2024-09-11T23:32:13.409 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T23:32:13.436 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T23:32:13.543 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901489' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBD+FohOSK2jb4q460gIdh7FZzdzPTqp7PGqKWR1p2XT6D3S8/UYqrpTqp2EZCfyzO3gAt0hfrADsRPqAnGITKo= smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSZ377PDtY50TiR0/u0vRmm9TF9PwwGQw6VpSZuEFqV1jVEiJUJ9yK7Nj9VC3mSnu5L2+r7dy+aGztXZpV/Pmk= smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIEDZmFVkEOvWqCWeTWM7U9UYnglBHRKhcluYHjt9h/+X//RQisZtU6GI6INN6w7RuJ+8qktOMCdF8yX1rHd+ZU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T23:32:13.543 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-11T23:32:13.544 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-11T23:32:13.544 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T23:32:13.546 INFO:teuthology.task.internal:Checking packages... 2024-09-11T23:32:13.568 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T23:32:13.568 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T23:32:13.569 INFO:teuthology.packaging:ref: None 2024-09-11T23:32:13.569 INFO:teuthology.packaging:tag: None 2024-09-11T23:32:13.569 INFO:teuthology.packaging:branch: squid 2024-09-11T23:32:13.569 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:32:13.569 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-11T23:32:13.821 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-11T23:32:13.823 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T23:32:13.829 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T23:32:13.829 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T23:32:13.941 INFO:teuthology.task.internal:Saving configuration 2024-09-11T23:32:13.962 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T23:32:14.035 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T23:32:14.066 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 23:27:20.121576', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBD+FohOSK2jb4q460gIdh7FZzdzPTqp7PGqKWR1p2XT6D3S8/UYqrpTqp2EZCfyzO3gAt0hfrADsRPqAnGITKo='} 2024-09-11T23:32:14.093 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 23:27:20.117670', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSZ377PDtY50TiR0/u0vRmm9TF9PwwGQw6VpSZuEFqV1jVEiJUJ9yK7Nj9VC3mSnu5L2+r7dy+aGztXZpV/Pmk='} 2024-09-11T23:32:14.125 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 23:27:20.119699', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIEDZmFVkEOvWqCWeTWM7U9UYnglBHRKhcluYHjt9h/+X//RQisZtU6GI6INN6w7RuJ+8qktOMCdF8yX1rHd+ZU='} 2024-09-11T23:32:14.125 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T23:32:14.132 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-11T23:32:14.132 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-11T23:32:14.132 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-11T23:32:14.132 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T23:32:14.263 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4c8cf9bbe0>, signals=[15]) 2024-09-11T23:32:14.263 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T23:32:14.270 INFO:teuthology.task.internal:Opening connections... 2024-09-11T23:32:14.270 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-09-11T23:32:14.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:32:14.356 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-09-11T23:32:14.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:32:14.441 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-09-11T23:32:14.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:32:14.523 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T23:32:14.530 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-09-11T23:32:14.548 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-09-11T23:32:14.548 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-09-11T23:32:14.606 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T23:32:14.607 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T23:32:14.608 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T23:32:14.608 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-09-11T23:32:14.634 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-09-11T23:32:14.652 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-09-11T23:32:14.653 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T23:32:14.711 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-09-11T23:32:14.712 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T23:32:14.712 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T23:32:14.712 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T23:32:14.712 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-09-11T23:32:14.734 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-09-11T23:32:14.752 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-09-11T23:32:14.753 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-09-11T23:32:14.811 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-09-11T23:32:14.811 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-09-11T23:32:14.811 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-09-11T23:32:14.811 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-09-11T23:32:14.811 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-09-11T23:32:14.811 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-09-11T23:32:14.811 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T23:32:14.812 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-09-11T23:32:14.812 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-09-11T23:32:14.812 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T23:32:14.812 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-09-11T23:32:14.812 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T23:32:14.812 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T23:32:14.812 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T23:32:14.812 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-09-11T23:32:14.842 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T23:32:14.852 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T23:32:14.894 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T23:32:14.894 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-09-11T23:32:14.897 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-09-11T23:32:14.900 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-09-11T23:32:14.919 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T23:32:14.929 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T23:32:14.930 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-09-11T23:32:14.954 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-09-11T23:32:14.957 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-09-11T23:32:14.988 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T23:32:15.092 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T23:32:15.107 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-11T23:32:15.107 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-11T23:32:15.107 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-11T23:32:15.107 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:32:15.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:32:15.125 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-09-11T23:32:15.182 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-503.el9.x86_64 2024-09-11T23:32:15.182 INFO:teuthology.task.kernel:Running kernel on smithi062: 5.14.0-503.el9.x86_64 2024-09-11T23:32:15.182 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y kernel 2024-09-11T23:32:17.514 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-11T23:32:20.157 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-11T23:32:25.574 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-11T23:32:26.991 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 118 kB/s | 19 kB 00:00 2024-09-11T23:32:27.887 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-11T23:32:34.101 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 2.8 kB/s | 1.7 kB 00:00 2024-09-11T23:32:36.208 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T23:32:36.208 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T23:32:36.302 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:32:36.311 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-09-11T23:32:36.312 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-09-11T23:32:36.312 INFO:teuthology.orchestra.run.smithi062.stdout:Install 4 Packages 2024-09-11T23:32:36.312 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:32:36.312 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 79 M 2024-09-11T23:32:36.312 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 126 M 2024-09-11T23:32:36.312 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-09-11T23:32:36.469 INFO:teuthology.orchestra.run.smithi062.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 206 kB/s | 23 kB 00:00 2024-09-11T23:32:37.070 INFO:teuthology.orchestra.run.smithi062.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-11T23:32:37.970 INFO:teuthology.orchestra.run.smithi062.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-11T23:32:38.478 INFO:teuthology.orchestra.run.smithi062.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 14 MB/s | 29 MB 00:01 2024-09-11T23:32:38.478 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:32:38.478 INFO:teuthology.orchestra.run.smithi062.stdout:Total 36 MB/s | 79 MB 00:02 2024-09-11T23:32:39.175 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-09-11T23:32:39.540 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-09-11T23:32:39.541 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-09-11T23:32:40.520 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-09-11T23:32:40.521 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-09-11T23:32:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-09-11T23:32:43.771 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T23:32:43.853 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T23:32:46.647 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T23:32:46.781 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T23:32:51.909 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T23:32:51.949 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:32:57.099 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:33:36.221 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:33:36.263 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:33:36.534 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:33:36.535 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T23:33:36.535 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T23:33:36.535 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T23:33:37.396 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:33:37.397 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:33:37.397 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-09-11T23:33:37.397 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T23:33:37.397 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T23:33:37.397 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T23:33:37.397 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T23:33:37.397 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:33:37.397 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-09-11T23:33:37.761 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T23:33:37.842 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T23:33:37.842 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T23:33:37.843 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T23:33:37.843 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T23:33:37.843 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T23:33:37.843 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-11T23:33:37.843 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T23:33:37.843 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T23:33:37.901 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T23:33:37.902 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T23:33:37.902 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2024-09-11T23:33:37.933 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2024-09-11T23:33:37.933 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2024-09-11T23:33:37.933 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.06 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 82.el9 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Group : Unspecified 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 8274670 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : builder@centos.org 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : CentOS 2024-09-11T23:33:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T23:33:37.935 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2024-09-11T23:33:37.935 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2024-09-11T23:33:37.935 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:33:37.935 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T23:33:37.935 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T23:33:37.935 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T23:33:37.935 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2024-09-11T23:33:37.935 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:33:37.935 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T23:33:37.936 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T23:33:37.936 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 5.14.0-505.el9.x86_64 2024-09-11T23:33:37.936 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T23:33:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-09-11T23:33:40.042 INFO:teuthology.orchestra.run.smithi062.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T23:33:40.071 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-09-11T23:33:40.073 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-09-11T23:33:40.130 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.y8kfYHaWUI 2024-09-11T23:33:40.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.y8kfYHaWUI 2024-09-11T23:33:40.196 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.y8kfYHaWUI 2024-09-11T23:33:40.339 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.y8kfYHaWUI is 6KB 2024-09-11T23:33:40.359 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.y8kfYHaWUI 2024-09-11T23:33:40.373 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T23:33:40.443 INFO:teuthology.orchestra.run.smithi062.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T23:33:40.443 INFO:teuthology.orchestra.run.smithi062.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T23:33:40.444 INFO:teuthology.orchestra.run.smithi062.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T23:33:40.444 INFO:teuthology.orchestra.run.smithi062.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T23:33:40.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T23:33:40.593 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2024-09-11T23:34:10.596 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T23:34:10.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-11T23:34:10.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:34:28.929 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-09-11T23:34:37.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-11T23:34:37.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:34:41.021 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-09-11T23:34:53.022 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-11T23:34:53.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:35:24.953 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-09-11T23:35:25.295 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-11T23:35:25.296 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-11T23:35:25.296 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-09-11T23:35:25.311 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-505.el9.x86_64 2024-09-11T23:35:25.311 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T23:35:25.311 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T23:35:25.312 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T23:35:26.313 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:26.313 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T23:35:26.320 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T23:35:26.320 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:35:26.323 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:35:26.325 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:35:26.344 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T23:35:26.355 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T23:35:26.388 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T23:35:26.388 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:35:26.391 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:35:26.393 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:35:26.478 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T23:35:26.485 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T23:35:26.485 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:35:26.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:35:26.502 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:35:26.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:35:26.520 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:35:26.522 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:35:26.545 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:35:26.586 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:35:26.595 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:35:26.614 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:35:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:35:26.617 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:35:26.634 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:35:26.636 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T23:35:26.644 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T23:35:26.644 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:35:26.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:35:26.649 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:35:26.677 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T23:35:26.718 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T23:35:26.718 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:35:26.721 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:35:26.723 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:35:26.739 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:35:26.829 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:35:26.956 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:35:27.003 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:35:27.043 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:35:27.043 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:35:27.110 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:35:27.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:35:27.207 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:35:27.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:35:27.336 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:35:27.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:35:27.403 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:35:27.458 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:35:27.504 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:35:27.595 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:35:27.633 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:35:27.634 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:35:27.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-09-11T23:35:27.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-09-11T23:35:27.708 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-09-11T23:35:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:35:27.758 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:35:27.790 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:35:28.272 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T23:35:28.279 INFO:teuthology.task.internal:Starting timer... 2024-09-11T23:35:28.279 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T23:35:28.317 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T23:35:28.350 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-11T23:35:28.350 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-09-11T23:35:28.401 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-09-11T23:35:28.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-09-11T23:35:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-09-11T23:35:28.476 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-09-11T23:35:28.536 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-09-11T23:35:28.538 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T23:35:28.538 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-09-11T23:35:28.568 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-09-11T23:35:28.568 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-09-11T23:35:28.594 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-09-11T23:35:28.594 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-09-11T23:35:28.624 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-09-11T23:35:28.624 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi120.front.sepia.ceph.com': 'permissive'} 2024-09-11T23:35:28.625 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:35:28.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:35:28.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:35:28.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:35:28.686 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:35:28.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:35:28.716 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T23:35:28.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:35:28.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:35:28.786 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:35:28.822 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T23:35:28.832 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T23:35:28.835 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T23:35:28.974 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T23:35:28.985 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T23:35:28.986 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi083.front.sepia.ceph.com,smithi120.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T23:39:30.509 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi120.front.sepia.ceph.com')] 2024-09-11T23:39:30.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-11T23:39:30.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:39:30.596 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-09-11T23:39:30.689 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-11T23:39:30.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-11T23:39:30.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:39:30.790 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-09-11T23:39:30.869 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-11T23:39:30.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-11T23:39:30.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:39:30.969 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-09-11T23:39:31.054 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-11T23:39:31.055 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T23:39:31.395 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T23:39:31.395 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:39:31.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:39:31.399 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:39:31.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:39:31.402 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:39:31.402 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:39:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:39:31.433 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:39:31.437 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:39:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:39:31.449 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:39:31.452 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:39:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-09-11T23:39:31.476 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-09-11T23:39:31.479 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-09-11T23:39:31.487 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-09-11T23:39:31.489 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-09-11T23:39:31.490 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-09-11T23:39:31.502 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:39:31.504 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:39:31.505 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:39:31.517 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:39:31.519 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:39:31.520 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:39:31.577 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:39:31.579 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:39:31.582 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:39:31.582 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-09-11T23:39:31.582 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.582 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.582 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.583 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.583 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:39:31.583 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-11T23:39:31.584 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.584 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.584 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.584 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.586 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:39:31.590 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:39:31.590 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-09-11T23:39:31.590 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.590 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.590 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.590 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:39:31.591 INFO:teuthology.run_tasks:Running task install... 2024-09-11T23:39:31.599 DEBUG:teuthology.task.install:project ceph 2024-09-11T23:39:31.600 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T23:39:31.600 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T23:39:31.600 INFO:teuthology.task.install:Using flavor: default 2024-09-11T23:39:31.605 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T23:39:31.605 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T23:39:31.606 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:39:31.606 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:39:31.608 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:39:31.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:39:31.609 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:39:31.609 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:39:31.769 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:39:31.769 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:39:31.787 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:39:31.787 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:39:31.798 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:39:31.798 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:39:31.896 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:39:31.897 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:39:31.897 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:39:31.906 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:39:31.906 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:39:31.906 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:39:31.914 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:39:31.914 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:39:31.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:39:31.924 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:39:31.925 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:39:31.932 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:39:31.932 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:39:31.940 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:39:31.941 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:39:31.992 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:39:31.997 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:39:32.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:39:32.070 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:39:32.072 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:39:32.082 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:39:32.135 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-09-11T23:39:32.137 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-09-11T23:39:32.137 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-09-11T23:39:32.138 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-09-11T23:39:32.146 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-09-11T23:39:32.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-09-11T23:39:32.362 INFO:teuthology.orchestra.run.smithi120.stdout:45 files removed 2024-09-11T23:39:32.369 INFO:teuthology.orchestra.run.smithi062.stdout:45 files removed 2024-09-11T23:39:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:45 files removed 2024-09-11T23:39:32.389 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:39:32.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:39:32.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:39:32.935 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-11T23:39:32.964 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-11T23:39:32.968 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 285 kB/s | 84 kB 00:00 2024-09-11T23:39:33.220 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-11T23:39:33.248 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-11T23:39:33.252 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-11T23:39:33.495 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-11T23:39:33.519 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-11T23:39:33.524 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-11T23:39:34.379 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.2 MB 00:00 2024-09-11T23:39:34.970 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.2 MB 00:01 2024-09-11T23:39:35.149 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.2 MB 00:01 2024-09-11T23:39:36.388 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-11T23:39:37.309 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-11T23:39:39.113 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 20 MB 00:03 2024-09-11T23:39:40.690 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-11T23:39:41.918 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.5 MB 00:00 2024-09-11T23:39:42.082 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 102 kB/s | 19 kB 00:00 2024-09-11T23:39:43.245 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-11T23:39:43.585 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 19 kB 00:00 2024-09-11T23:39:44.289 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.5 MB 00:01 2024-09-11T23:39:44.431 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-11T23:39:45.840 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 19 kB 00:00 2024-09-11T23:39:46.736 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-11T23:39:48.693 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-11T23:39:49.979 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T23:39:50.986 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:39:50.987 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:39:50.998 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:39:50.999 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:39:51.000 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:39:51.001 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:39:51.002 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:39:51.005 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:39:51.006 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:39:51.067 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-11T23:39:51.080 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-11T23:39:51.080 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:39:51.081 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:39:51.082 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:39:51.083 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:39:51.084 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:39:51.085 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:39:51.086 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:39:51.087 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:39:51.088 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:39:51.089 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:39:51.090 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:39:51.091 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:39:51.092 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:39:51.093 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:39:51.094 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-11T23:39:51.094 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-11T23:39:51.094 INFO:teuthology.orchestra.run.smithi120.stdout:Install 148 Packages 2024-09-11T23:39:51.094 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 8 Packages 2024-09-11T23:39:51.094 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:39:51.094 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 226 M 2024-09-11T23:39:51.095 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-11T23:39:52.078 INFO:teuthology.orchestra.run.smithi120.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-11T23:39:52.191 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T23:39:52.303 INFO:teuthology.orchestra.run.smithi120.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.9 MB/s | 897 kB 00:00 2024-09-11T23:39:52.370 INFO:teuthology.orchestra.run.smithi120.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.1 MB/s | 145 kB 00:00 2024-09-11T23:39:52.418 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:39:52.418 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:39:52.431 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:39:52.431 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:39:52.432 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:39:52.433 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:39:52.435 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:39:52.437 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:39:52.438 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:39:52.462 INFO:teuthology.orchestra.run.smithi120.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-11T23:39:52.505 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-09-11T23:39:52.531 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================== 2024-09-11T23:39:52.531 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-09-11T23:39:52.531 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================== 2024-09-11T23:39:52.531 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:39:52.532 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:39:52.533 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:39:52.534 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:39:52.535 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:39:52.536 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:39:52.536 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:39:52.536 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:39:52.536 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:39:52.536 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:39:52.536 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:39:52.537 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:39:52.537 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:39:52.537 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:39:52.537 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:39:52.537 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:39:52.537 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:39:52.537 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:39:52.537 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:39:52.537 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:39:52.537 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:39:52.538 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:39:52.539 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:39:52.540 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:39:52.541 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:39:52.542 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:39:52.543 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:39:52.544 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:39:52.544 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:39:52.544 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:39:52.544 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:39:52.544 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:39:52.544 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:39:52.544 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:39:52.544 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:39:52.544 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:39:52.544 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:39:52.545 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:39:52.546 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:39:52.546 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-09-11T23:39:52.546 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:39:52.546 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:39:52.546 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-09-11T23:39:52.546 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================== 2024-09-11T23:39:52.546 INFO:teuthology.orchestra.run.smithi062.stdout:Install 148 Packages 2024-09-11T23:39:52.546 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 8 Packages 2024-09-11T23:39:52.546 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:39:52.547 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 226 M 2024-09-11T23:39:52.547 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-09-11T23:39:52.579 INFO:teuthology.orchestra.run.smithi120.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-11T23:39:52.621 INFO:teuthology.orchestra.run.smithi120.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-11T23:39:52.963 INFO:teuthology.orchestra.run.smithi120.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-11T23:39:53.166 INFO:teuthology.orchestra.run.smithi062.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 33 kB/s | 6.5 kB 00:00 2024-09-11T23:39:53.264 INFO:teuthology.orchestra.run.smithi120.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 21 MB 00:01 2024-09-11T23:39:53.314 INFO:teuthology.orchestra.run.smithi120.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 507 kB/s | 25 kB 00:00 2024-09-11T23:39:53.583 INFO:teuthology.orchestra.run.smithi062.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.1 MB/s | 897 kB 00:00 2024-09-11T23:39:53.658 INFO:teuthology.orchestra.run.smithi062.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 7.8 MB/s | 5.4 MB 00:00 2024-09-11T23:39:53.683 INFO:teuthology.orchestra.run.smithi062.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.4 MB/s | 145 kB 00:00 2024-09-11T23:39:53.705 INFO:teuthology.orchestra.run.smithi120.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 16 MB/s | 17 MB 00:01 2024-09-11T23:39:53.792 INFO:teuthology.orchestra.run.smithi062.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 18 MB/s | 2.4 MB 00:00 2024-09-11T23:39:53.899 INFO:teuthology.orchestra.run.smithi120.stdout:(11/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 11 MB/s | 11 MB 00:00 2024-09-11T23:39:53.923 INFO:teuthology.orchestra.run.smithi120.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.0 MB/s | 228 kB 00:00 2024-09-11T23:39:53.957 INFO:teuthology.orchestra.run.smithi120.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 469 kB/s | 27 kB 00:00 2024-09-11T23:39:53.973 INFO:teuthology.orchestra.run.smithi120.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 629 kB/s | 32 kB 00:00 2024-09-11T23:39:54.017 INFO:teuthology.orchestra.run.smithi062.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 21 MB/s | 4.7 MB 00:00 2024-09-11T23:39:54.024 INFO:teuthology.orchestra.run.smithi120.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-11T23:39:54.049 INFO:teuthology.orchestra.run.smithi120.stdout:(16/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.0 MB/s | 752 kB 00:00 2024-09-11T23:39:54.059 INFO:teuthology.orchestra.run.smithi062.stdout:(7/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 4.1 MB/s | 1.5 MB 00:00 2024-09-11T23:39:54.091 INFO:teuthology.orchestra.run.smithi120.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.8 MB/s | 127 kB 00:00 2024-09-11T23:39:54.125 INFO:teuthology.orchestra.run.smithi120.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.5 MB/s | 506 kB 00:00 2024-09-11T23:39:54.208 INFO:teuthology.orchestra.run.smithi120.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 542 kB/s | 45 kB 00:00 2024-09-11T23:39:54.292 INFO:teuthology.orchestra.run.smithi120.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-11T23:39:54.376 INFO:teuthology.orchestra.run.smithi120.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 162 kB 00:00 2024-09-11T23:39:54.468 INFO:teuthology.orchestra.run.smithi120.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 321 kB 00:00 2024-09-11T23:39:54.486 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:39:54.487 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:39:54.497 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:39:54.498 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:39:54.499 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:39:54.500 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:39:54.502 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:39:54.504 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:39:54.505 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:39:54.560 INFO:teuthology.orchestra.run.smithi120.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.2 MB/s | 301 kB 00:00 2024-09-11T23:39:54.567 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi083.stdout:====================================================================================== 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi083.stdout:====================================================================================== 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-11T23:39:54.577 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:39:54.578 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:39:54.579 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:39:54.580 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:39:54.581 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-11T23:39:54.582 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:39:54.583 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:39:54.584 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:39:54.585 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:39:54.586 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:39:54.587 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:39:54.588 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:39:54.589 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:39:54.590 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout:====================================================================================== 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout:Install 148 Packages 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 8 Packages 2024-09-11T23:39:54.591 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 226 M 2024-09-11T23:39:54.592 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-11T23:39:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.3 MB/s | 100 kB 00:00 2024-09-11T23:39:54.694 INFO:teuthology.orchestra.run.smithi120.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-11T23:39:54.751 INFO:teuthology.orchestra.run.smithi062.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 23 MB/s | 17 MB 00:00 2024-09-11T23:39:54.769 INFO:teuthology.orchestra.run.smithi120.stdout:(26/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 7.9 MB/s | 5.3 MB 00:00 2024-09-11T23:39:54.793 INFO:teuthology.orchestra.run.smithi062.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 610 kB/s | 25 kB 00:00 2024-09-11T23:39:54.851 INFO:teuthology.orchestra.run.smithi120.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-11T23:39:54.962 INFO:teuthology.orchestra.run.smithi120.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-11T23:39:54.989 INFO:teuthology.orchestra.run.smithi120.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 234 kB/s | 32 kB 00:00 2024-09-11T23:39:55.014 INFO:teuthology.orchestra.run.smithi120.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.8 MB/s | 146 kB 00:00 2024-09-11T23:39:55.077 INFO:teuthology.orchestra.run.smithi062.stdout:(10/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 10 MB/s | 21 MB 00:02 2024-09-11T23:39:55.085 INFO:teuthology.orchestra.run.smithi083.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 46 kB/s | 6.5 kB 00:00 2024-09-11T23:39:55.153 INFO:teuthology.orchestra.run.smithi062.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 228 kB 00:00 2024-09-11T23:39:55.252 INFO:teuthology.orchestra.run.smithi062.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 249 kB/s | 27 kB 00:00 2024-09-11T23:39:55.321 INFO:teuthology.orchestra.run.smithi120.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 11 MB/s | 3.6 MB 00:00 2024-09-11T23:39:55.344 INFO:teuthology.orchestra.run.smithi062.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 346 kB/s | 32 kB 00:00 2024-09-11T23:39:55.380 INFO:teuthology.orchestra.run.smithi120.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.2 MB/s | 249 kB 00:00 2024-09-11T23:39:55.461 INFO:teuthology.orchestra.run.smithi120.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 612 kB/s | 49 kB 00:00 2024-09-11T23:39:55.486 INFO:teuthology.orchestra.run.smithi062.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 5.2 MB/s | 752 kB 00:00 2024-09-11T23:39:55.544 INFO:teuthology.orchestra.run.smithi062.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.8 MB/s | 163 kB 00:00 2024-09-11T23:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-11T23:39:55.578 INFO:teuthology.orchestra.run.smithi120.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 140 kB/s | 16 kB 00:00 2024-09-11T23:39:55.611 INFO:teuthology.orchestra.run.smithi062.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-11T23:39:55.620 INFO:teuthology.orchestra.run.smithi120.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.2 MB/s | 285 kB 00:00 2024-09-11T23:39:55.718 INFO:teuthology.orchestra.run.smithi083.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.4 MB/s | 897 kB 00:00 2024-09-11T23:39:55.720 INFO:teuthology.orchestra.run.smithi062.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 4.6 MB/s | 506 kB 00:00 2024-09-11T23:39:55.728 INFO:teuthology.orchestra.run.smithi120.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 5.0 MB/s | 765 kB 00:00 2024-09-11T23:39:55.754 INFO:teuthology.orchestra.run.smithi120.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 136 kB/s | 18 kB 00:00 2024-09-11T23:39:55.802 INFO:teuthology.orchestra.run.smithi083.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.7 MB/s | 145 kB 00:00 2024-09-11T23:39:55.930 INFO:teuthology.orchestra.run.smithi120.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 398 kB/s | 80 kB 00:00 2024-09-11T23:39:56.022 INFO:teuthology.orchestra.run.smithi120.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-11T23:39:56.063 INFO:teuthology.orchestra.run.smithi120.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-11T23:39:56.097 INFO:teuthology.orchestra.run.smithi120.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-11T23:39:56.164 INFO:teuthology.orchestra.run.smithi120.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-09-11T23:39:56.178 INFO:teuthology.orchestra.run.smithi062.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-09-11T23:39:56.189 INFO:teuthology.orchestra.run.smithi120.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-11T23:39:56.223 INFO:teuthology.orchestra.run.smithi120.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-11T23:39:56.245 INFO:teuthology.orchestra.run.smithi062.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 679 kB/s | 45 kB 00:00 2024-09-11T23:39:56.248 INFO:teuthology.orchestra.run.smithi120.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-11T23:39:56.282 INFO:teuthology.orchestra.run.smithi120.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-11T23:39:56.321 INFO:teuthology.orchestra.run.smithi062.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.8 MB/s | 136 kB 00:00 2024-09-11T23:39:56.387 INFO:teuthology.orchestra.run.smithi062.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-11T23:39:56.469 INFO:teuthology.orchestra.run.smithi083.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.6 MB/s | 2.4 MB 00:00 2024-09-11T23:39:56.471 INFO:teuthology.orchestra.run.smithi062.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 321 kB 00:00 2024-09-11T23:39:56.546 INFO:teuthology.orchestra.run.smithi062.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.9 MB/s | 301 kB 00:00 2024-09-11T23:39:56.644 INFO:teuthology.orchestra.run.smithi083.stdout:(5/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 3.2 MB/s | 5.4 MB 00:01 2024-09-11T23:39:56.711 INFO:teuthology.orchestra.run.smithi083.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 6.3 MB/s | 1.5 MB 00:00 2024-09-11T23:39:56.962 INFO:teuthology.orchestra.run.smithi083.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 4.7 MB 00:00 2024-09-11T23:39:56.963 INFO:teuthology.orchestra.run.smithi062.stdout:(24/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 22 MB/s | 49 MB 00:02 2024-09-11T23:39:56.989 INFO:teuthology.orchestra.run.smithi062.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 225 kB/s | 100 kB 00:00 2024-09-11T23:39:57.014 INFO:teuthology.orchestra.run.smithi062.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T23:39:57.131 INFO:teuthology.orchestra.run.smithi062.stdout:(27/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 3.5 MB/s | 11 MB 00:03 2024-09-11T23:39:57.156 INFO:teuthology.orchestra.run.smithi062.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.2 MB/s | 171 kB 00:00 2024-09-11T23:39:57.191 INFO:teuthology.orchestra.run.smithi120.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-11T23:39:57.207 INFO:teuthology.orchestra.run.smithi062.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 423 kB/s | 32 kB 00:00 2024-09-11T23:39:57.232 INFO:teuthology.orchestra.run.smithi062.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-11T23:39:57.233 INFO:teuthology.orchestra.run.smithi120.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 700 kB/s | 689 kB 00:00 2024-09-11T23:39:57.307 INFO:teuthology.orchestra.run.smithi062.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 9.7 MB/s | 3.1 MB 00:00 2024-09-11T23:39:57.399 INFO:teuthology.orchestra.run.smithi062.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.7 MB/s | 249 kB 00:00 2024-09-11T23:39:57.491 INFO:teuthology.orchestra.run.smithi062.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 539 kB/s | 49 kB 00:00 2024-09-11T23:39:57.583 INFO:teuthology.orchestra.run.smithi062.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 180 kB/s | 16 kB 00:00 2024-09-11T23:39:57.700 INFO:teuthology.orchestra.run.smithi062.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-11T23:39:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:(8/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 14 MB/s | 11 MB 00:00 2024-09-11T23:39:57.725 INFO:teuthology.orchestra.run.smithi062.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 2.0 MB/s | 285 kB 00:00 2024-09-11T23:39:57.762 INFO:teuthology.orchestra.run.smithi083.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 433 kB/s | 25 kB 00:00 2024-09-11T23:39:57.767 INFO:teuthology.orchestra.run.smithi062.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 11 MB/s | 765 kB 00:00 2024-09-11T23:39:57.774 INFO:teuthology.orchestra.run.smithi120.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 40 kB/s | 22 kB 00:00 2024-09-11T23:39:57.784 INFO:teuthology.orchestra.run.smithi062.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 311 kB/s | 18 kB 00:00 2024-09-11T23:39:57.925 INFO:teuthology.orchestra.run.smithi120.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 146 kB/s | 107 kB 00:00 2024-09-11T23:39:57.937 INFO:teuthology.orchestra.run.smithi083.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 14 MB/s | 17 MB 00:01 2024-09-11T23:39:57.988 INFO:teuthology.orchestra.run.smithi083.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.5 MB/s | 228 kB 00:00 2024-09-11T23:39:58.026 INFO:teuthology.orchestra.run.smithi062.stdout:(39/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 4.3 MB/s | 3.6 MB 00:00 2024-09-11T23:39:58.046 INFO:teuthology.orchestra.run.smithi083.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 465 kB/s | 27 kB 00:00 2024-09-11T23:39:58.105 INFO:teuthology.orchestra.run.smithi083.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 545 kB/s | 32 kB 00:00 2024-09-11T23:39:58.188 INFO:teuthology.orchestra.run.smithi083.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.8 MB/s | 752 kB 00:00 2024-09-11T23:39:58.243 INFO:teuthology.orchestra.run.smithi062.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-09-11T23:39:58.255 INFO:teuthology.orchestra.run.smithi083.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-11T23:39:58.322 INFO:teuthology.orchestra.run.smithi083.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-11T23:39:58.398 INFO:teuthology.orchestra.run.smithi083.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.6 MB/s | 506 kB 00:00 2024-09-11T23:39:58.484 INFO:teuthology.orchestra.run.smithi120.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-11T23:39:58.510 INFO:teuthology.orchestra.run.smithi062.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-11T23:39:58.535 INFO:teuthology.orchestra.run.smithi062.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 867 kB/s | 253 kB 00:00 2024-09-11T23:39:58.577 INFO:teuthology.orchestra.run.smithi062.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 347 kB/s | 191 kB 00:00 2024-09-11T23:39:58.642 INFO:teuthology.orchestra.run.smithi120.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 362 kB/s | 313 kB 00:00 2024-09-11T23:39:58.644 INFO:teuthology.orchestra.run.smithi062.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 980 kB/s | 106 kB 00:00 2024-09-11T23:39:58.678 INFO:teuthology.orchestra.run.smithi062.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-11T23:39:58.720 INFO:teuthology.orchestra.run.smithi062.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-09-11T23:39:58.795 INFO:teuthology.orchestra.run.smithi062.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-11T23:39:58.815 INFO:teuthology.orchestra.run.smithi083.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-11T23:39:58.837 INFO:teuthology.orchestra.run.smithi062.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 648 kB/s | 125 kB 00:00 2024-09-11T23:39:58.857 INFO:teuthology.orchestra.run.smithi083.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 1.1 MB/s | 45 kB 00:00 2024-09-11T23:39:58.940 INFO:teuthology.orchestra.run.smithi083.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-11T23:39:58.962 INFO:teuthology.orchestra.run.smithi062.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 689 kB 00:00 2024-09-11T23:39:59.021 INFO:teuthology.orchestra.run.smithi062.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-11T23:39:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.7 MB/s | 162 kB 00:00 2024-09-11T23:39:59.037 INFO:teuthology.orchestra.run.smithi062.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 534 kB/s | 107 kB 00:00 2024-09-11T23:39:59.079 INFO:teuthology.orchestra.run.smithi062.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-11T23:39:59.105 INFO:teuthology.orchestra.run.smithi062.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-11T23:39:59.130 INFO:teuthology.orchestra.run.smithi062.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-11T23:39:59.134 INFO:teuthology.orchestra.run.smithi120.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 37 kB/s | 18 kB 00:00 2024-09-11T23:39:59.172 INFO:teuthology.orchestra.run.smithi062.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-11T23:39:59.197 INFO:teuthology.orchestra.run.smithi062.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-09-11T23:39:59.231 INFO:teuthology.orchestra.run.smithi062.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-11T23:39:59.256 INFO:teuthology.orchestra.run.smithi062.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-09-11T23:39:59.315 INFO:teuthology.orchestra.run.smithi062.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-11T23:39:59.348 INFO:teuthology.orchestra.run.smithi062.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-11T23:39:59.374 INFO:teuthology.orchestra.run.smithi062.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-11T23:39:59.416 INFO:teuthology.orchestra.run.smithi062.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 3.6 MB/s | 581 kB 00:00 2024-09-11T23:39:59.441 INFO:teuthology.orchestra.run.smithi062.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 514 kB/s | 47 kB 00:00 2024-09-11T23:39:59.466 INFO:teuthology.orchestra.run.smithi062.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-11T23:39:59.491 INFO:teuthology.orchestra.run.smithi062.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-11T23:39:59.507 INFO:teuthology.orchestra.run.smithi083.stdout:(22/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 28 MB/s | 49 MB 00:01 2024-09-11T23:39:59.517 INFO:teuthology.orchestra.run.smithi062.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-11T23:39:59.533 INFO:teuthology.orchestra.run.smithi083.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 641 kB/s | 321 kB 00:00 2024-09-11T23:39:59.567 INFO:teuthology.orchestra.run.smithi083.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-11T23:39:59.592 INFO:teuthology.orchestra.run.smithi083.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-11T23:39:59.617 INFO:teuthology.orchestra.run.smithi083.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T23:39:59.625 INFO:teuthology.orchestra.run.smithi062.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T23:39:59.676 INFO:teuthology.orchestra.run.smithi083.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-11T23:39:59.718 INFO:teuthology.orchestra.run.smithi083.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 765 kB/s | 32 kB 00:00 2024-09-11T23:39:59.725 INFO:teuthology.orchestra.run.smithi062.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 19 MB/s | 4.9 MB 00:00 2024-09-11T23:39:59.767 INFO:teuthology.orchestra.run.smithi062.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-11T23:39:59.785 INFO:teuthology.orchestra.run.smithi083.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 3.1 MB 00:00 2024-09-11T23:39:59.793 INFO:teuthology.orchestra.run.smithi062.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-11T23:39:59.810 INFO:teuthology.orchestra.run.smithi083.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.6 MB/s | 146 kB 00:00 2024-09-11T23:39:59.851 INFO:teuthology.orchestra.run.smithi062.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-11T23:39:59.885 INFO:teuthology.orchestra.run.smithi062.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-11T23:39:59.910 INFO:teuthology.orchestra.run.smithi062.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-11T23:39:59.960 INFO:teuthology.orchestra.run.smithi062.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-11T23:39:59.986 INFO:teuthology.orchestra.run.smithi083.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 18 MB/s | 3.6 MB 00:00 2024-09-11T23:40:00.027 INFO:teuthology.orchestra.run.smithi062.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-11T23:40:00.036 INFO:teuthology.orchestra.run.smithi083.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-11T23:40:00.051 INFO:teuthology.orchestra.run.smithi120.stdout:(55/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 7.2 MB/s | 49 MB 00:06 2024-09-11T23:40:00.076 INFO:teuthology.orchestra.run.smithi120.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 12 kB/s | 11 kB 00:00 2024-09-11T23:40:00.102 INFO:teuthology.orchestra.run.smithi062.stdout:(76/156): postgresql-server-13.16-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-09-11T23:40:00.128 INFO:teuthology.orchestra.run.smithi062.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-09-11T23:40:00.135 INFO:teuthology.orchestra.run.smithi120.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.8 MB/s | 3.0 MB 00:01 2024-09-11T23:40:00.136 INFO:teuthology.orchestra.run.smithi083.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 23 MB/s | 7.4 MB 00:00 2024-09-11T23:40:00.153 INFO:teuthology.orchestra.run.smithi062.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-09-11T23:40:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 394 kB/s | 49 kB 00:00 2024-09-11T23:40:00.187 INFO:teuthology.orchestra.run.smithi083.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 328 kB/s | 16 kB 00:00 2024-09-11T23:40:00.212 INFO:teuthology.orchestra.run.smithi083.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-11T23:40:00.228 INFO:teuthology.orchestra.run.smithi062.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-11T23:40:00.254 INFO:teuthology.orchestra.run.smithi062.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 152 kB/s | 19 kB 00:00 2024-09-11T23:40:00.257 INFO:teuthology.orchestra.run.smithi083.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 11 MB/s | 765 kB 00:00 2024-09-11T23:40:00.279 INFO:teuthology.orchestra.run.smithi062.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T23:40:00.282 INFO:teuthology.orchestra.run.smithi083.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 264 kB/s | 18 kB 00:00 2024-09-11T23:40:00.304 INFO:teuthology.orchestra.run.smithi062.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-09-11T23:40:00.338 INFO:teuthology.orchestra.run.smithi062.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-11T23:40:00.363 INFO:teuthology.orchestra.run.smithi062.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-11T23:40:00.397 INFO:teuthology.orchestra.run.smithi062.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-11T23:40:00.422 INFO:teuthology.orchestra.run.smithi062.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-09-11T23:40:00.432 INFO:teuthology.orchestra.run.smithi083.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 456 kB/s | 80 kB 00:00 2024-09-11T23:40:00.455 INFO:teuthology.orchestra.run.smithi062.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-11T23:40:00.481 INFO:teuthology.orchestra.run.smithi062.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-11T23:40:00.516 INFO:teuthology.orchestra.run.smithi083.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-11T23:40:00.564 INFO:teuthology.orchestra.run.smithi062.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-11T23:40:00.589 INFO:teuthology.orchestra.run.smithi062.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-09-11T23:40:00.608 INFO:teuthology.orchestra.run.smithi083.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-11T23:40:00.623 INFO:teuthology.orchestra.run.smithi062.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-09-11T23:40:00.649 INFO:teuthology.orchestra.run.smithi062.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 639 kB/s | 54 kB 00:00 2024-09-11T23:40:00.666 INFO:teuthology.orchestra.run.smithi083.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 801 kB 00:00 2024-09-11T23:40:00.682 INFO:teuthology.orchestra.run.smithi062.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-09-11T23:40:00.707 INFO:teuthology.orchestra.run.smithi062.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 753 kB/s | 44 kB 00:00 2024-09-11T23:40:00.717 INFO:teuthology.orchestra.run.smithi083.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-11T23:40:00.727 INFO:teuthology.orchestra.run.smithi120.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-09-11T23:40:00.749 INFO:teuthology.orchestra.run.smithi062.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-11T23:40:00.774 INFO:teuthology.orchestra.run.smithi062.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 536 kB/s | 36 kB 00:00 2024-09-11T23:40:00.808 INFO:teuthology.orchestra.run.smithi062.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-09-11T23:40:00.809 INFO:teuthology.orchestra.run.smithi083.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-11T23:40:00.833 INFO:teuthology.orchestra.run.smithi062.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-11T23:40:00.852 INFO:teuthology.orchestra.run.smithi120.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 200 kB/s | 160 kB 00:00 2024-09-11T23:40:00.867 INFO:teuthology.orchestra.run.smithi062.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-09-11T23:40:00.892 INFO:teuthology.orchestra.run.smithi083.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-11T23:40:00.892 INFO:teuthology.orchestra.run.smithi062.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-11T23:40:00.926 INFO:teuthology.orchestra.run.smithi062.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-11T23:40:00.943 INFO:teuthology.orchestra.run.smithi083.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 3.7 MB/s | 1.2 MB 00:00 2024-09-11T23:40:00.951 INFO:teuthology.orchestra.run.smithi062.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-09-11T23:40:00.993 INFO:teuthology.orchestra.run.smithi083.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-09-11T23:40:01.111 INFO:teuthology.orchestra.run.smithi120.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 679 kB/s | 662 kB 00:00 2024-09-11T23:40:01.152 INFO:teuthology.orchestra.run.smithi062.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-11T23:40:01.152 INFO:teuthology.orchestra.run.smithi083.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-11T23:40:01.177 INFO:teuthology.orchestra.run.smithi062.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-09-11T23:40:01.202 INFO:teuthology.orchestra.run.smithi083.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-11T23:40:01.235 INFO:teuthology.orchestra.run.smithi062.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-11T23:40:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-09-11T23:40:01.261 INFO:teuthology.orchestra.run.smithi062.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-11T23:40:01.278 INFO:teuthology.orchestra.run.smithi083.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 287 kB/s | 22 kB 00:00 2024-09-11T23:40:01.294 INFO:teuthology.orchestra.run.smithi062.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-09-11T23:40:01.303 INFO:teuthology.orchestra.run.smithi083.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-11T23:40:01.319 INFO:teuthology.orchestra.run.smithi062.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T23:40:01.328 INFO:teuthology.orchestra.run.smithi083.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-11T23:40:01.345 INFO:teuthology.orchestra.run.smithi062.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T23:40:01.371 INFO:teuthology.orchestra.run.smithi083.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 440 kB/s | 18 kB 00:00 2024-09-11T23:40:01.371 INFO:teuthology.orchestra.run.smithi062.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T23:40:01.404 INFO:teuthology.orchestra.run.smithi062.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-11T23:40:01.429 INFO:teuthology.orchestra.run.smithi083.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-11T23:40:01.437 INFO:teuthology.orchestra.run.smithi062.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-11T23:40:01.454 INFO:teuthology.orchestra.run.smithi083.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-11T23:40:01.478 INFO:teuthology.orchestra.run.smithi120.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 39 kB/s | 24 kB 00:00 2024-09-11T23:40:01.480 INFO:teuthology.orchestra.run.smithi083.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-11T23:40:01.505 INFO:teuthology.orchestra.run.smithi083.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-09-11T23:40:01.521 INFO:teuthology.orchestra.run.smithi062.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-11T23:40:01.547 INFO:teuthology.orchestra.run.smithi083.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-11T23:40:01.554 INFO:teuthology.orchestra.run.smithi062.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 4.7 MB/s | 548 kB 00:00 2024-09-11T23:40:01.579 INFO:teuthology.orchestra.run.smithi062.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-09-11T23:40:01.580 INFO:teuthology.orchestra.run.smithi083.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.6 MB/s | 581 kB 00:00 2024-09-11T23:40:01.605 INFO:teuthology.orchestra.run.smithi062.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-09-11T23:40:01.606 INFO:teuthology.orchestra.run.smithi083.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-11T23:40:01.630 INFO:teuthology.orchestra.run.smithi062.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-11T23:40:01.631 INFO:teuthology.orchestra.run.smithi083.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-11T23:40:01.655 INFO:teuthology.orchestra.run.smithi062.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T23:40:01.665 INFO:teuthology.orchestra.run.smithi083.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-11T23:40:01.690 INFO:teuthology.orchestra.run.smithi083.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-11T23:40:01.703 INFO:teuthology.orchestra.run.smithi120.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 596 kB/s | 581 kB 00:00 2024-09-11T23:40:01.715 INFO:teuthology.orchestra.run.smithi083.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-11T23:40:01.731 INFO:teuthology.orchestra.run.smithi062.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-09-11T23:40:01.753 INFO:teuthology.orchestra.run.smithi120.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-09-11T23:40:01.756 INFO:teuthology.orchestra.run.smithi062.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-09-11T23:40:01.781 INFO:teuthology.orchestra.run.smithi062.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-09-11T23:40:01.807 INFO:teuthology.orchestra.run.smithi062.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-11T23:40:01.815 INFO:teuthology.orchestra.run.smithi083.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 39 MB/s | 4.9 MB 00:00 2024-09-11T23:40:01.832 INFO:teuthology.orchestra.run.smithi062.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-11T23:40:01.857 INFO:teuthology.orchestra.run.smithi062.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-11T23:40:01.882 INFO:teuthology.orchestra.run.smithi062.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-11T23:40:01.883 INFO:teuthology.orchestra.run.smithi083.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-11T23:40:01.908 INFO:teuthology.orchestra.run.smithi062.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-11T23:40:01.933 INFO:teuthology.orchestra.run.smithi062.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-11T23:40:01.950 INFO:teuthology.orchestra.run.smithi083.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-11T23:40:01.958 INFO:teuthology.orchestra.run.smithi062.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-11T23:40:01.983 INFO:teuthology.orchestra.run.smithi083.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-11T23:40:01.984 INFO:teuthology.orchestra.run.smithi062.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:40:02.001 INFO:teuthology.orchestra.run.smithi062.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 4.7 MB/s | 79 kB 00:00 2024-09-11T23:40:02.008 INFO:teuthology.orchestra.run.smithi083.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-11T23:40:02.026 INFO:teuthology.orchestra.run.smithi062.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-11T23:40:02.059 INFO:teuthology.orchestra.run.smithi062.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-11T23:40:02.067 INFO:teuthology.orchestra.run.smithi083.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-11T23:40:02.093 INFO:teuthology.orchestra.run.smithi062.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-11T23:40:02.110 INFO:teuthology.orchestra.run.smithi062.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-11T23:40:02.135 INFO:teuthology.orchestra.run.smithi062.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-11T23:40:02.142 INFO:teuthology.orchestra.run.smithi083.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-09-11T23:40:02.152 INFO:teuthology.orchestra.run.smithi062.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 675 kB/s | 28 kB 00:00 2024-09-11T23:40:02.168 INFO:teuthology.orchestra.run.smithi083.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 871 kB/s | 87 kB 00:00 2024-09-11T23:40:02.169 INFO:teuthology.orchestra.run.smithi062.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 921 kB/s | 31 kB 00:00 2024-09-11T23:40:02.194 INFO:teuthology.orchestra.run.smithi062.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-09-11T23:40:02.210 INFO:teuthology.orchestra.run.smithi083.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-11T23:40:02.211 INFO:teuthology.orchestra.run.smithi062.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-11T23:40:02.228 INFO:teuthology.orchestra.run.smithi062.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-11T23:40:02.279 INFO:teuthology.orchestra.run.smithi062.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-11T23:40:02.294 INFO:teuthology.orchestra.run.smithi083.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-09-11T23:40:02.319 INFO:teuthology.orchestra.run.smithi083.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-11T23:40:02.329 INFO:teuthology.orchestra.run.smithi062.stdout:(142/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-09-11T23:40:02.344 INFO:teuthology.orchestra.run.smithi083.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-11T23:40:02.354 INFO:teuthology.orchestra.run.smithi120.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 65 kB/s | 38 kB 00:00 2024-09-11T23:40:02.354 INFO:teuthology.orchestra.run.smithi062.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-09-11T23:40:02.370 INFO:teuthology.orchestra.run.smithi083.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-11T23:40:02.379 INFO:teuthology.orchestra.run.smithi120.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 325 kB/s | 292 kB 00:00 2024-09-11T23:40:02.379 INFO:teuthology.orchestra.run.smithi062.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-09-11T23:40:02.395 INFO:teuthology.orchestra.run.smithi083.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-11T23:40:02.405 INFO:teuthology.orchestra.run.smithi062.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 401 kB/s | 20 kB 00:00 2024-09-11T23:40:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-11T23:40:02.430 INFO:teuthology.orchestra.run.smithi062.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T23:40:02.445 INFO:teuthology.orchestra.run.smithi083.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-11T23:40:02.464 INFO:teuthology.orchestra.run.smithi062.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-11T23:40:02.513 INFO:teuthology.orchestra.run.smithi120.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 233 kB/s | 188 kB 00:00 2024-09-11T23:40:02.514 INFO:teuthology.orchestra.run.smithi062.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T23:40:02.646 INFO:teuthology.orchestra.run.smithi083.stdout:(82/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 2.8 MB/s | 21 MB 00:07 2024-09-11T23:40:02.671 INFO:teuthology.orchestra.run.smithi083.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-09-11T23:40:02.713 INFO:teuthology.orchestra.run.smithi083.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-09-11T23:40:02.789 INFO:teuthology.orchestra.run.smithi083.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-11T23:40:02.814 INFO:teuthology.orchestra.run.smithi083.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-09-11T23:40:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.6 MB/s | 3.2 MB 00:00 2024-09-11T23:40:02.839 INFO:teuthology.orchestra.run.smithi083.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-11T23:40:02.864 INFO:teuthology.orchestra.run.smithi083.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-11T23:40:02.890 INFO:teuthology.orchestra.run.smithi083.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-11T23:40:02.915 INFO:teuthology.orchestra.run.smithi083.stdout:(90/156): python3-oauthlib-3.1.1-5.el9.noarch.r 828 kB/s | 222 kB 00:00 2024-09-11T23:40:02.940 INFO:teuthology.orchestra.run.smithi083.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-11T23:40:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 6.6 MB/s | 3.4 MB 00:00 2024-09-11T23:40:02.965 INFO:teuthology.orchestra.run.smithi083.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-11T23:40:02.991 INFO:teuthology.orchestra.run.smithi083.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-11T23:40:03.016 INFO:teuthology.orchestra.run.smithi083.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-11T23:40:03.041 INFO:teuthology.orchestra.run.smithi083.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-11T23:40:03.066 INFO:teuthology.orchestra.run.smithi083.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-09-11T23:40:03.092 INFO:teuthology.orchestra.run.smithi083.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-11T23:40:03.117 INFO:teuthology.orchestra.run.smithi083.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-11T23:40:03.142 INFO:teuthology.orchestra.run.smithi083.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T23:40:03.167 INFO:teuthology.orchestra.run.smithi083.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-11T23:40:03.193 INFO:teuthology.orchestra.run.smithi083.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T23:40:03.218 INFO:teuthology.orchestra.run.smithi083.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 886 kB/s | 45 kB 00:00 2024-09-11T23:40:03.231 INFO:teuthology.orchestra.run.smithi062.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 144 kB/s | 41 kB 00:00 2024-09-11T23:40:03.244 INFO:teuthology.orchestra.run.smithi083.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-09-11T23:40:03.294 INFO:teuthology.orchestra.run.smithi083.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-09-11T23:40:03.344 INFO:teuthology.orchestra.run.smithi083.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.3 MB/s | 548 kB 00:00 2024-09-11T23:40:03.369 INFO:teuthology.orchestra.run.smithi083.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 786 kB/s | 19 kB 00:00 2024-09-11T23:40:03.395 INFO:teuthology.orchestra.run.smithi083.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.1 MB/s | 309 kB 00:00 2024-09-11T23:40:03.420 INFO:teuthology.orchestra.run.smithi083.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T23:40:03.462 INFO:teuthology.orchestra.run.smithi083.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-11T23:40:03.488 INFO:teuthology.orchestra.run.smithi083.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T23:40:03.521 INFO:teuthology.orchestra.run.smithi083.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-11T23:40:03.523 INFO:teuthology.orchestra.run.smithi062.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 852 kB/s | 249 kB 00:00 2024-09-11T23:40:03.572 INFO:teuthology.orchestra.run.smithi083.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-11T23:40:03.588 INFO:teuthology.orchestra.run.smithi120.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-09-11T23:40:03.599 INFO:teuthology.orchestra.run.smithi062.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 2.4 MB/s | 1.8 MB 00:00 2024-09-11T23:40:03.613 INFO:teuthology.orchestra.run.smithi083.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-11T23:40:03.639 INFO:teuthology.orchestra.run.smithi083.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T23:40:03.663 INFO:teuthology.orchestra.run.smithi120.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.8 MB/s | 4.9 MB 00:01 2024-09-11T23:40:03.664 INFO:teuthology.orchestra.run.smithi083.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T23:40:03.689 INFO:teuthology.orchestra.run.smithi083.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-11T23:40:03.705 INFO:teuthology.orchestra.run.smithi120.stdout:(69/156): postgresql-13.16-1.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-09-11T23:40:03.723 INFO:teuthology.orchestra.run.smithi083.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-11T23:40:03.765 INFO:teuthology.orchestra.run.smithi083.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 8.4 MB/s | 358 kB 00:00 2024-09-11T23:40:03.824 INFO:teuthology.orchestra.run.smithi083.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.7 MB/s | 4.4 MB 00:00 2024-09-11T23:40:03.825 INFO:teuthology.orchestra.run.smithi062.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-09-11T23:40:03.849 INFO:teuthology.orchestra.run.smithi083.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-11T23:40:03.857 INFO:teuthology.orchestra.run.smithi062.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.1 MB/s | 1.4 MB 00:00 2024-09-11T23:40:03.875 INFO:teuthology.orchestra.run.smithi083.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-11T23:40:03.908 INFO:teuthology.orchestra.run.smithi083.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-11T23:40:03.962 INFO:teuthology.orchestra.run.smithi083.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 200 kB/s | 11 kB 00:00 2024-09-11T23:40:03.992 INFO:teuthology.orchestra.run.smithi083.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 593 kB/s | 18 kB 00:00 2024-09-11T23:40:04.017 INFO:teuthology.orchestra.run.smithi083.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-11T23:40:04.043 INFO:teuthology.orchestra.run.smithi083.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-09-11T23:40:04.068 INFO:teuthology.orchestra.run.smithi083.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 794 kB/s | 20 kB 00:00 2024-09-11T23:40:04.074 INFO:teuthology.orchestra.run.smithi062.stdout:(156/156): NetworkManager-1.49.90-1.el9.x86_64. 1.5 MB/s | 2.3 MB 00:01 2024-09-11T23:40:04.080 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:40:04.081 INFO:teuthology.orchestra.run.smithi062.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-11T23:40:04.135 INFO:teuthology.orchestra.run.smithi083.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.1 MB/s | 2.0 MB 00:00 2024-09-11T23:40:04.160 INFO:teuthology.orchestra.run.smithi083.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:40:04.186 INFO:teuthology.orchestra.run.smithi083.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T23:40:04.219 INFO:teuthology.orchestra.run.smithi083.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-11T23:40:04.245 INFO:teuthology.orchestra.run.smithi083.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-09-11T23:40:04.270 INFO:teuthology.orchestra.run.smithi083.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-11T23:40:04.295 INFO:teuthology.orchestra.run.smithi083.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-11T23:40:04.321 INFO:teuthology.orchestra.run.smithi083.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-11T23:40:04.346 INFO:teuthology.orchestra.run.smithi083.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-11T23:40:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-11T23:40:04.396 INFO:teuthology.orchestra.run.smithi083.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T23:40:04.422 INFO:teuthology.orchestra.run.smithi083.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-11T23:40:04.422 INFO:teuthology.orchestra.run.smithi120.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 181 kB/s | 137 kB 00:00 2024-09-11T23:40:04.447 INFO:teuthology.orchestra.run.smithi083.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-11T23:40:04.472 INFO:teuthology.orchestra.run.smithi083.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-11T23:40:04.506 INFO:teuthology.orchestra.run.smithi083.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-11T23:40:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-11T23:40:04.548 INFO:teuthology.orchestra.run.smithi083.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-11T23:40:04.582 INFO:teuthology.orchestra.run.smithi083.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-11T23:40:04.607 INFO:teuthology.orchestra.run.smithi083.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-11T23:40:04.639 INFO:teuthology.orchestra.run.smithi120.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 866 kB/s | 910 kB 00:01 2024-09-11T23:40:04.658 INFO:teuthology.orchestra.run.smithi083.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-11T23:40:04.900 INFO:teuthology.orchestra.run.smithi083.stdout:(148/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-11T23:40:04.958 INFO:teuthology.orchestra.run.smithi083.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-11T23:40:04.998 INFO:teuthology.orchestra.run.smithi120.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-09-11T23:40:05.327 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-09-11T23:40:05.383 INFO:teuthology.orchestra.run.smithi083.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 4.8 MB/s | 2.3 MB 00:00 2024-09-11T23:40:05.398 INFO:teuthology.orchestra.run.smithi120.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 115 kB/s | 87 kB 00:00 2024-09-11T23:40:05.494 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-09-11T23:40:05.495 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-09-11T23:40:05.506 INFO:teuthology.orchestra.run.smithi120.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 953 kB/s | 1.0 MB 00:01 2024-09-11T23:40:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:(151/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.2 MB/s | 16 MB 00:02 2024-09-11T23:40:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 222 kB/s | 41 kB 00:00 2024-09-11T23:40:05.621 INFO:teuthology.orchestra.run.smithi083.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 2.8 MB/s | 1.8 MB 00:00 2024-09-11T23:40:05.785 INFO:teuthology.orchestra.run.smithi083.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 1.0 MB/s | 249 kB 00:00 2024-09-11T23:40:05.852 INFO:teuthology.orchestra.run.smithi083.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.8 MB/s | 1.4 MB 00:00 2024-09-11T23:40:06.190 INFO:teuthology.orchestra.run.smithi120.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-09-11T23:40:06.215 INFO:teuthology.orchestra.run.smithi120.stdout:(76/156): python3-colorama-0.4.6-3.el9.noarch.r 75 kB/s | 61 kB 00:00 2024-09-11T23:40:06.227 INFO:teuthology.orchestra.run.smithi083.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 2.4 MB 00:00 2024-09-11T23:40:06.233 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:40:06.233 INFO:teuthology.orchestra.run.smithi083.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-11T23:40:06.357 INFO:teuthology.orchestra.run.smithi120.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 288 kB/s | 245 kB 00:00 2024-09-11T23:40:06.782 INFO:teuthology.orchestra.run.smithi120.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 26 kB 00:00 2024-09-11T23:40:06.916 INFO:teuthology.orchestra.run.smithi120.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 34 kB/s | 19 kB 00:00 2024-09-11T23:40:07.033 INFO:teuthology.orchestra.run.smithi120.stdout:(80/156): python3-jinja2-2.11.3-6.el9.noarch.rp 295 kB/s | 248 kB 00:00 2024-09-11T23:40:07.461 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-11T23:40:07.508 INFO:teuthology.orchestra.run.smithi120.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 237 kB/s | 172 kB 00:00 2024-09-11T23:40:07.525 INFO:teuthology.orchestra.run.smithi120.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 57 kB/s | 35 kB 00:00 2024-09-11T23:40:07.609 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-11T23:40:07.609 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-11T23:40:07.791 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-09-11T23:40:07.791 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-09-11T23:40:08.292 INFO:teuthology.orchestra.run.smithi120.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-09-11T23:40:08.317 INFO:teuthology.orchestra.run.smithi120.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 310 kB/s | 250 kB 00:00 2024-09-11T23:40:08.343 INFO:teuthology.orchestra.run.smithi120.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 272 kB/s | 222 kB 00:00 2024-09-11T23:40:08.893 INFO:teuthology.orchestra.run.smithi120.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 74 kB/s | 42 kB 00:00 2024-09-11T23:40:08.935 INFO:teuthology.orchestra.run.smithi120.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 120 kB/s | 77 kB 00:00 2024-09-11T23:40:09.279 INFO:teuthology.orchestra.run.smithi120.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 288 kB/s | 269 kB 00:00 2024-09-11T23:40:09.400 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-11T23:40:09.400 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-11T23:40:09.744 INFO:teuthology.orchestra.run.smithi120.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 187 kB/s | 159 kB 00:00 2024-09-11T23:40:09.903 INFO:teuthology.orchestra.run.smithi120.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 83 kB/s | 51 kB 00:00 2024-09-11T23:40:09.928 INFO:teuthology.orchestra.run.smithi120.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 281 kB/s | 279 kB 00:00 2024-09-11T23:40:10.421 INFO:teuthology.orchestra.run.smithi120.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 79 kB/s | 54 kB 00:00 2024-09-11T23:40:10.545 INFO:teuthology.orchestra.run.smithi120.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-09-11T23:40:11.054 INFO:teuthology.orchestra.run.smithi120.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 70 kB/s | 44 kB 00:00 2024-09-11T23:40:11.363 INFO:teuthology.orchestra.run.smithi120.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-11T23:40:11.388 INFO:teuthology.orchestra.run.smithi120.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 262 kB/s | 221 kB 00:00 2024-09-11T23:40:11.771 INFO:teuthology.orchestra.run.smithi120.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 50 kB/s | 36 kB 00:00 2024-09-11T23:40:11.955 INFO:teuthology.orchestra.run.smithi120.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 80 kB/s | 47 kB 00:00 2024-09-11T23:40:12.255 INFO:teuthology.orchestra.run.smithi120.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 351 kB/s | 304 kB 00:00 2024-09-11T23:40:12.381 INFO:teuthology.orchestra.run.smithi120.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 95 kB/s | 58 kB 00:00 2024-09-11T23:40:12.672 INFO:teuthology.orchestra.run.smithi120.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 134 kB/s | 96 kB 00:00 2024-09-11T23:40:12.764 INFO:teuthology.orchestra.run.smithi120.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-09-11T23:40:12.890 INFO:teuthology.orchestra.run.smithi120.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-09-11T23:40:12.948 INFO:teuthology.orchestra.run.smithi120.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-11T23:40:12.977 INFO:teuthology.orchestra.run.smithi120.stdout:(105/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 89 kB/s | 64 kB 00:00 2024-09-11T23:40:13.007 INFO:teuthology.orchestra.run.smithi120.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.3 MB/s | 309 kB 00:00 2024-09-11T23:40:13.032 INFO:teuthology.orchestra.run.smithi120.stdout:(107/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 69 kB/s | 45 kB 00:00 2024-09-11T23:40:13.058 INFO:teuthology.orchestra.run.smithi120.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 232 kB/s | 19 kB 00:00 2024-09-11T23:40:13.091 INFO:teuthology.orchestra.run.smithi120.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T23:40:13.117 INFO:teuthology.orchestra.run.smithi120.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 310 kB/s | 26 kB 00:00 2024-09-11T23:40:13.142 INFO:teuthology.orchestra.run.smithi120.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-09-11T23:40:13.167 INFO:teuthology.orchestra.run.smithi120.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-11T23:40:13.218 INFO:teuthology.orchestra.run.smithi120.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-11T23:40:13.251 INFO:teuthology.orchestra.run.smithi120.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-11T23:40:13.310 INFO:teuthology.orchestra.run.smithi120.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-09-11T23:40:13.335 INFO:teuthology.orchestra.run.smithi120.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 384 kB/s | 32 kB 00:00 2024-09-11T23:40:13.361 INFO:teuthology.orchestra.run.smithi120.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-11T23:40:13.428 INFO:teuthology.orchestra.run.smithi120.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-11T23:40:13.453 INFO:teuthology.orchestra.run.smithi120.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-09-11T23:40:13.478 INFO:teuthology.orchestra.run.smithi120.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-09-11T23:40:13.504 INFO:teuthology.orchestra.run.smithi120.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-11T23:40:13.537 INFO:teuthology.orchestra.run.smithi120.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-09-11T23:40:13.554 INFO:teuthology.orchestra.run.smithi120.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 651 kB/s | 11 kB 00:00 2024-09-11T23:40:13.580 INFO:teuthology.orchestra.run.smithi120.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-11T23:40:13.605 INFO:teuthology.orchestra.run.smithi120.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-11T23:40:13.622 INFO:teuthology.orchestra.run.smithi120.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 554 kB/s | 23 kB 00:00 2024-09-11T23:40:13.639 INFO:teuthology.orchestra.run.smithi120.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 580 kB/s | 19 kB 00:00 2024-09-11T23:40:13.656 INFO:teuthology.orchestra.run.smithi120.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 588 kB/s | 20 kB 00:00 2024-09-11T23:40:13.681 INFO:teuthology.orchestra.run.smithi120.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:40:13.706 INFO:teuthology.orchestra.run.smithi120.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T23:40:13.732 INFO:teuthology.orchestra.run.smithi120.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-11T23:40:13.774 INFO:teuthology.orchestra.run.smithi120.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-11T23:40:13.808 INFO:teuthology.orchestra.run.smithi120.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-11T23:40:13.833 INFO:teuthology.orchestra.run.smithi120.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-11T23:40:13.858 INFO:teuthology.orchestra.run.smithi120.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-11T23:40:13.883 INFO:teuthology.orchestra.run.smithi120.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-11T23:40:13.908 INFO:teuthology.orchestra.run.smithi120.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-11T23:40:13.934 INFO:teuthology.orchestra.run.smithi120.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T23:40:13.967 INFO:teuthology.orchestra.run.smithi120.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-11T23:40:13.993 INFO:teuthology.orchestra.run.smithi120.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-09-11T23:40:14.027 INFO:teuthology.orchestra.run.smithi120.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-11T23:40:14.052 INFO:teuthology.orchestra.run.smithi120.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-11T23:40:14.086 INFO:teuthology.orchestra.run.smithi120.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-11T23:40:14.145 INFO:teuthology.orchestra.run.smithi120.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-09-11T23:40:14.170 INFO:teuthology.orchestra.run.smithi120.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-09-11T23:40:14.204 INFO:teuthology.orchestra.run.smithi120.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-11T23:40:14.229 INFO:teuthology.orchestra.run.smithi120.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-09-11T23:40:14.279 INFO:teuthology.orchestra.run.smithi120.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T23:40:14.578 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-09-11T23:40:14.579 INFO:teuthology.orchestra.run.smithi120.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 9.1 MB/s | 3.4 MB 00:00 2024-09-11T23:40:14.638 INFO:teuthology.orchestra.run.smithi120.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 7.7 MB/s | 3.2 MB 00:00 2024-09-11T23:40:14.670 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:40:14.709 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:40:14.713 INFO:teuthology.orchestra.run.smithi120.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 5.3 MB/s | 2.3 MB 00:00 2024-09-11T23:40:14.763 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:40:14.772 INFO:teuthology.orchestra.run.smithi120.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-11T23:40:14.804 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:40:14.829 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:40:14.856 INFO:teuthology.orchestra.run.smithi120.stdout:(153/156): NetworkManager-team-1.49.90-1.el9.x8 188 kB/s | 41 kB 00:00 2024-09-11T23:40:14.886 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:40:14.906 INFO:teuthology.orchestra.run.smithi120.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 5.5 MB/s | 1.8 MB 00:00 2024-09-11T23:40:14.940 INFO:teuthology.orchestra.run.smithi120.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 8.1 MB/s | 1.4 MB 00:00 2024-09-11T23:40:15.148 INFO:teuthology.orchestra.run.smithi120.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.2 MB/s | 2.4 MB 00:00 2024-09-11T23:40:15.155 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:40:15.155 INFO:teuthology.orchestra.run.smithi120.stdout:Total 9.4 MB/s | 226 MB 00:24 2024-09-11T23:40:15.238 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:40:15.347 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:40:15.473 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:40:15.499 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:40:15.587 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-11T23:40:15.591 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:40:15.620 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:40:15.689 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:40:15.828 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:40:15.859 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:40:15.886 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:40:15.992 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:40:16.008 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:40:16.020 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:40:16.066 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:40:16.072 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:40:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:40:16.107 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:40:16.124 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:40:16.151 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:40:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:40:16.384 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-11T23:40:16.482 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:40:16.533 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-11T23:40:16.533 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-11T23:40:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:40:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:40:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:40:16.721 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:40:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:40:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:40:16.833 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:40:16.864 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:40:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:40:17.021 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:40:17.085 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:40:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:40:17.113 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:40:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:40:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:40:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:40:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:40:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:40:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:40:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:40:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:40:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:40:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:40:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:40:17.534 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:40:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:40:17.605 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:40:17.662 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:40:17.681 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:40:17.689 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:40:17.760 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:40:17.797 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:40:17.813 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:40:17.843 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:40:17.886 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:40:17.895 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:40:17.957 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:40:17.963 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:40:17.997 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:40:18.041 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:40:18.068 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:40:18.149 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:40:18.184 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:40:18.192 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:40:18.268 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:40:18.294 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:40:18.341 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-11T23:40:18.342 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-11T23:40:18.382 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:40:18.404 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:40:18.429 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:40:18.462 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:40:18.467 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:40:18.531 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:40:18.585 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:40:18.617 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:40:18.660 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:40:18.693 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:40:18.721 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:40:18.789 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:40:18.843 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:40:18.882 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:40:18.926 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:40:19.001 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:40:19.008 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:40:19.037 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:40:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:40:19.091 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:40:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:40:19.135 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:40:19.178 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:40:19.180 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:40:19.216 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:40:19.223 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:40:19.267 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:40:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:40:19.439 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:40:19.492 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:40:19.587 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:40:19.761 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:40:19.817 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:40:19.890 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:40:19.999 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:40:20.017 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:40:20.123 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:40:20.159 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:40:20.202 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:40:20.240 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:40:20.241 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:40:20.281 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:40:20.365 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:40:20.398 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:40:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:40:20.442 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:40:20.485 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:40:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:40:20.542 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:40:20.630 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:40:20.647 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:40:20.668 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:40:20.691 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:40:20.709 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:40:20.737 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:40:20.774 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:40:20.814 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:40:20.817 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:40:20.851 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:40:20.867 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:40:20.896 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:40:20.924 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:40:20.949 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:40:21.003 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:40:21.044 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:40:21.056 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:40:21.086 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:40:21.187 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:40:21.196 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:40:21.231 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:40:21.236 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:40:21.260 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:40:21.276 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:40:21.298 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:40:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:40:21.326 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:40:21.341 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:40:21.381 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:40:21.400 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:40:21.411 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:40:21.445 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:40:21.507 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:40:21.529 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:40:21.549 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:40:21.561 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:40:21.583 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:40:21.589 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:40:21.623 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:40:21.662 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:40:21.672 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:40:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:40:21.714 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:40:21.732 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:40:21.745 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:40:21.766 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:40:21.799 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:40:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:40:21.851 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:40:21.876 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:40:21.904 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:40:21.963 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:40:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:40:22.046 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:40:22.149 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:40:22.197 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:40:22.659 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:40:22.709 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:40:22.743 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:40:22.845 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:40:22.899 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:40:22.910 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:40:22.944 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:40:22.969 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:40:23.003 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:40:23.033 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:40:23.079 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:40:23.100 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:40:23.158 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:40:23.202 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:40:23.269 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:40:23.281 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:40:23.309 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:40:23.330 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:40:23.389 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:40:23.522 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:40:23.561 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:40:23.638 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:40:24.070 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:40:24.178 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:40:24.278 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:40:24.384 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:40:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-11T23:40:24.910 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:40:24.949 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:40:25.011 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:40:25.045 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:40:25.070 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:40:25.114 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:40:25.460 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:40:25.489 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:40:25.572 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:40:25.572 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:40:25.601 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:40:25.608 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:40:25.634 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:40:25.686 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:40:25.714 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:40:25.789 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:40:25.819 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:40:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:40:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:40:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:40:25.965 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:40:25.965 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:40:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:40:26.025 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:40:26.053 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:40:26.098 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:40:26.126 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:40:26.157 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:40:26.163 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:40:26.193 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:40:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:40:26.250 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:40:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:40:26.281 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:40:26.293 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:40:26.371 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:40:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:40:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:40:26.465 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:40:26.595 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:40:26.667 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:40:26.691 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:40:26.703 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:40:26.754 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:40:26.784 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:40:26.811 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:40:26.888 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:40:26.926 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:40:26.959 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:40:27.014 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:40:27.045 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:40:27.109 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:40:27.227 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:40:27.321 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:40:27.360 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:40:27.395 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:40:27.463 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:40:27.508 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:40:27.540 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:40:27.584 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:40:27.616 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:40:27.645 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:40:27.721 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:40:27.782 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:40:27.814 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:40:27.849 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:40:27.925 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:40:27.960 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:40:28.004 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:40:28.032 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:40:28.101 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:40:28.156 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:40:28.229 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:40:28.363 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:40:28.416 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:40:28.941 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:40:29.029 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:40:29.059 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:40:29.101 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:40:29.139 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:40:29.181 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:40:29.214 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:40:29.256 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:40:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:40:29.361 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:40:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:40:29.432 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:40:29.486 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:40:29.558 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:40:29.600 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:40:29.636 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:40:29.690 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:40:29.727 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:40:29.772 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:40:29.814 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:40:29.833 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:40:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:40:29.926 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:40:29.950 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:40:29.961 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:40:29.962 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:40:29.962 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:40:29.962 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:40:29.962 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:30.000 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:40:30.002 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:40:30.041 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:40:30.057 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:40:30.057 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:40:30.057 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:40:30.057 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:40:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:40:30.200 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:40:30.240 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:40:30.266 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:40:30.290 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:40:30.365 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:40:30.492 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:40:30.523 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:40:30.551 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:40:30.585 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:40:30.633 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:40:30.674 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:40:30.703 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:40:30.738 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:40:30.771 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:40:30.813 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:40:30.847 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:40:30.935 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:40:30.986 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:40:31.026 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:40:31.129 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:40:31.185 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:40:31.900 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:40:31.949 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:40:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:40:32.091 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:40:32.147 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:40:32.191 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:40:32.270 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:40:32.311 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:40:32.494 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:40:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:40:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:40:33.233 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:40:33.340 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:40:34.569 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:40:34.648 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:40:34.676 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:40:34.709 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:40:35.020 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:40:35.072 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:40:35.147 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:40:35.186 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:40:35.221 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:40:35.704 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:40:35.734 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:40:35.818 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:40:35.844 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:40:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:40:38.166 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:40:38.708 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:40:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:40:38.836 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:40:38.836 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:40:38.837 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:40:38.837 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:40:38.863 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:40:41.883 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:40:41.883 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-09-11T23:40:41.884 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-09-11T23:40:41.884 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-09-11T23:40:41.884 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-09-11T23:40:41.884 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-09-11T23:40:41.884 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-09-11T23:40:41.884 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-09-11T23:40:41.884 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:41.935 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:40:41.935 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-09-11T23:40:41.935 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-09-11T23:40:41.935 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-09-11T23:40:41.935 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-09-11T23:40:41.936 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-09-11T23:40:41.936 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-09-11T23:40:41.936 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-09-11T23:40:41.936 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:42.058 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:40:42.074 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:40:42.074 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:42.075 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:40:42.075 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:42.107 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:40:42.123 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:40:42.123 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:42.123 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:40:42.123 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:43.571 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:40:43.597 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:40:43.598 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:43.598 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:40:43.598 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:40:43.598 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:40:43.598 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:43.660 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:40:43.688 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:40:43.688 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:43.688 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:40:43.689 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:40:43.689 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:40:43.689 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:43.822 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:40:43.845 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:40:43.845 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:43.846 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:40:43.846 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:40:43.846 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:40:43.846 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:43.928 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:40:43.952 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:40:43.952 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:43.953 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:40:43.953 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:40:43.953 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:40:43.953 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:44.280 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:40:44.304 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:40:44.304 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:44.305 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:40:44.305 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:40:44.305 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:40:44.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:40:44.412 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:40:44.436 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:40:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:40:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:40:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:40:44.436 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:40:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:40:44.542 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:40:44.601 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:40:44.664 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:40:44.717 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:40:44.764 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:40:44.842 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:40:44.857 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:40:44.905 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:40:44.939 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:40:44.984 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:40:45.074 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:40:45.087 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:40:45.140 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:40:45.180 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:40:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:40:45.223 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:40:45.285 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:40:45.320 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:40:45.347 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:40:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:40:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:40:45.738 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:40:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:40:45.771 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:40:45.800 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:40:45.803 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:40:45.839 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:40:45.867 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:40:45.902 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:40:46.885 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:40:46.923 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:40:47.002 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:40:47.037 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:40:47.888 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:40:47.916 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:40:48.019 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:40:48.051 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:40:48.079 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:40:48.186 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:40:48.195 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:40:48.226 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:40:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:40:48.251 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:48.251 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:40:48.251 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:40:48.251 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:40:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:40:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:40:48.348 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:40:48.381 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:40:48.407 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:40:48.407 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:48.407 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:40:48.407 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:40:48.407 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:40:48.407 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:48.412 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:40:48.457 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:40:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:40:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:40:48.570 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:40:48.625 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:40:49.397 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:40:49.429 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:40:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:40:49.453 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:49.453 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:40:49.453 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:40:49.453 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:40:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:49.505 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:40:49.528 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:40:49.528 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:49.529 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:40:49.529 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:49.581 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:40:49.597 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:40:49.604 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:40:49.605 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:49.605 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:40:49.605 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:40:49.605 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:40:49.605 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:49.626 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:40:49.650 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:40:49.650 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:49.650 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:40:49.650 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:40:49.650 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:40:49.651 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:49.693 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:40:49.717 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:40:49.717 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:49.717 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:40:49.718 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:49.769 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:40:49.793 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:40:49.793 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:49.794 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:40:49.794 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:40:49.794 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:40:49.794 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:49.907 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:40:49.930 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:40:49.930 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:49.930 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:40:49.931 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:40:49.931 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:40:49.931 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:40:50.111 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:40:50.136 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:40:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:40:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:40:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:40:50.136 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:40:50.538 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:40:50.538 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-09-11T23:40:50.538 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-09-11T23:40:50.538 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-09-11T23:40:50.539 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-09-11T23:40:50.539 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-09-11T23:40:50.539 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-09-11T23:40:50.539 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-09-11T23:40:50.539 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:40:50.703 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:40:50.719 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:40:50.719 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:50.719 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:40:50.719 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:40:52.206 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:40:52.233 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:40:52.233 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:52.233 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:40:52.233 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:40:52.233 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:40:52.234 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:40:52.466 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:40:52.489 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:40:52.490 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:52.490 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:40:52.490 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:40:52.490 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:40:52.490 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:40:52.934 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:40:52.958 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:40:52.958 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:52.958 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:40:52.958 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:40:52.958 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:40:52.958 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:40:53.064 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:40:53.121 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:40:53.176 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:40:53.363 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:40:53.411 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:40:53.488 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:40:53.578 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:40:53.716 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:40:53.753 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:40:53.784 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:40:53.812 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:40:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:40:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:40:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:40:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:40:54.385 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:40:54.469 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:40:54.510 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:40:54.559 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:40:54.600 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:40:54.635 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:40:54.827 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:40:54.874 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:40:54.904 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:40:54.937 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:40:54.957 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:40:54.980 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:40:55.007 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:40:55.048 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:40:55.079 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:40:55.106 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:40:55.314 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:40:55.353 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:40:55.383 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:40:55.413 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:40:55.425 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:40:55.459 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:40:55.462 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:40:55.586 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:40:55.628 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:40:55.650 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:40:56.091 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:40:56.140 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:40:56.161 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:40:56.425 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:40:56.454 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:40:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:40:56.554 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:40:56.600 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:40:56.641 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:40:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:40:56.723 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:40:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:40:56.753 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:40:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:40:56.764 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:56.764 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:40:56.764 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:40:56.764 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:40:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:40:56.779 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:40:56.813 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:40:56.813 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:40:56.816 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:40:56.828 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:40:56.905 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:40:56.906 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:40:56.925 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:40:56.972 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:40:57.006 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:40:57.032 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:40:57.037 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:40:57.037 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:40:57.044 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:40:57.145 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:40:57.197 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:40:57.220 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:40:57.220 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:40:57.294 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:40:57.301 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:40:57.323 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:40:57.348 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:40:57.364 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:40:57.374 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:40:57.374 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:40:57.458 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:40:57.459 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:40:57.534 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:40:57.559 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:40:57.564 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:40:57.564 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:40:57.752 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:40:57.752 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:40:57.845 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:40:57.895 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:40:57.913 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:40:57.935 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:40:57.959 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:40:57.959 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:57.959 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:40:57.959 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:40:57.959 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:40:57.959 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:40:58.003 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:40:58.026 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:40:58.026 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:58.026 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:40:58.026 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:40:58.079 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:40:58.102 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:40:58.102 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:58.102 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:40:58.102 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:40:58.102 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:40:58.102 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:40:58.413 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:40:58.436 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:40:58.436 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:40:58.436 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:40:58.436 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:40:58.436 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:40:58.436 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:41:02.983 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:41:03.025 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:41:03.065 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:41:03.098 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:41:03.125 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:41:03.305 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:41:03.348 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:41:03.377 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:41:03.407 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:41:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:41:03.455 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:41:03.455 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:41:03.456 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:41:03.456 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:41:03.456 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:41:03.457 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:41:03.457 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:41:03.457 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:41:03.457 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:41:03.457 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:41:03.458 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:41:03.458 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:41:03.458 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:41:03.458 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:41:03.459 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:41:03.459 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:41:03.459 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:41:03.459 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:41:03.459 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:41:03.460 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:41:03.460 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:41:03.460 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:41:03.460 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:41:03.460 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:41:03.461 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:41:03.461 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:41:03.461 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:41:03.461 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:41:03.461 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:41:03.462 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:41:03.462 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:41:03.462 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:41:03.462 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:41:03.462 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:41:03.462 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:41:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:41:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:41:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:41:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:41:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:41:03.464 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:41:03.464 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:41:03.464 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:41:03.464 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:41:03.464 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:41:03.465 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:41:03.465 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:41:03.465 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:41:03.465 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:41:03.465 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:41:03.465 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:41:03.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:41:03.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:41:03.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:41:03.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:41:03.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:41:03.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:41:03.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:41:03.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:41:03.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:41:03.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:41:03.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:41:03.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:41:03.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:41:03.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:41:03.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:41:03.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:41:03.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:41:03.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:41:03.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:41:03.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:41:03.470 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:41:03.471 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:41:03.472 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:41:03.473 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:41:03.474 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:41:03.475 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:41:03.476 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:41:03.476 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:41:03.476 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:41:03.476 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:41:03.476 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:41:03.476 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:41:03.490 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:41:03.491 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:41:03.492 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:41:03.492 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:41:03.492 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:41:03.492 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:41:03.492 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:41:03.492 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:41:03.492 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:41:03.492 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:41:03.492 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:41:03.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:41:03.494 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:41:03.495 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:41:03.495 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:41:03.495 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:41:03.495 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:41:03.495 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:41:03.495 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:41:03.495 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:41:03.496 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:41:03.496 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:41:03.496 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:41:03.496 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:41:03.496 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:41:03.496 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:41:03.496 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:41:03.496 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:41:03.496 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:41:03.496 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:41:03.497 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:41:03.497 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:41:03.497 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:41:03.497 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:41:03.497 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:41:03.497 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:41:03.497 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:41:03.498 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:41:03.499 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:41:03.500 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:41:03.500 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:41:03.500 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:41:03.500 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:41:03.500 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:41:03.500 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:41:03.500 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:41:03.500 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:41:03.500 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:41:03.500 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:41:03.501 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:41:03.502 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:41:03.503 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:41:03.504 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:41:03.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:41:03.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:41:03.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:41:03.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:41:03.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:41:03.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:41:03.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:41:03.505 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:41:04.077 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:41:04.119 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:41:04.140 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:41:04.974 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:41:05.059 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:41:05.100 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:41:05.180 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:41:05.220 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:41:05.246 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:41:05.272 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:41:05.272 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:41:05.356 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:41:05.356 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:41:05.440 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:41:05.465 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:41:05.471 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:41:05.471 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:41:05.651 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:41:05.651 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:41:05.727 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:41:05.791 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:41:05.962 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:41:05.962 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:41:05.962 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-09-11T23:41:05.962 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:05.962 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:05.962 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:05.962 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:05.962 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.962 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.962 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.963 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.964 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:41:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:41:05.967 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:41:05.968 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:41:05.969 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:41:05.970 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:41:05.971 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:41:05.972 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:41:05.973 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:41:05.974 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.974 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.974 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:05.974 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:41:05.974 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:41:05.974 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:41:05.974 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:41:05.974 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:41:05.974 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:41:05.975 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:41:05.975 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:41:05.975 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-11T23:41:06.388 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:41:06.388 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:41:06.388 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-09-11T23:41:06.388 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:06.388 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:06.388 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:06.388 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:06.388 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.388 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.389 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:41:06.389 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:41:06.389 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-09-11T23:41:06.389 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:41:06.389 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:41:06.389 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:41:06.389 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:41:06.389 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.389 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.390 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.390 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.390 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:06.390 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.390 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.390 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.390 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:06.390 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:06.390 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:06.391 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.392 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.392 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:06.392 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:41:06.392 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:41:06.392 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:41:06.392 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:41:06.392 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:41:06.392 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:41:06.392 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:41:06.392 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:41:06.393 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:41:06.394 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.394 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.394 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:41:06.394 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.394 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:41:06.394 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:41:06.394 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:41:06.394 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:41:06.394 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:41:06.395 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:41:06.395 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:41:06.395 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:41:06.395 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:41:06.395 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:41:06.395 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:41:06.395 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:41:06.395 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:41:06.395 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:41:06.395 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:41:06.396 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:41:06.396 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:41:06.396 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:41:06.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:41:06.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:41:06.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:41:06.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:41:06.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:41:06.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:41:06.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:41:06.398 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:41:06.398 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:41:06.398 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:41:06.398 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:41:06.398 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:41:06.398 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:41:06.398 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:41:06.398 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:41:06.399 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:41:06.399 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:41:06.399 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:41:06.399 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:41:06.399 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:41:06.399 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:41:06.399 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:41:06.399 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:41:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:41:06.401 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:41:06.401 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:41:06.401 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.401 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.401 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:41:06.401 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:41:06.401 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:41:06.401 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.401 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:41:06.401 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:41:06.402 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:41:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:41:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:41:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:41:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:41:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:41:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:41:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:41:06.404 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:41:06.404 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:41:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-09-11T23:41:06.408 DEBUG:teuthology.parallel:result is None 2024-09-11T23:41:06.813 DEBUG:teuthology.parallel:result is None 2024-09-11T23:41:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:41:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:41:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:41:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:41:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:41:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:41:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:41:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:41:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:41:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:41:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:41:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:41:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:41:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:41:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:41:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:41:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:41:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:41:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:41:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:41:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:41:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:41:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:41:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:41:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:41:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:41:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:41:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:41:14.308 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:41:14.308 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:41:14.308 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-09-11T23:41:14.308 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:14.308 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:14.308 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:14.308 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.309 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.310 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:41:14.311 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:41:14.312 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:41:14.313 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:41:14.314 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:41:14.315 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:41:14.316 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:41:14.317 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.318 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:41:14.319 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:41:14.320 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:41:14.321 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:41:14.321 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:41:14.321 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-11T23:41:14.763 DEBUG:teuthology.parallel:result is None 2024-09-11T23:41:14.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:41:14.919 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:41:14.947 INFO:teuthology.orchestra.run.smithi062.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:41:14.948 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:41:14.948 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:41:14.950 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:41:15.104 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:41:15.130 INFO:teuthology.orchestra.run.smithi083.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:41:15.131 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:41:15.131 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:41:15.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:41:15.281 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:41:15.304 INFO:teuthology.orchestra.run.smithi120.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:41:15.304 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:41:15.304 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:41:15.306 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T23:41:15.307 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:41:15.307 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:41:15.341 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:41:15.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:41:15.373 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:41:15.373 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:41:15.401 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T23:41:15.401 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:41:15.401 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:41:15.432 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:41:15.501 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:41:15.501 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:41:15.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:41:15.600 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:41:15.600 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:41:15.626 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:41:15.693 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T23:41:15.693 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:41:15.694 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:41:15.725 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:41:15.794 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:41:15.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:41:15.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:41:15.892 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:41:15.892 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:41:15.919 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:41:15.986 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T23:41:15.987 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:41:15.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:41:16.017 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:41:16.088 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:41:16.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:41:16.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:41:16.184 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:41:16.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:41:16.211 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:41:16.278 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T23:41:16.375 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-11T23:41:16.375 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:41:16.376 INFO:tasks.cephadm:Cluster fsid is 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:41:16.376 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T23:41:16.376 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.62', 'mon.b': '172.21.15.83', 'mon.c': '172.21.15.120'} 2024-09-11T23:41:16.376 INFO:tasks.cephadm:First mon is mon.a on smithi062 2024-09-11T23:41:16.376 INFO:tasks.cephadm:First mgr is x 2024-09-11T23:41:16.376 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T23:41:16.377 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2024-09-11T23:41:16.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-09-11T23:41:16.438 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname $(hostname -s) 2024-09-11T23:41:16.464 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-11T23:41:16.464 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:41:16.613 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-11T23:41:16.761 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:41:16.763 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T23:41:16.763 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T23:41:16.763 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:41:17.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:41 /home/ubuntu/cephtest/cephadm 2024-09-11T23:41:17.134 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:41:17.493 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:41 /home/ubuntu/cephtest/cephadm 2024-09-11T23:41:17.493 DEBUG:teuthology.orchestra.run.smithi120:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:41:17.884 INFO:teuthology.orchestra.run.smithi120.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:41 /home/ubuntu/cephtest/cephadm 2024-09-11T23:41:17.885 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:41:17.910 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:41:17.935 DEBUG:teuthology.orchestra.run.smithi120:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:41:17.971 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-11T23:41:17.971 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:41:17.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:41:17.978 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:41:19.600 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:41:19.637 INFO:teuthology.orchestra.run.smithi120.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:41:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:41:50.325 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:41:50.325 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:41:50.325 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:41:50.325 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2024-09-11T23:41:50.325 INFO:teuthology.orchestra.run.smithi062.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:41:50.325 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-11T23:41:50.325 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:41:57.672 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-11T23:41:57.673 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:41:57.673 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:41:57.673 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-09-11T23:41:57.673 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:41:57.673 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-11T23:41:57.673 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-11T23:42:07.956 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-11T23:42:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:42:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:42:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: "repo_digests": [ 2024-09-11T23:42:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:42:07.956 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-11T23:42:07.956 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-11T23:42:07.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2024-09-11T23:42:08.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-09-11T23:42:08.052 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph 2024-09-11T23:42:08.081 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2024-09-11T23:42:08.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-09-11T23:42:08.138 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /etc/ceph 2024-09-11T23:42:08.164 INFO:tasks.cephadm:Writing seed config... 2024-09-11T23:42:08.165 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-11T23:42:08.166 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-11T23:42:08.167 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-11T23:42:08.168 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-11T23:42:08.169 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T23:42:08.169 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-11T23:42:08.169 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:42:08.169 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T23:42:08.187 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 56675698-7097-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-11T23:42:08.188 DEBUG:teuthology.orchestra.run.smithi062:mon.a> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T23:42:08.230 DEBUG:teuthology.orchestra.run.smithi062:mgr.x> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T23:42:08.274 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T23:42:08.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 56675698-7097-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.62 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:42:08.488 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:42:08.488 INFO:teuthology.orchestra.run.smithi062.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '56675698-7097-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.62', '--skip-admin-label'] 2024-09-11T23:42:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T23:42:08.489 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying podman|docker is present... 2024-09-11T23:42:08.512 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 5.2.2 2024-09-11T23:42:08.512 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying lvm2 is present... 2024-09-11T23:42:08.513 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying time synchronization is in place... 2024-09-11T23:42:08.520 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T23:42:08.520 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T23:42:08.527 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T23:42:08.527 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-09-11T23:42:08.534 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2024-09-11T23:42:08.541 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2024-09-11T23:42:08.541 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2024-09-11T23:42:08.541 INFO:teuthology.orchestra.run.smithi062.stdout:Repeating the final host check... 2024-09-11T23:42:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 5.2.2 2024-09-11T23:42:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T23:42:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl is present 2024-09-11T23:42:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:lvcreate is present 2024-09-11T23:42:08.571 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T23:42:08.571 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T23:42:08.577 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T23:42:08.577 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-09-11T23:42:08.584 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2024-09-11T23:42:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2024-09-11T23:42:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2024-09-11T23:42:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:Host looks OK 2024-09-11T23:42:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:Cluster fsid: 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:Acquiring lock 139649713251328 on /run/cephadm/56675698-7097-11ef-bcea-c7b262605968.lock 2024-09-11T23:42:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:Lock 139649713251328 acquired on /run/cephadm/56675698-7097-11ef-bcea-c7b262605968.lock 2024-09-11T23:42:08.592 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 3300 ... 2024-09-11T23:42:08.592 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 6789 ... 2024-09-11T23:42:08.592 INFO:teuthology.orchestra.run.smithi062.stdout:Base mon IP(s) is [172.21.15.62:3300, 172.21.15.62:6789], mon addrv is [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-09-11T23:42:08.595 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.62 metric 100 2024-09-11T23:42:08.595 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.62 metric 100 2024-09-11T23:42:08.598 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T23:42:08.598 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T23:42:08.598 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T23:42:08.600 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T23:42:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T23:42:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T23:42:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T23:42:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febb:d1c3/64 scope link noprefixroute 2024-09-11T23:42:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T23:42:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-09-11T23:42:08.602 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-09-11T23:42:08.602 INFO:teuthology.orchestra.run.smithi062.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T23:42:08.602 INFO:teuthology.orchestra.run.smithi062.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T23:42:08.603 INFO:teuthology.orchestra.run.smithi062.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:42:09.091 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T23:42:09.091 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:42:09.092 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T23:42:09.092 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-11T23:42:09.092 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T23:42:09.092 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T23:42:09.092 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T23:42:09.972 INFO:teuthology.orchestra.run.smithi062.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T23:42:09.972 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T23:42:09.972 INFO:teuthology.orchestra.run.smithi062.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T23:42:10.962 INFO:teuthology.orchestra.run.smithi062.stdout:stat: stdout 167 167 2024-09-11T23:42:10.962 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial keys... 2024-09-11T23:42:11.901 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDTKuJmPN4uERAAWJPeXoi7BWgz4X+cmtyRHQ== 2024-09-11T23:42:12.874 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDUKuJmicSYCxAAReVW8ZseudBuWwKFIkAYaQ== 2024-09-11T23:42:13.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDVKuJm8u+HDBAA59TdRB6hgnkkjNcKp8yZ0Q== 2024-09-11T23:42:13.756 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial monmap... 2024-09-11T23:42:14.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T23:42:14.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-11T23:42:14.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:14.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T23:42:14.671 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool for a [v2:172.21.15.62:3300,v1:172.21.15.62:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T23:42:14.671 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = quincy 2024-09-11T23:42:14.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: set fsid to 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:14.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T23:42:14.671 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:42:14.671 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mon... 2024-09-11T23:42:15.609 INFO:teuthology.orchestra.run.smithi062.stdout:create mon.a on 2024-09-11T23:42:16.089 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T23:42:16.268 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T23:42:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-56675698-7097-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-56675698-7097-11ef-bcea-c7b262605968.target. 2024-09-11T23:42:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-56675698-7097-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-56675698-7097-11ef-bcea-c7b262605968.target. 2024-09-11T23:42:17.531 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-56675698-7097-11ef-bcea-c7b262605968@mon.a 2024-09-11T23:42:17.531 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-56675698-7097-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-56675698-7097-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T23:42:17.768 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-56675698-7097-11ef-bcea-c7b262605968.target.wants/ceph-56675698-7097-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-56675698-7097-11ef-bcea-c7b262605968@.service. 2024-09-11T23:42:18.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:18 smithi062 podman[30571]: 2024-09-11 23:42:17.906629385 +0000 UTC m=+0.016654912 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:42:19.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:18 smithi062 podman[30571]: 2024-09-11 23:42:18.750518178 +0000 UTC m=+0.860543702 container create b9f2805f864b23471047c4fb09db8182b8254cc00312f030263a11ebcace5a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:42:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:20 smithi062 podman[30571]: 2024-09-11 23:42:20.52375662 +0000 UTC m=+2.633782147 container init b9f2805f864b23471047c4fb09db8182b8254cc00312f030263a11ebcace5a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T23:42:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:20 smithi062 podman[30571]: 2024-09-11 23:42:20.528305329 +0000 UTC m=+2.638330854 container start b9f2805f864b23471047c4fb09db8182b8254cc00312f030263a11ebcace5a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:42:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:20 smithi062 ceph-mon[30590]: mkfs 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:20 smithi062 ceph-mon[30590]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T23:42:21.361 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:42:21.361 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-09-11T23:42:21.369 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:42:21.369 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-09-11T23:42:21.369 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-09-11T23:42:21.369 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T23:42:21.370 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon to start... 2024-09-11T23:42:21.370 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon... 2024-09-11T23:42:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 bash[30571]: b9f2805f864b23471047c4fb09db8182b8254cc00312f030263a11ebcace5a23 2024-09-11T23:42:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 systemd[1]: Started Ceph mon.a for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:42:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T23:42:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: monmap epoch 1 2024-09-11T23:42:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: last_changed 2024-09-11T23:42:14.107883+0000 2024-09-11T23:42:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:42:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: min_mon_release 19 (squid) 2024-09-11T23:42:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: election_strategy: 1 2024-09-11T23:42:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:42:21.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: fsmap 2024-09-11T23:42:21.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T23:42:21.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:21 smithi062 ceph-mon[30590]: mgrmap e1: no daemons active 2024-09-11T23:42:24.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:24 smithi062 ceph-mon[30590]: from='client.? 172.21.15.62:0/4039180097' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T23:42:27.272 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T23:42:27.272 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout id: 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout services: 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 3s) 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout data: 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T23:42:27.273 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.332+0000 7f33aae30640 1 Processor -- start 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.332+0000 7f33aae30640 1 -- start start 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.332+0000 7f33aae30640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a4107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.332+0000 7f33aae30640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f33a4107ff0 con 0x7f33a4107620 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.333+0000 7f33a8bcc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a4107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.333+0000 7f33a8bcc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a4107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38270/0 (socket says 172.21.15.62:38270) 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.333+0000 7f33a8bcc640 1 -- 172.21.15.62:0/3267787301 learned_addr learned my addr 172.21.15.62:0/3267787301 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.333+0000 7f33a8bcc640 1 -- 172.21.15.62:0/3267787301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a4108820 con 0x7f33a4107620 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.336+0000 7f33a8bcc640 1 --2- 172.21.15.62:0/3267787301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a4107a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3394009820 tx=0x7f339402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acaac3314b413d77 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.336+0000 7f33a37fe640 1 -- 172.21.15.62:0/3267787301 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3394037070 con 0x7f33a4107620 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.336+0000 7f33a37fe640 1 -- 172.21.15.62:0/3267787301 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3394032440 con 0x7f33a4107620 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.336+0000 7f33a37fe640 1 -- 172.21.15.62:0/3267787301 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3394032740 con 0x7f33a4107620 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.336+0000 7f33aae30640 1 -- 172.21.15.62:0/3267787301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 msgr2=0x7f33a4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.336+0000 7f33aae30640 1 --2- 172.21.15.62:0/3267787301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a4107a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3394009820 tx=0x7f339402a270 comp rx=0 tx=0).stop 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.337+0000 7f33aae30640 1 -- 172.21.15.62:0/3267787301 shutdown_connections 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.337+0000 7f33aae30640 1 --2- 172.21.15.62:0/3267787301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a4107a20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.337+0000 7f33aae30640 1 -- 172.21.15.62:0/3267787301 >> 172.21.15.62:0/3267787301 conn(0x7f33a4102d90 msgr2=0x7f33a41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.337+0000 7f33aae30640 1 -- 172.21.15.62:0/3267787301 shutdown_connections 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.337+0000 7f33aae30640 1 -- 172.21.15.62:0/3267787301 wait complete. 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.337+0000 7f33aae30640 1 Processor -- start 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.337+0000 7f33aae30640 1 -- start start 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33aae30640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:27.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33aae30640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f33a4108dc0 con 0x7f33a4107620 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33a8bcc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33a8bcc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38280/0 (socket says 172.21.15.62:38280) 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33a8bcc640 1 -- 172.21.15.62:0/4039180097 learned_addr learned my addr 172.21.15.62:0/4039180097 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33a8bcc640 1 -- 172.21.15.62:0/4039180097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a419b300 con 0x7f33a4107620 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33a8bcc640 1 --2- 172.21.15.62:0/4039180097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a419adc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f33940090b0 tx=0x7f33940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33a1ffb640 1 -- 172.21.15.62:0/4039180097 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3394041070 con 0x7f33a4107620 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33a1ffb640 1 -- 172.21.15.62:0/4039180097 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339403c7a0 con 0x7f33a4107620 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33aae30640 1 -- 172.21.15.62:0/4039180097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a419b590 con 0x7f33a4107620 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.338+0000 7f33a1ffb640 1 -- 172.21.15.62:0/4039180097 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3394037040 con 0x7f33a4107620 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.339+0000 7f33aae30640 1 -- 172.21.15.62:0/4039180097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a419e280 con 0x7f33a4107620 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.339+0000 7f33a1ffb640 1 -- 172.21.15.62:0/4039180097 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3394044070 con 0x7f33a4107620 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.339+0000 7f33a1ffb640 1 -- 172.21.15.62:0/4039180097 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f339403d3f0 con 0x7f33a4107620 2024-09-11T23:42:27.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.340+0000 7f33aae30640 1 -- 172.21.15.62:0/4039180097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a4109410 con 0x7f33a4107620 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.342+0000 7f33a1ffb640 1 -- 172.21.15.62:0/4039180097 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3394046450 con 0x7f33a4107620 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.404+0000 7f33aae30640 1 -- 172.21.15.62:0/4039180097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f33a419e7e0 con 0x7f33a4107620 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.404+0000 7f33a1ffb640 1 -- 172.21.15.62:0/4039180097 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7f3394046630 con 0x7f33a4107620 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.405+0000 7f33aae30640 1 -- 172.21.15.62:0/4039180097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 msgr2=0x7f33a419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.405+0000 7f33aae30640 1 --2- 172.21.15.62:0/4039180097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a419adc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f33940090b0 tx=0x7f33940046c0 comp rx=0 tx=0).stop 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.405+0000 7f33aae30640 1 -- 172.21.15.62:0/4039180097 shutdown_connections 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.405+0000 7f33aae30640 1 --2- 172.21.15.62:0/4039180097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a4107620 0x7f33a419adc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.405+0000 7f33aae30640 1 -- 172.21.15.62:0/4039180097 >> 172.21.15.62:0/4039180097 conn(0x7f33a4102d90 msgr2=0x7f33a418f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.405+0000 7f33aae30640 1 -- 172.21.15.62:0/4039180097 shutdown_connections 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:24.405+0000 7f33aae30640 1 -- 172.21.15.62:0/4039180097 wait complete. 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:mon is available 2024-09-11T23:42:27.277 INFO:teuthology.orchestra.run.smithi062.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T23:42:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:27 smithi062 ceph-mon[30590]: from='client.? 172.21.15.62:0/1836356747' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T23:42:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:27 smithi062 ceph-mon[30590]: from='client.? 172.21.15.62:0/1836356747' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T23:42:28.428 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:28.428 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T23:42:28.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:28.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-09-11T23:42:28.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T23:42:28.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T23:42:28.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T23:42:28.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T23:42:28.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:28.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T23:42:28.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T23:42:28.429 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.658+0000 7faf80adc640 1 Processor -- start 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.659+0000 7faf80adc640 1 -- start start 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.659+0000 7faf80adc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c10a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.659+0000 7faf80adc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c07bdb0 con 0x7faf7c108100 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.659+0000 7faf7a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c10a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.659+0000 7faf7a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c10a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38296/0 (socket says 172.21.15.62:38296) 2024-09-11T23:42:28.430 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.659+0000 7faf7a59c640 1 -- 172.21.15.62:0/2865464163 learned_addr learned my addr 172.21.15.62:0/2865464163 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.660+0000 7faf7a59c640 1 -- 172.21.15.62:0/2865464163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c10aa30 con 0x7faf7c108100 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.660+0000 7faf7a59c640 1 --2- 172.21.15.62:0/2865464163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c10a4f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009820 tx=0x7faf6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c813db30feeacce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.660+0000 7faf79d9b640 1 -- 172.21.15.62:0/2865464163 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf6c037070 con 0x7faf7c108100 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.661+0000 7faf79d9b640 1 -- 172.21.15.62:0/2865464163 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c032440 con 0x7faf7c108100 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.661+0000 7faf79d9b640 1 -- 172.21.15.62:0/2865464163 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf6c032740 con 0x7faf7c108100 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.661+0000 7faf80adc640 1 -- 172.21.15.62:0/2865464163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 msgr2=0x7faf7c10a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.661+0000 7faf80adc640 1 --2- 172.21.15.62:0/2865464163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c10a4f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009820 tx=0x7faf6c02a270 comp rx=0 tx=0).stop 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.661+0000 7faf80adc640 1 -- 172.21.15.62:0/2865464163 shutdown_connections 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.661+0000 7faf80adc640 1 --2- 172.21.15.62:0/2865464163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c10a4f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.661+0000 7faf80adc640 1 -- 172.21.15.62:0/2865464163 >> 172.21.15.62:0/2865464163 conn(0x7faf7c101ca0 msgr2=0x7faf7c1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.661+0000 7faf80adc640 1 -- 172.21.15.62:0/2865464163 shutdown_connections 2024-09-11T23:42:28.431 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.661+0000 7faf80adc640 1 -- 172.21.15.62:0/2865464163 wait complete. 2024-09-11T23:42:28.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.662+0000 7faf80adc640 1 Processor -- start 2024-09-11T23:42:28.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.662+0000 7faf80adc640 1 -- start start 2024-09-11T23:42:28.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.662+0000 7faf80adc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c107980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:28.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.662+0000 7faf80adc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c10ae50 con 0x7faf7c108100 2024-09-11T23:42:28.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.662+0000 7faf7a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:28.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.662+0000 7faf7a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38302/0 (socket says 172.21.15.62:38302) 2024-09-11T23:42:28.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.662+0000 7faf7a59c640 1 -- 172.21.15.62:0/1836356747 learned_addr learned my addr 172.21.15.62:0/1836356747 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:28.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.663+0000 7faf7a59c640 1 -- 172.21.15.62:0/1836356747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c107ec0 con 0x7faf7c108100 2024-09-11T23:42:28.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.663+0000 7faf7a59c640 1 --2- 172.21.15.62:0/1836356747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c107980 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0090b0 tx=0x7faf6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:28.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.663+0000 7faf737fe640 1 -- 172.21.15.62:0/1836356747 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf6c041070 con 0x7faf7c108100 2024-09-11T23:42:28.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.663+0000 7faf737fe640 1 -- 172.21.15.62:0/1836356747 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c03c7a0 con 0x7faf7c108100 2024-09-11T23:42:28.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.663+0000 7faf737fe640 1 -- 172.21.15.62:0/1836356747 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf6c037040 con 0x7faf7c108100 2024-09-11T23:42:28.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.663+0000 7faf80adc640 1 -- 172.21.15.62:0/1836356747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c1060b0 con 0x7faf7c108100 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.663+0000 7faf80adc640 1 -- 172.21.15.62:0/1836356747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c1065f0 con 0x7faf7c108100 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.663+0000 7faf737fe640 1 -- 172.21.15.62:0/1836356747 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faf6c044070 con 0x7faf7c108100 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.663+0000 7faf737fe640 1 -- 172.21.15.62:0/1836356747 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faf6c03d3f0 con 0x7faf7c108100 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.664+0000 7faf80adc640 1 -- 172.21.15.62:0/1836356747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7c108560 con 0x7faf7c108100 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.666+0000 7faf737fe640 1 -- 172.21.15.62:0/1836356747 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7faf6c046450 con 0x7faf7c108100 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.726+0000 7faf80adc640 1 -- 172.21.15.62:0/1836356747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7faf7c1946f0 con 0x7faf7c108100 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.729+0000 7faf737fe640 1 -- 172.21.15.62:0/1836356747 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7faf6c046630 con 0x7faf7c108100 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.730+0000 7faf737fe640 1 -- 172.21.15.62:0/1836356747 <== mon.0 v2:172.21.15.62:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faf6c0584b0 con 0x7faf7c108100 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.731+0000 7faf80adc640 1 -- 172.21.15.62:0/1836356747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 msgr2=0x7faf7c107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.731+0000 7faf80adc640 1 --2- 172.21.15.62:0/1836356747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c107980 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0090b0 tx=0x7faf6c0046c0 comp rx=0 tx=0).stop 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.731+0000 7faf80adc640 1 -- 172.21.15.62:0/1836356747 shutdown_connections 2024-09-11T23:42:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.731+0000 7faf80adc640 1 --2- 172.21.15.62:0/1836356747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c108100 0x7faf7c107980 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:28.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.731+0000 7faf80adc640 1 -- 172.21.15.62:0/1836356747 >> 172.21.15.62:0/1836356747 conn(0x7faf7c101ca0 msgr2=0x7faf7c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:28.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.731+0000 7faf80adc640 1 -- 172.21.15.62:0/1836356747 shutdown_connections 2024-09-11T23:42:28.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:27.731+0000 7faf80adc640 1 -- 172.21.15.62:0/1836356747 wait complete. 2024-09-11T23:42:28.435 INFO:teuthology.orchestra.run.smithi062.stdout:Generating new minimal ceph.conf... 2024-09-11T23:42:29.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:28 smithi062 ceph-mon[30590]: from='client.? 172.21.15.62:0/3807928238' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:42:29.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.847+0000 7f6fe8dbd640 1 Processor -- start 2024-09-11T23:42:29.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.847+0000 7f6fe8dbd640 1 -- start start 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.847+0000 7f6fe8dbd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe4107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.847+0000 7f6fe8dbd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe4107ff0 con 0x7f6fe4107620 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.847+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe4107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.847+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe4107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38316/0 (socket says 172.21.15.62:38316) 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.847+0000 7f6fe259c640 1 -- 172.21.15.62:0/1359300440 learned_addr learned my addr 172.21.15.62:0/1359300440 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.847+0000 7f6fe259c640 1 -- 172.21.15.62:0/1359300440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe4108820 con 0x7f6fe4107620 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.848+0000 7f6fe259c640 1 --2- 172.21.15.62:0/1359300440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe4107a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009820 tx=0x7f6fd402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf32252b8d876618 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.848+0000 7f6fe1d9b640 1 -- 172.21.15.62:0/1359300440 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fd4037070 con 0x7f6fe4107620 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.848+0000 7f6fe1d9b640 1 -- 172.21.15.62:0/1359300440 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6fd4032900 con 0x7f6fe4107620 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.848+0000 7f6fe1d9b640 1 -- 172.21.15.62:0/1359300440 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fd4030850 con 0x7f6fe4107620 2024-09-11T23:42:29.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.849+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/1359300440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 msgr2=0x7f6fe4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.849+0000 7f6fe8dbd640 1 --2- 172.21.15.62:0/1359300440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe4107a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009820 tx=0x7f6fd402a270 comp rx=0 tx=0).stop 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.849+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/1359300440 shutdown_connections 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.849+0000 7f6fe8dbd640 1 --2- 172.21.15.62:0/1359300440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe4107a20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.849+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/1359300440 >> 172.21.15.62:0/1359300440 conn(0x7f6fe4102db0 msgr2=0x7f6fe41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.849+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/1359300440 shutdown_connections 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.849+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/1359300440 wait complete. 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.850+0000 7f6fe8dbd640 1 Processor -- start 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.850+0000 7f6fe8dbd640 1 -- start start 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.850+0000 7f6fe8dbd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.850+0000 7f6fe8dbd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe410bbd0 con 0x7f6fe4107620 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.850+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.850+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe419d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38332/0 (socket says 172.21.15.62:38332) 2024-09-11T23:42:29.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.850+0000 7f6fe259c640 1 -- 172.21.15.62:0/3807928238 learned_addr learned my addr 172.21.15.62:0/3807928238 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.850+0000 7f6fe259c640 1 -- 172.21.15.62:0/3807928238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe419ddc0 con 0x7f6fe4107620 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.851+0000 7f6fe259c640 1 --2- 172.21.15.62:0/3807928238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe419d880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009940 tx=0x7f6fd40326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.851+0000 7f6fdb7fe640 1 -- 172.21.15.62:0/3807928238 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fd4042070 con 0x7f6fe4107620 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.851+0000 7f6fdb7fe640 1 -- 172.21.15.62:0/3807928238 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6fd4032be0 con 0x7f6fe4107620 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.851+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/3807928238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe419e050 con 0x7f6fe4107620 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.851+0000 7f6fdb7fe640 1 -- 172.21.15.62:0/3807928238 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fd4037040 con 0x7f6fe4107620 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.851+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/3807928238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe41a0d40 con 0x7f6fe4107620 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.851+0000 7f6fdb7fe640 1 -- 172.21.15.62:0/3807928238 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6fd4045070 con 0x7f6fe4107620 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.851+0000 7f6fdb7fe640 1 -- 172.21.15.62:0/3807928238 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6fd40479f0 con 0x7f6fe4107620 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.852+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/3807928238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe410bd50 con 0x7f6fe4107620 2024-09-11T23:42:29.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.854+0000 7f6fdb7fe640 1 -- 172.21.15.62:0/3807928238 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6fd4032d90 con 0x7f6fe4107620 2024-09-11T23:42:29.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.913+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/3807928238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6fe41a0f10 con 0x7f6fe4107620 2024-09-11T23:42:29.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.914+0000 7f6fdb7fe640 1 -- 172.21.15.62:0/3807928238 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f6fd403d430 con 0x7f6fe4107620 2024-09-11T23:42:29.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.915+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/3807928238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 msgr2=0x7f6fe419d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:29.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.915+0000 7f6fe8dbd640 1 --2- 172.21.15.62:0/3807928238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe419d880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009940 tx=0x7f6fd40326e0 comp rx=0 tx=0).stop 2024-09-11T23:42:29.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.915+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/3807928238 shutdown_connections 2024-09-11T23:42:29.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.915+0000 7f6fe8dbd640 1 --2- 172.21.15.62:0/3807928238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fe4107620 0x7f6fe419d880 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:29.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.915+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/3807928238 >> 172.21.15.62:0/3807928238 conn(0x7f6fe4102db0 msgr2=0x7f6fe4103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:29.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.915+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/3807928238 shutdown_connections 2024-09-11T23:42:29.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:28.915+0000 7f6fe8dbd640 1 -- 172.21.15.62:0/3807928238 wait complete. 2024-09-11T23:42:29.417 INFO:teuthology.orchestra.run.smithi062.stdout:Restarting the monitor... 2024-09-11T23:42:29.700 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:29 smithi062 systemd[1]: Stopping Ceph mon.a for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:42:29.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:29 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a[30586]: 2024-09-11T23:42:29.730+0000 7f4453049640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:42:29.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:29 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a[30586]: 2024-09-11T23:42:29.730+0000 7f4453049640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T23:42:29.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:29 smithi062 podman[30833]: 2024-09-11 23:42:29.844578408 +0000 UTC m=+0.267904136 container died b9f2805f864b23471047c4fb09db8182b8254cc00312f030263a11ebcace5a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T23:42:30.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 podman[30833]: 2024-09-11 23:42:30.123005836 +0000 UTC m=+0.546331554 container cleanup b9f2805f864b23471047c4fb09db8182b8254cc00312f030263a11ebcace5a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-11T23:42:30.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 bash[30833]: ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a 2024-09-11T23:42:30.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 podman[30845]: 2024-09-11 23:42:30.290636086 +0000 UTC m=+0.450044317 container remove b9f2805f864b23471047c4fb09db8182b8254cc00312f030263a11ebcace5a23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T23:42:30.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 systemd[1]: ceph-56675698-7097-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T23:42:30.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 systemd[1]: Stopped Ceph mon.a for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:42:30.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 systemd[1]: Starting Ceph mon.a for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:42:31.045 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 podman[30939]: 2024-09-11 23:42:30.656209956 +0000 UTC m=+0.016294058 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 podman[30939]: 2024-09-11 23:42:30.758423167 +0000 UTC m=+0.118507257 container create 577ea3d597ecdd3a9728a41ca756b69a136118806b5ce9b0e8a9d027840ef3e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 podman[30939]: 2024-09-11 23:42:30.952544136 +0000 UTC m=+0.312628225 container init 577ea3d597ecdd3a9728a41ca756b69a136118806b5ce9b0e8a9d027840ef3e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 podman[30939]: 2024-09-11 23:42:30.955694637 +0000 UTC m=+0.315778728 container start 577ea3d597ecdd3a9728a41ca756b69a136118806b5ce9b0e8a9d027840ef3e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: pidfile_write: ignore empty --pid-file 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: load: jerasure load: lrc 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: RocksDB version: 7.9.2 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Git sha 0 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: DB SUMMARY 2024-09-11T23:42:31.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: DB Session ID: HVFYD9HGW5I6FYBMC4ZI 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: CURRENT file: CURRENT 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105565 ; 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.error_if_exists: 0 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.create_if_missing: 0 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.paranoid_checks: 1 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.env: 0x560e9d59c940 2024-09-11T23:42:31.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.info_log: 0x560e9f2296e0 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.statistics: (nil) 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.use_fsync: 0 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_log_file_size: 0 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.allow_fallocate: 1 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T23:42:31.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.use_direct_reads: 0 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.db_log_dir: 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.wal_dir: 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T23:42:31.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.write_buffer_manager: 0x560e9f238dc0 2024-09-11T23:42:31.050 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T23:42:31.050 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T23:42:31.050 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T23:42:31.050 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T23:42:31.050 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T23:42:31.051 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T23:42:31.051 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T23:42:31.051 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T23:42:31.051 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.unordered_write: 0 2024-09-11T23:42:31.051 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.row_cache: None 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.wal_filter: None 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.two_write_queues: 0 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.wal_compression: 0 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.atomic_flush: 0 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T23:42:31.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.log_readahead_size: 0 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_background_jobs: 2 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_background_compactions: -1 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_subcompactions: 1 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T23:42:31.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_open_files: -1 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_background_flushes: -1 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Compression algorithms supported: 2024-09-11T23:42:31.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: kZSTD supported: 0 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: kXpressCompression supported: 0 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: kBZip2Compression supported: 0 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: kLZ4Compression supported: 1 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: kZlibCompression supported: 1 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: kSnappyCompression supported: 1 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.merge_operator: 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_filter: None 2024-09-11T23:42:31.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e9f229320) 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_type: 0 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_index_type: 0 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_shortening: 1 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout: checksum: 4 2024-09-11T23:42:31.056 INFO:journalctl@ceph.mon.a.smithi062.stdout: no_block_cache: 0 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache: 0x560e9f21d1f0 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_options: 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: capacity : 536870912 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_shard_bits : 4 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: strict_capacity_limit : 0 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_compressed: (nil) 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: persistent_cache: (nil) 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size: 4096 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size_deviation: 10 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_restart_interval: 16 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_block_restart_interval: 1 2024-09-11T23:42:31.057 INFO:journalctl@ceph.mon.a.smithi062.stdout: metadata_block_size: 4096 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: partition_filters: 0 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: use_delta_encoding: 1 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: filter_policy: bloomfilter 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: whole_key_filtering: 1 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: verify_compression: 0 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: format_version: 5 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_index_compression: 1 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_align: 0 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: max_auto_readahead_size: 262144 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: prepopulate_block_cache: 0 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: initial_auto_readahead_size: 8192 2024-09-11T23:42:31.058 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compression: NoCompression 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.num_levels: 7 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T23:42:31.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T23:42:31.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T23:42:31.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T23:42:31.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.inplace_update_support: 0 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T23:42:31.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.bloom_locality: 0 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.max_successive_merges: 0 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.ttl: 2592000 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T23:42:31.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.enable_blob_files: false 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.min_blob_size: 0 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d49ac17e-43f1-498b-bc73-96fcfd573a13 2024-09-11T23:42:31.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726098150982592, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726098150983693, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100484, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 98369, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14320, "raw_average_key_size": 49, "raw_value_size": 90616, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726098150, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d49ac17e-43f1-498b-bc73-96fcfd573a13", "db_session_id": "HVFYD9HGW5I6FYBMC4ZI", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726098150983792, "job": 1, "event": "recovery_finished"} 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560e9f24ae00 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: DB pointer 0x560e9f346000 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** DB Stats ** 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T23:42:31.066 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: L0 2/0 99.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: Sum 2/0 99.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T23:42:31.067 INFO:journalctl@ceph.mon.a.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative compaction: 0.00 GB write, 28.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval compaction: 0.00 GB write, 28.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache BinnedLRUCache@0x560e9f21d1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-11T23:42:31.068 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: starting mon.a rank 0 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: mon.a@-1(???) e1 preinit fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: mon.a@-1(???).mds e1 new map 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: mon.a@-1(???).mds e1 print_map 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout: e1 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout: btime 2024-09-11T23:42:20:574216+0000 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout: legacy client fscid: -1 2024-09-11T23:42:31.069 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout: No filesystems configured 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:30 smithi062 ceph-mon[30958]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: monmap epoch 1 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: last_changed 2024-09-11T23:42:14.107883+0000 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: min_mon_release 19 (squid) 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: election_strategy: 1 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:42:31.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: fsmap 2024-09-11T23:42:31.071 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T23:42:31.071 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 ceph-mon[30958]: mgrmap e1: no daemons active 2024-09-11T23:42:31.075 INFO:teuthology.orchestra.run.smithi062.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T23:42:31.302 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 bash[30939]: 577ea3d597ecdd3a9728a41ca756b69a136118806b5ce9b0e8a9d027840ef3e8 2024-09-11T23:42:31.302 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:31 smithi062 systemd[1]: Started Ceph mon.a for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:42:32.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.480+0000 7fb742fa1640 1 Processor -- start 2024-09-11T23:42:32.600 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.480+0000 7fb742fa1640 1 -- start start 2024-09-11T23:42:32.600 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.480+0000 7fb742fa1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c107870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:32.600 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.480+0000 7fb742fa1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c107e40 con 0x7fb73c107470 2024-09-11T23:42:32.601 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.481+0000 7fb741f9f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c107870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:32.601 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.481+0000 7fb741f9f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c107870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32836/0 (socket says 172.21.15.62:32836) 2024-09-11T23:42:32.601 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.481+0000 7fb741f9f640 1 -- 172.21.15.62:0/3460090319 learned_addr learned my addr 172.21.15.62:0/3460090319 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:32.601 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.481+0000 7fb741f9f640 1 -- 172.21.15.62:0/3460090319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c108620 con 0x7fb73c107470 2024-09-11T23:42:32.601 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.483+0000 7fb741f9f640 1 --2- 172.21.15.62:0/3460090319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c107870 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb734009820 tx=0x7fb73402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffbecb128f3af75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:32.601 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.483+0000 7fb740f9d640 1 -- 172.21.15.62:0/3460090319 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb734037070 con 0x7fb73c107470 2024-09-11T23:42:32.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.483+0000 7fb740f9d640 1 -- 172.21.15.62:0/3460090319 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb734032900 con 0x7fb73c107470 2024-09-11T23:42:32.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.483+0000 7fb740f9d640 1 -- 172.21.15.62:0/3460090319 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb734030850 con 0x7fb73c107470 2024-09-11T23:42:32.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.483+0000 7fb742fa1640 1 -- 172.21.15.62:0/3460090319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 msgr2=0x7fb73c107870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:32.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.483+0000 7fb742fa1640 1 --2- 172.21.15.62:0/3460090319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c107870 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb734009820 tx=0x7fb73402a270 comp rx=0 tx=0).stop 2024-09-11T23:42:32.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.484+0000 7fb742fa1640 1 -- 172.21.15.62:0/3460090319 shutdown_connections 2024-09-11T23:42:32.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.484+0000 7fb742fa1640 1 --2- 172.21.15.62:0/3460090319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c107870 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.484+0000 7fb742fa1640 1 -- 172.21.15.62:0/3460090319 >> 172.21.15.62:0/3460090319 conn(0x7fb73c102c00 msgr2=0x7fb73c105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.484+0000 7fb742fa1640 1 -- 172.21.15.62:0/3460090319 shutdown_connections 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.484+0000 7fb742fa1640 1 -- 172.21.15.62:0/3460090319 wait complete. 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.484+0000 7fb742fa1640 1 Processor -- start 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.484+0000 7fb742fa1640 1 -- start start 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.484+0000 7fb742fa1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.484+0000 7fb742fa1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c10b9d0 con 0x7fb73c107470 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.485+0000 7fb741f9f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.485+0000 7fb741f9f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32848/0 (socket says 172.21.15.62:32848) 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.485+0000 7fb741f9f640 1 -- 172.21.15.62:0/525374225 learned_addr learned my addr 172.21.15.62:0/525374225 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.485+0000 7fb741f9f640 1 -- 172.21.15.62:0/525374225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c19b980 con 0x7fb73c107470 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.485+0000 7fb741f9f640 1 --2- 172.21.15.62:0/525374225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c19b440 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb734009940 tx=0x7fb734032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:32.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.485+0000 7fb73affd640 1 -- 172.21.15.62:0/525374225 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb734042070 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.485+0000 7fb73affd640 1 -- 172.21.15.62:0/525374225 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb7340308c0 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.485+0000 7fb73affd640 1 -- 172.21.15.62:0/525374225 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb734037040 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.485+0000 7fb742fa1640 1 -- 172.21.15.62:0/525374225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73c19bc10 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.485+0000 7fb742fa1640 1 -- 172.21.15.62:0/525374225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73c19e900 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.486+0000 7fb73affd640 1 -- 172.21.15.62:0/525374225 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb734045070 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.486+0000 7fb73affd640 1 -- 172.21.15.62:0/525374225 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb734047bb0 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.486+0000 7fb742fa1640 1 -- 172.21.15.62:0/525374225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73c10bb50 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.489+0000 7fb73affd640 1 -- 172.21.15.62:0/525374225 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb7340327b0 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.548+0000 7fb742fa1640 1 -- 172.21.15.62:0/525374225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fb73c1078f0 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.550+0000 7fb73affd640 1 -- 172.21.15.62:0/525374225 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fb73403c760 con 0x7fb73c107470 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.550+0000 7fb742fa1640 1 -- 172.21.15.62:0/525374225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 msgr2=0x7fb73c19b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.550+0000 7fb742fa1640 1 --2- 172.21.15.62:0/525374225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c19b440 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb734009940 tx=0x7fb734032ce0 comp rx=0 tx=0).stop 2024-09-11T23:42:32.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.550+0000 7fb742fa1640 1 -- 172.21.15.62:0/525374225 shutdown_connections 2024-09-11T23:42:32.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.551+0000 7fb742fa1640 1 --2- 172.21.15.62:0/525374225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c107470 0x7fb73c19b440 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:32.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.551+0000 7fb742fa1640 1 -- 172.21.15.62:0/525374225 >> 172.21.15.62:0/525374225 conn(0x7fb73c102c00 msgr2=0x7fb73c1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:32.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.551+0000 7fb742fa1640 1 -- 172.21.15.62:0/525374225 shutdown_connections 2024-09-11T23:42:32.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:31.551+0000 7fb742fa1640 1 -- 172.21.15.62:0/525374225 wait complete. 2024-09-11T23:42:32.607 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T23:42:32.625 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:42:32.625 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mgr... 2024-09-11T23:42:32.625 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T23:42:32.626 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T23:42:32.893 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.x 2024-09-11T23:42:32.893 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-11T23:42:32.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:32 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/525374225' entity='client.admin' 2024-09-11T23:42:33.047 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-56675698-7097-11ef-bcea-c7b262605968.target.wants/ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-56675698-7097-11ef-bcea-c7b262605968@.service. 2024-09-11T23:42:33.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 systemd[1]: Starting Ceph mgr.x for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:42:33.572 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 podman[31169]: 2024-09-11 23:42:33.185935017 +0000 UTC m=+0.015762844 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:42:33.572 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 podman[31169]: 2024-09-11 23:42:33.295406047 +0000 UTC m=+0.125233873 container create bff0f724599e2f53ce30551a7483a7e67958d4e590fb8ff3cae309fb50e7407e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T23:42:33.572 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 podman[31169]: 2024-09-11 23:42:33.48971122 +0000 UTC m=+0.319539047 container init bff0f724599e2f53ce30551a7483a7e67958d4e590fb8ff3cae309fb50e7407e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T23:42:33.572 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 podman[31169]: 2024-09-11 23:42:33.492613619 +0000 UTC m=+0.322441446 container start bff0f724599e2f53ce30551a7483a7e67958d4e590fb8ff3cae309fb50e7407e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:42:33.572 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 bash[31169]: bff0f724599e2f53ce30551a7483a7e67958d4e590fb8ff3cae309fb50e7407e 2024-09-11T23:42:33.588 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:42:33.588 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-09-11T23:42:33.596 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:42:33.596 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-09-11T23:42:33.596 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-09-11T23:42:33.596 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T23:42:33.606 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:42:33.606 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-09-11T23:42:33.615 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:42:33.615 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-09-11T23:42:33.615 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-09-11T23:42:33.615 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T23:42:33.615 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr to start... 2024-09-11T23:42:33.615 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr... 2024-09-11T23:42:33.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 systemd[1]: Started Ceph mgr.x for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:42:33.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:33.626+0000 7fba8cb52040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:42:33.844 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:33.692+0000 7fba8cb52040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:42:34.109 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:33.853+0000 7fba8cb52040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:42:34.110 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:33 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:33.922+0000 7fba8cb52040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:42:34.110 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:34 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:34.081+0000 7fba8cb52040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:42:34.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:34 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2043688957' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:42:34.420 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:34 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:34.238+0000 7fba8cb52040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:42:34.695 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:34 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:34.574+0000 7fba8cb52040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:42:34.695 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:34 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:34.638+0000 7fba8cb52040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:42:34.698 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:42:34.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:42:34.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:42:34.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:42:34.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:42:34.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:42:34.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:34.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:42:20:574216+0000", 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:42:34.712 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:42:20.575005+0000", 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.040+0000 7f80c08da640 1 Processor -- start 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.040+0000 7f80c08da640 1 -- start start 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.040+0000 7f80c08da640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc10a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.040+0000 7f80c08da640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc07be30 con 0x7f80bc108080 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.041+0000 7f80ba59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc10a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.713 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.041+0000 7f80ba59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc10a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32868/0 (socket says 172.21.15.62:32868) 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.041+0000 7f80ba59c640 1 -- 172.21.15.62:0/3190430348 learned_addr learned my addr 172.21.15.62:0/3190430348 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.041+0000 7f80ba59c640 1 -- 172.21.15.62:0/3190430348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc10a9b0 con 0x7f80bc108080 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.042+0000 7f80ba59c640 1 --2- 172.21.15.62:0/3190430348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc10a470 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009820 tx=0x7f80ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a605fda1112be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.042+0000 7f80b9d9b640 1 -- 172.21.15.62:0/3190430348 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80ac037070 con 0x7f80bc108080 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.042+0000 7f80b9d9b640 1 -- 172.21.15.62:0/3190430348 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80ac032900 con 0x7f80bc108080 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.042+0000 7f80b9d9b640 1 -- 172.21.15.62:0/3190430348 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80ac030850 con 0x7f80bc108080 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.043+0000 7f80c08da640 1 -- 172.21.15.62:0/3190430348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 msgr2=0x7f80bc10a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.043+0000 7f80c08da640 1 --2- 172.21.15.62:0/3190430348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc10a470 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009820 tx=0x7f80ac02a270 comp rx=0 tx=0).stop 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.043+0000 7f80c08da640 1 -- 172.21.15.62:0/3190430348 shutdown_connections 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.043+0000 7f80c08da640 1 --2- 172.21.15.62:0/3190430348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc10a470 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.043+0000 7f80c08da640 1 -- 172.21.15.62:0/3190430348 >> 172.21.15.62:0/3190430348 conn(0x7f80bc101ca0 msgr2=0x7f80bc1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.043+0000 7f80c08da640 1 -- 172.21.15.62:0/3190430348 shutdown_connections 2024-09-11T23:42:34.714 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.043+0000 7f80c08da640 1 -- 172.21.15.62:0/3190430348 wait complete. 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.043+0000 7f80c08da640 1 Processor -- start 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.044+0000 7f80c08da640 1 -- start start 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.044+0000 7f80c08da640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc107900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.044+0000 7f80c08da640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc10b9f0 con 0x7f80bc108080 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.044+0000 7f80ba59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.044+0000 7f80ba59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32874/0 (socket says 172.21.15.62:32874) 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.044+0000 7f80ba59c640 1 -- 172.21.15.62:0/2043688957 learned_addr learned my addr 172.21.15.62:0/2043688957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.044+0000 7f80ba59c640 1 -- 172.21.15.62:0/2043688957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc107e40 con 0x7f80bc108080 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.044+0000 7f80ba59c640 1 --2- 172.21.15.62:0/2043688957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc107900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80ac031000 tx=0x7f80ac0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.045+0000 7f80b37fe640 1 -- 172.21.15.62:0/2043688957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80ac042070 con 0x7f80bc108080 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.045+0000 7f80b37fe640 1 -- 172.21.15.62:0/2043688957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80ac032be0 con 0x7f80bc108080 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.045+0000 7f80c08da640 1 -- 172.21.15.62:0/2043688957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc106030 con 0x7f80bc108080 2024-09-11T23:42:34.715 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.045+0000 7f80b37fe640 1 -- 172.21.15.62:0/2043688957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80ac037040 con 0x7f80bc108080 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.045+0000 7f80c08da640 1 -- 172.21.15.62:0/2043688957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc106570 con 0x7f80bc108080 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.045+0000 7f80b37fe640 1 -- 172.21.15.62:0/2043688957 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80ac045070 con 0x7f80bc108080 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.045+0000 7f80b37fe640 1 -- 172.21.15.62:0/2043688957 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80ac047ad0 con 0x7f80bc108080 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.046+0000 7f80c08da640 1 -- 172.21.15.62:0/2043688957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc1084e0 con 0x7f80bc108080 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.048+0000 7f80b37fe640 1 -- 172.21.15.62:0/2043688957 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f80ac04c030 con 0x7f80bc108080 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.110+0000 7f80c08da640 1 -- 172.21.15.62:0/2043688957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f80bc106c30 con 0x7f80bc108080 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.111+0000 7f80b37fe640 1 -- 172.21.15.62:0/2043688957 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f80ac032d90 con 0x7f80bc108080 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.112+0000 7f80c08da640 1 -- 172.21.15.62:0/2043688957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 msgr2=0x7f80bc107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.112+0000 7f80c08da640 1 --2- 172.21.15.62:0/2043688957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc107900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80ac031000 tx=0x7f80ac0326e0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.112+0000 7f80c08da640 1 -- 172.21.15.62:0/2043688957 shutdown_connections 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.112+0000 7f80c08da640 1 --2- 172.21.15.62:0/2043688957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80bc108080 0x7f80bc107900 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.112+0000 7f80c08da640 1 -- 172.21.15.62:0/2043688957 >> 172.21.15.62:0/2043688957 conn(0x7f80bc101ca0 msgr2=0x7f80bc102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.112+0000 7f80c08da640 1 -- 172.21.15.62:0/2043688957 shutdown_connections 2024-09-11T23:42:34.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:34.112+0000 7f80c08da640 1 -- 172.21.15.62:0/2043688957 wait complete. 2024-09-11T23:42:34.717 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (1/15)... 2024-09-11T23:42:34.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:34 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:34.833+0000 7fba8cb52040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:42:36.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:35 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:35.761+0000 7fba8cb52040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:42:36.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:35 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:35.831+0000 7fba8cb52040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:42:36.864 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:36 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:36.591+0000 7fba8cb52040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:42:36.865 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:36 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:36.712+0000 7fba8cb52040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:42:37.120 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:36 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:36.865+0000 7fba8cb52040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:42:37.120 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:36 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:36.933+0000 7fba8cb52040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:42:37.120 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:37.092+0000 7fba8cb52040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:42:37.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:37 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1901732066' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:42:37.402 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:37.166+0000 7fba8cb52040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:42:37.402 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:37.236+0000 7fba8cb52040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:42:37.402 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:37.402+0000 7fba8cb52040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:42:37.695 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:37.475+0000 7fba8cb52040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:42:37.695 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:37.545+0000 7fba8cb52040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:42:37.695 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:37.633+0000 7fba8cb52040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:42:37.695 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:37.696+0000 7fba8cb52040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:42:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:42:37.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:42:37.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:42:37.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:42:37.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:42:20:574216+0000", 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:42:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:42:20.575005+0000", 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.121+0000 7f0eafbde640 1 Processor -- start 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.121+0000 7f0eafbde640 1 -- start start 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.121+0000 7f0eafbde640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea810a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.121+0000 7f0eafbde640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea807be30 con 0x7f0ea8108080 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.121+0000 7f0ead97a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea810a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.121+0000 7f0ead97a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea810a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32884/0 (socket says 172.21.15.62:32884) 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.121+0000 7f0ead97a640 1 -- 172.21.15.62:0/3481045824 learned_addr learned my addr 172.21.15.62:0/3481045824 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.122+0000 7f0ead97a640 1 -- 172.21.15.62:0/3481045824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea810a9b0 con 0x7f0ea8108080 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.122+0000 7f0ead97a640 1 --2- 172.21.15.62:0/3481045824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea810a470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009820 tx=0x7f0ea402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94114e7f282e1122 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.122+0000 7f0eac978640 1 -- 172.21.15.62:0/3481045824 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ea4037070 con 0x7f0ea8108080 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.123+0000 7f0eac978640 1 -- 172.21.15.62:0/3481045824 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ea4032900 con 0x7f0ea8108080 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.123+0000 7f0eac978640 1 -- 172.21.15.62:0/3481045824 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ea4030850 con 0x7f0ea8108080 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.123+0000 7f0eafbde640 1 -- 172.21.15.62:0/3481045824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 msgr2=0x7f0ea810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.123+0000 7f0eafbde640 1 --2- 172.21.15.62:0/3481045824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea810a470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009820 tx=0x7f0ea402a270 comp rx=0 tx=0).stop 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.123+0000 7f0eafbde640 1 -- 172.21.15.62:0/3481045824 shutdown_connections 2024-09-11T23:42:37.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.123+0000 7f0eafbde640 1 --2- 172.21.15.62:0/3481045824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea810a470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.123+0000 7f0eafbde640 1 -- 172.21.15.62:0/3481045824 >> 172.21.15.62:0/3481045824 conn(0x7f0ea8101ca0 msgr2=0x7f0ea81040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.123+0000 7f0eafbde640 1 -- 172.21.15.62:0/3481045824 shutdown_connections 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.123+0000 7f0eafbde640 1 -- 172.21.15.62:0/3481045824 wait complete. 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.124+0000 7f0eafbde640 1 Processor -- start 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.124+0000 7f0eafbde640 1 -- start start 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.124+0000 7f0eafbde640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.124+0000 7f0eafbde640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea810b9f0 con 0x7f0ea8108080 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.124+0000 7f0ead97a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.124+0000 7f0ead97a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32900/0 (socket says 172.21.15.62:32900) 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.124+0000 7f0ead97a640 1 -- 172.21.15.62:0/1901732066 learned_addr learned my addr 172.21.15.62:0/1901732066 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.125+0000 7f0ead97a640 1 -- 172.21.15.62:0/1901732066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea819bb70 con 0x7f0ea8108080 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.125+0000 7f0ead97a640 1 --2- 172.21.15.62:0/1901732066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea819b630 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40097f0 tx=0x7f0ea4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:37.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.125+0000 7f0e9effd640 1 -- 172.21.15.62:0/1901732066 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ea4042070 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.125+0000 7f0e9effd640 1 -- 172.21.15.62:0/1901732066 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ea4032ca0 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.125+0000 7f0eafbde640 1 -- 172.21.15.62:0/1901732066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea819be00 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.125+0000 7f0e9effd640 1 -- 172.21.15.62:0/1901732066 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ea4037040 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.125+0000 7f0eafbde640 1 -- 172.21.15.62:0/1901732066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea819eaf0 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.125+0000 7f0e9effd640 1 -- 172.21.15.62:0/1901732066 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ea4045070 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.125+0000 7f0e9effd640 1 -- 172.21.15.62:0/1901732066 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ea4047ad0 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.126+0000 7f0eafbde640 1 -- 172.21.15.62:0/1901732066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea81084e0 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.129+0000 7f0e9effd640 1 -- 172.21.15.62:0/1901732066 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0ea403db10 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.192+0000 7f0eafbde640 1 -- 172.21.15.62:0/1901732066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0ea819eff0 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.192+0000 7f0e9effd640 1 -- 172.21.15.62:0/1901732066 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f0ea403d430 con 0x7f0ea8108080 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.193+0000 7f0eafbde640 1 -- 172.21.15.62:0/1901732066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 msgr2=0x7f0ea819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.193+0000 7f0eafbde640 1 --2- 172.21.15.62:0/1901732066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea819b630 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40097f0 tx=0x7f0ea4030ec0 comp rx=0 tx=0).stop 2024-09-11T23:42:37.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.193+0000 7f0eafbde640 1 -- 172.21.15.62:0/1901732066 shutdown_connections 2024-09-11T23:42:37.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.193+0000 7f0eafbde640 1 --2- 172.21.15.62:0/1901732066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ea8108080 0x7f0ea819b630 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:37.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.193+0000 7f0eafbde640 1 -- 172.21.15.62:0/1901732066 >> 172.21.15.62:0/1901732066 conn(0x7f0ea8101ca0 msgr2=0x7f0ea8102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:37.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.193+0000 7f0eafbde640 1 -- 172.21.15.62:0/1901732066 shutdown_connections 2024-09-11T23:42:37.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:37.193+0000 7f0eafbde640 1 -- 172.21.15.62:0/1901732066 wait complete. 2024-09-11T23:42:37.873 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (2/15)... 2024-09-11T23:42:37.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:37.759+0000 7fba8cb52040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:42:38.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:38.590+0000 7fba8cb52040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:42:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: Activating manager daemon x 2024-09-11T23:42:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: mgrmap e2: x(active, starting, since 0.00311995s) 2024-09-11T23:42:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: from='mgr.14100 172.21.15.62:0/2005117982' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:42:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: from='mgr.14100 172.21.15.62:0/2005117982' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:42:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: from='mgr.14100 172.21.15.62:0/2005117982' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:42:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: from='mgr.14100 172.21.15.62:0/2005117982' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:42:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: from='mgr.14100 172.21.15.62:0/2005117982' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:42:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: Manager daemon x is now available 2024-09-11T23:42:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: from='mgr.14100 172.21.15.62:0/2005117982' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:42:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: from='mgr.14100 172.21.15.62:0/2005117982' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:42:38.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: from='mgr.14100 172.21.15.62:0/2005117982' entity='mgr.x' 2024-09-11T23:42:38.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: from='mgr.14100 172.21.15.62:0/2005117982' entity='mgr.x' 2024-09-11T23:42:38.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:38 smithi062 ceph-mon[30958]: from='mgr.14100 172.21.15.62:0/2005117982' entity='mgr.x' 2024-09-11T23:42:40.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:40 smithi062 ceph-mon[30958]: mgrmap e3: x(active, since 1.00723s) 2024-09-11T23:42:40.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:40 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1768887084' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:42:41.079 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:42:41.080 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:42:41.081 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:42:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:42:20:574216+0000", 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:42:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:42:20.575005+0000", 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.284+0000 7f5781dec640 1 Processor -- start 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.284+0000 7f5781dec640 1 -- start start 2024-09-11T23:42:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.284+0000 7f5781dec640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.284+0000 7f5781dec640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f577c07c7f0 con 0x7f577c07de00 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.284+0000 7f577b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.284+0000 7f577b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32966/0 (socket says 172.21.15.62:32966) 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.284+0000 7f577b7fe640 1 -- 172.21.15.62:0/3156974296 learned_addr learned my addr 172.21.15.62:0/3156974296 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.285+0000 7f577b7fe640 1 -- 172.21.15.62:0/3156974296 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c07c970 con 0x7f577c07de00 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.285+0000 7f577b7fe640 1 --2- 172.21.15.62:0/3156974296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c07c2b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f576c009820 tx=0x7f576c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23504c214c2b8c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.285+0000 7f577a7fc640 1 -- 172.21.15.62:0/3156974296 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f576c037070 con 0x7f577c07de00 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.286+0000 7f577a7fc640 1 -- 172.21.15.62:0/3156974296 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f576c032900 con 0x7f577c07de00 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.286+0000 7f577a7fc640 1 -- 172.21.15.62:0/3156974296 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f576c030850 con 0x7f577c07de00 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.286+0000 7f5781dec640 1 -- 172.21.15.62:0/3156974296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 msgr2=0x7f577c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.286+0000 7f5781dec640 1 --2- 172.21.15.62:0/3156974296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c07c2b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f576c009820 tx=0x7f576c02a270 comp rx=0 tx=0).stop 2024-09-11T23:42:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.286+0000 7f5781dec640 1 -- 172.21.15.62:0/3156974296 shutdown_connections 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.286+0000 7f5781dec640 1 --2- 172.21.15.62:0/3156974296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c07c2b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.286+0000 7f5781dec640 1 -- 172.21.15.62:0/3156974296 >> 172.21.15.62:0/3156974296 conn(0x7f577c102da0 msgr2=0x7f577c105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.286+0000 7f5781dec640 1 -- 172.21.15.62:0/3156974296 shutdown_connections 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.286+0000 7f5781dec640 1 -- 172.21.15.62:0/3156974296 wait complete. 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.287+0000 7f5781dec640 1 Processor -- start 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.287+0000 7f5781dec640 1 -- start start 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.287+0000 7f5781dec640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.287+0000 7f5781dec640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f577c10dc60 con 0x7f577c07de00 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.287+0000 7f577b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.287+0000 7f577b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c19d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32978/0 (socket says 172.21.15.62:32978) 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.287+0000 7f577b7fe640 1 -- 172.21.15.62:0/1768887084 learned_addr learned my addr 172.21.15.62:0/1768887084 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.288+0000 7f577b7fe640 1 -- 172.21.15.62:0/1768887084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c19dd60 con 0x7f577c07de00 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.288+0000 7f577b7fe640 1 --2- 172.21.15.62:0/1768887084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c19d820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f576c0097f0 tx=0x7f576c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:41.086 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.288+0000 7f5778ff9640 1 -- 172.21.15.62:0/1768887084 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f576c042070 con 0x7f577c07de00 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.288+0000 7f5781dec640 1 -- 172.21.15.62:0/1768887084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c19dff0 con 0x7f577c07de00 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.288+0000 7f5778ff9640 1 -- 172.21.15.62:0/1768887084 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f576c032ca0 con 0x7f577c07de00 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.288+0000 7f5778ff9640 1 -- 172.21.15.62:0/1768887084 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f576c037040 con 0x7f577c07de00 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.288+0000 7f5781dec640 1 -- 172.21.15.62:0/1768887084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c1a0ce0 con 0x7f577c07de00 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.289+0000 7f5778ff9640 1 -- 172.21.15.62:0/1768887084 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f576c045070 con 0x7f577c07de00 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.289+0000 7f5781dec640 1 -- 172.21.15.62:0/1768887084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577c07c330 con 0x7f577c07de00 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.289+0000 7f5778ff9640 1 --2- 172.21.15.62:0/1768887084 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f5764038e10 0x7f576403b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.289+0000 7f5778ff9640 1 -- 172.21.15.62:0/1768887084 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f576c06bec0 con 0x7f577c07de00 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.289+0000 7f577affd640 1 --2- 172.21.15.62:0/1768887084 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f5764038e10 0x7f576403b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.290+0000 7f577affd640 1 --2- 172.21.15.62:0/1768887084 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f5764038e10 0x7f576403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f57700098c0 tx=0x7f5770006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.294+0000 7f5778ff9640 1 -- 172.21.15.62:0/1768887084 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f576c03d8f0 con 0x7f577c07de00 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.537+0000 7f5781dec640 1 -- 172.21.15.62:0/1768887084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f577c1a1360 con 0x7f577c07de00 2024-09-11T23:42:41.087 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.538+0000 7f5778ff9640 1 -- 172.21.15.62:0/1768887084 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f576c045520 con 0x7f577c07de00 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.540+0000 7f5781dec640 1 -- 172.21.15.62:0/1768887084 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f5764038e10 msgr2=0x7f576403b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.540+0000 7f5781dec640 1 --2- 172.21.15.62:0/1768887084 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f5764038e10 0x7f576403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f57700098c0 tx=0x7f5770006db0 comp rx=0 tx=0).stop 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.540+0000 7f5781dec640 1 -- 172.21.15.62:0/1768887084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 msgr2=0x7f577c19d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.540+0000 7f5781dec640 1 --2- 172.21.15.62:0/1768887084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c19d820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f576c0097f0 tx=0x7f576c030ec0 comp rx=0 tx=0).stop 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.540+0000 7f5781dec640 1 -- 172.21.15.62:0/1768887084 shutdown_connections 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.540+0000 7f5781dec640 1 --2- 172.21.15.62:0/1768887084 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f5764038e10 0x7f576403b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.540+0000 7f5781dec640 1 --2- 172.21.15.62:0/1768887084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577c07de00 0x7f577c19d820 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.540+0000 7f5781dec640 1 -- 172.21.15.62:0/1768887084 >> 172.21.15.62:0/1768887084 conn(0x7f577c102da0 msgr2=0x7f577c1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.540+0000 7f5781dec640 1 -- 172.21.15.62:0/1768887084 shutdown_connections 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:40.540+0000 7f5781dec640 1 -- 172.21.15.62:0/1768887084 wait complete. 2024-09-11T23:42:41.088 INFO:teuthology.orchestra.run.smithi062.stdout:mgr is available 2024-09-11T23:42:41.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:41 smithi062 ceph-mon[30958]: mgrmap e4: x(active, since 2s) 2024-09-11T23:42:42.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:42.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T23:42:42.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:42.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-09-11T23:42:42.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T23:42:42.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.442+0000 7fe42b5b7640 1 Processor -- start 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.442+0000 7fe42b5b7640 1 -- start start 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.442+0000 7fe42b5b7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42410a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.442+0000 7fe42b5b7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe42407be40 con 0x7fe4241080c0 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.443+0000 7fe429353640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42410a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.443+0000 7fe429353640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42410a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42244/0 (socket says 172.21.15.62:42244) 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.443+0000 7fe429353640 1 -- 172.21.15.62:0/272732154 learned_addr learned my addr 172.21.15.62:0/272732154 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.443+0000 7fe429353640 1 -- 172.21.15.62:0/272732154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42410a9f0 con 0x7fe4241080c0 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.444+0000 7fe429353640 1 --2- 172.21.15.62:0/272732154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42410a4b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe420009820 tx=0x7fe42002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db56cd78d257592 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.444+0000 7fe41bfff640 1 -- 172.21.15.62:0/272732154 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420037070 con 0x7fe4241080c0 2024-09-11T23:42:42.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.444+0000 7fe41bfff640 1 -- 172.21.15.62:0/272732154 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe420032900 con 0x7fe4241080c0 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.444+0000 7fe41bfff640 1 -- 172.21.15.62:0/272732154 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420030850 con 0x7fe4241080c0 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.444+0000 7fe42b5b7640 1 -- 172.21.15.62:0/272732154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 msgr2=0x7fe42410a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.444+0000 7fe42b5b7640 1 --2- 172.21.15.62:0/272732154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42410a4b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe420009820 tx=0x7fe42002a270 comp rx=0 tx=0).stop 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.444+0000 7fe42b5b7640 1 -- 172.21.15.62:0/272732154 shutdown_connections 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.444+0000 7fe42b5b7640 1 --2- 172.21.15.62:0/272732154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42410a4b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.444+0000 7fe42b5b7640 1 -- 172.21.15.62:0/272732154 >> 172.21.15.62:0/272732154 conn(0x7fe424101c80 msgr2=0x7fe4241040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.445+0000 7fe42b5b7640 1 -- 172.21.15.62:0/272732154 shutdown_connections 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.445+0000 7fe42b5b7640 1 -- 172.21.15.62:0/272732154 wait complete. 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.445+0000 7fe42b5b7640 1 Processor -- start 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.445+0000 7fe42b5b7640 1 -- start start 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.445+0000 7fe42b5b7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.445+0000 7fe42b5b7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe42410ba30 con 0x7fe4241080c0 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.445+0000 7fe429353640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.445+0000 7fe429353640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42419b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42250/0 (socket says 172.21.15.62:42250) 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.445+0000 7fe429353640 1 -- 172.21.15.62:0/1256073443 learned_addr learned my addr 172.21.15.62:0/1256073443 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.446+0000 7fe429353640 1 -- 172.21.15.62:0/1256073443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42419bb70 con 0x7fe4241080c0 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.446+0000 7fe429353640 1 --2- 172.21.15.62:0/1256073443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42419b630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe420031000 tx=0x7fe4200326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.446+0000 7fe41a7fc640 1 -- 172.21.15.62:0/1256073443 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420042070 con 0x7fe4241080c0 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.446+0000 7fe42b5b7640 1 -- 172.21.15.62:0/1256073443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42419be00 con 0x7fe4241080c0 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.446+0000 7fe41a7fc640 1 -- 172.21.15.62:0/1256073443 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe420032be0 con 0x7fe4241080c0 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.446+0000 7fe41a7fc640 1 -- 172.21.15.62:0/1256073443 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420037040 con 0x7fe4241080c0 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.446+0000 7fe42b5b7640 1 -- 172.21.15.62:0/1256073443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42419eaf0 con 0x7fe4241080c0 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.447+0000 7fe41a7fc640 1 -- 172.21.15.62:0/1256073443 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fe420045070 con 0x7fe4241080c0 2024-09-11T23:42:42.245 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.447+0000 7fe41a7fc640 1 --2- 172.21.15.62:0/1256073443 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fe410038f20 0x7fe41003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.447+0000 7fe41a7fc640 1 -- 172.21.15.62:0/1256073443 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe42006bf50 con 0x7fe4241080c0 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.447+0000 7fe42b5b7640 1 -- 172.21.15.62:0/1256073443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe424108520 con 0x7fe4241080c0 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.447+0000 7fe428b52640 1 --2- 172.21.15.62:0/1256073443 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fe410038f20 0x7fe41003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.448+0000 7fe428b52640 1 --2- 172.21.15.62:0/1256073443 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fe410038f20 0x7fe41003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe4140098c0 tx=0x7fe414006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.452+0000 7fe41a7fc640 1 -- 172.21.15.62:0/1256073443 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe420042210 con 0x7fe4241080c0 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.628+0000 7fe42b5b7640 1 -- 172.21.15.62:0/1256073443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe4241a0260 con 0x7fe4241080c0 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.629+0000 7fe41a7fc640 1 -- 172.21.15.62:0/1256073443 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7fe420032d90 con 0x7fe4241080c0 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.631+0000 7fe42b5b7640 1 -- 172.21.15.62:0/1256073443 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fe410038f20 msgr2=0x7fe41003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.631+0000 7fe42b5b7640 1 --2- 172.21.15.62:0/1256073443 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fe410038f20 0x7fe41003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe4140098c0 tx=0x7fe414006db0 comp rx=0 tx=0).stop 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.631+0000 7fe42b5b7640 1 -- 172.21.15.62:0/1256073443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 msgr2=0x7fe42419b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.631+0000 7fe42b5b7640 1 --2- 172.21.15.62:0/1256073443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42419b630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe420031000 tx=0x7fe4200326e0 comp rx=0 tx=0).stop 2024-09-11T23:42:42.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.631+0000 7fe42b5b7640 1 -- 172.21.15.62:0/1256073443 shutdown_connections 2024-09-11T23:42:42.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.631+0000 7fe42b5b7640 1 --2- 172.21.15.62:0/1256073443 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fe410038f20 0x7fe41003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:42.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.631+0000 7fe42b5b7640 1 --2- 172.21.15.62:0/1256073443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4241080c0 0x7fe42419b630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:42.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.631+0000 7fe42b5b7640 1 -- 172.21.15.62:0/1256073443 >> 172.21.15.62:0/1256073443 conn(0x7fe424101c80 msgr2=0x7fe424102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:42.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.631+0000 7fe42b5b7640 1 -- 172.21.15.62:0/1256073443 shutdown_connections 2024-09-11T23:42:42.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:41.631+0000 7fe42b5b7640 1 -- 172.21.15.62:0/1256073443 wait complete. 2024-09-11T23:42:42.247 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling cephadm module... 2024-09-11T23:42:42.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:42 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1256073443' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T23:42:43.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:43 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/925899247' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T23:42:43.864 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: ignoring --setuser ceph since I am not root 2024-09-11T23:42:43.864 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: ignoring --setgroup ceph since I am not root 2024-09-11T23:42:43.864 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:43.753+0000 7f1dfe8db040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:42:43.864 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:43.813+0000 7f1dfe8db040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:42:44.181 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:43.968+0000 7f1dfe8db040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:42:44.181 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:44.038+0000 7f1dfe8db040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:42:44.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.628+0000 7f51ce240640 1 Processor -- start 2024-09-11T23:42:44.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.628+0000 7f51ce240640 1 -- start start 2024-09-11T23:42:44.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.629+0000 7f51ce240640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:44.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.629+0000 7f51ce240640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51c807c880 con 0x7f51c807de50 2024-09-11T23:42:44.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.629+0000 7f51c77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:44.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.629+0000 7f51c77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42262/0 (socket says 172.21.15.62:42262) 2024-09-11T23:42:44.188 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.629+0000 7f51c77fe640 1 -- 172.21.15.62:0/1966909049 learned_addr learned my addr 172.21.15.62:0/1966909049 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.629+0000 7f51c77fe640 1 -- 172.21.15.62:0/1966909049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c807ca00 con 0x7f51c807de50 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.630+0000 7f51c77fe640 1 --2- 172.21.15.62:0/1966909049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c807c2b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009820 tx=0x7f51b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133d5f66778f5c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.630+0000 7f51c67fc640 1 -- 172.21.15.62:0/1966909049 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51b8037070 con 0x7f51c807de50 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.630+0000 7f51c67fc640 1 -- 172.21.15.62:0/1966909049 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f51b8032900 con 0x7f51c807de50 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.630+0000 7f51c67fc640 1 -- 172.21.15.62:0/1966909049 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51b8030850 con 0x7f51c807de50 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.630+0000 7f51ce240640 1 -- 172.21.15.62:0/1966909049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 msgr2=0x7f51c807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.630+0000 7f51ce240640 1 --2- 172.21.15.62:0/1966909049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c807c2b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009820 tx=0x7f51b802a270 comp rx=0 tx=0).stop 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.631+0000 7f51ce240640 1 -- 172.21.15.62:0/1966909049 shutdown_connections 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.631+0000 7f51ce240640 1 --2- 172.21.15.62:0/1966909049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c807c2b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.631+0000 7f51ce240640 1 -- 172.21.15.62:0/1966909049 >> 172.21.15.62:0/1966909049 conn(0x7f51c8102db0 msgr2=0x7f51c81051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.631+0000 7f51ce240640 1 -- 172.21.15.62:0/1966909049 shutdown_connections 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.631+0000 7f51ce240640 1 -- 172.21.15.62:0/1966909049 wait complete. 2024-09-11T23:42:44.189 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.631+0000 7f51ce240640 1 Processor -- start 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.631+0000 7f51ce240640 1 -- start start 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51ce240640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51ce240640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51c810dc50 con 0x7f51c807de50 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51c77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51c77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c819d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42264/0 (socket says 172.21.15.62:42264) 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51c77fe640 1 -- 172.21.15.62:0/925899247 learned_addr learned my addr 172.21.15.62:0/925899247 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51c77fe640 1 -- 172.21.15.62:0/925899247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c819dda0 con 0x7f51c807de50 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51c77fe640 1 --2- 172.21.15.62:0/925899247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c819d860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f51b8031000 tx=0x7f51b80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51c4ff9640 1 -- 172.21.15.62:0/925899247 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51b8042070 con 0x7f51c807de50 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51c4ff9640 1 -- 172.21.15.62:0/925899247 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f51b8032be0 con 0x7f51c807de50 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51c4ff9640 1 -- 172.21.15.62:0/925899247 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51b8037040 con 0x7f51c807de50 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51ce240640 1 -- 172.21.15.62:0/925899247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c819e030 con 0x7f51c807de50 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.632+0000 7f51ce240640 1 -- 172.21.15.62:0/925899247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c81a0d20 con 0x7f51c807de50 2024-09-11T23:42:44.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.633+0000 7f51c4ff9640 1 -- 172.21.15.62:0/925899247 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f51b8045070 con 0x7f51c807de50 2024-09-11T23:42:44.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.633+0000 7f51ce240640 1 -- 172.21.15.62:0/925899247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c807c330 con 0x7f51c807de50 2024-09-11T23:42:44.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.633+0000 7f51c4ff9640 1 --2- 172.21.15.62:0/925899247 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f51b4038f70 0x7f51b403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:44.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.634+0000 7f51c4ff9640 1 -- 172.21.15.62:0/925899247 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f51b806bee0 con 0x7f51c807de50 2024-09-11T23:42:44.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.634+0000 7f51c6ffd640 1 --2- 172.21.15.62:0/925899247 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f51b4038f70 0x7f51b403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:44.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.634+0000 7f51c6ffd640 1 --2- 172.21.15.62:0/925899247 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f51b4038f70 0x7f51b403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f51b00098c0 tx=0x7f51b0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:44.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.638+0000 7f51c4ff9640 1 -- 172.21.15.62:0/925899247 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51b8032d90 con 0x7f51c807de50 2024-09-11T23:42:44.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:42.860+0000 7f51ce240640 1 -- 172.21.15.62:0/925899247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f51c81a0ef0 con 0x7f51c807de50 2024-09-11T23:42:44.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.603+0000 7f51c4ff9640 1 -- 172.21.15.62:0/925899247 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f51b806b900 con 0x7f51c807de50 2024-09-11T23:42:44.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.605+0000 7f51c4ff9640 1 -- 172.21.15.62:0/925899247 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f51b8039d60 con 0x7f51c807de50 2024-09-11T23:42:44.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.606+0000 7f51ce240640 1 -- 172.21.15.62:0/925899247 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f51b4038f70 msgr2=0x7f51b403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:44.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.606+0000 7f51ce240640 1 --2- 172.21.15.62:0/925899247 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f51b4038f70 0x7f51b403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f51b00098c0 tx=0x7f51b0006db0 comp rx=0 tx=0).stop 2024-09-11T23:42:44.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.606+0000 7f51ce240640 1 -- 172.21.15.62:0/925899247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 msgr2=0x7f51c819d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:44.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.606+0000 7f51ce240640 1 --2- 172.21.15.62:0/925899247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c819d860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f51b8031000 tx=0x7f51b80326e0 comp rx=0 tx=0).stop 2024-09-11T23:42:44.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.607+0000 7f51ce240640 1 -- 172.21.15.62:0/925899247 shutdown_connections 2024-09-11T23:42:44.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.607+0000 7f51ce240640 1 --2- 172.21.15.62:0/925899247 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f51b4038f70 0x7f51b403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:44.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.607+0000 7f51ce240640 1 --2- 172.21.15.62:0/925899247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c807de50 0x7f51c819d860 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:44.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.607+0000 7f51ce240640 1 -- 172.21.15.62:0/925899247 >> 172.21.15.62:0/925899247 conn(0x7f51c8102db0 msgr2=0x7f51c81039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:44.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.607+0000 7f51ce240640 1 -- 172.21.15.62:0/925899247 shutdown_connections 2024-09-11T23:42:44.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:43.607+0000 7f51ce240640 1 -- 172.21.15.62:0/925899247 wait complete. 2024-09-11T23:42:44.452 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:44.193+0000 7f1dfe8db040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:42:44.452 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:44.346+0000 7f1dfe8db040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:42:44.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:44 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/925899247' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T23:42:44.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:44 smithi062 ceph-mon[30958]: mgrmap e5: x(active, since 5s) 2024-09-11T23:42:44.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:44.679+0000 7f1dfe8db040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:42:45.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:44.744+0000 7f1dfe8db040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:42:45.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:44.941+0000 7f1dfe8db040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:42:45.461 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:42:45.461 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T23:42:45.461 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:42:45.461 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.606+0000 7fee687ee640 1 Processor -- start 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.606+0000 7fee687ee640 1 -- start start 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.606+0000 7fee687ee640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee60106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.606+0000 7fee687ee640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee60106ee0 con 0x7fee60106510 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.607+0000 7fee6658a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee60106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.607+0000 7fee6658a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee60106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42294/0 (socket says 172.21.15.62:42294) 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.607+0000 7fee6658a640 1 -- 172.21.15.62:0/3210745002 learned_addr learned my addr 172.21.15.62:0/3210745002 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.607+0000 7fee6658a640 1 -- 172.21.15.62:0/3210745002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee601076c0 con 0x7fee60106510 2024-09-11T23:42:45.462 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.608+0000 7fee6658a640 1 --2- 172.21.15.62:0/3210745002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee60106910 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009820 tx=0x7fee5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a287b1c46eb6fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.608+0000 7fee65588640 1 -- 172.21.15.62:0/3210745002 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee5c037070 con 0x7fee60106510 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.608+0000 7fee65588640 1 -- 172.21.15.62:0/3210745002 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fee5c032900 con 0x7fee60106510 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.608+0000 7fee65588640 1 -- 172.21.15.62:0/3210745002 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee5c030850 con 0x7fee60106510 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.608+0000 7fee687ee640 1 -- 172.21.15.62:0/3210745002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 msgr2=0x7fee60106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.608+0000 7fee687ee640 1 --2- 172.21.15.62:0/3210745002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee60106910 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009820 tx=0x7fee5c02a270 comp rx=0 tx=0).stop 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.609+0000 7fee687ee640 1 -- 172.21.15.62:0/3210745002 shutdown_connections 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.609+0000 7fee687ee640 1 --2- 172.21.15.62:0/3210745002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee60106910 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.609+0000 7fee687ee640 1 -- 172.21.15.62:0/3210745002 >> 172.21.15.62:0/3210745002 conn(0x7fee60101ca0 msgr2=0x7fee601040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.609+0000 7fee687ee640 1 -- 172.21.15.62:0/3210745002 shutdown_connections 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.609+0000 7fee687ee640 1 -- 172.21.15.62:0/3210745002 wait complete. 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.609+0000 7fee687ee640 1 Processor -- start 2024-09-11T23:42:45.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.609+0000 7fee687ee640 1 -- start start 2024-09-11T23:42:45.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.609+0000 7fee687ee640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee601916f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:45.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.609+0000 7fee687ee640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee6010aa70 con 0x7fee60106510 2024-09-11T23:42:45.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.610+0000 7fee6658a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee601916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:45.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.610+0000 7fee6658a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee601916f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42298/0 (socket says 172.21.15.62:42298) 2024-09-11T23:42:45.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.610+0000 7fee6658a640 1 -- 172.21.15.62:0/1454834338 learned_addr learned my addr 172.21.15.62:0/1454834338 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:45.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.610+0000 7fee6658a640 1 -- 172.21.15.62:0/1454834338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee60191c30 con 0x7fee60106510 2024-09-11T23:42:45.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.610+0000 7fee6658a640 1 --2- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee601916f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0097f0 tx=0x7fee5c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:45.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.610+0000 7fee577fe640 1 -- 172.21.15.62:0/1454834338 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee5c042070 con 0x7fee60106510 2024-09-11T23:42:45.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.610+0000 7fee577fe640 1 -- 172.21.15.62:0/1454834338 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fee5c032ca0 con 0x7fee60106510 2024-09-11T23:42:45.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.610+0000 7fee687ee640 1 -- 172.21.15.62:0/1454834338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6018fe20 con 0x7fee60106510 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.610+0000 7fee687ee640 1 -- 172.21.15.62:0/1454834338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee60190360 con 0x7fee60106510 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.610+0000 7fee577fe640 1 -- 172.21.15.62:0/1454834338 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee5c037040 con 0x7fee60106510 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.611+0000 7fee577fe640 1 -- 172.21.15.62:0/1454834338 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fee5c045070 con 0x7fee60106510 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.611+0000 7fee577fe640 1 --2- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fee4c038fc0 0x7fee4c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.611+0000 7fee577fe640 1 -- 172.21.15.62:0/1454834338 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee5c06cee0 con 0x7fee60106510 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.611+0000 7fee65d89640 1 -- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fee4c038fc0 msgr2=0x7fee4c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1330539138 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.612+0000 7fee65d89640 1 --2- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fee4c038fc0 0x7fee4c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.612+0000 7fee687ee640 1 -- 172.21.15.62:0/1454834338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee44005290 con 0x7fee60106510 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.616+0000 7fee577fe640 1 -- 172.21.15.62:0/1454834338 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee5c06c900 con 0x7fee60106510 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.813+0000 7fee65d89640 1 -- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fee4c038fc0 msgr2=0x7fee4c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1330539138 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.813+0000 7fee65d89640 1 --2- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fee4c038fc0 0x7fee4c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:42:45.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.839+0000 7fee687ee640 1 -- 172.21.15.62:0/1454834338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fee44005d90 con 0x7fee60106510 2024-09-11T23:42:45.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.839+0000 7fee577fe640 1 -- 172.21.15.62:0/1454834338 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fee5c045520 con 0x7fee60106510 2024-09-11T23:42:45.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.841+0000 7fee687ee640 1 -- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fee4c038fc0 msgr2=0x7fee4c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:42:45.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.841+0000 7fee687ee640 1 --2- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fee4c038fc0 0x7fee4c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:45.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.841+0000 7fee687ee640 1 -- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 msgr2=0x7fee601916f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:45.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.841+0000 7fee687ee640 1 --2- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee601916f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0097f0 tx=0x7fee5c030ec0 comp rx=0 tx=0).stop 2024-09-11T23:42:45.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.841+0000 7fee687ee640 1 -- 172.21.15.62:0/1454834338 shutdown_connections 2024-09-11T23:42:45.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.841+0000 7fee687ee640 1 --2- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7fee4c038fc0 0x7fee4c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:45.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.841+0000 7fee687ee640 1 --2- 172.21.15.62:0/1454834338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60106510 0x7fee601916f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:45.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.841+0000 7fee687ee640 1 -- 172.21.15.62:0/1454834338 >> 172.21.15.62:0/1454834338 conn(0x7fee60101ca0 msgr2=0x7fee60102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:45.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.841+0000 7fee687ee640 1 -- 172.21.15.62:0/1454834338 shutdown_connections 2024-09-11T23:42:45.467 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:44.841+0000 7fee687ee640 1 -- 172.21.15.62:0/1454834338 wait complete. 2024-09-11T23:42:45.467 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2024-09-11T23:42:45.467 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 5... 2024-09-11T23:42:45.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:45 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1454834338' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T23:42:46.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:45 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:45.883+0000 7f1dfe8db040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:42:46.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:45 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:45.955+0000 7f1dfe8db040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:42:46.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:46 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:46.698+0000 7f1dfe8db040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:42:46.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:46 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:46.819+0000 7f1dfe8db040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:42:47.221 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:46 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:46.967+0000 7f1dfe8db040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:42:47.221 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:47.031+0000 7f1dfe8db040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:42:47.221 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:47.186+0000 7f1dfe8db040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:42:47.484 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:47.257+0000 7f1dfe8db040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:42:47.484 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:47.320+0000 7f1dfe8db040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:42:47.484 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:47.485+0000 7f1dfe8db040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:42:47.769 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:47.553+0000 7f1dfe8db040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:42:47.769 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:47.622+0000 7f1dfe8db040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:42:47.769 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:47.708+0000 7f1dfe8db040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:42:47.769 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:47.770+0000 7f1dfe8db040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:42:48.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:47.832+0000 7f1dfe8db040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:42:48.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:48.655+0000 7f1dfe8db040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: Active manager daemon x restarted 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: Activating manager daemon x 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: mgrmap e6: x(active, starting, since 0.00342857s) 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: Manager daemon x is now available 2024-09-11T23:42:48.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:42:48.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:42:48.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:42:48.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:42:48.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:42:48.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:48 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:42:50.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:42:50.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T23:42:50.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T23:42:50.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:42:50.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.813+0000 7f190282c640 1 Processor -- start 2024-09-11T23:42:50.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.813+0000 7f190282c640 1 -- start start 2024-09-11T23:42:50.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.814+0000 7f190282c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc1062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:50.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.814+0000 7f190282c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc07be20 con 0x7f18fc105ec0 2024-09-11T23:42:50.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.814+0000 7f18fbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc1062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:50.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.814+0000 7f18fbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc1062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42300/0 (socket says 172.21.15.62:42300) 2024-09-11T23:42:50.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.814+0000 7f18fbfff640 1 -- 172.21.15.62:0/1335895935 learned_addr learned my addr 172.21.15.62:0/1335895935 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:50.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.814+0000 7f18fbfff640 1 -- 172.21.15.62:0/1335895935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc106800 con 0x7f18fc105ec0 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.815+0000 7f18fbfff640 1 --2- 172.21.15.62:0/1335895935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc1062c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009820 tx=0x7f18ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce225a442a224796 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.815+0000 7f18faffd640 1 -- 172.21.15.62:0/1335895935 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18ec037070 con 0x7f18fc105ec0 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.815+0000 7f18faffd640 1 -- 172.21.15.62:0/1335895935 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f18ec032900 con 0x7f18fc105ec0 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.815+0000 7f18faffd640 1 -- 172.21.15.62:0/1335895935 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18ec030850 con 0x7f18fc105ec0 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.816+0000 7f190282c640 1 -- 172.21.15.62:0/1335895935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 msgr2=0x7f18fc1062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.816+0000 7f190282c640 1 --2- 172.21.15.62:0/1335895935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc1062c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009820 tx=0x7f18ec02a270 comp rx=0 tx=0).stop 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.816+0000 7f190282c640 1 -- 172.21.15.62:0/1335895935 shutdown_connections 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.816+0000 7f190282c640 1 --2- 172.21.15.62:0/1335895935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc1062c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.816+0000 7f190282c640 1 -- 172.21.15.62:0/1335895935 >> 172.21.15.62:0/1335895935 conn(0x7f18fc101d10 msgr2=0x7f18fc104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.816+0000 7f190282c640 1 -- 172.21.15.62:0/1335895935 shutdown_connections 2024-09-11T23:42:50.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.816+0000 7f190282c640 1 -- 172.21.15.62:0/1335895935 wait complete. 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.816+0000 7f190282c640 1 Processor -- start 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.816+0000 7f190282c640 1 -- start start 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.817+0000 7f190282c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.817+0000 7f190282c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc107900 con 0x7f18fc105ec0 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.817+0000 7f18fbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.817+0000 7f18fbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42310/0 (socket says 172.21.15.62:42310) 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.817+0000 7f18fbfff640 1 -- 172.21.15.62:0/1827676600 learned_addr learned my addr 172.21.15.62:0/1827676600 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.817+0000 7f18fbfff640 1 -- 172.21.15.62:0/1827676600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc19dcf0 con 0x7f18fc105ec0 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.817+0000 7f18fbfff640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc19d7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0097f0 tx=0x7f18ec030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.818+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18ec037040 con 0x7f18fc105ec0 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.818+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f18ec040070 con 0x7f18fc105ec0 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.818+0000 7f190282c640 1 -- 172.21.15.62:0/1827676600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18fc19df80 con 0x7f18fc105ec0 2024-09-11T23:42:50.240 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.818+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18ec03a390 con 0x7f18fc105ec0 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.818+0000 7f190282c640 1 -- 172.21.15.62:0/1827676600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18fc1a0c70 con 0x7f18fc105ec0 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.819+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f18ec03a630 con 0x7f18fc105ec0 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.819+0000 7f18f97fa640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 0x7f18e403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.819+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 --> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f18e403bb40 con 0x7f18e4038f70 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.819+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f18ec06c470 con 0x7f18fc105ec0 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.819+0000 7f18fb7fe640 1 -- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 msgr2=0x7f18e403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1330539138 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:45.819+0000 7f18fb7fe640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 0x7f18e403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:46.020+0000 7f18fb7fe640 1 -- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 msgr2=0x7f18e403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1330539138 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:46.020+0000 7f18fb7fe640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 0x7f18e403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:46.421+0000 7f18fb7fe640 1 -- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 msgr2=0x7f18e403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1330539138 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:46.421+0000 7f18fb7fe640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 0x7f18e403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T23:42:50.241 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:47.222+0000 7f18fb7fe640 1 -- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 msgr2=0x7f18e403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1330539138 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:47.222+0000 7f18fb7fe640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 0x7f18e403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:48.659+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f18ec06b270 con 0x7f18fc105ec0 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:48.659+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 msgr2=0x7f18e403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:48.659+0000 7f18f97fa640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 0x7f18e403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.660+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f18ec045460 con 0x7f18fc105ec0 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.660+0000 7f18f97fa640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f18e403c8a0 0x7f18e403ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.660+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 --> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f18ec03de70 con 0x7f18e403c8a0 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.661+0000 7f18fb7fe640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f18e403c8a0 0x7f18e403ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.661+0000 7f18fb7fe640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f18e403c8a0 0x7f18e403ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18f0003980 tx=0x7f18f0006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.662+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 <== mgr.14118 v2:172.21.15.62:6800/3180930925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f18ec03de70 con 0x7f18e403c8a0 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.669+0000 7f18e2ffd640 1 -- 172.21.15.62:0/1827676600 --> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f18fc1062c0 con 0x7f18e403c8a0 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.669+0000 7f18f97fa640 1 -- 172.21.15.62:0/1827676600 <== mgr.14118 v2:172.21.15.62:6800/3180930925 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f18fc1062c0 con 0x7f18e403c8a0 2024-09-11T23:42:50.242 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.669+0000 7f18e2ffd640 1 -- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f18e403c8a0 msgr2=0x7f18e403ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.669+0000 7f18e2ffd640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f18e403c8a0 0x7f18e403ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18f0003980 tx=0x7f18f0006e10 comp rx=0 tx=0).stop 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.669+0000 7f18e2ffd640 1 -- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 msgr2=0x7f18fc19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.669+0000 7f18e2ffd640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc19d7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0097f0 tx=0x7f18ec030d80 comp rx=0 tx=0).stop 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.669+0000 7f18e2ffd640 1 -- 172.21.15.62:0/1827676600 shutdown_connections 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.670+0000 7f18e2ffd640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f18e403c8a0 0x7f18e403ec90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.670+0000 7f18e2ffd640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:6800/1330539138,v1:172.21.15.62:6801/1330539138] conn(0x7f18e4038f70 0x7f18e403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.670+0000 7f18e2ffd640 1 --2- 172.21.15.62:0/1827676600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc105ec0 0x7f18fc19d7b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.670+0000 7f18e2ffd640 1 -- 172.21.15.62:0/1827676600 >> 172.21.15.62:0/1827676600 conn(0x7f18fc101d10 msgr2=0x7f18fc191ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.670+0000 7f18e2ffd640 1 -- 172.21.15.62:0/1827676600 shutdown_connections 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:49.670+0000 7f18e2ffd640 1 -- 172.21.15.62:0/1827676600 wait complete. 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 5 is available 2024-09-11T23:42:50.243 INFO:teuthology.orchestra.run.smithi062.stdout:Setting orchestrator backend to cephadm... 2024-09-11T23:42:50.639 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:50 smithi062 ceph-mon[30958]: Found migration_current of "None". Setting to last migration. 2024-09-11T23:42:50.639 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:50 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:42:50.640 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:50 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:42:50.640 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:50 smithi062 ceph-mon[30958]: mgrmap e7: x(active, since 1.00563s) 2024-09-11T23:42:51.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.646+0000 7fcac2ee0640 1 Processor -- start 2024-09-11T23:42:51.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.646+0000 7fcac2ee0640 1 -- start start 2024-09-11T23:42:51.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.646+0000 7fcac2ee0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc10a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:51.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.646+0000 7fcac2ee0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc07bdf0 con 0x7fcabc1080c0 2024-09-11T23:42:51.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.646+0000 7fcac0c7c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc10a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:51.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.646+0000 7fcac0c7c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc10a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42382/0 (socket says 172.21.15.62:42382) 2024-09-11T23:42:51.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.646+0000 7fcac0c7c640 1 -- 172.21.15.62:0/1070127066 learned_addr learned my addr 172.21.15.62:0/1070127066 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:51.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.646+0000 7fcac0c7c640 1 -- 172.21.15.62:0/1070127066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc10a9f0 con 0x7fcabc1080c0 2024-09-11T23:42:51.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.647+0000 7fcac0c7c640 1 --2- 172.21.15.62:0/1070127066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc10a4b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009820 tx=0x7fcab002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4c2da78964445f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:51.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.647+0000 7fcabb7fe640 1 -- 172.21.15.62:0/1070127066 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcab0037070 con 0x7fcabc1080c0 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.647+0000 7fcabb7fe640 1 -- 172.21.15.62:0/1070127066 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcab0032900 con 0x7fcabc1080c0 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.647+0000 7fcabb7fe640 1 -- 172.21.15.62:0/1070127066 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcab0030850 con 0x7fcabc1080c0 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.648+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1070127066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 msgr2=0x7fcabc10a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.648+0000 7fcac2ee0640 1 --2- 172.21.15.62:0/1070127066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc10a4b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009820 tx=0x7fcab002a270 comp rx=0 tx=0).stop 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.648+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1070127066 shutdown_connections 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.648+0000 7fcac2ee0640 1 --2- 172.21.15.62:0/1070127066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc10a4b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.648+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1070127066 >> 172.21.15.62:0/1070127066 conn(0x7fcabc101ca0 msgr2=0x7fcabc1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.648+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1070127066 shutdown_connections 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.648+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1070127066 wait complete. 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.649+0000 7fcac2ee0640 1 Processor -- start 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.649+0000 7fcac2ee0640 1 -- start start 2024-09-11T23:42:51.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.649+0000 7fcac2ee0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc107940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.649+0000 7fcac2ee0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10b9d0 con 0x7fcabc1080c0 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.649+0000 7fcac0c7c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc107940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.649+0000 7fcac0c7c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc107940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42392/0 (socket says 172.21.15.62:42392) 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.649+0000 7fcac0c7c640 1 -- 172.21.15.62:0/1967269141 learned_addr learned my addr 172.21.15.62:0/1967269141 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.649+0000 7fcac0c7c640 1 -- 172.21.15.62:0/1967269141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc107e80 con 0x7fcabc1080c0 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.649+0000 7fcac0c7c640 1 --2- 172.21.15.62:0/1967269141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc107940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcab0031000 tx=0x7fcab00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.650+0000 7fcab9ffb640 1 -- 172.21.15.62:0/1967269141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcab0042070 con 0x7fcabc1080c0 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.650+0000 7fcab9ffb640 1 -- 172.21.15.62:0/1967269141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcab0032be0 con 0x7fcabc1080c0 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.650+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1967269141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc106070 con 0x7fcabc1080c0 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.650+0000 7fcab9ffb640 1 -- 172.21.15.62:0/1967269141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcab0037040 con 0x7fcabc1080c0 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.650+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1967269141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1065b0 con 0x7fcabc1080c0 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.651+0000 7fcab9ffb640 1 -- 172.21.15.62:0/1967269141 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fcab0045070 con 0x7fcabc1080c0 2024-09-11T23:42:51.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.651+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1967269141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcabc108520 con 0x7fcabc1080c0 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.651+0000 7fcab9ffb640 1 --2- 172.21.15.62:0/1967269141 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fcaac038eb0 0x7fcaac03b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.651+0000 7fcab9ffb640 1 -- 172.21.15.62:0/1967269141 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcab006bf00 con 0x7fcabc1080c0 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.651+0000 7fcabbfff640 1 --2- 172.21.15.62:0/1967269141 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fcaac038eb0 0x7fcaac03b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.651+0000 7fcabbfff640 1 --2- 172.21.15.62:0/1967269141 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fcaac038eb0 0x7fcaac03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80098c0 tx=0x7fcaa8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.660+0000 7fcab9ffb640 1 -- 172.21.15.62:0/1967269141 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcab003a540 con 0x7fcabc1080c0 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.852+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1967269141 --> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fcabc106e00 con 0x7fcaac038eb0 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.856+0000 7fcab9ffb640 1 -- 172.21.15.62:0/1967269141 <== mgr.14118 v2:172.21.15.62:6800/3180930925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcabc106e00 con 0x7fcaac038eb0 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.859+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1967269141 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fcaac038eb0 msgr2=0x7fcaac03b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.859+0000 7fcac2ee0640 1 --2- 172.21.15.62:0/1967269141 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fcaac038eb0 0x7fcaac03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80098c0 tx=0x7fcaa8006db0 comp rx=0 tx=0).stop 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.859+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1967269141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 msgr2=0x7fcabc107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.859+0000 7fcac2ee0640 1 --2- 172.21.15.62:0/1967269141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc107940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcab0031000 tx=0x7fcab00326e0 comp rx=0 tx=0).stop 2024-09-11T23:42:51.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.859+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1967269141 shutdown_connections 2024-09-11T23:42:51.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.859+0000 7fcac2ee0640 1 --2- 172.21.15.62:0/1967269141 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fcaac038eb0 0x7fcaac03b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:51.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.859+0000 7fcac2ee0640 1 --2- 172.21.15.62:0/1967269141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcabc1080c0 0x7fcabc107940 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:51.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.859+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1967269141 >> 172.21.15.62:0/1967269141 conn(0x7fcabc101ca0 msgr2=0x7fcabc102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:51.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.859+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1967269141 shutdown_connections 2024-09-11T23:42:51.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:50.859+0000 7fcac2ee0640 1 -- 172.21.15.62:0/1967269141 wait complete. 2024-09-11T23:42:51.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-mon[30958]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T23:42:51.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-mon[30958]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T23:42:51.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:42:51.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:42:51.748 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:51.526+0000 7f1d65bf5640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:42:51] ENGINE Error in HTTPServer.serve 2024-09-11T23:42:51.748 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: Traceback (most recent call last): 2024-09-11T23:42:51.748 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:42:51.748 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self._connections.run(self.expiration_interval) 2024-09-11T23:42:51.748 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:42:51.748 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self._run(expiration_interval) 2024-09-11T23:42:51.748 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:42:51.748 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:42:51.748 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: s = self.context.wrap_socket( 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: return self.sslsocket_class._create( 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self.do_handshake() 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self._sslobj.do_handshake() 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:42:51.749 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:52 smithi062 ceph-mon[30958]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:52 smithi062 ceph-mon[30958]: [11/Sep/2024:23:42:51] ENGINE Bus STARTING 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:52 smithi062 ceph-mon[30958]: [11/Sep/2024:23:42:51] ENGINE Serving on http://172.21.15.62:8765 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:52 smithi062 ceph-mon[30958]: [11/Sep/2024:23:42:51] ENGINE Serving on https://172.21.15.62:7150 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:52 smithi062 ceph-mon[30958]: [11/Sep/2024:23:42:51] ENGINE Bus STARTED 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:52 smithi062 ceph-mon[30958]: [11/Sep/2024:23:42:51] ENGINE Error in HTTPServer.serve 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:42:52.710 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._run(expiration_interval) 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: s = self.context.wrap_socket( 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.sslsocket_class._create( 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: self.do_handshake() 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._sslobj.do_handshake() 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:42:52.711 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:52 smithi062 ceph-mon[30958]: mgrmap e8: x(active, since 2s) 2024-09-11T23:42:52.712 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:52 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:42:52.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T23:42:52.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.936+0000 7efe386db640 1 Processor -- start 2024-09-11T23:42:52.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.936+0000 7efe386db640 1 -- start start 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.937+0000 7efe386db640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe30106900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.937+0000 7efe386db640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efe30106ed0 con 0x7efe30106500 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.937+0000 7efe36477640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe30106900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.937+0000 7efe36477640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe30106900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54926/0 (socket says 172.21.15.62:54926) 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.937+0000 7efe36477640 1 -- 172.21.15.62:0/3418615857 learned_addr learned my addr 172.21.15.62:0/3418615857 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.937+0000 7efe36477640 1 -- 172.21.15.62:0/3418615857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe30107700 con 0x7efe30106500 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.938+0000 7efe36477640 1 --2- 172.21.15.62:0/3418615857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe30106900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009820 tx=0x7efe2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8458847d8d90705 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.938+0000 7efe35475640 1 -- 172.21.15.62:0/3418615857 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe2c037070 con 0x7efe30106500 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.938+0000 7efe35475640 1 -- 172.21.15.62:0/3418615857 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe2c032900 con 0x7efe30106500 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.938+0000 7efe35475640 1 -- 172.21.15.62:0/3418615857 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe2c030850 con 0x7efe30106500 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.939+0000 7efe386db640 1 -- 172.21.15.62:0/3418615857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 msgr2=0x7efe30106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:52.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.939+0000 7efe386db640 1 --2- 172.21.15.62:0/3418615857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe30106900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009820 tx=0x7efe2c02a270 comp rx=0 tx=0).stop 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.939+0000 7efe386db640 1 -- 172.21.15.62:0/3418615857 shutdown_connections 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.939+0000 7efe386db640 1 --2- 172.21.15.62:0/3418615857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe30106900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.939+0000 7efe386db640 1 -- 172.21.15.62:0/3418615857 >> 172.21.15.62:0/3418615857 conn(0x7efe30101d10 msgr2=0x7efe30104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.939+0000 7efe386db640 1 -- 172.21.15.62:0/3418615857 shutdown_connections 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.939+0000 7efe386db640 1 -- 172.21.15.62:0/3418615857 wait complete. 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.940+0000 7efe386db640 1 Processor -- start 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.940+0000 7efe386db640 1 -- start start 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.940+0000 7efe386db640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe3019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.940+0000 7efe386db640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efe3010aab0 con 0x7efe30106500 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.940+0000 7efe36477640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe3019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.940+0000 7efe36477640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe3019b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54932/0 (socket says 172.21.15.62:54932) 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.940+0000 7efe36477640 1 -- 172.21.15.62:0/3817959840 learned_addr learned my addr 172.21.15.62:0/3817959840 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:52.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.940+0000 7efe36477640 1 -- 172.21.15.62:0/3817959840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3019bb50 con 0x7efe30106500 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.940+0000 7efe36477640 1 --2- 172.21.15.62:0/3817959840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe3019b610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097f0 tx=0x7efe2c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.941+0000 7efe237fe640 1 -- 172.21.15.62:0/3817959840 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe2c037040 con 0x7efe30106500 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.941+0000 7efe386db640 1 -- 172.21.15.62:0/3817959840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3019bde0 con 0x7efe30106500 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.941+0000 7efe386db640 1 -- 172.21.15.62:0/3817959840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3019ead0 con 0x7efe30106500 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.941+0000 7efe237fe640 1 -- 172.21.15.62:0/3817959840 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe2c040070 con 0x7efe30106500 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.941+0000 7efe237fe640 1 -- 172.21.15.62:0/3817959840 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe2c03a390 con 0x7efe30106500 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.942+0000 7efe386db640 1 -- 172.21.15.62:0/3817959840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe14005290 con 0x7efe30106500 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.942+0000 7efe237fe640 1 -- 172.21.15.62:0/3817959840 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efe2c03a6b0 con 0x7efe30106500 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.942+0000 7efe237fe640 1 --2- 172.21.15.62:0/3817959840 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7efe1c038f70 0x7efe1c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.942+0000 7efe237fe640 1 -- 172.21.15.62:0/3817959840 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efe2c06c560 con 0x7efe30106500 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.942+0000 7efe35c76640 1 --2- 172.21.15.62:0/3817959840 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7efe1c038f70 0x7efe1c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:52.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.943+0000 7efe35c76640 1 --2- 172.21.15.62:0/3817959840 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7efe1c038f70 0x7efe1c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe240098c0 tx=0x7efe24006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:51.951+0000 7efe237fe640 1 -- 172.21.15.62:0/3817959840 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe2c043790 con 0x7efe30106500 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.132+0000 7efe386db640 1 -- 172.21.15.62:0/3817959840 --> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7efe14000bc0 con 0x7efe1c038f70 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.132+0000 7efe237fe640 1 -- 172.21.15.62:0/3817959840 <== mgr.14118 v2:172.21.15.62:6800/3180930925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7efe14000bc0 con 0x7efe1c038f70 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.135+0000 7efe386db640 1 -- 172.21.15.62:0/3817959840 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7efe1c038f70 msgr2=0x7efe1c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.135+0000 7efe386db640 1 --2- 172.21.15.62:0/3817959840 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7efe1c038f70 0x7efe1c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe240098c0 tx=0x7efe24006db0 comp rx=0 tx=0).stop 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.135+0000 7efe386db640 1 -- 172.21.15.62:0/3817959840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 msgr2=0x7efe3019b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.135+0000 7efe386db640 1 --2- 172.21.15.62:0/3817959840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe3019b610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097f0 tx=0x7efe2c030d80 comp rx=0 tx=0).stop 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.135+0000 7efe386db640 1 -- 172.21.15.62:0/3817959840 shutdown_connections 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.135+0000 7efe386db640 1 --2- 172.21.15.62:0/3817959840 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7efe1c038f70 0x7efe1c03b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.135+0000 7efe386db640 1 --2- 172.21.15.62:0/3817959840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30106500 0x7efe3019b610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.135+0000 7efe386db640 1 -- 172.21.15.62:0/3817959840 >> 172.21.15.62:0/3817959840 conn(0x7efe30101d10 msgr2=0x7efe30102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:52.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.135+0000 7efe386db640 1 -- 172.21.15.62:0/3817959840 shutdown_connections 2024-09-11T23:42:52.722 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:52.135+0000 7efe386db640 1 -- 172.21.15.62:0/3817959840 wait complete. 2024-09-11T23:42:52.722 INFO:teuthology.orchestra.run.smithi062.stdout:Generating ssh key... 2024-09-11T23:42:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-mon[30958]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:42:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:42:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:42:53.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: Generating public/private rsa key pair. 2024-09-11T23:42:53.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: Your identification has been saved in /tmp/tmpwlnqjlhz/key 2024-09-11T23:42:53.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: Your public key has been saved in /tmp/tmpwlnqjlhz/key.pub 2024-09-11T23:42:53.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: The key fingerprint is: 2024-09-11T23:42:53.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: SHA256:1Qbdt98H5y1iYmRtik12uoo+yxeB8S2SwND3UTHGnlE ceph-56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:53.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: The key's randomart image is: 2024-09-11T23:42:53.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: +---[RSA 3072]----+ 2024-09-11T23:42:53.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: | .+ o*+E. | 2024-09-11T23:42:53.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: | + o ..o+. . .| 2024-09-11T23:42:53.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: | o * +.+o ..| 2024-09-11T23:42:53.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: | + =.O.+ ...| 2024-09-11T23:42:53.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: | .SX = ++| 2024-09-11T23:42:53.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: | o * o . *| 2024-09-11T23:42:53.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: | o + . ..| 2024-09-11T23:42:53.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: | .... . | 2024-09-11T23:42:53.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: | .=+.. | 2024-09-11T23:42:53.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:42:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: +----[SHA256]-----+ 2024-09-11T23:42:54.232 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.095+0000 7f4a19d1c640 1 Processor -- start 2024-09-11T23:42:54.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.095+0000 7f4a19d1c640 1 -- start start 2024-09-11T23:42:54.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.095+0000 7f4a19d1c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1410a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:54.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.095+0000 7f4a19d1c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1407be40 con 0x7f4a141080d0 2024-09-11T23:42:54.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.095+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1410a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:54.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.095+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1410a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54936/0 (socket says 172.21.15.62:54936) 2024-09-11T23:42:54.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.095+0000 7f4a137fe640 1 -- 172.21.15.62:0/894858028 learned_addr learned my addr 172.21.15.62:0/894858028 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:54.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.096+0000 7f4a137fe640 1 -- 172.21.15.62:0/894858028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1410aa00 con 0x7f4a141080d0 2024-09-11T23:42:54.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.096+0000 7f4a137fe640 1 --2- 172.21.15.62:0/894858028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1410a4c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009820 tx=0x7f4a0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ba66bd0c7834bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:54.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.096+0000 7f4a12ffd640 1 -- 172.21.15.62:0/894858028 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a04037070 con 0x7f4a141080d0 2024-09-11T23:42:54.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.096+0000 7f4a12ffd640 1 -- 172.21.15.62:0/894858028 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a04032900 con 0x7f4a141080d0 2024-09-11T23:42:54.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.096+0000 7f4a12ffd640 1 -- 172.21.15.62:0/894858028 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a04030850 con 0x7f4a141080d0 2024-09-11T23:42:54.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.097+0000 7f4a19d1c640 1 -- 172.21.15.62:0/894858028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 msgr2=0x7f4a1410a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:54.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.097+0000 7f4a19d1c640 1 --2- 172.21.15.62:0/894858028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1410a4c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009820 tx=0x7f4a0402a270 comp rx=0 tx=0).stop 2024-09-11T23:42:54.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.097+0000 7f4a19d1c640 1 -- 172.21.15.62:0/894858028 shutdown_connections 2024-09-11T23:42:54.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.097+0000 7f4a19d1c640 1 --2- 172.21.15.62:0/894858028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1410a4c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:54.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.097+0000 7f4a19d1c640 1 -- 172.21.15.62:0/894858028 >> 172.21.15.62:0/894858028 conn(0x7f4a14101cb0 msgr2=0x7f4a141040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:54.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.097+0000 7f4a19d1c640 1 -- 172.21.15.62:0/894858028 shutdown_connections 2024-09-11T23:42:54.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.097+0000 7f4a19d1c640 1 -- 172.21.15.62:0/894858028 wait complete. 2024-09-11T23:42:54.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a19d1c640 1 Processor -- start 2024-09-11T23:42:54.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a19d1c640 1 -- start start 2024-09-11T23:42:54.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a19d1c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:54.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a19d1c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1410ba40 con 0x7f4a141080d0 2024-09-11T23:42:54.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:54.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54942/0 (socket says 172.21.15.62:54942) 2024-09-11T23:42:54.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a137fe640 1 -- 172.21.15.62:0/3887260533 learned_addr learned my addr 172.21.15.62:0/3887260533 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:54.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a137fe640 1 -- 172.21.15.62:0/3887260533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1419bba0 con 0x7f4a141080d0 2024-09-11T23:42:54.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a137fe640 1 --2- 172.21.15.62:0/3887260533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1419b660 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a04031000 tx=0x7f4a040326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:54.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a117fa640 1 -- 172.21.15.62:0/3887260533 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a04042070 con 0x7f4a141080d0 2024-09-11T23:42:54.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.098+0000 7f4a117fa640 1 -- 172.21.15.62:0/3887260533 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a04032be0 con 0x7f4a141080d0 2024-09-11T23:42:54.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.099+0000 7f4a19d1c640 1 -- 172.21.15.62:0/3887260533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1419be30 con 0x7f4a141080d0 2024-09-11T23:42:54.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.099+0000 7f4a117fa640 1 -- 172.21.15.62:0/3887260533 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a04037040 con 0x7f4a141080d0 2024-09-11T23:42:54.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.099+0000 7f4a19d1c640 1 -- 172.21.15.62:0/3887260533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1419eb20 con 0x7f4a141080d0 2024-09-11T23:42:54.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.099+0000 7f4a117fa640 1 -- 172.21.15.62:0/3887260533 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4a04045070 con 0x7f4a141080d0 2024-09-11T23:42:54.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.099+0000 7f4a117fa640 1 --2- 172.21.15.62:0/3887260533 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f4a00038fc0 0x7f4a0003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:54.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.099+0000 7f4a19d1c640 1 -- 172.21.15.62:0/3887260533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a14108530 con 0x7f4a141080d0 2024-09-11T23:42:54.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.100+0000 7f4a117fa640 1 -- 172.21.15.62:0/3887260533 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4a0406bf30 con 0x7f4a141080d0 2024-09-11T23:42:54.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.100+0000 7f4a0bfff640 1 --2- 172.21.15.62:0/3887260533 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f4a00038fc0 0x7f4a0003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:54.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.100+0000 7f4a0bfff640 1 --2- 172.21.15.62:0/3887260533 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f4a00038fc0 0x7f4a0003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0098c0 tx=0x7f49fc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:54.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.104+0000 7f4a117fa640 1 -- 172.21.15.62:0/3887260533 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a0402f2a0 con 0x7f4a141080d0 2024-09-11T23:42:54.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.284+0000 7f4a19d1c640 1 -- 172.21.15.62:0/3887260533 --> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f4a14102ff0 con 0x7f4a00038fc0 2024-09-11T23:42:54.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.520+0000 7f4a117fa640 1 -- 172.21.15.62:0/3887260533 <== mgr.14118 v2:172.21.15.62:6800/3180930925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4a14102ff0 con 0x7f4a00038fc0 2024-09-11T23:42:54.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.522+0000 7f4a19d1c640 1 -- 172.21.15.62:0/3887260533 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f4a00038fc0 msgr2=0x7f4a0003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:54.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.522+0000 7f4a19d1c640 1 --2- 172.21.15.62:0/3887260533 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f4a00038fc0 0x7f4a0003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0098c0 tx=0x7f49fc006db0 comp rx=0 tx=0).stop 2024-09-11T23:42:54.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.522+0000 7f4a19d1c640 1 -- 172.21.15.62:0/3887260533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 msgr2=0x7f4a1419b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:54.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.522+0000 7f4a19d1c640 1 --2- 172.21.15.62:0/3887260533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1419b660 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a04031000 tx=0x7f4a040326e0 comp rx=0 tx=0).stop 2024-09-11T23:42:54.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.523+0000 7f4a19d1c640 1 -- 172.21.15.62:0/3887260533 shutdown_connections 2024-09-11T23:42:54.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.523+0000 7f4a19d1c640 1 --2- 172.21.15.62:0/3887260533 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f4a00038fc0 0x7f4a0003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:54.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.523+0000 7f4a19d1c640 1 --2- 172.21.15.62:0/3887260533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a141080d0 0x7f4a1419b660 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:54.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.523+0000 7f4a19d1c640 1 -- 172.21.15.62:0/3887260533 >> 172.21.15.62:0/3887260533 conn(0x7f4a14101cb0 msgr2=0x7f4a141028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:54.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.523+0000 7f4a19d1c640 1 -- 172.21.15.62:0/3887260533 shutdown_connections 2024-09-11T23:42:54.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:53.523+0000 7f4a19d1c640 1 -- 172.21.15.62:0/3887260533 wait complete. 2024-09-11T23:42:54.786 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:54 smithi062 ceph-mon[30958]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:42:54.786 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:54 smithi062 ceph-mon[30958]: Generating ssh key... 2024-09-11T23:42:55.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzFhWzYrJcmK8rnIwIL0yfLwE89WskHh1HtYsybI3N531Sg2JpOPl2vgUjQedjqF6qPjpuPDYtn5fLba3MjxzZW2PwpWgCXb2Ac5jKIE0cpePjkRIMQgI3+QraFOylA5AtaM01SzUFeUQ2PcTTZcQo0aitjeNGS8D7BoswMZSwXFDWiM+FBlVRPRbB0qt07Md9wNa9y93rbi5w3kAESf6HyYngyicbu/i647omFN5w6qIS0bZOqInDL6YxtfvtkCIW1/KQQA9Xr6io8/hctBX2TxTnINdV+qrmW1OGIA4SNYHt0p6A1hzGrzCyVr/SGLyKZs9Sn5TRzK1mDXji5KlcY1cosPTBQ0gDDaEseOAVZDJsosSw2z4FkvNLJcbD2p8wbgWOC03QkcmPW+qzqC4aNLKiYEbpwGfE//dnzH6EHe0ENO1ivKX8xmQqe5GuHXYOpDGfjlEEavkWS8K9JEdfZnD/pldtJWwpCYgFt4Ov0IJlkFPpqBPVSZpnH08t+PE= ceph-56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:42:55.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.598+0000 7fac1826e640 1 Processor -- start 2024-09-11T23:42:55.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.598+0000 7fac1826e640 1 -- start start 2024-09-11T23:42:55.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.598+0000 7fac1826e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1007c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.598+0000 7fac1826e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fac1007c8e0 con 0x7fac1007deb0 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.599+0000 7fac1600a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1007c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.599+0000 7fac1600a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1007c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54956/0 (socket says 172.21.15.62:54956) 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.599+0000 7fac1600a640 1 -- 172.21.15.62:0/1597863214 learned_addr learned my addr 172.21.15.62:0/1597863214 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.599+0000 7fac1600a640 1 -- 172.21.15.62:0/1597863214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1007ca60 con 0x7fac1007deb0 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.600+0000 7fac1600a640 1 --2- 172.21.15.62:0/1597863214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1007c310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009820 tx=0x7fac0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9ff0c6de743d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.600+0000 7fac15008640 1 -- 172.21.15.62:0/1597863214 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac0c037070 con 0x7fac1007deb0 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.600+0000 7fac15008640 1 -- 172.21.15.62:0/1597863214 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac0c032900 con 0x7fac1007deb0 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.600+0000 7fac15008640 1 -- 172.21.15.62:0/1597863214 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac0c030850 con 0x7fac1007deb0 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.600+0000 7fac1826e640 1 -- 172.21.15.62:0/1597863214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 msgr2=0x7fac1007c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.600+0000 7fac1826e640 1 --2- 172.21.15.62:0/1597863214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1007c310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009820 tx=0x7fac0c02a270 comp rx=0 tx=0).stop 2024-09-11T23:42:55.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1826e640 1 -- 172.21.15.62:0/1597863214 shutdown_connections 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1826e640 1 --2- 172.21.15.62:0/1597863214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1007c310 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1826e640 1 -- 172.21.15.62:0/1597863214 >> 172.21.15.62:0/1597863214 conn(0x7fac10102db0 msgr2=0x7fac101051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1826e640 1 -- 172.21.15.62:0/1597863214 shutdown_connections 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1826e640 1 -- 172.21.15.62:0/1597863214 wait complete. 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1826e640 1 Processor -- start 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1826e640 1 -- start start 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1826e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1019d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1826e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fac1010dc50 con 0x7fac1007deb0 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1600a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1019d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1600a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1019d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54958/0 (socket says 172.21.15.62:54958) 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.601+0000 7fac1600a640 1 -- 172.21.15.62:0/2056510622 learned_addr learned my addr 172.21.15.62:0/2056510622 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:42:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.602+0000 7fac1600a640 1 -- 172.21.15.62:0/2056510622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1019ddf0 con 0x7fac1007deb0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.602+0000 7fac1600a640 1 --2- 172.21.15.62:0/2056510622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1019d8b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009940 tx=0x7fac0c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.602+0000 7fac037fe640 1 -- 172.21.15.62:0/2056510622 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac0c042070 con 0x7fac1007deb0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.602+0000 7fac037fe640 1 -- 172.21.15.62:0/2056510622 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac0c032be0 con 0x7fac1007deb0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.602+0000 7fac1826e640 1 -- 172.21.15.62:0/2056510622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1019e080 con 0x7fac1007deb0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.602+0000 7fac037fe640 1 -- 172.21.15.62:0/2056510622 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac0c037040 con 0x7fac1007deb0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.602+0000 7fac1826e640 1 -- 172.21.15.62:0/2056510622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac101a0d70 con 0x7fac1007deb0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.603+0000 7fac037fe640 1 -- 172.21.15.62:0/2056510622 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fac0c045070 con 0x7fac1007deb0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.603+0000 7fac1826e640 1 -- 172.21.15.62:0/2056510622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1007c390 con 0x7fac1007deb0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.603+0000 7fac037fe640 1 --2- 172.21.15.62:0/2056510622 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fabfc038f70 0x7fabfc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.603+0000 7fac037fe640 1 -- 172.21.15.62:0/2056510622 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fac0c06bfb0 con 0x7fac1007deb0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.603+0000 7fac15809640 1 --2- 172.21.15.62:0/2056510622 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fabfc038f70 0x7fabfc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.604+0000 7fac15809640 1 --2- 172.21.15.62:0/2056510622 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fabfc038f70 0x7fabfc03b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac040098c0 tx=0x7fac04006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.608+0000 7fac037fe640 1 -- 172.21.15.62:0/2056510622 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac0c06b900 con 0x7fac1007deb0 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.786+0000 7fac1826e640 1 -- 172.21.15.62:0/2056510622 --> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fac101040e0 con 0x7fabfc038f70 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.787+0000 7fac037fe640 1 -- 172.21.15.62:0/2056510622 <== mgr.14118 v2:172.21.15.62:6800/3180930925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fac101040e0 con 0x7fabfc038f70 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.789+0000 7fac1826e640 1 -- 172.21.15.62:0/2056510622 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fabfc038f70 msgr2=0x7fabfc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.789+0000 7fac1826e640 1 --2- 172.21.15.62:0/2056510622 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fabfc038f70 0x7fabfc03b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac040098c0 tx=0x7fac04006db0 comp rx=0 tx=0).stop 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.789+0000 7fac1826e640 1 -- 172.21.15.62:0/2056510622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 msgr2=0x7fac1019d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.789+0000 7fac1826e640 1 --2- 172.21.15.62:0/2056510622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1019d8b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009940 tx=0x7fac0c0326e0 comp rx=0 tx=0).stop 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.789+0000 7fac1826e640 1 -- 172.21.15.62:0/2056510622 shutdown_connections 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.789+0000 7fac1826e640 1 --2- 172.21.15.62:0/2056510622 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fabfc038f70 0x7fabfc03b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.789+0000 7fac1826e640 1 --2- 172.21.15.62:0/2056510622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac1007deb0 0x7fac1019d8b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.789+0000 7fac1826e640 1 -- 172.21.15.62:0/2056510622 >> 172.21.15.62:0/2056510622 conn(0x7fac10102db0 msgr2=0x7fac101039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.789+0000 7fac1826e640 1 -- 172.21.15.62:0/2056510622 shutdown_connections 2024-09-11T23:42:55.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:54.789+0000 7fac1826e640 1 -- 172.21.15.62:0/2056510622 wait complete. 2024-09-11T23:42:55.369 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:42:55.369 INFO:teuthology.orchestra.run.smithi062.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T23:42:55.370 INFO:teuthology.orchestra.run.smithi062.stdout:Adding host smithi062... 2024-09-11T23:42:55.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:55 smithi062 ceph-mon[30958]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:42:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:56 smithi062 ceph-mon[30958]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:42:57.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:42:57 smithi062 ceph-mon[30958]: Deploying cephadm binary to smithi062 2024-09-11T23:43:00.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Added host 'smithi062' with addr '172.21.15.62' 2024-09-11T23:43:00.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.823+0000 7fc0aeb79640 1 Processor -- start 2024-09-11T23:43:00.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.823+0000 7fc0aeb79640 1 -- start start 2024-09-11T23:43:00.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.823+0000 7fc0aeb79640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a8105cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:00.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.823+0000 7fc0aeb79640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a8106290 con 0x7fc0a81058c0 2024-09-11T23:43:00.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.824+0000 7fc0ac915640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a8105cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.824+0000 7fc0ac915640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a8105cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54960/0 (socket says 172.21.15.62:54960) 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.824+0000 7fc0ac915640 1 -- 172.21.15.62:0/3418284666 learned_addr learned my addr 172.21.15.62:0/3418284666 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.824+0000 7fc0ac915640 1 -- 172.21.15.62:0/3418284666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a81088a0 con 0x7fc0a81058c0 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.825+0000 7fc0ac915640 1 --2- 172.21.15.62:0/3418284666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a8105cc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009820 tx=0x7fc0a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c651203ed6f9fef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.825+0000 7fc09f7fe640 1 -- 172.21.15.62:0/3418284666 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0a4037070 con 0x7fc0a81058c0 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.825+0000 7fc09f7fe640 1 -- 172.21.15.62:0/3418284666 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc0a4032900 con 0x7fc0a81058c0 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.825+0000 7fc09f7fe640 1 -- 172.21.15.62:0/3418284666 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0a4030850 con 0x7fc0a81058c0 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.825+0000 7fc0aeb79640 1 -- 172.21.15.62:0/3418284666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 msgr2=0x7fc0a8105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.825+0000 7fc0aeb79640 1 --2- 172.21.15.62:0/3418284666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a8105cc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009820 tx=0x7fc0a402a270 comp rx=0 tx=0).stop 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.825+0000 7fc0aeb79640 1 -- 172.21.15.62:0/3418284666 shutdown_connections 2024-09-11T23:43:00.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.825+0000 7fc0aeb79640 1 --2- 172.21.15.62:0/3418284666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a8105cc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.825+0000 7fc0aeb79640 1 -- 172.21.15.62:0/3418284666 >> 172.21.15.62:0/3418284666 conn(0x7fc0a8101450 msgr2=0x7fc0a8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.826+0000 7fc0aeb79640 1 -- 172.21.15.62:0/3418284666 shutdown_connections 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.826+0000 7fc0aeb79640 1 -- 172.21.15.62:0/3418284666 wait complete. 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.826+0000 7fc0aeb79640 1 Processor -- start 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.826+0000 7fc0aeb79640 1 -- start start 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.826+0000 7fc0aeb79640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.826+0000 7fc0aeb79640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810bad0 con 0x7fc0a81058c0 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.826+0000 7fc0ac915640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.826+0000 7fc0ac915640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a819d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54962/0 (socket says 172.21.15.62:54962) 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.826+0000 7fc0ac915640 1 -- 172.21.15.62:0/253476709 learned_addr learned my addr 172.21.15.62:0/253476709 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.827+0000 7fc0ac915640 1 -- 172.21.15.62:0/253476709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a819dda0 con 0x7fc0a81058c0 2024-09-11T23:43:00.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.827+0000 7fc0ac915640 1 --2- 172.21.15.62:0/253476709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a819d860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009940 tx=0x7fc0a4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.827+0000 7fc09dffb640 1 -- 172.21.15.62:0/253476709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0a4042070 con 0x7fc0a81058c0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.827+0000 7fc09dffb640 1 -- 172.21.15.62:0/253476709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc0a40308c0 con 0x7fc0a81058c0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.827+0000 7fc0aeb79640 1 -- 172.21.15.62:0/253476709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a819e030 con 0x7fc0a81058c0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.827+0000 7fc09dffb640 1 -- 172.21.15.62:0/253476709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0a4037040 con 0x7fc0a81058c0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.827+0000 7fc0aeb79640 1 -- 172.21.15.62:0/253476709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a81a0d20 con 0x7fc0a81058c0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.828+0000 7fc09dffb640 1 -- 172.21.15.62:0/253476709 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc0a4045070 con 0x7fc0a81058c0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.828+0000 7fc0aeb79640 1 -- 172.21.15.62:0/253476709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a8105d40 con 0x7fc0a81058c0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.828+0000 7fc09dffb640 1 --2- 172.21.15.62:0/253476709 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc090038f70 0x7fc09003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.828+0000 7fc09dffb640 1 -- 172.21.15.62:0/253476709 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc0a406c070 con 0x7fc0a81058c0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.829+0000 7fc09ffff640 1 --2- 172.21.15.62:0/253476709 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc090038f70 0x7fc09003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.829+0000 7fc09ffff640 1 --2- 172.21.15.62:0/253476709 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc090038f70 0x7fc09003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc0980098c0 tx=0x7fc098006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:00.492 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:55.833+0000 7fc09dffb640 1 -- 172.21.15.62:0/253476709 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0a403cd90 con 0x7fc0a81058c0 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:56.013+0000 7fc0aeb79640 1 -- 172.21.15.62:0/253476709 --> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}) -- 0x7fc0a81026b0 con 0x7fc090038f70 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.939+0000 7fc09dffb640 1 -- 172.21.15.62:0/253476709 <== mgr.14118 v2:172.21.15.62:6800/3180930925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc0a81026b0 con 0x7fc090038f70 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.942+0000 7fc0aeb79640 1 -- 172.21.15.62:0/253476709 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc090038f70 msgr2=0x7fc09003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.942+0000 7fc0aeb79640 1 --2- 172.21.15.62:0/253476709 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc090038f70 0x7fc09003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc0980098c0 tx=0x7fc098006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.942+0000 7fc0aeb79640 1 -- 172.21.15.62:0/253476709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 msgr2=0x7fc0a819d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.942+0000 7fc0aeb79640 1 --2- 172.21.15.62:0/253476709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a819d860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009940 tx=0x7fc0a4032ce0 comp rx=0 tx=0).stop 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.942+0000 7fc0aeb79640 1 -- 172.21.15.62:0/253476709 shutdown_connections 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.942+0000 7fc0aeb79640 1 --2- 172.21.15.62:0/253476709 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc090038f70 0x7fc09003b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.942+0000 7fc0aeb79640 1 --2- 172.21.15.62:0/253476709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0a81058c0 0x7fc0a819d860 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.942+0000 7fc0aeb79640 1 -- 172.21.15.62:0/253476709 >> 172.21.15.62:0/253476709 conn(0x7fc0a8101450 msgr2=0x7fc0a8101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.942+0000 7fc0aeb79640 1 -- 172.21.15.62:0/253476709 shutdown_connections 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:42:59.942+0000 7fc0aeb79640 1 -- 172.21.15.62:0/253476709 wait complete. 2024-09-11T23:43:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mon service... 2024-09-11T23:43:01.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:00 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:43:01.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:00 smithi062 ceph-mon[30958]: Added host smithi062 2024-09-11T23:43:01.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:00 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:43:01.841 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T23:43:01.842 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.891+0000 7fadb2bdc640 1 Processor -- start 2024-09-11T23:43:01.842 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.891+0000 7fadb2bdc640 1 -- start start 2024-09-11T23:43:01.842 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.892+0000 7fadb2bdc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:01.842 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.892+0000 7fadb2bdc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fadac07c880 con 0x7fadac07de50 2024-09-11T23:43:01.842 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.892+0000 7fadb0978640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:01.842 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.892+0000 7fadb0978640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54978/0 (socket says 172.21.15.62:54978) 2024-09-11T23:43:01.842 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.892+0000 7fadb0978640 1 -- 172.21.15.62:0/3139614687 learned_addr learned my addr 172.21.15.62:0/3139614687 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:01.842 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.892+0000 7fadb0978640 1 -- 172.21.15.62:0/3139614687 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac07ca00 con 0x7fadac07de50 2024-09-11T23:43:01.843 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.893+0000 7fadb0978640 1 --2- 172.21.15.62:0/3139614687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac07c2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fada0009820 tx=0x7fada002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5095a6d7e327b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:01.843 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.893+0000 7fadab7fe640 1 -- 172.21.15.62:0/3139614687 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada0037070 con 0x7fadac07de50 2024-09-11T23:43:01.843 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.893+0000 7fadab7fe640 1 -- 172.21.15.62:0/3139614687 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fada0032900 con 0x7fadac07de50 2024-09-11T23:43:01.843 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.893+0000 7fadab7fe640 1 -- 172.21.15.62:0/3139614687 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada0030850 con 0x7fadac07de50 2024-09-11T23:43:01.843 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.894+0000 7fadb2bdc640 1 -- 172.21.15.62:0/3139614687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 msgr2=0x7fadac07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:01.843 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.894+0000 7fadb2bdc640 1 --2- 172.21.15.62:0/3139614687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac07c2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fada0009820 tx=0x7fada002a270 comp rx=0 tx=0).stop 2024-09-11T23:43:01.843 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.894+0000 7fadb2bdc640 1 -- 172.21.15.62:0/3139614687 shutdown_connections 2024-09-11T23:43:01.843 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.894+0000 7fadb2bdc640 1 --2- 172.21.15.62:0/3139614687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac07c2b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:01.843 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.894+0000 7fadb2bdc640 1 -- 172.21.15.62:0/3139614687 >> 172.21.15.62:0/3139614687 conn(0x7fadac102db0 msgr2=0x7fadac1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.894+0000 7fadb2bdc640 1 -- 172.21.15.62:0/3139614687 shutdown_connections 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.894+0000 7fadb2bdc640 1 -- 172.21.15.62:0/3139614687 wait complete. 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.895+0000 7fadb2bdc640 1 Processor -- start 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.895+0000 7fadb2bdc640 1 -- start start 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.895+0000 7fadb2bdc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac19d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.895+0000 7fadb2bdc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fadac10dc50 con 0x7fadac07de50 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.895+0000 7fadb0978640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac19d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.895+0000 7fadb0978640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac19d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54988/0 (socket says 172.21.15.62:54988) 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.895+0000 7fadb0978640 1 -- 172.21.15.62:0/1299789109 learned_addr learned my addr 172.21.15.62:0/1299789109 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.895+0000 7fadb0978640 1 -- 172.21.15.62:0/1299789109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac19dd80 con 0x7fadac07de50 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.895+0000 7fadb0978640 1 --2- 172.21.15.62:0/1299789109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac19d840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fada0031000 tx=0x7fada00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:01.844 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.896+0000 7fada9ffb640 1 -- 172.21.15.62:0/1299789109 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada0042070 con 0x7fadac07de50 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.896+0000 7fadb2bdc640 1 -- 172.21.15.62:0/1299789109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac19e010 con 0x7fadac07de50 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.896+0000 7fada9ffb640 1 -- 172.21.15.62:0/1299789109 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fada0032be0 con 0x7fadac07de50 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.896+0000 7fada9ffb640 1 -- 172.21.15.62:0/1299789109 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada0037040 con 0x7fadac07de50 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.896+0000 7fadb2bdc640 1 -- 172.21.15.62:0/1299789109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac1a0d00 con 0x7fadac07de50 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.897+0000 7fada9ffb640 1 -- 172.21.15.62:0/1299789109 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fada0045070 con 0x7fadac07de50 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.897+0000 7fada9ffb640 1 --2- 172.21.15.62:0/1299789109 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fad9c038f70 0x7fad9c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.897+0000 7fada9ffb640 1 -- 172.21.15.62:0/1299789109 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fada006c230 con 0x7fadac07de50 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.897+0000 7fadabfff640 1 --2- 172.21.15.62:0/1299789109 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fad9c038f70 0x7fad9c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.897+0000 7fadb2bdc640 1 -- 172.21.15.62:0/1299789109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadac07c330 con 0x7fadac07de50 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.898+0000 7fadabfff640 1 --2- 172.21.15.62:0/1299789109 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fad9c038f70 0x7fad9c03b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fad980098c0 tx=0x7fad98006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:01.845 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:00.902+0000 7fada9ffb640 1 -- 172.21.15.62:0/1299789109 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fada004c030 con 0x7fadac07de50 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.109+0000 7fadb2bdc640 1 -- 172.21.15.62:0/1299789109 --> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fadac104100 con 0x7fad9c038f70 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.114+0000 7fada9ffb640 1 -- 172.21.15.62:0/1299789109 <== mgr.14118 v2:172.21.15.62:6800/3180930925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fadac104100 con 0x7fad9c038f70 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.116+0000 7fadb2bdc640 1 -- 172.21.15.62:0/1299789109 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fad9c038f70 msgr2=0x7fad9c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.116+0000 7fadb2bdc640 1 --2- 172.21.15.62:0/1299789109 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fad9c038f70 0x7fad9c03b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fad980098c0 tx=0x7fad98006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.116+0000 7fadb2bdc640 1 -- 172.21.15.62:0/1299789109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 msgr2=0x7fadac19d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.116+0000 7fadb2bdc640 1 --2- 172.21.15.62:0/1299789109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac19d840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fada0031000 tx=0x7fada00326e0 comp rx=0 tx=0).stop 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.116+0000 7fadb2bdc640 1 -- 172.21.15.62:0/1299789109 shutdown_connections 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.116+0000 7fadb2bdc640 1 --2- 172.21.15.62:0/1299789109 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fad9c038f70 0x7fad9c03b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.116+0000 7fadb2bdc640 1 --2- 172.21.15.62:0/1299789109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac07de50 0x7fadac19d840 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.116+0000 7fadb2bdc640 1 -- 172.21.15.62:0/1299789109 >> 172.21.15.62:0/1299789109 conn(0x7fadac102db0 msgr2=0x7fadac1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.116+0000 7fadb2bdc640 1 -- 172.21.15.62:0/1299789109 shutdown_connections 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:01.116+0000 7fadb2bdc640 1 -- 172.21.15.62:0/1299789109 wait complete. 2024-09-11T23:43:01.846 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mgr service... 2024-09-11T23:43:02.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:02 smithi062 ceph-mon[30958]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:43:02.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:02 smithi062 ceph-mon[30958]: Saving service mon spec with placement count:5 2024-09-11T23:43:02.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:02 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:43:03.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:03 smithi062 ceph-mon[30958]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:43:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:03 smithi062 ceph-mon[30958]: Saving service mgr spec with placement count:2 2024-09-11T23:43:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:03 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:43:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:03 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:43:03.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T23:43:03.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.235+0000 7f862dbec640 1 Processor -- start 2024-09-11T23:43:03.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.235+0000 7f862dbec640 1 -- start start 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.235+0000 7f862dbec640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f8628107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.235+0000 7f862dbec640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8628107ff0 con 0x7f8628107620 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.235+0000 7f86277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f8628107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.235+0000 7f86277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f8628107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47646/0 (socket says 172.21.15.62:47646) 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.236+0000 7f86277fe640 1 -- 172.21.15.62:0/75154719 learned_addr learned my addr 172.21.15.62:0/75154719 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.236+0000 7f86277fe640 1 -- 172.21.15.62:0/75154719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8628108820 con 0x7f8628107620 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.236+0000 7f86277fe640 1 --2- 172.21.15.62:0/75154719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f8628107a20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8618009e80 tx=0x7f861802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe77c866da3973a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.237+0000 7f86267fc640 1 -- 172.21.15.62:0/75154719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8618036070 con 0x7f8628107620 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.237+0000 7f86267fc640 1 -- 172.21.15.62:0/75154719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f86180318c0 con 0x7f8628107620 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.237+0000 7f86267fc640 1 -- 172.21.15.62:0/75154719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8618007ab0 con 0x7f8628107620 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.237+0000 7f862dbec640 1 -- 172.21.15.62:0/75154719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 msgr2=0x7f8628107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.237+0000 7f862dbec640 1 --2- 172.21.15.62:0/75154719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f8628107a20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8618009e80 tx=0x7f861802a7a0 comp rx=0 tx=0).stop 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.237+0000 7f862dbec640 1 -- 172.21.15.62:0/75154719 shutdown_connections 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.237+0000 7f862dbec640 1 --2- 172.21.15.62:0/75154719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f8628107a20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.237+0000 7f862dbec640 1 -- 172.21.15.62:0/75154719 >> 172.21.15.62:0/75154719 conn(0x7f8628102db0 msgr2=0x7f86281051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.237+0000 7f862dbec640 1 -- 172.21.15.62:0/75154719 shutdown_connections 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.237+0000 7f862dbec640 1 -- 172.21.15.62:0/75154719 wait complete. 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.238+0000 7f862dbec640 1 Processor -- start 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.238+0000 7f862dbec640 1 -- start start 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.238+0000 7f862dbec640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f862819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.238+0000 7f862dbec640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f862810bbd0 con 0x7f8628107620 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.238+0000 7f86277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f862819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.238+0000 7f86277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f862819b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47658/0 (socket says 172.21.15.62:47658) 2024-09-11T23:43:03.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.238+0000 7f86277fe640 1 -- 172.21.15.62:0/4005095526 learned_addr learned my addr 172.21.15.62:0/4005095526 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.239+0000 7f86277fe640 1 -- 172.21.15.62:0/4005095526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862819bc00 con 0x7f8628107620 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.239+0000 7f86277fe640 1 --2- 172.21.15.62:0/4005095526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f862819b6c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8618007ce0 tx=0x7f8618031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.239+0000 7f8624ff9640 1 -- 172.21.15.62:0/4005095526 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8618036050 con 0x7f8628107620 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.239+0000 7f862dbec640 1 -- 172.21.15.62:0/4005095526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862819be90 con 0x7f8628107620 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.239+0000 7f8624ff9640 1 -- 172.21.15.62:0/4005095526 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8618042070 con 0x7f8628107620 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.239+0000 7f8624ff9640 1 -- 172.21.15.62:0/4005095526 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f861803d8a0 con 0x7f8628107620 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.239+0000 7f862dbec640 1 -- 172.21.15.62:0/4005095526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862819eb80 con 0x7f8628107620 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.240+0000 7f8624ff9640 1 -- 172.21.15.62:0/4005095526 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f861803b610 con 0x7f8628107620 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.240+0000 7f862dbec640 1 -- 172.21.15.62:0/4005095526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862810bd50 con 0x7f8628107620 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.240+0000 7f8624ff9640 1 --2- 172.21.15.62:0/4005095526 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f8614038f70 0x7f861403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.240+0000 7f8624ff9640 1 -- 172.21.15.62:0/4005095526 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8618007e00 con 0x7f8628107620 2024-09-11T23:43:03.814 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.241+0000 7f8626ffd640 1 --2- 172.21.15.62:0/4005095526 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f8614038f70 0x7f861403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.241+0000 7f8626ffd640 1 --2- 172.21.15.62:0/4005095526 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f8614038f70 0x7f861403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f86100098c0 tx=0x7f8610006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.245+0000 7f8624ff9640 1 -- 172.21.15.62:0/4005095526 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f861806cc80 con 0x7f8628107620 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.435+0000 7f862dbec640 1 -- 172.21.15.62:0/4005095526 --> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8628104090 con 0x7f8614038f70 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.442+0000 7f8624ff9640 1 -- 172.21.15.62:0/4005095526 <== mgr.14118 v2:172.21.15.62:6800/3180930925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8628104090 con 0x7f8614038f70 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.444+0000 7f862dbec640 1 -- 172.21.15.62:0/4005095526 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f8614038f70 msgr2=0x7f861403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.444+0000 7f862dbec640 1 --2- 172.21.15.62:0/4005095526 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f8614038f70 0x7f861403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f86100098c0 tx=0x7f8610006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.444+0000 7f862dbec640 1 -- 172.21.15.62:0/4005095526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 msgr2=0x7f862819b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.444+0000 7f862dbec640 1 --2- 172.21.15.62:0/4005095526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f862819b6c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8618007ce0 tx=0x7f8618031ca0 comp rx=0 tx=0).stop 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.444+0000 7f862dbec640 1 -- 172.21.15.62:0/4005095526 shutdown_connections 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.444+0000 7f862dbec640 1 --2- 172.21.15.62:0/4005095526 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f8614038f70 0x7f861403b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.444+0000 7f862dbec640 1 --2- 172.21.15.62:0/4005095526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8628107620 0x7f862819b6c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.444+0000 7f862dbec640 1 -- 172.21.15.62:0/4005095526 >> 172.21.15.62:0/4005095526 conn(0x7f8628102db0 msgr2=0x7f8628103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:03.815 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.444+0000 7f862dbec640 1 -- 172.21.15.62:0/4005095526 shutdown_connections 2024-09-11T23:43:03.816 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:02.444+0000 7f862dbec640 1 -- 172.21.15.62:0/4005095526 wait complete. 2024-09-11T23:43:05.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:04 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:43:05.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:04 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/628810920' entity='client.admin' 2024-09-11T23:43:05.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.274+0000 7fba2ff37640 1 Processor -- start 2024-09-11T23:43:05.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.275+0000 7fba2ff37640 1 -- start start 2024-09-11T23:43:05.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.275+0000 7fba2ff37640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba28107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.275+0000 7fba2ff37640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba28107ff0 con 0x7fba28107620 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.275+0000 7fba2dcd3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba28107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.275+0000 7fba2dcd3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba28107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47660/0 (socket says 172.21.15.62:47660) 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.275+0000 7fba2dcd3640 1 -- 172.21.15.62:0/3751505593 learned_addr learned my addr 172.21.15.62:0/3751505593 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.275+0000 7fba2dcd3640 1 -- 172.21.15.62:0/3751505593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba28108820 con 0x7fba28107620 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.276+0000 7fba2dcd3640 1 --2- 172.21.15.62:0/3751505593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba28107a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fba24009820 tx=0x7fba2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ec9c4776607881 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.276+0000 7fba2ccd1640 1 -- 172.21.15.62:0/3751505593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba24037070 con 0x7fba28107620 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.276+0000 7fba2ccd1640 1 -- 172.21.15.62:0/3751505593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba24032920 con 0x7fba28107620 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.276+0000 7fba2ccd1640 1 -- 172.21.15.62:0/3751505593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba24030810 con 0x7fba28107620 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.277+0000 7fba2ff37640 1 -- 172.21.15.62:0/3751505593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 msgr2=0x7fba28107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:05.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.277+0000 7fba2ff37640 1 --2- 172.21.15.62:0/3751505593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba28107a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fba24009820 tx=0x7fba2402a270 comp rx=0 tx=0).stop 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.277+0000 7fba2ff37640 1 -- 172.21.15.62:0/3751505593 shutdown_connections 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.277+0000 7fba2ff37640 1 --2- 172.21.15.62:0/3751505593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba28107a20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.277+0000 7fba2ff37640 1 -- 172.21.15.62:0/3751505593 >> 172.21.15.62:0/3751505593 conn(0x7fba28102db0 msgr2=0x7fba281051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.277+0000 7fba2ff37640 1 -- 172.21.15.62:0/3751505593 shutdown_connections 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.277+0000 7fba2ff37640 1 -- 172.21.15.62:0/3751505593 wait complete. 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.277+0000 7fba2ff37640 1 Processor -- start 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba2ff37640 1 -- start start 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba2ff37640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba2807db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba2ff37640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba2810bbd0 con 0x7fba28107620 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba2dcd3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba2807db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba2dcd3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba2807db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47666/0 (socket says 172.21.15.62:47666) 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba2dcd3640 1 -- 172.21.15.62:0/628810920 learned_addr learned my addr 172.21.15.62:0/628810920 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba2dcd3640 1 -- 172.21.15.62:0/628810920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2807e080 con 0x7fba28107620 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba2dcd3640 1 --2- 172.21.15.62:0/628810920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba2807db40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba240097f0 tx=0x7fba240327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba1affd640 1 -- 172.21.15.62:0/628810920 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba24037040 con 0x7fba28107620 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba1affd640 1 -- 172.21.15.62:0/628810920 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba24030c40 con 0x7fba28107620 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.278+0000 7fba2ff37640 1 -- 172.21.15.62:0/628810920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2807c270 con 0x7fba28107620 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.279+0000 7fba1affd640 1 -- 172.21.15.62:0/628810920 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba2403a5b0 con 0x7fba28107620 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.279+0000 7fba2ff37640 1 -- 172.21.15.62:0/628810920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2807c7b0 con 0x7fba28107620 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.279+0000 7fba1affd640 1 -- 172.21.15.62:0/628810920 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fba2403b3f0 con 0x7fba28107620 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.279+0000 7fba2ff37640 1 -- 172.21.15.62:0/628810920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba2810bd50 con 0x7fba28107620 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.280+0000 7fba1affd640 1 --2- 172.21.15.62:0/628810920 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fba14038fc0 0x7fba1403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.280+0000 7fba1affd640 1 -- 172.21.15.62:0/628810920 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fba2406c370 con 0x7fba28107620 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.280+0000 7fba2d4d2640 1 --2- 172.21.15.62:0/628810920 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fba14038fc0 0x7fba1403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:05.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.280+0000 7fba2d4d2640 1 --2- 172.21.15.62:0/628810920 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fba14038fc0 0x7fba1403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0098c0 tx=0x7fba1c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.284+0000 7fba1affd640 1 -- 172.21.15.62:0/628810920 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba24044dc0 con 0x7fba28107620 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.481+0000 7fba2ff37640 1 -- 172.21.15.62:0/628810920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fba28107aa0 con 0x7fba28107620 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.484+0000 7fba1affd640 1 -- 172.21.15.62:0/628810920 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fba240313a0 con 0x7fba28107620 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.486+0000 7fba2ff37640 1 -- 172.21.15.62:0/628810920 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fba14038fc0 msgr2=0x7fba1403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.486+0000 7fba2ff37640 1 --2- 172.21.15.62:0/628810920 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fba14038fc0 0x7fba1403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0098c0 tx=0x7fba1c006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.486+0000 7fba2ff37640 1 -- 172.21.15.62:0/628810920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 msgr2=0x7fba2807db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.486+0000 7fba2ff37640 1 --2- 172.21.15.62:0/628810920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba2807db40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba240097f0 tx=0x7fba240327b0 comp rx=0 tx=0).stop 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.486+0000 7fba2ff37640 1 -- 172.21.15.62:0/628810920 shutdown_connections 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.486+0000 7fba2ff37640 1 --2- 172.21.15.62:0/628810920 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fba14038fc0 0x7fba1403b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.486+0000 7fba2ff37640 1 --2- 172.21.15.62:0/628810920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba28107620 0x7fba2807db40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.486+0000 7fba2ff37640 1 -- 172.21.15.62:0/628810920 >> 172.21.15.62:0/628810920 conn(0x7fba28102db0 msgr2=0x7fba28103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.486+0000 7fba2ff37640 1 -- 172.21.15.62:0/628810920 shutdown_connections 2024-09-11T23:43:05.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:04.486+0000 7fba2ff37640 1 -- 172.21.15.62:0/628810920 wait complete. 2024-09-11T23:43:06.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:06 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:43:06.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:06 smithi062 ceph-mon[30958]: from='mgr.14118 172.21.15.62:0/4049862298' entity='mgr.x' 2024-09-11T23:43:06.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:06 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1002147021' entity='client.admin' 2024-09-11T23:43:06.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.170+0000 7fb2a982e640 1 Processor -- start 2024-09-11T23:43:06.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.170+0000 7fb2a982e640 1 -- start start 2024-09-11T23:43:06.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.171+0000 7fb2a982e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a4106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.171+0000 7fb2a982e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a4106ee0 con 0x7fb2a4106510 2024-09-11T23:43:06.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.171+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a4106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.171+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a4106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47672/0 (socket says 172.21.15.62:47672) 2024-09-11T23:43:06.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.171+0000 7fb2a2ffd640 1 -- 172.21.15.62:0/405910930 learned_addr learned my addr 172.21.15.62:0/405910930 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:06.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.171+0000 7fb2a2ffd640 1 -- 172.21.15.62:0/405910930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a41076c0 con 0x7fb2a4106510 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.172+0000 7fb2a2ffd640 1 --2- 172.21.15.62:0/405910930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a4106910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb294009820 tx=0x7fb29402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74c5572abcda784 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.172+0000 7fb2a27fc640 1 -- 172.21.15.62:0/405910930 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb294037070 con 0x7fb2a4106510 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.172+0000 7fb2a27fc640 1 -- 172.21.15.62:0/405910930 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb294032920 con 0x7fb2a4106510 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.172+0000 7fb2a27fc640 1 -- 172.21.15.62:0/405910930 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb294030810 con 0x7fb2a4106510 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.173+0000 7fb2a982e640 1 -- 172.21.15.62:0/405910930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 msgr2=0x7fb2a4106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.173+0000 7fb2a982e640 1 --2- 172.21.15.62:0/405910930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a4106910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb294009820 tx=0x7fb29402a270 comp rx=0 tx=0).stop 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.173+0000 7fb2a982e640 1 -- 172.21.15.62:0/405910930 shutdown_connections 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.173+0000 7fb2a982e640 1 --2- 172.21.15.62:0/405910930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a4106910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.173+0000 7fb2a982e640 1 -- 172.21.15.62:0/405910930 >> 172.21.15.62:0/405910930 conn(0x7fb2a4101ca0 msgr2=0x7fb2a41040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.173+0000 7fb2a982e640 1 -- 172.21.15.62:0/405910930 shutdown_connections 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.173+0000 7fb2a982e640 1 -- 172.21.15.62:0/405910930 wait complete. 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.174+0000 7fb2a982e640 1 Processor -- start 2024-09-11T23:43:06.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.174+0000 7fb2a982e640 1 -- start start 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.174+0000 7fb2a982e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a41916f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.174+0000 7fb2a982e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a410aa70 con 0x7fb2a4106510 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.174+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a41916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.174+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a41916f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47674/0 (socket says 172.21.15.62:47674) 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.174+0000 7fb2a2ffd640 1 -- 172.21.15.62:0/1002147021 learned_addr learned my addr 172.21.15.62:0/1002147021 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.174+0000 7fb2a2ffd640 1 -- 172.21.15.62:0/1002147021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a4191c30 con 0x7fb2a4106510 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.174+0000 7fb2a2ffd640 1 --2- 172.21.15.62:0/1002147021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a41916f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2940097f0 tx=0x7fb2940327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.175+0000 7fb2a882c640 1 -- 172.21.15.62:0/1002147021 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb294037040 con 0x7fb2a4106510 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.175+0000 7fb2a982e640 1 -- 172.21.15.62:0/1002147021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a418fe20 con 0x7fb2a4106510 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.175+0000 7fb2a882c640 1 -- 172.21.15.62:0/1002147021 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb294030c40 con 0x7fb2a4106510 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.175+0000 7fb2a882c640 1 -- 172.21.15.62:0/1002147021 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb29403a5b0 con 0x7fb2a4106510 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.175+0000 7fb2a982e640 1 -- 172.21.15.62:0/1002147021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a4190360 con 0x7fb2a4106510 2024-09-11T23:43:06.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.176+0000 7fb2a882c640 1 -- 172.21.15.62:0/1002147021 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb29403a7f0 con 0x7fb2a4106510 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.176+0000 7fb2a982e640 1 -- 172.21.15.62:0/1002147021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a410abf0 con 0x7fb2a4106510 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.176+0000 7fb2a882c640 1 --2- 172.21.15.62:0/1002147021 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fb290038f70 0x7fb29003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.176+0000 7fb2a882c640 1 -- 172.21.15.62:0/1002147021 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb29406c3d0 con 0x7fb2a4106510 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.176+0000 7fb29bfff640 1 --2- 172.21.15.62:0/1002147021 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fb290038f70 0x7fb29003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.177+0000 7fb29bfff640 1 --2- 172.21.15.62:0/1002147021 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fb290038f70 0x7fb29003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0098c0 tx=0x7fb28c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.181+0000 7fb2a882c640 1 -- 172.21.15.62:0/1002147021 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb294030df0 con 0x7fb2a4106510 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.357+0000 7fb2a982e640 1 -- 172.21.15.62:0/1002147021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb2a4106990 con 0x7fb2a4106510 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.360+0000 7fb2a882c640 1 -- 172.21.15.62:0/1002147021 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb294031c00 con 0x7fb2a4106510 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.362+0000 7fb2a982e640 1 -- 172.21.15.62:0/1002147021 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fb290038f70 msgr2=0x7fb29003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.362+0000 7fb2a982e640 1 --2- 172.21.15.62:0/1002147021 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fb290038f70 0x7fb29003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0098c0 tx=0x7fb28c006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.362+0000 7fb2a982e640 1 -- 172.21.15.62:0/1002147021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 msgr2=0x7fb2a41916f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.949 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.362+0000 7fb2a982e640 1 --2- 172.21.15.62:0/1002147021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a41916f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2940097f0 tx=0x7fb2940327b0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.949 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.362+0000 7fb2a982e640 1 -- 172.21.15.62:0/1002147021 shutdown_connections 2024-09-11T23:43:06.949 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.362+0000 7fb2a982e640 1 --2- 172.21.15.62:0/1002147021 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fb290038f70 0x7fb29003b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.949 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.362+0000 7fb2a982e640 1 --2- 172.21.15.62:0/1002147021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a4106510 0x7fb2a41916f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.949 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.362+0000 7fb2a982e640 1 -- 172.21.15.62:0/1002147021 >> 172.21.15.62:0/1002147021 conn(0x7fb2a4101ca0 msgr2=0x7fb2a4102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:06.949 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.362+0000 7fb2a982e640 1 -- 172.21.15.62:0/1002147021 shutdown_connections 2024-09-11T23:43:06.949 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:06.363+0000 7fb2a982e640 1 -- 172.21.15.62:0/1002147021 wait complete. 2024-09-11T23:43:06.949 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling the dashboard module... 2024-09-11T23:43:07.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:07 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/601328150' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T23:43:07.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: ignoring --setuser ceph since I am not root 2024-09-11T23:43:07.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: ignoring --setgroup ceph since I am not root 2024-09-11T23:43:07.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:07.816+0000 7f9fc5c5c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:43:07.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:07.875+0000 7f9fc5c5c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:43:08.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:08.033+0000 7f9fc5c5c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:43:08.216 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:08.101+0000 7f9fc5c5c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:43:08.656 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:08.255+0000 7f9fc5c5c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:43:08.656 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:08.403+0000 7f9fc5c5c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:43:08.663 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.356+0000 7fedafca9640 1 Processor -- start 2024-09-11T23:43:08.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.356+0000 7fedafca9640 1 -- start start 2024-09-11T23:43:08.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.356+0000 7fedafca9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda8107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:08.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.356+0000 7fedafca9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feda8107ff0 con 0x7feda8107620 2024-09-11T23:43:08.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.356+0000 7fedada45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda8107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:08.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.357+0000 7fedada45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda8107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47682/0 (socket says 172.21.15.62:47682) 2024-09-11T23:43:08.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.357+0000 7fedada45640 1 -- 172.21.15.62:0/3576748375 learned_addr learned my addr 172.21.15.62:0/3576748375 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:08.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.357+0000 7fedada45640 1 -- 172.21.15.62:0/3576748375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda8108820 con 0x7feda8107620 2024-09-11T23:43:08.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.358+0000 7fedada45640 1 --2- 172.21.15.62:0/3576748375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda8107a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feda4009820 tx=0x7feda402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cf1faa7f1726b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:08.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.358+0000 7fedaca43640 1 -- 172.21.15.62:0/3576748375 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feda4037070 con 0x7feda8107620 2024-09-11T23:43:08.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.358+0000 7fedaca43640 1 -- 172.21.15.62:0/3576748375 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda4032920 con 0x7feda8107620 2024-09-11T23:43:08.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.358+0000 7fedaca43640 1 -- 172.21.15.62:0/3576748375 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feda4030810 con 0x7feda8107620 2024-09-11T23:43:08.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.358+0000 7fedafca9640 1 -- 172.21.15.62:0/3576748375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 msgr2=0x7feda8107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:08.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.358+0000 7fedafca9640 1 --2- 172.21.15.62:0/3576748375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda8107a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feda4009820 tx=0x7feda402a270 comp rx=0 tx=0).stop 2024-09-11T23:43:08.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.358+0000 7fedafca9640 1 -- 172.21.15.62:0/3576748375 shutdown_connections 2024-09-11T23:43:08.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.358+0000 7fedafca9640 1 --2- 172.21.15.62:0/3576748375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda8107a20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.358+0000 7fedafca9640 1 -- 172.21.15.62:0/3576748375 >> 172.21.15.62:0/3576748375 conn(0x7feda8102db0 msgr2=0x7feda81051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:08.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.358+0000 7fedafca9640 1 -- 172.21.15.62:0/3576748375 shutdown_connections 2024-09-11T23:43:08.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.359+0000 7fedafca9640 1 -- 172.21.15.62:0/3576748375 wait complete. 2024-09-11T23:43:08.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.359+0000 7fedafca9640 1 Processor -- start 2024-09-11T23:43:08.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.359+0000 7fedafca9640 1 -- start start 2024-09-11T23:43:08.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.359+0000 7fedafca9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda819d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:08.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.359+0000 7fedafca9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feda810bbd0 con 0x7feda8107620 2024-09-11T23:43:08.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.359+0000 7fedada45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda819d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:08.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.359+0000 7fedada45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda819d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47688/0 (socket says 172.21.15.62:47688) 2024-09-11T23:43:08.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.359+0000 7fedada45640 1 -- 172.21.15.62:0/601328150 learned_addr learned my addr 172.21.15.62:0/601328150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:08.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.360+0000 7fedada45640 1 -- 172.21.15.62:0/601328150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda819de00 con 0x7feda8107620 2024-09-11T23:43:08.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.360+0000 7fedada45640 1 --2- 172.21.15.62:0/601328150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda819d8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feda4030560 tx=0x7feda4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:08.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.360+0000 7fed9effd640 1 -- 172.21.15.62:0/601328150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feda4037040 con 0x7feda8107620 2024-09-11T23:43:08.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.360+0000 7fed9effd640 1 -- 172.21.15.62:0/601328150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda4040070 con 0x7feda8107620 2024-09-11T23:43:08.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.360+0000 7fedafca9640 1 -- 172.21.15.62:0/601328150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda819e090 con 0x7feda8107620 2024-09-11T23:43:08.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.360+0000 7fed9effd640 1 -- 172.21.15.62:0/601328150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feda403a6d0 con 0x7feda8107620 2024-09-11T23:43:08.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.360+0000 7fedafca9640 1 -- 172.21.15.62:0/601328150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda81a0d80 con 0x7feda8107620 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.361+0000 7fed9effd640 1 -- 172.21.15.62:0/601328150 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7feda403bb20 con 0x7feda8107620 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.361+0000 7fedafca9640 1 -- 172.21.15.62:0/601328150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda810bd50 con 0x7feda8107620 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.361+0000 7fed9effd640 1 --2- 172.21.15.62:0/601328150 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fed94038fc0 0x7fed9403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.361+0000 7fed9effd640 1 -- 172.21.15.62:0/601328150 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feda406c4c0 con 0x7feda8107620 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.361+0000 7fedad244640 1 --2- 172.21.15.62:0/601328150 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fed94038fc0 0x7fed9403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.362+0000 7fedad244640 1 --2- 172.21.15.62:0/601328150 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fed94038fc0 0x7fed9403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed980098c0 tx=0x7fed98006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.366+0000 7fed9effd640 1 -- 172.21.15.62:0/601328150 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feda40328f0 con 0x7feda8107620 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.610+0000 7fedafca9640 1 -- 172.21.15.62:0/601328150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7feda81a0f50 con 0x7feda8107620 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.661+0000 7fed9effd640 1 -- 172.21.15.62:0/601328150 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7feda404a200 con 0x7feda8107620 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.663+0000 7fed9effd640 1 -- 172.21.15.62:0/601328150 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7feda403a870 con 0x7feda8107620 2024-09-11T23:43:08.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.664+0000 7fedafca9640 1 -- 172.21.15.62:0/601328150 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fed94038fc0 msgr2=0x7fed9403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:08.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.664+0000 7fedafca9640 1 --2- 172.21.15.62:0/601328150 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fed94038fc0 0x7fed9403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed980098c0 tx=0x7fed98006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.664+0000 7fedafca9640 1 -- 172.21.15.62:0/601328150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 msgr2=0x7feda819d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:08.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.664+0000 7fedafca9640 1 --2- 172.21.15.62:0/601328150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda819d8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feda4030560 tx=0x7feda4032670 comp rx=0 tx=0).stop 2024-09-11T23:43:08.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.664+0000 7fedafca9640 1 -- 172.21.15.62:0/601328150 shutdown_connections 2024-09-11T23:43:08.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.664+0000 7fedafca9640 1 --2- 172.21.15.62:0/601328150 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fed94038fc0 0x7fed9403b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.664+0000 7fedafca9640 1 --2- 172.21.15.62:0/601328150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda8107620 0x7feda819d8c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.664+0000 7fedafca9640 1 -- 172.21.15.62:0/601328150 >> 172.21.15.62:0/601328150 conn(0x7feda8102db0 msgr2=0x7feda8103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:08.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.664+0000 7fedafca9640 1 -- 172.21.15.62:0/601328150 shutdown_connections 2024-09-11T23:43:08.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:07.664+0000 7fedafca9640 1 -- 172.21.15.62:0/601328150 wait complete. 2024-09-11T23:43:08.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:08 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/601328150' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T23:43:08.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:08 smithi062 ceph-mon[30958]: mgrmap e9: x(active, since 19s) 2024-09-11T23:43:08.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:08.733+0000 7f9fc5c5c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:43:08.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:08.797+0000 7f9fc5c5c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:43:09.303 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:08.991+0000 7f9fc5c5c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:43:09.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:09 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1684669430' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T23:43:09.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:09 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:09.928+0000 7f9fc5c5c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:43:10.315 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:10 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:09.999+0000 7f9fc5c5c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:43:10.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:43:10.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T23:43:10.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:43:10.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T23:43:10.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T23:43:10.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:43:10.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.500+0000 7fc45da4b640 1 Processor -- start 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.500+0000 7fc45da4b640 1 -- start start 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.500+0000 7fc45da4b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45810a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.500+0000 7fc45da4b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc45807be80 con 0x7fc458108040 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.501+0000 7fc456ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45810a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.501+0000 7fc456ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45810a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47702/0 (socket says 172.21.15.62:47702) 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.501+0000 7fc456ffd640 1 -- 172.21.15.62:0/418695618 learned_addr learned my addr 172.21.15.62:0/418695618 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.501+0000 7fc456ffd640 1 -- 172.21.15.62:0/418695618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45810a970 con 0x7fc458108040 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.502+0000 7fc456ffd640 1 --2- 172.21.15.62:0/418695618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45810a430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc448009820 tx=0x7fc44802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f0463c776808c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.502+0000 7fc4567fc640 1 -- 172.21.15.62:0/418695618 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc448037070 con 0x7fc458108040 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.502+0000 7fc4567fc640 1 -- 172.21.15.62:0/418695618 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc448032920 con 0x7fc458108040 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.502+0000 7fc4567fc640 1 -- 172.21.15.62:0/418695618 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc448030810 con 0x7fc458108040 2024-09-11T23:43:10.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.503+0000 7fc45da4b640 1 -- 172.21.15.62:0/418695618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 msgr2=0x7fc45810a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.503+0000 7fc45da4b640 1 --2- 172.21.15.62:0/418695618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45810a430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc448009820 tx=0x7fc44802a270 comp rx=0 tx=0).stop 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.503+0000 7fc45da4b640 1 -- 172.21.15.62:0/418695618 shutdown_connections 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.503+0000 7fc45da4b640 1 --2- 172.21.15.62:0/418695618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45810a430 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.503+0000 7fc45da4b640 1 -- 172.21.15.62:0/418695618 >> 172.21.15.62:0/418695618 conn(0x7fc458101d00 msgr2=0x7fc458104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.503+0000 7fc45da4b640 1 -- 172.21.15.62:0/418695618 shutdown_connections 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.503+0000 7fc45da4b640 1 -- 172.21.15.62:0/418695618 wait complete. 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.503+0000 7fc45da4b640 1 Processor -- start 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.503+0000 7fc45da4b640 1 -- start start 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.504+0000 7fc45da4b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.504+0000 7fc45da4b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc45810ba50 con 0x7fc458108040 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.504+0000 7fc456ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.504+0000 7fc456ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45819b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47704/0 (socket says 172.21.15.62:47704) 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.504+0000 7fc456ffd640 1 -- 172.21.15.62:0/1684669430 learned_addr learned my addr 172.21.15.62:0/1684669430 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.504+0000 7fc456ffd640 1 -- 172.21.15.62:0/1684669430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45819bb30 con 0x7fc458108040 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.504+0000 7fc456ffd640 1 --2- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45819b5f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4480097f0 tx=0x7fc4480327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.504+0000 7fc45ca49640 1 -- 172.21.15.62:0/1684669430 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc448037040 con 0x7fc458108040 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.504+0000 7fc45ca49640 1 -- 172.21.15.62:0/1684669430 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc448030cc0 con 0x7fc458108040 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.504+0000 7fc45da4b640 1 -- 172.21.15.62:0/1684669430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45819bdc0 con 0x7fc458108040 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.505+0000 7fc45ca49640 1 -- 172.21.15.62:0/1684669430 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc44803a6b0 con 0x7fc458108040 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.505+0000 7fc45da4b640 1 -- 172.21.15.62:0/1684669430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45819eab0 con 0x7fc458108040 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.505+0000 7fc45ca49640 1 -- 172.21.15.62:0/1684669430 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fc44803a8a0 con 0x7fc458108040 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.506+0000 7fc45ca49640 1 --2- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc444038fc0 0x7fc44403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.506+0000 7fc45ca49640 1 -- 172.21.15.62:0/1684669430 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc44806c510 con 0x7fc458108040 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.506+0000 7fc44ffff640 1 -- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc444038fc0 msgr2=0x7fc44403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3180930925 2024-09-11T23:43:10.326 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.506+0000 7fc44ffff640 1 --2- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc444038fc0 0x7fc44403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.506+0000 7fc45da4b640 1 -- 172.21.15.62:0/1684669430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4581084a0 con 0x7fc458108040 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.511+0000 7fc45ca49640 1 -- 172.21.15.62:0/1684669430 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc44802fd40 con 0x7fc458108040 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.707+0000 7fc44ffff640 1 -- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc444038fc0 msgr2=0x7fc44403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3180930925 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.707+0000 7fc44ffff640 1 --2- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc444038fc0 0x7fc44403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.735+0000 7fc45da4b640 1 -- 172.21.15.62:0/1684669430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc45819ef90 con 0x7fc458108040 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.736+0000 7fc45ca49640 1 -- 172.21.15.62:0/1684669430 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fc4480448d0 con 0x7fc458108040 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.738+0000 7fc45da4b640 1 -- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc444038fc0 msgr2=0x7fc44403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.738+0000 7fc45da4b640 1 --2- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc444038fc0 0x7fc44403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.738+0000 7fc45da4b640 1 -- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 msgr2=0x7fc45819b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.738+0000 7fc45da4b640 1 --2- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45819b5f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4480097f0 tx=0x7fc4480327b0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.738+0000 7fc45da4b640 1 -- 172.21.15.62:0/1684669430 shutdown_connections 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.738+0000 7fc45da4b640 1 --2- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7fc444038fc0 0x7fc44403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.738+0000 7fc45da4b640 1 --2- 172.21.15.62:0/1684669430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc458108040 0x7fc45819b5f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.328 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.738+0000 7fc45da4b640 1 -- 172.21.15.62:0/1684669430 >> 172.21.15.62:0/1684669430 conn(0x7fc458101d00 msgr2=0x7fc4581028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:10.328 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.738+0000 7fc45da4b640 1 -- 172.21.15.62:0/1684669430 shutdown_connections 2024-09-11T23:43:10.328 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:09.738+0000 7fc45da4b640 1 -- 172.21.15.62:0/1684669430 wait complete. 2024-09-11T23:43:10.328 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2024-09-11T23:43:10.328 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 9... 2024-09-11T23:43:10.939 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:10 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:10.760+0000 7f9fc5c5c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:43:10.939 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:10 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:10.880+0000 7f9fc5c5c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:43:11.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.029+0000 7f9fc5c5c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:43:11.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.092+0000 7f9fc5c5c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:43:11.547 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.248+0000 7f9fc5c5c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:43:11.547 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.319+0000 7f9fc5c5c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:43:11.547 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.381+0000 7f9fc5c5c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:43:11.547 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.547+0000 7f9fc5c5c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:43:11.831 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.615+0000 7f9fc5c5c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:43:11.832 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.684+0000 7f9fc5c5c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:43:11.832 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.770+0000 7f9fc5c5c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:43:11.832 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.831+0000 7f9fc5c5c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:43:12.141 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:11.893+0000 7f9fc5c5c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:43:12.972 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:12.717+0000 7f9fc5c5c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:43:12.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: Active manager daemon x restarted 2024-09-11T23:43:12.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: Activating manager daemon x 2024-09-11T23:43:12.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T23:43:12.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: mgrmap e10: x(active, starting, since 0.00346859s) 2024-09-11T23:43:12.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:43:12.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:43:12.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:43:12.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:43:12.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:43:12.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: Manager daemon x is now available 2024-09-11T23:43:12.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:43:12.974 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:43:12.974 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:43:14.742 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:43:14.742 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T23:43:14.742 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T23:43:14.742 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:43:14.742 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.732+0000 7f78e98c6640 1 Processor -- start 2024-09-11T23:43:14.742 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.732+0000 7f78e98c6640 1 -- start start 2024-09-11T23:43:14.742 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.732+0000 7f78e98c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e4107ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.742 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.732+0000 7f78e98c6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4108090 con 0x7f78e41076c0 2024-09-11T23:43:14.742 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.732+0000 7f78e2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e4107ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.732+0000 7f78e2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e4107ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47716/0 (socket says 172.21.15.62:47716) 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.732+0000 7f78e2ffd640 1 -- 172.21.15.62:0/3506144822 learned_addr learned my addr 172.21.15.62:0/3506144822 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.733+0000 7f78e2ffd640 1 -- 172.21.15.62:0/3506144822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e4108820 con 0x7f78e41076c0 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.733+0000 7f78e2ffd640 1 --2- 172.21.15.62:0/3506144822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e4107ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009820 tx=0x7f78d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64423b05df5a8f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.734+0000 7f78e27fc640 1 -- 172.21.15.62:0/3506144822 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78d4037070 con 0x7f78e41076c0 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.734+0000 7f78e27fc640 1 -- 172.21.15.62:0/3506144822 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d4032920 con 0x7f78e41076c0 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.734+0000 7f78e27fc640 1 -- 172.21.15.62:0/3506144822 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78d4030810 con 0x7f78e41076c0 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.735+0000 7f78e98c6640 1 -- 172.21.15.62:0/3506144822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 msgr2=0x7f78e4107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.735+0000 7f78e98c6640 1 --2- 172.21.15.62:0/3506144822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e4107ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009820 tx=0x7f78d402a270 comp rx=0 tx=0).stop 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.735+0000 7f78e98c6640 1 -- 172.21.15.62:0/3506144822 shutdown_connections 2024-09-11T23:43:14.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.735+0000 7f78e98c6640 1 --2- 172.21.15.62:0/3506144822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e4107ac0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.735+0000 7f78e98c6640 1 -- 172.21.15.62:0/3506144822 >> 172.21.15.62:0/3506144822 conn(0x7f78e4102e50 msgr2=0x7f78e4105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:14.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.735+0000 7f78e98c6640 1 -- 172.21.15.62:0/3506144822 shutdown_connections 2024-09-11T23:43:14.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.735+0000 7f78e98c6640 1 -- 172.21.15.62:0/3506144822 wait complete. 2024-09-11T23:43:14.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.735+0000 7f78e98c6640 1 Processor -- start 2024-09-11T23:43:14.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.735+0000 7f78e98c6640 1 -- start start 2024-09-11T23:43:14.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e98c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e98c6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f78e410bbd0 con 0x7f78e41076c0 2024-09-11T23:43:14.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e419d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47728/0 (socket says 172.21.15.62:47728) 2024-09-11T23:43:14.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e2ffd640 1 -- 172.21.15.62:0/4130646537 learned_addr learned my addr 172.21.15.62:0/4130646537 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e2ffd640 1 -- 172.21.15.62:0/4130646537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e419dd90 con 0x7f78e41076c0 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e2ffd640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e419d850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78d40097f0 tx=0x7f78d40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78d4037040 con 0x7f78e41076c0 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e98c6640 1 -- 172.21.15.62:0/4130646537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e419e020 con 0x7f78e41076c0 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d4030cc0 con 0x7f78e41076c0 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.736+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78d403a640 con 0x7f78e41076c0 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.737+0000 7f78e98c6640 1 -- 172.21.15.62:0/4130646537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e41a0d10 con 0x7f78e41076c0 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.737+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f78d403a940 con 0x7f78e41076c0 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.738+0000 7f78e88c4640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 0x7f78d003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.738+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 --> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f78d003bb90 con 0x7f78d0038fc0 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.738+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f78d406c590 con 0x7f78e41076c0 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.738+0000 7f78dbfff640 1 -- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 msgr2=0x7f78d003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3180930925 2024-09-11T23:43:14.745 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.738+0000 7f78dbfff640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 0x7f78d003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.939+0000 7f78dbfff640 1 -- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 msgr2=0x7f78d003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3180930925 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:10.939+0000 7f78dbfff640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 0x7f78d003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:11.340+0000 7f78dbfff640 1 -- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 msgr2=0x7f78d003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3180930925 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:11.340+0000 7f78dbfff640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 0x7f78d003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:12.141+0000 7f78dbfff640 1 -- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 msgr2=0x7f78d003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3180930925 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:12.141+0000 7f78dbfff640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 0x7f78d003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:12.721+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f78d406b040 con 0x7f78e41076c0 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:12.721+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 msgr2=0x7f78d003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:12.721+0000 7f78e88c4640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 0x7f78d003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.722+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f78d4044920 con 0x7f78e41076c0 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.722+0000 7f78e88c4640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f78d003ca80 0x7f78d003ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.723+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f78d402dec0 con 0x7f78d003ca80 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.723+0000 7f78dbfff640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f78d003ca80 0x7f78d003ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.723+0000 7f78dbfff640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f78d003ca80 0x7f78d003ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f78cc003980 tx=0x7f78cc006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.724+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f78d402dec0 con 0x7f78d003ca80 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.731+0000 7f78e98c6640 1 -- 172.21.15.62:0/4130646537 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f78e410bd50 con 0x7f78d003ca80 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.731+0000 7f78e88c4640 1 -- 172.21.15.62:0/4130646537 <== mgr.14150 v2:172.21.15.62:6800/3106394779 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f78e410bd50 con 0x7f78d003ca80 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 -- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f78d003ca80 msgr2=0x7f78d003ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f78d003ca80 0x7f78d003ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f78cc003980 tx=0x7f78cc006e10 comp rx=0 tx=0).stop 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 -- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 msgr2=0x7f78e419d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e419d850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78d40097f0 tx=0x7f78d40327b0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 -- 172.21.15.62:0/4130646537 shutdown_connections 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f78d003ca80 0x7f78d003ee70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:6800/3180930925,v1:172.21.15.62:6801/3180930925] conn(0x7f78d0038fc0 0x7f78d003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 --2- 172.21.15.62:0/4130646537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e41076c0 0x7f78e419d850 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 -- 172.21.15.62:0/4130646537 >> 172.21.15.62:0/4130646537 conn(0x7f78e4102e50 msgr2=0x7f78e41039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:14.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 -- 172.21.15.62:0/4130646537 shutdown_connections 2024-09-11T23:43:14.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:13.732+0000 7f78e98c6640 1 -- 172.21.15.62:0/4130646537 wait complete. 2024-09-11T23:43:14.748 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 9 is available 2024-09-11T23:43:14.748 INFO:teuthology.orchestra.run.smithi062.stdout:Generating a dashboard self-signed certificate... 2024-09-11T23:43:14.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:14 smithi062 ceph-mon[30958]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T23:43:14.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:14 smithi062 ceph-mon[30958]: mgrmap e11: x(active, since 1.00576s) 2024-09-11T23:43:14.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:14 smithi062 ceph-mon[30958]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T23:43:14.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:15.281 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:15.100+0000 7f9f2b725640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:43:15] ENGINE Error in HTTPServer.serve 2024-09-11T23:43:15.281 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: Traceback (most recent call last): 2024-09-11T23:43:15.281 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:43:15.281 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self._connections.run(self.expiration_interval) 2024-09-11T23:43:15.281 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:43:15.281 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self._run(expiration_interval) 2024-09-11T23:43:15.282 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:43:15.282 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:43:15.282 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:43:15.282 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:43:15.282 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:43:15.282 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: s = self.context.wrap_socket( 2024-09-11T23:43:15.282 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:43:15.282 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: return self.sslsocket_class._create( 2024-09-11T23:43:15.282 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:43:15.282 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self.do_handshake() 2024-09-11T23:43:15.283 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:43:15.283 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self._sslobj.do_handshake() 2024-09-11T23:43:15.283 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:43:15.283 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:43:15 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:43:15.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T23:43:15.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.171+0000 7fe5ce206640 1 Processor -- start 2024-09-11T23:43:15.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.171+0000 7fe5ce206640 1 -- start start 2024-09-11T23:43:15.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.171+0000 7fe5ce206640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c810a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.171+0000 7fe5ce206640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c807bdb0 con 0x7fe5c8108100 2024-09-11T23:43:15.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.171+0000 7fe5c77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c810a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.171+0000 7fe5c77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c810a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51212/0 (socket says 172.21.15.62:51212) 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.171+0000 7fe5c77fe640 1 -- 172.21.15.62:0/2319638679 learned_addr learned my addr 172.21.15.62:0/2319638679 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.172+0000 7fe5c77fe640 1 -- 172.21.15.62:0/2319638679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c810aa30 con 0x7fe5c8108100 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.172+0000 7fe5c77fe640 1 --2- 172.21.15.62:0/2319638679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c810a4f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009820 tx=0x7fe5b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaac86fcdf30546 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.172+0000 7fe5c67fc640 1 -- 172.21.15.62:0/2319638679 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5b8037070 con 0x7fe5c8108100 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.172+0000 7fe5c67fc640 1 -- 172.21.15.62:0/2319638679 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b8032920 con 0x7fe5c8108100 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.172+0000 7fe5c67fc640 1 -- 172.21.15.62:0/2319638679 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5b8030810 con 0x7fe5c8108100 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.173+0000 7fe5ce206640 1 -- 172.21.15.62:0/2319638679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 msgr2=0x7fe5c810a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.173+0000 7fe5ce206640 1 --2- 172.21.15.62:0/2319638679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c810a4f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009820 tx=0x7fe5b802a270 comp rx=0 tx=0).stop 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.173+0000 7fe5ce206640 1 -- 172.21.15.62:0/2319638679 shutdown_connections 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.173+0000 7fe5ce206640 1 --2- 172.21.15.62:0/2319638679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c810a4f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.173+0000 7fe5ce206640 1 -- 172.21.15.62:0/2319638679 >> 172.21.15.62:0/2319638679 conn(0x7fe5c8101ca0 msgr2=0x7fe5c81040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.173+0000 7fe5ce206640 1 -- 172.21.15.62:0/2319638679 shutdown_connections 2024-09-11T23:43:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.173+0000 7fe5ce206640 1 -- 172.21.15.62:0/2319638679 wait complete. 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.174+0000 7fe5ce206640 1 Processor -- start 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.174+0000 7fe5ce206640 1 -- start start 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.174+0000 7fe5ce206640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c819b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.174+0000 7fe5ce206640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c810b9f0 con 0x7fe5c8108100 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.174+0000 7fe5c77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c819b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.174+0000 7fe5c77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c819b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51218/0 (socket says 172.21.15.62:51218) 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.174+0000 7fe5c77fe640 1 -- 172.21.15.62:0/2480430143 learned_addr learned my addr 172.21.15.62:0/2480430143 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.174+0000 7fe5c77fe640 1 -- 172.21.15.62:0/2480430143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c819bb90 con 0x7fe5c8108100 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.175+0000 7fe5c77fe640 1 --2- 172.21.15.62:0/2480430143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c819b650 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8030420 tx=0x7fe5b8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.175+0000 7fe5c4ff9640 1 -- 172.21.15.62:0/2480430143 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5b8037040 con 0x7fe5c8108100 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.175+0000 7fe5ce206640 1 -- 172.21.15.62:0/2480430143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c819be20 con 0x7fe5c8108100 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.175+0000 7fe5c4ff9640 1 -- 172.21.15.62:0/2480430143 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b8040070 con 0x7fe5c8108100 2024-09-11T23:43:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.175+0000 7fe5c4ff9640 1 -- 172.21.15.62:0/2480430143 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5b803a6e0 con 0x7fe5c8108100 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.175+0000 7fe5ce206640 1 -- 172.21.15.62:0/2480430143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c819eb10 con 0x7fe5c8108100 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.176+0000 7fe5c4ff9640 1 -- 172.21.15.62:0/2480430143 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fe5b803a9e0 con 0x7fe5c8108100 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.176+0000 7fe5c4ff9640 1 --2- 172.21.15.62:0/2480430143 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe5b0038eb0 0x7fe5b003b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.176+0000 7fe5c4ff9640 1 -- 172.21.15.62:0/2480430143 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe5b806c640 con 0x7fe5c8108100 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.176+0000 7fe5ce206640 1 -- 172.21.15.62:0/2480430143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c8108560 con 0x7fe5c8108100 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.176+0000 7fe5c6ffd640 1 --2- 172.21.15.62:0/2480430143 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe5b0038eb0 0x7fe5b003b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.176+0000 7fe5c6ffd640 1 --2- 172.21.15.62:0/2480430143 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe5b0038eb0 0x7fe5b003b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0098c0 tx=0x7fe5bc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.181+0000 7fe5c4ff9640 1 -- 172.21.15.62:0/2480430143 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5b8043bf0 con 0x7fe5c8108100 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.280+0000 7fe5c4ff9640 1 -- 172.21.15.62:0/2480430143 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe5b8045980 con 0x7fe5c8108100 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.370+0000 7fe5ce206640 1 -- 172.21.15.62:0/2480430143 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fe5c8102f80 con 0x7fe5b0038eb0 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.414+0000 7fe5c4ff9640 1 -- 172.21.15.62:0/2480430143 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fe5c8102f80 con 0x7fe5b0038eb0 2024-09-11T23:43:16.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.417+0000 7fe5ce206640 1 -- 172.21.15.62:0/2480430143 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe5b0038eb0 msgr2=0x7fe5b003b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:16.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.417+0000 7fe5ce206640 1 --2- 172.21.15.62:0/2480430143 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe5b0038eb0 0x7fe5b003b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0098c0 tx=0x7fe5bc006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:16.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.417+0000 7fe5ce206640 1 -- 172.21.15.62:0/2480430143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 msgr2=0x7fe5c819b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:16.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.417+0000 7fe5ce206640 1 --2- 172.21.15.62:0/2480430143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c819b650 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8030420 tx=0x7fe5b8032670 comp rx=0 tx=0).stop 2024-09-11T23:43:16.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.417+0000 7fe5ce206640 1 -- 172.21.15.62:0/2480430143 shutdown_connections 2024-09-11T23:43:16.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.417+0000 7fe5ce206640 1 --2- 172.21.15.62:0/2480430143 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe5b0038eb0 0x7fe5b003b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:16.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.417+0000 7fe5ce206640 1 --2- 172.21.15.62:0/2480430143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c8108100 0x7fe5c819b650 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:16.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.417+0000 7fe5ce206640 1 -- 172.21.15.62:0/2480430143 >> 172.21.15.62:0/2480430143 conn(0x7fe5c8101ca0 msgr2=0x7fe5c8102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:16.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.417+0000 7fe5ce206640 1 -- 172.21.15.62:0/2480430143 shutdown_connections 2024-09-11T23:43:16.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:15.417+0000 7fe5ce206640 1 -- 172.21.15.62:0/2480430143 wait complete. 2024-09-11T23:43:16.001 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial admin user... 2024-09-11T23:43:16.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:16 smithi062 ceph-mon[30958]: [11/Sep/2024:23:43:14] ENGINE Bus STARTING 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:16 smithi062 ceph-mon[30958]: [11/Sep/2024:23:43:14] ENGINE Serving on http://172.21.15.62:8765 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:16 smithi062 ceph-mon[30958]: [11/Sep/2024:23:43:15] ENGINE Serving on https://172.21.15.62:7150 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:16 smithi062 ceph-mon[30958]: [11/Sep/2024:23:43:15] ENGINE Bus STARTED 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:16 smithi062 ceph-mon[30958]: [11/Sep/2024:23:43:15] ENGINE Error in HTTPServer.serve 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._run(expiration_interval) 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:43:16.426 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout: s = self.context.wrap_socket( 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.sslsocket_class._create( 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout: self.do_handshake() 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._sslobj.do_handshake() 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:16 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:16 smithi062 ceph-mon[30958]: mgrmap e12: x(active, since 2s) 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:16 smithi062 ceph-mon[30958]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:43:16.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:16 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:16.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:16 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:17.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:17 smithi062 ceph-mon[30958]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:43:17.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:17.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$rpKTKZHMso/rKNW4negqeOhsZHxHLf2YwsSr4zYyYqdxvNreVFH.C", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726098196, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T23:43:17.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.428+0000 7fcef7b2b640 1 Processor -- start 2024-09-11T23:43:17.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.428+0000 7fcef7b2b640 1 -- start start 2024-09-11T23:43:17.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.428+0000 7fcef7b2b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef010a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.429+0000 7fcef7b2b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcef007be30 con 0x7fcef0108080 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.429+0000 7fcef58c7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef010a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.429+0000 7fcef58c7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef010a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51226/0 (socket says 172.21.15.62:51226) 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.429+0000 7fcef58c7640 1 -- 172.21.15.62:0/2860799286 learned_addr learned my addr 172.21.15.62:0/2860799286 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.429+0000 7fcef58c7640 1 -- 172.21.15.62:0/2860799286 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef010a9b0 con 0x7fcef0108080 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.430+0000 7fcef58c7640 1 --2- 172.21.15.62:0/2860799286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef010a470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009e80 tx=0x7fcee002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346a7431a03ba3df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.430+0000 7fcef48c5640 1 -- 172.21.15.62:0/2860799286 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee0036070 con 0x7fcef0108080 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.430+0000 7fcef48c5640 1 -- 172.21.15.62:0/2860799286 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcee00318e0 con 0x7fcef0108080 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.430+0000 7fcef48c5640 1 -- 172.21.15.62:0/2860799286 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee0007a00 con 0x7fcef0108080 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.430+0000 7fcef7b2b640 1 -- 172.21.15.62:0/2860799286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 msgr2=0x7fcef010a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.430+0000 7fcef7b2b640 1 --2- 172.21.15.62:0/2860799286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef010a470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009e80 tx=0x7fcee002a7a0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.431+0000 7fcef7b2b640 1 -- 172.21.15.62:0/2860799286 shutdown_connections 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.431+0000 7fcef7b2b640 1 --2- 172.21.15.62:0/2860799286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef010a470 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.431+0000 7fcef7b2b640 1 -- 172.21.15.62:0/2860799286 >> 172.21.15.62:0/2860799286 conn(0x7fcef0101ca0 msgr2=0x7fcef01040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.431+0000 7fcef7b2b640 1 -- 172.21.15.62:0/2860799286 shutdown_connections 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.431+0000 7fcef7b2b640 1 -- 172.21.15.62:0/2860799286 wait complete. 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.431+0000 7fcef7b2b640 1 Processor -- start 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.431+0000 7fcef7b2b640 1 -- start start 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.431+0000 7fcef7b2b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.431+0000 7fcef7b2b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcef010ba30 con 0x7fcef0108080 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.432+0000 7fcef58c7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.432+0000 7fcef58c7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef019b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51228/0 (socket says 172.21.15.62:51228) 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.432+0000 7fcef58c7640 1 -- 172.21.15.62:0/703068377 learned_addr learned my addr 172.21.15.62:0/703068377 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.432+0000 7fcef58c7640 1 -- 172.21.15.62:0/703068377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef019bb40 con 0x7fcef0108080 2024-09-11T23:43:17.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.432+0000 7fcef58c7640 1 --2- 172.21.15.62:0/703068377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef019b600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcee002acc0 tx=0x7fcee0031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.432+0000 7fceeeffd640 1 -- 172.21.15.62:0/703068377 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee0036050 con 0x7fcef0108080 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.432+0000 7fcef7b2b640 1 -- 172.21.15.62:0/703068377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef019bdd0 con 0x7fcef0108080 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.432+0000 7fceeeffd640 1 -- 172.21.15.62:0/703068377 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcee0037040 con 0x7fcef0108080 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.432+0000 7fceeeffd640 1 -- 172.21.15.62:0/703068377 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee003d970 con 0x7fcef0108080 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.432+0000 7fcef7b2b640 1 -- 172.21.15.62:0/703068377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef019eac0 con 0x7fcef0108080 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.433+0000 7fceeeffd640 1 -- 172.21.15.62:0/703068377 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fcee0044020 con 0x7fcef0108080 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.433+0000 7fcef7b2b640 1 -- 172.21.15.62:0/703068377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef01084e0 con 0x7fcef0108080 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.433+0000 7fceeeffd640 1 --2- 172.21.15.62:0/703068377 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fced8038fc0 0x7fced803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.433+0000 7fceeeffd640 1 -- 172.21.15.62:0/703068377 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcee0042020 con 0x7fcef0108080 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.434+0000 7fcef50c6640 1 --2- 172.21.15.62:0/703068377 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fced8038fc0 0x7fced803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.434+0000 7fcef50c6640 1 --2- 172.21.15.62:0/703068377 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fced8038fc0 0x7fced803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcee40098c0 tx=0x7fcee4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:17.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.438+0000 7fceeeffd640 1 -- 172.21.15.62:0/703068377 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcee004b020 con 0x7fcef0108080 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.630+0000 7fcef7b2b640 1 -- 172.21.15.62:0/703068377 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fcef0102f90 con 0x7fced8038fc0 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.879+0000 7fceeeffd640 1 -- 172.21.15.62:0/703068377 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fcef0102f90 con 0x7fced8038fc0 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.881+0000 7fcef7b2b640 1 -- 172.21.15.62:0/703068377 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fced8038fc0 msgr2=0x7fced803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.881+0000 7fcef7b2b640 1 --2- 172.21.15.62:0/703068377 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fced8038fc0 0x7fced803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcee40098c0 tx=0x7fcee4006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.881+0000 7fcef7b2b640 1 -- 172.21.15.62:0/703068377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 msgr2=0x7fcef019b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.881+0000 7fcef7b2b640 1 --2- 172.21.15.62:0/703068377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef019b600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcee002acc0 tx=0x7fcee0031cc0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.882+0000 7fcef7b2b640 1 -- 172.21.15.62:0/703068377 shutdown_connections 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.882+0000 7fcef7b2b640 1 --2- 172.21.15.62:0/703068377 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fced8038fc0 0x7fced803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.882+0000 7fcef7b2b640 1 --2- 172.21.15.62:0/703068377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef0108080 0x7fcef019b600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.882+0000 7fcef7b2b640 1 -- 172.21.15.62:0/703068377 >> 172.21.15.62:0/703068377 conn(0x7fcef0101ca0 msgr2=0x7fcef0102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.882+0000 7fcef7b2b640 1 -- 172.21.15.62:0/703068377 shutdown_connections 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:16.882+0000 7fcef7b2b640 1 -- 172.21.15.62:0/703068377 wait complete. 2024-09-11T23:43:17.703 INFO:teuthology.orchestra.run.smithi062.stdout:Fetching dashboard port number... 2024-09-11T23:43:18.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:18 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2677752035' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T23:43:19.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T23:43:19.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.511+0000 7fd58acc0640 1 Processor -- start 2024-09-11T23:43:19.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.511+0000 7fd58acc0640 1 -- start start 2024-09-11T23:43:19.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.511+0000 7fd58acc0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58410a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:19.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.511+0000 7fd58acc0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd58407bdc0 con 0x7fd584108150 2024-09-11T23:43:19.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.511+0000 7fd588a5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58410a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:19.275 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.511+0000 7fd588a5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58410a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51240/0 (socket says 172.21.15.62:51240) 2024-09-11T23:43:19.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.511+0000 7fd588a5c640 1 -- 172.21.15.62:0/3883989675 learned_addr learned my addr 172.21.15.62:0/3883989675 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:19.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.512+0000 7fd588a5c640 1 -- 172.21.15.62:0/3883989675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58410aa80 con 0x7fd584108150 2024-09-11T23:43:19.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.512+0000 7fd588a5c640 1 --2- 172.21.15.62:0/3883989675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58410a540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd578009820 tx=0x7fd57802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec6ff1f9a143502 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:19.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.512+0000 7fd5837fe640 1 -- 172.21.15.62:0/3883989675 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd578037070 con 0x7fd584108150 2024-09-11T23:43:19.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.512+0000 7fd5837fe640 1 -- 172.21.15.62:0/3883989675 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd578032920 con 0x7fd584108150 2024-09-11T23:43:19.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.512+0000 7fd5837fe640 1 -- 172.21.15.62:0/3883989675 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd578030810 con 0x7fd584108150 2024-09-11T23:43:19.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.513+0000 7fd58acc0640 1 -- 172.21.15.62:0/3883989675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 msgr2=0x7fd58410a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:19.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.513+0000 7fd58acc0640 1 --2- 172.21.15.62:0/3883989675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58410a540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd578009820 tx=0x7fd57802a270 comp rx=0 tx=0).stop 2024-09-11T23:43:19.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.513+0000 7fd58acc0640 1 -- 172.21.15.62:0/3883989675 shutdown_connections 2024-09-11T23:43:19.276 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.513+0000 7fd58acc0640 1 --2- 172.21.15.62:0/3883989675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58410a540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.513+0000 7fd58acc0640 1 -- 172.21.15.62:0/3883989675 >> 172.21.15.62:0/3883989675 conn(0x7fd584101cb0 msgr2=0x7fd5841040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.513+0000 7fd58acc0640 1 -- 172.21.15.62:0/3883989675 shutdown_connections 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.513+0000 7fd58acc0640 1 -- 172.21.15.62:0/3883989675 wait complete. 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.514+0000 7fd58acc0640 1 Processor -- start 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.514+0000 7fd58acc0640 1 -- start start 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.514+0000 7fd58acc0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58419b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.514+0000 7fd58acc0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd58410baa0 con 0x7fd584108150 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.514+0000 7fd588a5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58419b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.514+0000 7fd588a5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58419b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51244/0 (socket says 172.21.15.62:51244) 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.514+0000 7fd588a5c640 1 -- 172.21.15.62:0/2677752035 learned_addr learned my addr 172.21.15.62:0/2677752035 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.514+0000 7fd588a5c640 1 -- 172.21.15.62:0/2677752035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58419bbb0 con 0x7fd584108150 2024-09-11T23:43:19.277 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.515+0000 7fd588a5c640 1 --2- 172.21.15.62:0/2677752035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58419b670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd578030420 tx=0x7fd578032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.515+0000 7fd581ffb640 1 -- 172.21.15.62:0/2677752035 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd578037040 con 0x7fd584108150 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.515+0000 7fd581ffb640 1 -- 172.21.15.62:0/2677752035 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd578040070 con 0x7fd584108150 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.515+0000 7fd581ffb640 1 -- 172.21.15.62:0/2677752035 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd57803a6e0 con 0x7fd584108150 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.515+0000 7fd58acc0640 1 -- 172.21.15.62:0/2677752035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58419be40 con 0x7fd584108150 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.515+0000 7fd58acc0640 1 -- 172.21.15.62:0/2677752035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58419eb30 con 0x7fd584108150 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.516+0000 7fd581ffb640 1 -- 172.21.15.62:0/2677752035 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fd57803bb20 con 0x7fd584108150 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.516+0000 7fd58acc0640 1 -- 172.21.15.62:0/2677752035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5841085b0 con 0x7fd584108150 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.516+0000 7fd581ffb640 1 --2- 172.21.15.62:0/2677752035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd574038fc0 0x7fd57403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.516+0000 7fd581ffb640 1 -- 172.21.15.62:0/2677752035 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd57806c730 con 0x7fd584108150 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.516+0000 7fd583fff640 1 --2- 172.21.15.62:0/2677752035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd574038fc0 0x7fd57403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:19.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.517+0000 7fd583fff640 1 --2- 172.21.15.62:0/2677752035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd574038fc0 0x7fd57403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098c0 tx=0x7fd570006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.521+0000 7fd581ffb640 1 -- 172.21.15.62:0/2677752035 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd57806ec10 con 0x7fd584108150 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.698+0000 7fd58acc0640 1 -- 172.21.15.62:0/2677752035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fd58419f120 con 0x7fd584108150 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.698+0000 7fd581ffb640 1 -- 172.21.15.62:0/2677752035 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fd57806edf0 con 0x7fd584108150 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.700+0000 7fd58acc0640 1 -- 172.21.15.62:0/2677752035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd574038fc0 msgr2=0x7fd57403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.700+0000 7fd58acc0640 1 --2- 172.21.15.62:0/2677752035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd574038fc0 0x7fd57403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098c0 tx=0x7fd570006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.700+0000 7fd58acc0640 1 -- 172.21.15.62:0/2677752035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 msgr2=0x7fd58419b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.700+0000 7fd58acc0640 1 --2- 172.21.15.62:0/2677752035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58419b670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd578030420 tx=0x7fd578032670 comp rx=0 tx=0).stop 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.701+0000 7fd58acc0640 1 -- 172.21.15.62:0/2677752035 shutdown_connections 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.701+0000 7fd58acc0640 1 --2- 172.21.15.62:0/2677752035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd574038fc0 0x7fd57403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.701+0000 7fd58acc0640 1 --2- 172.21.15.62:0/2677752035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd584108150 0x7fd58419b670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:19.279 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.701+0000 7fd58acc0640 1 -- 172.21.15.62:0/2677752035 >> 172.21.15.62:0/2677752035 conn(0x7fd584101cb0 msgr2=0x7fd5841028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:19.280 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.701+0000 7fd58acc0640 1 -- 172.21.15.62:0/2677752035 shutdown_connections 2024-09-11T23:43:19.280 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:18.701+0000 7fd58acc0640 1 -- 172.21.15.62:0/2677752035 wait complete. 2024-09-11T23:43:19.282 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:43:19.282 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-09-11T23:43:19.291 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:43:19.291 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-09-11T23:43:19.291 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-09-11T23:43:19.291 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T23:43:19.293 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph Dashboard is now available at: 2024-09-11T23:43:19.293 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:19.293 INFO:teuthology.orchestra.run.smithi062.stdout: URL: https://smithi062.front.sepia.ceph.com:8443/ 2024-09-11T23:43:19.293 INFO:teuthology.orchestra.run.smithi062.stdout: User: admin 2024-09-11T23:43:19.293 INFO:teuthology.orchestra.run.smithi062.stdout: Password: 6gloediaco 2024-09-11T23:43:19.293 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:19.293 INFO:teuthology.orchestra.run.smithi062.stdout:Saving cluster configuration to /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config directory 2024-09-11T23:43:19.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:19 smithi062 ceph-mon[30958]: mgrmap e13: x(active, since 6s) 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.680+0000 7f37ca446640 1 Processor -- start 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.680+0000 7f37ca446640 1 -- start start 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.680+0000 7f37ca446640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c4107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.680+0000 7f37ca446640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4107ff0 con 0x7f37c4107620 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.680+0000 7f37c3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c4107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.680+0000 7f37c3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c4107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51250/0 (socket says 172.21.15.62:51250) 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.680+0000 7f37c3fff640 1 -- 172.21.15.62:0/3873283691 learned_addr learned my addr 172.21.15.62:0/3873283691 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.681+0000 7f37c3fff640 1 -- 172.21.15.62:0/3873283691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c4108820 con 0x7f37c4107620 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.681+0000 7f37c3fff640 1 --2- 172.21.15.62:0/3873283691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c4107a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009820 tx=0x7f37b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c2b12956efc86b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.682+0000 7f37c37fe640 1 -- 172.21.15.62:0/3873283691 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37b4037070 con 0x7f37c4107620 2024-09-11T23:43:20.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.682+0000 7f37c37fe640 1 -- 172.21.15.62:0/3873283691 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b4032920 con 0x7f37c4107620 2024-09-11T23:43:20.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.682+0000 7f37c37fe640 1 -- 172.21.15.62:0/3873283691 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37b4030810 con 0x7f37c4107620 2024-09-11T23:43:20.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.682+0000 7f37ca446640 1 -- 172.21.15.62:0/3873283691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 msgr2=0x7f37c4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.682+0000 7f37ca446640 1 --2- 172.21.15.62:0/3873283691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c4107a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009820 tx=0x7f37b402a270 comp rx=0 tx=0).stop 2024-09-11T23:43:20.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.682+0000 7f37ca446640 1 -- 172.21.15.62:0/3873283691 shutdown_connections 2024-09-11T23:43:20.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.682+0000 7f37ca446640 1 --2- 172.21.15.62:0/3873283691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c4107a20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.682+0000 7f37ca446640 1 -- 172.21.15.62:0/3873283691 >> 172.21.15.62:0/3873283691 conn(0x7f37c4102db0 msgr2=0x7f37c41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:20.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.682+0000 7f37ca446640 1 -- 172.21.15.62:0/3873283691 shutdown_connections 2024-09-11T23:43:20.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.683+0000 7f37ca446640 1 -- 172.21.15.62:0/3873283691 wait complete. 2024-09-11T23:43:20.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.683+0000 7f37ca446640 1 Processor -- start 2024-09-11T23:43:20.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.683+0000 7f37ca446640 1 -- start start 2024-09-11T23:43:20.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.683+0000 7f37ca446640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.683+0000 7f37ca446640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f37c410bbd0 con 0x7f37c4107620 2024-09-11T23:43:20.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.683+0000 7f37c3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.683+0000 7f37c3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c419d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51254/0 (socket says 172.21.15.62:51254) 2024-09-11T23:43:20.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.683+0000 7f37c3fff640 1 -- 172.21.15.62:0/2054589278 learned_addr learned my addr 172.21.15.62:0/2054589278 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:20.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.684+0000 7f37c3fff640 1 -- 172.21.15.62:0/2054589278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c419dde0 con 0x7f37c4107620 2024-09-11T23:43:20.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.684+0000 7f37c3fff640 1 --2- 172.21.15.62:0/2054589278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c419d8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f37b40097f0 tx=0x7f37b40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:20.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.684+0000 7f37c1ffb640 1 -- 172.21.15.62:0/2054589278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37b4037040 con 0x7f37c4107620 2024-09-11T23:43:20.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.684+0000 7f37c1ffb640 1 -- 172.21.15.62:0/2054589278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b4030c40 con 0x7f37c4107620 2024-09-11T23:43:20.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.684+0000 7f37ca446640 1 -- 172.21.15.62:0/2054589278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c419e070 con 0x7f37c4107620 2024-09-11T23:43:20.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.684+0000 7f37c1ffb640 1 -- 172.21.15.62:0/2054589278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37b403a5b0 con 0x7f37c4107620 2024-09-11T23:43:20.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.684+0000 7f37ca446640 1 -- 172.21.15.62:0/2054589278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c41a0d60 con 0x7f37c4107620 2024-09-11T23:43:20.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.685+0000 7f37c1ffb640 1 -- 172.21.15.62:0/2054589278 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f37b403a7f0 con 0x7f37c4107620 2024-09-11T23:43:20.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.685+0000 7f37ca446640 1 -- 172.21.15.62:0/2054589278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c410bd50 con 0x7f37c4107620 2024-09-11T23:43:20.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.686+0000 7f37c1ffb640 1 --2- 172.21.15.62:0/2054589278 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37b0039060 0x7f37b003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.686+0000 7f37c1ffb640 1 -- 172.21.15.62:0/2054589278 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f37b406c680 con 0x7f37c4107620 2024-09-11T23:43:20.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.686+0000 7f37bbfff640 1 --2- 172.21.15.62:0/2054589278 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37b0039060 0x7f37b003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.686+0000 7f37bbfff640 1 --2- 172.21.15.62:0/2054589278 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37b0039060 0x7f37b003b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0098c0 tx=0x7f37ac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.690+0000 7f37c1ffb640 1 -- 172.21.15.62:0/2054589278 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37b403b3f0 con 0x7f37c4107620 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.957+0000 7f37ca446640 1 -- 172.21.15.62:0/2054589278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f37c41a0f30 con 0x7f37c4107620 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.960+0000 7f37c1ffb640 1 -- 172.21.15.62:0/2054589278 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f37b4044920 con 0x7f37c4107620 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.962+0000 7f37ca446640 1 -- 172.21.15.62:0/2054589278 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37b0039060 msgr2=0x7f37b003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.962+0000 7f37ca446640 1 --2- 172.21.15.62:0/2054589278 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37b0039060 0x7f37b003b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0098c0 tx=0x7f37ac006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.962+0000 7f37ca446640 1 -- 172.21.15.62:0/2054589278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 msgr2=0x7f37c419d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.962+0000 7f37ca446640 1 --2- 172.21.15.62:0/2054589278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c419d8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f37b40097f0 tx=0x7f37b40327b0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.962+0000 7f37ca446640 1 -- 172.21.15.62:0/2054589278 shutdown_connections 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.962+0000 7f37ca446640 1 --2- 172.21.15.62:0/2054589278 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37b0039060 0x7f37b003b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.962+0000 7f37ca446640 1 --2- 172.21.15.62:0/2054589278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37c4107620 0x7f37c419d8a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.962+0000 7f37ca446640 1 -- 172.21.15.62:0/2054589278 >> 172.21.15.62:0/2054589278 conn(0x7f37c4102db0 msgr2=0x7f37c4103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.962+0000 7f37ca446640 1 -- 172.21.15.62:0/2054589278 shutdown_connections 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-09-11T23:43:19.962+0000 7f37ca446640 1 -- 172.21.15.62:0/2054589278 wait complete. 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:20.487 INFO:teuthology.orchestra.run.smithi062.stdout: ceph telemetry on 2024-09-11T23:43:20.488 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:20.488 INFO:teuthology.orchestra.run.smithi062.stdout:For more information see: 2024-09-11T23:43:20.488 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:20.488 INFO:teuthology.orchestra.run.smithi062.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-11T23:43:20.488 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:20.488 INFO:teuthology.orchestra.run.smithi062.stdout:Bootstrap complete. 2024-09-11T23:43:20.523 INFO:tasks.cephadm:Fetching config... 2024-09-11T23:43:20.523 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:43:20.523 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T23:43:20.540 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T23:43:20.541 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:43:20.541 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T23:43:20.596 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T23:43:20.596 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:43:20.596 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T23:43:20.666 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T23:43:20.666 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:43:20.666 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T23:43:20.722 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T23:43:20.722 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzFhWzYrJcmK8rnIwIL0yfLwE89WskHh1HtYsybI3N531Sg2JpOPl2vgUjQedjqF6qPjpuPDYtn5fLba3MjxzZW2PwpWgCXb2Ac5jKIE0cpePjkRIMQgI3+QraFOylA5AtaM01SzUFeUQ2PcTTZcQo0aitjeNGS8D7BoswMZSwXFDWiM+FBlVRPRbB0qt07Md9wNa9y93rbi5w3kAESf6HyYngyicbu/i647omFN5w6qIS0bZOqInDL6YxtfvtkCIW1/KQQA9Xr6io8/hctBX2TxTnINdV+qrmW1OGIA4SNYHt0p6A1hzGrzCyVr/SGLyKZs9Sn5TRzK1mDXji5KlcY1cosPTBQ0gDDaEseOAVZDJsosSw2z4FkvNLJcbD2p8wbgWOC03QkcmPW+qzqC4aNLKiYEbpwGfE//dnzH6EHe0ENO1ivKX8xmQqe5GuHXYOpDGfjlEEavkWS8K9JEdfZnD/pldtJWwpCYgFt4Ov0IJlkFPpqBPVSZpnH08t+PE= ceph-56675698-7097-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:43:20.802 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzFhWzYrJcmK8rnIwIL0yfLwE89WskHh1HtYsybI3N531Sg2JpOPl2vgUjQedjqF6qPjpuPDYtn5fLba3MjxzZW2PwpWgCXb2Ac5jKIE0cpePjkRIMQgI3+QraFOylA5AtaM01SzUFeUQ2PcTTZcQo0aitjeNGS8D7BoswMZSwXFDWiM+FBlVRPRbB0qt07Md9wNa9y93rbi5w3kAESf6HyYngyicbu/i647omFN5w6qIS0bZOqInDL6YxtfvtkCIW1/KQQA9Xr6io8/hctBX2TxTnINdV+qrmW1OGIA4SNYHt0p6A1hzGrzCyVr/SGLyKZs9Sn5TRzK1mDXji5KlcY1cosPTBQ0gDDaEseOAVZDJsosSw2z4FkvNLJcbD2p8wbgWOC03QkcmPW+qzqC4aNLKiYEbpwGfE//dnzH6EHe0ENO1ivKX8xmQqe5GuHXYOpDGfjlEEavkWS8K9JEdfZnD/pldtJWwpCYgFt4Ov0IJlkFPpqBPVSZpnH08t+PE= ceph-56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:43:20.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzFhWzYrJcmK8rnIwIL0yfLwE89WskHh1HtYsybI3N531Sg2JpOPl2vgUjQedjqF6qPjpuPDYtn5fLba3MjxzZW2PwpWgCXb2Ac5jKIE0cpePjkRIMQgI3+QraFOylA5AtaM01SzUFeUQ2PcTTZcQo0aitjeNGS8D7BoswMZSwXFDWiM+FBlVRPRbB0qt07Md9wNa9y93rbi5w3kAESf6HyYngyicbu/i647omFN5w6qIS0bZOqInDL6YxtfvtkCIW1/KQQA9Xr6io8/hctBX2TxTnINdV+qrmW1OGIA4SNYHt0p6A1hzGrzCyVr/SGLyKZs9Sn5TRzK1mDXji5KlcY1cosPTBQ0gDDaEseOAVZDJsosSw2z4FkvNLJcbD2p8wbgWOC03QkcmPW+qzqC4aNLKiYEbpwGfE//dnzH6EHe0ENO1ivKX8xmQqe5GuHXYOpDGfjlEEavkWS8K9JEdfZnD/pldtJWwpCYgFt4Ov0IJlkFPpqBPVSZpnH08t+PE= ceph-56675698-7097-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:43:20.855 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzFhWzYrJcmK8rnIwIL0yfLwE89WskHh1HtYsybI3N531Sg2JpOPl2vgUjQedjqF6qPjpuPDYtn5fLba3MjxzZW2PwpWgCXb2Ac5jKIE0cpePjkRIMQgI3+QraFOylA5AtaM01SzUFeUQ2PcTTZcQo0aitjeNGS8D7BoswMZSwXFDWiM+FBlVRPRbB0qt07Md9wNa9y93rbi5w3kAESf6HyYngyicbu/i647omFN5w6qIS0bZOqInDL6YxtfvtkCIW1/KQQA9Xr6io8/hctBX2TxTnINdV+qrmW1OGIA4SNYHt0p6A1hzGrzCyVr/SGLyKZs9Sn5TRzK1mDXji5KlcY1cosPTBQ0gDDaEseOAVZDJsosSw2z4FkvNLJcbD2p8wbgWOC03QkcmPW+qzqC4aNLKiYEbpwGfE//dnzH6EHe0ENO1ivKX8xmQqe5GuHXYOpDGfjlEEavkWS8K9JEdfZnD/pldtJWwpCYgFt4Ov0IJlkFPpqBPVSZpnH08t+PE= ceph-56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:43:20.869 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzFhWzYrJcmK8rnIwIL0yfLwE89WskHh1HtYsybI3N531Sg2JpOPl2vgUjQedjqF6qPjpuPDYtn5fLba3MjxzZW2PwpWgCXb2Ac5jKIE0cpePjkRIMQgI3+QraFOylA5AtaM01SzUFeUQ2PcTTZcQo0aitjeNGS8D7BoswMZSwXFDWiM+FBlVRPRbB0qt07Md9wNa9y93rbi5w3kAESf6HyYngyicbu/i647omFN5w6qIS0bZOqInDL6YxtfvtkCIW1/KQQA9Xr6io8/hctBX2TxTnINdV+qrmW1OGIA4SNYHt0p6A1hzGrzCyVr/SGLyKZs9Sn5TRzK1mDXji5KlcY1cosPTBQ0gDDaEseOAVZDJsosSw2z4FkvNLJcbD2p8wbgWOC03QkcmPW+qzqC4aNLKiYEbpwGfE//dnzH6EHe0ENO1ivKX8xmQqe5GuHXYOpDGfjlEEavkWS8K9JEdfZnD/pldtJWwpCYgFt4Ov0IJlkFPpqBPVSZpnH08t+PE= ceph-56675698-7097-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:43:20.911 INFO:teuthology.orchestra.run.smithi120.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzFhWzYrJcmK8rnIwIL0yfLwE89WskHh1HtYsybI3N531Sg2JpOPl2vgUjQedjqF6qPjpuPDYtn5fLba3MjxzZW2PwpWgCXb2Ac5jKIE0cpePjkRIMQgI3+QraFOylA5AtaM01SzUFeUQ2PcTTZcQo0aitjeNGS8D7BoswMZSwXFDWiM+FBlVRPRbB0qt07Md9wNa9y93rbi5w3kAESf6HyYngyicbu/i647omFN5w6qIS0bZOqInDL6YxtfvtkCIW1/KQQA9Xr6io8/hctBX2TxTnINdV+qrmW1OGIA4SNYHt0p6A1hzGrzCyVr/SGLyKZs9Sn5TRzK1mDXji5KlcY1cosPTBQ0gDDaEseOAVZDJsosSw2z4FkvNLJcbD2p8wbgWOC03QkcmPW+qzqC4aNLKiYEbpwGfE//dnzH6EHe0ENO1ivKX8xmQqe5GuHXYOpDGfjlEEavkWS8K9JEdfZnD/pldtJWwpCYgFt4Ov0IJlkFPpqBPVSZpnH08t+PE= ceph-56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:43:20.924 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T23:43:21.027 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:20 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2054589278' entity='client.admin' 2024-09-11T23:43:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.677+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2042979665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488100540 msgr2=0x7fd488100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.677+0000 7fd48cb6c640 1 --2- 172.21.15.62:0/2042979665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488100540 0x7fd488100940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd478009880 tx=0x7fd47802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:43:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.677+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2042979665 shutdown_connections 2024-09-11T23:43:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.677+0000 7fd48cb6c640 1 --2- 172.21.15.62:0/2042979665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488100540 0x7fd488100940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.678+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2042979665 >> 172.21.15.62:0/2042979665 conn(0x7fd4880fbcd0 msgr2=0x7fd4880fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.678+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2042979665 shutdown_connections 2024-09-11T23:43:21.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.678+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2042979665 wait complete. 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.678+0000 7fd48cb6c640 1 Processor -- start 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.678+0000 7fd48cb6c640 1 -- start start 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.678+0000 7fd48cb6c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488100540 0x7fd48806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.678+0000 7fd48cb6c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881065f0 con 0x7fd488100540 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.679+0000 7fd48659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488100540 0x7fd48806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.679+0000 7fd48659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488100540 0x7fd48806a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60276/0 (socket says 172.21.15.62:60276) 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.679+0000 7fd48659c640 1 -- 172.21.15.62:0/2619458434 learned_addr learned my addr 172.21.15.62:0/2619458434 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.679+0000 7fd48659c640 1 -- 172.21.15.62:0/2619458434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd488199a70 con 0x7fd488100540 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.679+0000 7fd48659c640 1 --2- 172.21.15.62:0/2619458434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488100540 0x7fd48806a170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd478004f60 tx=0x7fd4780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.679+0000 7fd4777fe640 1 -- 172.21.15.62:0/2619458434 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd47803f070 con 0x7fd488100540 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.679+0000 7fd4777fe640 1 -- 172.21.15.62:0/2619458434 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd478004d40 con 0x7fd488100540 2024-09-11T23:43:21.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.679+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2619458434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd488199c40 con 0x7fd488100540 2024-09-11T23:43:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.679+0000 7fd4777fe640 1 -- 172.21.15.62:0/2619458434 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd478035030 con 0x7fd488100540 2024-09-11T23:43:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.679+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2619458434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48819a060 con 0x7fd488100540 2024-09-11T23:43:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.680+0000 7fd4777fe640 1 -- 172.21.15.62:0/2619458434 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd478042070 con 0x7fd488100540 2024-09-11T23:43:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.680+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2619458434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd488106d70 con 0x7fd488100540 2024-09-11T23:43:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.681+0000 7fd4777fe640 1 --2- 172.21.15.62:0/2619458434 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd4700390c0 0x7fd47003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.681+0000 7fd4777fe640 1 -- 172.21.15.62:0/2619458434 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd47806ce00 con 0x7fd488100540 2024-09-11T23:43:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.681+0000 7fd485d9b640 1 --2- 172.21.15.62:0/2619458434 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd4700390c0 0x7fd47003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.681+0000 7fd485d9b640 1 --2- 172.21.15.62:0/2619458434 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd4700390c0 0x7fd47003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0098c0 tx=0x7fd47c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.689+0000 7fd4777fe640 1 -- 172.21.15.62:0/2619458434 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd478039350 con 0x7fd488100540 2024-09-11T23:43:21.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.874+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2619458434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fd48818f3b0 con 0x7fd488100540 2024-09-11T23:43:21.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.876+0000 7fd4777fe640 1 -- 172.21.15.62:0/2619458434 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fd47803b460 con 0x7fd488100540 2024-09-11T23:43:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.879+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2619458434 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd4700390c0 msgr2=0x7fd47003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.879+0000 7fd48cb6c640 1 --2- 172.21.15.62:0/2619458434 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd4700390c0 0x7fd47003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0098c0 tx=0x7fd47c006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.879+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2619458434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488100540 msgr2=0x7fd48806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.879+0000 7fd48cb6c640 1 --2- 172.21.15.62:0/2619458434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488100540 0x7fd48806a170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd478004f60 tx=0x7fd4780048f0 comp rx=0 tx=0).stop 2024-09-11T23:43:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.879+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2619458434 shutdown_connections 2024-09-11T23:43:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.879+0000 7fd48cb6c640 1 --2- 172.21.15.62:0/2619458434 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd4700390c0 0x7fd47003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.879+0000 7fd48cb6c640 1 --2- 172.21.15.62:0/2619458434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488100540 0x7fd48806a170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.879+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2619458434 >> 172.21.15.62:0/2619458434 conn(0x7fd4880fbcd0 msgr2=0x7fd4880fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.879+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2619458434 shutdown_connections 2024-09-11T23:43:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:21.879+0000 7fd48cb6c640 1 -- 172.21.15.62:0/2619458434 wait complete. 2024-09-11T23:43:23.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:22 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2619458434' entity='client.admin' 2024-09-11T23:43:23.362 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T23:43:23.362 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T23:43:24.043 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.618+0000 7fd1816f6640 1 -- 172.21.15.62:0/1589000282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd17c102100 msgr2=0x7fd17c1044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.618+0000 7fd1816f6640 1 --2- 172.21.15.62:0/1589000282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd17c102100 0x7fd17c1044f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009880 tx=0x7fd16c02a5d0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.618+0000 7fd1816f6640 1 -- 172.21.15.62:0/1589000282 shutdown_connections 2024-09-11T23:43:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.618+0000 7fd1816f6640 1 --2- 172.21.15.62:0/1589000282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd17c102100 0x7fd17c1044f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.618+0000 7fd1816f6640 1 -- 172.21.15.62:0/1589000282 >> 172.21.15.62:0/1589000282 conn(0x7fd17c0fbc80 msgr2=0x7fd17c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.618+0000 7fd1816f6640 1 -- 172.21.15.62:0/1589000282 shutdown_connections 2024-09-11T23:43:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.618+0000 7fd1816f6640 1 -- 172.21.15.62:0/1589000282 wait complete. 2024-09-11T23:43:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.619+0000 7fd1816f6640 1 Processor -- start 2024-09-11T23:43:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.619+0000 7fd1816f6640 1 -- start start 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.619+0000 7fd1816f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd17c102100 0x7fd17c101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.619+0000 7fd1816f6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c107710 con 0x7fd17c102100 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.619+0000 7fd17affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd17c102100 0x7fd17c101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.619+0000 7fd17affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd17c102100 0x7fd17c101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60284/0 (socket says 172.21.15.62:60284) 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.619+0000 7fd17affd640 1 -- 172.21.15.62:0/2632255480 learned_addr learned my addr 172.21.15.62:0/2632255480 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.619+0000 7fd17affd640 1 -- 172.21.15.62:0/2632255480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c101ec0 con 0x7fd17c102100 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.620+0000 7fd17affd640 1 --2- 172.21.15.62:0/2632255480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd17c102100 0x7fd17c101980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd16c004f60 tx=0x7fd16c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.620+0000 7fd178ff9640 1 -- 172.21.15.62:0/2632255480 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd16c03f070 con 0x7fd17c102100 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.620+0000 7fd178ff9640 1 -- 172.21.15.62:0/2632255480 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16c003ad0 con 0x7fd17c102100 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.620+0000 7fd178ff9640 1 -- 172.21.15.62:0/2632255480 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd16c035030 con 0x7fd17c102100 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.620+0000 7fd1816f6640 1 -- 172.21.15.62:0/2632255480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17c1000b0 con 0x7fd17c102100 2024-09-11T23:43:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.620+0000 7fd1816f6640 1 -- 172.21.15.62:0/2632255480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17c1005f0 con 0x7fd17c102100 2024-09-11T23:43:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.621+0000 7fd178ff9640 1 -- 172.21.15.62:0/2632255480 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd16c042070 con 0x7fd17c102100 2024-09-11T23:43:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.621+0000 7fd178ff9640 1 --2- 172.21.15.62:0/2632255480 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1680390c0 0x7fd16803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.621+0000 7fd1816f6640 1 -- 172.21.15.62:0/2632255480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17c107e90 con 0x7fd17c102100 2024-09-11T23:43:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.621+0000 7fd178ff9640 1 -- 172.21.15.62:0/2632255480 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd16c06cd50 con 0x7fd17c102100 2024-09-11T23:43:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.621+0000 7fd173fff640 1 --2- 172.21.15.62:0/2632255480 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1680390c0 0x7fd16803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.621+0000 7fd173fff640 1 --2- 172.21.15.62:0/2632255480 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1680390c0 0x7fd16803b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd1640098c0 tx=0x7fd164006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:24.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.626+0000 7fd178ff9640 1 -- 172.21.15.62:0/2632255480 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd16c04a260 con 0x7fd17c102100 2024-09-11T23:43:24.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.806+0000 7fd1816f6640 1 -- 172.21.15.62:0/2632255480 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fd17c190180 con 0x7fd1680390c0 2024-09-11T23:43:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.809+0000 7fd178ff9640 1 -- 172.21.15.62:0/2632255480 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd17c190180 con 0x7fd1680390c0 2024-09-11T23:43:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.811+0000 7fd1816f6640 1 -- 172.21.15.62:0/2632255480 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1680390c0 msgr2=0x7fd16803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.811+0000 7fd1816f6640 1 --2- 172.21.15.62:0/2632255480 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1680390c0 0x7fd16803b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd1640098c0 tx=0x7fd164006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.811+0000 7fd1816f6640 1 -- 172.21.15.62:0/2632255480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd17c102100 msgr2=0x7fd17c101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.811+0000 7fd1816f6640 1 --2- 172.21.15.62:0/2632255480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd17c102100 0x7fd17c101980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd16c004f60 tx=0x7fd16c0048f0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.811+0000 7fd1816f6640 1 -- 172.21.15.62:0/2632255480 shutdown_connections 2024-09-11T23:43:24.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.811+0000 7fd1816f6640 1 --2- 172.21.15.62:0/2632255480 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1680390c0 0x7fd16803b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.811+0000 7fd1816f6640 1 --2- 172.21.15.62:0/2632255480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd17c102100 0x7fd17c101980 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.811+0000 7fd1816f6640 1 -- 172.21.15.62:0/2632255480 >> 172.21.15.62:0/2632255480 conn(0x7fd17c0fbc80 msgr2=0x7fd17c0fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:24.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.811+0000 7fd1816f6640 1 -- 172.21.15.62:0/2632255480 shutdown_connections 2024-09-11T23:43:24.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:24.811+0000 7fd1816f6640 1 -- 172.21.15.62:0/2632255480 wait complete. 2024-09-11T23:43:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:25 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:25 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:25 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:43:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:25 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:25 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:43:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:25 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:25 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:25 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:43:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:25 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:25 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:43:25.666 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi083 2024-09-11T23:43:25.666 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:43:25.666 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/etc/ceph/ceph.conf 2024-09-11T23:43:25.686 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:43:25.686 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:43:25.744 INFO:tasks.cephadm:Adding host smithi083 to orchestrator... 2024-09-11T23:43:25.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch host add smithi083 2024-09-11T23:43:25.995 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.474+0000 7f72a48f4640 1 -- 172.21.15.62:0/1189035994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0106540 msgr2=0x7f72a0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.474+0000 7f72a48f4640 1 --2- 172.21.15.62:0/1189035994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0106540 0x7f72a0106920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7298009c10 tx=0x7f729802aa10 comp rx=0 tx=0).stop 2024-09-11T23:43:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.474+0000 7f72a48f4640 1 -- 172.21.15.62:0/1189035994 shutdown_connections 2024-09-11T23:43:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.474+0000 7f72a48f4640 1 --2- 172.21.15.62:0/1189035994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0106540 0x7f72a0106920 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.474+0000 7f72a48f4640 1 -- 172.21.15.62:0/1189035994 >> 172.21.15.62:0/1189035994 conn(0x7f72a0077310 msgr2=0x7f72a00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.474+0000 7f72a48f4640 1 -- 172.21.15.62:0/1189035994 shutdown_connections 2024-09-11T23:43:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.474+0000 7f72a48f4640 1 -- 172.21.15.62:0/1189035994 wait complete. 2024-09-11T23:43:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.475+0000 7f72a48f4640 1 Processor -- start 2024-09-11T23:43:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.475+0000 7f72a48f4640 1 -- start start 2024-09-11T23:43:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.475+0000 7f72a48f4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0106540 0x7f72a00fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.475+0000 7f72a48f4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72a010ab70 con 0x7f72a0106540 2024-09-11T23:43:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.475+0000 7f729e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0106540 0x7f72a00fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.475+0000 7f729e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0106540 0x7f72a00fd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60298/0 (socket says 172.21.15.62:60298) 2024-09-11T23:43:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.475+0000 7f729e59c640 1 -- 172.21.15.62:0/1223711321 learned_addr learned my addr 172.21.15.62:0/1223711321 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.475+0000 7f729e59c640 1 -- 172.21.15.62:0/1223711321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a00ff2d0 con 0x7f72a0106540 2024-09-11T23:43:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.476+0000 7f729e59c640 1 --2- 172.21.15.62:0/1223711321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0106540 0x7f72a00fd3e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7298005dd0 tx=0x7f7298030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.476+0000 7f72977fe640 1 -- 172.21.15.62:0/1223711321 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7298034070 con 0x7f72a0106540 2024-09-11T23:43:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.476+0000 7f72a48f4640 1 -- 172.21.15.62:0/1223711321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a00fd920 con 0x7f72a0106540 2024-09-11T23:43:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.476+0000 7f72977fe640 1 -- 172.21.15.62:0/1223711321 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f729802f8c0 con 0x7f72a0106540 2024-09-11T23:43:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.476+0000 7f72977fe640 1 -- 172.21.15.62:0/1223711321 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7298004520 con 0x7f72a0106540 2024-09-11T23:43:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.476+0000 7f72a48f4640 1 -- 172.21.15.62:0/1223711321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a00fdd20 con 0x7f72a0106540 2024-09-11T23:43:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.477+0000 7f72977fe640 1 -- 172.21.15.62:0/1223711321 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7298037020 con 0x7f72a0106540 2024-09-11T23:43:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.477+0000 7f72a48f4640 1 -- 172.21.15.62:0/1223711321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a0101bb0 con 0x7f72a0106540 2024-09-11T23:43:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.477+0000 7f72977fe640 1 --2- 172.21.15.62:0/1223711321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f728c039060 0x7f728c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.477+0000 7f72977fe640 1 -- 172.21.15.62:0/1223711321 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f729806cf10 con 0x7f72a0106540 2024-09-11T23:43:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.477+0000 7f729dd9b640 1 --2- 172.21.15.62:0/1223711321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f728c039060 0x7f728c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.478+0000 7f729dd9b640 1 --2- 172.21.15.62:0/1223711321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f728c039060 0x7f728c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f72880098c0 tx=0x7f7288006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.482+0000 7f72977fe640 1 -- 172.21.15.62:0/1223711321 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f729803f070 con 0x7f72a0106540 2024-09-11T23:43:26.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:26.660+0000 7f72a48f4640 1 -- 172.21.15.62:0/1223711321 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi083", "target": ["mon-mgr", ""]}) -- 0x7f72a00fe910 con 0x7f728c039060 2024-09-11T23:43:26.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:26 smithi062 ceph-mon[30958]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:43:26.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:26 smithi062 ceph-mon[30958]: Updating smithi062:/etc/ceph/ceph.conf 2024-09-11T23:43:26.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:26 smithi062 ceph-mon[30958]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:43:26.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:26 smithi062 ceph-mon[30958]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:43:26.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:26 smithi062 ceph-mon[30958]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:43:26.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:26 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:26.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:26 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:26.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:26 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:27 smithi062 ceph-mon[30958]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:43:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:28 smithi062 ceph-mon[30958]: Deploying cephadm binary to smithi083 2024-09-11T23:43:30.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.562+0000 7f72977fe640 1 -- 172.21.15.62:0/1223711321 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f72a00fe910 con 0x7f728c039060 2024-09-11T23:43:30.562 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi083' with addr '172.21.15.83' 2024-09-11T23:43:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.565+0000 7f72a48f4640 1 -- 172.21.15.62:0/1223711321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f728c039060 msgr2=0x7f728c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.565+0000 7f72a48f4640 1 --2- 172.21.15.62:0/1223711321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f728c039060 0x7f728c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f72880098c0 tx=0x7f7288006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.565+0000 7f72a48f4640 1 -- 172.21.15.62:0/1223711321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0106540 msgr2=0x7f72a00fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.565+0000 7f72a48f4640 1 --2- 172.21.15.62:0/1223711321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0106540 0x7f72a00fd3e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7298005dd0 tx=0x7f7298030c30 comp rx=0 tx=0).stop 2024-09-11T23:43:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.565+0000 7f72a48f4640 1 -- 172.21.15.62:0/1223711321 shutdown_connections 2024-09-11T23:43:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.565+0000 7f72a48f4640 1 --2- 172.21.15.62:0/1223711321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f728c039060 0x7f728c03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.565+0000 7f72a48f4640 1 --2- 172.21.15.62:0/1223711321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0106540 0x7f72a00fd3e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.565+0000 7f72a48f4640 1 -- 172.21.15.62:0/1223711321 >> 172.21.15.62:0/1223711321 conn(0x7f72a0077310 msgr2=0x7f72a00696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.565+0000 7f72a48f4640 1 -- 172.21.15.62:0/1223711321 shutdown_connections 2024-09-11T23:43:30.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:30.565+0000 7f72a48f4640 1 -- 172.21.15.62:0/1223711321 wait complete. 2024-09-11T23:43:31.174 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T23:43:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:31.804 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:31 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:31.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:31 smithi062 ceph-mon[30958]: Added host smithi083 2024-09-11T23:43:31.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:31 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:43:31.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:31 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:31.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:31 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:31.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.891+0000 7fd952651640 1 -- 172.21.15.62:0/2994954548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd94c107660 msgr2=0x7fd94c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:31.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.891+0000 7fd952651640 1 --2- 172.21.15.62:0/2994954548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd94c107660 0x7fd94c107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009880 tx=0x7fd93c02a5d0 comp rx=0 tx=0).stop 2024-09-11T23:43:31.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.891+0000 7fd952651640 1 -- 172.21.15.62:0/2994954548 shutdown_connections 2024-09-11T23:43:31.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.891+0000 7fd952651640 1 --2- 172.21.15.62:0/2994954548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd94c107660 0x7fd94c107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:31.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.891+0000 7fd952651640 1 -- 172.21.15.62:0/2994954548 >> 172.21.15.62:0/2994954548 conn(0x7fd94c0fd290 msgr2=0x7fd94c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:31.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.891+0000 7fd952651640 1 -- 172.21.15.62:0/2994954548 shutdown_connections 2024-09-11T23:43:31.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.891+0000 7fd952651640 1 -- 172.21.15.62:0/2994954548 wait complete. 2024-09-11T23:43:31.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.891+0000 7fd952651640 1 Processor -- start 2024-09-11T23:43:31.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.892+0000 7fd952651640 1 -- start start 2024-09-11T23:43:31.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.892+0000 7fd952651640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd94c107660 0x7fd94c197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:31.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.892+0000 7fd952651640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c10bc90 con 0x7fd94c107660 2024-09-11T23:43:31.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.892+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd94c107660 0x7fd94c197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:31.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.892+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd94c107660 0x7fd94c197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56050/0 (socket says 172.21.15.62:56050) 2024-09-11T23:43:31.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.892+0000 7fd94bfff640 1 -- 172.21.15.62:0/1909889427 learned_addr learned my addr 172.21.15.62:0/1909889427 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.892+0000 7fd94bfff640 1 -- 172.21.15.62:0/1909889427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c197670 con 0x7fd94c107660 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.893+0000 7fd94bfff640 1 --2- 172.21.15.62:0/1909889427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd94c107660 0x7fd94c197130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009230 tx=0x7fd93c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.893+0000 7fd9497fa640 1 -- 172.21.15.62:0/1909889427 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd93c03f070 con 0x7fd94c107660 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.893+0000 7fd9497fa640 1 -- 172.21.15.62:0/1909889427 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93c03bc80 con 0x7fd94c107660 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.893+0000 7fd952651640 1 -- 172.21.15.62:0/1909889427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c197900 con 0x7fd94c107660 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.893+0000 7fd9497fa640 1 -- 172.21.15.62:0/1909889427 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd93c035030 con 0x7fd94c107660 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.893+0000 7fd952651640 1 -- 172.21.15.62:0/1909889427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c19ba00 con 0x7fd94c107660 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.894+0000 7fd9497fa640 1 -- 172.21.15.62:0/1909889427 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd93c042070 con 0x7fd94c107660 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.894+0000 7fd9497fa640 1 --2- 172.21.15.62:0/1909889427 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9340390c0 0x7fd93403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.894+0000 7fd9497fa640 1 -- 172.21.15.62:0/1909889427 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd93c03b7a0 con 0x7fd94c107660 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.894+0000 7fd94b7fe640 1 --2- 172.21.15.62:0/1909889427 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9340390c0 0x7fd93403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.894+0000 7fd952651640 1 -- 172.21.15.62:0/1909889427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c102cd0 con 0x7fd94c107660 2024-09-11T23:43:31.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.895+0000 7fd94b7fe640 1 --2- 172.21.15.62:0/1909889427 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9340390c0 0x7fd93403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd9400098c0 tx=0x7fd940006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:31.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:31.899+0000 7fd9497fa640 1 -- 172.21.15.62:0/1909889427 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd93c030c30 con 0x7fd94c107660 2024-09-11T23:43:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.082+0000 7fd952651640 1 -- 172.21.15.62:0/1909889427 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd94c198490 con 0x7fd9340390c0 2024-09-11T23:43:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.084+0000 7fd9497fa640 1 -- 172.21.15.62:0/1909889427 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fd94c198490 con 0x7fd9340390c0 2024-09-11T23:43:32.084 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:32.084 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}] 2024-09-11T23:43:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.086+0000 7fd952651640 1 -- 172.21.15.62:0/1909889427 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9340390c0 msgr2=0x7fd93403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.086+0000 7fd952651640 1 --2- 172.21.15.62:0/1909889427 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9340390c0 0x7fd93403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd9400098c0 tx=0x7fd940006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.086+0000 7fd952651640 1 -- 172.21.15.62:0/1909889427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd94c107660 msgr2=0x7fd94c197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.086+0000 7fd952651640 1 --2- 172.21.15.62:0/1909889427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd94c107660 0x7fd94c197130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009230 tx=0x7fd93c003d40 comp rx=0 tx=0).stop 2024-09-11T23:43:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.086+0000 7fd952651640 1 -- 172.21.15.62:0/1909889427 shutdown_connections 2024-09-11T23:43:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.086+0000 7fd952651640 1 --2- 172.21.15.62:0/1909889427 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9340390c0 0x7fd93403b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.086+0000 7fd952651640 1 --2- 172.21.15.62:0/1909889427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd94c107660 0x7fd94c197130 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.086+0000 7fd952651640 1 -- 172.21.15.62:0/1909889427 >> 172.21.15.62:0/1909889427 conn(0x7fd94c0fd290 msgr2=0x7fd94c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.087+0000 7fd952651640 1 -- 172.21.15.62:0/1909889427 shutdown_connections 2024-09-11T23:43:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:32.087+0000 7fd952651640 1 -- 172.21.15.62:0/1909889427 wait complete. 2024-09-11T23:43:32.702 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi120 2024-09-11T23:43:32.702 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:43:32.702 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.conf 2024-09-11T23:43:32.719 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:43:32.719 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:43:32.774 INFO:tasks.cephadm:Adding host smithi120 to orchestrator... 2024-09-11T23:43:32.774 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch host add smithi120 2024-09-11T23:43:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:33.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:33.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:32 smithi062 ceph-mon[30958]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:43:33.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:32 smithi062 ceph-mon[30958]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:33.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.477+0000 7f4e744b4640 1 -- 172.21.15.62:0/960641021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e6c106560 msgr2=0x7f4e6c106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.477+0000 7f4e744b4640 1 --2- 172.21.15.62:0/960641021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e6c106560 0x7f4e6c106940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009880 tx=0x7f4e6802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.477+0000 7f4e744b4640 1 -- 172.21.15.62:0/960641021 shutdown_connections 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.477+0000 7f4e744b4640 1 --2- 172.21.15.62:0/960641021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e6c106560 0x7f4e6c106940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.477+0000 7f4e744b4640 1 -- 172.21.15.62:0/960641021 >> 172.21.15.62:0/960641021 conn(0x7f4e6c0fc1b0 msgr2=0x7f4e6c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.477+0000 7f4e744b4640 1 -- 172.21.15.62:0/960641021 shutdown_connections 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.477+0000 7f4e744b4640 1 -- 172.21.15.62:0/960641021 wait complete. 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.478+0000 7f4e744b4640 1 Processor -- start 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.478+0000 7f4e744b4640 1 -- start start 2024-09-11T23:43:33.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.478+0000 7f4e744b4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e6c106560 0x7f4e6c197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.478+0000 7f4e744b4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c10ab90 con 0x7f4e6c106560 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.478+0000 7f4e72250640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e6c106560 0x7f4e6c197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.478+0000 7f4e72250640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e6c106560 0x7f4e6c197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56064/0 (socket says 172.21.15.62:56064) 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.478+0000 7f4e72250640 1 -- 172.21.15.62:0/1305661834 learned_addr learned my addr 172.21.15.62:0/1305661834 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.479+0000 7f4e72250640 1 -- 172.21.15.62:0/1305661834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c197640 con 0x7f4e6c106560 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.479+0000 7f4e72250640 1 --2- 172.21.15.62:0/1305661834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e6c106560 0x7f4e6c197100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009230 tx=0x7f4e68003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.479+0000 7f4e637fe640 1 -- 172.21.15.62:0/1305661834 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e6803f070 con 0x7f4e6c106560 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.479+0000 7f4e744b4640 1 -- 172.21.15.62:0/1305661834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6c1978d0 con 0x7f4e6c106560 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.479+0000 7f4e637fe640 1 -- 172.21.15.62:0/1305661834 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6803bc80 con 0x7f4e6c106560 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.479+0000 7f4e637fe640 1 -- 172.21.15.62:0/1305661834 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e68035030 con 0x7f4e6c106560 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.479+0000 7f4e744b4640 1 -- 172.21.15.62:0/1305661834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6c19b9d0 con 0x7f4e6c106560 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.480+0000 7f4e637fe640 1 -- 172.21.15.62:0/1305661834 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4e68042070 con 0x7f4e6c106560 2024-09-11T23:43:33.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.480+0000 7f4e744b4640 1 -- 172.21.15.62:0/1305661834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e6c101bd0 con 0x7f4e6c106560 2024-09-11T23:43:33.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.480+0000 7f4e637fe640 1 --2- 172.21.15.62:0/1305661834 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e58039060 0x7f4e5803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.480+0000 7f4e637fe640 1 -- 172.21.15.62:0/1305661834 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4e6806cce0 con 0x7f4e6c106560 2024-09-11T23:43:33.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.480+0000 7f4e71a4f640 1 --2- 172.21.15.62:0/1305661834 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e58039060 0x7f4e5803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.481+0000 7f4e71a4f640 1 --2- 172.21.15.62:0/1305661834 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e58039060 0x7f4e5803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0098c0 tx=0x7f4e5c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:33.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.485+0000 7f4e637fe640 1 -- 172.21.15.62:0/1305661834 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e680372d0 con 0x7f4e6c106560 2024-09-11T23:43:33.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:33.664+0000 7f4e744b4640 1 -- 172.21.15.62:0/1305661834 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}) -- 0x7f4e6c1069c0 con 0x7f4e58039060 2024-09-11T23:43:34.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:33 smithi062 ceph-mon[30958]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:43:35.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:34 smithi062 ceph-mon[30958]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:35.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:35.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:35.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:35.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:35.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:43:35.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:35.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:43:36.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:36 smithi062 ceph-mon[30958]: Deploying cephadm binary to smithi120 2024-09-11T23:43:36.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:36 smithi062 ceph-mon[30958]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-11T23:43:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:36 smithi062 ceph-mon[30958]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:43:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:36 smithi062 ceph-mon[30958]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:43:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:36 smithi062 ceph-mon[30958]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:43:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:37.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.550+0000 7f4e637fe640 1 -- 172.21.15.62:0/1305661834 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f4e6c1069c0 con 0x7f4e58039060 2024-09-11T23:43:37.549 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi120' with addr '172.21.15.120' 2024-09-11T23:43:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.552+0000 7f4e744b4640 1 -- 172.21.15.62:0/1305661834 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e58039060 msgr2=0x7f4e5803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.552+0000 7f4e744b4640 1 --2- 172.21.15.62:0/1305661834 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e58039060 0x7f4e5803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0098c0 tx=0x7f4e5c006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.552+0000 7f4e744b4640 1 -- 172.21.15.62:0/1305661834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e6c106560 msgr2=0x7f4e6c197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.552+0000 7f4e744b4640 1 --2- 172.21.15.62:0/1305661834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e6c106560 0x7f4e6c197100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009230 tx=0x7f4e68003d40 comp rx=0 tx=0).stop 2024-09-11T23:43:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.552+0000 7f4e744b4640 1 -- 172.21.15.62:0/1305661834 shutdown_connections 2024-09-11T23:43:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.552+0000 7f4e744b4640 1 --2- 172.21.15.62:0/1305661834 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e58039060 0x7f4e5803b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.552+0000 7f4e744b4640 1 --2- 172.21.15.62:0/1305661834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e6c106560 0x7f4e6c197100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.552+0000 7f4e744b4640 1 -- 172.21.15.62:0/1305661834 >> 172.21.15.62:0/1305661834 conn(0x7f4e6c0fc1b0 msgr2=0x7f4e6c069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.552+0000 7f4e744b4640 1 -- 172.21.15.62:0/1305661834 shutdown_connections 2024-09-11T23:43:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:37.552+0000 7f4e744b4640 1 -- 172.21.15.62:0/1305661834 wait complete. 2024-09-11T23:43:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:37 smithi062 ceph-mon[30958]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:37 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:37 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:43:38.168 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T23:43:38.415 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:38.825 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:38 smithi062 ceph-mon[30958]: Added host smithi120 2024-09-11T23:43:38.825 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:38 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:38.825 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:38 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:38.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.834+0000 7f9d40750640 1 -- 172.21.15.62:0/3284407750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d38102750 msgr2=0x7f9d38102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:38.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.834+0000 7f9d40750640 1 --2- 172.21.15.62:0/3284407750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d38102750 0x7f9d38102b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009c20 tx=0x7f9d2802ab20 comp rx=0 tx=0).stop 2024-09-11T23:43:38.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.834+0000 7f9d40750640 1 -- 172.21.15.62:0/3284407750 shutdown_connections 2024-09-11T23:43:38.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.834+0000 7f9d40750640 1 --2- 172.21.15.62:0/3284407750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d38102750 0x7f9d38102b30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:38.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.834+0000 7f9d40750640 1 -- 172.21.15.62:0/3284407750 >> 172.21.15.62:0/3284407750 conn(0x7f9d380fc1b0 msgr2=0x7f9d380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:38.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.834+0000 7f9d40750640 1 -- 172.21.15.62:0/3284407750 shutdown_connections 2024-09-11T23:43:38.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.834+0000 7f9d40750640 1 -- 172.21.15.62:0/3284407750 wait complete. 2024-09-11T23:43:38.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.835+0000 7f9d40750640 1 Processor -- start 2024-09-11T23:43:38.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.835+0000 7f9d40750640 1 -- start start 2024-09-11T23:43:38.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.835+0000 7f9d40750640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d38102750 0x7f9d381971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:38.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.835+0000 7f9d40750640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3810bcc0 con 0x7f9d38102750 2024-09-11T23:43:38.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.836+0000 7f9d3e4ec640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d38102750 0x7f9d381971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:38.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.836+0000 7f9d3e4ec640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d38102750 0x7f9d381971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56076/0 (socket says 172.21.15.62:56076) 2024-09-11T23:43:38.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.836+0000 7f9d3e4ec640 1 -- 172.21.15.62:0/2439955589 learned_addr learned my addr 172.21.15.62:0/2439955589 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:38.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.836+0000 7f9d3e4ec640 1 -- 172.21.15.62:0/2439955589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d38197710 con 0x7f9d38102750 2024-09-11T23:43:38.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.836+0000 7f9d3e4ec640 1 --2- 172.21.15.62:0/2439955589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d38102750 0x7f9d381971d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9d28030040 tx=0x7f9d2802eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:38.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.836+0000 7f9d377fe640 1 -- 172.21.15.62:0/2439955589 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d28047070 con 0x7f9d38102750 2024-09-11T23:43:38.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.836+0000 7f9d377fe640 1 -- 172.21.15.62:0/2439955589 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d280368e0 con 0x7f9d38102750 2024-09-11T23:43:38.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.836+0000 7f9d377fe640 1 -- 172.21.15.62:0/2439955589 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d28042420 con 0x7f9d38102750 2024-09-11T23:43:38.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.836+0000 7f9d40750640 1 -- 172.21.15.62:0/2439955589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d381979a0 con 0x7f9d38102750 2024-09-11T23:43:38.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.836+0000 7f9d40750640 1 -- 172.21.15.62:0/2439955589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d38197e00 con 0x7f9d38102750 2024-09-11T23:43:38.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.837+0000 7f9d377fe640 1 -- 172.21.15.62:0/2439955589 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9d2804a070 con 0x7f9d38102750 2024-09-11T23:43:38.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.837+0000 7f9d40750640 1 -- 172.21.15.62:0/2439955589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d3806a7b0 con 0x7f9d38102750 2024-09-11T23:43:38.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.837+0000 7f9d377fe640 1 --2- 172.21.15.62:0/2439955589 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d200390c0 0x7f9d2003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:38.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.837+0000 7f9d377fe640 1 -- 172.21.15.62:0/2439955589 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9d2803d030 con 0x7f9d38102750 2024-09-11T23:43:38.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.837+0000 7f9d3dceb640 1 --2- 172.21.15.62:0/2439955589 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d200390c0 0x7f9d2003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:38.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.838+0000 7f9d3dceb640 1 --2- 172.21.15.62:0/2439955589 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d200390c0 0x7f9d2003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0098c0 tx=0x7f9d2c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:38.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:38.842+0000 7f9d377fe640 1 -- 172.21.15.62:0/2439955589 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d28032030 con 0x7f9d38102750 2024-09-11T23:43:39.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.024+0000 7f9d40750640 1 -- 172.21.15.62:0/2439955589 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d38198530 con 0x7f9d200390c0 2024-09-11T23:43:39.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.025+0000 7f9d377fe640 1 -- 172.21.15.62:0/2439955589 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f9d38198530 con 0x7f9d200390c0 2024-09-11T23:43:39.025 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:43:39.025 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}] 2024-09-11T23:43:39.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.028+0000 7f9d40750640 1 -- 172.21.15.62:0/2439955589 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d200390c0 msgr2=0x7f9d2003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.028+0000 7f9d40750640 1 --2- 172.21.15.62:0/2439955589 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d200390c0 0x7f9d2003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0098c0 tx=0x7f9d2c006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.028+0000 7f9d40750640 1 -- 172.21.15.62:0/2439955589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d38102750 msgr2=0x7f9d381971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.028+0000 7f9d40750640 1 --2- 172.21.15.62:0/2439955589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d38102750 0x7f9d381971d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9d28030040 tx=0x7f9d2802eb50 comp rx=0 tx=0).stop 2024-09-11T23:43:39.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.028+0000 7f9d40750640 1 -- 172.21.15.62:0/2439955589 shutdown_connections 2024-09-11T23:43:39.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.028+0000 7f9d40750640 1 --2- 172.21.15.62:0/2439955589 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d200390c0 0x7f9d2003b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.028+0000 7f9d40750640 1 --2- 172.21.15.62:0/2439955589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d38102750 0x7f9d381971d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.028+0000 7f9d40750640 1 -- 172.21.15.62:0/2439955589 >> 172.21.15.62:0/2439955589 conn(0x7f9d380fc1b0 msgr2=0x7f9d38105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:39.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.028+0000 7f9d40750640 1 -- 172.21.15.62:0/2439955589 shutdown_connections 2024-09-11T23:43:39.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:39.028+0000 7f9d40750640 1 -- 172.21.15.62:0/2439955589 wait complete. 2024-09-11T23:43:39.650 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T23:43:39.651 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T23:43:39.900 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:39.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:39 smithi062 ceph-mon[30958]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:39.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:39 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:40.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.325+0000 7f7afa8ba640 1 -- 172.21.15.62:0/2507479365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4102730 msgr2=0x7f7af4102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:40.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.325+0000 7f7afa8ba640 1 --2- 172.21.15.62:0/2507479365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4102730 0x7f7af4102b10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009810 tx=0x7f7ae402a540 comp rx=0 tx=0).stop 2024-09-11T23:43:40.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.325+0000 7f7afa8ba640 1 -- 172.21.15.62:0/2507479365 shutdown_connections 2024-09-11T23:43:40.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.325+0000 7f7afa8ba640 1 --2- 172.21.15.62:0/2507479365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4102730 0x7f7af4102b10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.325+0000 7f7afa8ba640 1 -- 172.21.15.62:0/2507479365 >> 172.21.15.62:0/2507479365 conn(0x7f7af4077310 msgr2=0x7f7af40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:40.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.325+0000 7f7afa8ba640 1 -- 172.21.15.62:0/2507479365 shutdown_connections 2024-09-11T23:43:40.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.325+0000 7f7afa8ba640 1 -- 172.21.15.62:0/2507479365 wait complete. 2024-09-11T23:43:40.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.325+0000 7f7afa8ba640 1 Processor -- start 2024-09-11T23:43:40.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.326+0000 7f7afa8ba640 1 -- start start 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.326+0000 7f7afa8ba640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4102730 0x7f7af4100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.326+0000 7f7afa8ba640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7af410bc80 con 0x7f7af4102730 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.326+0000 7f7af3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4102730 0x7f7af4100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.326+0000 7f7af3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4102730 0x7f7af4100390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56088/0 (socket says 172.21.15.62:56088) 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.326+0000 7f7af3fff640 1 -- 172.21.15.62:0/4197450910 learned_addr learned my addr 172.21.15.62:0/4197450910 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.326+0000 7f7af3fff640 1 -- 172.21.15.62:0/4197450910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af4102280 con 0x7f7af4102730 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.326+0000 7f7af3fff640 1 --2- 172.21.15.62:0/4197450910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4102730 0x7f7af4100390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40045e0 tx=0x7f7ae4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.326+0000 7f7af1ffb640 1 -- 172.21.15.62:0/4197450910 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ae4030e70 con 0x7f7af4102730 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.326+0000 7f7af1ffb640 1 -- 172.21.15.62:0/4197450910 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae40318e0 con 0x7f7af4102730 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.327+0000 7f7afa8ba640 1 -- 172.21.15.62:0/4197450910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af41008d0 con 0x7f7af4102730 2024-09-11T23:43:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.327+0000 7f7af1ffb640 1 -- 172.21.15.62:0/4197450910 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ae40049c0 con 0x7f7af4102730 2024-09-11T23:43:40.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.327+0000 7f7afa8ba640 1 -- 172.21.15.62:0/4197450910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af4100d90 con 0x7f7af4102730 2024-09-11T23:43:40.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.327+0000 7f7af1ffb640 1 -- 172.21.15.62:0/4197450910 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7ae4039040 con 0x7f7af4102730 2024-09-11T23:43:40.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.328+0000 7f7af1ffb640 1 --2- 172.21.15.62:0/4197450910 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ae00390c0 0x7f7ae003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:40.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.328+0000 7f7af1ffb640 1 -- 172.21.15.62:0/4197450910 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7ae406d250 con 0x7f7af4102730 2024-09-11T23:43:40.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.328+0000 7f7aebfff640 1 --2- 172.21.15.62:0/4197450910 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ae00390c0 0x7f7ae003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:40.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.328+0000 7f7afa8ba640 1 -- 172.21.15.62:0/4197450910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af4069f90 con 0x7f7af4102730 2024-09-11T23:43:40.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.328+0000 7f7aebfff640 1 --2- 172.21.15.62:0/4197450910 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ae00390c0 0x7f7ae003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0098c0 tx=0x7f7adc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:40.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.332+0000 7f7af1ffb640 1 -- 172.21.15.62:0/4197450910 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ae4040030 con 0x7f7af4102730 2024-09-11T23:43:40.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.514+0000 7f7afa8ba640 1 -- 172.21.15.62:0/4197450910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7af41018c0 con 0x7f7af4102730 2024-09-11T23:43:40.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.717+0000 7f7af1ffb640 1 -- 172.21.15.62:0/4197450910 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f7ae4037d70 con 0x7f7af4102730 2024-09-11T23:43:40.717 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2024-09-11T23:43:40.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.720+0000 7f7afa8ba640 1 -- 172.21.15.62:0/4197450910 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ae00390c0 msgr2=0x7f7ae003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:40.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.720+0000 7f7afa8ba640 1 --2- 172.21.15.62:0/4197450910 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ae00390c0 0x7f7ae003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0098c0 tx=0x7f7adc006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.720+0000 7f7afa8ba640 1 -- 172.21.15.62:0/4197450910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4102730 msgr2=0x7f7af4100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:40.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.720+0000 7f7afa8ba640 1 --2- 172.21.15.62:0/4197450910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4102730 0x7f7af4100390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40045e0 tx=0x7f7ae4004610 comp rx=0 tx=0).stop 2024-09-11T23:43:40.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.720+0000 7f7afa8ba640 1 -- 172.21.15.62:0/4197450910 shutdown_connections 2024-09-11T23:43:40.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.720+0000 7f7afa8ba640 1 --2- 172.21.15.62:0/4197450910 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ae00390c0 0x7f7ae003b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.720+0000 7f7afa8ba640 1 --2- 172.21.15.62:0/4197450910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4102730 0x7f7af4100390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.720+0000 7f7afa8ba640 1 -- 172.21.15.62:0/4197450910 >> 172.21.15.62:0/4197450910 conn(0x7f7af4077310 msgr2=0x7f7af41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:40.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.720+0000 7f7afa8ba640 1 -- 172.21.15.62:0/4197450910 shutdown_connections 2024-09-11T23:43:40.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:43:40.720+0000 7f7afa8ba640 1 -- 172.21.15.62:0/4197450910 wait complete. 2024-09-11T23:43:41.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:40 smithi062 ceph-mon[30958]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:43:41.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:40 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/4197450910' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T23:43:41.298 INFO:tasks.cephadm:Adding mon.a on smithi062 2024-09-11T23:43:41.298 INFO:tasks.cephadm:Adding mon.b on smithi083 2024-09-11T23:43:41.298 INFO:tasks.cephadm:Adding mon.c on smithi120 2024-09-11T23:43:41.298 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi062:172.21.15.62=a;smithi083:172.21.15.83=b;smithi120:172.21.15.120=c' 2024-09-11T23:43:41.666 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T23:43:41.711 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T23:43:42.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.179+0000 7f9bb03e2640 1 -- 172.21.15.120:0/470762453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8100540 msgr2=0x7f9ba8100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:42.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.179+0000 7f9bb03e2640 1 --2- 172.21.15.120:0/470762453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8100540 0x7f9ba8100940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009880 tx=0x7f9ba402a5d0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.179+0000 7f9bb03e2640 1 -- 172.21.15.120:0/470762453 shutdown_connections 2024-09-11T23:43:42.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.179+0000 7f9bb03e2640 1 --2- 172.21.15.120:0/470762453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8100540 0x7f9ba8100940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.179+0000 7f9bb03e2640 1 -- 172.21.15.120:0/470762453 >> 172.21.15.120:0/470762453 conn(0x7f9ba80fbcd0 msgr2=0x7f9ba80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:42.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.179+0000 7f9bb03e2640 1 -- 172.21.15.120:0/470762453 shutdown_connections 2024-09-11T23:43:42.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.179+0000 7f9bb03e2640 1 -- 172.21.15.120:0/470762453 wait complete. 2024-09-11T23:43:42.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.180+0000 7f9bb03e2640 1 Processor -- start 2024-09-11T23:43:42.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.180+0000 7f9bb03e2640 1 -- start start 2024-09-11T23:43:42.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.180+0000 7f9bb03e2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8100540 0x7f9ba80fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:42.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.180+0000 7f9bb03e2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba81065f0 con 0x7f9ba8100540 2024-09-11T23:43:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.181+0000 7f9bae17e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8100540 0x7f9ba80fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.181+0000 7f9bae17e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8100540 0x7f9ba80fdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.120:34328/0 (socket says 172.21.15.120:34328) 2024-09-11T23:43:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.181+0000 7f9bae17e640 1 -- 172.21.15.120:0/3537565640 learned_addr learned my addr 172.21.15.120:0/3537565640 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:43:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.181+0000 7f9bae17e640 1 -- 172.21.15.120:0/3537565640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba80fe0d0 con 0x7f9ba8100540 2024-09-11T23:43:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.181+0000 7f9bae17e640 1 --2- 172.21.15.120:0/3537565640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8100540 0x7f9ba80fdb90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4004f60 tx=0x7f9ba40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.181+0000 7f9b9f7fe640 1 -- 172.21.15.120:0/3537565640 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ba403f070 con 0x7f9ba8100540 2024-09-11T23:43:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.181+0000 7f9b9f7fe640 1 -- 172.21.15.120:0/3537565640 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ba4004d40 con 0x7f9ba8100540 2024-09-11T23:43:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.181+0000 7f9b9f7fe640 1 -- 172.21.15.120:0/3537565640 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ba4035030 con 0x7f9ba8100540 2024-09-11T23:43:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.181+0000 7f9bb03e2640 1 -- 172.21.15.120:0/3537565640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba80fc2c0 con 0x7f9ba8100540 2024-09-11T23:43:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.181+0000 7f9bb03e2640 1 -- 172.21.15.120:0/3537565640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba80fc800 con 0x7f9ba8100540 2024-09-11T23:43:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.182+0000 7f9b9f7fe640 1 -- 172.21.15.120:0/3537565640 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9ba4042070 con 0x7f9ba8100540 2024-09-11T23:43:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.182+0000 7f9b9f7fe640 1 --2- 172.21.15.120:0/3537565640 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9b940390c0 0x7f9b9403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.182+0000 7f9b9f7fe640 1 -- 172.21.15.120:0/3537565640 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9ba406cda0 con 0x7f9ba8100540 2024-09-11T23:43:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.182+0000 7f9bb03e2640 1 -- 172.21.15.120:0/3537565640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba8106d70 con 0x7f9ba8100540 2024-09-11T23:43:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.183+0000 7f9bad97d640 1 --2- 172.21.15.120:0/3537565640 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9b940390c0 0x7f9b9403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.183+0000 7f9bad97d640 1 --2- 172.21.15.120:0/3537565640 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9b940390c0 0x7f9b9403b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9b980098c0 tx=0x7f9b98006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.188+0000 7f9b9f7fe640 1 -- 172.21.15.120:0/3537565640 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ba4042360 con 0x7f9ba8100540 2024-09-11T23:43:42.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:41 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/4197450910' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T23:43:42.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:41 smithi062 ceph-mon[30958]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:43:42.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:41 smithi062 ceph-mon[30958]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:42.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:41 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:42.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:41 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:42.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:41 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:42.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:41 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:42.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:41 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:43:42.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:41 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:42.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:41 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:43:42.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.382+0000 7f9bb03e2640 1 -- 172.21.15.120:0/3537565640 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi083:172.21.15.83=b;smithi120:172.21.15.120=c", "target": ["mon-mgr", ""]}) -- 0x7f9ba81903c0 con 0x7f9b940390c0 2024-09-11T23:43:42.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.387+0000 7f9b9f7fe640 1 -- 172.21.15.120:0/3537565640 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9ba81903c0 con 0x7f9b940390c0 2024-09-11T23:43:42.387 INFO:teuthology.orchestra.run.smithi120.stdout:Scheduled mon update... 2024-09-11T23:43:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.389+0000 7f9bb03e2640 1 -- 172.21.15.120:0/3537565640 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9b940390c0 msgr2=0x7f9b9403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.389+0000 7f9bb03e2640 1 --2- 172.21.15.120:0/3537565640 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9b940390c0 0x7f9b9403b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9b980098c0 tx=0x7f9b98006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.389+0000 7f9bb03e2640 1 -- 172.21.15.120:0/3537565640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8100540 msgr2=0x7f9ba80fdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.389+0000 7f9bb03e2640 1 --2- 172.21.15.120:0/3537565640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8100540 0x7f9ba80fdb90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4004f60 tx=0x7f9ba40048f0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.389+0000 7f9bb03e2640 1 -- 172.21.15.120:0/3537565640 shutdown_connections 2024-09-11T23:43:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.389+0000 7f9bb03e2640 1 --2- 172.21.15.120:0/3537565640 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9b940390c0 0x7f9b9403b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.389+0000 7f9bb03e2640 1 --2- 172.21.15.120:0/3537565640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba8100540 0x7f9ba80fdb90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.389+0000 7f9bb03e2640 1 -- 172.21.15.120:0/3537565640 >> 172.21.15.120:0/3537565640 conn(0x7f9ba80fbcd0 msgr2=0x7f9ba8069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.389+0000 7f9bb03e2640 1 -- 172.21.15.120:0/3537565640 shutdown_connections 2024-09-11T23:43:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:42.389+0000 7f9bb03e2640 1 -- 172.21.15.120:0/3537565640 wait complete. 2024-09-11T23:43:43.010 DEBUG:teuthology.orchestra.run.smithi083:mon.b> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T23:43:43.013 DEBUG:teuthology.orchestra.run.smithi120:mon.c> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T23:43:43.016 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:43:43.016 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:43:43.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:42 smithi062 ceph-mon[30958]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-11T23:43:43.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:43:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:44.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:43 smithi062 ceph-mon[30958]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:43:44.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:43 smithi062 ceph-mon[30958]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:43:44.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:43 smithi062 ceph-mon[30958]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:43:44.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:43 smithi062 ceph-mon[30958]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi083:172.21.15.83=b;smithi120:172.21.15.120=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:43:44.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:43 smithi062 ceph-mon[30958]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi083:172.21.15.83=b;smithi120:172.21.15.120=c;count:3 2024-09-11T23:43:44.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:43 smithi062 ceph-mon[30958]: Deploying daemon mon.c on smithi120 2024-09-11T23:43:44.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:43 smithi062 ceph-mon[30958]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:45.141 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:43:45.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:44 smithi062 ceph-mon[30958]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.650+0000 7f8f937ea640 1 -- 172.21.15.120:0/3364202992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8c1066c0 msgr2=0x7f8f8c106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.650+0000 7f8f937ea640 1 --2- 172.21.15.120:0/3364202992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8c1066c0 0x7f8f8c106aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009880 tx=0x7f8f8802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.650+0000 7f8f937ea640 1 -- 172.21.15.120:0/3364202992 shutdown_connections 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.650+0000 7f8f937ea640 1 --2- 172.21.15.120:0/3364202992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8c1066c0 0x7f8f8c106aa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.650+0000 7f8f937ea640 1 -- 172.21.15.120:0/3364202992 >> 172.21.15.120:0/3364202992 conn(0x7f8f8c0fb9a0 msgr2=0x7f8f8c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.650+0000 7f8f937ea640 1 -- 172.21.15.120:0/3364202992 shutdown_connections 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.650+0000 7f8f937ea640 1 -- 172.21.15.120:0/3364202992 wait complete. 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.650+0000 7f8f937ea640 1 Processor -- start 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.650+0000 7f8f937ea640 1 -- start start 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.651+0000 7f8f937ea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8c1066c0 0x7f8f8c0fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:45.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.651+0000 7f8f937ea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c10ac50 con 0x7f8f8c1066c0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.651+0000 7f8f91586640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8c1066c0 0x7f8f8c0fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.651+0000 7f8f91586640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8c1066c0 0x7f8f8c0fbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.120:34356/0 (socket says 172.21.15.120:34356) 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.651+0000 7f8f91586640 1 -- 172.21.15.120:0/929514310 learned_addr learned my addr 172.21.15.120:0/929514310 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.651+0000 7f8f91586640 1 -- 172.21.15.120:0/929514310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c0fdd40 con 0x7f8f8c1066c0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.652+0000 7f8f91586640 1 --2- 172.21.15.120:0/929514310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8c1066c0 0x7f8f8c0fbe50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009230 tx=0x7f8f88003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.652+0000 7f8f827fc640 1 -- 172.21.15.120:0/929514310 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f8803f070 con 0x7f8f8c1066c0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.652+0000 7f8f827fc640 1 -- 172.21.15.120:0/929514310 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f8803bc80 con 0x7f8f8c1066c0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.652+0000 7f8f937ea640 1 -- 172.21.15.120:0/929514310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c0fc390 con 0x7f8f8c1066c0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.652+0000 7f8f827fc640 1 -- 172.21.15.120:0/929514310 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f88035030 con 0x7f8f8c1066c0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.652+0000 7f8f937ea640 1 -- 172.21.15.120:0/929514310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c0fc850 con 0x7f8f8c1066c0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.653+0000 7f8f827fc640 1 -- 172.21.15.120:0/929514310 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8f88042070 con 0x7f8f8c1066c0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.653+0000 7f8f937ea640 1 -- 172.21.15.120:0/929514310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8c101cd0 con 0x7f8f8c1066c0 2024-09-11T23:43:45.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.653+0000 7f8f827fc640 1 --2- 172.21.15.120:0/929514310 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8f780390c0 0x7f8f7803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:45.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.653+0000 7f8f827fc640 1 -- 172.21.15.120:0/929514310 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f8806ce90 con 0x7f8f8c1066c0 2024-09-11T23:43:45.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.653+0000 7f8f90d85640 1 --2- 172.21.15.120:0/929514310 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8f780390c0 0x7f8f7803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:45.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.654+0000 7f8f90d85640 1 --2- 172.21.15.120:0/929514310 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8f780390c0 0x7f8f7803b580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0098c0 tx=0x7f8f7c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:45.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.658+0000 7f8f827fc640 1 -- 172.21.15.120:0/929514310 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f880314c0 con 0x7f8f8c1066c0 2024-09-11T23:43:45.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.910+0000 7f8f937ea640 1 -- 172.21.15.120:0/929514310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8f8c0fd380 con 0x7f8f8c1066c0 2024-09-11T23:43:45.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.911+0000 7f8f827fc640 1 -- 172.21.15.120:0/929514310 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f8f8803a890 con 0x7f8f8c1066c0 2024-09-11T23:43:45.911 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:43:45.911 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":1,"fsid":"56675698-7097-11ef-bcea-c7b262605968","modified":"2024-09-11T23:42:14.107883Z","created":"2024-09-11T23:42:14.107883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T23:43:45.911 INFO:teuthology.orchestra.run.smithi120.stderr:dumped monmap epoch 1 2024-09-11T23:43:45.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.913+0000 7f8f937ea640 1 -- 172.21.15.120:0/929514310 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8f780390c0 msgr2=0x7f8f7803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:45.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.913+0000 7f8f937ea640 1 --2- 172.21.15.120:0/929514310 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8f780390c0 0x7f8f7803b580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0098c0 tx=0x7f8f7c006db0 comp rx=0 tx=0).stop 2024-09-11T23:43:45.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.913+0000 7f8f937ea640 1 -- 172.21.15.120:0/929514310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8c1066c0 msgr2=0x7f8f8c0fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:45.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.913+0000 7f8f937ea640 1 --2- 172.21.15.120:0/929514310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8c1066c0 0x7f8f8c0fbe50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009230 tx=0x7f8f88003d40 comp rx=0 tx=0).stop 2024-09-11T23:43:45.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.913+0000 7f8f937ea640 1 -- 172.21.15.120:0/929514310 shutdown_connections 2024-09-11T23:43:45.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.913+0000 7f8f937ea640 1 --2- 172.21.15.120:0/929514310 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8f780390c0 0x7f8f7803b580 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:45.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.913+0000 7f8f937ea640 1 --2- 172.21.15.120:0/929514310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8c1066c0 0x7f8f8c0fbe50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:45.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.913+0000 7f8f937ea640 1 -- 172.21.15.120:0/929514310 >> 172.21.15.120:0/929514310 conn(0x7f8f8c0fb9a0 msgr2=0x7f8f8c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:45.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.913+0000 7f8f937ea640 1 -- 172.21.15.120:0/929514310 shutdown_connections 2024-09-11T23:43:45.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:45.913+0000 7f8f937ea640 1 -- 172.21.15.120:0/929514310 wait complete. 2024-09-11T23:43:46.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:45 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/929514310' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:43:47.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:46 smithi120 systemd[1]: Started Ceph mon.c for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:43:47.481 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:43:47.482 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:43:47.733 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:43:52.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:43:52.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: mon.a calling monitor election 2024-09-11T23:43:52.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:52.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: mon.c calling monitor election 2024-09-11T23:43:52.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:52.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: monmap epoch 2 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: last_changed 2024-09-11T23:43:46.961907+0000 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: min_mon_release 19 (squid) 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: election_strategy: 1 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: 1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.c 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: fsmap 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: mgrmap e13: x(active, since 39s) 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: overall HEALTH_OK 2024-09-11T23:43:52.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:52.374 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:52.374 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:52.374 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:43:52.374 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:52.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:43:52.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: mon.a calling monitor election 2024-09-11T23:43:52.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:52.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: mon.c calling monitor election 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: monmap epoch 2 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: last_changed 2024-09-11T23:43:46.961907+0000 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: min_mon_release 19 (squid) 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: election_strategy: 1 2024-09-11T23:43:52.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:43:52.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: 1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.c 2024-09-11T23:43:52.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: fsmap 2024-09-11T23:43:52.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:43:52.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: mgrmap e13: x(active, since 39s) 2024-09-11T23:43:52.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: overall HEALTH_OK 2024-09-11T23:43:52.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:52.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:52.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:52.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:43:52.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:52.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.575+0000 7fa38992a640 1 -- 172.21.15.120:0/3585549278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa36c003530 msgr2=0x7fa36c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:52.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.575+0000 7fa38992a640 1 --2- 172.21.15.120:0/3585549278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa36c003530 0x7fa36c0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa3841016c0 tx=0x7fa37402c490 comp rx=0 tx=0).stop 2024-09-11T23:43:52.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.575+0000 7fa38992a640 1 -- 172.21.15.120:0/3585549278 shutdown_connections 2024-09-11T23:43:52.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.575+0000 7fa38992a640 1 --2- 172.21.15.120:0/3585549278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa36c003530 0x7fa36c0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:52.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.575+0000 7fa38992a640 1 -- 172.21.15.120:0/3585549278 >> 172.21.15.120:0/3585549278 conn(0x7fa3840fc280 msgr2=0x7fa3840fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:52.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.575+0000 7fa38992a640 1 -- 172.21.15.120:0/3585549278 shutdown_connections 2024-09-11T23:43:52.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.575+0000 7fa38992a640 1 -- 172.21.15.120:0/3585549278 wait complete. 2024-09-11T23:43:52.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.576+0000 7fa38992a640 1 Processor -- start 2024-09-11T23:43:52.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa38992a640 1 -- start start 2024-09-11T23:43:52.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa38992a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c003530 0x7fa384198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:52.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa38992a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa384198690 0x7fa384198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:52.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa38992a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa384108690 con 0x7fa384198690 2024-09-11T23:43:52.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa38992a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa384108510 con 0x7fa36c003530 2024-09-11T23:43:52.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa382ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c003530 0x7fa384198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:52.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa382ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c003530 0x7fa384198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49254/0 (socket says 172.21.15.120:49254) 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa382ffd640 1 -- 172.21.15.120:0/2539491319 learned_addr learned my addr 172.21.15.120:0/2539491319 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa382ffd640 1 -- 172.21.15.120:0/2539491319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c003530 msgr2=0x7fa384198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa382ffd640 1 -- 172.21.15.120:0/2539491319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c003530 msgr2=0x7fa384198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa382ffd640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c003530 0x7fa384198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa382ffd640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c003530 0x7fa384198150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.577+0000 7fa3827fc640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa384198690 0x7fa384198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.578+0000 7fa3827fc640 1 -- 172.21.15.120:0/2539491319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c003530 msgr2=0x7fa384198150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.578+0000 7fa3827fc640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c003530 0x7fa384198150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.578+0000 7fa3827fc640 1 -- 172.21.15.120:0/2539491319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3841a11c0 con 0x7fa384198690 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.578+0000 7fa3827fc640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa384198690 0x7fa384198af0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa37800b400 tx=0x7fa37800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:52.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.578+0000 7fa388928640 1 -- 172.21.15.120:0/2539491319 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa378011630 con 0x7fa384198690 2024-09-11T23:43:52.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.578+0000 7fa388928640 1 -- 172.21.15.120:0/2539491319 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa378011cb0 con 0x7fa384198690 2024-09-11T23:43:52.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.578+0000 7fa38992a640 1 -- 172.21.15.120:0/2539491319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3841a1390 con 0x7fa384198690 2024-09-11T23:43:52.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.578+0000 7fa388928640 1 -- 172.21.15.120:0/2539491319 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa378010d80 con 0x7fa384198690 2024-09-11T23:43:52.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.578+0000 7fa38992a640 1 -- 172.21.15.120:0/2539491319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3841a18f0 con 0x7fa384198690 2024-09-11T23:43:52.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.579+0000 7fa388928640 1 -- 172.21.15.120:0/2539491319 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa378018020 con 0x7fa384198690 2024-09-11T23:43:52.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.580+0000 7fa388928640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa370038e50 0x7fa37003b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:52.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.580+0000 7fa38992a640 1 -- 172.21.15.120:0/2539491319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa384101cc0 con 0x7fa384198690 2024-09-11T23:43:52.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.580+0000 7fa388928640 1 -- 172.21.15.120:0/2539491319 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa37804cce0 con 0x7fa384198690 2024-09-11T23:43:52.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.580+0000 7fa382ffd640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa370038e50 0x7fa37003b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:52.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.580+0000 7fa382ffd640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa370038e50 0x7fa37003b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa374012c10 tx=0x7fa3740323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:52.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.585+0000 7fa388928640 1 -- 172.21.15.120:0/2539491319 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa37801f020 con 0x7fa384198690 2024-09-11T23:43:52.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.816+0000 7fa38992a640 1 -- 172.21.15.120:0/2539491319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa384109840 con 0x7fa384198690 2024-09-11T23:43:52.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.816+0000 7fa388928640 1 -- 172.21.15.120:0/2539491319 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa378018310 con 0x7fa384198690 2024-09-11T23:43:52.817 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:43:52.817 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":2,"fsid":"56675698-7097-11ef-bcea-c7b262605968","modified":"2024-09-11T23:43:46.961907Z","created":"2024-09-11T23:42:14.107883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:43:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:dumped monmap epoch 2 2024-09-11T23:43:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 -- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa370038e50 msgr2=0x7fa37003b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa370038e50 0x7fa37003b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa374012c10 tx=0x7fa3740323a0 comp rx=0 tx=0).stop 2024-09-11T23:43:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 -- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa384198690 msgr2=0x7fa384198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa384198690 0x7fa384198af0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa37800b400 tx=0x7fa37800b8c0 comp rx=0 tx=0).stop 2024-09-11T23:43:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 -- 172.21.15.120:0/2539491319 shutdown_connections 2024-09-11T23:43:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa370038e50 0x7fa37003b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa384198690 0x7fa384198af0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 --2- 172.21.15.120:0/2539491319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c003530 0x7fa384198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 -- 172.21.15.120:0/2539491319 >> 172.21.15.120:0/2539491319 conn(0x7fa3840fc280 msgr2=0x7fa3840fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 -- 172.21.15.120:0/2539491319 shutdown_connections 2024-09-11T23:43:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:52.818+0000 7fa38992a640 1 -- 172.21.15.120:0/2539491319 wait complete. 2024-09-11T23:43:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:53 smithi120 ceph-mon[35182]: Deploying daemon mon.b on smithi083 2024-09-11T23:43:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:53 smithi120 ceph-mon[35182]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:53 smithi120 ceph-mon[35182]: from='client.? 172.21.15.120:0/2539491319' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:43:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:53.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:53 smithi062 ceph-mon[30958]: Deploying daemon mon.b on smithi083 2024-09-11T23:43:53.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:53 smithi062 ceph-mon[30958]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:53.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:53 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/2539491319' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:43:53.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:54.445 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:43:54.445 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:43:54.698 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:43:55.038 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:43:54 smithi120 ceph-mon[35182]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:55.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.118+0000 7f167ed07640 1 -- 172.21.15.120:0/3216049015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1678106630 msgr2=0x7f1678106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.118+0000 7f167ed07640 1 --2- 172.21.15.120:0/3216049015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1678106630 0x7f1678106a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1670009880 tx=0x7f167002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:43:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.118+0000 7f167ed07640 1 -- 172.21.15.120:0/3216049015 shutdown_connections 2024-09-11T23:43:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.118+0000 7f167ed07640 1 --2- 172.21.15.120:0/3216049015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1678106630 0x7f1678106a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.118+0000 7f167ed07640 1 -- 172.21.15.120:0/3216049015 >> 172.21.15.120:0/3216049015 conn(0x7f16780fc220 msgr2=0x7f16780fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.118+0000 7f167ed07640 1 -- 172.21.15.120:0/3216049015 shutdown_connections 2024-09-11T23:43:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.118+0000 7f167ed07640 1 -- 172.21.15.120:0/3216049015 wait complete. 2024-09-11T23:43:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f167ed07640 1 Processor -- start 2024-09-11T23:43:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f167ed07640 1 -- start start 2024-09-11T23:43:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f167ed07640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1678106630 0x7f16781962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f167ed07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1678196820 0x7f167819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f167ed07640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f167810ac60 con 0x7f1678106630 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f167ed07640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f167810aae0 con 0x7f1678196820 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f1677fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1678196820 0x7f167819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f1677fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1678196820 0x7f167819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49270/0 (socket says 172.21.15.120:49270) 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f1677fff640 1 -- 172.21.15.120:0/1852066582 learned_addr learned my addr 172.21.15.120:0/1852066582 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f1677fff640 1 -- 172.21.15.120:0/1852066582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1678196820 msgr2=0x7f167819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f1677fff640 1 -- 172.21.15.120:0/1852066582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1678196820 msgr2=0x7f167819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f1677fff640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1678196820 0x7f167819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f1677fff640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1678196820 0x7f167819abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.119+0000 7f167caa3640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1678106630 0x7f16781962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.120+0000 7f167caa3640 1 -- 172.21.15.120:0/1852066582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1678196820 msgr2=0x7f167819abb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.120+0000 7f167caa3640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1678196820 0x7f167819abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.120+0000 7f167caa3640 1 -- 172.21.15.120:0/1852066582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1678196e40 con 0x7f1678106630 2024-09-11T23:43:55.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.120+0000 7f167caa3640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1678106630 0x7f16781962e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1670005ab0 tx=0x7f1670032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:55.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.120+0000 7f1675ffb640 1 -- 172.21.15.120:0/1852066582 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1670035070 con 0x7f1678106630 2024-09-11T23:43:55.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.120+0000 7f1675ffb640 1 -- 172.21.15.120:0/1852066582 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1670032e40 con 0x7f1678106630 2024-09-11T23:43:55.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.120+0000 7f1675ffb640 1 -- 172.21.15.120:0/1852066582 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f167003bc10 con 0x7f1678106630 2024-09-11T23:43:55.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.120+0000 7f167ed07640 1 -- 172.21.15.120:0/1852066582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167819b0f0 con 0x7f1678106630 2024-09-11T23:43:55.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.121+0000 7f167ed07640 1 -- 172.21.15.120:0/1852066582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167819b5d0 con 0x7f1678106630 2024-09-11T23:43:55.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.122+0000 7f1675ffb640 1 -- 172.21.15.120:0/1852066582 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1670042030 con 0x7f1678106630 2024-09-11T23:43:55.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.122+0000 7f167ed07640 1 -- 172.21.15.120:0/1852066582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1678101ca0 con 0x7f1678106630 2024-09-11T23:43:55.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.122+0000 7f1675ffb640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1664038e00 0x7f166403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:55.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.122+0000 7f1675ffb640 1 -- 172.21.15.120:0/1852066582 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1670040070 con 0x7f1678106630 2024-09-11T23:43:55.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.122+0000 7f1677fff640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1664038e00 0x7f166403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:55.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.122+0000 7f1677fff640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1664038e00 0x7f166403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1660009100 tx=0x7f1660008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:55.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.127+0000 7f1675ffb640 1 -- 172.21.15.120:0/1852066582 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f167004a260 con 0x7f1678106630 2024-09-11T23:43:55.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:43:54 smithi062 ceph-mon[30958]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:55.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.356+0000 7f167ed07640 1 -- 172.21.15.120:0/1852066582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f167819b8b0 con 0x7f1678106630 2024-09-11T23:43:55.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.357+0000 7f1675ffb640 1 -- 172.21.15.120:0/1852066582 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f167003bdb0 con 0x7f1678106630 2024-09-11T23:43:55.357 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:43:55.357 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":2,"fsid":"56675698-7097-11ef-bcea-c7b262605968","modified":"2024-09-11T23:43:46.961907Z","created":"2024-09-11T23:42:14.107883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:43:55.358 INFO:teuthology.orchestra.run.smithi120.stderr:dumped monmap epoch 2 2024-09-11T23:43:55.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 -- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1664038e00 msgr2=0x7f166403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:55.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1664038e00 0x7f166403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1660009100 tx=0x7f1660008040 comp rx=0 tx=0).stop 2024-09-11T23:43:55.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 -- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1678106630 msgr2=0x7f16781962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:55.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1678106630 0x7f16781962e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1670005ab0 tx=0x7f1670032c50 comp rx=0 tx=0).stop 2024-09-11T23:43:55.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 -- 172.21.15.120:0/1852066582 shutdown_connections 2024-09-11T23:43:55.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1664038e00 0x7f166403b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:55.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1678196820 0x7f167819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:55.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 --2- 172.21.15.120:0/1852066582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1678106630 0x7f16781962e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:55.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 -- 172.21.15.120:0/1852066582 >> 172.21.15.120:0/1852066582 conn(0x7f16780fc220 msgr2=0x7f16780fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:55.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 -- 172.21.15.120:0/1852066582 shutdown_connections 2024-09-11T23:43:55.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:43:55.359+0000 7f167ed07640 1 -- 172.21.15.120:0/1852066582 wait complete. 2024-09-11T23:43:55.756 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-11T23:43:55.756 INFO:journalctl@ceph.mon.b.smithi083.stdout: e1 2024-09-11T23:43:55.756 INFO:journalctl@ceph.mon.b.smithi083.stdout: btime 2024-09-11T23:42:20:574216+0000 2024-09-11T23:43:55.756 INFO:journalctl@ceph.mon.b.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T23:43:55.756 INFO:journalctl@ceph.mon.b.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:43:55.757 INFO:journalctl@ceph.mon.b.smithi083.stdout: legacy client fscid: -1 2024-09-11T23:43:55.757 INFO:journalctl@ceph.mon.b.smithi083.stdout: 2024-09-11T23:43:55.757 INFO:journalctl@ceph.mon.b.smithi083.stdout: No filesystems configured 2024-09-11T23:43:55.757 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T23:43:55.757 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T23:43:55.757 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T23:43:55.757 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T23:43:55.757 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T23:43:55.757 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T23:43:56.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T23:43:56.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:43:56.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:43:56.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:43:56.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: Added host smithi120 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/4197450910' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/4197450910' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:43:56.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:43:56.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:56.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:43:56.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:43:56.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:43:56.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi083:172.21.15.83=b;smithi120:172.21.15.120=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:43:56.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi083:172.21.15.83=b;smithi120:172.21.15.120=c;count:3 2024-09-11T23:43:56.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: Deploying daemon mon.c on smithi120 2024-09-11T23:43:56.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/929514310' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.a calling monitor election 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.c calling monitor election 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:56.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: monmap epoch 2 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: last_changed 2024-09-11T23:43:46.961907+0000 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: min_mon_release 19 (squid) 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: election_strategy: 1 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: 1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.c 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: fsmap 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mgrmap e13: x(active, since 39s) 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: overall HEALTH_OK 2024-09-11T23:43:56.172 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: Deploying daemon mon.b on smithi083 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/2539491319' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 ceph-mon[34805]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 bash[34787]: 123c1129f1957ab1d9ef94d9d8711ad0e6d4e6607ee44f48f674c80e1f4afce2 2024-09-11T23:43:56.173 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:43:55 smithi083 systemd[1]: Started Ceph mon.b for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:43:56.936 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:43:56.936 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:43:57.187 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:44:00.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.858+0000 7f8e01b53640 1 -- 172.21.15.120:0/3292968828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8de4003530 msgr2=0x7f8de40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:00.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.858+0000 7f8e01b53640 1 --2- 172.21.15.120:0/3292968828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8de4003530 0x7f8de40059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc104d30 tx=0x7f8dec02c150 comp rx=0 tx=0).stop 2024-09-11T23:44:00.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.858+0000 7f8e01b53640 1 -- 172.21.15.120:0/3292968828 shutdown_connections 2024-09-11T23:44:00.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.858+0000 7f8e01b53640 1 --2- 172.21.15.120:0/3292968828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8de4003530 0x7f8de40059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.858+0000 7f8e01b53640 1 -- 172.21.15.120:0/3292968828 >> 172.21.15.120:0/3292968828 conn(0x7f8dfc0ff5c0 msgr2=0x7f8dfc1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:00.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.858+0000 7f8e01b53640 1 -- 172.21.15.120:0/3292968828 shutdown_connections 2024-09-11T23:44:00.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.858+0000 7f8e01b53640 1 -- 172.21.15.120:0/3292968828 wait complete. 2024-09-11T23:44:00.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.859+0000 7f8e01b53640 1 Processor -- start 2024-09-11T23:44:00.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.859+0000 7f8e01b53640 1 -- start start 2024-09-11T23:44:00.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.859+0000 7f8e01b53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de4003530 0x7f8dfc1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:00.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.859+0000 7f8e01b53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dfc199a20 0x7f8dfc19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:00.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.859+0000 7f8e01b53640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dfc19e2f0 0x7f8dfc19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:00.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.859+0000 7f8e01b53640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10b960 con 0x7f8dfc19e2f0 2024-09-11T23:44:00.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.859+0000 7f8e01b53640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10b7e0 con 0x7f8de4003530 2024-09-11T23:44:00.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.859+0000 7f8e01b53640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10bae0 con 0x7f8dfc199a20 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dfc19e2f0 0x7f8dfc19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dfc199a20 0x7f8dfc19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dfc19e2f0 0x7f8dfc19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.120:56674/0 (socket says 172.21.15.120:56674) 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfbfff640 1 -- 172.21.15.120:0/2034702986 learned_addr learned my addr 172.21.15.120:0/2034702986 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfb7fe640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de4003530 0x7f8dfc1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfaffd640 1 -- 172.21.15.120:0/2034702986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dfc199a20 msgr2=0x7f8dfc19ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfaffd640 1 -- 172.21.15.120:0/2034702986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dfc199a20 msgr2=0x7f8dfc19ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfaffd640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dfc199a20 0x7f8dfc19ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfaffd640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dfc199a20 0x7f8dfc19ddb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfb7fe640 1 -- 172.21.15.120:0/2034702986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de4003530 msgr2=0x7f8dfc1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfb7fe640 1 -- 172.21.15.120:0/2034702986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de4003530 msgr2=0x7f8dfc1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfb7fe640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de4003530 0x7f8dfc1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:44:00.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfbfff640 1 -- 172.21.15.120:0/2034702986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dfc199a20 msgr2=0x7f8dfc19ddb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfb7fe640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de4003530 0x7f8dfc1994e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfbfff640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dfc199a20 0x7f8dfc19ddb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfbfff640 1 -- 172.21.15.120:0/2034702986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de4003530 msgr2=0x7f8dfc1994e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfbfff640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de4003530 0x7f8dfc1994e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfbfff640 1 -- 172.21.15.120:0/2034702986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc1a6060 con 0x7f8dfc19e2f0 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.860+0000 7f8dfbfff640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dfc19e2f0 0x7f8dfc19e7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8df400ba50 tx=0x7f8df400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.861+0000 7f8df8ff9640 1 -- 172.21.15.120:0/2034702986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4019070 con 0x7f8dfc19e2f0 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.861+0000 7f8df8ff9640 1 -- 172.21.15.120:0/2034702986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df400cbe0 con 0x7f8dfc19e2f0 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.861+0000 7f8e01b53640 1 -- 172.21.15.120:0/2034702986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dfc1a6290 con 0x7f8dfc19e2f0 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.861+0000 7f8df8ff9640 1 -- 172.21.15.120:0/2034702986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df40076a0 con 0x7f8dfc19e2f0 2024-09-11T23:44:00.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.861+0000 7f8e01b53640 1 -- 172.21.15.120:0/2034702986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dfc1a6750 con 0x7f8dfc19e2f0 2024-09-11T23:44:00.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.862+0000 7f8df8ff9640 1 -- 172.21.15.120:0/2034702986 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8df401c070 con 0x7f8dfc19e2f0 2024-09-11T23:44:00.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.862+0000 7f8df8ff9640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8de80394f0 0x7f8de803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:00.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.862+0000 7f8df8ff9640 1 -- 172.21.15.120:0/2034702986 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8df404bd80 con 0x7f8dfc19e2f0 2024-09-11T23:44:00.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.862+0000 7f8e01b53640 1 -- 172.21.15.120:0/2034702986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dfc10ca90 con 0x7f8dfc19e2f0 2024-09-11T23:44:00.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.862+0000 7f8dfb7fe640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8de80394f0 0x7f8de803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:00.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.863+0000 7f8dfb7fe640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8de80394f0 0x7f8de803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8dec005c80 tx=0x7f8dec005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:00.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:00.867+0000 7f8df8ff9640 1 -- 172.21.15.120:0/2034702986 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8df400e0a0 con 0x7f8dfc19e2f0 2024-09-11T23:44:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.111+0000 7f8e01b53640 1 -- 172.21.15.120:0/2034702986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8dfc108290 con 0x7f8dfc19e2f0 2024-09-11T23:44:01.112 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:44:01.112 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.112 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:44:01.112 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: mon.a calling monitor election 2024-09-11T23:44:01.112 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: mon.c calling monitor election 2024-09-11T23:44:01.112 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:01.112 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.112 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.112 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: monmap epoch 3 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: last_changed 2024-09-11T23:43:55.763033+0000 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: min_mon_release 19 (squid) 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: election_strategy: 1 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:44:01.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: 1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.c 2024-09-11T23:44:01.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-09-11T23:44:01.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: fsmap 2024-09-11T23:44:01.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:44:01.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: mgrmap e13: x(active, since 48s) 2024-09-11T23:44:01.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: overall HEALTH_OK 2024-09-11T23:44:01.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:01.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:01.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:01.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.112+0000 7f8df8ff9640 1 -- 172.21.15.120:0/2034702986 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f8df40112a0 con 0x7f8dfc19e2f0 2024-09-11T23:44:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:44:01.115 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":3,"fsid":"56675698-7097-11ef-bcea-c7b262605968","modified":"2024-09-11T23:43:55.763033Z","created":"2024-09-11T23:42:14.107883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:44:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:dumped monmap epoch 3 2024-09-11T23:44:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 -- 172.21.15.120:0/2034702986 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8de80394f0 msgr2=0x7f8de803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8de80394f0 0x7f8de803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8dec005c80 tx=0x7f8dec005d10 comp rx=0 tx=0).stop 2024-09-11T23:44:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 -- 172.21.15.120:0/2034702986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dfc19e2f0 msgr2=0x7f8dfc19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dfc19e2f0 0x7f8dfc19e7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8df400ba50 tx=0x7f8df400bf10 comp rx=0 tx=0).stop 2024-09-11T23:44:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 -- 172.21.15.120:0/2034702986 shutdown_connections 2024-09-11T23:44:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8de80394f0 0x7f8de803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dfc19e2f0 0x7f8dfc19e7a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dfc199a20 0x7f8dfc19ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 --2- 172.21.15.120:0/2034702986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de4003530 0x7f8dfc1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 -- 172.21.15.120:0/2034702986 >> 172.21.15.120:0/2034702986 conn(0x7f8dfc0ff5c0 msgr2=0x7f8dfc1015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 -- 172.21.15.120:0/2034702986 shutdown_connections 2024-09-11T23:44:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:01.115+0000 7f8e01b53640 1 -- 172.21.15.120:0/2034702986 wait complete. 2024-09-11T23:44:01.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:44:01.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:44:01.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: mon.a calling monitor election 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: mon.c calling monitor election 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: monmap epoch 3 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: last_changed 2024-09-11T23:43:55.763033+0000 2024-09-11T23:44:01.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: min_mon_release 19 (squid) 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: election_strategy: 1 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: 1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.c 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: fsmap 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: mgrmap e13: x(active, since 48s) 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: overall HEALTH_OK 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:01.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:01.832 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:01.832 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:01 smithi120 ceph-mon[35182]: from='client.? 172.21.15.120:0/2034702986' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:44:01.832 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:01.857 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T23:44:01.857 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T23:44:02.123 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:02.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:02.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:01 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/2034702986' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:44:02.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:02.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.528+0000 7f4f9dd86640 1 -- 172.21.15.62:0/1572214123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f98106560 msgr2=0x7f4f98106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.528+0000 7f4f9dd86640 1 --2- 172.21.15.62:0/1572214123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f98106560 0x7f4f98106940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009880 tx=0x7f4f8802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:44:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.529+0000 7f4f9dd86640 1 -- 172.21.15.62:0/1572214123 shutdown_connections 2024-09-11T23:44:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.529+0000 7f4f9dd86640 1 --2- 172.21.15.62:0/1572214123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f98106560 0x7f4f98106940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.529+0000 7f4f9dd86640 1 -- 172.21.15.62:0/1572214123 >> 172.21.15.62:0/1572214123 conn(0x7f4f980fc1b0 msgr2=0x7f4f980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.529+0000 7f4f9dd86640 1 -- 172.21.15.62:0/1572214123 shutdown_connections 2024-09-11T23:44:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.529+0000 7f4f9dd86640 1 -- 172.21.15.62:0/1572214123 wait complete. 2024-09-11T23:44:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.529+0000 7f4f9dd86640 1 Processor -- start 2024-09-11T23:44:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f9dd86640 1 -- start start 2024-09-11T23:44:02.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f9dd86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98106560 0x7f4f981981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:02.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f9dd86640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f981986f0 0x7f4f98198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f9dd86640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981a1220 0x7f4f981a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f9dd86640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9810ab90 con 0x7f4f981986f0 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f9dd86640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9810aa10 con 0x7f4f981a1220 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f9dd86640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9810ad10 con 0x7f4f98106560 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f96ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f981986f0 0x7f4f98198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f96ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f981986f0 0x7f4f98198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45246/0 (socket says 172.21.15.62:45246) 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f96ffd640 1 -- 172.21.15.62:0/3130571508 learned_addr learned my addr 172.21.15.62:0/3130571508 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f97fff640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981a1220 0x7f4f981a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.530+0000 7f4f977fe640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98106560 0x7f4f981981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f97fff640 1 -- 172.21.15.62:0/3130571508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981a1220 msgr2=0x7f4f981a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f97fff640 1 -- 172.21.15.62:0/3130571508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981a1220 msgr2=0x7f4f981a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:44:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f97fff640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981a1220 0x7f4f981a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f97fff640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981a1220 0x7f4f981a3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f96ffd640 1 -- 172.21.15.62:0/3130571508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98106560 msgr2=0x7f4f981981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f96ffd640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98106560 0x7f4f981981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f96ffd640 1 -- 172.21.15.62:0/3130571508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981a1220 msgr2=0x7f4f981a3610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f96ffd640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981a1220 0x7f4f981a3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f96ffd640 1 -- 172.21.15.62:0/3130571508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f981a3db0 con 0x7f4f981986f0 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f977fe640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98106560 0x7f4f981981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f96ffd640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f981986f0 0x7f4f98198b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b520 tx=0x7f4f8c00b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f94ff9640 1 -- 172.21.15.62:0/3130571508 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c004230 con 0x7f4f981986f0 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f94ff9640 1 -- 172.21.15.62:0/3130571508 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c0043d0 con 0x7f4f981986f0 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f9dd86640 1 -- 172.21.15.62:0/3130571508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f981a40a0 con 0x7f4f981986f0 2024-09-11T23:44:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f94ff9640 1 -- 172.21.15.62:0/3130571508 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c003d60 con 0x7f4f981986f0 2024-09-11T23:44:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.531+0000 7f4f9dd86640 1 -- 172.21.15.62:0/3130571508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f981a4630 con 0x7f4f981986f0 2024-09-11T23:44:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.532+0000 7f4f94ff9640 1 -- 172.21.15.62:0/3130571508 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4f8c00e070 con 0x7f4f981986f0 2024-09-11T23:44:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.532+0000 7f4f94ff9640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4f800394f0 0x7f4f8003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.532+0000 7f4f94ff9640 1 -- 172.21.15.62:0/3130571508 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4f8c04b950 con 0x7f4f981986f0 2024-09-11T23:44:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.533+0000 7f4f977fe640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4f800394f0 0x7f4f8003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.533+0000 7f4f9dd86640 1 -- 172.21.15.62:0/3130571508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f981069c0 con 0x7f4f981986f0 2024-09-11T23:44:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.533+0000 7f4f977fe640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4f800394f0 0x7f4f8003b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4f8802ab40 tx=0x7f4f8802aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:02.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.538+0000 7f4f94ff9640 1 -- 172.21.15.62:0/3130571508 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f8c014020 con 0x7f4f981986f0 2024-09-11T23:44:02.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.716+0000 7f4f9dd86640 1 -- 172.21.15.62:0/3130571508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4f981a48f0 con 0x7f4f981986f0 2024-09-11T23:44:02.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.716+0000 7f4f94ff9640 1 -- 172.21.15.62:0/3130571508 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f4f8c020350 con 0x7f4f981986f0 2024-09-11T23:44:02.716 INFO:teuthology.orchestra.run.smithi062.stdout:# minimal ceph.conf for 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:44:02.717 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-09-11T23:44:02.717 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:44:02.717 INFO:teuthology.orchestra.run.smithi062.stdout: mon_host = [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-09-11T23:44:02.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 -- 172.21.15.62:0/3130571508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4f800394f0 msgr2=0x7f4f8003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:02.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4f800394f0 0x7f4f8003b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4f8802ab40 tx=0x7f4f8802aad0 comp rx=0 tx=0).stop 2024-09-11T23:44:02.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 -- 172.21.15.62:0/3130571508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f981986f0 msgr2=0x7f4f98198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:02.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f981986f0 0x7f4f98198b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b520 tx=0x7f4f8c00b9e0 comp rx=0 tx=0).stop 2024-09-11T23:44:02.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 -- 172.21.15.62:0/3130571508 shutdown_connections 2024-09-11T23:44:02.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4f800394f0 0x7f4f8003b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:02.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981a1220 0x7f4f981a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:02.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f981986f0 0x7f4f98198b50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:02.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 --2- 172.21.15.62:0/3130571508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98106560 0x7f4f981981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:02.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 -- 172.21.15.62:0/3130571508 >> 172.21.15.62:0/3130571508 conn(0x7f4f980fc1b0 msgr2=0x7f4f980fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:02.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 -- 172.21.15.62:0/3130571508 shutdown_connections 2024-09-11T23:44:02.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:02.718+0000 7f4f9dd86640 1 -- 172.21.15.62:0/3130571508 wait complete. 2024-09-11T23:44:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:44:03.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:03.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:44:03.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: mon.a calling monitor election 2024-09-11T23:44:03.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: mon.c calling monitor election 2024-09-11T23:44:03.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:03.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:03.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:03.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:03.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:03.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: monmap epoch 3 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: last_changed 2024-09-11T23:43:55.763033+0000 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: min_mon_release 19 (squid) 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: election_strategy: 1 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: 1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.c 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-09-11T23:44:03.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: fsmap 2024-09-11T23:44:03.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:44:03.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: mgrmap e13: x(active, since 48s) 2024-09-11T23:44:03.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: overall HEALTH_OK 2024-09-11T23:44:03.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:03.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/2034702986' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:44:03.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:03.439 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T23:44:03.439 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:44:03.439 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:44:03.468 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:44:03.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:44:03.534 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:44:03.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:44:03.562 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:44:03.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:44:03.629 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:44:03.629 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:44:03.743 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:44:03.743 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:44:03.768 INFO:tasks.cephadm:Adding mgr.x on smithi062 2024-09-11T23:44:03.769 INFO:tasks.cephadm:Adding mgr.y on smithi083 2024-09-11T23:44:03.769 INFO:tasks.cephadm:Adding mgr.z on smithi120 2024-09-11T23:44:03.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi062=x;smithi083=y;smithi120=z' 2024-09-11T23:44:03.796 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: mon.b calling monitor election 2024-09-11T23:44:03.796 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: mon.b calling monitor election 2024-09-11T23:44:03.796 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: mon.c calling monitor election 2024-09-11T23:44:03.796 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: mon.a calling monitor election 2024-09-11T23:44:03.796 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:44:03.796 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: monmap epoch 3 2024-09-11T23:44:03.796 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:44:03.796 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: last_changed 2024-09-11T23:43:55.763033+0000 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: min_mon_release 19 (squid) 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: election_strategy: 1 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: 1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.c 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: fsmap 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: mgrmap e13: x(active, since 50s) 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: overall HEALTH_OK 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:44:03.797 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:44:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:44:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.799 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.799 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.799 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T23:44:03.799 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:44:03.799 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:44:03.799 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:03.799 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:03 smithi062 ceph-mon[30958]: Reconfiguring daemon mon.a on smithi062 2024-09-11T23:44:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: mon.b calling monitor election 2024-09-11T23:44:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: mon.b calling monitor election 2024-09-11T23:44:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: mon.c calling monitor election 2024-09-11T23:44:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: mon.a calling monitor election 2024-09-11T23:44:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: monmap epoch 3 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: last_changed 2024-09-11T23:43:55.763033+0000 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: min_mon_release 19 (squid) 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: election_strategy: 1 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: 1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.c 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: fsmap 2024-09-11T23:44:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: mgrmap e13: x(active, since 50s) 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: overall HEALTH_OK 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:03.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T23:44:03.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:44:03.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:44:03.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:03.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:03 smithi083 ceph-mon[34805]: Reconfiguring daemon mon.a on smithi062 2024-09-11T23:44:04.041 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:44:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: mon.b calling monitor election 2024-09-11T23:44:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: mon.b calling monitor election 2024-09-11T23:44:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: mon.c calling monitor election 2024-09-11T23:44:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: mon.a calling monitor election 2024-09-11T23:44:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:44:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: monmap epoch 3 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: last_changed 2024-09-11T23:43:55.763033+0000 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: created 2024-09-11T23:42:14.107883+0000 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: min_mon_release 19 (squid) 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: election_strategy: 1 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: 1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.c 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: fsmap 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: mgrmap e13: x(active, since 50s) 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: overall HEALTH_OK 2024-09-11T23:44:04.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.124 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T23:44:04.124 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:44:04.124 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:44:04.124 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:04.124 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:03 smithi120 ceph-mon[35182]: Reconfiguring daemon mon.a on smithi062 2024-09-11T23:44:04.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.445+0000 7fd13dff9640 1 -- 172.21.15.120:0/3881884334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1381016e0 msgr2=0x7fd138101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:04.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.445+0000 7fd13dff9640 1 --2- 172.21.15.120:0/3881884334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1381016e0 0x7fd138101ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd128009c20 tx=0x7fd12802aa90 comp rx=0 tx=0).stop 2024-09-11T23:44:04.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.445+0000 7fd13dff9640 1 -- 172.21.15.120:0/3881884334 shutdown_connections 2024-09-11T23:44:04.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.445+0000 7fd13dff9640 1 --2- 172.21.15.120:0/3881884334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1381016e0 0x7fd138101ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:04.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.445+0000 7fd13dff9640 1 -- 172.21.15.120:0/3881884334 >> 172.21.15.120:0/3881884334 conn(0x7fd1380fce90 msgr2=0x7fd1380ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:04.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.445+0000 7fd13dff9640 1 -- 172.21.15.120:0/3881884334 shutdown_connections 2024-09-11T23:44:04.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.445+0000 7fd13dff9640 1 -- 172.21.15.120:0/3881884334 wait complete. 2024-09-11T23:44:04.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.445+0000 7fd13dff9640 1 Processor -- start 2024-09-11T23:44:04.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd13dff9640 1 -- start start 2024-09-11T23:44:04.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd13dff9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1381016e0 0x7fd138196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:04.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd13dff9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd138197510 0x7fd13819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:04.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd13dff9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd138197990 0x7fd13819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:04.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd13dff9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1381077d0 con 0x7fd138197990 2024-09-11T23:44:04.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd13dff9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd138107650 con 0x7fd1381016e0 2024-09-11T23:44:04.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd13dff9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd138107950 con 0x7fd138197510 2024-09-11T23:44:04.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd136ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd138197510 0x7fd13819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:04.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd136ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd138197510 0x7fd13819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38178/0 (socket says 172.21.15.120:38178) 2024-09-11T23:44:04.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd136ffd640 1 -- 172.21.15.120:0/4066261771 learned_addr learned my addr 172.21.15.120:0/4066261771 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:44:04.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd137fff640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd138197990 0x7fd13819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:04.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd1377fe640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1381016e0 0x7fd138196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:04.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd136ffd640 1 -- 172.21.15.120:0/4066261771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1381016e0 msgr2=0x7fd138196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd136ffd640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1381016e0 0x7fd138196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd136ffd640 1 -- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd138197990 msgr2=0x7fd13819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd136ffd640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd138197990 0x7fd13819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd136ffd640 1 -- 172.21.15.120:0/4066261771 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13819f3d0 con 0x7fd138197510 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd137fff640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd138197990 0x7fd13819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.446+0000 7fd136ffd640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd138197510 0x7fd13819c580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00b520 tx=0x7fd12c00b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.447+0000 7fd134ff9640 1 -- 172.21.15.120:0/4066261771 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c004230 con 0x7fd138197510 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.447+0000 7fd134ff9640 1 -- 172.21.15.120:0/4066261771 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd12c0043d0 con 0x7fd138197510 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.447+0000 7fd13dff9640 1 -- 172.21.15.120:0/4066261771 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13819f6c0 con 0x7fd138197510 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.447+0000 7fd134ff9640 1 -- 172.21.15.120:0/4066261771 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c003d60 con 0x7fd138197510 2024-09-11T23:44:04.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.447+0000 7fd13dff9640 1 -- 172.21.15.120:0/4066261771 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13819fcd0 con 0x7fd138197510 2024-09-11T23:44:04.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.447+0000 7fd134ff9640 1 -- 172.21.15.120:0/4066261771 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd12c00e070 con 0x7fd138197510 2024-09-11T23:44:04.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.448+0000 7fd13dff9640 1 -- 172.21.15.120:0/4066261771 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd138190590 con 0x7fd138197510 2024-09-11T23:44:04.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.448+0000 7fd134ff9640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1200394f0 0x7fd12003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:04.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.448+0000 7fd134ff9640 1 -- 172.21.15.120:0/4066261771 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd12c04b940 con 0x7fd138197510 2024-09-11T23:44:04.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.448+0000 7fd1377fe640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1200394f0 0x7fd12003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:04.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.448+0000 7fd1377fe640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1200394f0 0x7fd12003b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd1280098d0 tx=0x7fd12803a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:04.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.453+0000 7fd134ff9640 1 -- 172.21.15.120:0/4066261771 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd12c014020 con 0x7fd138197510 2024-09-11T23:44:04.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.629+0000 7fd13dff9640 1 -- 172.21.15.120:0/4066261771 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi083=y;smithi120=z", "target": ["mon-mgr", ""]}) -- 0x7fd1380626c0 con 0x7fd1200394f0 2024-09-11T23:44:04.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.634+0000 7fd134ff9640 1 -- 172.21.15.120:0/4066261771 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd1380626c0 con 0x7fd1200394f0 2024-09-11T23:44:04.634 INFO:teuthology.orchestra.run.smithi120.stdout:Scheduled mgr update... 2024-09-11T23:44:04.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 -- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1200394f0 msgr2=0x7fd12003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:04.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1200394f0 0x7fd12003b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd1280098d0 tx=0x7fd12803a040 comp rx=0 tx=0).stop 2024-09-11T23:44:04.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 -- 172.21.15.120:0/4066261771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd138197510 msgr2=0x7fd13819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:04.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd138197510 0x7fd13819c580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00b520 tx=0x7fd12c00b9e0 comp rx=0 tx=0).stop 2024-09-11T23:44:04.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 -- 172.21.15.120:0/4066261771 shutdown_connections 2024-09-11T23:44:04.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1200394f0 0x7fd12003b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:04.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd138197990 0x7fd13819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:04.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd138197510 0x7fd13819c580 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:04.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 --2- 172.21.15.120:0/4066261771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1381016e0 0x7fd138196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:04.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 -- 172.21.15.120:0/4066261771 >> 172.21.15.120:0/4066261771 conn(0x7fd1380fce90 msgr2=0x7fd1380fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:04.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 -- 172.21.15.120:0/4066261771 shutdown_connections 2024-09-11T23:44:04.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:44:04.636+0000 7fd13dff9640 1 -- 172.21.15.120:0/4066261771 wait complete. 2024-09-11T23:44:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:44:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:44:04.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:44:04.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:04.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: Reconfiguring daemon mon.b on smithi083 2024-09-11T23:44:04.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi083=y;smithi120=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:44:04.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: Saving service mgr spec with placement smithi062=x;smithi083=y;smithi120=z;count:3 2024-09-11T23:44:04.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:04.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:04 smithi083 ceph-mon[34805]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:05.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:05.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:05.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:05.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:44:05.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:44:05.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:44:05.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:05.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: Reconfiguring daemon mon.b on smithi083 2024-09-11T23:44:05.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi083=y;smithi120=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:44:05.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: Saving service mgr spec with placement smithi062=x;smithi083=y;smithi120=z;count:3 2024-09-11T23:44:05.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:05.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:04 smithi120 ceph-mon[35182]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:05.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:44:05.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:05.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:05.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:44:05.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:44:05.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:44:05.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:05.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: Reconfiguring daemon mon.b on smithi083 2024-09-11T23:44:05.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi083=y;smithi120=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:44:05.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: Saving service mgr spec with placement smithi062=x;smithi083=y;smithi120=z;count:3 2024-09-11T23:44:05.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:05.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:04 smithi062 ceph-mon[30958]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:05.273 DEBUG:teuthology.orchestra.run.smithi083:mgr.y> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T23:44:05.276 DEBUG:teuthology.orchestra.run.smithi120:mgr.z> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.z.service 2024-09-11T23:44:05.279 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T23:44:05.279 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:44:05.279 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:44:05.299 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:44:05.299 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-09-11T23:44:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:44:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:05.359 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-11T23:44:05.359 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:05.359 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:05.360 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-09-11 23:43:23.271504605 +0000 2024-09-11T23:44:05.360 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-09-11 23:38:59.151221107 +0000 2024-09-11T23:44:05.360 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-09-11 23:38:59.151221107 +0000 2024-09-11T23:44:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-09-11 23:38:59.151221107 +0000 2024-09-11T23:44:05.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:44:05.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-09-11T23:44:05.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-09-11T23:44:05.429 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000232016 s, 2.2 MB/s 2024-09-11T23:44:05.431 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:44:05.491 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-09-11T23:44:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:44:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:05.550 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T23:44:05.550 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:05.550 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:05.550 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-09-11 23:43:23.298504164 +0000 2024-09-11T23:44:05.550 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-09-11 23:38:59.502215435 +0000 2024-09-11T23:44:05.550 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-09-11 23:38:59.502215435 +0000 2024-09-11T23:44:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-09-11 23:38:59.502215435 +0000 2024-09-11T23:44:05.550 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:44:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-09-11T23:44:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-09-11T23:44:05.618 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00026584 s, 1.9 MB/s 2024-09-11T23:44:05.619 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:44:05.677 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-09-11T23:44:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:44:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:05.735 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-11T23:44:05.735 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:05.736 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:05.736 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-09-11 23:43:23.325503722 +0000 2024-09-11T23:44:05.736 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-09-11 23:38:59.831210118 +0000 2024-09-11T23:44:05.736 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-09-11 23:38:59.831210118 +0000 2024-09-11T23:44:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-09-11 23:38:59.831210118 +0000 2024-09-11T23:44:05.736 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:44:05.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-09-11T23:44:05.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-09-11T23:44:05.802 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000205479 s, 2.5 MB/s 2024-09-11T23:44:05.803 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:44:05.860 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-09-11T23:44:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:44:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:05.917 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-11T23:44:05.917 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:05.917 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:05.918 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-09-11 23:43:23.354503248 +0000 2024-09-11T23:44:05.918 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-09-11 23:39:00.155204882 +0000 2024-09-11T23:44:05.918 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-09-11 23:39:00.155204882 +0000 2024-09-11T23:44:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-09-11 23:39:00.155204882 +0000 2024-09-11T23:44:05.918 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:44:05.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-09-11T23:44:05.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-09-11T23:44:05.985 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000196944 s, 2.6 MB/s 2024-09-11T23:44:05.987 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:44:06.045 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-11T23:44:06.045 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:44:06.062 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:44:06.062 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-09-11T23:44:06.118 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:44:06.118 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:06.118 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T23:44:06.118 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:06.119 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:06.119 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-11 23:43:55.210639409 +0000 2024-09-11T23:44:06.119 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-11 23:39:10.943023793 +0000 2024-09-11T23:44:06.119 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-11 23:39:10.943023793 +0000 2024-09-11T23:44:06.119 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-11 23:39:10.943023793 +0000 2024-09-11T23:44:06.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:44:06.183 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-11T23:44:06.183 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-11T23:44:06.183 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000343458 s, 1.5 MB/s 2024-09-11T23:44:06.185 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:44:06.242 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-09-11T23:44:06.298 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:44:06.298 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:06.298 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-11T23:44:06.298 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:06.298 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:06.298 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-11 23:43:55.210639409 +0000 2024-09-11T23:44:06.298 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-11 23:39:11.266015663 +0000 2024-09-11T23:44:06.298 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-11 23:39:11.266015663 +0000 2024-09-11T23:44:06.299 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-11 23:39:11.266015663 +0000 2024-09-11T23:44:06.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:44:06.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-11T23:44:06.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-11T23:44:06.362 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000206235 s, 2.5 MB/s 2024-09-11T23:44:06.363 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:44:06.419 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-09-11T23:44:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:44:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:06.475 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-11T23:44:06.475 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:06.476 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:06.476 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-11 23:43:55.210639409 +0000 2024-09-11T23:44:06.476 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-11 23:39:11.591007482 +0000 2024-09-11T23:44:06.476 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-11 23:39:11.591007482 +0000 2024-09-11T23:44:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-11 23:39:11.591007482 +0000 2024-09-11T23:44:06.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:44:06.538 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-11T23:44:06.539 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-11T23:44:06.539 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000205818 s, 2.5 MB/s 2024-09-11T23:44:06.540 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:44:06.596 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-09-11T23:44:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:44:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:06.652 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-11T23:44:06.652 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:06.652 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:06.653 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-11 23:43:55.211639384 +0000 2024-09-11T23:44:06.653 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-11 23:39:11.936998773 +0000 2024-09-11T23:44:06.653 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-11 23:39:11.936998773 +0000 2024-09-11T23:44:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-11 23:39:11.936998773 +0000 2024-09-11T23:44:06.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:44:06.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-11T23:44:06.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-11T23:44:06.716 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000176705 s, 2.9 MB/s 2024-09-11T23:44:06.717 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:44:06.774 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-11T23:44:06.774 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:44:06.790 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:44:06.790 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-09-11T23:44:06.846 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:44:06.846 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:06.846 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T23:44:06.846 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:06.847 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:06.847 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-11 23:43:46.053871675 +0000 2024-09-11T23:44:06.847 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-11 23:39:08.837042773 +0000 2024-09-11T23:44:06.847 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-11 23:39:08.837042773 +0000 2024-09-11T23:44:06.847 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-11 23:39:08.837042773 +0000 2024-09-11T23:44:06.847 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:44:06.910 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-11T23:44:06.910 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-11T23:44:06.911 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000229013 s, 2.2 MB/s 2024-09-11T23:44:06.912 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:44:06.968 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-09-11T23:44:07.023 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:44:07.023 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:07.023 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-11T23:44:07.023 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:07.024 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:07.024 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-11 23:43:46.054871653 +0000 2024-09-11T23:44:07.024 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-11 23:39:09.151035647 +0000 2024-09-11T23:44:07.024 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-11 23:39:09.151035647 +0000 2024-09-11T23:44:07.025 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-11 23:39:09.151035647 +0000 2024-09-11T23:44:07.025 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:44:07.087 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-11T23:44:07.087 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-11T23:44:07.087 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.00019767 s, 2.6 MB/s 2024-09-11T23:44:07.088 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:44:07.146 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-09-11T23:44:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:44:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:07.202 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-11T23:44:07.202 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:07.202 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:07.202 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-11 23:43:46.054871653 +0000 2024-09-11T23:44:07.202 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-11 23:39:09.483028113 +0000 2024-09-11T23:44:07.202 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-11 23:39:09.483028113 +0000 2024-09-11T23:44:07.202 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-11 23:39:09.483028113 +0000 2024-09-11T23:44:07.202 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:44:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:06 smithi062 ceph-mon[30958]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:44:07.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:44:07.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:44:07.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:07.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:06 smithi062 ceph-mon[30958]: Reconfiguring daemon mon.c on smithi120 2024-09-11T23:44:07.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:06 smithi062 ceph-mon[30958]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:07.262 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:07.262 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:07.263 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:06 smithi120 ceph-mon[35182]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:44:07.263 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:44:07.263 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:44:07.263 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:07.263 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:06 smithi120 ceph-mon[35182]: Reconfiguring daemon mon.c on smithi120 2024-09-11T23:44:07.263 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:06 smithi120 ceph-mon[35182]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:07.264 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-11T23:44:07.264 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-11T23:44:07.264 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000171007 s, 3.0 MB/s 2024-09-11T23:44:07.266 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:44:07.322 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-09-11T23:44:07.377 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:44:07.377 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:44:07.377 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-11T23:44:07.377 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:44:07.377 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:44:07.377 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-11 23:43:46.054871653 +0000 2024-09-11T23:44:07.377 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-11 23:39:09.811020670 +0000 2024-09-11T23:44:07.377 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-11 23:39:09.811020670 +0000 2024-09-11T23:44:07.377 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-11 23:39:09.811020670 +0000 2024-09-11T23:44:07.378 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:44:07.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:07.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:07.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:06 smithi083 ceph-mon[34805]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:44:07.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:44:07.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:44:07.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:07.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:06 smithi083 ceph-mon[34805]: Reconfiguring daemon mon.c on smithi120 2024-09-11T23:44:07.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:06 smithi083 ceph-mon[34805]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-11T23:44:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-11T23:44:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000202576 s, 2.5 MB/s 2024-09-11T23:44:07.442 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:44:07.500 INFO:tasks.cephadm:Deploying osd.0 on smithi062 with /dev/vg_nvme/lv_4... 2024-09-11T23:44:07.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:44:07.761 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:44:08.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:44:08.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:44:08.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:08.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:08 smithi083 ceph-mon[34805]: Deploying daemon mgr.z on smithi120 2024-09-11T23:44:08.451 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:08.451 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:08.451 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:08.451 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:08.451 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:08.452 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:08.452 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:44:08.452 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:44:08.452 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:44:08.452 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:08.452 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:08 smithi120 ceph-mon[35182]: Deploying daemon mgr.z on smithi120 2024-09-11T23:44:08.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:08.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:08.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:08.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:08.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:08.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:08.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:44:08.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:44:08.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:44:08.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:08.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:08 smithi062 ceph-mon[30958]: Deploying daemon mgr.z on smithi120 2024-09-11T23:44:09.121 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 podman[36969]: 2024-09-11 23:44:09.047580165 +0000 UTC m=+0.146069086 container create 251ba42188807c0c33843fca6c71ce553c0fff016a06225e9674cbed9f4e7ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-11T23:44:09.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:09 smithi062 ceph-mon[30958]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:09.433 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:09 smithi120 ceph-mon[35182]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:09.433 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 podman[36969]: 2024-09-11 23:44:09.24997383 +0000 UTC m=+0.348462752 container init 251ba42188807c0c33843fca6c71ce553c0fff016a06225e9674cbed9f4e7ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T23:44:09.433 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 podman[36969]: 2024-09-11 23:44:09.252887194 +0000 UTC m=+0.351376115 container start 251ba42188807c0c33843fca6c71ce553c0fff016a06225e9674cbed9f4e7ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-11T23:44:09.433 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 ceph-mgr[36988]: -- 172.21.15.120:0/3949398842 <== mon.1 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb1f7f50e0 con 0x55cb1f7d3400 2024-09-11T23:44:09.433 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 bash[36969]: 251ba42188807c0c33843fca6c71ce553c0fff016a06225e9674cbed9f4e7ea2 2024-09-11T23:44:09.433 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 systemd[1]: Started Ceph mgr.z for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:44:09.433 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:09.374+0000 7fb1c2329040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:44:09.434 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:09.432+0000 7fb1c2329040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:44:09.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:09 smithi083 ceph-mon[34805]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:09.787 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:09.578+0000 7fb1c2329040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:44:09.787 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:09.642+0000 7fb1c2329040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:44:09.787 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:09.787+0000 7fb1c2329040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:44:10.081 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:44:10.107 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2024-09-11T23:44:10.120 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:09.924+0000 7fb1c2329040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:44:10.358 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:10.621 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:10.235+0000 7fb1c2329040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:44:10.621 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:10.294+0000 7fb1c2329040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:44:10.621 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:10.477+0000 7fb1c2329040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:44:10.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:10.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:10.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:10.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:44:10.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:44:10.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:44:10.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:10.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:10 smithi120 ceph-mon[35182]: Deploying daemon mgr.y on smithi083 2024-09-11T23:44:10.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:10.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:10.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:10.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:44:10.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:44:10.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:44:10.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:10.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:10 smithi083 ceph-mon[34805]: Deploying daemon mgr.y on smithi083 2024-09-11T23:44:10.698 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:10.698 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:10.698 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:10.698 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:44:10.698 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:44:10.698 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:44:10.698 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:10.698 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:10 smithi062 ceph-mon[30958]: Deploying daemon mgr.y on smithi083 2024-09-11T23:44:10.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.782+0000 7f57b1847640 1 -- 172.21.15.62:0/3739282931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ac1026a0 msgr2=0x7f57ac104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.782+0000 7f57b1847640 1 --2- 172.21.15.62:0/3739282931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ac1026a0 0x7f57ac104a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f579c009900 tx=0x7f579c02a410 comp rx=0 tx=0).stop 2024-09-11T23:44:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.782+0000 7f57b1847640 1 -- 172.21.15.62:0/3739282931 shutdown_connections 2024-09-11T23:44:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.782+0000 7f57b1847640 1 --2- 172.21.15.62:0/3739282931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac107900 0x7f57ac109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.782+0000 7f57b1847640 1 --2- 172.21.15.62:0/3739282931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ac104fd0 0x7f57ac1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.782+0000 7f57b1847640 1 --2- 172.21.15.62:0/3739282931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ac1026a0 0x7f57ac104a90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.782+0000 7f57b1847640 1 -- 172.21.15.62:0/3739282931 >> 172.21.15.62:0/3739282931 conn(0x7f57ac0fc2a0 msgr2=0x7f57ac0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.782+0000 7f57b1847640 1 -- 172.21.15.62:0/3739282931 shutdown_connections 2024-09-11T23:44:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.782+0000 7f57b1847640 1 -- 172.21.15.62:0/3739282931 wait complete. 2024-09-11T23:44:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57b1847640 1 Processor -- start 2024-09-11T23:44:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57b1847640 1 -- start start 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57b1847640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ac1026a0 0x7f57ac197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57b1847640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ac104fd0 0x7f57ac1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57b1847640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac107900 0x7f57ac19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57b1847640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac10d900 con 0x7f57ac1026a0 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57b1847640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac10d780 con 0x7f57ac107900 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57b1847640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac10da80 con 0x7f57ac104fd0 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57aaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ac1026a0 0x7f57ac197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57aaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ac1026a0 0x7f57ac197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45266/0 (socket says 172.21.15.62:45266) 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.783+0000 7f57aaffd640 1 -- 172.21.15.62:0/3317829293 learned_addr learned my addr 172.21.15.62:0/3317829293 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57aa7fc640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ac104fd0 0x7f57ac1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57ab7fe640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac107900 0x7f57ac19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57aaffd640 1 -- 172.21.15.62:0/3317829293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ac104fd0 msgr2=0x7f57ac1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57aaffd640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ac104fd0 0x7f57ac1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57aaffd640 1 -- 172.21.15.62:0/3317829293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac107900 msgr2=0x7f57ac19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57aaffd640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac107900 0x7f57ac19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57aaffd640 1 -- 172.21.15.62:0/3317829293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac19fb90 con 0x7f57ac1026a0 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57aa7fc640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ac104fd0 0x7f57ac1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57ab7fe640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac107900 0x7f57ac19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57aaffd640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ac1026a0 0x7f57ac197f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f579c031440 tx=0x7f579c032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57b0845640 1 -- 172.21.15.62:0/3317829293 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c03f030 con 0x7f57ac1026a0 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57b0845640 1 -- 172.21.15.62:0/3317829293 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579c003ce0 con 0x7f57ac1026a0 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57b1847640 1 -- 172.21.15.62:0/3317829293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ac19fe20 con 0x7f57ac1026a0 2024-09-11T23:44:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57b0845640 1 -- 172.21.15.62:0/3317829293 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c035070 con 0x7f57ac1026a0 2024-09-11T23:44:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.784+0000 7f57b1847640 1 -- 172.21.15.62:0/3317829293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ac1a0220 con 0x7f57ac1026a0 2024-09-11T23:44:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.785+0000 7f57b0845640 1 -- 172.21.15.62:0/3317829293 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f579c042030 con 0x7f57ac1026a0 2024-09-11T23:44:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.785+0000 7f57b1847640 1 -- 172.21.15.62:0/3317829293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57ac0fdd90 con 0x7f57ac1026a0 2024-09-11T23:44:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.786+0000 7f57b0845640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57980392c0 0x7f579803b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.786+0000 7f57b0845640 1 -- 172.21.15.62:0/3317829293 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f579c06cb50 con 0x7f57ac1026a0 2024-09-11T23:44:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.786+0000 7f57aa7fc640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57980392c0 0x7f579803b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.786+0000 7f57aa7fc640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57980392c0 0x7f579803b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5794005d00 tx=0x7f5794005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.790+0000 7f57b0845640 1 -- 172.21.15.62:0/3317829293 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f579c030c30 con 0x7f57ac1026a0 2024-09-11T23:44:10.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:10.979+0000 7f57b1847640 1 -- 172.21.15.62:0/3317829293 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f57ac062630 con 0x7f57980392c0 2024-09-11T23:44:11.337 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 systemd[1]: Starting Ceph mgr.y for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:44:11.599 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:11 smithi083 ceph-mon[34805]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:11.599 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:11 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:44:11.600 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:11 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:44:11.600 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:11 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:11.600 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 podman[36156]: 2024-09-11 23:44:11.240996941 +0000 UTC m=+0.015973586 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:44:11.600 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 podman[36156]: 2024-09-11 23:44:11.348013089 +0000 UTC m=+0.122989734 container create 15708569c0cbf53b0a1382cc9710b509fa12f4eb9edf9944ab608e607d67c266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-11T23:44:11.600 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 podman[36156]: 2024-09-11 23:44:11.600612293 +0000 UTC m=+0.375588937 container init 15708569c0cbf53b0a1382cc9710b509fa12f4eb9edf9944ab608e607d67c266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:44:11.621 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:11 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:11.350+0000 7fb1c2329040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:44:11.621 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:11 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:11.418+0000 7fb1c2329040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:44:11.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:11 smithi120 ceph-mon[35182]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:11.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:11 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:44:11.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:11 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:44:11.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:11 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:11.684 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:11 smithi062 ceph-mon[30958]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:11.684 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:11 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:44:11.684 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:11 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:44:11.684 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:11 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:11.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 podman[36156]: 2024-09-11 23:44:11.603603024 +0000 UTC m=+0.378579680 container start 15708569c0cbf53b0a1382cc9710b509fa12f4eb9edf9944ab608e607d67c266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-11T23:44:11.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 ceph-mgr[36175]: -- 172.21.15.83:0/1836203005 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5580a755d0e0 con 0x5580a753b400 2024-09-11T23:44:11.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 bash[36156]: 15708569c0cbf53b0a1382cc9710b509fa12f4eb9edf9944ab608e607d67c266 2024-09-11T23:44:11.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 systemd[1]: Started Ceph mgr.y for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:44:11.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:11.722+0000 7fd65660b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:44:11.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:11.784+0000 7fd65660b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:44:12.320 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:11 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:11.941+0000 7fd65660b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:44:12.320 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:12.010+0000 7fd65660b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:44:12.320 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:12.167+0000 7fd65660b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:44:12.320 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:12.320+0000 7fd65660b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:44:12.433 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:12.155+0000 7fb1c2329040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:44:12.433 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:12.289+0000 7fb1c2329040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:44:12.433 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:12.433+0000 7fb1c2329040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:44:12.434 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-mon[35182]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:44:12.434 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.434 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.434 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.434 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.434 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:12.576 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:12 smithi062 ceph-mon[30958]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:44:12.577 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.577 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.577 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.577 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.577 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:12 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:12.646 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-mon[34805]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:44:12.647 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.647 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.647 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.647 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:12.647 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:12.717 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:12.494+0000 7fb1c2329040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:44:12.718 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:12.645+0000 7fb1c2329040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:44:12.718 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:12.717+0000 7fb1c2329040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:44:12.901 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:12.646+0000 7fd65660b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:44:12.902 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:12.708+0000 7fd65660b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:44:13.006 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:12.778+0000 7fb1c2329040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:44:13.006 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:12.938+0000 7fb1c2329040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:44:13.006 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:13.007+0000 7fb1c2329040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:44:13.167 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:12.901+0000 7fd65660b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:44:13.280 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:13.079+0000 7fb1c2329040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:44:13.281 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:13.164+0000 7fb1c2329040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:44:13.281 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:13.223+0000 7fb1c2329040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:44:13.281 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:13.281+0000 7fb1c2329040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:44:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:13.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:13.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:44:13.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:44:13.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:13.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2081622907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74ed3595-d3b7-44d6-8aa3-1896f7602607"}]: dispatch 2024-09-11T23:44:13.623 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2081622907' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74ed3595-d3b7-44d6-8aa3-1896f7602607"}]': finished 2024-09-11T23:44:13.623 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:44:13.623 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2081622907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74ed3595-d3b7-44d6-8aa3-1896f7602607"}]: dispatch 2024-09-11T23:44:13.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2081622907' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74ed3595-d3b7-44d6-8aa3-1896f7602607"}]': finished 2024-09-11T23:44:13.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:44:13.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:13.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:44:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:44:13.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:13.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2081622907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74ed3595-d3b7-44d6-8aa3-1896f7602607"}]: dispatch 2024-09-11T23:44:13.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2081622907' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74ed3595-d3b7-44d6-8aa3-1896f7602607"}]': finished 2024-09-11T23:44:13.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:44:13.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:14.167 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:13.793+0000 7fd65660b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:44:14.167 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:13 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:13.860+0000 7fd65660b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:44:14.370 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:44:14 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:44:14.058+0000 7fb1c2329040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:44:14.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:14.415+0000 7f57b0845640 1 -- 172.21.15.62:0/3317829293 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f579c03a8e0 con 0x7f57ac1026a0 2024-09-11T23:44:14.667 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:14.576+0000 7fd65660b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:44:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-mon[34805]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:44:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-mon[34805]: Reconfiguring daemon mgr.x on smithi062 2024-09-11T23:44:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2550978246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:44:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-mon[34805]: Standby manager daemon z started 2024-09-11T23:44:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:44:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:44:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:44:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:44:14.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:14 smithi062 ceph-mon[30958]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:44:14.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:14 smithi062 ceph-mon[30958]: Reconfiguring daemon mgr.x on smithi062 2024-09-11T23:44:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:14 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2550978246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:44:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:14 smithi062 ceph-mon[30958]: Standby manager daemon z started 2024-09-11T23:44:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:14 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:44:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:14 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:44:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:14 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:44:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:14 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:44:14.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:14 smithi120 ceph-mon[35182]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:44:14.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:14 smithi120 ceph-mon[35182]: Reconfiguring daemon mgr.x on smithi062 2024-09-11T23:44:14.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:14 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2550978246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:44:14.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:14 smithi120 ceph-mon[35182]: Standby manager daemon z started 2024-09-11T23:44:14.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:14 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:44:14.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:14 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:44:14.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:14 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:44:14.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:14 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.120:0/947595523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:44:15.043 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:14.691+0000 7fd65660b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:44:15.043 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:14.834+0000 7fd65660b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:44:15.043 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:14.895+0000 7fd65660b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:44:15.043 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:15.043+0000 7fd65660b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:44:15.325 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:15.111+0000 7fd65660b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:44:15.325 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:15.170+0000 7fd65660b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:44:15.325 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:15.325+0000 7fd65660b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:44:15.598 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-mon[34805]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-11T23:44:15.598 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:44:15.598 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-mon[34805]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:15.598 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:15.390+0000 7fd65660b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:44:15.598 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:15.458+0000 7fd65660b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:44:15.599 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:15.540+0000 7fd65660b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:44:15.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:15 smithi062 ceph-mon[30958]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-11T23:44:15.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:15 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:44:15.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:15 smithi062 ceph-mon[30958]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:15.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:15 smithi120 ceph-mon[35182]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-11T23:44:15.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:15 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:44:15.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:15 smithi120 ceph-mon[35182]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:15.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:15.598+0000 7fd65660b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:44:15.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:15.657+0000 7fd65660b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:44:16.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:16.500+0000 7f57b0845640 1 -- 172.21.15.62:0/3317829293 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f579c03a3c0 con 0x7f57ac1026a0 2024-09-11T23:44:16.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:16 smithi120 ceph-mon[35182]: Standby manager daemon y started 2024-09-11T23:44:16.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:16 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:44:16.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:16 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:44:16.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:16 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:44:16.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:16 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:44:16.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:44:16 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:44:16.439+0000 7fd65660b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:44:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:16 smithi083 ceph-mon[34805]: Standby manager daemon y started 2024-09-11T23:44:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:16 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:44:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:16 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:44:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:16 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:44:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:16 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:44:16.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:16 smithi062 ceph-mon[30958]: Standby manager daemon y started 2024-09-11T23:44:16.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:16 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:44:16.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:16 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:44:16.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:16 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:44:16.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:16 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.83:0/1352997806' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:44:17.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:17 smithi120 ceph-mon[35182]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-11T23:44:17.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:17 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:44:17.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:17 smithi120 ceph-mon[35182]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:17 smithi083 ceph-mon[34805]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-11T23:44:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:17 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:44:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:17 smithi083 ceph-mon[34805]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:17.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:17 smithi062 ceph-mon[30958]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-11T23:44:17.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:44:17.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:17 smithi062 ceph-mon[30958]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:19.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:18 smithi062 ceph-mon[30958]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:19.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:18 smithi120 ceph-mon[35182]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:19.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:18 smithi083 ceph-mon[34805]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:21.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:21 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:21.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:21 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:21.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:21 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:21.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:21 smithi062 ceph-mon[30958]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:21 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:21 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:21 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:21 smithi120 ceph-mon[35182]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:21 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:21 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:21 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:21 smithi083 ceph-mon[34805]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:22.347 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:22 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:22.347 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:22 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:22.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:22 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:22.522 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:22 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:22.522 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:22 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:22.522 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:22 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:22.568 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:22 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:22.569 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:22 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:22.569 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:22 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:23.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:44:23.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:23.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:23 smithi062 ceph-mon[30958]: Deploying daemon osd.0 on smithi062 2024-09-11T23:44:23.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:23 smithi062 ceph-mon[30958]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:23.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:44:23.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:23.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:23 smithi120 ceph-mon[35182]: Deploying daemon osd.0 on smithi062 2024-09-11T23:44:23.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:23 smithi120 ceph-mon[35182]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:44:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:23 smithi083 ceph-mon[34805]: Deploying daemon osd.0 on smithi062 2024-09-11T23:44:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:23 smithi083 ceph-mon[34805]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:25.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:24 smithi062 ceph-mon[30958]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:24 smithi120 ceph-mon[35182]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:24 smithi083 ceph-mon[34805]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:26 smithi120 ceph-mon[35182]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:26 smithi083 ceph-mon[34805]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:26 smithi062 ceph-mon[30958]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:29.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:28 smithi120 ceph-mon[35182]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:29.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:28 smithi083 ceph-mon[34805]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:28 smithi062 ceph-mon[30958]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:30.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:29 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:30.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:29 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:30.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:29 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:30.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:29 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:30.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:29 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:30.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:29 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:30.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:29 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:30.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:29 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:30.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:29 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:30.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:30 smithi062 ceph-mon[30958]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:31.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:30 smithi120 ceph-mon[35182]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:31.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:30 smithi083 ceph-mon[34805]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:32.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:32.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:32.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:32.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:32.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:33.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.001+0000 7f57b0845640 1 -- 172.21.15.62:0/3317829293 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f57ac062630 con 0x7f57980392c0 2024-09-11T23:44:33.005 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 0 on host 'smithi062' 2024-09-11T23:44:33.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.003+0000 7f57b1847640 1 -- 172.21.15.62:0/3317829293 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57980392c0 msgr2=0x7f579803b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:33.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.003+0000 7f57b1847640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57980392c0 0x7f579803b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5794005d00 tx=0x7f5794005c70 comp rx=0 tx=0).stop 2024-09-11T23:44:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.003+0000 7f57b1847640 1 -- 172.21.15.62:0/3317829293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ac1026a0 msgr2=0x7f57ac197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.003+0000 7f57b1847640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ac1026a0 0x7f57ac197f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f579c031440 tx=0x7f579c032e20 comp rx=0 tx=0).stop 2024-09-11T23:44:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.003+0000 7f57b1847640 1 -- 172.21.15.62:0/3317829293 shutdown_connections 2024-09-11T23:44:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.003+0000 7f57b1847640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57980392c0 0x7f579803b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.003+0000 7f57b1847640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac107900 0x7f57ac19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.003+0000 7f57b1847640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57ac104fd0 0x7f57ac1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.003+0000 7f57b1847640 1 --2- 172.21.15.62:0/3317829293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ac1026a0 0x7f57ac197f60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.003+0000 7f57b1847640 1 -- 172.21.15.62:0/3317829293 >> 172.21.15.62:0/3317829293 conn(0x7f57ac0fc2a0 msgr2=0x7f57ac102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.004+0000 7f57b1847640 1 -- 172.21.15.62:0/3317829293 shutdown_connections 2024-09-11T23:44:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:44:33.004+0000 7f57b1847640 1 -- 172.21.15.62:0/3317829293 wait complete. 2024-09-11T23:44:33.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:33 smithi062 ceph-mon[30958]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:33.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:33 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:33.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:33 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:33.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:33 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:33 smithi120 ceph-mon[35182]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:33 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:33.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:33 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:33.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:33 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:33.636 DEBUG:teuthology.orchestra.run.smithi062:osd.0> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T23:44:33.639 INFO:tasks.cephadm:Deploying osd.1 on smithi083 with /dev/vg_nvme/lv_4... 2024-09-11T23:44:33.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:44:33.664 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:33 smithi083 ceph-mon[34805]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:33.664 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:33 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:33.664 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:33 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:33.664 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:33 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:33.884 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:44:35.040 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:34 smithi062 ceph-mon[30958]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:35.040 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:44:34 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0[44685]: 2024-09-11T23:44:34.870+0000 7f9498106640 -1 osd.0 0 log_to_monitors true 2024-09-11T23:44:35.047 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:34 smithi083 ceph-mon[34805]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:35.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:34 smithi120 ceph-mon[35182]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:35.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:35 smithi083 ceph-mon[34805]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:44:36.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:35 smithi120 ceph-mon[35182]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:44:36.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:35 smithi062 ceph-mon[30958]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:44:36.327 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:44:36.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-09-11T23:44:36.644 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:44:36.875 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:44:36.875 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:44:36.875 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:44:36.875 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:36.875 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi062 2024-09-11T23:44:36.875 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:36.875 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:36.875 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:44:36.875 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi062 to 17088M 2024-09-11T23:44:36.875 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:36.876 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:36.876 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:36.876 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:36.876 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:36 smithi083 ceph-mon[34805]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:36.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi062 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi062 to 17088M 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-mon[30958]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:36.967 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0[44685]: 2024-09-11T23:44:36.817+0000 7f9494089640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T23:44:36.967 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:44:36 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0[44685]: 2024-09-11T23:44:36.820+0000 7f948feb1640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:44:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.088+0000 7fb09f805640 1 -- 172.21.15.83:0/592666725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb098101cd0 msgr2=0x7fb098102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.088+0000 7fb09f805640 1 --2- 172.21.15.83:0/592666725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb098101cd0 0x7fb098102150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb088009900 tx=0x7fb08802a410 comp rx=0 tx=0).stop 2024-09-11T23:44:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.088+0000 7fb09f805640 1 -- 172.21.15.83:0/592666725 shutdown_connections 2024-09-11T23:44:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.088+0000 7fb09f805640 1 --2- 172.21.15.83:0/592666725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb098102690 0x7fb098108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.088+0000 7fb09f805640 1 --2- 172.21.15.83:0/592666725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb098101cd0 0x7fb098102150 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.088+0000 7fb09f805640 1 --2- 172.21.15.83:0/592666725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb098100ad0 0x7fb098100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.088+0000 7fb09f805640 1 -- 172.21.15.83:0/592666725 >> 172.21.15.83:0/592666725 conn(0x7fb0980fc280 msgr2=0x7fb0980fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.088+0000 7fb09f805640 1 -- 172.21.15.83:0/592666725 shutdown_connections 2024-09-11T23:44:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.088+0000 7fb09f805640 1 -- 172.21.15.83:0/592666725 wait complete. 2024-09-11T23:44:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09f805640 1 Processor -- start 2024-09-11T23:44:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09f805640 1 -- start start 2024-09-11T23:44:37.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09f805640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb098100ad0 0x7fb098197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09f805640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb098101cd0 0x7fb098198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09f805640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb098102690 0x7fb09819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09f805640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb09810c730 con 0x7fb098100ad0 2024-09-11T23:44:37.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09f805640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb09810c5b0 con 0x7fb098101cd0 2024-09-11T23:44:37.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09f805640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb09810c8b0 con 0x7fb098102690 2024-09-11T23:44:37.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09cda0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb098101cd0 0x7fb098198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09cda0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb098101cd0 0x7fb098198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39756/0 (socket says 172.21.15.83:39756) 2024-09-11T23:44:37.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09cda0640 1 -- 172.21.15.83:0/2406323333 learned_addr learned my addr 172.21.15.83:0/2406323333 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-11T23:44:37.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.089+0000 7fb09d5a1640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb098100ad0 0x7fb098197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09dda2640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb098102690 0x7fb09819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09cda0640 1 -- 172.21.15.83:0/2406323333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb098102690 msgr2=0x7fb09819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09cda0640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb098102690 0x7fb09819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09cda0640 1 -- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb098100ad0 msgr2=0x7fb098197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09cda0640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb098100ad0 0x7fb098197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09cda0640 1 -- 172.21.15.83:0/2406323333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09819fb40 con 0x7fb098101cd0 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09d5a1640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb098100ad0 0x7fb098197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09dda2640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb098102690 0x7fb09819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09cda0640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb098101cd0 0x7fb098198450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb088009230 tx=0x7fb088004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb08e7fc640 1 -- 172.21.15.83:0/2406323333 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08803f030 con 0x7fb098101cd0 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb08e7fc640 1 -- 172.21.15.83:0/2406323333 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb088003ce0 con 0x7fb098101cd0 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09f805640 1 -- 172.21.15.83:0/2406323333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09819fdd0 con 0x7fb098101cd0 2024-09-11T23:44:37.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb08e7fc640 1 -- 172.21.15.83:0/2406323333 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088035070 con 0x7fb098101cd0 2024-09-11T23:44:37.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.090+0000 7fb09f805640 1 -- 172.21.15.83:0/2406323333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0981a0270 con 0x7fb098101cd0 2024-09-11T23:44:37.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.091+0000 7fb09f805640 1 -- 172.21.15.83:0/2406323333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0981a04a0 con 0x7fb098101cd0 2024-09-11T23:44:37.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.092+0000 7fb08e7fc640 1 -- 172.21.15.83:0/2406323333 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fb088042030 con 0x7fb098101cd0 2024-09-11T23:44:37.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.093+0000 7fb08e7fc640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0840a2ba0 0x7fb0840a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.093+0000 7fb08e7fc640 1 -- 172.21.15.83:0/2406323333 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fb0880eda10 con 0x7fb098101cd0 2024-09-11T23:44:37.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.093+0000 7fb09d5a1640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0840a2ba0 0x7fb0840a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.094+0000 7fb09d5a1640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0840a2ba0 0x7fb0840a5060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb094005fd0 tx=0x7fb09400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.097+0000 7fb08e7fc640 1 -- 172.21.15.83:0/2406323333 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0880bc500 con 0x7fb098101cd0 2024-09-11T23:44:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:44:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:44:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:44:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi062 2024-09-11T23:44:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:44:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi062 to 17088M 2024-09-11T23:44:37.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:37.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:37.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:37.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:37.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:36 smithi120 ceph-mon[35182]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:37.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:37.271+0000 7fb09f805640 1 -- 172.21.15.83:0/2406323333 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb09806a680 con 0x7fb0840a2ba0 2024-09-11T23:44:37.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:37 smithi083 ceph-mon[34805]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:44:37.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:37 smithi083 ceph-mon[34805]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:44:37.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:37 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:37.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:37 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:37.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:37 smithi083 ceph-mon[34805]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:44:37.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:37 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:44:37.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:37 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:44:37.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:37 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:38.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:37 smithi120 ceph-mon[35182]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:44:38.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:37 smithi120 ceph-mon[35182]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:44:38.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:37 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:38.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:37 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:38.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:37 smithi120 ceph-mon[35182]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:44:38.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:37 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:44:38.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:37 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:44:38.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:37 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:38.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:37 smithi062 ceph-mon[30958]: from='osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:44:38.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:37 smithi062 ceph-mon[30958]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:44:38.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:37 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:38.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:37 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:38.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:37 smithi062 ceph-mon[30958]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:44:38.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:37 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:44:38.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:37 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:44:38.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:37 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:38.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:38 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:44:38.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:38 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:44:38.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:38 smithi083 ceph-mon[34805]: osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] boot 2024-09-11T23:44:38.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:38 smithi083 ceph-mon[34805]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:44:38.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:38 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:38.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:38 smithi083 ceph-mon[34805]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:39.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:38 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:44:39.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:38 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:44:39.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:38 smithi120 ceph-mon[35182]: osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] boot 2024-09-11T23:44:39.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:38 smithi120 ceph-mon[35182]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:44:39.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:38 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:39.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:38 smithi120 ceph-mon[35182]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:39.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:38 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:44:39.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:38 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:44:39.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:38 smithi062 ceph-mon[30958]: osd.0 [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] boot 2024-09-11T23:44:39.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:38 smithi062 ceph-mon[30958]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:44:39.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:38 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:44:39.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:38 smithi062 ceph-mon[30958]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:44:40.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:39 smithi120 ceph-mon[35182]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T23:44:40.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:39 smithi120 ceph-mon[35182]: from='client.? 172.21.15.83:0/1065359077' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f352db6b-9b13-4d9b-9a26-7d6f16e9dd05"}]: dispatch 2024-09-11T23:44:40.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:39 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f352db6b-9b13-4d9b-9a26-7d6f16e9dd05"}]: dispatch 2024-09-11T23:44:40.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:39 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f352db6b-9b13-4d9b-9a26-7d6f16e9dd05"}]': finished 2024-09-11T23:44:40.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:39 smithi120 ceph-mon[35182]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:44:40.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:39 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:44:40.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:39 smithi083 ceph-mon[34805]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T23:44:40.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:39 smithi083 ceph-mon[34805]: from='client.? 172.21.15.83:0/1065359077' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f352db6b-9b13-4d9b-9a26-7d6f16e9dd05"}]: dispatch 2024-09-11T23:44:40.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:39 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f352db6b-9b13-4d9b-9a26-7d6f16e9dd05"}]: dispatch 2024-09-11T23:44:40.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:39 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f352db6b-9b13-4d9b-9a26-7d6f16e9dd05"}]': finished 2024-09-11T23:44:40.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:39 smithi083 ceph-mon[34805]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:44:40.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:39 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:44:40.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:39 smithi062 ceph-mon[30958]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T23:44:40.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:39 smithi062 ceph-mon[30958]: from='client.? 172.21.15.83:0/1065359077' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f352db6b-9b13-4d9b-9a26-7d6f16e9dd05"}]: dispatch 2024-09-11T23:44:40.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:39 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f352db6b-9b13-4d9b-9a26-7d6f16e9dd05"}]: dispatch 2024-09-11T23:44:40.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:39 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f352db6b-9b13-4d9b-9a26-7d6f16e9dd05"}]': finished 2024-09-11T23:44:40.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:39 smithi062 ceph-mon[30958]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:44:40.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:39 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:44:41.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:40 smithi120 ceph-mon[35182]: from='client.? 172.21.15.83:0/1455146069' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:44:41.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:40 smithi120 ceph-mon[35182]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:41.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:40 smithi083 ceph-mon[34805]: from='client.? 172.21.15.83:0/1455146069' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:44:41.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:40 smithi083 ceph-mon[34805]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:41.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:40 smithi062 ceph-mon[30958]: from='client.? 172.21.15.83:0/1455146069' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:44:41.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:40 smithi062 ceph-mon[30958]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:43.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:42 smithi120 ceph-mon[35182]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:43.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:42 smithi083 ceph-mon[34805]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:43.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:42 smithi062 ceph-mon[30958]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:45.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:44 smithi120 ceph-mon[35182]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:45.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:44 smithi083 ceph-mon[34805]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:45.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:44 smithi062 ceph-mon[30958]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:46.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:46 smithi083 ceph-mon[34805]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:47.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:46 smithi120 ceph-mon[35182]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:47.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:46 smithi062 ceph-mon[30958]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:47.998 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:44:47.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:47.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:47 smithi083 ceph-mon[34805]: Deploying daemon osd.1 on smithi083 2024-09-11T23:44:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:44:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:47 smithi120 ceph-mon[35182]: Deploying daemon osd.1 on smithi083 2024-09-11T23:44:48.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:44:48.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:48.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:47 smithi062 ceph-mon[30958]: Deploying daemon osd.1 on smithi083 2024-09-11T23:44:49.102 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:48 smithi083 ceph-mon[34805]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:49.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:48 smithi120 ceph-mon[35182]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:48 smithi062 ceph-mon[30958]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:50.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:50 smithi083 ceph-mon[34805]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:51.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:50 smithi120 ceph-mon[35182]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:51.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:50 smithi062 ceph-mon[30958]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:52.890 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:52 smithi083 ceph-mon[34805]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:52 smithi120 ceph-mon[35182]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:53.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:52 smithi062 ceph-mon[30958]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:54.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:54.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:54.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:54.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:54.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:54.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:54.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:54.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:54.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:55.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:54 smithi120 ceph-mon[35182]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:55.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:54 smithi083 ceph-mon[34805]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:55.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:54 smithi062 ceph-mon[30958]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:57.646 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:57 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:57.646 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:57 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:57.646 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:57 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:57.646 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:57 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:57.646 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:57 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:57.646 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:57 smithi083 ceph-mon[34805]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:57.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:57 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:57.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:57 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:57.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:57 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:57.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:57 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:57.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:57 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:57.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:57 smithi062 ceph-mon[30958]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:57.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:57 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:57.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:57 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:57.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:57 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:57.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:57 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:57.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:57 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:57.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:57 smithi120 ceph-mon[35182]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:57.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.936+0000 7fb08e7fc640 1 -- 172.21.15.83:0/2406323333 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb09806a680 con 0x7fb0840a2ba0 2024-09-11T23:44:57.936 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 1 on host 'smithi083' 2024-09-11T23:44:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 -- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0840a2ba0 msgr2=0x7fb0840a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0840a2ba0 0x7fb0840a5060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb094005fd0 tx=0x7fb09400b410 comp rx=0 tx=0).stop 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 -- 172.21.15.83:0/2406323333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb098101cd0 msgr2=0x7fb098198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb098101cd0 0x7fb098198450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb088009230 tx=0x7fb088004360 comp rx=0 tx=0).stop 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 -- 172.21.15.83:0/2406323333 shutdown_connections 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0840a2ba0 0x7fb0840a5060 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb098102690 0x7fb09819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb098101cd0 0x7fb098198450 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 --2- 172.21.15.83:0/2406323333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb098100ad0 0x7fb098197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 -- 172.21.15.83:0/2406323333 >> 172.21.15.83:0/2406323333 conn(0x7fb0980fc280 msgr2=0x7fb098069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 -- 172.21.15.83:0/2406323333 shutdown_connections 2024-09-11T23:44:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:44:57.939+0000 7fb09f805640 1 -- 172.21.15.83:0/2406323333 wait complete. 2024-09-11T23:44:58.567 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:58 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:58.568 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:58 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:58.568 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:58 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:58.592 DEBUG:teuthology.orchestra.run.smithi083:osd.1> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T23:44:58.595 INFO:tasks.cephadm:Deploying osd.2 on smithi120 with /dev/vg_nvme/lv_4... 2024-09-11T23:44:58.595 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:44:58.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:58 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:58.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:58 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:58.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:58 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:44:58.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:58 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:58.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:58 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:58.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:58 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:44:59.612 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:44:59 smithi120 ceph-mon[35182]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:59.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:44:59 smithi083 ceph-mon[34805]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:44:59.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:44:59 smithi062 ceph-mon[30958]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:45:00.016 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:44:59 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1[39266]: 2024-09-11T23:44:59.795+0000 7f7b1432c640 -1 osd.1 0 log_to_monitors true 2024-09-11T23:45:00.510 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:00 smithi120 ceph-mon[35182]: from='osd.1 [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:45:00.510 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:00 smithi120 ceph-mon[35182]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:45:00.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:00 smithi083 ceph-mon[34805]: from='osd.1 [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:45:00.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:00 smithi083 ceph-mon[34805]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:45:00.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:00 smithi062 ceph-mon[30958]: from='osd.1 [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:45:00.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:00 smithi062 ceph-mon[30958]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:45:01.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:45:01.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='osd.1 [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:45:01.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:45:01.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:01.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:45:01.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:45:01.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.590 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:01.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:01.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:01.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.591 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:01 smithi120 ceph-mon[35182]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:45:01.593 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:45:01.621 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_4 2024-09-11T23:45:01.667 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1[39266]: 2024-09-11T23:45:01.416+0000 7f7b102af640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T23:45:01.667 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1[39266]: 2024-09-11T23:45:01.419+0000 7f7b0c0d7640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:45:01.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:45:01.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='osd.1 [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:45:01.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:45:01.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:01.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:45:01.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:45:01.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:01.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:01.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:01.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:01 smithi083 ceph-mon[34805]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:45:01.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:45:01.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='osd.1 [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:45:01.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:45:01.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:01.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:45:01.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:45:01.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:01.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:01.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:01.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:01.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:01 smithi062 ceph-mon[30958]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:45:01.882 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:45:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.379+0000 7ff838eb5640 1 -- 172.21.15.120:0/3241938349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff834100ad0 msgr2=0x7ff834100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.379+0000 7ff838eb5640 1 --2- 172.21.15.120:0/3241938349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff834100ad0 0x7ff834100ed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff824009900 tx=0x7ff82402a410 comp rx=0 tx=0).stop 2024-09-11T23:45:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.379+0000 7ff838eb5640 1 -- 172.21.15.120:0/3241938349 shutdown_connections 2024-09-11T23:45:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.379+0000 7ff838eb5640 1 --2- 172.21.15.120:0/3241938349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff834102690 0x7ff834108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.379+0000 7ff838eb5640 1 --2- 172.21.15.120:0/3241938349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff834101cd0 0x7ff834102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.379+0000 7ff838eb5640 1 --2- 172.21.15.120:0/3241938349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff834100ad0 0x7ff834100ed0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.379+0000 7ff838eb5640 1 -- 172.21.15.120:0/3241938349 >> 172.21.15.120:0/3241938349 conn(0x7ff8340fc280 msgr2=0x7ff8340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.379+0000 7ff838eb5640 1 -- 172.21.15.120:0/3241938349 shutdown_connections 2024-09-11T23:45:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.379+0000 7ff838eb5640 1 -- 172.21.15.120:0/3241938349 wait complete. 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.380+0000 7ff838eb5640 1 Processor -- start 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.380+0000 7ff838eb5640 1 -- start start 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.380+0000 7ff838eb5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff834100ad0 0x7ff834197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.380+0000 7ff838eb5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff834101cd0 0x7ff8341984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.380+0000 7ff838eb5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff834102690 0x7ff83419f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.380+0000 7ff838eb5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff83410d8e0 con 0x7ff834100ad0 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.380+0000 7ff838eb5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff83410d760 con 0x7ff834102690 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.380+0000 7ff838eb5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff83410da60 con 0x7ff834101cd0 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff831d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff834101cd0 0x7ff8341984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff831d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff834101cd0 0x7ff8341984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41684/0 (socket says 172.21.15.120:41684) 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff831d9b640 1 -- 172.21.15.120:0/3003254268 learned_addr learned my addr 172.21.15.120:0/3003254268 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:45:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff83259c640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff834100ad0 0x7ff834197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff832d9d640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff834102690 0x7ff83419f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff831d9b640 1 -- 172.21.15.120:0/3003254268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff834102690 msgr2=0x7ff83419f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff831d9b640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff834102690 0x7ff83419f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff831d9b640 1 -- 172.21.15.120:0/3003254268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff834100ad0 msgr2=0x7ff834197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff831d9b640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff834100ad0 0x7ff834197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff831d9b640 1 -- 172.21.15.120:0/3003254268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83419fba0 con 0x7ff834101cd0 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff831d9b640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff834101cd0 0x7ff8341984b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff82800d770 tx=0x7ff82800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff8237fe640 1 -- 172.21.15.120:0/3003254268 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828017070 con 0x7ff834101cd0 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff8237fe640 1 -- 172.21.15.120:0/3003254268 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff828005500 con 0x7ff834101cd0 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.381+0000 7ff838eb5640 1 -- 172.21.15.120:0/3003254268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83419fe90 con 0x7ff834101cd0 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.382+0000 7ff8237fe640 1 -- 172.21.15.120:0/3003254268 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828004e00 con 0x7ff834101cd0 2024-09-11T23:45:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.382+0000 7ff838eb5640 1 -- 172.21.15.120:0/3003254268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff834109960 con 0x7ff834101cd0 2024-09-11T23:45:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.383+0000 7ff838eb5640 1 -- 172.21.15.120:0/3003254268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff834107f20 con 0x7ff834101cd0 2024-09-11T23:45:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.384+0000 7ff8237fe640 1 -- 172.21.15.120:0/3003254268 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7ff828004fa0 con 0x7ff834101cd0 2024-09-11T23:45:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.385+0000 7ff8237fe640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff81c0a6f30 0x7ff81c0a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.385+0000 7ff8237fe640 1 -- 172.21.15.120:0/3003254268 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7ff8280cd020 con 0x7ff834101cd0 2024-09-11T23:45:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.385+0000 7ff83259c640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff81c0a6f30 0x7ff81c0a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.386+0000 7ff83259c640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff81c0a6f30 0x7ff81c0a93f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff824005ab0 tx=0x7ff824000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:02.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.389+0000 7ff8237fe640 1 -- 172.21.15.120:0/3003254268 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff82809b2a0 con 0x7ff834101cd0 2024-09-11T23:45:02.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:02.570+0000 7ff838eb5640 1 -- 172.21.15.120:0/3003254268 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff834062630 con 0x7ff81c0a6f30 2024-09-11T23:45:02.569 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:02 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi083 2024-09-11T23:45:02.569 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:02 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi083 to 17088M 2024-09-11T23:45:02.569 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:02 smithi120 ceph-mon[35182]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T23:45:02.569 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:02.569 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:02.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:02 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi083 2024-09-11T23:45:02.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:02 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi083 to 17088M 2024-09-11T23:45:02.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:02 smithi062 ceph-mon[30958]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T23:45:02.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:02.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:02 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi083 2024-09-11T23:45:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:02 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi083 to 17088M 2024-09-11T23:45:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:02 smithi083 ceph-mon[34805]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T23:45:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:03.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:03 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:45:03.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:03 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:45:03.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:03 smithi120 ceph-mon[35182]: osd.1 [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] boot 2024-09-11T23:45:03.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:03 smithi120 ceph-mon[35182]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:45:03.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:03.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:03 smithi120 ceph-mon[35182]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:45:03.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:45:03.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:45:03.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:03.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:03 smithi120 ceph-mon[35182]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:45:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:03 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:45:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:03 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:45:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:03 smithi083 ceph-mon[34805]: osd.1 [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] boot 2024-09-11T23:45:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:03 smithi083 ceph-mon[34805]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:45:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:03 smithi083 ceph-mon[34805]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:45:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:45:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:45:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:03.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:03 smithi083 ceph-mon[34805]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:45:03.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:03 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:45:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:03 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:45:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:03 smithi062 ceph-mon[30958]: osd.1 [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] boot 2024-09-11T23:45:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:03 smithi062 ceph-mon[30958]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:45:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:45:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:03 smithi062 ceph-mon[30958]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:45:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:45:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:45:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:03 smithi062 ceph-mon[30958]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:45:04.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:04 smithi062 ceph-mon[30958]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T23:45:04.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:04 smithi120 ceph-mon[35182]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T23:45:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:04 smithi083 ceph-mon[34805]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T23:45:05.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:05 smithi062 ceph-mon[30958]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:05.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:05 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/2056698487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f"}]: dispatch 2024-09-11T23:45:05.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:05 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f"}]: dispatch 2024-09-11T23:45:05.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:05 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f"}]': finished 2024-09-11T23:45:05.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:05 smithi062 ceph-mon[30958]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:45:05.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:05 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:05.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:05 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/147765176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:45:05.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:05 smithi120 ceph-mon[35182]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:05.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:05 smithi120 ceph-mon[35182]: from='client.? 172.21.15.120:0/2056698487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f"}]: dispatch 2024-09-11T23:45:05.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:05 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f"}]: dispatch 2024-09-11T23:45:05.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:05 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f"}]': finished 2024-09-11T23:45:05.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:05 smithi120 ceph-mon[35182]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:45:05.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:05 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:05.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:05 smithi120 ceph-mon[35182]: from='client.? 172.21.15.120:0/147765176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:45:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:05 smithi083 ceph-mon[34805]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:05 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/2056698487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f"}]: dispatch 2024-09-11T23:45:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:05 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f"}]: dispatch 2024-09-11T23:45:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:05 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f"}]': finished 2024-09-11T23:45:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:05 smithi083 ceph-mon[34805]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:45:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:05 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:05 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/147765176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:45:07.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:06 smithi120 ceph-mon[35182]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:07.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:06 smithi083 ceph-mon[34805]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:06 smithi062 ceph-mon[30958]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:09.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:08 smithi120 ceph-mon[35182]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:09.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:08 smithi083 ceph-mon[34805]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:09.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:08 smithi062 ceph-mon[30958]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:10.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:10 smithi120 ceph-mon[35182]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:11.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:10 smithi083 ceph-mon[34805]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:11.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:10 smithi062 ceph-mon[30958]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:13.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:12 smithi120 ceph-mon[35182]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:12 smithi083 ceph-mon[34805]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:13.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:12 smithi062 ceph-mon[30958]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:13.869 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:45:13.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:13.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:13 smithi120 ceph-mon[35182]: Deploying daemon osd.2 on smithi120 2024-09-11T23:45:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:45:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:13 smithi083 ceph-mon[34805]: Deploying daemon osd.2 on smithi120 2024-09-11T23:45:14.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:45:14.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:14.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:13 smithi062 ceph-mon[30958]: Deploying daemon osd.2 on smithi120 2024-09-11T23:45:15.077 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:14 smithi120 ceph-mon[35182]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:15.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:14 smithi083 ceph-mon[34805]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:15.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:14 smithi062 ceph-mon[30958]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:17.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:16 smithi120 ceph-mon[35182]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:17.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:16 smithi083 ceph-mon[34805]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:17.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:16 smithi062 ceph-mon[30958]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:18.998 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:18 smithi120 ceph-mon[35182]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:19.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:18 smithi083 ceph-mon[34805]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:19.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:18 smithi062 ceph-mon[30958]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:20.060 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:19 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:20.060 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:19 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:20.060 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:19 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:20.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:19 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:19 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:19 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:20.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:19 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:20.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:19 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:20.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:19 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:21.088 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:20 smithi120 ceph-mon[35182]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:21.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:20 smithi083 ceph-mon[34805]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:21.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:20 smithi062 ceph-mon[30958]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.240+0000 7ff8237fe640 1 -- 172.21.15.120:0/3003254268 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff834062630 con 0x7ff81c0a6f30 2024-09-11T23:45:23.241 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 2 on host 'smithi120' 2024-09-11T23:45:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 -- 172.21.15.120:0/3003254268 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff81c0a6f30 msgr2=0x7ff81c0a93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff81c0a6f30 0x7ff81c0a93f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff824005ab0 tx=0x7ff824000bc0 comp rx=0 tx=0).stop 2024-09-11T23:45:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 -- 172.21.15.120:0/3003254268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff834101cd0 msgr2=0x7ff8341984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff834101cd0 0x7ff8341984b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff82800d770 tx=0x7ff82800dc30 comp rx=0 tx=0).stop 2024-09-11T23:45:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 -- 172.21.15.120:0/3003254268 shutdown_connections 2024-09-11T23:45:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff81c0a6f30 0x7ff81c0a93f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff834102690 0x7ff83419f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff834101cd0 0x7ff8341984b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 --2- 172.21.15.120:0/3003254268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff834100ad0 0x7ff834197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:23.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 -- 172.21.15.120:0/3003254268 >> 172.21.15.120:0/3003254268 conn(0x7ff8340fc280 msgr2=0x7ff8341050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:23.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 -- 172.21.15.120:0/3003254268 shutdown_connections 2024-09-11T23:45:23.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:45:23.243+0000 7ff838eb5640 1 -- 172.21.15.120:0/3003254268 wait complete. 2024-09-11T23:45:23.558 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.558 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.558 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:23.558 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:23.558 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.558 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:23 smithi120 ceph-mon[35182]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:23.559 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:23.559 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.559 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:23 smithi083 ceph-mon[34805]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:23.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:23.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:23 smithi062 ceph-mon[30958]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:23.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:23.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:23.876 DEBUG:teuthology.orchestra.run.smithi120:osd.2> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T23:45:23.879 INFO:tasks.cephadm:Deploying osd.3 on smithi062 with /dev/vg_nvme/lv_3... 2024-09-11T23:45:23.879 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:45:24.130 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:24.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:24 smithi062 ceph-mon[30958]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:25.072 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:24 smithi120 ceph-mon[35182]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:25.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:24 smithi083 ceph-mon[34805]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:26.225 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:26 smithi062 ceph-mon[30958]: from='osd.2 [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:45:26.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:26 smithi062 ceph-mon[30958]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:45:26.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:26 smithi120 ceph-mon[35182]: from='osd.2 [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:45:26.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:26 smithi120 ceph-mon[35182]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:45:26.371 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:25 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2[40143]: 2024-09-11T23:45:25.956+0000 7f13f9014640 -1 osd.2 0 log_to_monitors true 2024-09-11T23:45:26.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:26 smithi083 ceph-mon[34805]: from='osd.2 [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:45:26.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:26 smithi083 ceph-mon[34805]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:45:26.845 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:45:26.874 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2024-09-11T23:45:27.133 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:45:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:45:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='osd.2 [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:45:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:45:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:27.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:27.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:27.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.371 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2[40143]: 2024-09-11T23:45:27.025+0000 7f13f4f97640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T23:45:27.371 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2[40143]: 2024-09-11T23:45:27.029+0000 7f13f0dbf640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:45:27.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:45:27.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:45:27.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:27.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='osd.2 [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:45:27.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:45:27.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:27.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:27.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:27.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:27.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:45:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:45:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='osd.2 [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:45:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:45:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:27.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:27.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:27.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:27.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:27.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.648+0000 7fddbdbc2640 1 -- 172.21.15.62:0/1752157305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb81037c0 msgr2=0x7fddb810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:27.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.648+0000 7fddbdbc2640 1 --2- 172.21.15.62:0/1752157305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb81037c0 0x7fddb810a050 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009900 tx=0x7fddb002a410 comp rx=0 tx=0).stop 2024-09-11T23:45:27.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.649+0000 7fddbdbc2640 1 -- 172.21.15.62:0/1752157305 shutdown_connections 2024-09-11T23:45:27.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.649+0000 7fddbdbc2640 1 --2- 172.21.15.62:0/1752157305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb81037c0 0x7fddb810a050 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.649+0000 7fddbdbc2640 1 --2- 172.21.15.62:0/1752157305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102e00 0x7fddb8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.649+0000 7fddbdbc2640 1 --2- 172.21.15.62:0/1752157305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddb8101c00 0x7fddb8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.649+0000 7fddbdbc2640 1 -- 172.21.15.62:0/1752157305 >> 172.21.15.62:0/1752157305 conn(0x7fddb80fd390 msgr2=0x7fddb80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:27.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.649+0000 7fddbdbc2640 1 -- 172.21.15.62:0/1752157305 shutdown_connections 2024-09-11T23:45:27.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.649+0000 7fddbdbc2640 1 -- 172.21.15.62:0/1752157305 wait complete. 2024-09-11T23:45:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.649+0000 7fddbdbc2640 1 Processor -- start 2024-09-11T23:45:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.649+0000 7fddbdbc2640 1 -- start start 2024-09-11T23:45:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddbdbc2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8101c00 0x7fddb8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddbdbc2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddb8102e00 0x7fddb8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddbdbc2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb81037c0 0x7fddb819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddbdbc2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fddb810d9d0 con 0x7fddb81037c0 2024-09-11T23:45:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddbdbc2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fddb810d850 con 0x7fddb8101c00 2024-09-11T23:45:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddbdbc2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fddb810db50 con 0x7fddb8102e00 2024-09-11T23:45:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb81037c0 0x7fddb819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb81037c0 0x7fddb819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46546/0 (socket says 172.21.15.62:46546) 2024-09-11T23:45:27.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb7fff640 1 -- 172.21.15.62:0/3011698928 learned_addr learned my addr 172.21.15.62:0/3011698928 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:45:27.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb6ffd640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddb8102e00 0x7fddb8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:27.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb77fe640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8101c00 0x7fddb8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:27.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb7fff640 1 -- 172.21.15.62:0/3011698928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddb8102e00 msgr2=0x7fddb8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:27.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb7fff640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddb8102e00 0x7fddb8198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb7fff640 1 -- 172.21.15.62:0/3011698928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8101c00 msgr2=0x7fddb8198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:27.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb7fff640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8101c00 0x7fddb8198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb7fff640 1 -- 172.21.15.62:0/3011698928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb819fc80 con 0x7fddb81037c0 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb77fe640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8101c00 0x7fddb8198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb7fff640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb81037c0 0x7fddb819f610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fddb00314e0 tx=0x7fddb0032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb4ff9640 1 -- 172.21.15.62:0/3011698928 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb003f030 con 0x7fddb81037c0 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb4ff9640 1 -- 172.21.15.62:0/3011698928 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb0003ce0 con 0x7fddb81037c0 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddbdbc2640 1 -- 172.21.15.62:0/3011698928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb819ff10 con 0x7fddb81037c0 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddb4ff9640 1 -- 172.21.15.62:0/3011698928 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0035070 con 0x7fddb81037c0 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.650+0000 7fddbdbc2640 1 -- 172.21.15.62:0/3011698928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb81a0440 con 0x7fddb81037c0 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.651+0000 7fddbdbc2640 1 -- 172.21.15.62:0/3011698928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb8108fd0 con 0x7fddb81037c0 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.653+0000 7fddb4ff9640 1 -- 172.21.15.62:0/3011698928 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fddb0042030 con 0x7fddb81037c0 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.653+0000 7fddb4ff9640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdda00a2ba0 0x7fdda00a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.653+0000 7fddb77fe640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdda00a2ba0 0x7fdda00a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:27.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.653+0000 7fddb4ff9640 1 -- 172.21.15.62:0/3011698928 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7fddb00ede40 con 0x7fddb81037c0 2024-09-11T23:45:27.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.654+0000 7fddb77fe640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdda00a2ba0 0x7fdda00a5060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fddb8102c60 tx=0x7fdda8009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:27.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.658+0000 7fddb4ff9640 1 -- 172.21.15.62:0/3011698928 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddb00bc4f0 con 0x7fddb81037c0 2024-09-11T23:45:27.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:27.842+0000 7fddbdbc2640 1 -- 172.21.15.62:0/3011698928 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fddb8062630 con 0x7fdda00a2ba0 2024-09-11T23:45:28.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:28 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi120 2024-09-11T23:45:28.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:28 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-11T23:45:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:28 smithi062 ceph-mon[30958]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:45:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:28 smithi062 ceph-mon[30958]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T23:45:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:28 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:28 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:28 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:45:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:28 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:45:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:28 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:28.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:28 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi120 2024-09-11T23:45:28.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:28 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-11T23:45:28.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:28 smithi120 ceph-mon[35182]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:45:28.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:28 smithi120 ceph-mon[35182]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T23:45:28.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:28 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:28.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:28 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:28.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:28 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:45:28.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:28 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:45:28.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:28 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:28 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi120 2024-09-11T23:45:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:28 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-11T23:45:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:28 smithi083 ceph-mon[34805]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:45:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:28 smithi083 ceph-mon[34805]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T23:45:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:28 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:28 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:28 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:45:28.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:28 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:45:28.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:28 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:29.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:29 smithi062 ceph-mon[30958]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:45:29.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:29 smithi062 ceph-mon[30958]: osd.2 [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] boot 2024-09-11T23:45:29.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:29 smithi062 ceph-mon[30958]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:45:29.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:29 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:29.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:29 smithi062 ceph-mon[30958]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:29.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:29 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:45:29.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:29 smithi120 ceph-mon[35182]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:45:29.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:29 smithi120 ceph-mon[35182]: osd.2 [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] boot 2024-09-11T23:45:29.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:29 smithi120 ceph-mon[35182]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:45:29.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:29 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:29.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:29 smithi120 ceph-mon[35182]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:29.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:29 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:45:29.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:29 smithi083 ceph-mon[34805]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:45:29.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:29 smithi083 ceph-mon[34805]: osd.2 [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] boot 2024-09-11T23:45:29.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:29 smithi083 ceph-mon[34805]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:45:29.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:29 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:45:29.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:29 smithi083 ceph-mon[34805]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:45:29.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:29 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:45:30.515 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:30 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:45:30.515 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:30 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:45:30.515 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:45:30.516 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:30 smithi062 ceph-mon[30958]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T23:45:30.516 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:45:30.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:30 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:45:30.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:30 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:45:30.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:45:30.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:30 smithi120 ceph-mon[35182]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T23:45:30.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:45:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:30 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:45:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:30 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:45:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:45:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:30 smithi083 ceph-mon[34805]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T23:45:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:45:31.215 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:45:30 smithi062 sudo[49388]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:45:31.215 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:45:30 smithi062 sudo[49388]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:31.215 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:45:30 smithi062 sudo[49388]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:31.216 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:45:30 smithi062 sudo[49388]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:31.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:31.295+0000 7fddb4ff9640 1 -- 172.21.15.62:0/3011698928 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fddb00f2070 con 0x7fddb81037c0 2024-09-11T23:45:31.572 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:31 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:45:31.572 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:31 smithi083 ceph-mon[34805]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T23:45:31.572 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:31 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1114479733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37bbe64a-1332-4aee-83f4-67fb8f0092b7"}]: dispatch 2024-09-11T23:45:31.572 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:31 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1114479733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37bbe64a-1332-4aee-83f4-67fb8f0092b7"}]': finished 2024-09-11T23:45:31.572 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:31 smithi083 ceph-mon[34805]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T23:45:31.572 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:31 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:31.573 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:31 smithi083 ceph-mon[34805]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:31.573 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:31 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2478456092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:45:31.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:31 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:45:31.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:31 smithi120 ceph-mon[35182]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T23:45:31.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:31 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1114479733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37bbe64a-1332-4aee-83f4-67fb8f0092b7"}]: dispatch 2024-09-11T23:45:31.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:31 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1114479733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37bbe64a-1332-4aee-83f4-67fb8f0092b7"}]': finished 2024-09-11T23:45:31.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:31 smithi120 ceph-mon[35182]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T23:45:31.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:31 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:31.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:31 smithi120 ceph-mon[35182]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:31.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:31 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2478456092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:45:31.715 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:45:31 smithi062 sudo[49437]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:45:31.716 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:45:31 smithi062 sudo[49437]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:31.716 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:45:31 smithi062 sudo[49437]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:31.716 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 11 23:45:31 smithi062 sudo[49437]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:31.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:31 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:45:31.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:31 smithi062 ceph-mon[30958]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T23:45:31.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:31 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1114479733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37bbe64a-1332-4aee-83f4-67fb8f0092b7"}]: dispatch 2024-09-11T23:45:31.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:31 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1114479733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37bbe64a-1332-4aee-83f4-67fb8f0092b7"}]': finished 2024-09-11T23:45:31.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:31 smithi062 ceph-mon[30958]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T23:45:31.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:31 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:31.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:31 smithi062 ceph-mon[30958]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:31.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:31 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2478456092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:45:31.826 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:45:31 smithi083 sudo[42955]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:45:31.826 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:45:31 smithi083 sudo[42955]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:31.826 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:45:31 smithi083 sudo[42955]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:31.826 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:45:31 smithi083 sudo[42955]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:32.167 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:45:31 smithi083 sudo[42958]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:45:32.167 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:45:31 smithi083 sudo[42958]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:32.167 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:45:31 smithi083 sudo[42958]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:32.167 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 11 23:45:31 smithi083 sudo[42958]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:32.354 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:32 smithi120 sudo[44040]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:45:32.354 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:32 smithi120 sudo[44040]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:32.354 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:32 smithi120 sudo[44040]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:32.354 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:32 smithi120 sudo[44040]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:32.355 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:32 smithi120 ceph-mon[35182]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:45:32.355 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:32 smithi120 ceph-mon[35182]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T23:45:32.355 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:32.355 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:32.355 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:32.621 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:32 smithi120 sudo[44043]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:45:32.621 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:32 smithi120 sudo[44043]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:32.621 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:32 smithi120 sudo[44043]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:32.621 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 11 23:45:32 smithi120 sudo[44043]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:32 smithi083 ceph-mon[34805]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:45:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:32 smithi083 ceph-mon[34805]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T23:45:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:32 smithi062 ceph-mon[30958]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:45:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:32 smithi062 ceph-mon[30958]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T23:45:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:32.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:32.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:33.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:32 smithi062 sudo[50184]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:45:33.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:32 smithi062 sudo[50184]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:33.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:32 smithi062 sudo[50184]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:33.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:32 smithi062 sudo[50184]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:33.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:33 smithi062 sudo[50188]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:45:33.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:33 smithi062 sudo[50188]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:33.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:33 smithi062 sudo[50188]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:33.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:33 smithi062 sudo[50188]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:33.787 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:33 smithi120 sudo[44046]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:45:33.787 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:33 smithi120 sudo[44046]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:33.787 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:33 smithi120 sudo[44046]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:33.787 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:33 smithi120 sudo[44046]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:34.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:33 smithi120 sudo[44050]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:45:34.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:33 smithi120 sudo[44050]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:34.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:33 smithi120 sudo[44050]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:34.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:33 smithi120 sudo[44050]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:34.287 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 sudo[42961]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:45:34.287 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 sudo[42961]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:34.287 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 sudo[42961]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:34.287 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 sudo[42961]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:34.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:34.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:34 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:45:34.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:34 smithi062 ceph-mon[30958]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:34.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:34 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:45:34.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:45:34.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:45:34.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:45:34.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:45:34.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:45:34.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:45:34.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:34 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:34.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:34 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:45:34.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:34 smithi120 ceph-mon[35182]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:34.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:34 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:45:34.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:34 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:45:34.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:34 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:45:34.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:34 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:45:34.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:34 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:45:34.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:34 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:45:34.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:34 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:45:34.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 sudo[42964]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:45:34.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 sudo[42964]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:45:34.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 sudo[42964]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:45:34.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 sudo[42964]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:45:34.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:34.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:45:34.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 ceph-mon[34805]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:34.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:45:34.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:45:34.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:45:34.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:45:34.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:45:34.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:45:34.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:34 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:45:35.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:35 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:45:35.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:35 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:45:35.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:35 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:45:35.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:45:35.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:45:35.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:45:35.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:35 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:45:35.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:35 smithi062 ceph-mon[30958]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:35.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:35 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:45:35.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:35 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:45:35.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:35 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:45:35.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:45:35.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:45:35.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:45:35.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:35 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:45:35.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:35 smithi120 ceph-mon[35182]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:35.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:35 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:45:35.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:35 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:45:35.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:35 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:45:35.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:45:35.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:45:35.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:45:35.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:35 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:45:35.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:35 smithi083 ceph-mon[34805]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:36.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:36 smithi062 ceph-mon[30958]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:36 smithi120 ceph-mon[35182]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:37.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:36 smithi083 ceph-mon[34805]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:38.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:37 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:45:38.224 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:37 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:38.375 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:37 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:45:38.375 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:37 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:38.429 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:37 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:45:38.430 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:37 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:39.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:38 smithi062 ceph-mon[30958]: Deploying daemon osd.3 on smithi062 2024-09-11T23:45:39.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:38 smithi062 ceph-mon[30958]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T23:45:39.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:38 smithi120 ceph-mon[35182]: Deploying daemon osd.3 on smithi062 2024-09-11T23:45:39.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:38 smithi120 ceph-mon[35182]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T23:45:39.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:38 smithi083 ceph-mon[34805]: Deploying daemon osd.3 on smithi062 2024-09-11T23:45:39.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:38 smithi083 ceph-mon[34805]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T23:45:40.944 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:40 smithi062 ceph-mon[30958]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T23:45:41.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:40 smithi120 ceph-mon[35182]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T23:45:41.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:40 smithi083 ceph-mon[34805]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T23:45:43.037 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:42 smithi062 ceph-mon[30958]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T23:45:43.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:42 smithi120 ceph-mon[35182]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T23:45:43.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:42 smithi083 ceph-mon[34805]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T23:45:44.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:44.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:44.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:45.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:45 smithi083 ceph-mon[34805]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T23:45:45.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:45 smithi062 ceph-mon[30958]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T23:45:45.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:45 smithi120 ceph-mon[35182]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T23:45:47.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:46 smithi120 ceph-mon[35182]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:45:47.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:46 smithi083 ceph-mon[34805]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:45:47.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:46 smithi062 ceph-mon[30958]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:45:48.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:48 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:48.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:48 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:48.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:48 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:48.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:48 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:48.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:48 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:48.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:48 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:48.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:48 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:48.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:48 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:48.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:48 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:48.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:48 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.384+0000 7fddb4ff9640 1 -- 172.21.15.62:0/3011698928 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fddb8062630 con 0x7fdda00a2ba0 2024-09-11T23:45:48.383 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 3 on host 'smithi062' 2024-09-11T23:45:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 -- 172.21.15.62:0/3011698928 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdda00a2ba0 msgr2=0x7fdda00a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdda00a2ba0 0x7fdda00a5060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fddb8102c60 tx=0x7fdda8009570 comp rx=0 tx=0).stop 2024-09-11T23:45:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 -- 172.21.15.62:0/3011698928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb81037c0 msgr2=0x7fddb819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb81037c0 0x7fddb819f610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fddb00314e0 tx=0x7fddb0032cd0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 -- 172.21.15.62:0/3011698928 shutdown_connections 2024-09-11T23:45:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdda00a2ba0 0x7fdda00a5060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb81037c0 0x7fddb819f610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddb8102e00 0x7fddb8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 --2- 172.21.15.62:0/3011698928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8101c00 0x7fddb8198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 -- 172.21.15.62:0/3011698928 >> 172.21.15.62:0/3011698928 conn(0x7fddb80fd390 msgr2=0x7fddb81061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 -- 172.21.15.62:0/3011698928 shutdown_connections 2024-09-11T23:45:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:45:48.386+0000 7fddbdbc2640 1 -- 172.21.15.62:0/3011698928 wait complete. 2024-09-11T23:45:48.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:48 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:48.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:48 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:48.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:48 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:48.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:48 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:48.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:48 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:49.040 DEBUG:teuthology.orchestra.run.smithi062:osd.3> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T23:45:49.042 INFO:tasks.cephadm:Deploying osd.4 on smithi083 with /dev/vg_nvme/lv_3... 2024-09-11T23:45:49.042 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:45:49.294 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:45:49.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:49 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:49.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:49 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:49.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:49 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:49.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:49 smithi120 ceph-mon[35182]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:45:49.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:49 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:49.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:49 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:49.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:49 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:49.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:49 smithi062 ceph-mon[30958]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:45:49.415 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:49 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:49.415 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:49 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:49.415 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:49 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:49.415 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:49 smithi083 ceph-mon[34805]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:45:50.216 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 11 23:45:49 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3[51397]: 2024-09-11T23:45:49.903+0000 7efd1ffd1640 -1 osd.3 0 log_to_monitors true 2024-09-11T23:45:50.487 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:50 smithi083 ceph-mon[34805]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:45:50.573 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:50 smithi062 ceph-mon[30958]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:45:50.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:50 smithi120 ceph-mon[35182]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:45:51.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:51 smithi083 ceph-mon[34805]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:45:51.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:51 smithi083 ceph-mon[34805]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T23:45:51.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:51 smithi083 ceph-mon[34805]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:45:51.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:51 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:51.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:51 smithi083 ceph-mon[34805]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:51.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:51 smithi062 ceph-mon[30958]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:45:51.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:51 smithi062 ceph-mon[30958]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T23:45:51.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:51 smithi062 ceph-mon[30958]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:45:51.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:51 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:51.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:51 smithi062 ceph-mon[30958]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:51.661 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 11 23:45:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3[51397]: 2024-09-11T23:45:51.398+0000 7efd1bf54640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T23:45:51.662 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 11 23:45:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3[51397]: 2024-09-11T23:45:51.402+0000 7efd17d7c640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:45:51.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:51 smithi120 ceph-mon[35182]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:45:51.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:51 smithi120 ceph-mon[35182]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T23:45:51.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:51 smithi120 ceph-mon[35182]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:45:51.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:51 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:51.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:51 smithi120 ceph-mon[35182]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:45:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:45:52.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-09-11T23:45:52.354 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:45:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:45:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T23:45:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:52.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:52.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.772 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:45:52.772 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T23:45:52.772 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:52.772 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:52.772 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.773 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.773 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:52.773 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:45:52.773 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.773 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:52.773 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:52.773 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:45:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.808+0000 7f24224c1640 1 -- 172.21.15.83:0/1627414725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c1026d0 msgr2=0x7f241c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.808+0000 7f24224c1640 1 --2- 172.21.15.83:0/1627414725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c1026d0 0x7f241c108f60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2414009ca0 tx=0x7f241402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:45:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.808+0000 7f24224c1640 1 -- 172.21.15.83:0/1627414725 shutdown_connections 2024-09-11T23:45:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.808+0000 7f24224c1640 1 --2- 172.21.15.83:0/1627414725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c1026d0 0x7f241c108f60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.808+0000 7f24224c1640 1 --2- 172.21.15.83:0/1627414725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c101d10 0x7f241c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.808+0000 7f24224c1640 1 --2- 172.21.15.83:0/1627414725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f241c100b10 0x7f241c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.808+0000 7f24224c1640 1 -- 172.21.15.83:0/1627414725 >> 172.21.15.83:0/1627414725 conn(0x7f241c0fc2a0 msgr2=0x7f241c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.808+0000 7f24224c1640 1 -- 172.21.15.83:0/1627414725 shutdown_connections 2024-09-11T23:45:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.808+0000 7f24224c1640 1 -- 172.21.15.83:0/1627414725 wait complete. 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.808+0000 7f24224c1640 1 Processor -- start 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f24224c1640 1 -- start start 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f24224c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f241c100b10 0x7f241c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f24224c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c101d10 0x7f241c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f24224c1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c1026d0 0x7f241c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f24224c1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10d8c0 con 0x7f241c1026d0 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f24224c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10d740 con 0x7f241c101d10 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f24224c1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10da40 con 0x7f241c100b10 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f241b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c101d10 0x7f241c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f241b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c101d10 0x7f241c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52128/0 (socket says 172.21.15.83:52128) 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f241b7fe640 1 -- 172.21.15.83:0/3076838169 learned_addr learned my addr 172.21.15.83:0/3076838169 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-11T23:45:52.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f2420a5e640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c1026d0 0x7f241c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f241bfff640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f241c100b10 0x7f241c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f241b7fe640 1 -- 172.21.15.83:0/3076838169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f241c100b10 msgr2=0x7f241c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f241b7fe640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f241c100b10 0x7f241c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f241b7fe640 1 -- 172.21.15.83:0/3076838169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c1026d0 msgr2=0x7f241c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f241b7fe640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c1026d0 0x7f241c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.809+0000 7f241b7fe640 1 -- 172.21.15.83:0/3076838169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c19faf0 con 0x7f241c101d10 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.810+0000 7f241b7fe640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c101d10 0x7f241c198400 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f240400d950 tx=0x7f240400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.810+0000 7f24197fa640 1 -- 172.21.15.83:0/3076838169 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404017030 con 0x7f241c101d10 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.810+0000 7f24224c1640 1 -- 172.21.15.83:0/3076838169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c19fde0 con 0x7f241c101d10 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.810+0000 7f24197fa640 1 -- 172.21.15.83:0/3076838169 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2404004020 con 0x7f241c101d10 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.810+0000 7f24197fa640 1 -- 172.21.15.83:0/3076838169 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24040047a0 con 0x7f241c101d10 2024-09-11T23:45:52.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.810+0000 7f24224c1640 1 -- 172.21.15.83:0/3076838169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c1a0290 con 0x7f241c101d10 2024-09-11T23:45:52.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.811+0000 7f24224c1640 1 -- 172.21.15.83:0/3076838169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241c107f10 con 0x7f241c101d10 2024-09-11T23:45:52.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.812+0000 7f24197fa640 1 -- 172.21.15.83:0/3076838169 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f24040041d0 con 0x7f241c101d10 2024-09-11T23:45:52.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.812+0000 7f24197fa640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f24080a2bb0 0x7f24080a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:52.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.813+0000 7f24197fa640 1 -- 172.21.15.83:0/3076838169 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f24040cd740 con 0x7f241c101d10 2024-09-11T23:45:52.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.813+0000 7f241bfff640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f24080a2bb0 0x7f24080a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:52.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.813+0000 7f241bfff640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f24080a2bb0 0x7f24080a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f240c0058c0 tx=0x7f240c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:52.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.817+0000 7f24197fa640 1 -- 172.21.15.83:0/3076838169 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f240409b8a0 con 0x7f241c101d10 2024-09-11T23:45:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:45:52.991+0000 7f24224c1640 1 -- 172.21.15.83:0/3076838169 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f241c062630 con 0x7f24080a2bb0 2024-09-11T23:45:53.644 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:45:53.645 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:45:53.645 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi062 2024-09-11T23:45:53.645 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi062 to 8544M 2024-09-11T23:45:53.645 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] boot 2024-09-11T23:45:53.645 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T23:45:53.645 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:53.645 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:45:53.645 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:45:53.645 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:45:53.645 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:45:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:45:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi062 2024-09-11T23:45:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi062 to 8544M 2024-09-11T23:45:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] boot 2024-09-11T23:45:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T23:45:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:45:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:45:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:45:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:53.793 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:45:53.793 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:45:53.793 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi062 2024-09-11T23:45:53.793 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi062 to 8544M 2024-09-11T23:45:53.793 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: osd.3 [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] boot 2024-09-11T23:45:53.793 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T23:45:53.793 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:45:53.793 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:45:53.793 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:45:53.793 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:45:53.794 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:54.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:54 smithi083 ceph-mon[34805]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:45:54.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:54 smithi083 ceph-mon[34805]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T23:45:54.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:54 smithi062 ceph-mon[30958]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:45:54.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:54 smithi062 ceph-mon[30958]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T23:45:54.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:54 smithi120 ceph-mon[35182]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:45:54.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:54 smithi120 ceph-mon[35182]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T23:45:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:55 smithi062 ceph-mon[30958]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T23:45:55.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:55 smithi062 ceph-mon[30958]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:45:55.723 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:55 smithi083 ceph-mon[34805]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T23:45:55.723 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:55 smithi083 ceph-mon[34805]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:45:55.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:55 smithi120 ceph-mon[35182]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T23:45:55.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:55 smithi120 ceph-mon[35182]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:45:56.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:56 smithi083 ceph-mon[34805]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T23:45:56.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:56 smithi083 ceph-mon[34805]: from='client.? 172.21.15.83:0/2437652546' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb698b7-3471-4503-8039-6519c4d2943e"}]: dispatch 2024-09-11T23:45:56.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:56 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb698b7-3471-4503-8039-6519c4d2943e"}]: dispatch 2024-09-11T23:45:56.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:56 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb698b7-3471-4503-8039-6519c4d2943e"}]': finished 2024-09-11T23:45:56.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:56 smithi083 ceph-mon[34805]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T23:45:56.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:56 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:45:56.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:56 smithi083 ceph-mon[34805]: from='client.? 172.21.15.83:0/457907595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:45:56.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:56 smithi062 ceph-mon[30958]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T23:45:56.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:56 smithi062 ceph-mon[30958]: from='client.? 172.21.15.83:0/2437652546' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb698b7-3471-4503-8039-6519c4d2943e"}]: dispatch 2024-09-11T23:45:56.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:56 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb698b7-3471-4503-8039-6519c4d2943e"}]: dispatch 2024-09-11T23:45:56.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:56 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb698b7-3471-4503-8039-6519c4d2943e"}]': finished 2024-09-11T23:45:56.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:56 smithi062 ceph-mon[30958]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T23:45:56.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:56 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:45:56.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:56 smithi062 ceph-mon[30958]: from='client.? 172.21.15.83:0/457907595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:45:56.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:56 smithi120 ceph-mon[35182]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T23:45:56.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:56 smithi120 ceph-mon[35182]: from='client.? 172.21.15.83:0/2437652546' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb698b7-3471-4503-8039-6519c4d2943e"}]: dispatch 2024-09-11T23:45:56.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:56 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb698b7-3471-4503-8039-6519c4d2943e"}]: dispatch 2024-09-11T23:45:56.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:56 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb698b7-3471-4503-8039-6519c4d2943e"}]': finished 2024-09-11T23:45:56.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:56 smithi120 ceph-mon[35182]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T23:45:56.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:56 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:45:56.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:56 smithi120 ceph-mon[35182]: from='client.? 172.21.15.83:0/457907595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:45:57.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:57 smithi062 ceph-mon[30958]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:45:57.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:57 smithi120 ceph-mon[35182]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:45:57.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:57 smithi083 ceph-mon[34805]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:45:59.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:45:58 smithi120 ceph-mon[35182]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-11T23:45:59.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:45:58 smithi083 ceph-mon[34805]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-11T23:45:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:45:58 smithi062 ceph-mon[30958]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-11T23:46:00.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:00 smithi083 ceph-mon[34805]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-11T23:46:01.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:00 smithi120 ceph-mon[35182]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-11T23:46:01.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:00 smithi062 ceph-mon[30958]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-11T23:46:03.117 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:02 smithi083 ceph-mon[34805]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:46:03.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:02 smithi120 ceph-mon[35182]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:46:03.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:02 smithi062 ceph-mon[30958]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:46:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:46:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:04.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:46:04.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:04.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:46:04.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:05.080 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:04 smithi120 ceph-mon[35182]: Deploying daemon osd.4 on smithi083 2024-09-11T23:46:05.080 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:04 smithi120 ceph-mon[35182]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:46:05.104 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:04 smithi083 ceph-mon[34805]: Deploying daemon osd.4 on smithi083 2024-09-11T23:46:05.104 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:04 smithi083 ceph-mon[34805]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:46:05.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:04 smithi062 ceph-mon[30958]: Deploying daemon osd.4 on smithi083 2024-09-11T23:46:05.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:04 smithi062 ceph-mon[30958]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:46:07.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:06 smithi120 ceph-mon[35182]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-11T23:46:07.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:06 smithi083 ceph-mon[34805]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-11T23:46:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:06 smithi062 ceph-mon[30958]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-11T23:46:09.086 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:08 smithi083 ceph-mon[34805]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:46:09.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:08 smithi120 ceph-mon[35182]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:46:09.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:08 smithi062 ceph-mon[30958]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:46:10.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:10.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:10.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:10.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:10.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:10.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:10.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:10.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:10.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:11.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:11 smithi062 ceph-mon[30958]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:11.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:11 smithi120 ceph-mon[35182]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:11.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:11 smithi083 ceph-mon[34805]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:13.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:12 smithi120 ceph-mon[35182]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:12 smithi083 ceph-mon[34805]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:13.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:12 smithi062 ceph-mon[30958]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.550+0000 7f24197fa640 1 -- 172.21.15.83:0/3076838169 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f241c062630 con 0x7f24080a2bb0 2024-09-11T23:46:13.550 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 4 on host 'smithi083' 2024-09-11T23:46:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.552+0000 7f24224c1640 1 -- 172.21.15.83:0/3076838169 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f24080a2bb0 msgr2=0x7f24080a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.552+0000 7f24224c1640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f24080a2bb0 0x7f24080a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f240c0058c0 tx=0x7f240c00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:46:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.552+0000 7f24224c1640 1 -- 172.21.15.83:0/3076838169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c101d10 msgr2=0x7f241c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.552+0000 7f24224c1640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c101d10 0x7f241c198400 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f240400d950 tx=0x7f240400de10 comp rx=0 tx=0).stop 2024-09-11T23:46:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.553+0000 7f24224c1640 1 -- 172.21.15.83:0/3076838169 shutdown_connections 2024-09-11T23:46:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.553+0000 7f24224c1640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f24080a2bb0 0x7f24080a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.553+0000 7f24224c1640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f241c1026d0 0x7f241c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.553+0000 7f24224c1640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c101d10 0x7f241c198400 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.553+0000 7f24224c1640 1 --2- 172.21.15.83:0/3076838169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f241c100b10 0x7f241c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.553+0000 7f24224c1640 1 -- 172.21.15.83:0/3076838169 >> 172.21.15.83:0/3076838169 conn(0x7f241c0fc2a0 msgr2=0x7f241c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.553+0000 7f24224c1640 1 -- 172.21.15.83:0/3076838169 shutdown_connections 2024-09-11T23:46:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:46:13.553+0000 7f24224c1640 1 -- 172.21.15.83:0/3076838169 wait complete. 2024-09-11T23:46:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:14.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:14.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:14.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:14.258 DEBUG:teuthology.orchestra.run.smithi083:osd.4> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T23:46:14.260 INFO:tasks.cephadm:Deploying osd.5 on smithi120 with /dev/vg_nvme/lv_3... 2024-09-11T23:46:14.261 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:46:14.513 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:46:14.940 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:14 smithi120 ceph-mon[35182]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:15.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:14 smithi083 ceph-mon[34805]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:15.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:14 smithi062 ceph-mon[30958]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:16.288 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:16 smithi120 ceph-mon[35182]: from='osd.4 [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:46:16.289 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:16 smithi120 ceph-mon[35182]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:46:16.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:16 smithi083 ceph-mon[34805]: from='osd.4 [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:46:16.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:16 smithi083 ceph-mon[34805]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:46:16.329 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 11 23:46:16 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-11T23:46:16.058+0000 7f4a1d872640 -1 osd.4 0 log_to_monitors true 2024-09-11T23:46:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:16 smithi062 ceph-mon[30958]: from='osd.4 [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:46:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:16 smithi062 ceph-mon[30958]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:46:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:46:17.116 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_3 2024-09-11T23:46:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:46:17.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:46:17.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T23:46:17.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:17.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='osd.4 [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:46:17.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:46:17.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:17.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:17.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:17.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:17.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:17.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:17 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:46:17.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T23:46:17.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:17.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='osd.4 [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:46:17.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:46:17.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:17.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:17.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:17.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:17.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:17.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:46:17.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T23:46:17.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='osd.4 [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:17 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:17.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.824+0000 7faf6e41d640 1 -- 172.21.15.120:0/2790684998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf68107900 msgr2=0x7faf68109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:17.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.824+0000 7faf6e41d640 1 --2- 172.21.15.120:0/2790684998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf68107900 0x7faf68109d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faf60009ca0 tx=0x7faf6002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:46:17.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.824+0000 7faf6e41d640 1 -- 172.21.15.120:0/2790684998 shutdown_connections 2024-09-11T23:46:17.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.824+0000 7faf6e41d640 1 --2- 172.21.15.120:0/2790684998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf68107900 0x7faf68109d50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.824+0000 7faf6e41d640 1 --2- 172.21.15.120:0/2790684998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf68104fd0 0x7faf681073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.824+0000 7faf6e41d640 1 --2- 172.21.15.120:0/2790684998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf681026a0 0x7faf68104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.824+0000 7faf6e41d640 1 -- 172.21.15.120:0/2790684998 >> 172.21.15.120:0/2790684998 conn(0x7faf680fc2a0 msgr2=0x7faf680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.824+0000 7faf6e41d640 1 -- 172.21.15.120:0/2790684998 shutdown_connections 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.824+0000 7faf6e41d640 1 -- 172.21.15.120:0/2790684998 wait complete. 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.825+0000 7faf6e41d640 1 Processor -- start 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.825+0000 7faf6e41d640 1 -- start start 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.825+0000 7faf6e41d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf681026a0 0x7faf68197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.825+0000 7faf6e41d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf68104fd0 0x7faf68198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.825+0000 7faf6e41d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf68107900 0x7faf6819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:17.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.825+0000 7faf6e41d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faf6810d8a0 con 0x7faf681026a0 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.825+0000 7faf6e41d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf6810d720 con 0x7faf68104fd0 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.825+0000 7faf6e41d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faf6810da20 con 0x7faf68107900 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6c9ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf68107900 0x7faf6819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6c9ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf68107900 0x7faf6819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49846/0 (socket says 172.21.15.120:49846) 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6c9ba640 1 -- 172.21.15.120:0/3178621872 learned_addr learned my addr 172.21.15.120:0/3178621872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf677fe640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf68104fd0 0x7faf68198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf67fff640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf681026a0 0x7faf68197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6c9ba640 1 -- 172.21.15.120:0/3178621872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf68104fd0 msgr2=0x7faf68198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6c9ba640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf68104fd0 0x7faf68198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6c9ba640 1 -- 172.21.15.120:0/3178621872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf681026a0 msgr2=0x7faf68197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6c9ba640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf681026a0 0x7faf68197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:17.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6c9ba640 1 -- 172.21.15.120:0/3178621872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6819fa70 con 0x7faf68107900 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf677fe640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf68104fd0 0x7faf68198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6c9ba640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf68107900 0x7faf6819f400 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faf60005e30 tx=0x7faf60031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf657fa640 1 -- 172.21.15.120:0/3178621872 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60003b70 con 0x7faf68107900 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6e41d640 1 -- 172.21.15.120:0/3178621872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6819fd00 con 0x7faf68107900 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf657fa640 1 -- 172.21.15.120:0/3178621872 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf60003d10 con 0x7faf68107900 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf657fa640 1 -- 172.21.15.120:0/3178621872 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6002fd30 con 0x7faf68107900 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.826+0000 7faf6e41d640 1 -- 172.21.15.120:0/3178621872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf681a01e0 con 0x7faf68107900 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.827+0000 7faf6e41d640 1 -- 172.21.15.120:0/3178621872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf680fdd90 con 0x7faf68107900 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.828+0000 7faf657fa640 1 -- 172.21.15.120:0/3178621872 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faf60004050 con 0x7faf68107900 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.829+0000 7faf657fa640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faf500a2bb0 0x7faf500a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.830+0000 7faf657fa640 1 -- 172.21.15.120:0/3178621872 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7faf600ef060 con 0x7faf68107900 2024-09-11T23:46:17.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.830+0000 7faf67fff640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faf500a2bb0 0x7faf500a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:17.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.830+0000 7faf67fff640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faf500a2bb0 0x7faf500a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf5800bbc0 tx=0x7faf5800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:17.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:17.834+0000 7faf657fa640 1 -- 172.21.15.120:0/3178621872 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf600bd270 con 0x7faf68107900 2024-09-11T23:46:17.917 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-11T23:46:17.537+0000 7f4a197f5640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T23:46:17.917 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 11 23:46:17 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-11T23:46:17.544+0000 7f4a11615640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:46:18.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:18.007+0000 7faf6e41d640 1 -- 172.21.15.120:0/3178621872 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7faf68062630 con 0x7faf500a2bb0 2024-09-11T23:46:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:18 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi083 2024-09-11T23:46:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:18 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi083 to 8544M 2024-09-11T23:46:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:18 smithi120 ceph-mon[35182]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:46:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:18 smithi120 ceph-mon[35182]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T23:46:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:18 smithi120 ceph-mon[35182]: from='osd.4 ' entity='osd.4' 2024-09-11T23:46:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:46:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:46:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:18.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:18 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi083 2024-09-11T23:46:18.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:18 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi083 to 8544M 2024-09-11T23:46:18.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:18 smithi083 ceph-mon[34805]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:46:18.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:18 smithi083 ceph-mon[34805]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T23:46:18.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:18.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:18.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:18 smithi083 ceph-mon[34805]: from='osd.4 ' entity='osd.4' 2024-09-11T23:46:18.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:46:18.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:46:18.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:18.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:18 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi083 2024-09-11T23:46:18.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:18 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi083 to 8544M 2024-09-11T23:46:18.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:18 smithi062 ceph-mon[30958]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:46:18.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:18 smithi062 ceph-mon[30958]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T23:46:18.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:18.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:18.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:18 smithi062 ceph-mon[30958]: from='osd.4 ' entity='osd.4' 2024-09-11T23:46:18.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:46:18.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:46:18.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:19.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:19 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:46:19.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:19 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:46:19.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:19 smithi083 ceph-mon[34805]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:46:19.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:19 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:19.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:19 smithi083 ceph-mon[34805]: osd.4 [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] boot 2024-09-11T23:46:19.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:19 smithi083 ceph-mon[34805]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T23:46:19.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:19 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:19.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:19 smithi083 ceph-mon[34805]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:19.431 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:19 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:46:19.431 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:19 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:46:19.431 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:19 smithi120 ceph-mon[35182]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:46:19.432 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:19 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:19.432 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:19 smithi120 ceph-mon[35182]: osd.4 [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] boot 2024-09-11T23:46:19.432 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:19 smithi120 ceph-mon[35182]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T23:46:19.432 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:19 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:19.432 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:19 smithi120 ceph-mon[35182]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:19.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:19 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:46:19.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:19 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:46:19.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:19 smithi062 ceph-mon[30958]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:46:19.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:19 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:19.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:19 smithi062 ceph-mon[30958]: osd.4 [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] boot 2024-09-11T23:46:19.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:19 smithi062 ceph-mon[30958]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T23:46:19.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:19 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:46:19.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:19 smithi062 ceph-mon[30958]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-11T23:46:20.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:20 smithi120 ceph-mon[35182]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T23:46:20.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:20 smithi120 ceph-mon[35182]: from='client.? 172.21.15.120:0/2323695557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce904ade-27b5-4171-af4c-7c4894f40c16"}]: dispatch 2024-09-11T23:46:20.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:20 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce904ade-27b5-4171-af4c-7c4894f40c16"}]: dispatch 2024-09-11T23:46:20.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:20 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce904ade-27b5-4171-af4c-7c4894f40c16"}]': finished 2024-09-11T23:46:20.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:20 smithi120 ceph-mon[35182]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T23:46:20.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:20 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:20 smithi083 ceph-mon[34805]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T23:46:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:20 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/2323695557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce904ade-27b5-4171-af4c-7c4894f40c16"}]: dispatch 2024-09-11T23:46:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:20 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce904ade-27b5-4171-af4c-7c4894f40c16"}]: dispatch 2024-09-11T23:46:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:20 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce904ade-27b5-4171-af4c-7c4894f40c16"}]': finished 2024-09-11T23:46:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:20 smithi083 ceph-mon[34805]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T23:46:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:20 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:20.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:20 smithi062 ceph-mon[30958]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T23:46:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:20 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/2323695557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce904ade-27b5-4171-af4c-7c4894f40c16"}]: dispatch 2024-09-11T23:46:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:20 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce904ade-27b5-4171-af4c-7c4894f40c16"}]: dispatch 2024-09-11T23:46:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:20 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce904ade-27b5-4171-af4c-7c4894f40c16"}]': finished 2024-09-11T23:46:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:20 smithi062 ceph-mon[30958]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T23:46:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:20 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:21 smithi120 ceph-mon[35182]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:21 smithi120 ceph-mon[35182]: from='client.? 172.21.15.120:0/1758315000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:46:21.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:21 smithi083 ceph-mon[34805]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:21.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:21 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/1758315000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:46:21.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:21 smithi062 ceph-mon[30958]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:21.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:21 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/1758315000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:46:23.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:22 smithi120 ceph-mon[35182]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:22 smithi083 ceph-mon[34805]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:23.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:22 smithi062 ceph-mon[30958]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:24 smithi120 ceph-mon[35182]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:24 smithi083 ceph-mon[34805]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:24 smithi062 ceph-mon[30958]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:26.979 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:26 smithi120 ceph-mon[35182]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:26 smithi083 ceph-mon[34805]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:27.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:26 smithi062 ceph-mon[30958]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:29.024 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:28 smithi120 ceph-mon[35182]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:29.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:28 smithi083 ceph-mon[34805]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:28 smithi062 ceph-mon[30958]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:30.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:29 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:46:30.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:29 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:30.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:29 smithi120 ceph-mon[35182]: Deploying daemon osd.5 on smithi120 2024-09-11T23:46:30.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:29 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:46:30.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:29 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:30.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:29 smithi083 ceph-mon[34805]: Deploying daemon osd.5 on smithi120 2024-09-11T23:46:30.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:29 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:46:30.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:29 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:30.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:29 smithi062 ceph-mon[30958]: Deploying daemon osd.5 on smithi120 2024-09-11T23:46:32.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:31 smithi120 ceph-mon[35182]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:32.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:31 smithi083 ceph-mon[34805]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:32.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:31 smithi062 ceph-mon[30958]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:34.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:33 smithi120 ceph-mon[35182]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:34.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:33 smithi083 ceph-mon[34805]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:34.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:33 smithi062 ceph-mon[30958]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:36.113 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:35 smithi120 ceph-mon[35182]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:36.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:36.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:36.114 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:36.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:35 smithi083 ceph-mon[34805]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:36.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:36.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:36.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:36.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:35 smithi062 ceph-mon[30958]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:36.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:36.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:36.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:38.078 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:37 smithi120 ceph-mon[35182]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:38.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:37 smithi083 ceph-mon[34805]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:38.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:37 smithi062 ceph-mon[30958]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:39.582 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:39 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:39.583 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:39 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:39.583 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:39 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:39.583 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:39 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:39.583 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:39 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:39.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.623+0000 7faf657fa640 1 -- 172.21.15.120:0/3178621872 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faf68062630 con 0x7faf500a2bb0 2024-09-11T23:46:39.623 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 5 on host 'smithi120' 2024-09-11T23:46:39.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.625+0000 7faf6e41d640 1 -- 172.21.15.120:0/3178621872 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faf500a2bb0 msgr2=0x7faf500a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:39.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.625+0000 7faf6e41d640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faf500a2bb0 0x7faf500a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf5800bbc0 tx=0x7faf5800b390 comp rx=0 tx=0).stop 2024-09-11T23:46:39.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.626+0000 7faf6e41d640 1 -- 172.21.15.120:0/3178621872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf68107900 msgr2=0x7faf6819f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:39.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.626+0000 7faf6e41d640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf68107900 0x7faf6819f400 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faf60005e30 tx=0x7faf60031ee0 comp rx=0 tx=0).stop 2024-09-11T23:46:39.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.626+0000 7faf6e41d640 1 -- 172.21.15.120:0/3178621872 shutdown_connections 2024-09-11T23:46:39.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.626+0000 7faf6e41d640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faf500a2bb0 0x7faf500a5070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:39.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.626+0000 7faf6e41d640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf68107900 0x7faf6819f400 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:39.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.626+0000 7faf6e41d640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf68104fd0 0x7faf68198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:39.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.626+0000 7faf6e41d640 1 --2- 172.21.15.120:0/3178621872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf681026a0 0x7faf68197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:39.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.626+0000 7faf6e41d640 1 -- 172.21.15.120:0/3178621872 >> 172.21.15.120:0/3178621872 conn(0x7faf680fc2a0 msgr2=0x7faf68102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:39.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.626+0000 7faf6e41d640 1 -- 172.21.15.120:0/3178621872 shutdown_connections 2024-09-11T23:46:39.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:46:39.626+0000 7faf6e41d640 1 -- 172.21.15.120:0/3178621872 wait complete. 2024-09-11T23:46:39.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:39 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:39.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:39 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:39.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:39 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:39.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:39 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:39.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:39 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:39.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:39 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:39.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:39 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:39.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:39 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:39.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:39 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:39.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:39 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:40.347 DEBUG:teuthology.orchestra.run.smithi120:osd.5> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T23:46:40.349 INFO:tasks.cephadm:Deploying osd.6 on smithi062 with /dev/vg_nvme/lv_2... 2024-09-11T23:46:40.349 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:46:40.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:40 smithi120 ceph-mon[35182]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:40.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:40 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:40.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:40 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:40.575 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:40 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:40.614 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:40.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:40 smithi083 ceph-mon[34805]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:40.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:40 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:40.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:40 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:40.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:40 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:40.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:40 smithi062 ceph-mon[30958]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:40.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:40 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:40.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:40 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:40.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:40 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:41.580 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 11 23:46:41 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-11T23:46:41.327+0000 7f8639e52640 -1 osd.5 0 log_to_monitors true 2024-09-11T23:46:42.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:42 smithi120 ceph-mon[35182]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:42.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:42 smithi120 ceph-mon[35182]: from='osd.5 [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:46:42.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:42 smithi120 ceph-mon[35182]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:46:42.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:42 smithi083 ceph-mon[34805]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:42.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:42 smithi083 ceph-mon[34805]: from='osd.5 [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:46:42.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:42 smithi083 ceph-mon[34805]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:46:42.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:42 smithi062 ceph-mon[30958]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:42.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:42 smithi062 ceph-mon[30958]: from='osd.5 [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:46:42.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:42 smithi062 ceph-mon[30958]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:46:43.319 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:46:43.346 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2024-09-11T23:46:43.612 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='osd.5 [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:43.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:43.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:46:43.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T23:46:43.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:43.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='osd.5 [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:46:43.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:46:43.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:43.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:43.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:43.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:43.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:43 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:46:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T23:46:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='osd.5 [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:46:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:46:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:43.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:46:43.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:43.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:43.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:43.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:43 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:46:44.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.074+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/1286718562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4102690 msgr2=0x7fdfe4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:44.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.074+0000 7fdfeb9fa640 1 --2- 172.21.15.62:0/1286718562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4102690 0x7fdfe4108f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009ca0 tx=0x7fdfdc02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.074+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/1286718562 shutdown_connections 2024-09-11T23:46:44.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.074+0000 7fdfeb9fa640 1 --2- 172.21.15.62:0/1286718562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4102690 0x7fdfe4108f20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.074+0000 7fdfeb9fa640 1 --2- 172.21.15.62:0/1286718562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfe4101cd0 0x7fdfe4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.074+0000 7fdfeb9fa640 1 --2- 172.21.15.62:0/1286718562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfe4100ad0 0x7fdfe4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.074+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/1286718562 >> 172.21.15.62:0/1286718562 conn(0x7fdfe40fc280 msgr2=0x7fdfe40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:44.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.074+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/1286718562 shutdown_connections 2024-09-11T23:46:44.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.074+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/1286718562 wait complete. 2024-09-11T23:46:44.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfeb9fa640 1 Processor -- start 2024-09-11T23:46:44.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfeb9fa640 1 -- start start 2024-09-11T23:46:44.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfeb9fa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4100ad0 0x7fdfe406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfeb9fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfe4101cd0 0x7fdfe419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfeb9fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfe4102690 0x7fdfe419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfeb9fa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe410c730 con 0x7fdfe4100ad0 2024-09-11T23:46:44.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfeb9fa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe410c5b0 con 0x7fdfe4102690 2024-09-11T23:46:44.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfeb9fa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe410c8b0 con 0x7fdfe4101cd0 2024-09-11T23:46:44.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfe9796640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4100ad0 0x7fdfe406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfe9796640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4100ad0 0x7fdfe406ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40554/0 (socket says 172.21.15.62:40554) 2024-09-11T23:46:44.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.075+0000 7fdfe9796640 1 -- 172.21.15.62:0/2320802562 learned_addr learned my addr 172.21.15.62:0/2320802562 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:46:44.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfe9f97640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfe4102690 0x7fdfe419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfe8f95640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfe4101cd0 0x7fdfe419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfe9796640 1 -- 172.21.15.62:0/2320802562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfe4101cd0 msgr2=0x7fdfe419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:44.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfe9796640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfe4101cd0 0x7fdfe419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfe9796640 1 -- 172.21.15.62:0/2320802562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfe4102690 msgr2=0x7fdfe419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:44.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfe9796640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfe4102690 0x7fdfe419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfe9796640 1 -- 172.21.15.62:0/2320802562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe419fa00 con 0x7fdfe4100ad0 2024-09-11T23:46:44.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfe8f95640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfe4101cd0 0x7fdfe419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:46:44.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfe9796640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4100ad0 0x7fdfe406ab10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000d950 tx=0x7fdfe000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:44.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfda7fc640 1 -- 172.21.15.62:0/2320802562 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0017020 con 0x7fdfe4100ad0 2024-09-11T23:46:44.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/2320802562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe419fcf0 con 0x7fdfe4100ad0 2024-09-11T23:46:44.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfda7fc640 1 -- 172.21.15.62:0/2320802562 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe0009c30 con 0x7fdfe4100ad0 2024-09-11T23:46:44.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/2320802562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe41a01c0 con 0x7fdfe4100ad0 2024-09-11T23:46:44.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.076+0000 7fdfda7fc640 1 -- 172.21.15.62:0/2320802562 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000f740 con 0x7fdfe4100ad0 2024-09-11T23:46:44.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.077+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/2320802562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe4107e00 con 0x7fdfe4100ad0 2024-09-11T23:46:44.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.079+0000 7fdfda7fc640 1 -- 172.21.15.62:0/2320802562 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdfe001a020 con 0x7fdfe4100ad0 2024-09-11T23:46:44.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.079+0000 7fdfda7fc640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdfd00a2c00 0x7fdfd00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.079+0000 7fdfda7fc640 1 -- 172.21.15.62:0/2320802562 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4253+0+0 (secure 0 0 0) 0x7fdfe00cddf0 con 0x7fdfe4100ad0 2024-09-11T23:46:44.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.079+0000 7fdfe8f95640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdfd00a2c00 0x7fdfd00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.080+0000 7fdfe8f95640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdfd00a2c00 0x7fdfd00a50c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdfe406a890 tx=0x7fdfd4005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:44.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.083+0000 7fdfda7fc640 1 -- 172.21.15.62:0/2320802562 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdfe0021070 con 0x7fdfe4100ad0 2024-09-11T23:46:44.121 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-11T23:46:43.681+0000 7f8635dd5640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T23:46:44.121 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 11 23:46:43 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-11T23:46:43.687+0000 7f862dbf5640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:46:44.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:46:44.269+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/2320802562 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdfe4062630 con 0x7fdfd00a2c00 2024-09-11T23:46:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi120 2024-09-11T23:46:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-11T23:46:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:46:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T23:46:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: from='osd.5 ' entity='osd.5' 2024-09-11T23:46:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:46:44.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:46:44.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:44.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi120 2024-09-11T23:46:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-11T23:46:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:46:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T23:46:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: from='osd.5 ' entity='osd.5' 2024-09-11T23:46:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:46:44.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:46:44.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:44.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:44.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:44.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi120 2024-09-11T23:46:44.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-11T23:46:44.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:46:44.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T23:46:44.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:44.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:44.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: from='osd.5 ' entity='osd.5' 2024-09-11T23:46:44.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:46:44.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:46:44.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:44.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:45.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:45 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:46:45.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:45 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:46:45.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:45 smithi120 ceph-mon[35182]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:46:45.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:45 smithi120 ceph-mon[35182]: osd.5 [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] boot 2024-09-11T23:46:45.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:45 smithi120 ceph-mon[35182]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T23:46:45.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:45 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:45.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:45 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:46:45.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:45 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:46:45.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:45 smithi062 ceph-mon[30958]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:46:45.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:45 smithi062 ceph-mon[30958]: osd.5 [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] boot 2024-09-11T23:46:45.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:45 smithi062 ceph-mon[30958]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T23:46:45.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:45 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:45.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:45 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:46:45.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:45 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:46:45.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:45 smithi083 ceph-mon[34805]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:46:45.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:45 smithi083 ceph-mon[34805]: osd.5 [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] boot 2024-09-11T23:46:45.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:45 smithi083 ceph-mon[34805]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T23:46:45.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:45 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:46:46.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:46 smithi120 ceph-mon[35182]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:46.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:46 smithi120 ceph-mon[35182]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T23:46:46.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:46 smithi083 ceph-mon[34805]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:46.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:46 smithi083 ceph-mon[34805]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T23:46:46.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:46 smithi062 ceph-mon[30958]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:46:46.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:46 smithi062 ceph-mon[30958]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T23:46:47.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:47 smithi120 ceph-mon[35182]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T23:46:47.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:47 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2504376930' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ea78a9f-4447-413d-a698-12bf6a2f1398"}]: dispatch 2024-09-11T23:46:47.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:47 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2504376930' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ea78a9f-4447-413d-a698-12bf6a2f1398"}]': finished 2024-09-11T23:46:47.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:47 smithi120 ceph-mon[35182]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T23:46:47.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:46:47.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:47 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3105672035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:46:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:47 smithi083 ceph-mon[34805]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T23:46:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:47 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2504376930' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ea78a9f-4447-413d-a698-12bf6a2f1398"}]: dispatch 2024-09-11T23:46:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:47 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2504376930' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ea78a9f-4447-413d-a698-12bf6a2f1398"}]': finished 2024-09-11T23:46:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:47 smithi083 ceph-mon[34805]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T23:46:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:46:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:47 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3105672035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:46:47.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:47 smithi062 ceph-mon[30958]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T23:46:47.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:47 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2504376930' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ea78a9f-4447-413d-a698-12bf6a2f1398"}]: dispatch 2024-09-11T23:46:47.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:47 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2504376930' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ea78a9f-4447-413d-a698-12bf6a2f1398"}]': finished 2024-09-11T23:46:47.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:47 smithi062 ceph-mon[30958]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T23:46:47.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:46:47.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:47 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3105672035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:46:48.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:48 smithi062 ceph-mon[30958]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:46:48.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:48 smithi062 ceph-mon[30958]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T23:46:48.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:48 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:46:49.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:48 smithi120 ceph-mon[35182]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:46:49.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:48 smithi120 ceph-mon[35182]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T23:46:49.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:48 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:46:49.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:48 smithi083 ceph-mon[34805]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:46:49.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:48 smithi083 ceph-mon[34805]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T23:46:49.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:48 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:46:50.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:50 smithi062 ceph-mon[30958]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:46:51.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:50 smithi120 ceph-mon[35182]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:46:51.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:50 smithi083 ceph-mon[34805]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:46:52.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:52 smithi062 ceph-mon[30958]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-11T23:46:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:52 smithi120 ceph-mon[35182]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-11T23:46:53.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:52 smithi083 ceph-mon[34805]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-11T23:46:54.787 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:54 smithi062 ceph-mon[30958]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-11T23:46:54.969 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:54 smithi083 ceph-mon[34805]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-11T23:46:55.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:54 smithi120 ceph-mon[35182]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-11T23:46:57.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:56 smithi062 ceph-mon[30958]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:46:57.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:56 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:46:57.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:56 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:57.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:56 smithi062 ceph-mon[30958]: Deploying daemon osd.6 on smithi062 2024-09-11T23:46:57.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:56 smithi120 ceph-mon[35182]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:46:57.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:56 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:46:57.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:56 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:57.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:56 smithi120 ceph-mon[35182]: Deploying daemon osd.6 on smithi062 2024-09-11T23:46:57.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:56 smithi083 ceph-mon[34805]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:46:57.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:56 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:46:57.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:56 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:57.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:56 smithi083 ceph-mon[34805]: Deploying daemon osd.6 on smithi062 2024-09-11T23:46:58.940 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:46:58 smithi062 ceph-mon[30958]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:46:59.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:46:58 smithi120 ceph-mon[35182]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:46:59.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:46:58 smithi083 ceph-mon[34805]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:47:01.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:00 smithi062 ceph-mon[30958]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T23:47:01.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:00 smithi120 ceph-mon[35182]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T23:47:01.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:00 smithi083 ceph-mon[34805]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T23:47:02.932 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:02 smithi062 ceph-mon[30958]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:02.932 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:02.933 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:02.933 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:02 smithi120 ceph-mon[35182]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:02 smithi083 ceph-mon[34805]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:05.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:04 smithi120 ceph-mon[35182]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:05.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:04 smithi083 ceph-mon[34805]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:05.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:04 smithi062 ceph-mon[30958]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:05.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.972+0000 7fdfda7fc640 1 -- 172.21.15.62:0/2320802562 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdfe4062630 con 0x7fdfd00a2c00 2024-09-11T23:47:05.972 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 6 on host 'smithi062' 2024-09-11T23:47:05.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/2320802562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdfd00a2c00 msgr2=0x7fdfd00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdfd00a2c00 0x7fdfd00a50c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdfe406a890 tx=0x7fdfd4005d20 comp rx=0 tx=0).stop 2024-09-11T23:47:05.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/2320802562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4100ad0 msgr2=0x7fdfe406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4100ad0 0x7fdfe406ab10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000d950 tx=0x7fdfe000de10 comp rx=0 tx=0).stop 2024-09-11T23:47:05.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/2320802562 shutdown_connections 2024-09-11T23:47:05.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdfd00a2c00 0x7fdfd00a50c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfe4102690 0x7fdfe419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfe4101cd0 0x7fdfe419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 --2- 172.21.15.62:0/2320802562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4100ad0 0x7fdfe406ab10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/2320802562 >> 172.21.15.62:0/2320802562 conn(0x7fdfe40fc280 msgr2=0x7fdfe4104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:05.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/2320802562 shutdown_connections 2024-09-11T23:47:05.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:47:05.974+0000 7fdfeb9fa640 1 -- 172.21.15.62:0/2320802562 wait complete. 2024-09-11T23:47:06.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:06 smithi062 ceph-mon[30958]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:06.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:06.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:06.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:06.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:06 smithi120 ceph-mon[35182]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:06.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:06.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:06.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:06.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:06 smithi083 ceph-mon[34805]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:06.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:06.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:06.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:06.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:06.675 DEBUG:teuthology.orchestra.run.smithi062:osd.6> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T23:47:06.677 INFO:tasks.cephadm:Deploying osd.7 on smithi083 with /dev/vg_nvme/lv_2... 2024-09-11T23:47:06.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:47:06.931 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:47:08.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:08 smithi062 ceph-mon[30958]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:08.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:08 smithi062 ceph-mon[30958]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:47:08.203 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 11 23:47:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-11T23:47:07.856+0000 7ff32e630640 -1 osd.6 0 log_to_monitors true 2024-09-11T23:47:08.322 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:08 smithi083 ceph-mon[34805]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:08.323 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:08 smithi083 ceph-mon[34805]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:47:08.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:08 smithi120 ceph-mon[35182]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:08.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:08 smithi120 ceph-mon[35182]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:47:09.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:09 smithi062 ceph-mon[30958]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:47:09.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:09 smithi062 ceph-mon[30958]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T23:47:09.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:09 smithi062 ceph-mon[30958]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:47:09.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:09 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:09.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:09 smithi062 ceph-mon[30958]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:47:09.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:09 smithi062 ceph-mon[30958]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T23:47:09.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:09 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:09.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:09 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:09.295 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:09 smithi083 ceph-mon[34805]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:47:09.296 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:09 smithi083 ceph-mon[34805]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T23:47:09.296 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:09 smithi083 ceph-mon[34805]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:47:09.296 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:09 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:09.296 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:09 smithi083 ceph-mon[34805]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:47:09.296 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:09 smithi083 ceph-mon[34805]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T23:47:09.296 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:09 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:09.296 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:09 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:09.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:09 smithi120 ceph-mon[35182]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:47:09.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:09 smithi120 ceph-mon[35182]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T23:47:09.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:09 smithi120 ceph-mon[35182]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:47:09.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:09 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:09.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:09 smithi120 ceph-mon[35182]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:47:09.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:09 smithi120 ceph-mon[35182]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T23:47:09.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:09 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:09.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:09 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:09.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:47:09.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-09-11T23:47:09.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 11 23:47:09 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-11T23:47:09.447+0000 7ff32a5b3640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T23:47:09.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 11 23:47:09 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-11T23:47:09.457+0000 7ff3223d3640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:47:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:47:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.379+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/3293782560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e88104f90 msgr2=0x7f4e88107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.379+0000 7f4e8f2c5640 1 --2- 172.21.15.83:0/3293782560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e88104f90 0x7f4e88107380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009900 tx=0x7f4e7c02a410 comp rx=0 tx=0).stop 2024-09-11T23:47:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/3293782560 shutdown_connections 2024-09-11T23:47:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 --2- 172.21.15.83:0/3293782560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e881078c0 0x7f4e88109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 --2- 172.21.15.83:0/3293782560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e88104f90 0x7f4e88107380 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 --2- 172.21.15.83:0/3293782560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e88102660 0x7f4e88104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/3293782560 >> 172.21.15.83:0/3293782560 conn(0x7f4e880fc280 msgr2=0x7f4e880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/3293782560 shutdown_connections 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/3293782560 wait complete. 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 Processor -- start 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 -- start start 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e88102660 0x7f4e881980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e88104f90 0x7f4e88198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e881078c0 0x7f4e8819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8810d930 con 0x7f4e881078c0 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8810d7b0 con 0x7f4e88104f90 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.380+0000 7f4e8f2c5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8810dab0 con 0x7f4e88102660 2024-09-11T23:47:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8c860640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e88104f90 0x7f4e88198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8c860640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e88104f90 0x7f4e88198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34792/0 (socket says 172.21.15.83:34792) 2024-09-11T23:47:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8c860640 1 -- 172.21.15.83:0/4103211725 learned_addr learned my addr 172.21.15.83:0/4103211725 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-11T23:47:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8d862640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e881078c0 0x7f4e8819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8c860640 1 -- 172.21.15.83:0/4103211725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e88102660 msgr2=0x7f4e881980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8c860640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e88102660 0x7f4e881980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8c860640 1 -- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e881078c0 msgr2=0x7f4e8819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8c860640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e881078c0 0x7f4e8819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8c860640 1 -- 172.21.15.83:0/4103211725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8819fcf0 con 0x7f4e88104f90 2024-09-11T23:47:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8d862640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e881078c0 0x7f4e8819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:47:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8c860640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e88104f90 0x7f4e88198600 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c031610 tx=0x7f4e7c0043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e867fc640 1 -- 172.21.15.83:0/4103211725 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c03f030 con 0x7f4e88104f90 2024-09-11T23:47:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e867fc640 1 -- 172.21.15.83:0/4103211725 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e7c003ce0 con 0x7f4e88104f90 2024-09-11T23:47:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/4103211725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8819ff80 con 0x7f4e88104f90 2024-09-11T23:47:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e867fc640 1 -- 172.21.15.83:0/4103211725 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c035070 con 0x7f4e88104f90 2024-09-11T23:47:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.381+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/4103211725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e881a0450 con 0x7f4e88104f90 2024-09-11T23:47:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.382+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/4103211725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e880fdd70 con 0x7f4e88104f90 2024-09-11T23:47:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.383+0000 7f4e867fc640 1 -- 172.21.15.83:0/4103211725 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4e7c042030 con 0x7f4e88104f90 2024-09-11T23:47:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.384+0000 7f4e867fc640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e700a2bb0 0x7f4e700a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:10.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.384+0000 7f4e867fc640 1 -- 172.21.15.83:0/4103211725 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4685+0+0 (secure 0 0 0) 0x7f4e7c0eeac0 con 0x7f4e88104f90 2024-09-11T23:47:10.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.384+0000 7f4e8d061640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e700a2bb0 0x7f4e700a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:10.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.385+0000 7f4e8d061640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e700a2bb0 0x7f4e700a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009710 tx=0x7f4e80006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:10.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.387+0000 7f4e867fc640 1 -- 172.21.15.83:0/4103211725 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e7c0bc850 con 0x7f4e88104f90 2024-09-11T23:47:10.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:10.564+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/4103211725 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4e88062630 con 0x7f4e700a2bb0 2024-09-11T23:47:10.563 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:10.563 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' 2024-09-11T23:47:10.564 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.564 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.564 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:10.564 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:10.564 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:10.564 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.564 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:10.564 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:10.564 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.564 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:10 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:10.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:10.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' 2024-09-11T23:47:10.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:10.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:10.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:10.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:10.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:10.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:10 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:10.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:10.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356]' entity='osd.6' 2024-09-11T23:47:10.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:10.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:10.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:10.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:10.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:10.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:10.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:10 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:11.654 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:47:11.654 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:47:11.654 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi062 2024-09-11T23:47:11.654 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi062 to 5696M 2024-09-11T23:47:11.654 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] boot 2024-09-11T23:47:11.654 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T23:47:11.654 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:11.654 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:47:11.655 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:47:11.655 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:47:11.655 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:11 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:11.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:47:11.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:47:11.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi062 2024-09-11T23:47:11.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi062 to 5696M 2024-09-11T23:47:11.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] boot 2024-09-11T23:47:11.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T23:47:11.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:11.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:47:11.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:47:11.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:47:11.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:11 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi062 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi062 to 5696M 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: osd.6 [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] boot 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:47:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:11 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:12.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:12 smithi083 ceph-mon[34805]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:12.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:12 smithi083 ceph-mon[34805]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T23:47:12.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:12 smithi062 ceph-mon[30958]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:12.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:12 smithi062 ceph-mon[30958]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T23:47:12.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:12 smithi120 ceph-mon[35182]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:47:12.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:12 smithi120 ceph-mon[35182]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T23:47:13.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:13 smithi083 ceph-mon[34805]: from='client.? 172.21.15.83:0/1957733901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9eb34260-3535-4e62-8aa5-276282a679cb"}]: dispatch 2024-09-11T23:47:13.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:13 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9eb34260-3535-4e62-8aa5-276282a679cb"}]: dispatch 2024-09-11T23:47:13.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:13 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9eb34260-3535-4e62-8aa5-276282a679cb"}]': finished 2024-09-11T23:47:13.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:13 smithi083 ceph-mon[34805]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T23:47:13.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:13.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:13 smithi120 ceph-mon[35182]: from='client.? 172.21.15.83:0/1957733901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9eb34260-3535-4e62-8aa5-276282a679cb"}]: dispatch 2024-09-11T23:47:13.873 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:13 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9eb34260-3535-4e62-8aa5-276282a679cb"}]: dispatch 2024-09-11T23:47:13.873 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:13 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9eb34260-3535-4e62-8aa5-276282a679cb"}]': finished 2024-09-11T23:47:13.873 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:13 smithi120 ceph-mon[35182]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T23:47:13.873 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:13.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:13 smithi062 ceph-mon[30958]: from='client.? 172.21.15.83:0/1957733901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9eb34260-3535-4e62-8aa5-276282a679cb"}]: dispatch 2024-09-11T23:47:13.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:13 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9eb34260-3535-4e62-8aa5-276282a679cb"}]: dispatch 2024-09-11T23:47:13.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:13 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9eb34260-3535-4e62-8aa5-276282a679cb"}]': finished 2024-09-11T23:47:13.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:13 smithi062 ceph-mon[30958]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T23:47:13.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:14.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:14 smithi120 ceph-mon[35182]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:14.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:14 smithi120 ceph-mon[35182]: from='client.? 172.21.15.83:0/2526573542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:47:14.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:14 smithi083 ceph-mon[34805]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:14.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:14 smithi083 ceph-mon[34805]: from='client.? 172.21.15.83:0/2526573542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:47:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:14 smithi062 ceph-mon[30958]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:14 smithi062 ceph-mon[30958]: from='client.? 172.21.15.83:0/2526573542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:47:16.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:16 smithi120 ceph-mon[35182]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:16 smithi083 ceph-mon[34805]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:16.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:16 smithi062 ceph-mon[30958]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:18 smithi083 ceph-mon[34805]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:18.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:18 smithi120 ceph-mon[35182]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:18.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:18 smithi062 ceph-mon[30958]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:20 smithi083 ceph-mon[34805]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:20.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:20 smithi120 ceph-mon[35182]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:20.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:20 smithi062 ceph-mon[30958]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:21.741 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:21 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:47:21.741 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:21 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:21.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:21 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:47:21.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:21 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:21.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:21 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:47:21.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:21 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:22.739 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:22 smithi083 ceph-mon[34805]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:22.739 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:22 smithi083 ceph-mon[34805]: Deploying daemon osd.7 on smithi083 2024-09-11T23:47:22.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:22 smithi120 ceph-mon[35182]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:22.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:22 smithi120 ceph-mon[35182]: Deploying daemon osd.7 on smithi083 2024-09-11T23:47:22.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:22 smithi062 ceph-mon[30958]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:22.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:22 smithi062 ceph-mon[30958]: Deploying daemon osd.7 on smithi083 2024-09-11T23:47:24.785 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:24 smithi083 ceph-mon[34805]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:24.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:24 smithi120 ceph-mon[35182]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:24.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:24 smithi062 ceph-mon[30958]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:26.614 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:26 smithi083 ceph-mon[34805]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:26.767 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:26 smithi062 ceph-mon[30958]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:26.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:26 smithi120 ceph-mon[35182]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:27.719 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:27.719 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:27.719 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:28.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:28 smithi083 ceph-mon[34805]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:28.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:28 smithi120 ceph-mon[35182]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:28.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:28 smithi062 ceph-mon[30958]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:30 smithi120 ceph-mon[35182]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:30 smithi083 ceph-mon[34805]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:30 smithi062 ceph-mon[30958]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:30.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:30.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:30.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:30.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:31.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.331+0000 7f4e867fc640 1 -- 172.21.15.83:0/4103211725 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4e88062630 con 0x7f4e700a2bb0 2024-09-11T23:47:31.331 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 7 on host 'smithi083' 2024-09-11T23:47:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e700a2bb0 msgr2=0x7f4e700a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e700a2bb0 0x7f4e700a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009710 tx=0x7f4e80006c90 comp rx=0 tx=0).stop 2024-09-11T23:47:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/4103211725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e88104f90 msgr2=0x7f4e88198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e88104f90 0x7f4e88198600 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c031610 tx=0x7f4e7c0043b0 comp rx=0 tx=0).stop 2024-09-11T23:47:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/4103211725 shutdown_connections 2024-09-11T23:47:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4e700a2bb0 0x7f4e700a5070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e881078c0 0x7f4e8819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e88104f90 0x7f4e88198600 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 --2- 172.21.15.83:0/4103211725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e88102660 0x7f4e881980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/4103211725 >> 172.21.15.83:0/4103211725 conn(0x7f4e880fc280 msgr2=0x7f4e88103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/4103211725 shutdown_connections 2024-09-11T23:47:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:47:31.333+0000 7f4e8f2c5640 1 -- 172.21.15.83:0/4103211725 wait complete. 2024-09-11T23:47:31.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:31 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:31.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:31 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:31.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:31 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:31.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:31 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:31.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:31 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:31.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:31 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:31.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:31 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:31.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:31 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:31.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:31 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:32.115 DEBUG:teuthology.orchestra.run.smithi083:osd.7> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T23:47:32.118 INFO:tasks.cephadm:Deploying osd.8 on smithi120 with /dev/vg_nvme/lv_2... 2024-09-11T23:47:32.118 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:47:32.371 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:47:32.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:32 smithi083 ceph-mon[34805]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:32.762 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:32 smithi120 ceph-mon[35182]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:32.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:32 smithi062 ceph-mon[30958]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:33.167 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 11 23:47:33 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-11T23:47:33.066+0000 7f421bec5640 -1 osd.7 0 log_to_monitors true 2024-09-11T23:47:33.782 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:33 smithi083 ceph-mon[34805]: from='osd.7 [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:47:33.783 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:33 smithi083 ceph-mon[34805]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:47:33.788 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:33 smithi120 ceph-mon[35182]: from='osd.7 [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:47:33.788 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:33 smithi120 ceph-mon[35182]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:47:33.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:33 smithi062 ceph-mon[30958]: from='osd.7 [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:47:33.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:33 smithi062 ceph-mon[30958]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:47:34.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:34 smithi120 ceph-mon[35182]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:34.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:34 smithi120 ceph-mon[35182]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:47:34.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:34 smithi120 ceph-mon[35182]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T23:47:34.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:34 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:34.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:34 smithi120 ceph-mon[35182]: from='osd.7 [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:47:34.800 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:34 smithi120 ceph-mon[35182]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:47:34.917 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 11 23:47:34 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-11T23:47:34.524+0000 7f4217e48640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T23:47:34.917 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 11 23:47:34 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-11T23:47:34.534+0000 7f4213c70640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:47:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:34 smithi083 ceph-mon[34805]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:34 smithi083 ceph-mon[34805]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:47:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:34 smithi083 ceph-mon[34805]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T23:47:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:34 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:34 smithi083 ceph-mon[34805]: from='osd.7 [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:47:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:34 smithi083 ceph-mon[34805]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:47:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:34 smithi062 ceph-mon[30958]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:34 smithi062 ceph-mon[30958]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:47:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:34 smithi062 ceph-mon[30958]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T23:47:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:34 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:34 smithi062 ceph-mon[30958]: from='osd.7 [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:47:34.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:34 smithi062 ceph-mon[30958]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:47:35.121 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:47:35.145 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_2 2024-09-11T23:47:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:47:35.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:47:35.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T23:47:35.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:35.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:35.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.805 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:35.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:35.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:35.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:35.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:35.806 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:35 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.830+0000 7fd930e4c640 1 -- 172.21.15.120:0/3548968578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd92c107940 msgr2=0x7fd92c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:35.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.830+0000 7fd930e4c640 1 --2- 172.21.15.120:0/3548968578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd92c107940 0x7fd92c109d30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd924009ca0 tx=0x7fd92402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:47:35.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.830+0000 7fd930e4c640 1 -- 172.21.15.120:0/3548968578 shutdown_connections 2024-09-11T23:47:35.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.830+0000 7fd930e4c640 1 --2- 172.21.15.120:0/3548968578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd92c107940 0x7fd92c109d30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:35.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.830+0000 7fd930e4c640 1 --2- 172.21.15.120:0/3548968578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd92c105010 0x7fd92c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:35.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.830+0000 7fd930e4c640 1 --2- 172.21.15.120:0/3548968578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd92c1026e0 0x7fd92c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:35.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.830+0000 7fd930e4c640 1 -- 172.21.15.120:0/3548968578 >> 172.21.15.120:0/3548968578 conn(0x7fd92c0fc280 msgr2=0x7fd92c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:35.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.830+0000 7fd930e4c640 1 -- 172.21.15.120:0/3548968578 shutdown_connections 2024-09-11T23:47:35.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.830+0000 7fd930e4c640 1 -- 172.21.15.120:0/3548968578 wait complete. 2024-09-11T23:47:35.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.830+0000 7fd930e4c640 1 Processor -- start 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd930e4c640 1 -- start start 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd930e4c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd92c1026e0 0x7fd92c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd930e4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd92c105010 0x7fd92c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd930e4c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd92c107940 0x7fd92c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd930e4c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c10d8e0 con 0x7fd92c107940 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd930e4c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c10d760 con 0x7fd92c105010 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd930e4c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c10da60 con 0x7fd92c1026e0 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd92a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd92c1026e0 0x7fd92c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd92a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd92c1026e0 0x7fd92c197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60714/0 (socket says 172.21.15.120:60714) 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd92a59c640 1 -- 172.21.15.120:0/3902659886 learned_addr learned my addr 172.21.15.120:0/3902659886 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd929d9b640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd92c105010 0x7fd92c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:35.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd92ad9d640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd92c107940 0x7fd92c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd92a59c640 1 -- 172.21.15.120:0/3902659886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd92c105010 msgr2=0x7fd92c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd92a59c640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd92c105010 0x7fd92c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd92a59c640 1 -- 172.21.15.120:0/3902659886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd92c107940 msgr2=0x7fd92c19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd92a59c640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd92c107940 0x7fd92c19f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd92a59c640 1 -- 172.21.15.120:0/3902659886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92c19fb60 con 0x7fd92c1026e0 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd92a59c640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd92c1026e0 0x7fd92c197f30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00b4a0 tx=0x7fd91c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd91b7fe640 1 -- 172.21.15.120:0/3902659886 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c014070 con 0x7fd92c1026e0 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd91b7fe640 1 -- 172.21.15.120:0/3902659886 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd91c010af0 con 0x7fd92c1026e0 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.831+0000 7fd930e4c640 1 -- 172.21.15.120:0/3902659886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92c19fe50 con 0x7fd92c1026e0 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.832+0000 7fd91b7fe640 1 -- 172.21.15.120:0/3902659886 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c017d10 con 0x7fd92c1026e0 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.832+0000 7fd930e4c640 1 -- 172.21.15.120:0/3902659886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92c1a0320 con 0x7fd92c1026e0 2024-09-11T23:47:35.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.833+0000 7fd930e4c640 1 -- 172.21.15.120:0/3902659886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd92c0fdd70 con 0x7fd92c1026e0 2024-09-11T23:47:35.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.834+0000 7fd91b7fe640 1 -- 172.21.15.120:0/3902659886 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd91c017070 con 0x7fd92c1026e0 2024-09-11T23:47:35.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.834+0000 7fd91b7fe640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9140a2bb0 0x7fd9140a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:35.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.835+0000 7fd91b7fe640 1 -- 172.21.15.120:0/3902659886 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5430+0+0 (secure 0 0 0) 0x7fd91c0cdcd0 con 0x7fd92c1026e0 2024-09-11T23:47:35.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.835+0000 7fd929d9b640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9140a2bb0 0x7fd9140a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:35.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.835+0000 7fd929d9b640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9140a2bb0 0x7fd9140a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd920005fd0 tx=0x7fd920005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:35.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:35.838+0000 7fd91b7fe640 1 -- 172.21.15.120:0/3902659886 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd91c09b770 con 0x7fd92c1026e0 2024-09-11T23:47:35.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:47:35.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T23:47:35.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:35.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:35.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:35.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:35.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:35.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:35.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:35.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:35 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:47:35.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:35.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:35 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:36.011+0000 7fd930e4c640 1 -- 172.21.15.120:0/3902659886 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd92c062630 con 0x7fd9140a2bb0 2024-09-11T23:47:36.735 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:47:36.735 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:47:36.735 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:36.735 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi083 2024-09-11T23:47:36.735 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi083 to 5696M 2024-09-11T23:47:36.735 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:36.735 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: osd.7 [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] boot 2024-09-11T23:47:36.735 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T23:47:36.736 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:36.736 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:47:36.736 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:47:36.736 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:36.736 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:36 smithi120 ceph-mon[35182]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T23:47:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:47:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:47:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi083 2024-09-11T23:47:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi083 to 5696M 2024-09-11T23:47:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: osd.7 [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] boot 2024-09-11T23:47:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T23:47:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:47:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:47:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:36 smithi083 ceph-mon[34805]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T23:47:36.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi083 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi083 to 5696M 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: osd.7 [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] boot 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:36.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:36 smithi062 ceph-mon[30958]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T23:47:37.708 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:37 smithi120 ceph-mon[35182]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:47:37.708 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:37 smithi120 ceph-mon[35182]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T23:47:37.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:37 smithi083 ceph-mon[34805]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:47:37.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:37 smithi083 ceph-mon[34805]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T23:47:37.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:37 smithi062 ceph-mon[30958]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:47:37.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:37 smithi062 ceph-mon[30958]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T23:47:38.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:38 smithi083 ceph-mon[34805]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:38.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:38 smithi083 ceph-mon[34805]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T23:47:38.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:38 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/1247357355' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2244a62-94a3-4ebf-9e65-aadfc025c3c7"}]: dispatch 2024-09-11T23:47:38.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:38 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2244a62-94a3-4ebf-9e65-aadfc025c3c7"}]: dispatch 2024-09-11T23:47:38.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:38 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2244a62-94a3-4ebf-9e65-aadfc025c3c7"}]': finished 2024-09-11T23:47:38.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:38 smithi083 ceph-mon[34805]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:47:38.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:38 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:47:38.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:38 smithi120 ceph-mon[35182]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:38.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:38 smithi120 ceph-mon[35182]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T23:47:38.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:38 smithi120 ceph-mon[35182]: from='client.? 172.21.15.120:0/1247357355' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2244a62-94a3-4ebf-9e65-aadfc025c3c7"}]: dispatch 2024-09-11T23:47:38.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:38 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2244a62-94a3-4ebf-9e65-aadfc025c3c7"}]: dispatch 2024-09-11T23:47:38.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:38 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2244a62-94a3-4ebf-9e65-aadfc025c3c7"}]': finished 2024-09-11T23:47:38.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:38 smithi120 ceph-mon[35182]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:47:38.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:38 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:47:38.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:38 smithi062 ceph-mon[30958]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:38.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:38 smithi062 ceph-mon[30958]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T23:47:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:38 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/1247357355' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2244a62-94a3-4ebf-9e65-aadfc025c3c7"}]: dispatch 2024-09-11T23:47:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:38 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2244a62-94a3-4ebf-9e65-aadfc025c3c7"}]: dispatch 2024-09-11T23:47:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:38 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2244a62-94a3-4ebf-9e65-aadfc025c3c7"}]': finished 2024-09-11T23:47:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:38 smithi062 ceph-mon[30958]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:47:38.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:38 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:47:39.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:39 smithi120 ceph-mon[35182]: from='client.? 172.21.15.120:0/1024698839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:47:39.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:39 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/1024698839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:47:39.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:39 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/1024698839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:47:40.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:40 smithi120 ceph-mon[35182]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:40.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:40 smithi083 ceph-mon[34805]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:40.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:40 smithi062 ceph-mon[30958]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:42.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:42 smithi120 ceph-mon[35182]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:42.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:42 smithi083 ceph-mon[34805]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:42.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:42 smithi062 ceph-mon[30958]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:44.818 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:44 smithi120 ceph-mon[35182]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:44.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:44 smithi083 ceph-mon[34805]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:44.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:44 smithi062 ceph-mon[30958]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:46.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:46 smithi120 ceph-mon[35182]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:46.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:46 smithi083 ceph-mon[34805]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:46.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:46 smithi062 ceph-mon[30958]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:47.764 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:47:47.764 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:47.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:47:47.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:47.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:47:47.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:48.694 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:48 smithi120 ceph-mon[35182]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:48.694 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:48 smithi120 ceph-mon[35182]: Deploying daemon osd.8 on smithi120 2024-09-11T23:47:48.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:48 smithi083 ceph-mon[34805]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:48.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:48 smithi083 ceph-mon[34805]: Deploying daemon osd.8 on smithi120 2024-09-11T23:47:48.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:48 smithi062 ceph-mon[30958]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:48.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:48 smithi062 ceph-mon[30958]: Deploying daemon osd.8 on smithi120 2024-09-11T23:47:50.741 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:50 smithi120 ceph-mon[35182]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:50.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:50 smithi083 ceph-mon[34805]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:50.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:50 smithi062 ceph-mon[30958]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:52.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:52 smithi120 ceph-mon[35182]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:52.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:52 smithi083 ceph-mon[34805]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:52.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:52 smithi062 ceph-mon[30958]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:53.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:53.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:53.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:53.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:53.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:53.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:53.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:53.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:53.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:54.683 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:54 smithi120 ceph-mon[35182]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:54.831 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:54 smithi083 ceph-mon[34805]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:54.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:54 smithi062 ceph-mon[30958]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:56.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:56 smithi120 ceph-mon[35182]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:56 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:56 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:56 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:56 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:56 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:56.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:56 smithi083 ceph-mon[34805]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:56.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:56 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:56.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:56 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:56.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:56 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:56.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:56 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:56.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:56 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:56 smithi062 ceph-mon[30958]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:56 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:56 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:56 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:56 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:56 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:57.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.103+0000 7fd91b7fe640 1 -- 172.21.15.120:0/3902659886 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd92c062630 con 0x7fd9140a2bb0 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 8 on host 'smithi120' 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 -- 172.21.15.120:0/3902659886 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9140a2bb0 msgr2=0x7fd9140a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9140a2bb0 0x7fd9140a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd920005fd0 tx=0x7fd920005850 comp rx=0 tx=0).stop 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 -- 172.21.15.120:0/3902659886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd92c1026e0 msgr2=0x7fd92c197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd92c1026e0 0x7fd92c197f30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00b4a0 tx=0x7fd91c00b960 comp rx=0 tx=0).stop 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 -- 172.21.15.120:0/3902659886 shutdown_connections 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9140a2bb0 0x7fd9140a5070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd92c107940 0x7fd92c19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd92c105010 0x7fd92c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 --2- 172.21.15.120:0/3902659886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd92c1026e0 0x7fd92c197f30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 -- 172.21.15.120:0/3902659886 >> 172.21.15.120:0/3902659886 conn(0x7fd92c0fc280 msgr2=0x7fd92c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:57.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 -- 172.21.15.120:0/3902659886 shutdown_connections 2024-09-11T23:47:57.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:47:57.106+0000 7fd930e4c640 1 -- 172.21.15.120:0/3902659886 wait complete. 2024-09-11T23:47:57.776 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:57 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:57.776 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:57 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:57.776 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:57 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:57.803 DEBUG:teuthology.orchestra.run.smithi120:osd.8> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T23:47:57.806 INFO:tasks.cephadm:Deploying osd.9 on smithi062 with /dev/vg_nvme/lv_1... 2024-09-11T23:47:57.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:47:57.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:57 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:57.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:57 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:57.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:57 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:57.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:57 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:57.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:57 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:57.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:57 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:47:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:58.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:58 smithi120 ceph-mon[35182]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:58.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:58 smithi083 ceph-mon[34805]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:58 smithi062 ceph-mon[30958]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:47:59.431 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 11 23:47:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-11T23:47:59.275+0000 7fd28e984640 -1 osd.8 0 log_to_monitors true 2024-09-11T23:47:59.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:59 smithi120 ceph-mon[35182]: from='osd.8 [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:47:59.873 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:47:59 smithi120 ceph-mon[35182]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:47:59.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:59 smithi083 ceph-mon[34805]: from='osd.8 [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:47:59.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:47:59 smithi083 ceph-mon[34805]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:47:59.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:59 smithi062 ceph-mon[30958]: from='osd.8 [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:47:59.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:47:59 smithi062 ceph-mon[30958]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:48:00.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 11 23:48:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-11T23:48:00.611+0000 7fd28a907640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T23:48:00.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 11 23:48:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-11T23:48:00.619+0000 7fd28672f640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:48:00.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:00 smithi120 ceph-mon[35182]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:48:00.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:00 smithi120 ceph-mon[35182]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:48:00.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:00 smithi120 ceph-mon[35182]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T23:48:00.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:00 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:00.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:00 smithi120 ceph-mon[35182]: from='osd.8 [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:48:00.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:00 smithi120 ceph-mon[35182]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:48:00.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:00 smithi083 ceph-mon[34805]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:48:00.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:00 smithi083 ceph-mon[34805]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:48:00.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:00 smithi083 ceph-mon[34805]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T23:48:00.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:00 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:00.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:00 smithi083 ceph-mon[34805]: from='osd.8 [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:48:00.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:00 smithi083 ceph-mon[34805]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:48:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:00 smithi062 ceph-mon[30958]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:48:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:00 smithi062 ceph-mon[30958]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:48:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:00 smithi062 ceph-mon[30958]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T23:48:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:00 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:00 smithi062 ceph-mon[30958]: from='osd.8 [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:48:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:00 smithi062 ceph-mon[30958]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:48:01.075 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:48:01.100 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2024-09-11T23:48:01.354 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:01.760 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:48:01.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T23:48:01.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:01.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:01.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:01.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:01.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:01.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:01.762 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:01.762 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.762 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: osd.8 [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] boot 2024-09-11T23:48:01.762 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T23:48:01.762 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:01.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.846+0000 7feef48cb640 1 -- 172.21.15.62:0/3322098194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef0101d10 msgr2=0x7feef0102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:01.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.846+0000 7feef48cb640 1 --2- 172.21.15.62:0/3322098194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef0101d10 0x7feef0102190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feed8009900 tx=0x7feed802a410 comp rx=0 tx=0).stop 2024-09-11T23:48:01.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.846+0000 7feef48cb640 1 -- 172.21.15.62:0/3322098194 shutdown_connections 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.846+0000 7feef48cb640 1 --2- 172.21.15.62:0/3322098194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef01026d0 0x7feef0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.846+0000 7feef48cb640 1 --2- 172.21.15.62:0/3322098194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef0101d10 0x7feef0102190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.846+0000 7feef48cb640 1 --2- 172.21.15.62:0/3322098194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feef0100b10 0x7feef0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.846+0000 7feef48cb640 1 -- 172.21.15.62:0/3322098194 >> 172.21.15.62:0/3322098194 conn(0x7feef00fc2a0 msgr2=0x7feef00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.846+0000 7feef48cb640 1 -- 172.21.15.62:0/3322098194 shutdown_connections 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.846+0000 7feef48cb640 1 -- 172.21.15.62:0/3322098194 wait complete. 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feef48cb640 1 Processor -- start 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feef48cb640 1 -- start start 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feef48cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feef0100b10 0x7feef0197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feef48cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0101d10 0x7feef0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:01.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feef48cb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef01026d0 0x7feef019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feef48cb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feef010d8c0 con 0x7feef01026d0 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feef48cb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feef010d740 con 0x7feef0101d10 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feef48cb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feef010da40 con 0x7feef0100b10 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feeeed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef01026d0 0x7feef019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feeeed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef01026d0 0x7feef019f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47024/0 (socket says 172.21.15.62:47024) 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.847+0000 7feeeed9d640 1 -- 172.21.15.62:0/1008196864 learned_addr learned my addr 172.21.15.62:0/1008196864 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feeee59c640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feef0100b10 0x7feef0197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feee7fff640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0101d10 0x7feef0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feeeed9d640 1 -- 172.21.15.62:0/1008196864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feef0100b10 msgr2=0x7feef0197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feeeed9d640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feef0100b10 0x7feef0197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feeeed9d640 1 -- 172.21.15.62:0/1008196864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0101d10 msgr2=0x7feef0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:01.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feeeed9d640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0101d10 0x7feef0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feeeed9d640 1 -- 172.21.15.62:0/1008196864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef019fa70 con 0x7feef01026d0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feeeed9d640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef01026d0 0x7feef019f400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feee800eb10 tx=0x7feee800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feee77fe640 1 -- 172.21.15.62:0/1008196864 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8004b20 con 0x7feef01026d0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feee77fe640 1 -- 172.21.15.62:0/1008196864 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee8004cc0 con 0x7feef01026d0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feef48cb640 1 -- 172.21.15.62:0/1008196864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef019fd60 con 0x7feef01026d0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feee77fe640 1 -- 172.21.15.62:0/1008196864 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8010760 con 0x7feef01026d0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.848+0000 7feef48cb640 1 -- 172.21.15.62:0/1008196864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01a0210 con 0x7feef01026d0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.849+0000 7feef48cb640 1 -- 172.21.15.62:0/1008196864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef0107e90 con 0x7feef01026d0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.850+0000 7feee77fe640 1 -- 172.21.15.62:0/1008196864 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feee8005020 con 0x7feef01026d0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.851+0000 7feee77fe640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feedc0a2c30 0x7feedc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.851+0000 7feeee59c640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feedc0a2c30 0x7feedc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.851+0000 7feee77fe640 1 -- 172.21.15.62:0/1008196864 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5862+0+0 (secure 0 0 0) 0x7feee80cebb0 con 0x7feef01026d0 2024-09-11T23:48:01.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.851+0000 7feeee59c640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feedc0a2c30 0x7feedc0a50f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feef0101b70 tx=0x7feee0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:01.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:01.855+0000 7feee77fe640 1 -- 172.21.15.62:0/1008196864 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feee809c480 con 0x7feef01026d0 2024-09-11T23:48:01.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: osd.8 [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] boot 2024-09-11T23:48:01.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T23:48:01.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:01.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:48:01.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T23:48:01.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:01.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:01.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:01.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:01.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:01.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:01.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:01.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:01.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: osd.8 [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] boot 2024-09-11T23:48:01.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T23:48:01.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:48:02.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:02.040+0000 7feef48cb640 1 -- 172.21.15.62:0/1008196864 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7feef0062630 con 0x7feedc0a2c30 2024-09-11T23:48:02.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:02 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:48:02.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:02 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:48:02.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:02 smithi062 ceph-mon[30958]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:48:02.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:02 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi120 2024-09-11T23:48:02.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:02 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-11T23:48:02.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:48:02.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:02 smithi062 ceph-mon[30958]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T23:48:02.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:48:02.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:02.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:02 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:48:02.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:02 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:48:02.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:02 smithi120 ceph-mon[35182]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:48:02.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:02 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi120 2024-09-11T23:48:02.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:02 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-11T23:48:02.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:48:02.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:02 smithi120 ceph-mon[35182]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T23:48:02.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:48:02.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:02 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:48:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:02 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:48:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:02 smithi083 ceph-mon[34805]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:48:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:02 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi120 2024-09-11T23:48:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:02 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-11T23:48:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:48:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:02 smithi083 ceph-mon[34805]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T23:48:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:48:02.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:03.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:03 smithi120 ceph-mon[35182]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:03.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:03 smithi120 ceph-mon[35182]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T23:48:03.892 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:03 smithi062 ceph-mon[30958]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:03.892 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:03 smithi062 ceph-mon[30958]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T23:48:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:03 smithi083 ceph-mon[34805]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:03 smithi083 ceph-mon[34805]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T23:48:04.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:04 smithi120 ceph-mon[35182]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:04.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:04 smithi120 ceph-mon[35182]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T23:48:04.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:04 smithi062 ceph-mon[30958]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:04.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:04 smithi062 ceph-mon[30958]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T23:48:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:04 smithi083 ceph-mon[34805]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:04 smithi083 ceph-mon[34805]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T23:48:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:05 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3454339005' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0"}]: dispatch 2024-09-11T23:48:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:05 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3454339005' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0"}]': finished 2024-09-11T23:48:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:05 smithi083 ceph-mon[34805]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T23:48:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:05 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:05 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3318574528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:48:05.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:05 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3454339005' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0"}]: dispatch 2024-09-11T23:48:05.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:05 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3454339005' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0"}]': finished 2024-09-11T23:48:05.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:05 smithi062 ceph-mon[30958]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T23:48:05.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:05 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:05.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:05 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3318574528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:48:06.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:05 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3454339005' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0"}]: dispatch 2024-09-11T23:48:06.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:05 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3454339005' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0"}]': finished 2024-09-11T23:48:06.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:05 smithi120 ceph-mon[35182]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T23:48:06.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:05 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:06.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:05 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3318574528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:48:06.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:06 smithi083 ceph-mon[34805]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:06.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:06 smithi062 ceph-mon[30958]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:07.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:06 smithi120 ceph-mon[35182]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:08.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:08 smithi083 ceph-mon[34805]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-11T23:48:08.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:08 smithi062 ceph-mon[30958]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-11T23:48:09.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:08 smithi120 ceph-mon[35182]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-11T23:48:10.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:10 smithi083 ceph-mon[34805]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:48:10.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:10 smithi062 ceph-mon[30958]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:48:11.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:10 smithi120 ceph-mon[35182]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:48:12.697 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:12 smithi062 ceph-mon[30958]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T23:48:12.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:12 smithi083 ceph-mon[34805]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T23:48:13.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:12 smithi120 ceph-mon[35182]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T23:48:13.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:48:13.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:13.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:13 smithi062 ceph-mon[30958]: Deploying daemon osd.9 on smithi062 2024-09-11T23:48:13.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:48:13.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:13.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:13 smithi083 ceph-mon[34805]: Deploying daemon osd.9 on smithi062 2024-09-11T23:48:14.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:48:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:13 smithi120 ceph-mon[35182]: Deploying daemon osd.9 on smithi062 2024-09-11T23:48:14.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:14 smithi062 ceph-mon[30958]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-11T23:48:14.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:14 smithi083 ceph-mon[34805]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-11T23:48:15.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:14 smithi120 ceph-mon[35182]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-11T23:48:16.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:16 smithi062 ceph-mon[30958]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-11T23:48:16.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:16 smithi083 ceph-mon[34805]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-11T23:48:17.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:16 smithi120 ceph-mon[35182]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-11T23:48:18.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:18 smithi083 ceph-mon[34805]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:48:18.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:18 smithi062 ceph-mon[30958]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:48:19.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:18 smithi120 ceph-mon[35182]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:48:19.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:19 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:19.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:19 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:19.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:19 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:19 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:19 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:19 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:20.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:19 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:19 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:19 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:20 smithi083 ceph-mon[34805]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:20.941 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:20 smithi062 ceph-mon[30958]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:21.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:20 smithi120 ceph-mon[35182]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:22.822 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 9 on host 'smithi062' 2024-09-11T23:48:22.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.821+0000 7feee77fe640 1 -- 172.21.15.62:0/1008196864 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7feef0062630 con 0x7feedc0a2c30 2024-09-11T23:48:22.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 -- 172.21.15.62:0/1008196864 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feedc0a2c30 msgr2=0x7feedc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feedc0a2c30 0x7feedc0a50f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feef0101b70 tx=0x7feee0005f90 comp rx=0 tx=0).stop 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 -- 172.21.15.62:0/1008196864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef01026d0 msgr2=0x7feef019f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef01026d0 0x7feef019f400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feee800eb10 tx=0x7feee800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 -- 172.21.15.62:0/1008196864 shutdown_connections 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feedc0a2c30 0x7feedc0a50f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef01026d0 0x7feef019f400 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feef0101d10 0x7feef0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 --2- 172.21.15.62:0/1008196864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feef0100b10 0x7feef0197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 -- 172.21.15.62:0/1008196864 >> 172.21.15.62:0/1008196864 conn(0x7feef00fc2a0 msgr2=0x7feef0105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 -- 172.21.15.62:0/1008196864 shutdown_connections 2024-09-11T23:48:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:48:22.824+0000 7feef48cb640 1 -- 172.21.15.62:0/1008196864 wait complete. 2024-09-11T23:48:22.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:22 smithi083 ceph-mon[34805]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:22.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:22 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:22.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:22 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:22.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:22 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:22.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:22 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:22.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:22 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:22.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:22 smithi062 ceph-mon[30958]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:22.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:22 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:22.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:22 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:22.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:22 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:22.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:22 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:22.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:22 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:23.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:22 smithi120 ceph-mon[35182]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:22 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:22 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:22 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:22 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:22 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:23.561 DEBUG:teuthology.orchestra.run.smithi062:osd.9> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T23:48:23.564 INFO:tasks.cephadm:Deploying osd.10 on smithi083 with /dev/vg_nvme/lv_1... 2024-09-11T23:48:23.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:48:23.784 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:23.784 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:23.784 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:23 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:23.824 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:48:23.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:23.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:23.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:23 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:24.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:24.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:24.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:23 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:24.883 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:24 smithi062 ceph-mon[30958]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:24.981 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:24 smithi083 ceph-mon[34805]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:24 smithi120 ceph-mon[35182]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:25.197 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 11 23:48:25 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-11T23:48:25.026+0000 7ff8db10c640 -1 osd.9 0 log_to_monitors true 2024-09-11T23:48:25.852 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:25 smithi083 ceph-mon[34805]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:48:25.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:25 smithi062 ceph-mon[30958]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:48:26.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:25 smithi120 ceph-mon[35182]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:48:26.516 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-11T23:48:26.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-09-11T23:48:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:48:26.833 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:26 smithi083 ceph-mon[34805]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:26.833 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:26 smithi083 ceph-mon[34805]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:48:26.833 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:26 smithi083 ceph-mon[34805]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T23:48:26.833 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:26 smithi083 ceph-mon[34805]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:48:26.833 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:26 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:26.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:26 smithi062 ceph-mon[30958]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:26.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:26 smithi062 ceph-mon[30958]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:48:26.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:26 smithi062 ceph-mon[30958]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T23:48:26.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:26 smithi062 ceph-mon[30958]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:48:26.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:26 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:26.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 11 23:48:26 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-11T23:48:26.688+0000 7ff8d708f640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T23:48:26.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 11 23:48:26 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-11T23:48:26.700+0000 7ff8d2eb7640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:48:27.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:26 smithi120 ceph-mon[35182]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:26 smithi120 ceph-mon[35182]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:48:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:26 smithi120 ceph-mon[35182]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T23:48:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:26 smithi120 ceph-mon[35182]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-09-11T23:48:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:26 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 -- 172.21.15.83:0/3473160588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f3c102660 msgr2=0x7f5f3c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 --2- 172.21.15.83:0/3473160588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f3c102660 0x7f5f3c104a50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009900 tx=0x7f5f2c02a410 comp rx=0 tx=0).stop 2024-09-11T23:48:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 -- 172.21.15.83:0/3473160588 shutdown_connections 2024-09-11T23:48:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 --2- 172.21.15.83:0/3473160588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3c1078c0 0x7f5f3c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 --2- 172.21.15.83:0/3473160588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c104f90 0x7f5f3c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 --2- 172.21.15.83:0/3473160588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f3c102660 0x7f5f3c104a50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 -- 172.21.15.83:0/3473160588 >> 172.21.15.83:0/3473160588 conn(0x7f5f3c0fc280 msgr2=0x7f5f3c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 -- 172.21.15.83:0/3473160588 shutdown_connections 2024-09-11T23:48:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 -- 172.21.15.83:0/3473160588 wait complete. 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 Processor -- start 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.293+0000 7f5f41f18640 1 -- start start 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f41f18640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c102660 0x7f5f3c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f41f18640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3c104f90 0x7f5f3c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f41f18640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f3c1078c0 0x7f5f3c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f41f18640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c10d870 con 0x7f5f3c104f90 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f41f18640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c10d6f0 con 0x7f5f3c1078c0 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f41f18640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c10d9f0 con 0x7f5f3c102660 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f3c1078c0 0x7f5f3c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f3c1078c0 0x7f5f3c19f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39418/0 (socket says 172.21.15.83:39418) 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3bfff640 1 -- 172.21.15.83:0/212265722 learned_addr learned my addr 172.21.15.83:0/212265722 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-11T23:48:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3affd640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3c104f90 0x7f5f3c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3b7fe640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c102660 0x7f5f3c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3bfff640 1 -- 172.21.15.83:0/212265722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c102660 msgr2=0x7f5f3c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3bfff640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c102660 0x7f5f3c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3bfff640 1 -- 172.21.15.83:0/212265722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3c104f90 msgr2=0x7f5f3c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3bfff640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3c104f90 0x7f5f3c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3bfff640 1 -- 172.21.15.83:0/212265722 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3c19fb50 con 0x7f5f3c1078c0 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f3bfff640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f3c1078c0 0x7f5f3c19f4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400eb10 tx=0x7f5f3400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f38ff9640 1 -- 172.21.15.83:0/212265722 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34004b20 con 0x7f5f3c1078c0 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f38ff9640 1 -- 172.21.15.83:0/212265722 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f34004cc0 con 0x7f5f3c1078c0 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f41f18640 1 -- 172.21.15.83:0/212265722 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3c19fe40 con 0x7f5f3c1078c0 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f38ff9640 1 -- 172.21.15.83:0/212265722 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34010760 con 0x7f5f3c1078c0 2024-09-11T23:48:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.294+0000 7f5f41f18640 1 -- 172.21.15.83:0/212265722 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3c1a02a0 con 0x7f5f3c1078c0 2024-09-11T23:48:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.295+0000 7f5f41f18640 1 -- 172.21.15.83:0/212265722 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3c0fdd70 con 0x7f5f3c1078c0 2024-09-11T23:48:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.297+0000 7f5f38ff9640 1 -- 172.21.15.83:0/212265722 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5f34010900 con 0x7f5f3c1078c0 2024-09-11T23:48:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.298+0000 7f5f38ff9640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5f240a2bb0 0x7f5f240a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.298+0000 7f5f38ff9640 1 -- 172.21.15.83:0/212265722 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7f5f340cef20 con 0x7f5f3c1078c0 2024-09-11T23:48:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.298+0000 7f5f3b7fe640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5f240a2bb0 0x7f5f240a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.298+0000 7f5f3b7fe640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5f240a2bb0 0x7f5f240a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c000c00 tx=0x7f5f2c0315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.301+0000 7f5f38ff9640 1 -- 172.21.15.83:0/212265722 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f340541b0 con 0x7f5f3c1078c0 2024-09-11T23:48:27.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:27.479+0000 7f5f41f18640 1 -- 172.21.15.83:0/212265722 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5f3c062630 con 0x7f5f240a2bb0 2024-09-11T23:48:27.821 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi062 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi062 to 4272M 2024-09-11T23:48:27.822 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:27.823 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:27.823 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:27.823 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:27.823 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:48:27.823 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:48:27.823 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:48:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi062 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi062 to 4272M 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:27.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:27.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:48:27.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:48:27.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:28.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-09-11T23:48:28.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi062 2024-09-11T23:48:28.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T23:48:28.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:28.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:28.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:28.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:28.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:28.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:28.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:28.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:28.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi062 to 4272M 2024-09-11T23:48:28.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:28.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:28.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:28.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:28.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:48:28.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:48:28.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:28.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:28 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:48:28.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:28 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:48:28.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:28 smithi083 ceph-mon[34805]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:28.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:28 smithi083 ceph-mon[34805]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:28.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:28 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:28.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:28 smithi083 ceph-mon[34805]: osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] boot 2024-09-11T23:48:28.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:28 smithi083 ceph-mon[34805]: osdmap e63: 10 total, 10 up, 10 in 2024-09-11T23:48:28.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:28 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:28.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:28 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:48:28.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:28 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:48:28.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:28 smithi062 ceph-mon[30958]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:28.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:28 smithi062 ceph-mon[30958]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:28.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:28 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:28.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:28 smithi062 ceph-mon[30958]: osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] boot 2024-09-11T23:48:28.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:28 smithi062 ceph-mon[30958]: osdmap e63: 10 total, 10 up, 10 in 2024-09-11T23:48:28.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:28 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:29.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:28 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:48:29.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:28 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:48:29.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:28 smithi120 ceph-mon[35182]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:48:29.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:28 smithi120 ceph-mon[35182]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:29.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:28 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:29.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:28 smithi120 ceph-mon[35182]: osd.9 [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] boot 2024-09-11T23:48:29.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:28 smithi120 ceph-mon[35182]: osdmap e63: 10 total, 10 up, 10 in 2024-09-11T23:48:29.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:28 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:48:29.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:29 smithi083 ceph-mon[34805]: osdmap e64: 10 total, 10 up, 10 in 2024-09-11T23:48:29.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:29 smithi062 ceph-mon[30958]: osdmap e64: 10 total, 10 up, 10 in 2024-09-11T23:48:30.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:29 smithi120 ceph-mon[35182]: osdmap e64: 10 total, 10 up, 10 in 2024-09-11T23:48:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:30 smithi083 ceph-mon[34805]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:30 smithi083 ceph-mon[34805]: from='client.? 172.21.15.83:0/3579676000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7"}]: dispatch 2024-09-11T23:48:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:30 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7"}]: dispatch 2024-09-11T23:48:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:30 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7"}]': finished 2024-09-11T23:48:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:30 smithi083 ceph-mon[34805]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T23:48:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:30 smithi083 ceph-mon[34805]: from='client.? 172.21.15.83:0/2051537444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:48:31.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:30 smithi120 ceph-mon[35182]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:31.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:30 smithi120 ceph-mon[35182]: from='client.? 172.21.15.83:0/3579676000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7"}]: dispatch 2024-09-11T23:48:31.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:30 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7"}]: dispatch 2024-09-11T23:48:31.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:30 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7"}]': finished 2024-09-11T23:48:31.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:30 smithi120 ceph-mon[35182]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T23:48:31.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:31.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:30 smithi120 ceph-mon[35182]: from='client.? 172.21.15.83:0/2051537444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:48:31.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:30 smithi062 ceph-mon[30958]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:31.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:30 smithi062 ceph-mon[30958]: from='client.? 172.21.15.83:0/3579676000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7"}]: dispatch 2024-09-11T23:48:31.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:30 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7"}]: dispatch 2024-09-11T23:48:31.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:30 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7"}]': finished 2024-09-11T23:48:31.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:30 smithi062 ceph-mon[30958]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T23:48:31.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:31.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:30 smithi062 ceph-mon[30958]: from='client.? 172.21.15.83:0/2051537444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:48:33.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:32 smithi120 ceph-mon[35182]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:33.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:32 smithi083 ceph-mon[34805]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:33.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:32 smithi062 ceph-mon[30958]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:35.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:34 smithi120 ceph-mon[35182]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:35.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:34 smithi083 ceph-mon[34805]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:35.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:34 smithi062 ceph-mon[30958]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:36 smithi120 ceph-mon[35182]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:37.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:36 smithi083 ceph-mon[34805]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:37.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:36 smithi062 ceph-mon[30958]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:38.794 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:38 smithi083 ceph-mon[34805]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:38.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:38 smithi062 ceph-mon[30958]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:39.129 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:38 smithi120 ceph-mon[35182]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:39.893 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:39 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:48:39.893 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:39 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:40.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:39 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:48:40.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:39 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:40.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:39 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:48:40.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:39 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:40.781 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:40 smithi083 ceph-mon[34805]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:40.782 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:40 smithi083 ceph-mon[34805]: Deploying daemon osd.10 on smithi083 2024-09-11T23:48:41.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:40 smithi120 ceph-mon[35182]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:41.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:40 smithi120 ceph-mon[35182]: Deploying daemon osd.10 on smithi083 2024-09-11T23:48:41.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:40 smithi062 ceph-mon[30958]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:41.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:40 smithi062 ceph-mon[30958]: Deploying daemon osd.10 on smithi083 2024-09-11T23:48:42.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:42 smithi083 ceph-mon[34805]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:43.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:42 smithi120 ceph-mon[35182]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:43.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:42 smithi062 ceph-mon[30958]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:45.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:44 smithi120 ceph-mon[35182]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:45.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:45.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:44 smithi083 ceph-mon[34805]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:45.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:45.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:44 smithi062 ceph-mon[30958]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:45.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:46.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:45 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:46.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:45 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:46.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:45 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:46.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:45 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:46.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:45 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:46.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:45 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:46.907 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:46 smithi083 ceph-mon[34805]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:47.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:46 smithi120 ceph-mon[35182]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:47.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:46 smithi062 ceph-mon[30958]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:48.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.292+0000 7f5f38ff9640 1 -- 172.21.15.83:0/212265722 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5f3c062630 con 0x7f5f240a2bb0 2024-09-11T23:48:48.292 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 10 on host 'smithi083' 2024-09-11T23:48:48.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 -- 172.21.15.83:0/212265722 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5f240a2bb0 msgr2=0x7f5f240a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:48.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5f240a2bb0 0x7f5f240a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c000c00 tx=0x7f5f2c0315d0 comp rx=0 tx=0).stop 2024-09-11T23:48:48.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 -- 172.21.15.83:0/212265722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f3c1078c0 msgr2=0x7f5f3c19f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:48.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f3c1078c0 0x7f5f3c19f4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400eb10 tx=0x7f5f3400c2d0 comp rx=0 tx=0).stop 2024-09-11T23:48:48.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 -- 172.21.15.83:0/212265722 shutdown_connections 2024-09-11T23:48:48.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5f240a2bb0 0x7f5f240a5070 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:48.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f3c1078c0 0x7f5f3c19f4e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:48.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f3c104f90 0x7f5f3c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:48.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 --2- 172.21.15.83:0/212265722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3c102660 0x7f5f3c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:48.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 -- 172.21.15.83:0/212265722 >> 172.21.15.83:0/212265722 conn(0x7f5f3c0fc280 msgr2=0x7f5f3c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:48.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 -- 172.21.15.83:0/212265722 shutdown_connections 2024-09-11T23:48:48.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-11T23:48:48.295+0000 7f5f41f18640 1 -- 172.21.15.83:0/212265722 wait complete. 2024-09-11T23:48:48.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:48 smithi083 ceph-mon[34805]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:48.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:48 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:48.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:48 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:48.980 DEBUG:teuthology.orchestra.run.smithi083:osd.10> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T23:48:48.982 INFO:tasks.cephadm:Deploying osd.11 on smithi120 with /dev/vg_nvme/lv_1... 2024-09-11T23:48:48.982 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:48:49.013 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:48 smithi120 ceph-mon[35182]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:49.013 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:48 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:49.013 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:48 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:48 smithi062 ceph-mon[30958]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:48 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:48 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:49.240 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:48:50.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:50 smithi062 ceph-mon[30958]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:50.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:50 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:50.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:50 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:50.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:50 smithi083 ceph-mon[34805]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:50.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:50 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:50.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:50 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:50.610 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:50 smithi120 ceph-mon[35182]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:50.610 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:50 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:50.610 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:50 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:50.822 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 11 23:48:50 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-11T23:48:50.547+0000 7f761de7e640 -1 osd.10 0 log_to_monitors true 2024-09-11T23:48:51.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:51 smithi083 ceph-mon[34805]: from='osd.10 [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:48:51.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:51 smithi083 ceph-mon[34805]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:48:51.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:51 smithi062 ceph-mon[30958]: from='osd.10 [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:48:51.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:51 smithi062 ceph-mon[30958]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:48:51.476 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:51 smithi120 ceph-mon[35182]: from='osd.10 [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:48:51.476 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:51 smithi120 ceph-mon[35182]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:48:52.114 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-11T23:48:52.141 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_1 2024-09-11T23:48:52.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:52 smithi120 ceph-mon[35182]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:52.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:52 smithi120 ceph-mon[35182]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:48:52.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:52 smithi120 ceph-mon[35182]: from='osd.10 [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:48:52.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:52 smithi120 ceph-mon[35182]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T23:48:52.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:52.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:52 smithi120 ceph-mon[35182]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:48:52.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 11 23:48:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-11T23:48:52.223+0000 7f7619e01640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T23:48:52.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 11 23:48:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-11T23:48:52.233+0000 7f7615c29640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:48:52.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:52 smithi083 ceph-mon[34805]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:52.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:52 smithi083 ceph-mon[34805]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:48:52.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:52 smithi083 ceph-mon[34805]: from='osd.10 [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:48:52.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:52 smithi083 ceph-mon[34805]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T23:48:52.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:52.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:52 smithi083 ceph-mon[34805]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:48:52.434 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:48:52.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:52 smithi062 ceph-mon[30958]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:52.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:52 smithi062 ceph-mon[30958]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:48:52.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:52 smithi062 ceph-mon[30958]: from='osd.10 [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:48:52.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:52 smithi062 ceph-mon[30958]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T23:48:52.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:52 smithi062 ceph-mon[30958]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-09-11T23:48:52.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.948+0000 7f71d5fa1640 1 -- 172.21.15.120:0/2634537256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71d0101d10 msgr2=0x7f71d0102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:52.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.948+0000 7f71d5fa1640 1 --2- 172.21.15.120:0/2634537256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71d0101d10 0x7f71d0102190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009900 tx=0x7f71c402a410 comp rx=0 tx=0).stop 2024-09-11T23:48:52.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.949+0000 7f71d5fa1640 1 -- 172.21.15.120:0/2634537256 shutdown_connections 2024-09-11T23:48:52.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.949+0000 7f71d5fa1640 1 --2- 172.21.15.120:0/2634537256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d01026d0 0x7f71d0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:52.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.949+0000 7f71d5fa1640 1 --2- 172.21.15.120:0/2634537256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71d0101d10 0x7f71d0102190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:52.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.949+0000 7f71d5fa1640 1 --2- 172.21.15.120:0/2634537256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d0100b10 0x7f71d0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:52.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.949+0000 7f71d5fa1640 1 -- 172.21.15.120:0/2634537256 >> 172.21.15.120:0/2634537256 conn(0x7f71d00fc2a0 msgr2=0x7f71d00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:52.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.949+0000 7f71d5fa1640 1 -- 172.21.15.120:0/2634537256 shutdown_connections 2024-09-11T23:48:52.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.949+0000 7f71d5fa1640 1 -- 172.21.15.120:0/2634537256 wait complete. 2024-09-11T23:48:52.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.949+0000 7f71d5fa1640 1 Processor -- start 2024-09-11T23:48:52.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71d5fa1640 1 -- start start 2024-09-11T23:48:52.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71d5fa1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d0100b10 0x7f71d0196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71d5fa1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71d0101d10 0x7f71d0197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71d5fa1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d01026d0 0x7f71d019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71d5fa1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f71d010c770 con 0x7f71d01026d0 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71d5fa1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71d010c5f0 con 0x7f71d0100b10 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71d5fa1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71d010c8f0 con 0x7f71d0101d10 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71ceffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71d0101d10 0x7f71d0197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71cf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d0100b10 0x7f71d0196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71cf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d0100b10 0x7f71d0196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.120:50394/0 (socket says 172.21.15.120:50394) 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71cf7fe640 1 -- 172.21.15.120:0/3909749328 learned_addr learned my addr 172.21.15.120:0/3909749328 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.950+0000 7f71cffff640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d01026d0 0x7f71d019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:52.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71ceffd640 1 -- 172.21.15.120:0/3909749328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d0100b10 msgr2=0x7f71d0196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:52.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71ceffd640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d0100b10 0x7f71d0196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:52.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71ceffd640 1 -- 172.21.15.120:0/3909749328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d01026d0 msgr2=0x7f71d019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:52.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71ceffd640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d01026d0 0x7f71d019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:52.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71ceffd640 1 -- 172.21.15.120:0/3909749328 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d019ea20 con 0x7f71d0101d10 2024-09-11T23:48:52.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71ceffd640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71d0101d10 0x7f71d0197330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71c4031440 tx=0x7f71c4031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:52.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71ccff9640 1 -- 172.21.15.120:0/3909749328 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4035050 con 0x7f71d0101d10 2024-09-11T23:48:52.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71d5fa1640 1 -- 172.21.15.120:0/3909749328 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d006a790 con 0x7f71d0101d10 2024-09-11T23:48:52.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71ccff9640 1 -- 172.21.15.120:0/3909749328 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c4003ad0 con 0x7f71d0101d10 2024-09-11T23:48:52.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71ccff9640 1 -- 172.21.15.120:0/3909749328 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4038b80 con 0x7f71d0101d10 2024-09-11T23:48:52.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.951+0000 7f71d5fa1640 1 -- 172.21.15.120:0/3909749328 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d006acc0 con 0x7f71d0101d10 2024-09-11T23:48:52.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.952+0000 7f71d5fa1640 1 -- 172.21.15.120:0/3909749328 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d0100f10 con 0x7f71d0101d10 2024-09-11T23:48:52.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.953+0000 7f71ccff9640 1 -- 172.21.15.120:0/3909749328 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f71c40035f0 con 0x7f71d0101d10 2024-09-11T23:48:52.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.954+0000 7f71ccff9640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f71b80a2bb0 0x7f71b80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:52.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.954+0000 7f71ccff9640 1 -- 172.21.15.120:0/3909749328 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6414+0+0 (secure 0 0 0) 0x7f71c40f09d0 con 0x7f71d0101d10 2024-09-11T23:48:52.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.954+0000 7f71cf7fe640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f71b80a2bb0 0x7f71b80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:52.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.955+0000 7f71cf7fe640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f71b80a2bb0 0x7f71b80a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71c00058c0 tx=0x7f71c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:52.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:52.957+0000 7f71ccff9640 1 -- 172.21.15.120:0/3909749328 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71c40be0a0 con 0x7f71d0101d10 2024-09-11T23:48:53.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:48:53.131+0000 7f71d5fa1640 1 -- 172.21.15.120:0/3909749328 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f71d006af20 con 0x7f71b80a2bb0 2024-09-11T23:48:53.537 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:48:53.537 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T23:48:53.537 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi083 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi083 to 4272M 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.538 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:53.539 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:53.539 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:53.539 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.539 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:48:53.539 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:48:53.539 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:53.539 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: osd.10 [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] boot 2024-09-11T23:48:53.539 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: osdmap e68: 11 total, 11 up, 11 in 2024-09-11T23:48:53.539 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:53.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:48:53.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T23:48:53.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:53.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi083 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi083 to 4272M 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:53.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:53.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:53.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:48:53.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:48:53.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:53.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: osd.10 [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] boot 2024-09-11T23:48:53.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: osdmap e68: 11 total, 11 up, 11 in 2024-09-11T23:48:53.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-11T23:48:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T23:48:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi083 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi083 to 4272M 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:53.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:53.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:53.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:53.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:48:53.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:48:53.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:48:53.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:53.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: osd.10 [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] boot 2024-09-11T23:48:53.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: osdmap e68: 11 total, 11 up, 11 in 2024-09-11T23:48:53.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:48:54.571 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:54 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:48:54.571 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:54 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:48:54.571 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:54 smithi083 ceph-mon[34805]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:54.571 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:54 smithi083 ceph-mon[34805]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:54.571 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:54 smithi083 ceph-mon[34805]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T23:48:54.601 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:54 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:48:54.601 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:54 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:48:54.601 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:54 smithi120 ceph-mon[35182]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:54.601 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:54 smithi120 ceph-mon[35182]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:54.601 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:54 smithi120 ceph-mon[35182]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T23:48:54.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:54 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:48:54.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:54 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:48:54.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:54 smithi062 ceph-mon[30958]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:48:54.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:54 smithi062 ceph-mon[30958]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:54.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:54 smithi062 ceph-mon[30958]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T23:48:56.518 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:56 smithi062 ceph-mon[30958]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:48:56.518 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:56 smithi062 ceph-mon[30958]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T23:48:56.518 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:56 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/3838834463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37bcf111-5dde-49c5-8f40-0a1a5644a081"}]: dispatch 2024-09-11T23:48:56.518 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:56 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37bcf111-5dde-49c5-8f40-0a1a5644a081"}]: dispatch 2024-09-11T23:48:56.518 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:56 smithi062 ceph-mon[30958]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37bcf111-5dde-49c5-8f40-0a1a5644a081"}]': finished 2024-09-11T23:48:56.518 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:56 smithi062 ceph-mon[30958]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T23:48:56.518 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:56 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:48:56.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:56 smithi120 ceph-mon[35182]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:48:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:56 smithi120 ceph-mon[35182]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T23:48:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:56 smithi120 ceph-mon[35182]: from='client.? 172.21.15.120:0/3838834463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37bcf111-5dde-49c5-8f40-0a1a5644a081"}]: dispatch 2024-09-11T23:48:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:56 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37bcf111-5dde-49c5-8f40-0a1a5644a081"}]: dispatch 2024-09-11T23:48:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:56 smithi120 ceph-mon[35182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37bcf111-5dde-49c5-8f40-0a1a5644a081"}]': finished 2024-09-11T23:48:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:56 smithi120 ceph-mon[35182]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T23:48:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:56 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:48:56.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:56 smithi083 ceph-mon[34805]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:48:56.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:56 smithi083 ceph-mon[34805]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T23:48:56.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:56 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/3838834463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37bcf111-5dde-49c5-8f40-0a1a5644a081"}]: dispatch 2024-09-11T23:48:56.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:56 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37bcf111-5dde-49c5-8f40-0a1a5644a081"}]: dispatch 2024-09-11T23:48:56.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:56 smithi083 ceph-mon[34805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37bcf111-5dde-49c5-8f40-0a1a5644a081"}]': finished 2024-09-11T23:48:56.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:56 smithi083 ceph-mon[34805]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T23:48:56.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:56 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:48:57.537 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:57 smithi062 ceph-mon[30958]: from='client.? 172.21.15.120:0/2839897852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:48:57.537 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:57 smithi062 ceph-mon[30958]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:48:57.537 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:57 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:48:57.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:57 smithi120 ceph-mon[35182]: from='client.? 172.21.15.120:0/2839897852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:48:57.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:57 smithi120 ceph-mon[35182]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:48:57.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:57 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:48:57.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:57 smithi083 ceph-mon[34805]: from='client.? 172.21.15.120:0/2839897852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:48:57.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:57 smithi083 ceph-mon[34805]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:48:57.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:57 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:48:58.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:48:58 smithi083 ceph-mon[34805]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 162 KiB/s, 0 objects/s recovering 2024-09-11T23:48:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:48:58 smithi062 ceph-mon[30958]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 162 KiB/s, 0 objects/s recovering 2024-09-11T23:48:59.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:48:58 smithi120 ceph-mon[35182]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 162 KiB/s, 0 objects/s recovering 2024-09-11T23:49:00.828 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:00 smithi120 ceph-mon[35182]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-09-11T23:49:00.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:00 smithi083 ceph-mon[34805]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-09-11T23:49:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:00 smithi062 ceph-mon[30958]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-09-11T23:49:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:02 smithi083 ceph-mon[34805]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:49:02.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:02 smithi062 ceph-mon[30958]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:49:03.079 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:02 smithi120 ceph-mon[35182]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:49:04.944 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:04 smithi120 ceph-mon[35182]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-11T23:49:05.037 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:04 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:49:05.037 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:04 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:05.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:04 smithi062 ceph-mon[30958]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-11T23:49:05.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:04 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:49:05.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:04 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:05.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:04 smithi083 ceph-mon[34805]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-11T23:49:05.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:04 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:49:05.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:04 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:05.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:05 smithi062 ceph-mon[30958]: Deploying daemon osd.11 on smithi120 2024-09-11T23:49:05.971 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:05 smithi120 ceph-mon[35182]: Deploying daemon osd.11 on smithi120 2024-09-11T23:49:06.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:05 smithi083 ceph-mon[34805]: Deploying daemon osd.11 on smithi120 2024-09-11T23:49:06.792 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:06 smithi120 ceph-mon[35182]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-11T23:49:06.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:06 smithi062 ceph-mon[30958]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-11T23:49:07.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:06 smithi083 ceph-mon[34805]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-11T23:49:08.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:08 smithi120 ceph-mon[35182]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:08.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:08 smithi062 ceph-mon[30958]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:09.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:08 smithi083 ceph-mon[34805]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:10.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:10 smithi120 ceph-mon[35182]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:10.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:10 smithi062 ceph-mon[30958]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:11.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:10 smithi083 ceph-mon[34805]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:11.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:11 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:11 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:11.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:11 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:11.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:11 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:11.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:11 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:11.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:11 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:11 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:11 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:11 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:12.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:12 smithi120 ceph-mon[35182]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:12.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:12 smithi062 ceph-mon[30958]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:12 smithi083 ceph-mon[34805]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:14.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.319+0000 7f71ccff9640 1 -- 172.21.15.120:0/3909749328 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f71d006af20 con 0x7f71b80a2bb0 2024-09-11T23:49:14.320 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 11 on host 'smithi120' 2024-09-11T23:49:14.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.321+0000 7f71d5fa1640 1 -- 172.21.15.120:0/3909749328 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f71b80a2bb0 msgr2=0x7f71b80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:14.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.321+0000 7f71d5fa1640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f71b80a2bb0 0x7f71b80a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71c00058c0 tx=0x7f71c0005f90 comp rx=0 tx=0).stop 2024-09-11T23:49:14.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.322+0000 7f71d5fa1640 1 -- 172.21.15.120:0/3909749328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71d0101d10 msgr2=0x7f71d0197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:14.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.322+0000 7f71d5fa1640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71d0101d10 0x7f71d0197330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71c4031440 tx=0x7f71c4031b60 comp rx=0 tx=0).stop 2024-09-11T23:49:14.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.322+0000 7f71d5fa1640 1 -- 172.21.15.120:0/3909749328 shutdown_connections 2024-09-11T23:49:14.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.322+0000 7f71d5fa1640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f71b80a2bb0 0x7f71b80a5070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:14.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.322+0000 7f71d5fa1640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71d01026d0 0x7f71d019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:14.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.322+0000 7f71d5fa1640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71d0101d10 0x7f71d0197330 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:14.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.322+0000 7f71d5fa1640 1 --2- 172.21.15.120:0/3909749328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d0100b10 0x7f71d0196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:14.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.322+0000 7f71d5fa1640 1 -- 172.21.15.120:0/3909749328 >> 172.21.15.120:0/3909749328 conn(0x7f71d00fc2a0 msgr2=0x7f71d0105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:14.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.322+0000 7f71d5fa1640 1 -- 172.21.15.120:0/3909749328 shutdown_connections 2024-09-11T23:49:14.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:14.322+0000 7f71d5fa1640 1 -- 172.21.15.120:0/3909749328 wait complete. 2024-09-11T23:49:14.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:14 smithi083 ceph-mon[34805]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:14.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:14.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:14.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:14.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:14.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:14.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:14.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:14.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:14 smithi062 ceph-mon[30958]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:14.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:14.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:14.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:14.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:14.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:14.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:15.034 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:14 smithi120 ceph-mon[35182]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:15.034 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:15.034 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:15.034 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:15.034 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:15.034 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:15.036 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:15.036 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:15.036 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:15.061 DEBUG:teuthology.orchestra.run.smithi120:osd.11> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T23:49:15.064 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T23:49:15.064 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:49:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:15.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.866+0000 7fe6acfe5640 1 -- 172.21.15.62:0/334396237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6a8101cd0 msgr2=0x7fe6a8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:15.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.866+0000 7fe6acfe5640 1 --2- 172.21.15.62:0/334396237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6a8101cd0 0x7fe6a8102150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe690009900 tx=0x7fe69002a410 comp rx=0 tx=0).stop 2024-09-11T23:49:15.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.866+0000 7fe6acfe5640 1 -- 172.21.15.62:0/334396237 shutdown_connections 2024-09-11T23:49:15.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.866+0000 7fe6acfe5640 1 --2- 172.21.15.62:0/334396237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8102690 0x7fe6a8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:15.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.866+0000 7fe6acfe5640 1 --2- 172.21.15.62:0/334396237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6a8101cd0 0x7fe6a8102150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:15.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.866+0000 7fe6acfe5640 1 --2- 172.21.15.62:0/334396237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6a8100ad0 0x7fe6a8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:15.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.866+0000 7fe6acfe5640 1 -- 172.21.15.62:0/334396237 >> 172.21.15.62:0/334396237 conn(0x7fe6a80fc280 msgr2=0x7fe6a80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:15.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.866+0000 7fe6acfe5640 1 -- 172.21.15.62:0/334396237 shutdown_connections 2024-09-11T23:49:15.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.866+0000 7fe6acfe5640 1 -- 172.21.15.62:0/334396237 wait complete. 2024-09-11T23:49:15.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.866+0000 7fe6acfe5640 1 Processor -- start 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6acfe5640 1 -- start start 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6acfe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6a8100ad0 0x7fe6a8104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6acfe5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6a8101cd0 0x7fe6a81030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6acfe5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8102690 0x7fe6a81035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6acfe5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a810c730 con 0x7fe6a8100ad0 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6acfe5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a810c5b0 con 0x7fe6a8101cd0 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6acfe5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a810c8b0 con 0x7fe6a8102690 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6a659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6a8100ad0 0x7fe6a8104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6a659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6a8100ad0 0x7fe6a8104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46590/0 (socket says 172.21.15.62:46590) 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6a659c640 1 -- 172.21.15.62:0/3909960375 learned_addr learned my addr 172.21.15.62:0/3909960375 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6a6d9d640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8102690 0x7fe6a81035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe69ffff640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6a8101cd0 0x7fe6a81030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6a659c640 1 -- 172.21.15.62:0/3909960375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8102690 msgr2=0x7fe6a81035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6a659c640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8102690 0x7fe6a81035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6a659c640 1 -- 172.21.15.62:0/3909960375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6a8101cd0 msgr2=0x7fe6a81030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6a659c640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6a8101cd0 0x7fe6a81030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6a659c640 1 -- 172.21.15.62:0/3909960375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a8103eb0 con 0x7fe6a8100ad0 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe69ffff640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6a8101cd0 0x7fe6a81030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.867+0000 7fe6a659c640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6a8100ad0 0x7fe6a8104a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe69800c970 tx=0x7fe69800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.868+0000 7fe69f7fe640 1 -- 172.21.15.62:0/3909960375 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698007c10 con 0x7fe6a8100ad0 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.868+0000 7fe69f7fe640 1 -- 172.21.15.62:0/3909960375 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe698007db0 con 0x7fe6a8100ad0 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.868+0000 7fe6acfe5640 1 -- 172.21.15.62:0/3909960375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a81a1fb0 con 0x7fe6a8100ad0 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.868+0000 7fe69f7fe640 1 -- 172.21.15.62:0/3909960375 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69800f940 con 0x7fe6a8100ad0 2024-09-11T23:49:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.868+0000 7fe6acfe5640 1 -- 172.21.15.62:0/3909960375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a81a2400 con 0x7fe6a8100ad0 2024-09-11T23:49:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.869+0000 7fe6acfe5640 1 -- 172.21.15.62:0/3909960375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a81a2690 con 0x7fe6a8100ad0 2024-09-11T23:49:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.870+0000 7fe69f7fe640 1 -- 172.21.15.62:0/3909960375 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe69800fae0 con 0x7fe6a8100ad0 2024-09-11T23:49:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.871+0000 7fe69f7fe640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe6940a2b80 0x7fe6940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.871+0000 7fe69ffff640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe6940a2b80 0x7fe6940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.871+0000 7fe69f7fe640 1 -- 172.21.15.62:0/3909960375 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7fe6980cf0a0 con 0x7fe6a8100ad0 2024-09-11T23:49:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.872+0000 7fe69ffff640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe6940a2b80 0x7fe6940a5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe690000c00 tx=0x7fe6900095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:15.874+0000 7fe69f7fe640 1 -- 172.21.15.62:0/3909960375 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe69809c700 con 0x7fe6a8100ad0 2024-09-11T23:49:16.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.052+0000 7fe6acfe5640 1 -- 172.21.15.62:0/3909960375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe6a81a2980 con 0x7fe6a8100ad0 2024-09-11T23:49:16.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.052+0000 7fe69f7fe640 1 -- 172.21.15.62:0/3909960375 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7fe69809c000 con 0x7fe6a8100ad0 2024-09-11T23:49:16.052 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:16.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 -- 172.21.15.62:0/3909960375 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe6940a2b80 msgr2=0x7fe6940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:16.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe6940a2b80 0x7fe6940a5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe690000c00 tx=0x7fe6900095f0 comp rx=0 tx=0).stop 2024-09-11T23:49:16.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 -- 172.21.15.62:0/3909960375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6a8100ad0 msgr2=0x7fe6a8104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:16.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6a8100ad0 0x7fe6a8104a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe69800c970 tx=0x7fe69800ce30 comp rx=0 tx=0).stop 2024-09-11T23:49:16.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 -- 172.21.15.62:0/3909960375 shutdown_connections 2024-09-11T23:49:16.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe6940a2b80 0x7fe6940a5040 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:16.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a8102690 0x7fe6a81035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:16.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6a8101cd0 0x7fe6a81030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:16.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 --2- 172.21.15.62:0/3909960375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6a8100ad0 0x7fe6a8104a60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:16.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 -- 172.21.15.62:0/3909960375 >> 172.21.15.62:0/3909960375 conn(0x7fe6a80fc280 msgr2=0x7fe6a80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:16.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 -- 172.21.15.62:0/3909960375 shutdown_connections 2024-09-11T23:49:16.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:16.056+0000 7fe6acfe5640 1 -- 172.21.15.62:0/3909960375 wait complete. 2024-09-11T23:49:16.802 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:16 smithi062 ceph-mon[30958]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:16.802 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:16 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3909960375' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:49:16.809 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:16 smithi120 ceph-mon[35182]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:16.809 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:16 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3909960375' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:49:16.830 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726098533,"num_in_osds":12,"osd_in_since":1726098535,"num_remapped_pgs":0} 2024-09-11T23:49:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:16 smithi083 ceph-mon[34805]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:16 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3909960375' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:49:17.121 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 11 23:49:16 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-11T23:49:16.808+0000 7f03ddfe8640 -1 osd.11 0 log_to_monitors true 2024-09-11T23:49:17.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:49:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:17 smithi083 ceph-mon[34805]: from='osd.11 [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:49:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:17 smithi083 ceph-mon[34805]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:49:17.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:17 smithi062 ceph-mon[30958]: from='osd.11 [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:49:17.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:17 smithi062 ceph-mon[30958]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:49:17.985 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:17 smithi120 ceph-mon[35182]: from='osd.11 [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:49:17.985 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:17 smithi120 ceph-mon[35182]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:49:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:18.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.606+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/4294126205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e18101d10 msgr2=0x7f5e18102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:18.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.606+0000 7f5e1e4d3640 1 --2- 172.21.15.62:0/4294126205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e18101d10 0x7f5e18102190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009900 tx=0x7f5e0c02a410 comp rx=0 tx=0).stop 2024-09-11T23:49:18.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.606+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/4294126205 shutdown_connections 2024-09-11T23:49:18.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.606+0000 7f5e1e4d3640 1 --2- 172.21.15.62:0/4294126205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e181026d0 0x7f5e18108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:18.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.606+0000 7f5e1e4d3640 1 --2- 172.21.15.62:0/4294126205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e18101d10 0x7f5e18102190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:18.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.606+0000 7f5e1e4d3640 1 --2- 172.21.15.62:0/4294126205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e18100b10 0x7f5e18100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:18.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.606+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/4294126205 >> 172.21.15.62:0/4294126205 conn(0x7f5e180fc2a0 msgr2=0x7f5e180fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.606+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/4294126205 shutdown_connections 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.606+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/4294126205 wait complete. 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.607+0000 7f5e1e4d3640 1 Processor -- start 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.607+0000 7f5e1e4d3640 1 -- start start 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.607+0000 7f5e1e4d3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e18100b10 0x7f5e18197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.607+0000 7f5e1e4d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e18101d10 0x7f5e18198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.607+0000 7f5e1e4d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e181026d0 0x7f5e1819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.607+0000 7f5e1e4d3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1810d8c0 con 0x7f5e181026d0 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.607+0000 7f5e1e4d3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1810d740 con 0x7f5e18101d10 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.607+0000 7f5e1e4d3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1810da40 con 0x7f5e18100b10 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1ca70640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e181026d0 0x7f5e1819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1ca70640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e181026d0 0x7f5e1819f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46608/0 (socket says 172.21.15.62:46608) 2024-09-11T23:49:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1ca70640 1 -- 172.21.15.62:0/160877315 learned_addr learned my addr 172.21.15.62:0/160877315 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e17fff640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e18100b10 0x7f5e18197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e177fe640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e18101d10 0x7f5e18198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1ca70640 1 -- 172.21.15.62:0/160877315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e18100b10 msgr2=0x7f5e18197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1ca70640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e18100b10 0x7f5e18197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1ca70640 1 -- 172.21.15.62:0/160877315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e18101d10 msgr2=0x7f5e18198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1ca70640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e18101d10 0x7f5e18198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1ca70640 1 -- 172.21.15.62:0/160877315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1819fb30 con 0x7f5e181026d0 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1ca70640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e181026d0 0x7f5e1819f4c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000eaa0 tx=0x7f5e1000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e157fa640 1 -- 172.21.15.62:0/160877315 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000cdb0 con 0x7f5e181026d0 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/160877315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1819fe20 con 0x7f5e181026d0 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e157fa640 1 -- 172.21.15.62:0/160877315 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e10005020 con 0x7f5e181026d0 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e157fa640 1 -- 172.21.15.62:0/160877315 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100116b0 con 0x7f5e181026d0 2024-09-11T23:49:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.608+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/160877315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e181a0280 con 0x7f5e181026d0 2024-09-11T23:49:18.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.609+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/160877315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e18107f60 con 0x7f5e181026d0 2024-09-11T23:49:18.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.611+0000 7f5e157fa640 1 -- 172.21.15.62:0/160877315 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5e10005560 con 0x7f5e181026d0 2024-09-11T23:49:18.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.611+0000 7f5e157fa640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5e000a2bb0 0x7f5e000a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:18.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.611+0000 7f5e17fff640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5e000a2bb0 0x7f5e000a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:18.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.611+0000 7f5e157fa640 1 -- 172.21.15.62:0/160877315 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6831+0+0 (secure 0 0 0) 0x7f5e100cef70 con 0x7f5e181026d0 2024-09-11T23:49:18.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.612+0000 7f5e17fff640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5e000a2bb0 0x7f5e000a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e18101b70 tx=0x7f5e0800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:18.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.614+0000 7f5e157fa640 1 -- 172.21.15.62:0/160877315 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e10016030 con 0x7f5e181026d0 2024-09-11T23:49:18.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.791+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/160877315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5e1802d990 con 0x7f5e181026d0 2024-09-11T23:49:18.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:18.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:49:18.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T23:49:18.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:18.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='osd.11 [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:49:18.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:49:18.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:18.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:18.793 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:18.793 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:18.793 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:18.793 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:18.793 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:18.793 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:18.793 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:18.793 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:18 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.792+0000 7f5e157fa640 1 -- 172.21.15.62:0/160877315 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f5e1009be50 con 0x7f5e181026d0 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/160877315 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5e000a2bb0 msgr2=0x7f5e000a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5e000a2bb0 0x7f5e000a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e18101b70 tx=0x7f5e0800b390 comp rx=0 tx=0).stop 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/160877315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e181026d0 msgr2=0x7f5e1819f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e181026d0 0x7f5e1819f4c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000eaa0 tx=0x7f5e1000ef60 comp rx=0 tx=0).stop 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/160877315 shutdown_connections 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5e000a2bb0 0x7f5e000a5070 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e181026d0 0x7f5e1819f4c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e18101d10 0x7f5e18198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 --2- 172.21.15.62:0/160877315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e18100b10 0x7f5e18197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/160877315 >> 172.21.15.62:0/160877315 conn(0x7f5e180fc2a0 msgr2=0x7f5e18105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/160877315 shutdown_connections 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:18.794+0000 7f5e1e4d3640 1 -- 172.21.15.62:0/160877315 wait complete. 2024-09-11T23:49:18.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:18.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:49:18.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T23:49:18.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:18.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='osd.11 [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:49:18.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:49:18.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:18.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:18.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:18.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:18.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:18.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:18.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:18.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:18.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:18.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:18 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:19.121 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-11T23:49:18.648+0000 7f03d9f6b640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T23:49:19.121 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-11T23:49:18.661+0000 7f03d5d93640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:49:19.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:19.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:49:19.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T23:49:19.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:19.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='osd.11 [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:49:19.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-11T23:49:19.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:19.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:19.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:19.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:19.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:19.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:49:19.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:19.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:19.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:19.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:18 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:19.579 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726098533,"num_in_osds":12,"osd_in_since":1726098535,"num_remapped_pgs":0} 2024-09-11T23:49:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:19 smithi083 ceph-mon[34805]: Detected new or changed devices on smithi120 2024-09-11T23:49:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:19 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-11T23:49:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:19 smithi083 ceph-mon[34805]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:49:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:19 smithi083 ceph-mon[34805]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T23:49:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:19 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:19 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:19 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/160877315' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:49:19.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:19 smithi062 ceph-mon[30958]: Detected new or changed devices on smithi120 2024-09-11T23:49:19.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:19 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-11T23:49:19.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:19 smithi062 ceph-mon[30958]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:49:19.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:19 smithi062 ceph-mon[30958]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T23:49:19.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:19 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:19.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:19 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:19.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:19 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/160877315' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:49:20.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:19 smithi120 ceph-mon[35182]: Detected new or changed devices on smithi120 2024-09-11T23:49:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:19 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-11T23:49:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:19 smithi120 ceph-mon[35182]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-11T23:49:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:19 smithi120 ceph-mon[35182]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T23:49:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:19 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:19 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:19 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/160877315' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:49:20.580 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:49:20.851 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:20 smithi083 ceph-mon[34805]: purged_snaps scrub starts 2024-09-11T23:49:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:20 smithi083 ceph-mon[34805]: purged_snaps scrub ok 2024-09-11T23:49:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:20 smithi083 ceph-mon[34805]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:20 smithi083 ceph-mon[34805]: osd.11 [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] boot 2024-09-11T23:49:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:20 smithi083 ceph-mon[34805]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T23:49:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:20 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:20.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:20 smithi062 ceph-mon[30958]: purged_snaps scrub starts 2024-09-11T23:49:20.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:20 smithi062 ceph-mon[30958]: purged_snaps scrub ok 2024-09-11T23:49:20.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:20 smithi062 ceph-mon[30958]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:20.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:20 smithi062 ceph-mon[30958]: osd.11 [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] boot 2024-09-11T23:49:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:20 smithi062 ceph-mon[30958]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T23:49:20.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:20 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:21.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:20 smithi120 ceph-mon[35182]: purged_snaps scrub starts 2024-09-11T23:49:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:20 smithi120 ceph-mon[35182]: purged_snaps scrub ok 2024-09-11T23:49:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:20 smithi120 ceph-mon[35182]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:49:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:20 smithi120 ceph-mon[35182]: osd.11 [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] boot 2024-09-11T23:49:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:20 smithi120 ceph-mon[35182]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T23:49:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:20 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:49:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.314+0000 7fef63acf640 1 -- 172.21.15.62:0/657989009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef5c1026d0 msgr2=0x7fef5c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.314+0000 7fef63acf640 1 --2- 172.21.15.62:0/657989009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef5c1026d0 0x7fef5c108f60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fef54009ca0 tx=0x7fef5402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:49:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.314+0000 7fef63acf640 1 -- 172.21.15.62:0/657989009 shutdown_connections 2024-09-11T23:49:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.314+0000 7fef63acf640 1 --2- 172.21.15.62:0/657989009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef5c1026d0 0x7fef5c108f60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.314+0000 7fef63acf640 1 --2- 172.21.15.62:0/657989009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5c101d10 0x7fef5c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.314+0000 7fef63acf640 1 --2- 172.21.15.62:0/657989009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef5c100b10 0x7fef5c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.314+0000 7fef63acf640 1 -- 172.21.15.62:0/657989009 >> 172.21.15.62:0/657989009 conn(0x7fef5c0fc2a0 msgr2=0x7fef5c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.314+0000 7fef63acf640 1 -- 172.21.15.62:0/657989009 shutdown_connections 2024-09-11T23:49:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.314+0000 7fef63acf640 1 -- 172.21.15.62:0/657989009 wait complete. 2024-09-11T23:49:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef63acf640 1 Processor -- start 2024-09-11T23:49:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef63acf640 1 -- start start 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef63acf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5c100b10 0x7fef5c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef63acf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef5c101d10 0x7fef5c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef63acf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef5c1026d0 0x7fef5c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef63acf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c10c770 con 0x7fef5c1026d0 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef63acf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c10c5f0 con 0x7fef5c100b10 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef63acf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c10c8f0 con 0x7fef5c101d10 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef6206c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef5c1026d0 0x7fef5c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef6206c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef5c1026d0 0x7fef5c1a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48256/0 (socket says 172.21.15.62:48256) 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.315+0000 7fef6206c640 1 -- 172.21.15.62:0/1363705756 learned_addr learned my addr 172.21.15.62:0/1363705756 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef6186b640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5c100b10 0x7fef5c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef6106a640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef5c101d10 0x7fef5c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef6206c640 1 -- 172.21.15.62:0/1363705756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef5c101d10 msgr2=0x7fef5c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef6206c640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef5c101d10 0x7fef5c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef6206c640 1 -- 172.21.15.62:0/1363705756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5c100b10 msgr2=0x7fef5c199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef6206c640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5c100b10 0x7fef5c199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef6206c640 1 -- 172.21.15.62:0/1363705756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c1a1c20 con 0x7fef5c1026d0 2024-09-11T23:49:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef6186b640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5c100b10 0x7fef5c199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:49:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef6206c640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef5c1026d0 0x7fef5c1a15b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef540098d0 tx=0x7fef5402f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef52ffd640 1 -- 172.21.15.62:0/1363705756 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5402fc30 con 0x7fef5c1026d0 2024-09-11T23:49:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef63acf640 1 -- 172.21.15.62:0/1363705756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c1a1eb0 con 0x7fef5c1026d0 2024-09-11T23:49:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef52ffd640 1 -- 172.21.15.62:0/1363705756 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef54003e50 con 0x7fef5c1026d0 2024-09-11T23:49:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef52ffd640 1 -- 172.21.15.62:0/1363705756 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef540305c0 con 0x7fef5c1026d0 2024-09-11T23:49:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.316+0000 7fef63acf640 1 -- 172.21.15.62:0/1363705756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c1a2360 con 0x7fef5c1026d0 2024-09-11T23:49:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.317+0000 7fef63acf640 1 -- 172.21.15.62:0/1363705756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c1a25c0 con 0x7fef5c1026d0 2024-09-11T23:49:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.318+0000 7fef52ffd640 1 -- 172.21.15.62:0/1363705756 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fef54004b40 con 0x7fef5c1026d0 2024-09-11T23:49:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.319+0000 7fef52ffd640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fef480a2bb0 0x7fef480a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.319+0000 7fef6186b640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fef480a2bb0 0x7fef480a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.319+0000 7fef52ffd640 1 -- 172.21.15.62:0/1363705756 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fef540f0140 con 0x7fef5c1026d0 2024-09-11T23:49:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.320+0000 7fef6186b640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fef480a2bb0 0x7fef480a5070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef5c101b70 tx=0x7fef58005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:21.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.322+0000 7fef52ffd640 1 -- 172.21.15.62:0/1363705756 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef540433d0 con 0x7fef5c1026d0 2024-09-11T23:49:21.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.499+0000 7fef63acf640 1 -- 172.21.15.62:0/1363705756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fef5c1a2950 con 0x7fef5c1026d0 2024-09-11T23:49:21.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.499+0000 7fef52ffd640 1 -- 172.21.15.62:0/1363705756 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7fef540bd540 con 0x7fef5c1026d0 2024-09-11T23:49:21.499 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 -- 172.21.15.62:0/1363705756 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fef480a2bb0 msgr2=0x7fef480a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fef480a2bb0 0x7fef480a5070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef5c101b70 tx=0x7fef58005d30 comp rx=0 tx=0).stop 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 -- 172.21.15.62:0/1363705756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef5c1026d0 msgr2=0x7fef5c1a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef5c1026d0 0x7fef5c1a15b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef540098d0 tx=0x7fef5402f840 comp rx=0 tx=0).stop 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 -- 172.21.15.62:0/1363705756 shutdown_connections 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fef480a2bb0 0x7fef480a5070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef5c1026d0 0x7fef5c1a15b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef5c101d10 0x7fef5c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 --2- 172.21.15.62:0/1363705756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5c100b10 0x7fef5c199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 -- 172.21.15.62:0/1363705756 >> 172.21.15.62:0/1363705756 conn(0x7fef5c0fc2a0 msgr2=0x7fef5c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 -- 172.21.15.62:0/1363705756 shutdown_connections 2024-09-11T23:49:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:21.502+0000 7fef63acf640 1 -- 172.21.15.62:0/1363705756 wait complete. 2024-09-11T23:49:21.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:21 smithi083 ceph-mon[34805]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T23:49:21.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:21 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1363705756' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:49:21.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:21 smithi062 ceph-mon[30958]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T23:49:21.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:21 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1363705756' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:49:22.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:21 smithi120 ceph-mon[35182]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T23:49:22.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:21 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1363705756' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:49:22.237 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726098559,"num_in_osds":12,"osd_in_since":1726098535,"num_remapped_pgs":0} 2024-09-11T23:49:22.237 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:49:22.490 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:22.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:22 smithi062 ceph-mon[30958]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:22.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.908+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/245682233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb8103660 msgr2=0x7f5fb810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:22.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.908+0000 7f5fbd9bb640 1 --2- 172.21.15.62:0/245682233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb8103660 0x7f5fb810bb50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009900 tx=0x7f5fac02a480 comp rx=0 tx=0).stop 2024-09-11T23:49:22.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.908+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/245682233 shutdown_connections 2024-09-11T23:49:22.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.908+0000 7f5fbd9bb640 1 --2- 172.21.15.62:0/245682233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb810c170 0x7f5fb810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:22.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.908+0000 7f5fbd9bb640 1 --2- 172.21.15.62:0/245682233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb8103660 0x7f5fb810bb50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:22.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.908+0000 7f5fbd9bb640 1 --2- 172.21.15.62:0/245682233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb8102d40 0x7f5fb8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:22.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.908+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/245682233 >> 172.21.15.62:0/245682233 conn(0x7f5fb80fc7a0 msgr2=0x7f5fb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:22.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.908+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/245682233 shutdown_connections 2024-09-11T23:49:22.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.909+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/245682233 wait complete. 2024-09-11T23:49:22.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.909+0000 7f5fbd9bb640 1 Processor -- start 2024-09-11T23:49:22.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.910+0000 7f5fbd9bb640 1 -- start start 2024-09-11T23:49:22.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fbd9bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb8102d40 0x7f5fb81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:22.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fbd9bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb8103660 0x7f5fb8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:22.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fbd9bb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb810c170 0x7f5fb8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:22.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fbd9bb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8111f70 con 0x7f5fb810c170 2024-09-11T23:49:22.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fbd9bb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8111df0 con 0x7f5fb8103660 2024-09-11T23:49:22.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fbd9bb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81120f0 con 0x7f5fb8102d40 2024-09-11T23:49:22.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb810c170 0x7f5fb8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb810c170 0x7f5fb8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48276/0 (socket says 172.21.15.62:48276) 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb77fe640 1 -- 172.21.15.62:0/2770099915 learned_addr learned my addr 172.21.15.62:0/2770099915 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb6ffd640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb8102d40 0x7f5fb81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb67fc640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb8103660 0x7f5fb8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb77fe640 1 -- 172.21.15.62:0/2770099915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb8102d40 msgr2=0x7f5fb81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb77fe640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb8102d40 0x7f5fb81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb77fe640 1 -- 172.21.15.62:0/2770099915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb8103660 msgr2=0x7f5fb8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb77fe640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb8103660 0x7f5fb8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb77fe640 1 -- 172.21.15.62:0/2770099915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb81a68f0 con 0x7f5fb810c170 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.911+0000 7f5fb77fe640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb810c170 0x7f5fb8101420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000eaa0 tx=0x7f5fb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.912+0000 7f5fbc9b9640 1 -- 172.21.15.62:0/2770099915 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000cdb0 con 0x7f5fb810c170 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.912+0000 7f5fbc9b9640 1 -- 172.21.15.62:0/2770099915 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb0005020 con 0x7f5fb810c170 2024-09-11T23:49:22.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.912+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/2770099915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb81a6be0 con 0x7f5fb810c170 2024-09-11T23:49:22.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.912+0000 7f5fbc9b9640 1 -- 172.21.15.62:0/2770099915 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00116b0 con 0x7f5fb810c170 2024-09-11T23:49:22.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.912+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/2770099915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb81a70a0 con 0x7f5fb810c170 2024-09-11T23:49:22.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.913+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/2770099915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb8103120 con 0x7f5fb810c170 2024-09-11T23:49:22.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.914+0000 7f5fbc9b9640 1 -- 172.21.15.62:0/2770099915 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5fb0005560 con 0x7f5fb810c170 2024-09-11T23:49:22.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.915+0000 7f5fbc9b9640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5fa00a2b80 0x7f5fa00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:22.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.915+0000 7f5fb6ffd640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5fa00a2b80 0x7f5fa00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:22.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.915+0000 7f5fbc9b9640 1 -- 172.21.15.62:0/2770099915 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f5fb00cf090 con 0x7f5fb810c170 2024-09-11T23:49:22.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.915+0000 7f5fb6ffd640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5fa00a2b80 0x7f5fa00a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80058c0 tx=0x7f5fa800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:22.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:22 smithi083 ceph-mon[34805]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:22.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:22.918+0000 7f5fbc9b9640 1 -- 172.21.15.62:0/2770099915 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5fb0016030 con 0x7f5fb810c170 2024-09-11T23:49:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.099+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/2770099915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fb8101ed0 con 0x7f5fb810c170 2024-09-11T23:49:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.100+0000 7f5fbc9b9640 1 -- 172.21.15.62:0/2770099915 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18433 (secure 0 0 0) 0x7f5fb8101ed0 con 0x7f5fb810c170 2024-09-11T23:49:23.100 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:23.101 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":76,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:49:20.652597+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:49:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/2770099915 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5fa00a2b80 msgr2=0x7f5fa00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5fa00a2b80 0x7f5fa00a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80058c0 tx=0x7f5fa800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:49:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/2770099915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb810c170 msgr2=0x7f5fb8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb810c170 0x7f5fb8101420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000eaa0 tx=0x7f5fb000ef60 comp rx=0 tx=0).stop 2024-09-11T23:49:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/2770099915 shutdown_connections 2024-09-11T23:49:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5fa00a2b80 0x7f5fa00a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb810c170 0x7f5fb8101420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb8103660 0x7f5fb8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 --2- 172.21.15.62:0/2770099915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fb8102d40 0x7f5fb81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/2770099915 >> 172.21.15.62:0/2770099915 conn(0x7f5fb80fc7a0 msgr2=0x7f5fb80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:23.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/2770099915 shutdown_connections 2024-09-11T23:49:23.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:23.103+0000 7f5fbd9bb640 1 -- 172.21.15.62:0/2770099915 wait complete. 2024-09-11T23:49:23.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:22 smithi120 ceph-mon[35182]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:23.913 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:45:29.220992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:49:23.914 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T23:49:23.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:23 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2770099915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:49:24.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:23 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2770099915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:49:24.165 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:24.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:23 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2770099915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:49:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.647+0000 7f37bd1a0640 1 -- 172.21.15.62:0/2506949096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37b8101270 msgr2=0x7f37b810d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.647+0000 7f37bd1a0640 1 --2- 172.21.15.62:0/2506949096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37b8101270 0x7f37b810d7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009cd0 tx=0x7f37b002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 -- 172.21.15.62:0/2506949096 shutdown_connections 2024-09-11T23:49:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 --2- 172.21.15.62:0/2506949096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37b8101270 0x7f37b810d7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 --2- 172.21.15.62:0/2506949096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37b81008d0 0x7f37b8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 --2- 172.21.15.62:0/2506949096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b81068d0 0x7f37b8106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 -- 172.21.15.62:0/2506949096 >> 172.21.15.62:0/2506949096 conn(0x7f37b80fc610 msgr2=0x7f37b80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 -- 172.21.15.62:0/2506949096 shutdown_connections 2024-09-11T23:49:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 -- 172.21.15.62:0/2506949096 wait complete. 2024-09-11T23:49:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 Processor -- start 2024-09-11T23:49:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 -- start start 2024-09-11T23:49:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37b81008d0 0x7f37b8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37b8101270 0x7f37b81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b81068d0 0x7f37b81a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8110cd0 con 0x7f37b8101270 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8110b50 con 0x7f37b81068d0 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.648+0000 7f37bd1a0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8110e50 con 0x7f37b81008d0 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37affff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37b8101270 0x7f37b81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37affff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37b8101270 0x7f37b81995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48306/0 (socket says 172.21.15.62:48306) 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37affff640 1 -- 172.21.15.62:0/1881185694 learned_addr learned my addr 172.21.15.62:0/1881185694 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37b7fff640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37b81008d0 0x7f37b8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37bc99f640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b81068d0 0x7f37b81a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37affff640 1 -- 172.21.15.62:0/1881185694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37b81008d0 msgr2=0x7f37b8199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37affff640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37b81008d0 0x7f37b8199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37affff640 1 -- 172.21.15.62:0/1881185694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b81068d0 msgr2=0x7f37b81a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37affff640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b81068d0 0x7f37b81a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37affff640 1 -- 172.21.15.62:0/1881185694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b81a4640 con 0x7f37b8101270 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37affff640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37b8101270 0x7f37b81995c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37a000e9f0 tx=0x7f37a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37b5ffb640 1 -- 172.21.15.62:0/1881185694 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a000ccc0 con 0x7f37b8101270 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37bd1a0640 1 -- 172.21.15.62:0/1881185694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b81a4810 con 0x7f37b8101270 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37b5ffb640 1 -- 172.21.15.62:0/1881185694 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a000ce60 con 0x7f37b8101270 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37b5ffb640 1 -- 172.21.15.62:0/1881185694 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0010690 con 0x7f37b8101270 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.649+0000 7f37bd1a0640 1 -- 172.21.15.62:0/1881185694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81a4d20 con 0x7f37b8101270 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.650+0000 7f37bd1a0640 1 -- 172.21.15.62:0/1881185694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b8100d30 con 0x7f37b8101270 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.651+0000 7f37b5ffb640 1 -- 172.21.15.62:0/1881185694 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f37a00046c0 con 0x7f37b8101270 2024-09-11T23:49:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.652+0000 7f37b5ffb640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37a40a2b80 0x7f37a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.652+0000 7f37b7fff640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37a40a2b80 0x7f37a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.653+0000 7f37b5ffb640 1 -- 172.21.15.62:0/1881185694 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f37a00cf060 con 0x7f37b8101270 2024-09-11T23:49:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.653+0000 7f37b7fff640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37a40a2b80 0x7f37a40a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37a800ab60 tx=0x7f37a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.657+0000 7f37b5ffb640 1 -- 172.21.15.62:0/1881185694 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37a009c460 con 0x7f37b8101270 2024-09-11T23:49:24.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.831+0000 7f37bd1a0640 1 -- 172.21.15.62:0/1881185694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f37b81a4fe0 con 0x7f37b8101270 2024-09-11T23:49:24.831 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:24 smithi062 ceph-mon[30958]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:24.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.832+0000 7f37b5ffb640 1 -- 172.21.15.62:0/1881185694 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f37a009bd60 con 0x7f37b8101270 2024-09-11T23:49:24.832 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-09-11T23:49:24.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.833+0000 7f37bd1a0640 1 -- 172.21.15.62:0/1881185694 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37a40a2b80 msgr2=0x7f37a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:24.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37a40a2b80 0x7f37a40a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37a800ab60 tx=0x7f37a800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 -- 172.21.15.62:0/1881185694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37b8101270 msgr2=0x7f37b81995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:24.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37b8101270 0x7f37b81995c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37a000e9f0 tx=0x7f37a000edf0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 -- 172.21.15.62:0/1881185694 shutdown_connections 2024-09-11T23:49:24.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f37a40a2b80 0x7f37a40a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b81068d0 0x7f37b81a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37b8101270 0x7f37b81995c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 --2- 172.21.15.62:0/1881185694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37b81008d0 0x7f37b8199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:24.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 -- 172.21.15.62:0/1881185694 >> 172.21.15.62:0/1881185694 conn(0x7f37b80fc610 msgr2=0x7f37b80fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:24.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 -- 172.21.15.62:0/1881185694 shutdown_connections 2024-09-11T23:49:24.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:24.834+0000 7f37bd1a0640 1 -- 172.21.15.62:0/1881185694 wait complete. 2024-09-11T23:49:25.068 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:24 smithi083 ceph-mon[34805]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:25.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:24 smithi120 ceph-mon[35182]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:25.586 INFO:tasks.cephadm:Adding mds.a on smithi062 2024-09-11T23:49:25.586 INFO:tasks.cephadm:Adding mds.d on smithi062 2024-09-11T23:49:25.586 INFO:tasks.cephadm:Adding mds.g on smithi062 2024-09-11T23:49:25.586 INFO:tasks.cephadm:Adding mds.j on smithi062 2024-09-11T23:49:25.586 INFO:tasks.cephadm:Adding mds.b on smithi083 2024-09-11T23:49:25.586 INFO:tasks.cephadm:Adding mds.e on smithi083 2024-09-11T23:49:25.587 INFO:tasks.cephadm:Adding mds.h on smithi083 2024-09-11T23:49:25.587 INFO:tasks.cephadm:Adding mds.k on smithi083 2024-09-11T23:49:25.587 INFO:tasks.cephadm:Adding mds.c on smithi120 2024-09-11T23:49:25.587 INFO:tasks.cephadm:Adding mds.f on smithi120 2024-09-11T23:49:25.587 INFO:tasks.cephadm:Adding mds.i on smithi120 2024-09-11T23:49:25.587 INFO:tasks.cephadm:Adding mds.l on smithi120 2024-09-11T23:49:25.587 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi120=c;smithi120=f;smithi120=i;smithi120=l' 2024-09-11T23:49:25.839 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:49:25.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:25 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1881185694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:49:25.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:25 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1881185694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:49:26.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:25 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1881185694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:49:26.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.348+0000 7fd15720c640 1 -- 172.21.15.120:0/454369176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150100b10 msgr2=0x7fd150100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:26.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.348+0000 7fd15720c640 1 --2- 172.21.15.120:0/454369176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150100b10 0x7fd150100f10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd144009900 tx=0x7fd14402a410 comp rx=0 tx=0).stop 2024-09-11T23:49:26.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.348+0000 7fd15720c640 1 -- 172.21.15.120:0/454369176 shutdown_connections 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.348+0000 7fd15720c640 1 --2- 172.21.15.120:0/454369176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1501026d0 0x7fd150108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.348+0000 7fd15720c640 1 --2- 172.21.15.120:0/454369176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd150101d10 0x7fd150102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.348+0000 7fd15720c640 1 --2- 172.21.15.120:0/454369176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150100b10 0x7fd150100f10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.348+0000 7fd15720c640 1 -- 172.21.15.120:0/454369176 >> 172.21.15.120:0/454369176 conn(0x7fd1500fc2a0 msgr2=0x7fd1500fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.348+0000 7fd15720c640 1 -- 172.21.15.120:0/454369176 shutdown_connections 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.348+0000 7fd15720c640 1 -- 172.21.15.120:0/454369176 wait complete. 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd15720c640 1 Processor -- start 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd15720c640 1 -- start start 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd15720c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150100b10 0x7fd150104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd15720c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd150101d10 0x7fd1501030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:26.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd15720c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1501026d0 0x7fd150103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd15720c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd15010c790 con 0x7fd1501026d0 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd15720c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd15010c610 con 0x7fd150101d10 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd15720c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd15010c910 con 0x7fd150100b10 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd154fa8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150100b10 0x7fd150104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd154fa8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150100b10 0x7fd150104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44022/0 (socket says 172.21.15.120:44022) 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd154fa8640 1 -- 172.21.15.120:0/982619133 learned_addr learned my addr 172.21.15.120:0/982619133 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd14ffff640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd150101d10 0x7fd1501030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd1557a9640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1501026d0 0x7fd150103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd154fa8640 1 -- 172.21.15.120:0/982619133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd150101d10 msgr2=0x7fd1501030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd154fa8640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd150101d10 0x7fd1501030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:26.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd154fa8640 1 -- 172.21.15.120:0/982619133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1501026d0 msgr2=0x7fd150103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd154fa8640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1501026d0 0x7fd150103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.349+0000 7fd154fa8640 1 -- 172.21.15.120:0/982619133 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd150103ef0 con 0x7fd150100b10 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.350+0000 7fd154fa8640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150100b10 0x7fd150104aa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd144031610 tx=0x7fd1440043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.350+0000 7fd14dffb640 1 -- 172.21.15.120:0/982619133 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14403f030 con 0x7fd150100b10 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.350+0000 7fd15720c640 1 -- 172.21.15.120:0/982619133 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1501a20d0 con 0x7fd150100b10 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.350+0000 7fd14dffb640 1 -- 172.21.15.120:0/982619133 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd144003ce0 con 0x7fd150100b10 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.350+0000 7fd14dffb640 1 -- 172.21.15.120:0/982619133 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144035070 con 0x7fd150100b10 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.350+0000 7fd15720c640 1 -- 172.21.15.120:0/982619133 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1501a2580 con 0x7fd150100b10 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.351+0000 7fd15720c640 1 -- 172.21.15.120:0/982619133 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1501a27b0 con 0x7fd150100b10 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.352+0000 7fd14dffb640 1 -- 172.21.15.120:0/982619133 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd144042030 con 0x7fd150100b10 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.353+0000 7fd14dffb640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1400a2b80 0x7fd1400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.353+0000 7fd14dffb640 1 -- 172.21.15.120:0/982619133 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd1440ef140 con 0x7fd150100b10 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.353+0000 7fd14ffff640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1400a2b80 0x7fd1400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:26.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.353+0000 7fd14ffff640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1400a2b80 0x7fd1400a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd150104820 tx=0x7fd13c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:26.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.357+0000 7fd14dffb640 1 -- 172.21.15.120:0/982619133 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1440bc540 con 0x7fd150100b10 2024-09-11T23:49:26.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.532+0000 7fd15720c640 1 -- 172.21.15.120:0/982619133 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi120=c;smithi120=f;smithi120=i;smithi120=l", "target": ["mon-mgr", ""]}) -- 0x7fd15006b010 con 0x7fd1400a2b80 2024-09-11T23:49:26.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.539+0000 7fd14dffb640 1 -- 172.21.15.120:0/982619133 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd15006b010 con 0x7fd1400a2b80 2024-09-11T23:49:26.539 INFO:teuthology.orchestra.run.smithi120.stdout:Scheduled mds.all update... 2024-09-11T23:49:26.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 -- 172.21.15.120:0/982619133 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1400a2b80 msgr2=0x7fd1400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:26.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1400a2b80 0x7fd1400a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd150104820 tx=0x7fd13c00b390 comp rx=0 tx=0).stop 2024-09-11T23:49:26.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 -- 172.21.15.120:0/982619133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150100b10 msgr2=0x7fd150104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:26.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150100b10 0x7fd150104aa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd144031610 tx=0x7fd1440043b0 comp rx=0 tx=0).stop 2024-09-11T23:49:26.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 -- 172.21.15.120:0/982619133 shutdown_connections 2024-09-11T23:49:26.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1400a2b80 0x7fd1400a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:26.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1501026d0 0x7fd150103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:26.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd150101d10 0x7fd1501030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:26.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 --2- 172.21.15.120:0/982619133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150100b10 0x7fd150104aa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:26.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 -- 172.21.15.120:0/982619133 >> 172.21.15.120:0/982619133 conn(0x7fd1500fc2a0 msgr2=0x7fd15006a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:26.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 -- 172.21.15.120:0/982619133 shutdown_connections 2024-09-11T23:49:26.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-11T23:49:26.541+0000 7fd15720c640 1 -- 172.21.15.120:0/982619133 wait complete. 2024-09-11T23:49:26.867 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:26 smithi062 ceph-mon[30958]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:26.867 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:26 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:26.867 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:26 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:26.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:26 smithi120 ceph-mon[35182]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:26.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:26 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:26.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:26 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:26 smithi083 ceph-mon[34805]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:26 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:26 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:27.273 DEBUG:teuthology.orchestra.run.smithi062:mds.a> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T23:49:27.276 DEBUG:teuthology.orchestra.run.smithi062:mds.d> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T23:49:27.320 DEBUG:teuthology.orchestra.run.smithi062:mds.g> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T23:49:27.363 DEBUG:teuthology.orchestra.run.smithi062:mds.j> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T23:49:27.405 DEBUG:teuthology.orchestra.run.smithi083:mds.b> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T23:49:27.408 DEBUG:teuthology.orchestra.run.smithi083:mds.e> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T23:49:27.451 DEBUG:teuthology.orchestra.run.smithi083:mds.h> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T23:49:27.495 DEBUG:teuthology.orchestra.run.smithi083:mds.k> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T23:49:27.538 DEBUG:teuthology.orchestra.run.smithi120:mds.c> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T23:49:27.541 DEBUG:teuthology.orchestra.run.smithi120:mds.f> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T23:49:27.583 DEBUG:teuthology.orchestra.run.smithi120:mds.i> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T23:49:27.627 DEBUG:teuthology.orchestra.run.smithi120:mds.l> sudo journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T23:49:27.669 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T23:49:27.670 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:27 smithi120 ceph-mon[35182]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi120=c;smithi120=f;smithi120=i;smithi120=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:49:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:27 smithi120 ceph-mon[35182]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi120=c;smithi120=f;smithi120=i;smithi120=l;count:12 2024-09-11T23:49:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:27.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:27 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:27.931 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:27 smithi062 ceph-mon[30958]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi120=c;smithi120=f;smithi120=i;smithi120=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:49:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:27 smithi062 ceph-mon[30958]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi120=c;smithi120=f;smithi120=i;smithi120=l;count:12 2024-09-11T23:49:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:27.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:27 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:27 smithi083 ceph-mon[34805]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi120=c;smithi120=f;smithi120=i;smithi120=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:49:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:27 smithi083 ceph-mon[34805]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi120=c;smithi120=f;smithi120=i;smithi120=l;count:12 2024-09-11T23:49:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:27 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:28.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.423+0000 7fd9b3937640 1 -- 172.21.15.62:0/2221683014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9ac112b90 msgr2=0x7fd9ac114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:28.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.423+0000 7fd9b3937640 1 --2- 172.21.15.62:0/2221683014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9ac112b90 0x7fd9ac114f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009ca0 tx=0x7fd9a402a950 comp rx=0 tx=0).stop 2024-09-11T23:49:28.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.423+0000 7fd9b3937640 1 -- 172.21.15.62:0/2221683014 shutdown_connections 2024-09-11T23:49:28.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.423+0000 7fd9b3937640 1 --2- 172.21.15.62:0/2221683014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9ac112b90 0x7fd9ac114f80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:28.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.423+0000 7fd9b3937640 1 --2- 172.21.15.62:0/2221683014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ac074eb0 0x7fd9ac112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.423+0000 7fd9b3937640 1 --2- 172.21.15.62:0/2221683014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac074590 0x7fd9ac074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.423+0000 7fd9b3937640 1 -- 172.21.15.62:0/2221683014 >> 172.21.15.62:0/2221683014 conn(0x7fd9ac0fd8c0 msgr2=0x7fd9ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.423+0000 7fd9b3937640 1 -- 172.21.15.62:0/2221683014 shutdown_connections 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.423+0000 7fd9b3937640 1 -- 172.21.15.62:0/2221683014 wait complete. 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.423+0000 7fd9b3937640 1 Processor -- start 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b3937640 1 -- start start 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b3937640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ac074590 0x7fd9ac1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b3937640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac074eb0 0x7fd9ac1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b3937640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9ac112b90 0x7fd9ac19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b3937640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac118550 con 0x7fd9ac074590 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b3937640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1183d0 con 0x7fd9ac074eb0 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b3937640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1186d0 con 0x7fd9ac112b90 2024-09-11T23:49:28.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b16d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ac074590 0x7fd9ac1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b16d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ac074590 0x7fd9ac1a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48322/0 (socket says 172.21.15.62:48322) 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b16d3640 1 -- 172.21.15.62:0/734195783 learned_addr learned my addr 172.21.15.62:0/734195783 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b1ed4640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9ac112b90 0x7fd9ac19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b0ed2640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac074eb0 0x7fd9ac1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b16d3640 1 -- 172.21.15.62:0/734195783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9ac112b90 msgr2=0x7fd9ac19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b16d3640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9ac112b90 0x7fd9ac19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b16d3640 1 -- 172.21.15.62:0/734195783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac074eb0 msgr2=0x7fd9ac1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b16d3640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac074eb0 0x7fd9ac1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b16d3640 1 -- 172.21.15.62:0/734195783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac19efa0 con 0x7fd9ac074590 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b0ed2640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac074eb0 0x7fd9ac1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.424+0000 7fd9b16d3640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ac074590 0x7fd9ac1a46b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800e9f0 tx=0x7fd9a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:28.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.425+0000 7fd9a27fc640 1 -- 172.21.15.62:0/734195783 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800cd80 con 0x7fd9ac074590 2024-09-11T23:49:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.425+0000 7fd9b3937640 1 -- 172.21.15.62:0/734195783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ac19f290 con 0x7fd9ac074590 2024-09-11T23:49:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.425+0000 7fd9a27fc640 1 -- 172.21.15.62:0/734195783 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a8010910 con 0x7fd9ac074590 2024-09-11T23:49:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.425+0000 7fd9a27fc640 1 -- 172.21.15.62:0/734195783 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80188f0 con 0x7fd9ac074590 2024-09-11T23:49:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.425+0000 7fd9b3937640 1 -- 172.21.15.62:0/734195783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ac1ab580 con 0x7fd9ac074590 2024-09-11T23:49:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.426+0000 7fd9b3937640 1 -- 172.21.15.62:0/734195783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ac074970 con 0x7fd9ac074590 2024-09-11T23:49:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.427+0000 7fd9a27fc640 1 -- 172.21.15.62:0/734195783 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd9a8010ac0 con 0x7fd9ac074590 2024-09-11T23:49:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.427+0000 7fd9a27fc640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9980a2b80 0x7fd9980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.428+0000 7fd9b0ed2640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9980a2b80 0x7fd9980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.428+0000 7fd9a27fc640 1 -- 172.21.15.62:0/734195783 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd9a80cefe0 con 0x7fd9ac074590 2024-09-11T23:49:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.428+0000 7fd9b0ed2640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9980a2b80 0x7fd9980a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac114c40 tx=0x7fd99c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:28.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.432+0000 7fd9a27fc640 1 -- 172.21.15.62:0/734195783 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9a809c3e0 con 0x7fd9ac074590 2024-09-11T23:49:28.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.644+0000 7fd9b3937640 1 -- 172.21.15.62:0/734195783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9ac1ab880 con 0x7fd9ac074590 2024-09-11T23:49:28.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.645+0000 7fd9a27fc640 1 -- 172.21.15.62:0/734195783 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fd9a809bce0 con 0x7fd9ac074590 2024-09-11T23:49:28.645 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:28.645 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"btime":"2024-09-11T23:42:20:574216+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T23:49:28.645 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 1 2024-09-11T23:49:28.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 -- 172.21.15.62:0/734195783 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9980a2b80 msgr2=0x7fd9980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:28.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9980a2b80 0x7fd9980a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac114c40 tx=0x7fd99c005f90 comp rx=0 tx=0).stop 2024-09-11T23:49:28.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 -- 172.21.15.62:0/734195783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ac074590 msgr2=0x7fd9ac1a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:28.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ac074590 0x7fd9ac1a46b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800e9f0 tx=0x7fd9a800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:49:28.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 -- 172.21.15.62:0/734195783 shutdown_connections 2024-09-11T23:49:28.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd9980a2b80 0x7fd9980a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:28.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9ac112b90 0x7fd9ac19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:28.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9ac074eb0 0x7fd9ac1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:28.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 --2- 172.21.15.62:0/734195783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ac074590 0x7fd9ac1a46b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:28.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 -- 172.21.15.62:0/734195783 >> 172.21.15.62:0/734195783 conn(0x7fd9ac0fd8c0 msgr2=0x7fd9ac0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:28.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 -- 172.21.15.62:0/734195783 shutdown_connections 2024-09-11T23:49:28.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:28.647+0000 7fd9b3937640 1 -- 172.21.15.62:0/734195783 wait complete. 2024-09-11T23:49:28.753 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:28 smithi120 ceph-mon[35182]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:28.753 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:28 smithi120 ceph-mon[35182]: Deploying daemon mds.l on smithi120 2024-09-11T23:49:28.753 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:28 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/734195783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:28.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:28 smithi062 ceph-mon[30958]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:28.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:28 smithi062 ceph-mon[30958]: Deploying daemon mds.l on smithi120 2024-09-11T23:49:28.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:28 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/734195783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:29.121 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 11 23:49:28 smithi120 systemd[1]: Starting Ceph mds.l for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:49:29.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:28 smithi083 ceph-mon[34805]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:29.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:28 smithi083 ceph-mon[34805]: Deploying daemon mds.l on smithi120 2024-09-11T23:49:29.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:28 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/734195783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:29.430 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 11 23:49:29 smithi120 podman[64991]: 2024-09-11 23:49:29.056960407 +0000 UTC m=+0.015592146 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:49:29.430 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 11 23:49:29 smithi120 podman[64991]: 2024-09-11 23:49:29.176386423 +0000 UTC m=+0.135018217 container create 86a7bf9d1c1250931d7a1514c875921f31a94915ee9020d23f0180cff375b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T23:49:29.430 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 11 23:49:29 smithi120 podman[64991]: 2024-09-11 23:49:29.431262058 +0000 UTC m=+0.389893809 container init 86a7bf9d1c1250931d7a1514c875921f31a94915ee9020d23f0180cff375b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T23:49:29.871 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 11 23:49:29 smithi120 podman[64991]: 2024-09-11 23:49:29.434194959 +0000 UTC m=+0.392826714 container start 86a7bf9d1c1250931d7a1514c875921f31a94915ee9020d23f0180cff375b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:49:29.871 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 11 23:49:29 smithi120 ceph-mds[65010]: -- 172.21.15.120:0/953852990 <== mon.1 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a385deb860 con 0x55a385dc8800 2024-09-11T23:49:29.871 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 11 23:49:29 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-l[65006]: starting mds.l at 2024-09-11T23:49:29.871 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 11 23:49:29 smithi120 bash[64991]: 86a7bf9d1c1250931d7a1514c875921f31a94915ee9020d23f0180cff375b001 2024-09-11T23:49:29.871 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 11 23:49:29 smithi120 systemd[1]: Started Ceph mds.l for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.120:6832/1221552790,v1:172.21.15.120:6833/1221552790] up:boot 2024-09-11T23:49:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: fsmap 1 up:standby 2024-09-11T23:49:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:49:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:30.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:30.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:30.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:30.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:30 smithi083 ceph-mon[34805]: Deploying daemon mds.k on smithi083 2024-09-11T23:49:30.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:30.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.120:6832/1221552790,v1:172.21.15.120:6833/1221552790] up:boot 2024-09-11T23:49:30.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: fsmap 1 up:standby 2024-09-11T23:49:30.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:49:30.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:30.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:30.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:30.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:30.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:30.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:30.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:30 smithi062 ceph-mon[30958]: Deploying daemon mds.k on smithi083 2024-09-11T23:49:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.120:6832/1221552790,v1:172.21.15.120:6833/1221552790] up:boot 2024-09-11T23:49:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: fsmap 1 up:standby 2024-09-11T23:49:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:49:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:30.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:30.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:30 smithi120 ceph-mon[35182]: Deploying daemon mds.k on smithi083 2024-09-11T23:49:31.398 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:31.882 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 11 23:49:31 smithi083 systemd[1]: Starting Ceph mds.k for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:49:32.153 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 11 23:49:31 smithi083 podman[63458]: 2024-09-11 23:49:31.785553153 +0000 UTC m=+0.015758197 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:49:32.153 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 11 23:49:31 smithi083 podman[63458]: 2024-09-11 23:49:31.901702006 +0000 UTC m=+0.131907086 container create b49664dd6ce399d04c2d05d250624485ae599aba8e0544fdb9fb12475195a232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T23:49:32.153 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 11 23:49:32 smithi083 podman[63458]: 2024-09-11 23:49:32.154360537 +0000 UTC m=+0.384565575 container init b49664dd6ce399d04c2d05d250624485ae599aba8e0544fdb9fb12475195a232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T23:49:32.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.147+0000 7f5c32ec4640 1 -- 172.21.15.62:0/3621754501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c2c106f10 msgr2=0x7f5c2c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:32.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.147+0000 7f5c32ec4640 1 --2- 172.21.15.62:0/3621754501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c2c106f10 0x7f5c2c10ea90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009ca0 tx=0x7f5c2402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.147+0000 7f5c32ec4640 1 -- 172.21.15.62:0/3621754501 shutdown_connections 2024-09-11T23:49:32.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.147+0000 7f5c32ec4640 1 --2- 172.21.15.62:0/3621754501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c2c106f10 0x7f5c2c10ea90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.147+0000 7f5c32ec4640 1 --2- 172.21.15.62:0/3621754501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2c102790 0x7f5c2c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.147+0000 7f5c32ec4640 1 --2- 172.21.15.62:0/3621754501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c2c101de0 0x7f5c2c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.147+0000 7f5c32ec4640 1 -- 172.21.15.62:0/3621754501 >> 172.21.15.62:0/3621754501 conn(0x7f5c2c0fd8c0 msgr2=0x7f5c2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.147+0000 7f5c32ec4640 1 -- 172.21.15.62:0/3621754501 shutdown_connections 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.147+0000 7f5c32ec4640 1 -- 172.21.15.62:0/3621754501 wait complete. 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c32ec4640 1 Processor -- start 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c32ec4640 1 -- start start 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c32ec4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c2c101de0 0x7f5c2c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c32ec4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2c102790 0x7f5c2c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c32ec4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c2c106f10 0x7f5c2c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c32ec4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c111f90 con 0x7f5c2c101de0 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c32ec4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c111e10 con 0x7f5c2c102790 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c32ec4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c112110 con 0x7f5c2c106f10 2024-09-11T23:49:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c30c60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c2c101de0 0x7f5c2c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c30c60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c2c101de0 0x7f5c2c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57304/0 (socket says 172.21.15.62:57304) 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c30c60640 1 -- 172.21.15.62:0/703855479 learned_addr learned my addr 172.21.15.62:0/703855479 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c2bfff640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2c102790 0x7f5c2c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c31461640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c2c106f10 0x7f5c2c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c30c60640 1 -- 172.21.15.62:0/703855479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c2c106f10 msgr2=0x7f5c2c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c30c60640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c2c106f10 0x7f5c2c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c30c60640 1 -- 172.21.15.62:0/703855479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2c102790 msgr2=0x7f5c2c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c30c60640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2c102790 0x7f5c2c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c30c60640 1 -- 172.21.15.62:0/703855479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c19f620 con 0x7f5c2c101de0 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.148+0000 7f5c31461640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c2c106f10 0x7f5c2c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.149+0000 7f5c30c60640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c2c101de0 0x7f5c2c19a670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400b4a0 tx=0x7f5c1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:32.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.149+0000 7f5c29ffb640 1 -- 172.21.15.62:0/703855479 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14014070 con 0x7f5c2c101de0 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.149+0000 7f5c29ffb640 1 -- 172.21.15.62:0/703855479 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c14010af0 con 0x7f5c2c101de0 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.149+0000 7f5c32ec4640 1 -- 172.21.15.62:0/703855479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c19f910 con 0x7f5c2c101de0 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.149+0000 7f5c29ffb640 1 -- 172.21.15.62:0/703855479 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14017d10 con 0x7f5c2c101de0 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.149+0000 7f5c32ec4640 1 -- 172.21.15.62:0/703855479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c1a71f0 con 0x7f5c2c101de0 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.150+0000 7f5c32ec4640 1 -- 172.21.15.62:0/703855479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2c1021c0 con 0x7f5c2c101de0 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.151+0000 7f5c29ffb640 1 -- 172.21.15.62:0/703855479 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5c14017070 con 0x7f5c2c101de0 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.152+0000 7f5c29ffb640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5c180a2b80 0x7f5c180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.152+0000 7f5c2bfff640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5c180a2b80 0x7f5c180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.152+0000 7f5c29ffb640 1 -- 172.21.15.62:0/703855479 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f5c140cdf70 con 0x7f5c2c101de0 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.152+0000 7f5c2bfff640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5c180a2b80 0x7f5c180a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c1088e0 tx=0x7f5c1c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.156+0000 7f5c29ffb640 1 -- 172.21.15.62:0/703855479 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c1409b370 con 0x7f5c2c101de0 2024-09-11T23:49:32.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.369+0000 7f5c32ec4640 1 -- 172.21.15.62:0/703855479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c2c19b9d0 con 0x7f5c2c101de0 2024-09-11T23:49:32.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.370+0000 7f5c29ffb640 1 -- 172.21.15.62:0/703855479 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f5c1409ac70 con 0x7f5c2c101de0 2024-09-11T23:49:32.370 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:32.370 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":2,"btime":"2024-09-11T23:49:29:462544+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T23:49:32.370 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 2 2024-09-11T23:49:32.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.372+0000 7f5c32ec4640 1 -- 172.21.15.62:0/703855479 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5c180a2b80 msgr2=0x7f5c180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.372+0000 7f5c32ec4640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5c180a2b80 0x7f5c180a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c1088e0 tx=0x7f5c1c00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.372+0000 7f5c32ec4640 1 -- 172.21.15.62:0/703855479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c2c101de0 msgr2=0x7f5c2c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.372+0000 7f5c32ec4640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c2c101de0 0x7f5c2c19a670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400b4a0 tx=0x7f5c1400b960 comp rx=0 tx=0).stop 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.372+0000 7f5c32ec4640 1 -- 172.21.15.62:0/703855479 shutdown_connections 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.372+0000 7f5c32ec4640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5c180a2b80 0x7f5c180a5040 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c1088e0 tx=0x7f5c1c00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.372+0000 7f5c32ec4640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c2c106f10 0x7f5c2c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.372+0000 7f5c32ec4640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2c102790 0x7f5c2c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.372+0000 7f5c32ec4640 1 --2- 172.21.15.62:0/703855479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c2c101de0 0x7f5c2c19a670 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.372+0000 7f5c32ec4640 1 -- 172.21.15.62:0/703855479 >> 172.21.15.62:0/703855479 conn(0x7f5c2c0fd8c0 msgr2=0x7f5c2c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.373+0000 7f5c32ec4640 1 -- 172.21.15.62:0/703855479 shutdown_connections 2024-09-11T23:49:32.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:32.373+0000 7f5c32ec4640 1 -- 172.21.15.62:0/703855479 wait complete. 2024-09-11T23:49:32.417 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 11 23:49:32 smithi083 podman[63458]: 2024-09-11 23:49:32.157385776 +0000 UTC m=+0.387590823 container start b49664dd6ce399d04c2d05d250624485ae599aba8e0544fdb9fb12475195a232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-k, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True) 2024-09-11T23:49:32.417 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 11 23:49:32 smithi083 ceph-mds[63477]: -- 172.21.15.83:0/488555377 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ce30dd3860 con 0x55ce30db1000 2024-09-11T23:49:32.417 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 11 23:49:32 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-k[63473]: starting mds.k at 2024-09-11T23:49:32.417 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 11 23:49:32 smithi083 bash[63458]: b49664dd6ce399d04c2d05d250624485ae599aba8e0544fdb9fb12475195a232 2024-09-11T23:49:32.418 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 11 23:49:32 smithi083 systemd[1]: Started Ceph mds.k for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:32 smithi062 ceph-mon[30958]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:32.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:32.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:32.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:32.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:32.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:32.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:32 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:32.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:32 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/703855479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:32.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:32 smithi120 ceph-mon[35182]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:32.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:32.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:32.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:32.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:32.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:32.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:32 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:32.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:32 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/703855479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:32.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:32 smithi083 ceph-mon[34805]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:32.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:32.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:32.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:32.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:32.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:32.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:32 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:32.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:32 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/703855479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:33.593 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:33 smithi083 ceph-mon[34805]: Deploying daemon mds.j on smithi062 2024-09-11T23:49:33.594 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:33 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.83:6832/3300661503,v1:172.21.15.83:6833/3300661503] up:boot 2024-09-11T23:49:33.594 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:33 smithi083 ceph-mon[34805]: fsmap 2 up:standby 2024-09-11T23:49:33.594 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:33 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:49:33.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:33 smithi062 ceph-mon[30958]: Deploying daemon mds.j on smithi062 2024-09-11T23:49:33.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:33 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.83:6832/3300661503,v1:172.21.15.83:6833/3300661503] up:boot 2024-09-11T23:49:33.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:33 smithi062 ceph-mon[30958]: fsmap 2 up:standby 2024-09-11T23:49:33.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:33 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:49:33.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:33 smithi120 ceph-mon[35182]: Deploying daemon mds.j on smithi062 2024-09-11T23:49:33.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:33 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.83:6832/3300661503,v1:172.21.15.83:6833/3300661503] up:boot 2024-09-11T23:49:33.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:33 smithi120 ceph-mon[35182]: fsmap 2 up:standby 2024-09-11T23:49:33.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:33 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:49:34.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:34 smithi062 ceph-mon[30958]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:34.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:34 smithi120 ceph-mon[35182]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:34 smithi083 ceph-mon[34805]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:34.965 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 11 23:49:34 smithi062 systemd[1]: Starting Ceph mds.j for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:49:35.135 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:35.296 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 11 23:49:35 smithi062 podman[70852]: 2024-09-11 23:49:34.934013509 +0000 UTC m=+0.016266055 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:49:35.296 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 11 23:49:35 smithi062 podman[70852]: 2024-09-11 23:49:35.03658572 +0000 UTC m=+0.118838250 container create 200292cae1191b3f6574068ce6bb59cffbed917465a2e609ac979a8710441cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T23:49:35.297 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 11 23:49:35 smithi062 podman[70852]: 2024-09-11 23:49:35.29785453 +0000 UTC m=+0.380107062 container init 200292cae1191b3f6574068ce6bb59cffbed917465a2e609ac979a8710441cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T23:49:35.483 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:35.608 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 11 23:49:35 smithi062 podman[70852]: 2024-09-11 23:49:35.301011221 +0000 UTC m=+0.383263758 container start 200292cae1191b3f6574068ce6bb59cffbed917465a2e609ac979a8710441cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-11T23:49:35.608 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 11 23:49:35 smithi062 ceph-mds[70889]: -- 172.21.15.62:0/1889774463 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5613bdc31860 con 0x5613bdc0e800 2024-09-11T23:49:35.608 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 11 23:49:35 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-j[70875]: starting mds.j at 2024-09-11T23:49:35.608 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 11 23:49:35 smithi062 bash[70852]: 200292cae1191b3f6574068ce6bb59cffbed917465a2e609ac979a8710441cef 2024-09-11T23:49:35.609 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 11 23:49:35 smithi062 systemd[1]: Started Ceph mds.j for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:36.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.992+0000 7f3fc3fff640 1 -- 172.21.15.62:0/3183063831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc4102380 msgr2=0x7f3fc410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:36.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.992+0000 7f3fc3fff640 1 --2- 172.21.15.62:0/3183063831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc4102380 0x7f3fc410e9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009cd0 tx=0x7f3fbc02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.993+0000 7f3fc3fff640 1 -- 172.21.15.62:0/3183063831 shutdown_connections 2024-09-11T23:49:36.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.993+0000 7f3fc3fff640 1 --2- 172.21.15.62:0/3183063831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc4102380 0x7f3fc410e9c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.993+0000 7f3fc3fff640 1 --2- 172.21.15.62:0/3183063831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc41019e0 0x7f3fc4101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.993+0000 7f3fc3fff640 1 --2- 172.21.15.62:0/3183063831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fc4107af0 0x7f3fc4107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.993+0000 7f3fc3fff640 1 -- 172.21.15.62:0/3183063831 >> 172.21.15.62:0/3183063831 conn(0x7f3fc40fd700 msgr2=0x7f3fc40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.993+0000 7f3fc3fff640 1 -- 172.21.15.62:0/3183063831 shutdown_connections 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.993+0000 7f3fc3fff640 1 -- 172.21.15.62:0/3183063831 wait complete. 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.993+0000 7f3fc3fff640 1 Processor -- start 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc3fff640 1 -- start start 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fc41019e0 0x7f3fc41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc4102380 0x7f3fc4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc4107af0 0x7f3fc419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc3fff640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4111ef0 con 0x7f3fc4102380 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc3fff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4111d70 con 0x7f3fc4107af0 2024-09-11T23:49:36.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc3fff640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4112070 con 0x7f3fc41019e0 2024-09-11T23:49:36.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc27fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc4102380 0x7f3fc4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:36.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc27fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc4102380 0x7f3fc4198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57348/0 (socket says 172.21.15.62:57348) 2024-09-11T23:49:36.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc27fc640 1 -- 172.21.15.62:0/694272511 learned_addr learned my addr 172.21.15.62:0/694272511 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:36.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc2ffd640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fc41019e0 0x7f3fc41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:36.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc37fe640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc4107af0 0x7f3fc419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:36.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc27fc640 1 -- 172.21.15.62:0/694272511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fc41019e0 msgr2=0x7f3fc41982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:36.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc27fc640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fc41019e0 0x7f3fc41982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc27fc640 1 -- 172.21.15.62:0/694272511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc4107af0 msgr2=0x7f3fc419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:36.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc27fc640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc4107af0 0x7f3fc419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc27fc640 1 -- 172.21.15.62:0/694272511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc419d2a0 con 0x7f3fc4102380 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.994+0000 7f3fc27fc640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc4102380 0x7f3fc4198830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e9f0 tx=0x7f3fb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.995+0000 7f3fc8a00640 1 -- 172.21.15.62:0/694272511 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800cd80 con 0x7f3fc4102380 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.995+0000 7f3fc8a00640 1 -- 172.21.15.62:0/694272511 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb8010910 con 0x7f3fc4102380 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.995+0000 7f3fc3fff640 1 -- 172.21.15.62:0/694272511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc419d590 con 0x7f3fc4102380 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.995+0000 7f3fc8a00640 1 -- 172.21.15.62:0/694272511 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8018990 con 0x7f3fc4102380 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.995+0000 7f3fc3fff640 1 -- 172.21.15.62:0/694272511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc41a4e70 con 0x7f3fc4102380 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.996+0000 7f3fc3fff640 1 -- 172.21.15.62:0/694272511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc4101e40 con 0x7f3fc4102380 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.997+0000 7f3fc8a00640 1 -- 172.21.15.62:0/694272511 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3fb8010ac0 con 0x7f3fc4102380 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.998+0000 7f3fc8a00640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3fac0a2b80 0x7f3fac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.998+0000 7f3fc2ffd640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3fac0a2b80 0x7f3fac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.998+0000 7f3fc8a00640 1 -- 172.21.15.62:0/694272511 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f3fb80cf440 con 0x7f3fc4102380 2024-09-11T23:49:36.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:35.998+0000 7f3fc2ffd640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3fac0a2b80 0x7f3fac0a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4005d00 tx=0x7f3fb400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:36.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.001+0000 7f3fc8a00640 1 -- 172.21.15.62:0/694272511 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3fb809c7a0 con 0x7f3fc4102380 2024-09-11T23:49:36.219 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:36.219 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":4,"btime":"2024-09-11T23:49:35:484433+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T23:49:36.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.219+0000 7f3fc3fff640 1 -- 172.21.15.62:0/694272511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fc4199650 con 0x7f3fc4102380 2024-09-11T23:49:36.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.219+0000 7f3fc8a00640 1 -- 172.21.15.62:0/694272511 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f3fc4199650 con 0x7f3fc4102380 2024-09-11T23:49:36.220 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 4 2024-09-11T23:49:36.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 -- 172.21.15.62:0/694272511 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3fac0a2b80 msgr2=0x7f3fac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:36.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3fac0a2b80 0x7f3fac0a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4005d00 tx=0x7f3fb400b410 comp rx=0 tx=0).stop 2024-09-11T23:49:36.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 -- 172.21.15.62:0/694272511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc4102380 msgr2=0x7f3fc4198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:36.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc4102380 0x7f3fc4198830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e9f0 tx=0x7f3fb800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 -- 172.21.15.62:0/694272511 shutdown_connections 2024-09-11T23:49:36.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3fac0a2b80 0x7f3fac0a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc4107af0 0x7f3fc419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc4102380 0x7f3fc4198830 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 --2- 172.21.15.62:0/694272511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fc41019e0 0x7f3fc41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 -- 172.21.15.62:0/694272511 >> 172.21.15.62:0/694272511 conn(0x7f3fc40fd700 msgr2=0x7f3fc40fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:36.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 -- 172.21.15.62:0/694272511 shutdown_connections 2024-09-11T23:49:36.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:36.221+0000 7f3fc3fff640 1 -- 172.21.15.62:0/694272511 wait complete. 2024-09-11T23:49:36.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:36.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:36.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:36.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:36.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:36.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:36.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:36.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: Deploying daemon mds.d on smithi062 2024-09-11T23:49:36.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.62:6834/1692157360,v1:172.21.15.62:6835/1692157360] up:boot 2024-09-11T23:49:36.623 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: fsmap 3 up:standby 2024-09-11T23:49:36.623 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:49:36.623 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:36 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/694272511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:36.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: Deploying daemon mds.d on smithi062 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.62:6834/1692157360,v1:172.21.15.62:6835/1692157360] up:boot 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: fsmap 3 up:standby 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:49:36.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:36 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/694272511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: Deploying daemon mds.d on smithi062 2024-09-11T23:49:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.62:6834/1692157360,v1:172.21.15.62:6835/1692157360] up:boot 2024-09-11T23:49:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: fsmap 3 up:standby 2024-09-11T23:49:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:49:36.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:36 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/694272511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:38.466 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 11 23:49:38 smithi062 systemd[1]: Starting Ceph mds.d for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:49:38.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:38 smithi062 ceph-mon[30958]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:38.792 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 11 23:49:38 smithi062 podman[71269]: 2024-09-11 23:49:38.529747578 +0000 UTC m=+0.024624144 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:49:38.792 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 11 23:49:38 smithi062 podman[71269]: 2024-09-11 23:49:38.636014521 +0000 UTC m=+0.130891118 container create 2d408df0c1e5dddfa72418fa5cabc67dc871a8f8bd1c22b35fdd24f6b87fa773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T23:49:38.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:38 smithi120 ceph-mon[35182]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:38.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:38 smithi083 ceph-mon[34805]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:39.215 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 11 23:49:38 smithi062 podman[71269]: 2024-09-11 23:49:38.930322964 +0000 UTC m=+0.425199516 container init 2d408df0c1e5dddfa72418fa5cabc67dc871a8f8bd1c22b35fdd24f6b87fa773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T23:49:39.215 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 11 23:49:38 smithi062 podman[71269]: 2024-09-11 23:49:38.933441959 +0000 UTC m=+0.428318514 container start 2d408df0c1e5dddfa72418fa5cabc67dc871a8f8bd1c22b35fdd24f6b87fa773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-11T23:49:39.216 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 11 23:49:38 smithi062 ceph-mds[71289]: -- 172.21.15.62:0/29402721 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608e7277860 con 0x5608e7255000 2024-09-11T23:49:39.216 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 11 23:49:38 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-d[71285]: starting mds.d at 2024-09-11T23:49:39.216 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 11 23:49:38 smithi062 bash[71269]: 2d408df0c1e5dddfa72418fa5cabc67dc871a8f8bd1c22b35fdd24f6b87fa773 2024-09-11T23:49:39.216 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 11 23:49:39 smithi062 systemd[1]: Started Ceph mds.d for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:39.544 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:40.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:40.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:40.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:40.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:40.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:40.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:40.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:40.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: Deploying daemon mds.e on smithi083 2024-09-11T23:49:40.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.62:6836/1495913627,v1:172.21.15.62:6837/1495913627] up:boot 2024-09-11T23:49:40.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: fsmap 4 up:standby 2024-09-11T23:49:40.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:40 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:49:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.329+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/1236435079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc4102570 msgr2=0x7fbfc41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.329+0000 7fbfcc5f7640 1 --2- 172.21.15.62:0/1236435079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc4102570 0x7fbfc41029b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009900 tx=0x7fbfc002a480 comp rx=0 tx=0).stop 2024-09-11T23:49:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.329+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/1236435079 shutdown_connections 2024-09-11T23:49:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.329+0000 7fbfcc5f7640 1 --2- 172.21.15.62:0/1236435079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfc4102ef0 0x7fbfc410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.329+0000 7fbfcc5f7640 1 --2- 172.21.15.62:0/1236435079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc4102570 0x7fbfc41029b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.329+0000 7fbfcc5f7640 1 --2- 172.21.15.62:0/1236435079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4100e20 0x7fbfc4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.329+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/1236435079 >> 172.21.15.62:0/1236435079 conn(0x7fbfc40fc7d0 msgr2=0x7fbfc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.329+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/1236435079 shutdown_connections 2024-09-11T23:49:40.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.329+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/1236435079 wait complete. 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcc5f7640 1 Processor -- start 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcc5f7640 1 -- start start 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcc5f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfc4100e20 0x7fbfc41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcc5f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4102570 0x7fbfc4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcc5f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc4102ef0 0x7fbfc419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcc5f7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4110f50 con 0x7fbfc4102ef0 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcc5f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4110dd0 con 0x7fbfc4102570 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcc5f7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41110d0 con 0x7fbfc4100e20 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcab94640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc4102ef0 0x7fbfc419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcab94640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc4102ef0 0x7fbfc419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57372/0 (socket says 172.21.15.62:57372) 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfc9b92640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4102570 0x7fbfc4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcab94640 1 -- 172.21.15.62:0/3570324394 learned_addr learned my addr 172.21.15.62:0/3570324394 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:40.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfca393640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfc4100e20 0x7fbfc41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcab94640 1 -- 172.21.15.62:0/3570324394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfc4100e20 msgr2=0x7fbfc41985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcab94640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfc4100e20 0x7fbfc41985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcab94640 1 -- 172.21.15.62:0/3570324394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4102570 msgr2=0x7fbfc4198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcab94640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4102570 0x7fbfc4198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.330+0000 7fbfcab94640 1 -- 172.21.15.62:0/3570324394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc419d550 con 0x7fbfc4102ef0 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.331+0000 7fbfcab94640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc4102ef0 0x7fbfc419ce70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00cad0 tx=0x7fbfbc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.331+0000 7fbfb77fe640 1 -- 172.21.15.62:0/3570324394 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00ae20 con 0x7fbfc4102ef0 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.331+0000 7fbfb77fe640 1 -- 172.21.15.62:0/3570324394 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc005020 con 0x7fbfc4102ef0 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.331+0000 7fbfb77fe640 1 -- 172.21.15.62:0/3570324394 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00f6f0 con 0x7fbfc4102ef0 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.331+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/3570324394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc419d840 con 0x7fbfc4102ef0 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.331+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/3570324394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc41a5120 con 0x7fbfc4102ef0 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.332+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/3570324394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc4101200 con 0x7fbfc4102ef0 2024-09-11T23:49:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.333+0000 7fbfb77fe640 1 -- 172.21.15.62:0/3570324394 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbfbc0051d0 con 0x7fbfc4102ef0 2024-09-11T23:49:40.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.334+0000 7fbfb77fe640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbfb00a2c00 0x7fbfb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:40.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.334+0000 7fbfb77fe640 1 -- 172.21.15.62:0/3570324394 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fbfbc0ceb20 con 0x7fbfc4102ef0 2024-09-11T23:49:40.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.334+0000 7fbfca393640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbfb00a2c00 0x7fbfb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:40.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.334+0000 7fbfca393640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbfb00a2c00 0x7fbfb00a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005fd0 tx=0x7fbfb8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:40.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.338+0000 7fbfb77fe640 1 -- 172.21.15.62:0/3570324394 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbfbc014030 con 0x7fbfc4102ef0 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: Deploying daemon mds.e on smithi083 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.62:6836/1495913627,v1:172.21.15.62:6837/1495913627] up:boot 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: fsmap 4 up:standby 2024-09-11T23:49:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:40 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:49:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: Deploying daemon mds.e on smithi083 2024-09-11T23:49:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.62:6836/1495913627,v1:172.21.15.62:6837/1495913627] up:boot 2024-09-11T23:49:40.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: fsmap 4 up:standby 2024-09-11T23:49:40.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:40 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:49:40.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.550+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/3570324394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfc410a0a0 con 0x7fbfc4102ef0 2024-09-11T23:49:40.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.551+0000 7fbfb77fe640 1 -- 172.21.15.62:0/3570324394 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7fbfbc09bf20 con 0x7fbfc4102ef0 2024-09-11T23:49:40.550 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:40.550 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":5,"btime":"2024-09-11T23:49:39:468068+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T23:49:40.550 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 5 2024-09-11T23:49:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/3570324394 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbfb00a2c00 msgr2=0x7fbfb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbfb00a2c00 0x7fbfb00a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005fd0 tx=0x7fbfb8005850 comp rx=0 tx=0).stop 2024-09-11T23:49:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/3570324394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc4102ef0 msgr2=0x7fbfc419ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc4102ef0 0x7fbfc419ce70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00cad0 tx=0x7fbfbc00cf90 comp rx=0 tx=0).stop 2024-09-11T23:49:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/3570324394 shutdown_connections 2024-09-11T23:49:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbfb00a2c00 0x7fbfb00a50c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc4102ef0 0x7fbfc419ce70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4102570 0x7fbfc4198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 --2- 172.21.15.62:0/3570324394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfc4100e20 0x7fbfc41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/3570324394 >> 172.21.15.62:0/3570324394 conn(0x7fbfc40fc7d0 msgr2=0x7fbfc406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/3570324394 shutdown_connections 2024-09-11T23:49:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:40.553+0000 7fbfcc5f7640 1 -- 172.21.15.62:0/3570324394 wait complete. 2024-09-11T23:49:41.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:41 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3570324394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:41.331 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:41 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3570324394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:41.331 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 11 23:49:41 smithi083 systemd[1]: Starting Ceph mds.e for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:49:41.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:41 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3570324394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:41.594 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 11 23:49:41 smithi083 podman[63771]: 2024-09-11 23:49:41.234945577 +0000 UTC m=+0.016102402 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:49:41.594 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 11 23:49:41 smithi083 podman[63771]: 2024-09-11 23:49:41.350401658 +0000 UTC m=+0.131558485 container create f8cbd6eead053188c46dd19c86db5afb333af9037862f5c3771087962062957d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:49:41.594 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 11 23:49:41 smithi083 podman[63771]: 2024-09-11 23:49:41.595237578 +0000 UTC m=+0.376394401 container init f8cbd6eead053188c46dd19c86db5afb333af9037862f5c3771087962062957d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T23:49:41.917 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 11 23:49:41 smithi083 podman[63771]: 2024-09-11 23:49:41.598753622 +0000 UTC m=+0.379910448 container start f8cbd6eead053188c46dd19c86db5afb333af9037862f5c3771087962062957d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-11T23:49:41.917 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 11 23:49:41 smithi083 ceph-mds[63790]: -- 172.21.15.83:0/1285834947 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558946ce5860 con 0x558946cc3400 2024-09-11T23:49:41.917 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 11 23:49:41 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e[63786]: starting mds.e at 2024-09-11T23:49:41.917 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 11 23:49:41 smithi083 bash[63771]: f8cbd6eead053188c46dd19c86db5afb333af9037862f5c3771087962062957d 2024-09-11T23:49:41.917 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 11 23:49:41 smithi083 systemd[1]: Started Ceph mds.e for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:43.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:42 smithi120 ceph-mon[35182]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:43.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:42 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:43.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:42 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:43.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:42 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:43.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:42 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:43.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:42 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:43.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:42 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:43.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:42 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.83:6834/3040146550,v1:172.21.15.83:6835/3040146550] up:boot 2024-09-11T23:49:43.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:42 smithi120 ceph-mon[35182]: fsmap 5 up:standby 2024-09-11T23:49:43.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:42 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:49:43.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:42 smithi083 ceph-mon[34805]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:43.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:42 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:43.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:42 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:43.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:42 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:43.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:42 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:43.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:42 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:43.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:42 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:43.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:42 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.83:6834/3040146550,v1:172.21.15.83:6835/3040146550] up:boot 2024-09-11T23:49:43.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:42 smithi083 ceph-mon[34805]: fsmap 5 up:standby 2024-09-11T23:49:43.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:42 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:49:43.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:42 smithi062 ceph-mon[30958]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:43.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:42 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.83:6834/3040146550,v1:172.21.15.83:6835/3040146550] up:boot 2024-09-11T23:49:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:42 smithi062 ceph-mon[30958]: fsmap 5 up:standby 2024-09-11T23:49:43.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:42 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:49:43.296 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:43.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:43 smithi120 ceph-mon[35182]: Deploying daemon mds.f on smithi120 2024-09-11T23:49:43.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:43 smithi062 ceph-mon[30958]: Deploying daemon mds.f on smithi120 2024-09-11T23:49:44.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.041+0000 7f4fe787f640 1 -- 172.21.15.62:0/861359034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe0101280 msgr2=0x7f4fe010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:44.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.041+0000 7f4fe787f640 1 --2- 172.21.15.62:0/861359034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe0101280 0x7f4fe010df40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009900 tx=0x7f4fd402a410 comp rx=0 tx=0).stop 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.041+0000 7f4fe787f640 1 -- 172.21.15.62:0/861359034 shutdown_connections 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.041+0000 7f4fe787f640 1 --2- 172.21.15.62:0/861359034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe010e480 0x7f4fe0110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.041+0000 7f4fe787f640 1 --2- 172.21.15.62:0/861359034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe0101280 0x7f4fe010df40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.041+0000 7f4fe787f640 1 --2- 172.21.15.62:0/861359034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe01008d0 0x7f4fe0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.041+0000 7f4fe787f640 1 -- 172.21.15.62:0/861359034 >> 172.21.15.62:0/861359034 conn(0x7f4fe00fc7a0 msgr2=0x7f4fe00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.041+0000 7f4fe787f640 1 -- 172.21.15.62:0/861359034 shutdown_connections 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.041+0000 7f4fe787f640 1 -- 172.21.15.62:0/861359034 wait complete. 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.042+0000 7f4fe787f640 1 Processor -- start 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.042+0000 7f4fe787f640 1 -- start start 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe787f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe01008d0 0x7f4fe019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe787f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe0101280 0x7f4fe019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe787f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe010e480 0x7f4fe019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe787f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe0114190 con 0x7f4fe01008d0 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe787f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe0114010 con 0x7f4fe010e480 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe787f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe0114310 con 0x7f4fe0101280 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe561b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe01008d0 0x7f4fe019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe561b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe01008d0 0x7f4fe019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39282/0 (socket says 172.21.15.62:39282) 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe561b640 1 -- 172.21.15.62:0/2370296091 learned_addr learned my addr 172.21.15.62:0/2370296091 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe4e1a640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe0101280 0x7f4fe019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe5e1c640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe010e480 0x7f4fe019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe561b640 1 -- 172.21.15.62:0/2370296091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe0101280 msgr2=0x7f4fe019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe561b640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe0101280 0x7f4fe019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe561b640 1 -- 172.21.15.62:0/2370296091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe010e480 msgr2=0x7f4fe019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe561b640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe010e480 0x7f4fe019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe561b640 1 -- 172.21.15.62:0/2370296091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe019f640 con 0x7f4fe01008d0 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe561b640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe01008d0 0x7f4fe019a690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00d950 tx=0x7f4fdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fd27fc640 1 -- 172.21.15.62:0/2370296091 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc017030 con 0x7f4fe01008d0 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fe787f640 1 -- 172.21.15.62:0/2370296091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe019f930 con 0x7f4fe01008d0 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fd27fc640 1 -- 172.21.15.62:0/2370296091 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc004020 con 0x7f4fe01008d0 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.043+0000 7f4fd27fc640 1 -- 172.21.15.62:0/2370296091 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0047a0 con 0x7f4fe01008d0 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.044+0000 7f4fe787f640 1 -- 172.21.15.62:0/2370296091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe01a7210 con 0x7f4fe01008d0 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.045+0000 7f4fe787f640 1 -- 172.21.15.62:0/2370296091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe0100cb0 con 0x7f4fe01008d0 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.046+0000 7f4fd27fc640 1 -- 172.21.15.62:0/2370296091 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4fdc0041d0 con 0x7f4fe01008d0 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.046+0000 7f4fd27fc640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4fcc0a2b80 0x7f4fcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.047+0000 7f4fe4e1a640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4fcc0a2b80 0x7f4fcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:44.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.047+0000 7f4fd27fc640 1 -- 172.21.15.62:0/2370296091 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f4fdc0cec20 con 0x7f4fe01008d0 2024-09-11T23:49:44.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.047+0000 7f4fe4e1a640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4fcc0a2b80 0x7f4fcc0a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009230 tx=0x7f4fd40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:44.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.050+0000 7f4fd27fc640 1 -- 172.21.15.62:0/2370296091 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fdc09c020 con 0x7f4fe01008d0 2024-09-11T23:49:44.066 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:43 smithi083 ceph-mon[34805]: Deploying daemon mds.f on smithi120 2024-09-11T23:49:44.234 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 11 23:49:44 smithi120 podman[65292]: 2024-09-11 23:49:44.212316378 +0000 UTC m=+0.353010835 container start 33a9ed429ceea66cdbdc090aedb31cb63027112df98af3d0c94c8051672473b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-f, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T23:49:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.265+0000 7f4fe787f640 1 -- 172.21.15.62:0/2370296091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fe019b9f0 con 0x7f4fe01008d0 2024-09-11T23:49:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.266+0000 7f4fd27fc640 1 -- 172.21.15.62:0/2370296091 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4738 (secure 0 0 0) 0x7f4fdc09b920 con 0x7f4fe01008d0 2024-09-11T23:49:44.265 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:44.265 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":6,"btime":"2024-09-11T23:49:42:053622+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T23:49:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 6 2024-09-11T23:49:44.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 -- 172.21.15.62:0/2370296091 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4fcc0a2b80 msgr2=0x7f4fcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4fcc0a2b80 0x7f4fcc0a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009230 tx=0x7f4fd40058d0 comp rx=0 tx=0).stop 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 -- 172.21.15.62:0/2370296091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe01008d0 msgr2=0x7f4fe019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe01008d0 0x7f4fe019a690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00d950 tx=0x7f4fdc00de10 comp rx=0 tx=0).stop 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 -- 172.21.15.62:0/2370296091 shutdown_connections 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4fcc0a2b80 0x7f4fcc0a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe010e480 0x7f4fe019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe0101280 0x7f4fe019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 --2- 172.21.15.62:0/2370296091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe01008d0 0x7f4fe019a690 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 -- 172.21.15.62:0/2370296091 >> 172.21.15.62:0/2370296091 conn(0x7f4fe00fc7a0 msgr2=0x7f4fe00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 -- 172.21.15.62:0/2370296091 shutdown_connections 2024-09-11T23:49:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:44.268+0000 7f4fe787f640 1 -- 172.21.15.62:0/2370296091 wait complete. 2024-09-11T23:49:44.621 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 11 23:49:44 smithi120 ceph-mds[65311]: -- 172.21.15.120:0/3296432094 <== mon.1 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560fb1aa7860 con 0x560fb1a85400 2024-09-11T23:49:44.621 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 11 23:49:44 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-f[65307]: starting mds.f at 2024-09-11T23:49:44.621 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 11 23:49:44 smithi120 bash[65292]: 33a9ed429ceea66cdbdc090aedb31cb63027112df98af3d0c94c8051672473b8 2024-09-11T23:49:44.621 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 11 23:49:44 smithi120 systemd[1]: Started Ceph mds.f for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:44.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:44 smithi083 ceph-mon[34805]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:44.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:44 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2370296091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:44.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:44.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:44.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:44.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:44.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:44.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:44 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:45.053 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:44 smithi120 ceph-mon[35182]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:45.054 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:44 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2370296091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:45.054 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:45.054 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:45.054 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:45.054 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:45.054 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:45.054 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:44 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:45.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:44 smithi062 ceph-mon[30958]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:45.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:44 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2370296091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:45.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:45.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:45.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:45.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:45.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:45.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:44 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:45.784 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:45 smithi120 ceph-mon[35182]: Deploying daemon mds.c on smithi120 2024-09-11T23:49:45.785 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:45 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.120:6834/2559115046,v1:172.21.15.120:6835/2559115046] up:boot 2024-09-11T23:49:45.785 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:45 smithi120 ceph-mon[35182]: fsmap 6 up:standby 2024-09-11T23:49:46.065 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:45 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:49:46.065 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:45 smithi120 ceph-mon[35182]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:46.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:45 smithi083 ceph-mon[34805]: Deploying daemon mds.c on smithi120 2024-09-11T23:49:46.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:45 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.120:6834/2559115046,v1:172.21.15.120:6835/2559115046] up:boot 2024-09-11T23:49:46.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:45 smithi083 ceph-mon[34805]: fsmap 6 up:standby 2024-09-11T23:49:46.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:45 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:49:46.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:45 smithi083 ceph-mon[34805]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:46.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:45 smithi062 ceph-mon[30958]: Deploying daemon mds.c on smithi120 2024-09-11T23:49:46.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:45 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.120:6834/2559115046,v1:172.21.15.120:6835/2559115046] up:boot 2024-09-11T23:49:46.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:45 smithi062 ceph-mon[30958]: fsmap 6 up:standby 2024-09-11T23:49:46.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:45 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:49:46.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:45 smithi062 ceph-mon[30958]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:46.371 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 11 23:49:46 smithi120 systemd[1]: Starting Ceph mds.c for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:49:46.762 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 11 23:49:46 smithi120 podman[65592]: 2024-09-11 23:49:46.364683625 +0000 UTC m=+0.016319027 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:49:46.762 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 11 23:49:46 smithi120 podman[65592]: 2024-09-11 23:49:46.494364645 +0000 UTC m=+0.146000047 container create 518513c8e43a83968dcf6de9ad4501530bdc978fd60a242b739e9de0013fd9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T23:49:46.918 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:47.121 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 11 23:49:46 smithi120 podman[65592]: 2024-09-11 23:49:46.763865592 +0000 UTC m=+0.415500994 container init 518513c8e43a83968dcf6de9ad4501530bdc978fd60a242b739e9de0013fd9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T23:49:47.121 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 11 23:49:46 smithi120 podman[65592]: 2024-09-11 23:49:46.766775868 +0000 UTC m=+0.418411270 container start 518513c8e43a83968dcf6de9ad4501530bdc978fd60a242b739e9de0013fd9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T23:49:47.121 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 11 23:49:46 smithi120 ceph-mds[65611]: -- 172.21.15.120:0/745255003 <== mon.1 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea30da9c20 con 0x55ea30d87400 2024-09-11T23:49:47.121 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 11 23:49:46 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c[65607]: starting mds.c at 2024-09-11T23:49:47.121 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 11 23:49:46 smithi120 bash[65592]: 518513c8e43a83968dcf6de9ad4501530bdc978fd60a242b739e9de0013fd9c3 2024-09-11T23:49:47.121 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 11 23:49:46 smithi120 systemd[1]: Started Ceph mds.c for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:47.185 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.653+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/4210715009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee4102d30 msgr2=0x7f7ee4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.653+0000 7f7eeb0c0640 1 --2- 172.21.15.62:0/4210715009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee4102d30 0x7f7ee4103110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009930 tx=0x7f7ed802a420 comp rx=0 tx=0).stop 2024-09-11T23:49:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.653+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/4210715009 shutdown_connections 2024-09-11T23:49:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.653+0000 7f7eeb0c0640 1 --2- 172.21.15.62:0/4210715009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ee41082a0 0x7f7ee410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.653+0000 7f7eeb0c0640 1 --2- 172.21.15.62:0/4210715009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee41036e0 0x7f7ee4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.653+0000 7f7eeb0c0640 1 --2- 172.21.15.62:0/4210715009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee4102d30 0x7f7ee4103110 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.653+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/4210715009 >> 172.21.15.62:0/4210715009 conn(0x7f7ee40fc7d0 msgr2=0x7f7ee40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.653+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/4210715009 shutdown_connections 2024-09-11T23:49:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.653+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/4210715009 wait complete. 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7eeb0c0640 1 Processor -- start 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7eeb0c0640 1 -- start start 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7eeb0c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ee4102d30 0x7f7ee4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7eeb0c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee41036e0 0x7f7ee4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7eeb0c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee41082a0 0x7f7ee419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7eeb0c0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41120a0 con 0x7f7ee41036e0 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7eeb0c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee4111f20 con 0x7f7ee41082a0 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7eeb0c0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee4112220 con 0x7f7ee4102d30 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7ee3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee41036e0 0x7f7ee4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7ee8e5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ee4102d30 0x7f7ee4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7ee3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee41036e0 0x7f7ee4198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39302/0 (socket says 172.21.15.62:39302) 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7ee3fff640 1 -- 172.21.15.62:0/3726418190 learned_addr learned my addr 172.21.15.62:0/3726418190 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7ee965d640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee41082a0 0x7f7ee419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.654+0000 7f7ee3fff640 1 -- 172.21.15.62:0/3726418190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ee4102d30 msgr2=0x7f7ee4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.655+0000 7f7ee3fff640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ee4102d30 0x7f7ee4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.655+0000 7f7ee3fff640 1 -- 172.21.15.62:0/3726418190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee41082a0 msgr2=0x7f7ee419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.655+0000 7f7ee3fff640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee41082a0 0x7f7ee419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.655+0000 7f7ee3fff640 1 -- 172.21.15.62:0/3726418190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee419d4d0 con 0x7f7ee41036e0 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.655+0000 7f7ee3fff640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee41036e0 0x7f7ee4198a60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000d740 tx=0x7f7ed000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.655+0000 7f7ee1ffb640 1 -- 172.21.15.62:0/3726418190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0017070 con 0x7f7ee41036e0 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.655+0000 7f7ee1ffb640 1 -- 172.21.15.62:0/3726418190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed000de50 con 0x7f7ee41036e0 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.655+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/3726418190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee419d7c0 con 0x7f7ee41036e0 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.655+0000 7f7ee1ffb640 1 -- 172.21.15.62:0/3726418190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed000f6b0 con 0x7f7ee41036e0 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.655+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/3726418190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee41a50a0 con 0x7f7ee41036e0 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.656+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/3726418190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee406a850 con 0x7f7ee41036e0 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.657+0000 7f7ee1ffb640 1 -- 172.21.15.62:0/3726418190 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7ed001a070 con 0x7f7ee41036e0 2024-09-11T23:49:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.658+0000 7f7ee1ffb640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ed40a2c00 0x7f7ed40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:47.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.659+0000 7f7ee8e5c640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ed40a2c00 0x7f7ed40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:47.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.659+0000 7f7ee1ffb640 1 -- 172.21.15.62:0/3726418190 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f7ed00ce4e0 con 0x7f7ee41036e0 2024-09-11T23:49:47.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.659+0000 7f7ee8e5c640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ed40a2c00 0x7f7ed40a50c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009900 tx=0x7f7ed80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:47.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.661+0000 7f7ee1ffb640 1 -- 172.21.15.62:0/3726418190 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ed009b270 con 0x7f7ee41036e0 2024-09-11T23:49:47.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.875+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/3726418190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ee41a5270 con 0x7f7ee41036e0 2024-09-11T23:49:47.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.876+0000 7f7ee1ffb640 1 -- 172.21.15.62:0/3726418190 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6394 (secure 0 0 0) 0x7f7ed009f570 con 0x7f7ee41036e0 2024-09-11T23:49:47.875 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:47.875 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"btime":"2024-09-11T23:49:46:844929+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-11T23:49:47.875 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-09-11T23:49:47.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/3726418190 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ed40a2c00 msgr2=0x7f7ed40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:47.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ed40a2c00 0x7f7ed40a50c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009900 tx=0x7f7ed80058d0 comp rx=0 tx=0).stop 2024-09-11T23:49:47.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/3726418190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee41036e0 msgr2=0x7f7ee4198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:47.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee41036e0 0x7f7ee4198a60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000d740 tx=0x7f7ed000dc00 comp rx=0 tx=0).stop 2024-09-11T23:49:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/3726418190 shutdown_connections 2024-09-11T23:49:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7ed40a2c00 0x7f7ed40a50c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee41082a0 0x7f7ee419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee41036e0 0x7f7ee4198a60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 --2- 172.21.15.62:0/3726418190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ee4102d30 0x7f7ee4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/3726418190 >> 172.21.15.62:0/3726418190 conn(0x7f7ee40fc7d0 msgr2=0x7f7ee4105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/3726418190 shutdown_connections 2024-09-11T23:49:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:47.878+0000 7f7eeb0c0640 1 -- 172.21.15.62:0/3726418190 wait complete. 2024-09-11T23:49:47.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:47.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] up:boot 2024-09-11T23:49:47.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: fsmap 7 up:standby 2024-09-11T23:49:47.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:49:47.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:47.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:47.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:47.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:47.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:47.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:47.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:47 smithi083 ceph-mon[34805]: Deploying daemon mds.h on smithi083 2024-09-11T23:49:48.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] up:boot 2024-09-11T23:49:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: fsmap 7 up:standby 2024-09-11T23:49:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:49:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:48.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:47 smithi120 ceph-mon[35182]: Deploying daemon mds.h on smithi083 2024-09-11T23:49:48.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:48.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] up:boot 2024-09-11T23:49:48.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: fsmap 7 up:standby 2024-09-11T23:49:48.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:49:48.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:48.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:48.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:48.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:48.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:48.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:48.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:47 smithi062 ceph-mon[30958]: Deploying daemon mds.h on smithi083 2024-09-11T23:49:49.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:48 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3726418190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:49.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:48 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3726418190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:49.167 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 11 23:49:49 smithi083 systemd[1]: Starting Ceph mds.h for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:49:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:48 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3726418190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:49.476 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 11 23:49:49 smithi083 podman[64071]: 2024-09-11 23:49:49.119101426 +0000 UTC m=+0.016048848 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:49:49.476 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 11 23:49:49 smithi083 podman[64071]: 2024-09-11 23:49:49.257987158 +0000 UTC m=+0.154934560 container create e72378e7512a938e859583a07f8a1b2075e58f6e4d5ec63af432681d3a918112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:49:49.917 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 11 23:49:49 smithi083 podman[64071]: 2024-09-11 23:49:49.477454035 +0000 UTC m=+0.374401436 container init e72378e7512a938e859583a07f8a1b2075e58f6e4d5ec63af432681d3a918112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:49:49.917 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 11 23:49:49 smithi083 podman[64071]: 2024-09-11 23:49:49.480157652 +0000 UTC m=+0.377105063 container start e72378e7512a938e859583a07f8a1b2075e58f6e4d5ec63af432681d3a918112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T23:49:49.917 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 11 23:49:49 smithi083 ceph-mds[64090]: -- 172.21.15.83:0/1716280061 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555933b5d860 con 0x555933b3b000 2024-09-11T23:49:49.917 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 11 23:49:49 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h[64086]: starting mds.h at 2024-09-11T23:49:49.917 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 11 23:49:49 smithi083 bash[64071]: e72378e7512a938e859583a07f8a1b2075e58f6e4d5ec63af432681d3a918112 2024-09-11T23:49:49.918 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 11 23:49:49 smithi083 systemd[1]: Started Ceph mds.h for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:50.577 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:50.761 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:50.761 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:50.762 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:50.762 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:50.762 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:50.762 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:50.762 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:50.762 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: Deploying daemon mds.i on smithi120 2024-09-11T23:49:50.762 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.83:6836/337493680,v1:172.21.15.83:6837/337493680] up:boot 2024-09-11T23:49:50.762 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: fsmap 8 up:standby 2024-09-11T23:49:50.762 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:50 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:49:50.849 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:50.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:50.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:50.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:50.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:50.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:50.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:50.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:50.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: Deploying daemon mds.i on smithi120 2024-09-11T23:49:50.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.83:6836/337493680,v1:172.21.15.83:6837/337493680] up:boot 2024-09-11T23:49:50.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: fsmap 8 up:standby 2024-09-11T23:49:50.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:50 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:49:50.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:50.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:50.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:50.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:50.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:50.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:50.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:50.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: Deploying daemon mds.i on smithi120 2024-09-11T23:49:50.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.83:6836/337493680,v1:172.21.15.83:6837/337493680] up:boot 2024-09-11T23:49:50.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: fsmap 8 up:standby 2024-09-11T23:49:50.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:50 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:49:51.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.377+0000 7f8c158db640 1 -- 172.21.15.62:0/3639951059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c101082e0 msgr2=0x7f8c1010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:51.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.377+0000 7f8c158db640 1 --2- 172.21.15.62:0/3639951059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c101082e0 0x7f8c1010ec50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009930 tx=0x7f8c0802a420 comp rx=0 tx=0).stop 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.377+0000 7f8c158db640 1 -- 172.21.15.62:0/3639951059 shutdown_connections 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.377+0000 7f8c158db640 1 --2- 172.21.15.62:0/3639951059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c101082e0 0x7f8c1010ec50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.377+0000 7f8c158db640 1 --2- 172.21.15.62:0/3639951059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c10103760 0x7f8c10107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.377+0000 7f8c158db640 1 --2- 172.21.15.62:0/3639951059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c10102db0 0x7f8c10103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.377+0000 7f8c158db640 1 -- 172.21.15.62:0/3639951059 >> 172.21.15.62:0/3639951059 conn(0x7f8c100fc7d0 msgr2=0x7f8c100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.377+0000 7f8c158db640 1 -- 172.21.15.62:0/3639951059 shutdown_connections 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.377+0000 7f8c158db640 1 -- 172.21.15.62:0/3639951059 wait complete. 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c158db640 1 Processor -- start 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c158db640 1 -- start start 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c158db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c10102db0 0x7f8c101985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c158db640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c10103760 0x7f8c10198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c158db640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c101082e0 0x7f8c1019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c158db640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8c101120b0 con 0x7f8c10103760 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c158db640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10111f30 con 0x7f8c10102db0 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c158db640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10112230 con 0x7f8c101082e0 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c0e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c10103760 0x7f8c10198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c0e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c10103760 0x7f8c10198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57804/0 (socket says 172.21.15.62:57804) 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c0e7fc640 1 -- 172.21.15.62:0/903253412 learned_addr learned my addr 172.21.15.62:0/903253412 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.378+0000 7f8c0effd640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c10102db0 0x7f8c101985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c0f7fe640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c101082e0 0x7f8c1019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c0e7fc640 1 -- 172.21.15.62:0/903253412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c101082e0 msgr2=0x7f8c1019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c0e7fc640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c101082e0 0x7f8c1019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c0e7fc640 1 -- 172.21.15.62:0/903253412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c10102db0 msgr2=0x7f8c101985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c0e7fc640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c10102db0 0x7f8c101985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c0e7fc640 1 -- 172.21.15.62:0/903253412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1019d560 con 0x7f8c10103760 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c0e7fc640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c10103760 0x7f8c10198af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800e9c0 tx=0x7f8bf800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c148d9640 1 -- 172.21.15.62:0/903253412 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800ccc0 con 0x7f8c10103760 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c158db640 1 -- 172.21.15.62:0/903253412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1019d850 con 0x7f8c10103760 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c148d9640 1 -- 172.21.15.62:0/903253412 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf8005500 con 0x7f8c10103760 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c148d9640 1 -- 172.21.15.62:0/903253412 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8010690 con 0x7f8c10103760 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.379+0000 7f8c158db640 1 -- 172.21.15.62:0/903253412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c101a5130 con 0x7f8c10103760 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.380+0000 7f8c158db640 1 -- 172.21.15.62:0/903253412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c10103190 con 0x7f8c10103760 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.381+0000 7f8c148d9640 1 -- 172.21.15.62:0/903253412 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8bf8004e80 con 0x7f8c10103760 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.382+0000 7f8c148d9640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8bfc0a2c00 0x7f8bfc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.382+0000 7f8c0effd640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8bfc0a2c00 0x7f8bfc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:51.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.382+0000 7f8c148d9640 1 -- 172.21.15.62:0/903253412 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f8bf80cef20 con 0x7f8c10103760 2024-09-11T23:49:51.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.383+0000 7f8c0effd640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8bfc0a2c00 0x7f8bfc0a50c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005fd0 tx=0x7f8c00009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:51.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.386+0000 7f8c148d9640 1 -- 172.21.15.62:0/903253412 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8bf809c320 con 0x7f8c10103760 2024-09-11T23:49:51.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.596+0000 7f8c158db640 1 -- 172.21.15.62:0/903253412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c1010a2c0 con 0x7f8c10103760 2024-09-11T23:49:51.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.597+0000 7f8c148d9640 1 -- 172.21.15.62:0/903253412 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7216 (secure 0 0 0) 0x7f8bf8086070 con 0x7f8c10103760 2024-09-11T23:49:51.596 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:51.596 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":9,"btime":"2024-09-11T23:49:49:853945+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-11T23:49:51.597 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 9 2024-09-11T23:49:51.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.599+0000 7f8c158db640 1 -- 172.21.15.62:0/903253412 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8bfc0a2c00 msgr2=0x7f8bfc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:51.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.599+0000 7f8c158db640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8bfc0a2c00 0x7f8bfc0a50c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005fd0 tx=0x7f8c00009500 comp rx=0 tx=0).stop 2024-09-11T23:49:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.600+0000 7f8c158db640 1 -- 172.21.15.62:0/903253412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c10103760 msgr2=0x7f8c10198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.600+0000 7f8c158db640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c10103760 0x7f8c10198af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800e9c0 tx=0x7f8bf800ee80 comp rx=0 tx=0).stop 2024-09-11T23:49:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.600+0000 7f8c158db640 1 -- 172.21.15.62:0/903253412 shutdown_connections 2024-09-11T23:49:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.600+0000 7f8c158db640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8bfc0a2c00 0x7f8bfc0a50c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.600+0000 7f8c158db640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c101082e0 0x7f8c1019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.600+0000 7f8c158db640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c10103760 0x7f8c10198af0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.600+0000 7f8c158db640 1 --2- 172.21.15.62:0/903253412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c10102db0 0x7f8c101985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.600+0000 7f8c158db640 1 -- 172.21.15.62:0/903253412 >> 172.21.15.62:0/903253412 conn(0x7f8c100fc7d0 msgr2=0x7f8c10105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.600+0000 7f8c158db640 1 -- 172.21.15.62:0/903253412 shutdown_connections 2024-09-11T23:49:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:51.600+0000 7f8c158db640 1 -- 172.21.15.62:0/903253412 wait complete. 2024-09-11T23:49:51.642 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 11 23:49:51 smithi120 systemd[1]: Starting Ceph mds.i for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:49:51.903 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:51 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/903253412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:51.903 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 11 23:49:51 smithi120 podman[65890]: 2024-09-11 23:49:51.676081426 +0000 UTC m=+0.016385223 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:49:51.903 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 11 23:49:51 smithi120 podman[65890]: 2024-09-11 23:49:51.782342854 +0000 UTC m=+0.122646695 container create ab1501235c6d42b04a6e03e1c1df3c4916ab4a5e454001fb5d86bbdf008357f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T23:49:51.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:51 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/903253412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:51.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:51 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/903253412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:52.371 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 11 23:49:52 smithi120 podman[65890]: 2024-09-11 23:49:52.035133408 +0000 UTC m=+0.375437200 container init ab1501235c6d42b04a6e03e1c1df3c4916ab4a5e454001fb5d86bbdf008357f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:49:52.371 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 11 23:49:52 smithi120 podman[65890]: 2024-09-11 23:49:52.037948182 +0000 UTC m=+0.378251979 container start ab1501235c6d42b04a6e03e1c1df3c4916ab4a5e454001fb5d86bbdf008357f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T23:49:52.371 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 11 23:49:52 smithi120 ceph-mds[65909]: -- 172.21.15.120:0/3371876312 <== mon.1 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0e26afc20 con 0x55d0e268d000 2024-09-11T23:49:52.371 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 11 23:49:52 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i[65905]: starting mds.i at 2024-09-11T23:49:52.371 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 11 23:49:52 smithi120 bash[65890]: ab1501235c6d42b04a6e03e1c1df3c4916ab4a5e454001fb5d86bbdf008357f8 2024-09-11T23:49:52.371 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 11 23:49:52 smithi120 systemd[1]: Started Ceph mds.i for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:52.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:52 smithi062 ceph-mon[30958]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:52.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:52.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:52.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:52.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:52.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:52.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:52 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:52.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:52 smithi083 ceph-mon[34805]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:52.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:52.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:52.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:52.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:52.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:52.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:52 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:52 smithi120 ceph-mon[35182]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:53.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:52 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:53.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:53 smithi062 ceph-mon[30958]: Deploying daemon mds.g on smithi062 2024-09-11T23:49:53.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:53 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] up:boot 2024-09-11T23:49:53.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:53 smithi062 ceph-mon[30958]: fsmap 9 up:standby 2024-09-11T23:49:53.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:53 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:49:53.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:53 smithi083 ceph-mon[34805]: Deploying daemon mds.g on smithi062 2024-09-11T23:49:53.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:53 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] up:boot 2024-09-11T23:49:53.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:53 smithi083 ceph-mon[34805]: fsmap 9 up:standby 2024-09-11T23:49:53.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:53 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:49:54.022 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:53 smithi120 ceph-mon[35182]: Deploying daemon mds.g on smithi062 2024-09-11T23:49:54.023 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:53 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] up:boot 2024-09-11T23:49:54.023 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:53 smithi120 ceph-mon[35182]: fsmap 9 up:standby 2024-09-11T23:49:54.023 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:53 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:49:54.215 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 11 23:49:54 smithi062 systemd[1]: Starting Ceph mds.g for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:49:54.247 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:54.580 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 11 23:49:54 smithi062 podman[72036]: 2024-09-11 23:49:54.218708035 +0000 UTC m=+0.016899703 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:49:54.580 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 11 23:49:54 smithi062 podman[72036]: 2024-09-11 23:49:54.336083982 +0000 UTC m=+0.134275635 container create 4589a60950bf4029192d42e5a65e55d0077760eed2b0b20ba6095fde2d19f38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T23:49:54.580 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 11 23:49:54 smithi062 podman[72036]: 2024-09-11 23:49:54.581192165 +0000 UTC m=+0.379383826 container init 4589a60950bf4029192d42e5a65e55d0077760eed2b0b20ba6095fde2d19f38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T23:49:54.746 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:54.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:54 smithi062 ceph-mon[30958]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:54.870 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 11 23:49:54 smithi062 podman[72036]: 2024-09-11 23:49:54.584479619 +0000 UTC m=+0.382671270 container start 4589a60950bf4029192d42e5a65e55d0077760eed2b0b20ba6095fde2d19f38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-11T23:49:54.870 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 11 23:49:54 smithi062 ceph-mds[72096]: -- 172.21.15.62:0/4069595547 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5643ddb73860 con 0x5643ddb50800 2024-09-11T23:49:54.870 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 11 23:49:54 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-g[72080]: starting mds.g at 2024-09-11T23:49:54.870 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 11 23:49:54 smithi062 bash[72036]: 4589a60950bf4029192d42e5a65e55d0077760eed2b0b20ba6095fde2d19f38a 2024-09-11T23:49:54.870 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 11 23:49:54 smithi062 systemd[1]: Started Ceph mds.g for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:54.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:54 smithi083 ceph-mon[34805]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:55.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:54 smithi120 ceph-mon[35182]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.235+0000 7f3920db4640 1 -- 172.21.15.62:0/396932110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c102790 msgr2=0x7f391c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.235+0000 7f3920db4640 1 --2- 172.21.15.62:0/396932110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c102790 0x7f391c1067e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3910009ca0 tx=0x7f391002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.235+0000 7f3920db4640 1 -- 172.21.15.62:0/396932110 shutdown_connections 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.235+0000 7f3920db4640 1 --2- 172.21.15.62:0/396932110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c106f10 0x7f391c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.235+0000 7f3920db4640 1 --2- 172.21.15.62:0/396932110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c102790 0x7f391c1067e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.235+0000 7f3920db4640 1 --2- 172.21.15.62:0/396932110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391c101de0 0x7f391c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.235+0000 7f3920db4640 1 -- 172.21.15.62:0/396932110 >> 172.21.15.62:0/396932110 conn(0x7f391c0fd8c0 msgr2=0x7f391c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.235+0000 7f3920db4640 1 -- 172.21.15.62:0/396932110 shutdown_connections 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.235+0000 7f3920db4640 1 -- 172.21.15.62:0/396932110 wait complete. 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f3920db4640 1 Processor -- start 2024-09-11T23:49:55.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f3920db4640 1 -- start start 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f3920db4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c101de0 0x7f391c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f3920db4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c102790 0x7f391c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f3920db4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391c106f10 0x7f391c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f3920db4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f391c111f90 con 0x7f391c101de0 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f3920db4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f391c111e10 con 0x7f391c102790 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f3920db4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f391c112110 con 0x7f391c106f10 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f391a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c101de0 0x7f391c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f391a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c101de0 0x7f391c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57858/0 (socket says 172.21.15.62:57858) 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.236+0000 7f391a59c640 1 -- 172.21.15.62:0/2909612123 learned_addr learned my addr 172.21.15.62:0/2909612123 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f391ad9d640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391c106f10 0x7f391c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f3919d9b640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c102790 0x7f391c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:55.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f391a59c640 1 -- 172.21.15.62:0/2909612123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391c106f10 msgr2=0x7f391c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f391a59c640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391c106f10 0x7f391c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f391a59c640 1 -- 172.21.15.62:0/2909612123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c102790 msgr2=0x7f391c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f391a59c640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c102790 0x7f391c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f391a59c640 1 -- 172.21.15.62:0/2909612123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c19d420 con 0x7f391c101de0 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f3919d9b640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c102790 0x7f391c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f391a59c640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c101de0 0x7f391c198470 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f391400d950 tx=0x7f391400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f390b7fe640 1 -- 172.21.15.62:0/2909612123 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914017030 con 0x7f391c101de0 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f3920db4640 1 -- 172.21.15.62:0/2909612123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c19d710 con 0x7f391c101de0 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f390b7fe640 1 -- 172.21.15.62:0/2909612123 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3914004020 con 0x7f391c101de0 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f390b7fe640 1 -- 172.21.15.62:0/2909612123 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39140047a0 con 0x7f391c101de0 2024-09-11T23:49:55.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.237+0000 7f3920db4640 1 -- 172.21.15.62:0/2909612123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c1a4ff0 con 0x7f391c101de0 2024-09-11T23:49:55.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.238+0000 7f3920db4640 1 -- 172.21.15.62:0/2909612123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391c1021c0 con 0x7f391c101de0 2024-09-11T23:49:55.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.240+0000 7f390b7fe640 1 -- 172.21.15.62:0/2909612123 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f39140041d0 con 0x7f391c101de0 2024-09-11T23:49:55.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.240+0000 7f390b7fe640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f39040a2b80 0x7f39040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:55.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.240+0000 7f3919d9b640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f39040a2b80 0x7f39040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:55.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.240+0000 7f390b7fe640 1 -- 172.21.15.62:0/2909612123 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f39140cec20 con 0x7f391c101de0 2024-09-11T23:49:55.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.241+0000 7f3919d9b640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f39040a2b80 0x7f39040a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f391c0fdce0 tx=0x7f3910000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:55.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.243+0000 7f390b7fe640 1 -- 172.21.15.62:0/2909612123 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f391409c020 con 0x7f391c101de0 2024-09-11T23:49:55.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.462+0000 7f3920db4640 1 -- 172.21.15.62:0/2909612123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f391c109b60 con 0x7f391c101de0 2024-09-11T23:49:55.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.463+0000 7f390b7fe640 1 -- 172.21.15.62:0/2909612123 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7f391409b920 con 0x7f391c101de0 2024-09-11T23:49:55.462 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:55.462 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":11,"btime":"2024-09-11T23:49:54:656734+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T23:49:55.462 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 11 2024-09-11T23:49:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 -- 172.21.15.62:0/2909612123 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f39040a2b80 msgr2=0x7f39040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f39040a2b80 0x7f39040a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f391c0fdce0 tx=0x7f3910000f40 comp rx=0 tx=0).stop 2024-09-11T23:49:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 -- 172.21.15.62:0/2909612123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c101de0 msgr2=0x7f391c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c101de0 0x7f391c198470 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f391400d950 tx=0x7f391400de10 comp rx=0 tx=0).stop 2024-09-11T23:49:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 -- 172.21.15.62:0/2909612123 shutdown_connections 2024-09-11T23:49:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f39040a2b80 0x7f39040a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391c106f10 0x7f391c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c102790 0x7f391c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 --2- 172.21.15.62:0/2909612123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f391c101de0 0x7f391c198470 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 -- 172.21.15.62:0/2909612123 >> 172.21.15.62:0/2909612123 conn(0x7f391c0fd8c0 msgr2=0x7f391c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 -- 172.21.15.62:0/2909612123 shutdown_connections 2024-09-11T23:49:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:55.466+0000 7f3920db4640 1 -- 172.21.15.62:0/2909612123 wait complete. 2024-09-11T23:49:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.62:6838/392450380,v1:172.21.15.62:6839/392450380] up:boot 2024-09-11T23:49:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: fsmap 10 up:standby 2024-09-11T23:49:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:49:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:55.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:55.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:55.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:55.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:55.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: Deploying daemon mds.a on smithi062 2024-09-11T23:49:55.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:55 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2909612123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:55.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.62:6838/392450380,v1:172.21.15.62:6839/392450380] up:boot 2024-09-11T23:49:55.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: fsmap 10 up:standby 2024-09-11T23:49:55.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:49:55.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:55.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:55.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:55.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:55.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:55.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:55.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: Deploying daemon mds.a on smithi062 2024-09-11T23:49:55.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:55 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2909612123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:56.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.62:6838/392450380,v1:172.21.15.62:6839/392450380] up:boot 2024-09-11T23:49:56.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: fsmap 10 up:standby 2024-09-11T23:49:56.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:49:56.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:56.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:56.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:56.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:56.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:56.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:56.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: Deploying daemon mds.a on smithi062 2024-09-11T23:49:56.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:55 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2909612123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:56 smithi062 ceph-mon[30958]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:57.038 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:56 smithi083 ceph-mon[34805]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:57.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:56 smithi120 ceph-mon[35182]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:57.716 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 11 23:49:57 smithi062 systemd[1]: Starting Ceph mds.a for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:49:58.028 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 11 23:49:57 smithi062 podman[72447]: 2024-09-11 23:49:57.64580103 +0000 UTC m=+0.016250014 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:49:58.029 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 11 23:49:57 smithi062 podman[72447]: 2024-09-11 23:49:57.76038645 +0000 UTC m=+0.130835422 container create 0ee0b356bad1fc23048d5968cffe6d72610bb3287acbb12b3cc3593b3890df84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:49:58.029 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 11 23:49:58 smithi062 podman[72447]: 2024-09-11 23:49:58.030044374 +0000 UTC m=+0.400493346 container init 0ee0b356bad1fc23048d5968cffe6d72610bb3287acbb12b3cc3593b3890df84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T23:49:58.465 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 11 23:49:58 smithi062 podman[72447]: 2024-09-11 23:49:58.033199594 +0000 UTC m=+0.403648566 container start 0ee0b356bad1fc23048d5968cffe6d72610bb3287acbb12b3cc3593b3890df84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:49:58.465 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 11 23:49:58 smithi062 ceph-mds[72467]: -- 172.21.15.62:0/332233075 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c1a5ca1860 con 0x55c1a5c7e800 2024-09-11T23:49:58.465 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 11 23:49:58 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-a[72463]: starting mds.a at 2024-09-11T23:49:58.465 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 11 23:49:58 smithi062 bash[72447]: 0ee0b356bad1fc23048d5968cffe6d72610bb3287acbb12b3cc3593b3890df84 2024-09-11T23:49:58.466 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 11 23:49:58 smithi062 systemd[1]: Started Ceph mds.a for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:49:58.678 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:58.850 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:58 smithi083 ceph-mon[34805]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:58.850 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:58 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:58.850 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:58 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:58.850 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:58 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:58.850 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:58 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:58.850 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:58 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:58.851 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:58 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:58.954 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:58 smithi062 ceph-mon[30958]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:58 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:58 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:58 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:58 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:58 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:58 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:59.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:58 smithi120 ceph-mon[35182]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:49:59.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:58 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:59.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:58 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:59.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:58 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:49:59.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:58 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:49:59.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:58 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:49:59.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:58 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.399+0000 7fed411e7640 1 -- 172.21.15.62:0/766888640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed3c101420 msgr2=0x7fed3c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.399+0000 7fed411e7640 1 --2- 172.21.15.62:0/766888640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed3c101420 0x7fed3c10d950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed3402a420 comp rx=0 tx=0).stop 2024-09-11T23:49:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.399+0000 7fed411e7640 1 -- 172.21.15.62:0/766888640 shutdown_connections 2024-09-11T23:49:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.399+0000 7fed411e7640 1 --2- 172.21.15.62:0/766888640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed3c101420 0x7fed3c10d950 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.399+0000 7fed411e7640 1 --2- 172.21.15.62:0/766888640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c100a80 0x7fed3c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.399+0000 7fed411e7640 1 --2- 172.21.15.62:0/766888640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c106a80 0x7fed3c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.399+0000 7fed411e7640 1 -- 172.21.15.62:0/766888640 >> 172.21.15.62:0/766888640 conn(0x7fed3c0fc7a0 msgr2=0x7fed3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.399+0000 7fed411e7640 1 -- 172.21.15.62:0/766888640 shutdown_connections 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.399+0000 7fed411e7640 1 -- 172.21.15.62:0/766888640 wait complete. 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed411e7640 1 Processor -- start 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed411e7640 1 -- start start 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed411e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c100a80 0x7fed3c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed411e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c101420 0x7fed3c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed411e7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed3c106a80 0x7fed3c1a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed411e7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c110e80 con 0x7fed3c106a80 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed411e7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c110d00 con 0x7fed3c101420 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed411e7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c111000 con 0x7fed3c100a80 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed3b59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed3c106a80 0x7fed3c1a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed3a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c101420 0x7fed3c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed3b59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed3c106a80 0x7fed3c1a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57896/0 (socket says 172.21.15.62:57896) 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.400+0000 7fed3b59e640 1 -- 172.21.15.62:0/3442958111 learned_addr learned my addr 172.21.15.62:0/3442958111 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed3ad9d640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c100a80 0x7fed3c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed3b59e640 1 -- 172.21.15.62:0/3442958111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c100a80 msgr2=0x7fed3c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed3b59e640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c100a80 0x7fed3c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed3b59e640 1 -- 172.21.15.62:0/3442958111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c101420 msgr2=0x7fed3c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed3b59e640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c101420 0x7fed3c1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed3b59e640 1 -- 172.21.15.62:0/3442958111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c1a4870 con 0x7fed3c106a80 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed3ad9d640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c100a80 0x7fed3c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed3b59e640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed3c106a80 0x7fed3c1a4330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fed34005ab0 tx=0x7fed340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed33fff640 1 -- 172.21.15.62:0/3442958111 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3403f030 con 0x7fed3c106a80 2024-09-11T23:49:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed411e7640 1 -- 172.21.15.62:0/3442958111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c1a4a40 con 0x7fed3c106a80 2024-09-11T23:49:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed33fff640 1 -- 172.21.15.62:0/3442958111 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed34003ce0 con 0x7fed3c106a80 2024-09-11T23:49:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed33fff640 1 -- 172.21.15.62:0/3442958111 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34035070 con 0x7fed3c106a80 2024-09-11T23:49:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.401+0000 7fed411e7640 1 -- 172.21.15.62:0/3442958111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c1a4e90 con 0x7fed3c106a80 2024-09-11T23:49:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.402+0000 7fed411e7640 1 -- 172.21.15.62:0/3442958111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3c100ee0 con 0x7fed3c106a80 2024-09-11T23:49:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.404+0000 7fed33fff640 1 -- 172.21.15.62:0/3442958111 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fed34042030 con 0x7fed3c106a80 2024-09-11T23:49:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.404+0000 7fed33fff640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fed280a2b80 0x7fed280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.405+0000 7fed3ad9d640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fed280a2b80 0x7fed280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.405+0000 7fed33fff640 1 -- 172.21.15.62:0/3442958111 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fed340ef020 con 0x7fed3c106a80 2024-09-11T23:49:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.405+0000 7fed3ad9d640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fed280a2b80 0x7fed280a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fed2c005d90 tx=0x7fed2c005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.409+0000 7fed33fff640 1 -- 172.21.15.62:0/3442958111 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed340bc420 con 0x7fed3c106a80 2024-09-11T23:49:59.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.619+0000 7fed411e7640 1 -- 172.21.15.62:0/3442958111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed3c108a80 con 0x7fed3c106a80 2024-09-11T23:49:59.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.620+0000 7fed33fff640 1 -- 172.21.15.62:0/3442958111 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7fed340bbd20 con 0x7fed3c106a80 2024-09-11T23:49:59.620 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:49:59.620 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":12,"btime":"2024-09-11T23:49:58:756939+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T23:49:59.621 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 12 2024-09-11T23:49:59.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 -- 172.21.15.62:0/3442958111 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fed280a2b80 msgr2=0x7fed280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:59.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fed280a2b80 0x7fed280a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fed2c005d90 tx=0x7fed2c005d20 comp rx=0 tx=0).stop 2024-09-11T23:49:59.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 -- 172.21.15.62:0/3442958111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed3c106a80 msgr2=0x7fed3c1a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:59.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed3c106a80 0x7fed3c1a4330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fed34005ab0 tx=0x7fed340042f0 comp rx=0 tx=0).stop 2024-09-11T23:49:59.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 -- 172.21.15.62:0/3442958111 shutdown_connections 2024-09-11T23:49:59.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fed280a2b80 0x7fed280a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:59.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed3c106a80 0x7fed3c1a4330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:59.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c101420 0x7fed3c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:59.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 --2- 172.21.15.62:0/3442958111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c100a80 0x7fed3c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:59.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 -- 172.21.15.62:0/3442958111 >> 172.21.15.62:0/3442958111 conn(0x7fed3c0fc7a0 msgr2=0x7fed3c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:59.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 -- 172.21.15.62:0/3442958111 shutdown_connections 2024-09-11T23:49:59.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:49:59.622+0000 7fed411e7640 1 -- 172.21.15.62:0/3442958111 wait complete. 2024-09-11T23:49:59.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:59 smithi083 ceph-mon[34805]: Deploying daemon mds.b on smithi083 2024-09-11T23:49:59.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:59 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.62:6840/4034635892,v1:172.21.15.62:6841/4034635892] up:boot 2024-09-11T23:49:59.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:59 smithi083 ceph-mon[34805]: fsmap 11 up:standby 2024-09-11T23:49:59.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:59 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:49:59.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:49:59 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3442958111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:50:00.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:59 smithi120 ceph-mon[35182]: Deploying daemon mds.b on smithi083 2024-09-11T23:50:00.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:59 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.62:6840/4034635892,v1:172.21.15.62:6841/4034635892] up:boot 2024-09-11T23:50:00.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:59 smithi120 ceph-mon[35182]: fsmap 11 up:standby 2024-09-11T23:50:00.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:59 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:50:00.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:49:59 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3442958111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:50:00.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:59 smithi062 ceph-mon[30958]: Deploying daemon mds.b on smithi083 2024-09-11T23:50:00.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:59 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.62:6840/4034635892,v1:172.21.15.62:6841/4034635892] up:boot 2024-09-11T23:50:00.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:59 smithi062 ceph-mon[30958]: fsmap 11 up:standby 2024-09-11T23:50:00.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:59 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:50:00.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:49:59 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3442958111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:50:00.417 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:00 smithi083 systemd[1]: Starting Ceph mds.b for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-11T23:50:00.685 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:00 smithi083 podman[64387]: 2024-09-11 23:50:00.326993858 +0000 UTC m=+0.016252202 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:50:00.685 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:00 smithi083 podman[64387]: 2024-09-11 23:50:00.448306071 +0000 UTC m=+0.137564404 container create 0c5fa4907a3f4af0398f02771cd99af69238a94e0c05ba523743e93646ae0c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:50:00.685 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:00 smithi083 podman[64387]: 2024-09-11 23:50:00.659609318 +0000 UTC m=+0.348867650 container init 0c5fa4907a3f4af0398f02771cd99af69238a94e0c05ba523743e93646ae0c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T23:50:00.685 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:00 smithi083 podman[64387]: 2024-09-11 23:50:00.662716005 +0000 UTC m=+0.351974338 container start 0c5fa4907a3f4af0398f02771cd99af69238a94e0c05ba523743e93646ae0c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T23:50:01.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:00 smithi120 ceph-mon[35182]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:01.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:00 smithi120 ceph-mon[35182]: overall HEALTH_OK 2024-09-11T23:50:01.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:00 smithi083 ceph-mon[34805]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:01.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:00 smithi083 ceph-mon[34805]: overall HEALTH_OK 2024-09-11T23:50:01.167 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:00 smithi083 ceph-mds[64406]: -- 172.21.15.83:0/871386526 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5578bcbdd860 con 0x5578bcbbb400 2024-09-11T23:50:01.167 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b[64402]: starting mds.b at 2024-09-11T23:50:01.167 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:00 smithi083 bash[64387]: 0c5fa4907a3f4af0398f02771cd99af69238a94e0c05ba523743e93646ae0c18 2024-09-11T23:50:01.167 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:00 smithi083 systemd[1]: Started Ceph mds.b for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-11T23:50:01.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:00 smithi062 ceph-mon[30958]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:01.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:00 smithi062 ceph-mon[30958]: overall HEALTH_OK 2024-09-11T23:50:01.801 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:01 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] up:boot 2024-09-11T23:50:01.801 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:01 smithi062 ceph-mon[30958]: fsmap 12 up:standby 2024-09-11T23:50:01.801 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:50:01.801 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.801 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.801 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.801 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.801 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.801 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:01 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:50:01.804 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:01 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] up:boot 2024-09-11T23:50:01.804 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:01 smithi083 ceph-mon[34805]: fsmap 12 up:standby 2024-09-11T23:50:01.804 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:50:01.804 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.805 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.805 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.805 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.805 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.805 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:01 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:50:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:01 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] up:boot 2024-09-11T23:50:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:01 smithi120 ceph-mon[35182]: fsmap 12 up:standby 2024-09-11T23:50:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:50:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:01 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:50:02.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:50:02.583 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:02.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:02 smithi062 ceph-mon[30958]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:02.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:50:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:50:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:02 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:02 smithi120 ceph-mon[35182]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:50:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:50:03.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:02 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.106+0000 7f61ef04a640 1 -- 172.21.15.62:0/1917366801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61e8100e20 msgr2=0x7f61e8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.106+0000 7f61ef04a640 1 --2- 172.21.15.62:0/1917366801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61e8100e20 0x7f61e8101200 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009900 tx=0x7f61dc02a480 comp rx=0 tx=0).stop 2024-09-11T23:50:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.106+0000 7f61ef04a640 1 -- 172.21.15.62:0/1917366801 shutdown_connections 2024-09-11T23:50:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.106+0000 7f61ef04a640 1 --2- 172.21.15.62:0/1917366801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e8102ef0 0x7f61e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.106+0000 7f61ef04a640 1 --2- 172.21.15.62:0/1917366801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61e8102570 0x7f61e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.106+0000 7f61ef04a640 1 --2- 172.21.15.62:0/1917366801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61e8100e20 0x7f61e8101200 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.106+0000 7f61ef04a640 1 -- 172.21.15.62:0/1917366801 >> 172.21.15.62:0/1917366801 conn(0x7f61e80fc7d0 msgr2=0x7f61e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.106+0000 7f61ef04a640 1 -- 172.21.15.62:0/1917366801 shutdown_connections 2024-09-11T23:50:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.106+0000 7f61ef04a640 1 -- 172.21.15.62:0/1917366801 wait complete. 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ef04a640 1 Processor -- start 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ef04a640 1 -- start start 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ef04a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e8100e20 0x7f61e81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ef04a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61e8102570 0x7f61e8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ef04a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61e8102ef0 0x7f61e819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ef04a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8110f80 con 0x7f61e8102ef0 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ef04a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8110e00 con 0x7f61e8102570 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ef04a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8111100 con 0x7f61e8100e20 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ed5e7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61e8102ef0 0x7f61e819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ed5e7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61e8102ef0 0x7f61e819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36038/0 (socket says 172.21.15.62:36038) 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61e8102570 0x7f61e8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ed5e7640 1 -- 172.21.15.62:0/28291599 learned_addr learned my addr 172.21.15.62:0/28291599 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.107+0000 7f61ecde6640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e8100e20 0x7f61e81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61ed5e7640 1 -- 172.21.15.62:0/28291599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e8100e20 msgr2=0x7f61e81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61ed5e7640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e8100e20 0x7f61e81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61ed5e7640 1 -- 172.21.15.62:0/28291599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61e8102570 msgr2=0x7f61e8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61ed5e7640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61e8102570 0x7f61e8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61ed5e7640 1 -- 172.21.15.62:0/28291599 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e819d4a0 con 0x7f61e8102ef0 2024-09-11T23:50:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61e7fff640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61e8102570 0x7f61e8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:50:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61ed5e7640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61e8102ef0 0x7f61e819cdc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009fd0 tx=0x7f61e000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61e5ffb640 1 -- 172.21.15.62:0/28291599 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0017070 con 0x7f61e8102ef0 2024-09-11T23:50:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61e5ffb640 1 -- 172.21.15.62:0/28291599 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e0003e50 con 0x7f61e8102ef0 2024-09-11T23:50:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61ef04a640 1 -- 172.21.15.62:0/28291599 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e819d790 con 0x7f61e8102ef0 2024-09-11T23:50:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61e5ffb640 1 -- 172.21.15.62:0/28291599 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e00106c0 con 0x7f61e8102ef0 2024-09-11T23:50:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.108+0000 7f61ef04a640 1 -- 172.21.15.62:0/28291599 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e81a5070 con 0x7f61e8102ef0 2024-09-11T23:50:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.109+0000 7f61ef04a640 1 -- 172.21.15.62:0/28291599 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e8101200 con 0x7f61e8102ef0 2024-09-11T23:50:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.110+0000 7f61e5ffb640 1 -- 172.21.15.62:0/28291599 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f61e0003970 con 0x7f61e8102ef0 2024-09-11T23:50:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.111+0000 7f61e5ffb640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61d80a2c00 0x7f61d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.111+0000 7f61ecde6640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61d80a2c00 0x7f61d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.111+0000 7f61e5ffb640 1 -- 172.21.15.62:0/28291599 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f61e00ce7e0 con 0x7f61e8102ef0 2024-09-11T23:50:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.112+0000 7f61ecde6640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61d80a2c00 0x7f61d80a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009230 tx=0x7f61dc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.119+0000 7f61e5ffb640 1 -- 172.21.15.62:0/28291599 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61e009b610 con 0x7f61e8102ef0 2024-09-11T23:50:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:02 smithi083 ceph-mon[34805]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:50:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:50:03.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:02 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:03.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.358+0000 7f61ef04a640 1 -- 172.21.15.62:0/28291599 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61e81a5240 con 0x7f61e8102ef0 2024-09-11T23:50:03.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.359+0000 7f61e5ffb640 1 -- 172.21.15.62:0/28291599 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7f61e009f910 con 0x7f61e8102ef0 2024-09-11T23:50:03.359 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:03.359 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":13,"btime":"2024-09-11T23:50:00:766945+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-11T23:50:03.360 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 13 2024-09-11T23:50:03.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 -- 172.21.15.62:0/28291599 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61d80a2c00 msgr2=0x7f61d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:03.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61d80a2c00 0x7f61d80a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009230 tx=0x7f61dc0058d0 comp rx=0 tx=0).stop 2024-09-11T23:50:03.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 -- 172.21.15.62:0/28291599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61e8102ef0 msgr2=0x7f61e819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:03.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61e8102ef0 0x7f61e819cdc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009fd0 tx=0x7f61e000b260 comp rx=0 tx=0).stop 2024-09-11T23:50:03.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 -- 172.21.15.62:0/28291599 shutdown_connections 2024-09-11T23:50:03.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61d80a2c00 0x7f61d80a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:03.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61e8102ef0 0x7f61e819cdc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:03.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61e8102570 0x7f61e8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:03.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 --2- 172.21.15.62:0/28291599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e8100e20 0x7f61e81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:03.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 -- 172.21.15.62:0/28291599 >> 172.21.15.62:0/28291599 conn(0x7f61e80fc7d0 msgr2=0x7f61e806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:03.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 -- 172.21.15.62:0/28291599 shutdown_connections 2024-09-11T23:50:03.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:03.361+0000 7f61ef04a640 1 -- 172.21.15.62:0/28291599 wait complete. 2024-09-11T23:50:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:03 smithi120 ceph-mon[35182]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:03 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:50:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:03 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/28291599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:50:04.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:04.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:03 smithi083 ceph-mon[34805]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:04.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:03 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:50:04.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:03 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/28291599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:50:04.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:04.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:03 smithi062 ceph-mon[30958]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:04.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:03 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:50:04.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:03 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/28291599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:50:04.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T23:50:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.386+0000 7f523b28f640 1 -- 172.21.15.62:0/2531936157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5234101de0 msgr2=0x7f52341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.386+0000 7f523b28f640 1 --2- 172.21.15.62:0/2531936157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5234101de0 0x7f52341021c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5230009900 tx=0x7f523002a410 comp rx=0 tx=0).stop 2024-09-11T23:50:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.386+0000 7f523b28f640 1 -- 172.21.15.62:0/2531936157 shutdown_connections 2024-09-11T23:50:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.386+0000 7f523b28f640 1 --2- 172.21.15.62:0/2531936157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5234106f10 0x7f523410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.386+0000 7f523b28f640 1 --2- 172.21.15.62:0/2531936157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5234102790 0x7f52341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.386+0000 7f523b28f640 1 --2- 172.21.15.62:0/2531936157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5234101de0 0x7f52341021c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.386+0000 7f523b28f640 1 -- 172.21.15.62:0/2531936157 >> 172.21.15.62:0/2531936157 conn(0x7f52340fd8c0 msgr2=0x7f52340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.386+0000 7f523b28f640 1 -- 172.21.15.62:0/2531936157 shutdown_connections 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.386+0000 7f523b28f640 1 -- 172.21.15.62:0/2531936157 wait complete. 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523b28f640 1 Processor -- start 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523b28f640 1 -- start start 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523b28f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5234101de0 0x7f5234198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523b28f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5234102790 0x7f5234198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523b28f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5234106f10 0x7f523419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523b28f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5234111fe0 con 0x7f5234102790 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523b28f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5234111e60 con 0x7f5234101de0 2024-09-11T23:50:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523b28f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5234112160 con 0x7f5234106f10 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523882a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5234102790 0x7f5234198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523902b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5234101de0 0x7f5234198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.387+0000 7f523882a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5234102790 0x7f5234198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36060/0 (socket says 172.21.15.62:36060) 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.388+0000 7f523882a640 1 -- 172.21.15.62:0/151553611 learned_addr learned my addr 172.21.15.62:0/151553611 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.388+0000 7f523982c640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5234106f10 0x7f523419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.388+0000 7f523882a640 1 -- 172.21.15.62:0/151553611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5234106f10 msgr2=0x7f523419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.388+0000 7f523882a640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5234106f10 0x7f523419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.388+0000 7f523882a640 1 -- 172.21.15.62:0/151553611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5234101de0 msgr2=0x7f5234198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.388+0000 7f523882a640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5234101de0 0x7f5234198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.388+0000 7f523882a640 1 -- 172.21.15.62:0/151553611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523419d3d0 con 0x7f5234102790 2024-09-11T23:50:05.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.388+0000 7f523982c640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5234106f10 0x7f523419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.388+0000 7f523902b640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5234101de0 0x7f5234198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.388+0000 7f523882a640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5234102790 0x7f5234198960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f522400b4a0 tx=0x7f522400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.391+0000 7f522a7fc640 1 -- 172.21.15.62:0/151553611 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224014070 con 0x7f5234102790 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.391+0000 7f523b28f640 1 -- 172.21.15.62:0/151553611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523419d6c0 con 0x7f5234102790 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.391+0000 7f522a7fc640 1 -- 172.21.15.62:0/151553611 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5224010af0 con 0x7f5234102790 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.391+0000 7f522a7fc640 1 -- 172.21.15.62:0/151553611 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224017d10 con 0x7f5234102790 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.392+0000 7f523b28f640 1 -- 172.21.15.62:0/151553611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52341a4fa0 con 0x7f5234102790 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.393+0000 7f523b28f640 1 -- 172.21.15.62:0/151553611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52341021c0 con 0x7f5234102790 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.394+0000 7f522a7fc640 1 -- 172.21.15.62:0/151553611 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5224017070 con 0x7f5234102790 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.395+0000 7f522a7fc640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f52200a2b80 0x7f52200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:05.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.395+0000 7f523902b640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f52200a2b80 0x7f52200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:05.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.395+0000 7f522a7fc640 1 -- 172.21.15.62:0/151553611 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f52240cdf70 con 0x7f5234102790 2024-09-11T23:50:05.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.395+0000 7f523902b640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f52200a2b80 0x7f52200a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5230000c00 tx=0x7f5230005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:05.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.399+0000 7f522a7fc640 1 -- 172.21.15.62:0/151553611 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f522409b370 con 0x7f5234102790 2024-09-11T23:50:05.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.585+0000 7f523b28f640 1 -- 172.21.15.62:0/151553611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5234109b60 con 0x7f5234102790 2024-09-11T23:50:05.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:05 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:05 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:05 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:05 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.585+0000 7f522a7fc640 1 -- 172.21.15.62:0/151553611 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f522409ac70 con 0x7f5234102790 2024-09-11T23:50:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:05.586 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-09-11T23:50:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-11T23:50:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-09-11T23:50:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-09-11T23:50:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:50:05.586 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 -- 172.21.15.62:0/151553611 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f52200a2b80 msgr2=0x7f52200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f52200a2b80 0x7f52200a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5230000c00 tx=0x7f5230005b50 comp rx=0 tx=0).stop 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 -- 172.21.15.62:0/151553611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5234102790 msgr2=0x7f5234198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5234102790 0x7f5234198960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f522400b4a0 tx=0x7f522400b960 comp rx=0 tx=0).stop 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 -- 172.21.15.62:0/151553611 shutdown_connections 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f52200a2b80 0x7f52200a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5234106f10 0x7f523419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5234102790 0x7f5234198960 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 --2- 172.21.15.62:0/151553611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5234101de0 0x7f5234198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 -- 172.21.15.62:0/151553611 >> 172.21.15.62:0/151553611 conn(0x7f52340fd8c0 msgr2=0x7f52340fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 -- 172.21.15.62:0/151553611 shutdown_connections 2024-09-11T23:50:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:05.588+0000 7f523b28f640 1 -- 172.21.15.62:0/151553611 wait complete. 2024-09-11T23:50:05.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:05 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:05 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:05 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:05 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:05 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:05 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:05 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:05.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:05 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:06.327 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T23:50:06.327 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T23:50:06.583 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:06.705 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:06 smithi062 ceph-mon[30958]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:06.705 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:06.705 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:06.705 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:50:06.705 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:50:06.705 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:06.705 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:06 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/151553611' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:50:06.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:06 smithi120 ceph-mon[35182]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:06.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:06.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:06.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:50:06.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:50:06.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:06.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:06 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/151553611' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:50:06.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:06 smithi083 ceph-mon[34805]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:06.957 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:06.958 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:06.958 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:50:06.958 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:50:06.958 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:50:06.958 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:06 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/151553611' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:50:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 -- 172.21.15.62:0/3601330474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce4108300 msgr2=0x7f5ce410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 --2- 172.21.15.62:0/3601330474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce4108300 0x7f5ce410ec70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009cd0 tx=0x7f5cd402a8d0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 -- 172.21.15.62:0/3601330474 shutdown_connections 2024-09-11T23:50:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 --2- 172.21.15.62:0/3601330474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce4108300 0x7f5ce410ec70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 --2- 172.21.15.62:0/3601330474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ce4103780 0x7f5ce4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 --2- 172.21.15.62:0/3601330474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce4102dd0 0x7f5ce41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 -- 172.21.15.62:0/3601330474 >> 172.21.15.62:0/3601330474 conn(0x7f5ce40fc7d0 msgr2=0x7f5ce40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 -- 172.21.15.62:0/3601330474 shutdown_connections 2024-09-11T23:50:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 -- 172.21.15.62:0/3601330474 wait complete. 2024-09-11T23:50:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 Processor -- start 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.081+0000 7f5ceb784640 1 -- start start 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ceb784640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce4102dd0 0x7f5ce41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ceb784640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce4103780 0x7f5ce4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ceb784640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ce4108300 0x7f5ce419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ceb784640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce41120d0 con 0x7f5ce4102dd0 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ceb784640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4111f50 con 0x7f5ce4108300 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ceb784640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4112250 con 0x7f5ce4103780 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce9520640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce4102dd0 0x7f5ce41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce9520640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce4102dd0 0x7f5ce41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36082/0 (socket says 172.21.15.62:36082) 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce9520640 1 -- 172.21.15.62:0/150963532 learned_addr learned my addr 172.21.15.62:0/150963532 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce8d1f640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce4103780 0x7f5ce4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce9d21640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ce4108300 0x7f5ce419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce9520640 1 -- 172.21.15.62:0/150963532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce4103780 msgr2=0x7f5ce4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce9520640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce4103780 0x7f5ce4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce9520640 1 -- 172.21.15.62:0/150963532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ce4108300 msgr2=0x7f5ce419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce9520640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ce4108300 0x7f5ce419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce9520640 1 -- 172.21.15.62:0/150963532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce419d480 con 0x7f5ce4102dd0 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce9520640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce4102dd0 0x7f5ce41984d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c920 tx=0x7f5cdc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce27fc640 1 -- 172.21.15.62:0/150963532 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc007c70 con 0x7f5ce4102dd0 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ceb784640 1 -- 172.21.15.62:0/150963532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce419d770 con 0x7f5ce4102dd0 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce27fc640 1 -- 172.21.15.62:0/150963532 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cdc007e10 con 0x7f5ce4102dd0 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.082+0000 7f5ce27fc640 1 -- 172.21.15.62:0/150963532 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc018890 con 0x7f5ce4102dd0 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.083+0000 7f5ceb784640 1 -- 172.21.15.62:0/150963532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce41a5050 con 0x7f5ce4102dd0 2024-09-11T23:50:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.083+0000 7f5ceb784640 1 -- 172.21.15.62:0/150963532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce41031b0 con 0x7f5ce4102dd0 2024-09-11T23:50:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.085+0000 7f5ce27fc640 1 -- 172.21.15.62:0/150963532 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5cdc018a70 con 0x7f5ce4102dd0 2024-09-11T23:50:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.085+0000 7f5ce27fc640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5ccc0a2b80 0x7f5ccc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.085+0000 7f5ce8d1f640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5ccc0a2b80 0x7f5ccc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.085+0000 7f5ce27fc640 1 -- 172.21.15.62:0/150963532 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f5cdc0ceda0 con 0x7f5ce4102dd0 2024-09-11T23:50:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.086+0000 7f5ce8d1f640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5ccc0a2b80 0x7f5ccc0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005d90 tx=0x7f5cd8005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.090+0000 7f5ce27fc640 1 -- 172.21.15.62:0/150963532 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cdc014030 con 0x7f5ce4102dd0 2024-09-11T23:50:07.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.264+0000 7f5ceb784640 1 -- 172.21.15.62:0/150963532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5ce41a5250 con 0x7f5ce4102dd0 2024-09-11T23:50:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.749+0000 7f5ce27fc640 1 -- 172.21.15.62:0/150963532 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7f5cdc00de20 con 0x7f5ce4102dd0 2024-09-11T23:50:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-09-11T23:50:07.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 -- 172.21.15.62:0/150963532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5ccc0a2b80 msgr2=0x7f5ccc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:07.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5ccc0a2b80 0x7f5ccc0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005d90 tx=0x7f5cd8005d20 comp rx=0 tx=0).stop 2024-09-11T23:50:07.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 -- 172.21.15.62:0/150963532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce4102dd0 msgr2=0x7f5ce41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:07.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce4102dd0 0x7f5ce41984d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c920 tx=0x7f5cdc00cde0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 -- 172.21.15.62:0/150963532 shutdown_connections 2024-09-11T23:50:07.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5ccc0a2b80 0x7f5ccc0a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ce4108300 0x7f5ce419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce4103780 0x7f5ce4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 --2- 172.21.15.62:0/150963532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce4102dd0 0x7f5ce41984d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:07.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 -- 172.21.15.62:0/150963532 >> 172.21.15.62:0/150963532 conn(0x7f5ce40fc7d0 msgr2=0x7f5ce40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:07.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 -- 172.21.15.62:0/150963532 shutdown_connections 2024-09-11T23:50:07.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:07.751+0000 7f5ceb784640 1 -- 172.21.15.62:0/150963532 wait complete. 2024-09-11T23:50:08.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:07 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/150963532' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:50:08.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:07 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/150963532' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:50:08.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:07 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/150963532' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:50:08.431 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T23:50:08.688 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:09.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:08 smithi062 ceph-mon[30958]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:09.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:08 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/150963532' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:50:09.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:08 smithi062 ceph-mon[30958]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:50:09.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:08 smithi062 ceph-mon[30958]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:50:09.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:08 smithi120 ceph-mon[35182]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:09.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:08 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/150963532' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:50:09.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:08 smithi120 ceph-mon[35182]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:50:09.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:08 smithi120 ceph-mon[35182]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:50:09.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:08 smithi083 ceph-mon[34805]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:09.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:08 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/150963532' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:50:09.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:08 smithi083 ceph-mon[34805]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:50:09.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:08 smithi083 ceph-mon[34805]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:50:09.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 -- 172.21.15.62:0/2032225091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcab0103780 msgr2=0x7fcab0107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:09.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 --2- 172.21.15.62:0/2032225091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcab0103780 0x7fcab0107bc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fca98009930 tx=0x7fca9802a420 comp rx=0 tx=0).stop 2024-09-11T23:50:09.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 -- 172.21.15.62:0/2032225091 shutdown_connections 2024-09-11T23:50:09.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 --2- 172.21.15.62:0/2032225091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcab0108300 0x7fcab010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 --2- 172.21.15.62:0/2032225091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcab0103780 0x7fcab0107bc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 --2- 172.21.15.62:0/2032225091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab0102dd0 0x7fcab01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 -- 172.21.15.62:0/2032225091 >> 172.21.15.62:0/2032225091 conn(0x7fcab00fc7d0 msgr2=0x7fcab00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 -- 172.21.15.62:0/2032225091 shutdown_connections 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 -- 172.21.15.62:0/2032225091 wait complete. 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 Processor -- start 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.193+0000 7fcab6b18640 1 -- start start 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab6b18640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab0102dd0 0x7fcab01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab6b18640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcab0103780 0x7fcab0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab6b18640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcab0108300 0x7fcab019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:09.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab6b18640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0112050 con 0x7fcab0108300 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab6b18640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0111ed0 con 0x7fcab0102dd0 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab6b18640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01121d0 con 0x7fcab0103780 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab50b5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcab0108300 0x7fcab019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab50b5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcab0108300 0x7fcab019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36090/0 (socket says 172.21.15.62:36090) 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab48b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab0102dd0 0x7fcab01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab50b5640 1 -- 172.21.15.62:0/791666554 learned_addr learned my addr 172.21.15.62:0/791666554 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcaaffff640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcab0103780 0x7fcab0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab50b5640 1 -- 172.21.15.62:0/791666554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcab0103780 msgr2=0x7fcab0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab50b5640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcab0103780 0x7fcab0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab50b5640 1 -- 172.21.15.62:0/791666554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab0102dd0 msgr2=0x7fcab01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab50b5640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab0102dd0 0x7fcab01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab50b5640 1 -- 172.21.15.62:0/791666554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab019d480 con 0x7fcab0108300 2024-09-11T23:50:09.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcab50b5640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcab0108300 0x7fcab019cda0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000cce0 tx=0x7fcaa000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:09.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.194+0000 7fcaadffb640 1 -- 172.21.15.62:0/791666554 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000f450 con 0x7fcab0108300 2024-09-11T23:50:09.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.195+0000 7fcaadffb640 1 -- 172.21.15.62:0/791666554 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa000fad0 con 0x7fcab0108300 2024-09-11T23:50:09.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.195+0000 7fcab6b18640 1 -- 172.21.15.62:0/791666554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab019d770 con 0x7fcab0108300 2024-09-11T23:50:09.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.195+0000 7fcaadffb640 1 -- 172.21.15.62:0/791666554 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0018b60 con 0x7fcab0108300 2024-09-11T23:50:09.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.195+0000 7fcab6b18640 1 -- 172.21.15.62:0/791666554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab01a5050 con 0x7fcab0108300 2024-09-11T23:50:09.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.196+0000 7fcab6b18640 1 -- 172.21.15.62:0/791666554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab01031b0 con 0x7fcab0108300 2024-09-11T23:50:09.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.197+0000 7fcaadffb640 1 -- 172.21.15.62:0/791666554 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcaa000fc80 con 0x7fcab0108300 2024-09-11T23:50:09.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.198+0000 7fcaadffb640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fca9c0a2c00 0x7fca9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:09.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.198+0000 7fcaadffb640 1 -- 172.21.15.62:0/791666554 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7fcaa00cf170 con 0x7fcab0108300 2024-09-11T23:50:09.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.198+0000 7fcab48b4640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fca9c0a2c00 0x7fca9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:09.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.198+0000 7fcab48b4640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fca9c0a2c00 0x7fca9c0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800abe0 tx=0x7fcaa800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:09.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.202+0000 7fcaadffb640 1 -- 172.21.15.62:0/791666554 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcaa0014030 con 0x7fcab0108300 2024-09-11T23:50:09.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.380+0000 7fcab6b18640 1 -- 172.21.15.62:0/791666554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcab010a290 con 0x7fcab0108300 2024-09-11T23:50:09.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.820+0000 7fcaadffb640 1 -- 172.21.15.62:0/791666554 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7fcaa009c400 con 0x7fcab0108300 2024-09-11T23:50:09.820 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-09-11T23:50:09.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 -- 172.21.15.62:0/791666554 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fca9c0a2c00 msgr2=0x7fca9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:09.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fca9c0a2c00 0x7fca9c0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800abe0 tx=0x7fcaa800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:50:09.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 -- 172.21.15.62:0/791666554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcab0108300 msgr2=0x7fcab019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:09.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcab0108300 0x7fcab019cda0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000cce0 tx=0x7fcaa000a620 comp rx=0 tx=0).stop 2024-09-11T23:50:09.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 -- 172.21.15.62:0/791666554 shutdown_connections 2024-09-11T23:50:09.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fca9c0a2c00 0x7fca9c0a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:09.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcab0108300 0x7fcab019cda0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:09.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcab0103780 0x7fcab0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:09.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 --2- 172.21.15.62:0/791666554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab0102dd0 0x7fcab01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:09.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 -- 172.21.15.62:0/791666554 >> 172.21.15.62:0/791666554 conn(0x7fcab00fc7d0 msgr2=0x7fcab01063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:09.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 -- 172.21.15.62:0/791666554 shutdown_connections 2024-09-11T23:50:09.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:09.822+0000 7fcab6b18640 1 -- 172.21.15.62:0/791666554 wait complete. 2024-09-11T23:50:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:09 smithi120 ceph-mon[35182]: pgmap v276: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:09 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/791666554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:50:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:09 smithi120 ceph-mon[35182]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:50:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:09 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/791666554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:50:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:09 smithi120 ceph-mon[35182]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:50:10.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:09 smithi062 ceph-mon[30958]: pgmap v276: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:10.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:09 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/791666554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:50:10.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:09 smithi062 ceph-mon[30958]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:50:10.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:09 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/791666554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:50:10.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:09 smithi062 ceph-mon[30958]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:50:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:09 smithi083 ceph-mon[34805]: pgmap v276: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:09 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/791666554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:50:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:09 smithi083 ceph-mon[34805]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:50:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:09 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/791666554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:50:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:09 smithi083 ceph-mon[34805]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:50:10.464 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T23:50:10.721 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:11.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.182+0000 7fba0fd3d640 1 -- 172.21.15.62:0/1500220020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba08103660 msgr2=0x7fba0810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:11.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.182+0000 7fba0fd3d640 1 --2- 172.21.15.62:0/1500220020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba08103660 0x7fba0810bb50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009900 tx=0x7fb9fc02a480 comp rx=0 tx=0).stop 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.182+0000 7fba0fd3d640 1 -- 172.21.15.62:0/1500220020 shutdown_connections 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.182+0000 7fba0fd3d640 1 --2- 172.21.15.62:0/1500220020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba0810c170 0x7fba0810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.182+0000 7fba0fd3d640 1 --2- 172.21.15.62:0/1500220020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba08103660 0x7fba0810bb50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.182+0000 7fba0fd3d640 1 --2- 172.21.15.62:0/1500220020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba08102d40 0x7fba08103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.182+0000 7fba0fd3d640 1 -- 172.21.15.62:0/1500220020 >> 172.21.15.62:0/1500220020 conn(0x7fba080fc7a0 msgr2=0x7fba080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.182+0000 7fba0fd3d640 1 -- 172.21.15.62:0/1500220020 shutdown_connections 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.182+0000 7fba0fd3d640 1 -- 172.21.15.62:0/1500220020 wait complete. 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0fd3d640 1 Processor -- start 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0fd3d640 1 -- start start 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0fd3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba08102d40 0x7fba08198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0fd3d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba08103660 0x7fba08198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0fd3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba0810c170 0x7fba0819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0fd3d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba08111fd0 con 0x7fba08103660 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0fd3d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba08111e50 con 0x7fba08102d40 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0fd3d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fba08112150 con 0x7fba0810c170 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0dad9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba08102d40 0x7fba08198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0d2d8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba08103660 0x7fba08198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0dad9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba08102d40 0x7fba08198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:52764/0 (socket says 172.21.15.62:52764) 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0dad9640 1 -- 172.21.15.62:0/2012309985 learned_addr learned my addr 172.21.15.62:0/2012309985 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0d2d8640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba08103660 0x7fba08198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56406/0 (socket says 172.21.15.62:56406) 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.183+0000 7fba0e2da640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba0810c170 0x7fba0819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fba0dad9640 1 -- 172.21.15.62:0/2012309985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba0810c170 msgr2=0x7fba0819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fba0dad9640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba0810c170 0x7fba0819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fba0dad9640 1 -- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba08103660 msgr2=0x7fba08198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fba0dad9640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba08103660 0x7fba08198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fba0dad9640 1 -- 172.21.15.62:0/2012309985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0819d510 con 0x7fba08102d40 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fba0d2d8640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba08103660 0x7fba08198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fba0dad9640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba08102d40 0x7fba08198560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fba0400b4a0 tx=0x7fba0400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fb9faffd640 1 -- 172.21.15.62:0/2012309985 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04014070 con 0x7fba08102d40 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fba0fd3d640 1 -- 172.21.15.62:0/2012309985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0819d800 con 0x7fba08102d40 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fb9faffd640 1 -- 172.21.15.62:0/2012309985 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba04010af0 con 0x7fba08102d40 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fb9faffd640 1 -- 172.21.15.62:0/2012309985 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04017d10 con 0x7fba08102d40 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.184+0000 7fba0fd3d640 1 -- 172.21.15.62:0/2012309985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba081a50e0 con 0x7fba08102d40 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.185+0000 7fba0fd3d640 1 -- 172.21.15.62:0/2012309985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba08103120 con 0x7fba08102d40 2024-09-11T23:50:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.186+0000 7fb9faffd640 1 -- 172.21.15.62:0/2012309985 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fba04017070 con 0x7fba08102d40 2024-09-11T23:50:11.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.187+0000 7fb9faffd640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb9f40a2b80 0x7fb9f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:11.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.187+0000 7fba0d2d8640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb9f40a2b80 0x7fb9f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:11.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.187+0000 7fb9faffd640 1 -- 172.21.15.62:0/2012309985 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7fba040ce1a0 con 0x7fba08102d40 2024-09-11T23:50:11.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.187+0000 7fba0d2d8640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb9f40a2b80 0x7fb9f40a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc000c00 tx=0x7fb9fc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:11.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.190+0000 7fb9faffd640 1 -- 172.21.15.62:0/2012309985 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba0409ac40 con 0x7fba08102d40 2024-09-11T23:50:11.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.404+0000 7fba0fd3d640 1 -- 172.21.15.62:0/2012309985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fba081998c0 con 0x7fba08102d40 2024-09-11T23:50:11.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.871+0000 7fb9faffd640 1 -- 172.21.15.62:0/2012309985 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fba081998c0 con 0x7fba08102d40 2024-09-11T23:50:11.871 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T23:50:11.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 -- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb9f40a2b80 msgr2=0x7fb9f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:11.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb9f40a2b80 0x7fb9f40a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc000c00 tx=0x7fb9fc0058d0 comp rx=0 tx=0).stop 2024-09-11T23:50:11.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 -- 172.21.15.62:0/2012309985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba08102d40 msgr2=0x7fba08198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:11.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba08102d40 0x7fba08198560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fba0400b4a0 tx=0x7fba0400b960 comp rx=0 tx=0).stop 2024-09-11T23:50:11.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 -- 172.21.15.62:0/2012309985 shutdown_connections 2024-09-11T23:50:11.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb9f40a2b80 0x7fb9f40a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:11.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba0810c170 0x7fba0819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:11.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba08103660 0x7fba08198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:11.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 --2- 172.21.15.62:0/2012309985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba08102d40 0x7fba08198560 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:11.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 -- 172.21.15.62:0/2012309985 >> 172.21.15.62:0/2012309985 conn(0x7fba080fc7a0 msgr2=0x7fba080fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:11.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 -- 172.21.15.62:0/2012309985 shutdown_connections 2024-09-11T23:50:11.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:11.874+0000 7fba0fd3d640 1 -- 172.21.15.62:0/2012309985 wait complete. 2024-09-11T23:50:12.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:11 smithi120 ceph-mon[35182]: pgmap v278: 97 pgs: 19 creating+peering, 66 unknown, 12 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:12.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:11 smithi120 ceph-mon[35182]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:50:12.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:11 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2012309985' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:50:12.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:11 smithi120 ceph-mon[35182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:50:12.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:11 smithi120 ceph-mon[35182]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T23:50:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:11 smithi083 ceph-mon[34805]: pgmap v278: 97 pgs: 19 creating+peering, 66 unknown, 12 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:11 smithi083 ceph-mon[34805]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:50:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:11 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2012309985' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:50:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:11 smithi083 ceph-mon[34805]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:50:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:11 smithi083 ceph-mon[34805]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T23:50:12.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:11 smithi062 ceph-mon[30958]: pgmap v278: 97 pgs: 19 creating+peering, 66 unknown, 12 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:12.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:11 smithi062 ceph-mon[30958]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:50:12.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:11 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2012309985' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:50:12.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:11 smithi062 ceph-mon[30958]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:50:12.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:11 smithi062 ceph-mon[30958]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T23:50:12.536 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:50:12.798 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:13.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:12 smithi120 ceph-mon[35182]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:50:13.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:12 smithi120 ceph-mon[35182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:50:13.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:12 smithi120 ceph-mon[35182]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T23:50:13.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:12 smithi120 ceph-mon[35182]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T23:50:13.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:12 smithi062 ceph-mon[30958]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:50:13.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:12 smithi062 ceph-mon[30958]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:50:13.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:12 smithi062 ceph-mon[30958]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T23:50:13.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:12 smithi062 ceph-mon[30958]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T23:50:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:12 smithi083 ceph-mon[34805]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:50:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:12 smithi083 ceph-mon[34805]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:50:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:12 smithi083 ceph-mon[34805]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T23:50:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:12 smithi083 ceph-mon[34805]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T23:50:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.212+0000 7fcb0801a640 1 -- 172.21.15.62:0/1237341860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb00100b30 msgr2=0x7fcb00100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.212+0000 7fcb0801a640 1 --2- 172.21.15.62:0/1237341860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb00100b30 0x7fcb00100f10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009900 tx=0x7fcafc02a470 comp rx=0 tx=0).stop 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.212+0000 7fcb0801a640 1 -- 172.21.15.62:0/1237341860 shutdown_connections 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.212+0000 7fcb0801a640 1 --2- 172.21.15.62:0/1237341860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb00105f40 0x7fcb001063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.212+0000 7fcb0801a640 1 --2- 172.21.15.62:0/1237341860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb00101450 0x7fcb00105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.212+0000 7fcb0801a640 1 --2- 172.21.15.62:0/1237341860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb00100b30 0x7fcb00100f10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.212+0000 7fcb0801a640 1 -- 172.21.15.62:0/1237341860 >> 172.21.15.62:0/1237341860 conn(0x7fcb000fc7a0 msgr2=0x7fcb000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.212+0000 7fcb0801a640 1 -- 172.21.15.62:0/1237341860 shutdown_connections 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.212+0000 7fcb0801a640 1 -- 172.21.15.62:0/1237341860 wait complete. 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb0801a640 1 Processor -- start 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb0801a640 1 -- start start 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb0801a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb00100b30 0x7fcb001a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb0801a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb00101450 0x7fcb001a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb0801a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb00105f40 0x7fcb0019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb0801a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001185c0 con 0x7fcb00100b30 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb0801a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00118440 con 0x7fcb00101450 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb0801a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00118740 con 0x7fcb00105f40 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb05db6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb00100b30 0x7fcb001a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb05db6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb00100b30 0x7fcb001a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56424/0 (socket says 172.21.15.62:56424) 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb05db6640 1 -- 172.21.15.62:0/3272519686 learned_addr learned my addr 172.21.15.62:0/3272519686 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.213+0000 7fcb055b5640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb00101450 0x7fcb001a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcb065b7640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb00105f40 0x7fcb0019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcb05db6640 1 -- 172.21.15.62:0/3272519686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb00105f40 msgr2=0x7fcb0019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcb05db6640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb00105f40 0x7fcb0019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcb05db6640 1 -- 172.21.15.62:0/3272519686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb00101450 msgr2=0x7fcb001a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcb05db6640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb00101450 0x7fcb001a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcb05db6640 1 -- 172.21.15.62:0/3272519686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0019f080 con 0x7fcb00100b30 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcb05db6640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb00100b30 0x7fcb001a4790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcafc005ab0 tx=0x7fcafc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcaf6ffd640 1 -- 172.21.15.62:0/3272519686 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc03f030 con 0x7fcb00100b30 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcb0801a640 1 -- 172.21.15.62:0/3272519686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0019f310 con 0x7fcb00100b30 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcaf6ffd640 1 -- 172.21.15.62:0/3272519686 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcafc003ce0 con 0x7fcb00100b30 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcaf6ffd640 1 -- 172.21.15.62:0/3272519686 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc035070 con 0x7fcb00100b30 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.214+0000 7fcb0801a640 1 -- 172.21.15.62:0/3272519686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb001ab660 con 0x7fcb00100b30 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.215+0000 7fcb0801a640 1 -- 172.21.15.62:0/3272519686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb00105920 con 0x7fcb00100b30 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.216+0000 7fcaf6ffd640 1 -- 172.21.15.62:0/3272519686 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcafc042030 con 0x7fcb00100b30 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.217+0000 7fcaf6ffd640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaec0a2c00 0x7fcaec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:13.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.217+0000 7fcaf6ffd640 1 -- 172.21.15.62:0/3272519686 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fcafc0efa10 con 0x7fcb00100b30 2024-09-11T23:50:13.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.217+0000 7fcb055b5640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaec0a2c00 0x7fcaec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:13.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.217+0000 7fcb055b5640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaec0a2c00 0x7fcaec0a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcb000fec40 tx=0x7fcaf0009060 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:13.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.221+0000 7fcaf6ffd640 1 -- 172.21.15.62:0/3272519686 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcafc0bcac0 con 0x7fcb00100b30 2024-09-11T23:50:13.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.396+0000 7fcb0801a640 1 -- 172.21.15.62:0/3272519686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb001ab890 con 0x7fcb00100b30 2024-09-11T23:50:13.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.398+0000 7fcaf6ffd640 1 -- 172.21.15.62:0/3272519686 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22244 (secure 0 0 0) 0x7fcafc0bc3c0 con 0x7fcb00100b30 2024-09-11T23:50:13.398 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:13.398 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:50:11.406572+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:50:13.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 -- 172.21.15.62:0/3272519686 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaec0a2c00 msgr2=0x7fcaec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:13.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaec0a2c00 0x7fcaec0a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcb000fec40 tx=0x7fcaf0009060 comp rx=0 tx=0).stop 2024-09-11T23:50:13.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 -- 172.21.15.62:0/3272519686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb00100b30 msgr2=0x7fcb001a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:13.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb00100b30 0x7fcb001a4790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcafc005ab0 tx=0x7fcafc0042f0 comp rx=0 tx=0).stop 2024-09-11T23:50:13.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 -- 172.21.15.62:0/3272519686 shutdown_connections 2024-09-11T23:50:13.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaec0a2c00 0x7fcaec0a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:13.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb00105f40 0x7fcb0019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:13.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb00101450 0x7fcb001a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:13.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 --2- 172.21.15.62:0/3272519686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb00100b30 0x7fcb001a4790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:13.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 -- 172.21.15.62:0/3272519686 >> 172.21.15.62:0/3272519686 conn(0x7fcb000fc7a0 msgr2=0x7fcb0010f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:13.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 -- 172.21.15.62:0/3272519686 shutdown_connections 2024-09-11T23:50:13.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:13.400+0000 7fcb0801a640 1 -- 172.21.15.62:0/3272519686 wait complete. 2024-09-11T23:50:14.036 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:13 smithi062 ceph-mon[30958]: pgmap v281: 97 pgs: 19 creating+peering, 26 unknown, 52 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:14.036 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:13 smithi062 ceph-mon[30958]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:50:14.037 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:13 smithi062 ceph-mon[30958]: Cluster is now healthy 2024-09-11T23:50:14.037 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:13 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] up:active 2024-09-11T23:50:14.037 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:13 smithi062 ceph-mon[30958]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:14.037 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:13 smithi062 ceph-mon[30958]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:14.037 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:13 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3272519686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:14.066 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:50:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:13 smithi120 ceph-mon[35182]: pgmap v281: 97 pgs: 19 creating+peering, 26 unknown, 52 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:13 smithi120 ceph-mon[35182]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:50:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:13 smithi120 ceph-mon[35182]: Cluster is now healthy 2024-09-11T23:50:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:13 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] up:active 2024-09-11T23:50:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:13 smithi120 ceph-mon[35182]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:13 smithi120 ceph-mon[35182]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:14.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:13 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3272519686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:13 smithi083 ceph-mon[34805]: pgmap v281: 97 pgs: 19 creating+peering, 26 unknown, 52 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:50:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:13 smithi083 ceph-mon[34805]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:50:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:13 smithi083 ceph-mon[34805]: Cluster is now healthy 2024-09-11T23:50:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:13 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] up:active 2024-09-11T23:50:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:13 smithi083 ceph-mon[34805]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:13 smithi083 ceph-mon[34805]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:14.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:13 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3272519686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:14.322 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.771+0000 7fb0d0035640 1 -- 172.21.15.62:0/1821594941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c8106f50 msgr2=0x7fb0c810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.771+0000 7fb0d0035640 1 --2- 172.21.15.62:0/1821594941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c8106f50 0x7fb0c810eaf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009930 tx=0x7fb0c002a420 comp rx=0 tx=0).stop 2024-09-11T23:50:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.771+0000 7fb0d0035640 1 -- 172.21.15.62:0/1821594941 shutdown_connections 2024-09-11T23:50:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.771+0000 7fb0d0035640 1 --2- 172.21.15.62:0/1821594941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c8106f50 0x7fb0c810eaf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.771+0000 7fb0d0035640 1 --2- 172.21.15.62:0/1821594941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0c8100d00 0x7fb0c8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.771+0000 7fb0d0035640 1 --2- 172.21.15.62:0/1821594941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c8100350 0x7fb0c8100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.771+0000 7fb0d0035640 1 -- 172.21.15.62:0/1821594941 >> 172.21.15.62:0/1821594941 conn(0x7fb0c8077210 msgr2=0x7fb0c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.771+0000 7fb0d0035640 1 -- 172.21.15.62:0/1821594941 shutdown_connections 2024-09-11T23:50:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.771+0000 7fb0d0035640 1 -- 172.21.15.62:0/1821594941 wait complete. 2024-09-11T23:50:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0d0035640 1 Processor -- start 2024-09-11T23:50:14.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0d0035640 1 -- start start 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0d0035640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0c8100350 0x7fb0c8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0d0035640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c8100d00 0x7fb0c81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0d0035640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c8106f50 0x7fb0c819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0d0035640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8111fa0 con 0x7fb0c8106f50 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0d0035640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8111e20 con 0x7fb0c8100350 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0d0035640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8112120 con 0x7fb0c8100d00 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0ce5d2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c8106f50 0x7fb0c819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0ce5d2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c8106f50 0x7fb0c819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56442/0 (socket says 172.21.15.62:56442) 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0ce5d2640 1 -- 172.21.15.62:0/2081057071 learned_addr learned my addr 172.21.15.62:0/2081057071 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0cddd1640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0c8100350 0x7fb0c8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0cd5d0640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c8100d00 0x7fb0c81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:14.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0ce5d2640 1 -- 172.21.15.62:0/2081057071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c8100d00 msgr2=0x7fb0c81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0ce5d2640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c8100d00 0x7fb0c81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0ce5d2640 1 -- 172.21.15.62:0/2081057071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0c8100350 msgr2=0x7fb0c8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0ce5d2640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0c8100350 0x7fb0c8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.772+0000 7fb0ce5d2640 1 -- 172.21.15.62:0/2081057071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c819d330 con 0x7fb0c8106f50 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.773+0000 7fb0ce5d2640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c8106f50 0x7fb0c819cc50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0005ab0 tx=0x7fb0c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.773+0000 7fb0baffd640 1 -- 172.21.15.62:0/2081057071 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c003f030 con 0x7fb0c8106f50 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.773+0000 7fb0d0035640 1 -- 172.21.15.62:0/2081057071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c819d5c0 con 0x7fb0c8106f50 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.773+0000 7fb0baffd640 1 -- 172.21.15.62:0/2081057071 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c0003ce0 con 0x7fb0c8106f50 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.773+0000 7fb0baffd640 1 -- 172.21.15.62:0/2081057071 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0035070 con 0x7fb0c8106f50 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.773+0000 7fb0d0035640 1 -- 172.21.15.62:0/2081057071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c81a4f00 con 0x7fb0c8106f50 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.774+0000 7fb0d0035640 1 -- 172.21.15.62:0/2081057071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c8103410 con 0x7fb0c8106f50 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.775+0000 7fb0baffd640 1 -- 172.21.15.62:0/2081057071 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb0c0042030 con 0x7fb0c8106f50 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.776+0000 7fb0baffd640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0b40a2b80 0x7fb0b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:14.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.776+0000 7fb0cddd1640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0b40a2b80 0x7fb0b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:14.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.776+0000 7fb0baffd640 1 -- 172.21.15.62:0/2081057071 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb0c00ef980 con 0x7fb0c8106f50 2024-09-11T23:50:14.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.776+0000 7fb0cddd1640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0b40a2b80 0x7fb0b40a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4007350 tx=0x7fb0c40072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:14.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.779+0000 7fb0baffd640 1 -- 172.21.15.62:0/2081057071 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0c00bca30 con 0x7fb0c8106f50 2024-09-11T23:50:14.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.957+0000 7fb0d0035640 1 -- 172.21.15.62:0/2081057071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0c810a120 con 0x7fb0c8106f50 2024-09-11T23:50:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.958+0000 7fb0baffd640 1 -- 172.21.15.62:0/2081057071 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22244 (secure 0 0 0) 0x7fb0c00bc330 con 0x7fb0c8106f50 2024-09-11T23:50:14.958 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:14.958 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:50:11.406572+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:50:14.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.960+0000 7fb0d0035640 1 -- 172.21.15.62:0/2081057071 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0b40a2b80 msgr2=0x7fb0b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:14.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0b40a2b80 0x7fb0b40a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4007350 tx=0x7fb0c40072e0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 -- 172.21.15.62:0/2081057071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c8106f50 msgr2=0x7fb0c819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c8106f50 0x7fb0c819cc50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0005ab0 tx=0x7fb0c00042f0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 -- 172.21.15.62:0/2081057071 shutdown_connections 2024-09-11T23:50:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb0b40a2b80 0x7fb0b40a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c8106f50 0x7fb0c819cc50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c8100d00 0x7fb0c81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 --2- 172.21.15.62:0/2081057071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0c8100350 0x7fb0c8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 -- 172.21.15.62:0/2081057071 >> 172.21.15.62:0/2081057071 conn(0x7fb0c8077210 msgr2=0x7fb0c8105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 -- 172.21.15.62:0/2081057071 shutdown_connections 2024-09-11T23:50:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:14.961+0000 7fb0d0035640 1 -- 172.21.15.62:0/2081057071 wait complete. 2024-09-11T23:50:15.611 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T23:50:15.870 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:15 smithi062 ceph-mon[30958]: pgmap v282: 97 pgs: 12 creating+peering, 85 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-11T23:50:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:15 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2081057071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:16.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:15 smithi120 ceph-mon[35182]: pgmap v282: 97 pgs: 12 creating+peering, 85 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-11T23:50:16.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:15 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2081057071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:16.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:15 smithi083 ceph-mon[34805]: pgmap v282: 97 pgs: 12 creating+peering, 85 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-11T23:50:16.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:15 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2081057071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:16.167 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b[64402]: 2024-09-11T23:50:15.774+0000 7ff382e15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T23:50:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.334+0000 7feb5337e640 1 -- 172.21.15.62:0/2839107135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c102790 msgr2=0x7feb4c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.334+0000 7feb5337e640 1 --2- 172.21.15.62:0/2839107135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c102790 0x7feb4c1067e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009900 tx=0x7feb3c02a410 comp rx=0 tx=0).stop 2024-09-11T23:50:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.334+0000 7feb5337e640 1 -- 172.21.15.62:0/2839107135 shutdown_connections 2024-09-11T23:50:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.334+0000 7feb5337e640 1 --2- 172.21.15.62:0/2839107135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb4c106f10 0x7feb4c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.334+0000 7feb5337e640 1 --2- 172.21.15.62:0/2839107135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c102790 0x7feb4c1067e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.334+0000 7feb5337e640 1 --2- 172.21.15.62:0/2839107135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb4c101de0 0x7feb4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.334+0000 7feb5337e640 1 -- 172.21.15.62:0/2839107135 >> 172.21.15.62:0/2839107135 conn(0x7feb4c0fd8c0 msgr2=0x7feb4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.334+0000 7feb5337e640 1 -- 172.21.15.62:0/2839107135 shutdown_connections 2024-09-11T23:50:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.334+0000 7feb5337e640 1 -- 172.21.15.62:0/2839107135 wait complete. 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5337e640 1 Processor -- start 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5337e640 1 -- start start 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5337e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb4c101de0 0x7feb4c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5337e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb4c102790 0x7feb4c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5337e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c106f10 0x7feb4c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5337e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c111fe0 con 0x7feb4c106f10 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5337e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c111e60 con 0x7feb4c101de0 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5337e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c112160 con 0x7feb4c102790 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5191b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c106f10 0x7feb4c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5191b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c106f10 0x7feb4c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56458/0 (socket says 172.21.15.62:56458) 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5191b640 1 -- 172.21.15.62:0/4196743035 learned_addr learned my addr 172.21.15.62:0/4196743035 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.335+0000 7feb5111a640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb4c101de0 0x7feb4c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb50919640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb4c102790 0x7feb4c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb5191b640 1 -- 172.21.15.62:0/4196743035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb4c102790 msgr2=0x7feb4c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb5191b640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb4c102790 0x7feb4c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb5191b640 1 -- 172.21.15.62:0/4196743035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb4c101de0 msgr2=0x7feb4c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb5191b640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb4c101de0 0x7feb4c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb5191b640 1 -- 172.21.15.62:0/4196743035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4c19d3a0 con 0x7feb4c106f10 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb5191b640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c106f10 0x7feb4c19ccc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feb4400b970 tx=0x7feb4400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb427fc640 1 -- 172.21.15.62:0/4196743035 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44019070 con 0x7feb4c106f10 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb427fc640 1 -- 172.21.15.62:0/4196743035 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb44014440 con 0x7feb4c106f10 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb5337e640 1 -- 172.21.15.62:0/4196743035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4c19d690 con 0x7feb4c106f10 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb427fc640 1 -- 172.21.15.62:0/4196743035 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb440075d0 con 0x7feb4c106f10 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.336+0000 7feb5337e640 1 -- 172.21.15.62:0/4196743035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4c1a4f70 con 0x7feb4c106f10 2024-09-11T23:50:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.337+0000 7feb5337e640 1 -- 172.21.15.62:0/4196743035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb4c1021c0 con 0x7feb4c106f10 2024-09-11T23:50:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.338+0000 7feb427fc640 1 -- 172.21.15.62:0/4196743035 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feb4401c070 con 0x7feb4c106f10 2024-09-11T23:50:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.339+0000 7feb427fc640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feb380a2b80 0x7feb380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.339+0000 7feb5111a640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feb380a2b80 0x7feb380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.339+0000 7feb427fc640 1 -- 172.21.15.62:0/4196743035 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7feb440ce910 con 0x7feb4c106f10 2024-09-11T23:50:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.340+0000 7feb5111a640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feb380a2b80 0x7feb380a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feb4800abe0 tx=0x7feb4800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.342+0000 7feb427fc640 1 -- 172.21.15.62:0/4196743035 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb4409b370 con 0x7feb4c106f10 2024-09-11T23:50:16.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.554+0000 7feb5337e640 1 -- 172.21.15.62:0/4196743035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feb4c199750 con 0x7feb4c106f10 2024-09-11T23:50:16.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.838+0000 7feb427fc640 1 -- 172.21.15.62:0/4196743035 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7feb4c199750 con 0x7feb4c106f10 2024-09-11T23:50:16.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 -- 172.21.15.62:0/4196743035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feb380a2b80 msgr2=0x7feb380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:16.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feb380a2b80 0x7feb380a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feb4800abe0 tx=0x7feb4800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:50:16.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 -- 172.21.15.62:0/4196743035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c106f10 msgr2=0x7feb4c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:16.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c106f10 0x7feb4c19ccc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feb4400b970 tx=0x7feb4400bd70 comp rx=0 tx=0).stop 2024-09-11T23:50:16.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 -- 172.21.15.62:0/4196743035 shutdown_connections 2024-09-11T23:50:16.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feb380a2b80 0x7feb380a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c106f10 0x7feb4c19ccc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb4c102790 0x7feb4c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 --2- 172.21.15.62:0/4196743035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb4c101de0 0x7feb4c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 -- 172.21.15.62:0/4196743035 >> 172.21.15.62:0/4196743035 conn(0x7feb4c0fd8c0 msgr2=0x7feb4c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 -- 172.21.15.62:0/4196743035 shutdown_connections 2024-09-11T23:50:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:16.840+0000 7feb5337e640 1 -- 172.21.15.62:0/4196743035 wait complete. 2024-09-11T23:50:17.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:16 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/4196743035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:50:17.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:16 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/4196743035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:50:17.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:16 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/4196743035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:50:17.511 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-11T23:50:17.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-11T23:50:17.771 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:18.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:17 smithi120 ceph-mon[35182]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-11T23:50:18.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:17 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/4196743035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:50:18.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:17 smithi120 ceph-mon[35182]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:18.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:17 smithi062 ceph-mon[30958]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-11T23:50:18.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:17 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/4196743035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:50:18.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:17 smithi062 ceph-mon[30958]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:18.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:17 smithi083 ceph-mon[34805]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-11T23:50:18.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:17 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/4196743035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:50:18.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:17 smithi083 ceph-mon[34805]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:18.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.229+0000 7f55626d5640 1 -- 172.21.15.62:0/2057965974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f555c102ef0 msgr2=0x7f555c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:18.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.229+0000 7f55626d5640 1 --2- 172.21.15.62:0/2057965974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f555c102ef0 0x7f555c10da40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5554009ca0 tx=0x7f555402a950 comp rx=0 tx=0).stop 2024-09-11T23:50:18.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.229+0000 7f55626d5640 1 -- 172.21.15.62:0/2057965974 shutdown_connections 2024-09-11T23:50:18.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.229+0000 7f55626d5640 1 --2- 172.21.15.62:0/2057965974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f555c102ef0 0x7f555c10da40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:18.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.229+0000 7f55626d5640 1 --2- 172.21.15.62:0/2057965974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f555c102570 0x7f555c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:18.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.229+0000 7f55626d5640 1 --2- 172.21.15.62:0/2057965974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555c100e20 0x7f555c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.229+0000 7f55626d5640 1 -- 172.21.15.62:0/2057965974 >> 172.21.15.62:0/2057965974 conn(0x7f555c0fc7d0 msgr2=0x7f555c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.229+0000 7f55626d5640 1 -- 172.21.15.62:0/2057965974 shutdown_connections 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.229+0000 7f55626d5640 1 -- 172.21.15.62:0/2057965974 wait complete. 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f55626d5640 1 Processor -- start 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f55626d5640 1 -- start start 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f55626d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f555c100e20 0x7f555c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f55626d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555c102570 0x7f555c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f55626d5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f555c102ef0 0x7f555c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f55626d5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f555c110f80 con 0x7f555c100e20 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f55626d5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f555c110e00 con 0x7f555c102570 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f55626d5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f555c111100 con 0x7f555c102ef0 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f555bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f555c100e20 0x7f555c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f555bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f555c100e20 0x7f555c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56478/0 (socket says 172.21.15.62:56478) 2024-09-11T23:50:18.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f555bfff640 1 -- 172.21.15.62:0/2732052038 learned_addr learned my addr 172.21.15.62:0/2732052038 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f5553fff640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555c102570 0x7f555c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f5560c72640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f555c102ef0 0x7f555c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.230+0000 7f555bfff640 1 -- 172.21.15.62:0/2732052038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f555c102ef0 msgr2=0x7f555c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f555bfff640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f555c102ef0 0x7f555c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f555bfff640 1 -- 172.21.15.62:0/2732052038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555c102570 msgr2=0x7f555c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f555bfff640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555c102570 0x7f555c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f555bfff640 1 -- 172.21.15.62:0/2732052038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c1a6990 con 0x7f555c100e20 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f5560c72640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f555c102ef0 0x7f555c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f555bfff640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f555c100e20 0x7f555c0fcc80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f554c00d950 tx=0x7f554c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f5559ffb640 1 -- 172.21.15.62:0/2732052038 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c017070 con 0x7f555c100e20 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f55626d5640 1 -- 172.21.15.62:0/2732052038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555c1a6c80 con 0x7f555c100e20 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f5559ffb640 1 -- 172.21.15.62:0/2732052038 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f554c009c30 con 0x7f555c100e20 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f5559ffb640 1 -- 172.21.15.62:0/2732052038 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c00f6b0 con 0x7f555c100e20 2024-09-11T23:50:18.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.231+0000 7f55626d5640 1 -- 172.21.15.62:0/2732052038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f555c1a71c0 con 0x7f555c100e20 2024-09-11T23:50:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.232+0000 7f55626d5640 1 -- 172.21.15.62:0/2732052038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f555c101200 con 0x7f555c100e20 2024-09-11T23:50:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.233+0000 7f5559ffb640 1 -- 172.21.15.62:0/2732052038 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f554c0109d0 con 0x7f555c100e20 2024-09-11T23:50:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.234+0000 7f5559ffb640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f55480a2b80 0x7f55480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.234+0000 7f5559ffb640 1 -- 172.21.15.62:0/2732052038 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f554c0ceb10 con 0x7f555c100e20 2024-09-11T23:50:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.234+0000 7f5553fff640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f55480a2b80 0x7f55480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.235+0000 7f5553fff640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f55480a2b80 0x7f55480a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f555c0fe440 tx=0x7f5544005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.241+0000 7f5559ffb640 1 -- 172.21.15.62:0/2732052038 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f554c009de0 con 0x7f555c100e20 2024-09-11T23:50:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.457+0000 7f55626d5640 1 -- 172.21.15.62:0/2732052038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f555c02d7c0 con 0x7f555c100e20 2024-09-11T23:50:18.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.846+0000 7f5559ffb640 1 -- 172.21.15.62:0/2732052038 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f554c09bbc0 con 0x7f555c100e20 2024-09-11T23:50:18.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 -- 172.21.15.62:0/2732052038 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f55480a2b80 msgr2=0x7f55480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f55480a2b80 0x7f55480a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f555c0fe440 tx=0x7f5544005f90 comp rx=0 tx=0).stop 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 -- 172.21.15.62:0/2732052038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f555c100e20 msgr2=0x7f555c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f555c100e20 0x7f555c0fcc80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f554c00d950 tx=0x7f554c00de10 comp rx=0 tx=0).stop 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 -- 172.21.15.62:0/2732052038 shutdown_connections 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f55480a2b80 0x7f55480a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f555c102ef0 0x7f555c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f555c102570 0x7f555c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 --2- 172.21.15.62:0/2732052038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f555c100e20 0x7f555c0fcc80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 -- 172.21.15.62:0/2732052038 >> 172.21.15.62:0/2732052038 conn(0x7f555c0fc7d0 msgr2=0x7f555c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 -- 172.21.15.62:0/2732052038 shutdown_connections 2024-09-11T23:50:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:18.849+0000 7f55626d5640 1 -- 172.21.15.62:0/2732052038 wait complete. 2024-09-11T23:50:19.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:18 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2732052038' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-11T23:50:19.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:18 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2732052038' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-11T23:50:19.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:18 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2732052038' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-11T23:50:19.474 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T23:50:19.730 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:19 smithi120 ceph-mon[35182]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T23:50:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:19 smithi120 ceph-mon[35182]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:50:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:19 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2732052038' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-11T23:50:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:19 smithi120 ceph-mon[35182]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:50:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:19 smithi120 ceph-mon[35182]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:19 smithi120 ceph-mon[35182]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T23:50:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:19 smithi120 ceph-mon[35182]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T23:50:20.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:19 smithi062 ceph-mon[30958]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T23:50:20.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:19 smithi062 ceph-mon[30958]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:50:20.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:19 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2732052038' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-11T23:50:20.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:19 smithi062 ceph-mon[30958]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:50:20.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:19 smithi062 ceph-mon[30958]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:20.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:19 smithi062 ceph-mon[30958]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T23:50:20.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:19 smithi062 ceph-mon[30958]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T23:50:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:19 smithi083 ceph-mon[34805]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T23:50:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:19 smithi083 ceph-mon[34805]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:50:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:19 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2732052038' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-11T23:50:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:19 smithi083 ceph-mon[34805]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:50:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:19 smithi083 ceph-mon[34805]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:50:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:19 smithi083 ceph-mon[34805]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T23:50:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:19 smithi083 ceph-mon[34805]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T23:50:20.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.214+0000 7f1f3d392640 1 -- 172.21.15.62:0/2593959593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f38108290 msgr2=0x7f1f3810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:20.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.214+0000 7f1f3d392640 1 --2- 172.21.15.62:0/2593959593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f38108290 0x7f1f3810ec30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009930 tx=0x7f1f3002a420 comp rx=0 tx=0).stop 2024-09-11T23:50:20.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.214+0000 7f1f3d392640 1 -- 172.21.15.62:0/2593959593 shutdown_connections 2024-09-11T23:50:20.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.214+0000 7f1f3d392640 1 --2- 172.21.15.62:0/2593959593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f38108290 0x7f1f3810ec30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:20.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.214+0000 7f1f3d392640 1 --2- 172.21.15.62:0/2593959593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f38103740 0x7f1f38107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:20.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.214+0000 7f1f3d392640 1 --2- 172.21.15.62:0/2593959593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f38102d90 0x7f1f38103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:20.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.214+0000 7f1f3d392640 1 -- 172.21.15.62:0/2593959593 >> 172.21.15.62:0/2593959593 conn(0x7f1f380fc7d0 msgr2=0x7f1f380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:20.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.214+0000 7f1f3d392640 1 -- 172.21.15.62:0/2593959593 shutdown_connections 2024-09-11T23:50:20.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.214+0000 7f1f3d392640 1 -- 172.21.15.62:0/2593959593 wait complete. 2024-09-11T23:50:20.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.215+0000 7f1f3d392640 1 Processor -- start 2024-09-11T23:50:20.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.215+0000 7f1f3d392640 1 -- start start 2024-09-11T23:50:20.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.215+0000 7f1f3d392640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f38102d90 0x7f1f38198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:20.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.215+0000 7f1f3d392640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f38103740 0x7f1f38198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:20.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.215+0000 7f1f3d392640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f38108290 0x7f1f3819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:20.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.215+0000 7f1f3d392640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381120d0 con 0x7f1f38102d90 2024-09-11T23:50:20.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.215+0000 7f1f3d392640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38111f50 con 0x7f1f38103740 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.215+0000 7f1f3d392640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38112250 con 0x7f1f38108290 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f36ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f38102d90 0x7f1f38198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f36ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f38102d90 0x7f1f38198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56500/0 (socket says 172.21.15.62:56500) 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f36ffd640 1 -- 172.21.15.62:0/1541901233 learned_addr learned my addr 172.21.15.62:0/1541901233 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f367fc640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f38103740 0x7f1f38198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f377fe640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f38108290 0x7f1f3819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f36ffd640 1 -- 172.21.15.62:0/1541901233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f38108290 msgr2=0x7f1f3819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f36ffd640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f38108290 0x7f1f3819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f36ffd640 1 -- 172.21.15.62:0/1541901233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f38103740 msgr2=0x7f1f38198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f36ffd640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f38103740 0x7f1f38198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f36ffd640 1 -- 172.21.15.62:0/1541901233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3819d500 con 0x7f1f38102d90 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f36ffd640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f38102d90 0x7f1f38198550 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800cce0 tx=0x7f1f2800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f2ffff640 1 -- 172.21.15.62:0/1541901233 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800f450 con 0x7f1f38102d90 2024-09-11T23:50:20.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f3d392640 1 -- 172.21.15.62:0/1541901233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3819d7f0 con 0x7f1f38102d90 2024-09-11T23:50:20.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f2ffff640 1 -- 172.21.15.62:0/1541901233 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f2800fad0 con 0x7f1f38102d90 2024-09-11T23:50:20.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f2ffff640 1 -- 172.21.15.62:0/1541901233 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28018b80 con 0x7f1f38102d90 2024-09-11T23:50:20.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.216+0000 7f1f3d392640 1 -- 172.21.15.62:0/1541901233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f381a50d0 con 0x7f1f38102d90 2024-09-11T23:50:20.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.218+0000 7f1f3d392640 1 -- 172.21.15.62:0/1541901233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f38103170 con 0x7f1f38102d90 2024-09-11T23:50:20.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.218+0000 7f1f2ffff640 1 -- 172.21.15.62:0/1541901233 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1f2800f5f0 con 0x7f1f38102d90 2024-09-11T23:50:20.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.219+0000 7f1f2ffff640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1f240a2b80 0x7f1f240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:20.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.219+0000 7f1f367fc640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1f240a2b80 0x7f1f240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:20.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.219+0000 7f1f2ffff640 1 -- 172.21.15.62:0/1541901233 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f1f280cf540 con 0x7f1f38102d90 2024-09-11T23:50:20.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.220+0000 7f1f367fc640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1f240a2b80 0x7f1f240a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f380fdb60 tx=0x7f1f20009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.223+0000 7f1f2ffff640 1 -- 172.21.15.62:0/1541901233 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f28014030 con 0x7f1f38102d90 2024-09-11T23:50:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.439+0000 7f1f3d392640 1 -- 172.21.15.62:0/1541901233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f1f3810a2a0 con 0x7f1f38102d90 2024-09-11T23:50:20.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.861+0000 7f1f2ffff640 1 -- 172.21.15.62:0/1541901233 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f1f2809c5f0 con 0x7f1f38102d90 2024-09-11T23:50:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.864+0000 7f1f3d392640 1 -- 172.21.15.62:0/1541901233 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1f240a2b80 msgr2=0x7f1f240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.864+0000 7f1f3d392640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1f240a2b80 0x7f1f240a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f380fdb60 tx=0x7f1f20009110 comp rx=0 tx=0).stop 2024-09-11T23:50:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.864+0000 7f1f3d392640 1 -- 172.21.15.62:0/1541901233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f38102d90 msgr2=0x7f1f38198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.864+0000 7f1f3d392640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f38102d90 0x7f1f38198550 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800cce0 tx=0x7f1f2800a620 comp rx=0 tx=0).stop 2024-09-11T23:50:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.864+0000 7f1f3d392640 1 -- 172.21.15.62:0/1541901233 shutdown_connections 2024-09-11T23:50:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.864+0000 7f1f3d392640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f1f240a2b80 0x7f1f240a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.864+0000 7f1f3d392640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f38108290 0x7f1f3819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.864+0000 7f1f3d392640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f38103740 0x7f1f38198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.864+0000 7f1f3d392640 1 --2- 172.21.15.62:0/1541901233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f38102d90 0x7f1f38198550 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.864+0000 7f1f3d392640 1 -- 172.21.15.62:0/1541901233 >> 172.21.15.62:0/1541901233 conn(0x7f1f380fc7d0 msgr2=0x7f1f380fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.865+0000 7f1f3d392640 1 -- 172.21.15.62:0/1541901233 shutdown_connections 2024-09-11T23:50:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:20.865+0000 7f1f3d392640 1 -- 172.21.15.62:0/1541901233 wait complete. 2024-09-11T23:50:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:20 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] up:active 2024-09-11T23:50:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:20 smithi120 ceph-mon[35182]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:50:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:20 smithi120 ceph-mon[35182]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:50:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:20 smithi120 ceph-mon[35182]: Cluster is now healthy 2024-09-11T23:50:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:20 smithi120 ceph-mon[35182]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T23:50:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:20 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T23:50:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:20 smithi120 ceph-mon[35182]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T23:50:21.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:20 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1541901233' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:50:21.167 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 11 23:50:20 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b[64402]: 2024-09-11T23:50:20.775+0000 7ff382e15640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T23:50:21.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:20 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] up:active 2024-09-11T23:50:21.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:20 smithi083 ceph-mon[34805]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:50:21.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:20 smithi083 ceph-mon[34805]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:50:21.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:20 smithi083 ceph-mon[34805]: Cluster is now healthy 2024-09-11T23:50:21.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:20 smithi083 ceph-mon[34805]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T23:50:21.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:20 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T23:50:21.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:20 smithi083 ceph-mon[34805]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T23:50:21.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:20 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1541901233' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:50:21.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:20 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] up:active 2024-09-11T23:50:21.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:20 smithi062 ceph-mon[30958]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:50:21.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:20 smithi062 ceph-mon[30958]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:50:21.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:20 smithi062 ceph-mon[30958]: Cluster is now healthy 2024-09-11T23:50:21.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:20 smithi062 ceph-mon[30958]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T23:50:21.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:20 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T23:50:21.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:20 smithi062 ceph-mon[30958]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T23:50:21.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:20 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1541901233' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:50:21.576 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T23:50:21.840 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:22.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:21 smithi062 ceph-mon[30958]: pgmap v285: 97 pgs: 97 active+clean; 580 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-11T23:50:22.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:21 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1541901233' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:50:22.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:21 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] up:active 2024-09-11T23:50:22.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:21 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T23:50:22.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:21 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:50:22.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:21 smithi120 ceph-mon[35182]: pgmap v285: 97 pgs: 97 active+clean; 580 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-11T23:50:22.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:21 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1541901233' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:50:22.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:21 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] up:active 2024-09-11T23:50:22.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:21 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T23:50:22.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:21 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:50:22.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:21 smithi083 ceph-mon[34805]: pgmap v285: 97 pgs: 97 active+clean; 580 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-11T23:50:22.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:21 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1541901233' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:50:22.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:21 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] up:active 2024-09-11T23:50:22.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:21 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T23:50:22.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:21 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:50:22.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.329+0000 7fcbb3939640 1 -- 172.21.15.62:0/796694186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac101de0 msgr2=0x7fcbac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:22.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.329+0000 7fcbb3939640 1 --2- 172.21.15.62:0/796694186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac101de0 0x7fcbac1021c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009900 tx=0x7fcba802a410 comp rx=0 tx=0).stop 2024-09-11T23:50:22.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.329+0000 7fcbb3939640 1 -- 172.21.15.62:0/796694186 shutdown_connections 2024-09-11T23:50:22.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.329+0000 7fcbb3939640 1 --2- 172.21.15.62:0/796694186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbac106f10 0x7fcbac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:22.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.329+0000 7fcbb3939640 1 --2- 172.21.15.62:0/796694186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbac102790 0x7fcbac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:22.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.329+0000 7fcbb3939640 1 --2- 172.21.15.62:0/796694186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac101de0 0x7fcbac1021c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:22.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.329+0000 7fcbb3939640 1 -- 172.21.15.62:0/796694186 >> 172.21.15.62:0/796694186 conn(0x7fcbac0fd8c0 msgr2=0x7fcbac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:22.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.329+0000 7fcbb3939640 1 -- 172.21.15.62:0/796694186 shutdown_connections 2024-09-11T23:50:22.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.329+0000 7fcbb3939640 1 -- 172.21.15.62:0/796694186 wait complete. 2024-09-11T23:50:22.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.330+0000 7fcbb3939640 1 Processor -- start 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.330+0000 7fcbb3939640 1 -- start start 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.330+0000 7fcbb3939640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbac101de0 0x7fcbac1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.330+0000 7fcbb3939640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbac102790 0x7fcbac198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.330+0000 7fcbb3939640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac106f10 0x7fcbac19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.330+0000 7fcbb3939640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac112010 con 0x7fcbac106f10 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb3939640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac111e90 con 0x7fcbac101de0 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb3939640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac112190 con 0x7fcbac102790 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb1ed6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac106f10 0x7fcbac19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb1ed6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac106f10 0x7fcbac19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44502/0 (socket says 172.21.15.62:44502) 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb1ed6640 1 -- 172.21.15.62:0/602520659 learned_addr learned my addr 172.21.15.62:0/602520659 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb0ed4640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbac102790 0x7fcbac198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb16d5640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbac101de0 0x7fcbac1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb1ed6640 1 -- 172.21.15.62:0/602520659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbac102790 msgr2=0x7fcbac198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb1ed6640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbac102790 0x7fcbac198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb1ed6640 1 -- 172.21.15.62:0/602520659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbac101de0 msgr2=0x7fcbac1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb1ed6640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbac101de0 0x7fcbac1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb1ed6640 1 -- 172.21.15.62:0/602520659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbac19d560 con 0x7fcbac106f10 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb1ed6640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac106f10 0x7fcbac19ce80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcba400b970 tx=0x7fcba400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcba27fc640 1 -- 172.21.15.62:0/602520659 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4019070 con 0x7fcbac106f10 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb3939640 1 -- 172.21.15.62:0/602520659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbac19d850 con 0x7fcbac106f10 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcba27fc640 1 -- 172.21.15.62:0/602520659 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba4014440 con 0x7fcbac106f10 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.331+0000 7fcbb3939640 1 -- 172.21.15.62:0/602520659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbac1a5130 con 0x7fcbac106f10 2024-09-11T23:50:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.332+0000 7fcba27fc640 1 -- 172.21.15.62:0/602520659 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40075d0 con 0x7fcbac106f10 2024-09-11T23:50:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.332+0000 7fcbb3939640 1 -- 172.21.15.62:0/602520659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb90005290 con 0x7fcbac106f10 2024-09-11T23:50:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.334+0000 7fcba27fc640 1 -- 172.21.15.62:0/602520659 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcba401c070 con 0x7fcbac106f10 2024-09-11T23:50:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.334+0000 7fcba27fc640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb980a2b80 0x7fcb980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.335+0000 7fcbb16d5640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb980a2b80 0x7fcb980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.335+0000 7fcba27fc640 1 -- 172.21.15.62:0/602520659 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fcba40ce650 con 0x7fcbac106f10 2024-09-11T23:50:22.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.335+0000 7fcbb16d5640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb980a2b80 0x7fcb980a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcba80096c0 tx=0x7fcba8031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.338+0000 7fcba27fc640 1 -- 172.21.15.62:0/602520659 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcba409b700 con 0x7fcbac106f10 2024-09-11T23:50:22.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.551+0000 7fcbb3939640 1 -- 172.21.15.62:0/602520659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fcb90005580 con 0x7fcbac106f10 2024-09-11T23:50:22.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.868+0000 7fcba27fc640 1 -- 172.21.15.62:0/602520659 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fcba409b000 con 0x7fcbac106f10 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 -- 172.21.15.62:0/602520659 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb980a2b80 msgr2=0x7fcb980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb980a2b80 0x7fcb980a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcba80096c0 tx=0x7fcba8031490 comp rx=0 tx=0).stop 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 -- 172.21.15.62:0/602520659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac106f10 msgr2=0x7fcbac19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac106f10 0x7fcbac19ce80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcba400b970 tx=0x7fcba400bd70 comp rx=0 tx=0).stop 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 -- 172.21.15.62:0/602520659 shutdown_connections 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb980a2b80 0x7fcb980a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbac106f10 0x7fcbac19ce80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbac102790 0x7fcbac198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 --2- 172.21.15.62:0/602520659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbac101de0 0x7fcbac1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 -- 172.21.15.62:0/602520659 >> 172.21.15.62:0/602520659 conn(0x7fcbac0fd8c0 msgr2=0x7fcbac0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:22.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 -- 172.21.15.62:0/602520659 shutdown_connections 2024-09-11T23:50:22.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:22.871+0000 7fcbb3939640 1 -- 172.21.15.62:0/602520659 wait complete. 2024-09-11T23:50:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:22 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/602520659' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:50:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:22 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/602520659' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:50:23.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:22 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/602520659' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:50:23.568 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-11T23:50:23.569 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T23:50:23.838 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:24.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:23 smithi120 ceph-mon[35182]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 559 B/s rd, 3.1 KiB/s wr, 8 op/s 2024-09-11T23:50:24.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:23 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/602520659' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:50:24.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:23 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:50:24.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:23 smithi062 ceph-mon[30958]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 559 B/s rd, 3.1 KiB/s wr, 8 op/s 2024-09-11T23:50:24.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:23 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/602520659' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:50:24.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:23 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:50:24.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:23 smithi083 ceph-mon[34805]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 559 B/s rd, 3.1 KiB/s wr, 8 op/s 2024-09-11T23:50:24.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:23 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/602520659' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:50:24.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:23 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:50:24.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.338+0000 7ff2988cb640 1 -- 172.21.15.62:0/73555676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff294100e20 msgr2=0x7ff294101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:24.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.338+0000 7ff2988cb640 1 --2- 172.21.15.62:0/73555676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff294100e20 0x7ff294101200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff284009900 tx=0x7ff28402a480 comp rx=0 tx=0).stop 2024-09-11T23:50:24.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.338+0000 7ff2988cb640 1 -- 172.21.15.62:0/73555676 shutdown_connections 2024-09-11T23:50:24.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.338+0000 7ff2988cb640 1 --2- 172.21.15.62:0/73555676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294102ef0 0x7ff29410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.338+0000 7ff2988cb640 1 --2- 172.21.15.62:0/73555676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff294102570 0x7ff2941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.338+0000 7ff2988cb640 1 --2- 172.21.15.62:0/73555676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff294100e20 0x7ff294101200 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.338+0000 7ff2988cb640 1 -- 172.21.15.62:0/73555676 >> 172.21.15.62:0/73555676 conn(0x7ff2940fc7d0 msgr2=0x7ff2940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.338+0000 7ff2988cb640 1 -- 172.21.15.62:0/73555676 shutdown_connections 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.338+0000 7ff2988cb640 1 -- 172.21.15.62:0/73555676 wait complete. 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.338+0000 7ff2988cb640 1 Processor -- start 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff2988cb640 1 -- start start 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff2988cb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff294100e20 0x7ff2941973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff2988cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294102570 0x7ff294197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff2988cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff294102ef0 0x7ff29419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff2988cb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff294110fa0 con 0x7ff294100e20 2024-09-11T23:50:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff2988cb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff294110e20 con 0x7ff294102570 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff2988cb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff294111120 con 0x7ff294102ef0 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff29259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff294100e20 0x7ff2941973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff29259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff294100e20 0x7ff2941973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44516/0 (socket says 172.21.15.62:44516) 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff29259c640 1 -- 172.21.15.62:0/4232329983 learned_addr learned my addr 172.21.15.62:0/4232329983 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff29259c640 1 -- 172.21.15.62:0/4232329983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff294102ef0 msgr2=0x7ff29419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff292d9d640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff294102ef0 0x7ff29419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff29259c640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff294102ef0 0x7ff29419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff28bfff640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294102570 0x7ff294197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff29259c640 1 -- 172.21.15.62:0/4232329983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294102570 msgr2=0x7ff294197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff29259c640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294102570 0x7ff294197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff29259c640 1 -- 172.21.15.62:0/4232329983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29419c370 con 0x7ff294100e20 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.339+0000 7ff29259c640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff294100e20 0x7ff2941973c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff284006010 tx=0x7ff284003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.340+0000 7ff28b7fe640 1 -- 172.21.15.62:0/4232329983 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28403f070 con 0x7ff294100e20 2024-09-11T23:50:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.340+0000 7ff28b7fe640 1 -- 172.21.15.62:0/4232329983 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff284031e30 con 0x7ff294100e20 2024-09-11T23:50:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.340+0000 7ff2988cb640 1 -- 172.21.15.62:0/4232329983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29419c600 con 0x7ff294100e20 2024-09-11T23:50:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.340+0000 7ff28b7fe640 1 -- 172.21.15.62:0/4232329983 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284035030 con 0x7ff294100e20 2024-09-11T23:50:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.340+0000 7ff2988cb640 1 -- 172.21.15.62:0/4232329983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2941a3f40 con 0x7ff294100e20 2024-09-11T23:50:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.341+0000 7ff2988cb640 1 -- 172.21.15.62:0/4232329983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2941021b0 con 0x7ff294100e20 2024-09-11T23:50:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.342+0000 7ff28b7fe640 1 -- 172.21.15.62:0/4232329983 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff284042070 con 0x7ff294100e20 2024-09-11T23:50:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.343+0000 7ff28b7fe640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2800a2b80 0x7ff2800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.343+0000 7ff28bfff640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2800a2b80 0x7ff2800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.343+0000 7ff28b7fe640 1 -- 172.21.15.62:0/4232329983 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ff2840ef8f0 con 0x7ff294100e20 2024-09-11T23:50:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.343+0000 7ff28bfff640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2800a2b80 0x7ff2800a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff2940fec10 tx=0x7ff27c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.347+0000 7ff28b7fe640 1 -- 172.21.15.62:0/4232329983 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2840bc9a0 con 0x7ff294100e20 2024-09-11T23:50:24.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.551+0000 7ff2988cb640 1 -- 172.21.15.62:0/4232329983 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff294106f50 con 0x7ff2800a2b80 2024-09-11T23:50:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.575+0000 7ff28b7fe640 1 -- 172.21.15.62:0/4232329983 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff294106f50 con 0x7ff2800a2b80 2024-09-11T23:50:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 -- 172.21.15.62:0/4232329983 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2800a2b80 msgr2=0x7ff2800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2800a2b80 0x7ff2800a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff2940fec10 tx=0x7ff27c005cc0 comp rx=0 tx=0).stop 2024-09-11T23:50:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 -- 172.21.15.62:0/4232329983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff294100e20 msgr2=0x7ff2941973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff294100e20 0x7ff2941973c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff284006010 tx=0x7ff284003c80 comp rx=0 tx=0).stop 2024-09-11T23:50:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 -- 172.21.15.62:0/4232329983 shutdown_connections 2024-09-11T23:50:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2800a2b80 0x7ff2800a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff294102ef0 0x7ff29419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294102570 0x7ff294197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 --2- 172.21.15.62:0/4232329983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff294100e20 0x7ff2941973c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 -- 172.21.15.62:0/4232329983 >> 172.21.15.62:0/4232329983 conn(0x7ff2940fc7d0 msgr2=0x7ff294106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:24.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 -- 172.21.15.62:0/4232329983 shutdown_connections 2024-09-11T23:50:24.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:24.578+0000 7ff2988cb640 1 -- 172.21.15.62:0/4232329983 wait complete. 2024-09-11T23:50:24.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:50:24 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:50:24.556+0000 7f9f317f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:50:24.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:50:24 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:50:24.556+0000 7f9f317f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:50:24.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:50:24 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:50:24.556+0000 7f9f317f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:50:24.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:50:24 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:50:24.556+0000 7f9f317f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:50:24.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:50:24 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:50:24.556+0000 7f9f317f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:50:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:24 smithi083 ceph-mon[34805]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:24 smithi062 ceph-mon[30958]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:25.284 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:24 smithi120 ceph-mon[35182]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:25.374 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T23:50:25.634 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:26.008 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:25 smithi062 ceph-mon[30958]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-11T23:50:26.008 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:25 smithi062 ceph-mon[30958]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T23:50:26.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.094+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/101927393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c48101420 msgr2=0x7f2c4810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.094+0000 7f2c4ce1a640 1 --2- 172.21.15.62:0/101927393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c48101420 0x7f2c4810d950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009cd0 tx=0x7f2c4002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.094+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/101927393 shutdown_connections 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.094+0000 7f2c4ce1a640 1 --2- 172.21.15.62:0/101927393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c48101420 0x7f2c4810d950 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.094+0000 7f2c4ce1a640 1 --2- 172.21.15.62:0/101927393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c48100a80 0x7f2c48100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.094+0000 7f2c4ce1a640 1 --2- 172.21.15.62:0/101927393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c48106a80 0x7f2c48106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.094+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/101927393 >> 172.21.15.62:0/101927393 conn(0x7f2c480fc7a0 msgr2=0x7f2c480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.094+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/101927393 shutdown_connections 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.094+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/101927393 wait complete. 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c4ce1a640 1 Processor -- start 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c4ce1a640 1 -- start start 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c4ce1a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c48100a80 0x7f2c48197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c4ce1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c48101420 0x7f2c48197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c4ce1a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c48106a80 0x7f2c4819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c4ce1a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2c48110e80 con 0x7f2c48101420 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c4ce1a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c48110d00 con 0x7f2c48106a80 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c4ce1a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2c48111000 con 0x7f2c48100a80 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c3ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c48101420 0x7f2c48197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c3ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c48101420 0x7f2c48197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44544/0 (socket says 172.21.15.62:44544) 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c3ffff640 1 -- 172.21.15.62:0/1825872562 learned_addr learned my addr 172.21.15.62:0/1825872562 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c4659c640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c48100a80 0x7f2c48197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c46d9d640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c48106a80 0x7f2c4819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c3ffff640 1 -- 172.21.15.62:0/1825872562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c48100a80 msgr2=0x7f2c48197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c3ffff640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c48100a80 0x7f2c48197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c3ffff640 1 -- 172.21.15.62:0/1825872562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c48106a80 msgr2=0x7f2c4819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c3ffff640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c48106a80 0x7f2c4819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.095+0000 7f2c3ffff640 1 -- 172.21.15.62:0/1825872562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4819c2b0 con 0x7f2c48101420 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.096+0000 7f2c3ffff640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c48101420 0x7f2c48197840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000d770 tx=0x7f2c3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.096+0000 7f2c3f7fe640 1 -- 172.21.15.62:0/1825872562 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30017070 con 0x7f2c48101420 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.096+0000 7f2c3f7fe640 1 -- 172.21.15.62:0/1825872562 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c30005500 con 0x7f2c48101420 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.096+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/1825872562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4819c5a0 con 0x7f2c48101420 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.096+0000 7f2c3f7fe640 1 -- 172.21.15.62:0/1825872562 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30004e00 con 0x7f2c48101420 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.096+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/1825872562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c481a3e80 con 0x7f2c48101420 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.097+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/1825872562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c48100ee0 con 0x7f2c48101420 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.098+0000 7f2c3f7fe640 1 -- 172.21.15.62:0/1825872562 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2c30004fa0 con 0x7f2c48101420 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.099+0000 7f2c3f7fe640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2c340a2c10 0x7f2c340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.099+0000 7f2c4659c640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2c340a2c10 0x7f2c340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.099+0000 7f2c3f7fe640 1 -- 172.21.15.62:0/1825872562 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f2c300cef20 con 0x7f2c48101420 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.100+0000 7f2c4659c640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2c340a2c10 0x7f2c340a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c38005d00 tx=0x7f2c3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.102+0000 7f2c3f7fe640 1 -- 172.21.15.62:0/1825872562 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c3009b780 con 0x7f2c48101420 2024-09-11T23:50:26.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:25 smithi083 ceph-mon[34805]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-11T23:50:26.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:25 smithi083 ceph-mon[34805]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T23:50:26.238 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:25 smithi120 ceph-mon[35182]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-11T23:50:26.238 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:25 smithi120 ceph-mon[35182]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T23:50:26.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.304+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/1825872562 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2c48062630 con 0x7f2c340a2c10 2024-09-11T23:50:26.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.324+0000 7f2c3f7fe640 1 -- 172.21.15.62:0/1825872562 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2c48062630 con 0x7f2c340a2c10 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.326+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/1825872562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2c340a2c10 msgr2=0x7f2c340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.326+0000 7f2c4ce1a640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2c340a2c10 0x7f2c340a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c38005d00 tx=0x7f2c3800b410 comp rx=0 tx=0).stop 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.327+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/1825872562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c48101420 msgr2=0x7f2c48197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.327+0000 7f2c4ce1a640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c48101420 0x7f2c48197840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000d770 tx=0x7f2c3000dc30 comp rx=0 tx=0).stop 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.327+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/1825872562 shutdown_connections 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.327+0000 7f2c4ce1a640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2c340a2c10 0x7f2c340a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.327+0000 7f2c4ce1a640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c48106a80 0x7f2c4819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.327+0000 7f2c4ce1a640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c48101420 0x7f2c48197840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.327+0000 7f2c4ce1a640 1 --2- 172.21.15.62:0/1825872562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c48100a80 0x7f2c48197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.327+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/1825872562 >> 172.21.15.62:0/1825872562 conn(0x7f2c480fc7a0 msgr2=0x7f2c480fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.327+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/1825872562 shutdown_connections 2024-09-11T23:50:26.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:26.327+0000 7f2c4ce1a640 1 -- 172.21.15.62:0/1825872562 wait complete. 2024-09-11T23:50:27.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:26 smithi062 ceph-mon[30958]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:27.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T23:50:27.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:26 smithi083 ceph-mon[34805]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:27.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:26 smithi120 ceph-mon[35182]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:27.391 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.852+0000 7f4459868640 1 -- 172.21.15.62:0/1685362176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4454102570 msgr2=0x7f44541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.852+0000 7f4459868640 1 --2- 172.21.15.62:0/1685362176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4454102570 0x7f44541029b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4448009900 tx=0x7f444802a480 comp rx=0 tx=0).stop 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.852+0000 7f4459868640 1 -- 172.21.15.62:0/1685362176 shutdown_connections 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.852+0000 7f4459868640 1 --2- 172.21.15.62:0/1685362176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4454102ef0 0x7f445410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.852+0000 7f4459868640 1 --2- 172.21.15.62:0/1685362176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4454102570 0x7f44541029b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.852+0000 7f4459868640 1 --2- 172.21.15.62:0/1685362176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4454100e20 0x7f4454101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.852+0000 7f4459868640 1 -- 172.21.15.62:0/1685362176 >> 172.21.15.62:0/1685362176 conn(0x7f44540fc7d0 msgr2=0x7f44540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.852+0000 7f4459868640 1 -- 172.21.15.62:0/1685362176 shutdown_connections 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.852+0000 7f4459868640 1 -- 172.21.15.62:0/1685362176 wait complete. 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4459868640 1 Processor -- start 2024-09-11T23:50:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4459868640 1 -- start start 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4459868640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4454100e20 0x7f44540fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4459868640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4454102570 0x7f44540fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4459868640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4454102ef0 0x7f44540fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4459868640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4454110f80 con 0x7f4454100e20 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4459868640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4454110e00 con 0x7f4454102ef0 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4459868640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4454111100 con 0x7f4454102570 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4452ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4454100e20 0x7f44540fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4452ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4454100e20 0x7f44540fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44552/0 (socket says 172.21.15.62:44552) 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f4452ffd640 1 -- 172.21.15.62:0/1026863037 learned_addr learned my addr 172.21.15.62:0/1026863037 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f44527fc640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4454102570 0x7f44540fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.853+0000 7f44537fe640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4454102ef0 0x7f44540fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4452ffd640 1 -- 172.21.15.62:0/1026863037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4454102570 msgr2=0x7f44540fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4452ffd640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4454102570 0x7f44540fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4452ffd640 1 -- 172.21.15.62:0/1026863037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4454102ef0 msgr2=0x7f44540fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4452ffd640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4454102ef0 0x7f44540fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4452ffd640 1 -- 172.21.15.62:0/1026863037 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44541a69a0 con 0x7f4454100e20 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f44537fe640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4454102ef0 0x7f44540fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f44527fc640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4454102570 0x7f44540fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4452ffd640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4454100e20 0x7f44540fcc80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f444400c890 tx=0x7f444400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4458866640 1 -- 172.21.15.62:0/1026863037 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444004220 con 0x7f4454100e20 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4458866640 1 -- 172.21.15.62:0/1026863037 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44440043c0 con 0x7f4454100e20 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4459868640 1 -- 172.21.15.62:0/1026863037 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44541a6c90 con 0x7f4454100e20 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4458866640 1 -- 172.21.15.62:0/1026863037 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44440049e0 con 0x7f4454100e20 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.854+0000 7f4459868640 1 -- 172.21.15.62:0/1026863037 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44541a7180 con 0x7f4454100e20 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.855+0000 7f4459868640 1 -- 172.21.15.62:0/1026863037 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4454101200 con 0x7f4454100e20 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.856+0000 7f4458866640 1 -- 172.21.15.62:0/1026863037 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f44440079b0 con 0x7f4454100e20 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.857+0000 7f4458866640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f443c0a2c10 0x7f443c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.857+0000 7f44527fc640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f443c0a2c10 0x7f443c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.857+0000 7f4458866640 1 -- 172.21.15.62:0/1026863037 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f44440cf230 con 0x7f4454100e20 2024-09-11T23:50:27.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.857+0000 7f44527fc640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f443c0a2c10 0x7f443c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f44540fe710 tx=0x7f4448005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:27.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:27.861+0000 7f4458866640 1 -- 172.21.15.62:0/1026863037 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4444014030 con 0x7f4454100e20 2024-09-11T23:50:28.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.068+0000 7f4459868640 1 -- 172.21.15.62:0/1026863037 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4454062630 con 0x7f443c0a2c10 2024-09-11T23:50:28.068 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:27 smithi062 ceph-mon[30958]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-11T23:50:28.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.074+0000 7f4458866640 1 -- 172.21.15.62:0/1026863037 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4454062630 con 0x7f443c0a2c10 2024-09-11T23:50:28.074 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/qa/sv_0/0a897f11-b5d9-4c2f-9ca2-102ad6feb26c 2024-09-11T23:50:28.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 -- 172.21.15.62:0/1026863037 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f443c0a2c10 msgr2=0x7f443c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:28.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f443c0a2c10 0x7f443c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f44540fe710 tx=0x7f4448005a20 comp rx=0 tx=0).stop 2024-09-11T23:50:28.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 -- 172.21.15.62:0/1026863037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4454100e20 msgr2=0x7f44540fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:28.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4454100e20 0x7f44540fcc80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f444400c890 tx=0x7f444400cd50 comp rx=0 tx=0).stop 2024-09-11T23:50:28.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 -- 172.21.15.62:0/1026863037 shutdown_connections 2024-09-11T23:50:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f443c0a2c10 0x7f443c0a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4454102ef0 0x7f44540fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4454102570 0x7f44540fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 --2- 172.21.15.62:0/1026863037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4454100e20 0x7f44540fcc80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 -- 172.21.15.62:0/1026863037 >> 172.21.15.62:0/1026863037 conn(0x7f44540fc7d0 msgr2=0x7f445406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 -- 172.21.15.62:0/1026863037 shutdown_connections 2024-09-11T23:50:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:28.076+0000 7f4459868640 1 -- 172.21.15.62:0/1026863037 wait complete. 2024-09-11T23:50:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:27 smithi083 ceph-mon[34805]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-11T23:50:28.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:27 smithi120 ceph-mon[35182]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-11T23:50:28.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T23:50:29.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:28 smithi083 ceph-mon[34805]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:28 smithi062 ceph-mon[30958]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:29.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:28 smithi120 ceph-mon[35182]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.636+0000 7f9d44257640 1 -- 172.21.15.62:0/1039177369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d3c105ad0 msgr2=0x7f9d3c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.636+0000 7f9d44257640 1 --2- 172.21.15.62:0/1039177369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d3c105ad0 0x7f9d3c105eb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009900 tx=0x7f9d3802a410 comp rx=0 tx=0).stop 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.636+0000 7f9d44257640 1 -- 172.21.15.62:0/1039177369 shutdown_connections 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.636+0000 7f9d44257640 1 --2- 172.21.15.62:0/1039177369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c101b60 0x7f9d3c10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.636+0000 7f9d44257640 1 --2- 172.21.15.62:0/1039177369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d3c1011c0 0x7f9d3c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.636+0000 7f9d44257640 1 --2- 172.21.15.62:0/1039177369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d3c105ad0 0x7f9d3c105eb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.636+0000 7f9d44257640 1 -- 172.21.15.62:0/1039177369 >> 172.21.15.62:0/1039177369 conn(0x7f9d3c077210 msgr2=0x7f9d3c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.636+0000 7f9d44257640 1 -- 172.21.15.62:0/1039177369 shutdown_connections 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.636+0000 7f9d44257640 1 -- 172.21.15.62:0/1039177369 wait complete. 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.637+0000 7f9d44257640 1 Processor -- start 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.637+0000 7f9d44257640 1 -- start start 2024-09-11T23:50:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d44257640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d3c1011c0 0x7f9d3c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d44257640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d3c101b60 0x7f9d3c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d44257640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c105ad0 0x7f9d3c19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d44257640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c110ef0 con 0x7f9d3c1011c0 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d44257640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c110d70 con 0x7f9d3c105ad0 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d44257640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c111070 con 0x7f9d3c101b60 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d41ff3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d3c1011c0 0x7f9d3c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d41ff3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d3c1011c0 0x7f9d3c197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44568/0 (socket says 172.21.15.62:44568) 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d41ff3640 1 -- 172.21.15.62:0/306139738 learned_addr learned my addr 172.21.15.62:0/306139738 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d417f2640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d3c101b60 0x7f9d3c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d41ff3640 1 -- 172.21.15.62:0/306139738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d3c101b60 msgr2=0x7f9d3c197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d41ff3640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d3c101b60 0x7f9d3c197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d41ff3640 1 -- 172.21.15.62:0/306139738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c105ad0 msgr2=0x7f9d3c19bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d41ff3640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c105ad0 0x7f9d3c19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d41ff3640 1 -- 172.21.15.62:0/306139738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c19c300 con 0x7f9d3c1011c0 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d417f2640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d3c101b60 0x7f9d3c197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d41ff3640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d3c1011c0 0x7f9d3c197350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006010 tx=0x7f9d38003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d32ffd640 1 -- 172.21.15.62:0/306139738 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3803f070 con 0x7f9d3c1011c0 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d32ffd640 1 -- 172.21.15.62:0/306139738 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d38031e30 con 0x7f9d3c1011c0 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d44257640 1 -- 172.21.15.62:0/306139738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3c19c590 con 0x7f9d3c1011c0 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d32ffd640 1 -- 172.21.15.62:0/306139738 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38035030 con 0x7f9d3c1011c0 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.638+0000 7f9d44257640 1 -- 172.21.15.62:0/306139738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3c1a3ed0 con 0x7f9d3c1011c0 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.639+0000 7f9d44257640 1 -- 172.21.15.62:0/306139738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d3c101620 con 0x7f9d3c1011c0 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.640+0000 7f9d32ffd640 1 -- 172.21.15.62:0/306139738 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9d38042070 con 0x7f9d3c1011c0 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.641+0000 7f9d32ffd640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d280a2c10 0x7f9d280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:29.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.641+0000 7f9d417f2640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d280a2c10 0x7f9d280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:29.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.641+0000 7f9d32ffd640 1 -- 172.21.15.62:0/306139738 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f9d380efa20 con 0x7f9d3c1011c0 2024-09-11T23:50:29.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.642+0000 7f9d417f2640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d280a2c10 0x7f9d280a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c006fd0 tx=0x7f9d2c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:29.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.645+0000 7f9d32ffd640 1 -- 172.21.15.62:0/306139738 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d380bc9d0 con 0x7f9d3c1011c0 2024-09-11T23:50:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.845+0000 7f9d44257640 1 -- 172.21.15.62:0/306139738 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9d3c104190 con 0x7f9d280a2c10 2024-09-11T23:50:29.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.869+0000 7f9d32ffd640 1 -- 172.21.15.62:0/306139738 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d3c104190 con 0x7f9d280a2c10 2024-09-11T23:50:29.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 -- 172.21.15.62:0/306139738 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d280a2c10 msgr2=0x7f9d280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:29.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d280a2c10 0x7f9d280a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c006fd0 tx=0x7f9d2c008040 comp rx=0 tx=0).stop 2024-09-11T23:50:29.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 -- 172.21.15.62:0/306139738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d3c1011c0 msgr2=0x7f9d3c197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:29.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d3c1011c0 0x7f9d3c197350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006010 tx=0x7f9d38003c80 comp rx=0 tx=0).stop 2024-09-11T23:50:29.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 -- 172.21.15.62:0/306139738 shutdown_connections 2024-09-11T23:50:29.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f9d280a2c10 0x7f9d280a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:29.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d3c105ad0 0x7f9d3c19bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:29.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d3c101b60 0x7f9d3c197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:29.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 --2- 172.21.15.62:0/306139738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d3c1011c0 0x7f9d3c197350 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:29.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 -- 172.21.15.62:0/306139738 >> 172.21.15.62:0/306139738 conn(0x7f9d3c077210 msgr2=0x7f9d3c103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:29.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 -- 172.21.15.62:0/306139738 shutdown_connections 2024-09-11T23:50:29.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:29.871+0000 7f9d44257640 1 -- 172.21.15.62:0/306139738 wait complete. 2024-09-11T23:50:30.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:29 smithi083 ceph-mon[34805]: pgmap v289: 97 pgs: 97 active+clean; 602 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-11T23:50:30.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:29 smithi062 ceph-mon[30958]: pgmap v289: 97 pgs: 97 active+clean; 602 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-11T23:50:30.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:29 smithi120 ceph-mon[35182]: pgmap v289: 97 pgs: 97 active+clean; 602 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-11T23:50:30.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T23:50:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:31.166 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:30 smithi062 ceph-mon[30958]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:31.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:30 smithi083 ceph-mon[34805]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:31.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.254+0000 7f94b6502640 1 -- 172.21.15.62:0/576786922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b0102ef0 msgr2=0x7f94b010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:31.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.254+0000 7f94b6502640 1 --2- 172.21.15.62:0/576786922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b0102ef0 0x7f94b010da40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009ca0 tx=0x7f94a802a950 comp rx=0 tx=0).stop 2024-09-11T23:50:31.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.254+0000 7f94b6502640 1 -- 172.21.15.62:0/576786922 shutdown_connections 2024-09-11T23:50:31.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.254+0000 7f94b6502640 1 --2- 172.21.15.62:0/576786922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b0102ef0 0x7f94b010da40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:31.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.254+0000 7f94b6502640 1 --2- 172.21.15.62:0/576786922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b0102570 0x7f94b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.254+0000 7f94b6502640 1 --2- 172.21.15.62:0/576786922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0100e20 0x7f94b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.254+0000 7f94b6502640 1 -- 172.21.15.62:0/576786922 >> 172.21.15.62:0/576786922 conn(0x7f94b00fc7d0 msgr2=0x7f94b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.254+0000 7f94b6502640 1 -- 172.21.15.62:0/576786922 shutdown_connections 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.254+0000 7f94b6502640 1 -- 172.21.15.62:0/576786922 wait complete. 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94b6502640 1 Processor -- start 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94b6502640 1 -- start start 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94b6502640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0100e20 0x7f94b00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94b6502640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b0102570 0x7f94b00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94b6502640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b0102ef0 0x7f94b00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94b6502640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0110fd0 con 0x7f94b0102570 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94b6502640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0110e50 con 0x7f94b0100e20 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94b6502640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0111150 con 0x7f94b0102ef0 2024-09-11T23:50:31.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b0102570 0x7f94b00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b0102570 0x7f94b00fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51962/0 (socket says 172.21.15.62:51962) 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94a7fff640 1 -- 172.21.15.62:0/3225648357 learned_addr learned my addr 172.21.15.62:0/3225648357 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94b4a9f640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b0102ef0 0x7f94b00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94affff640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0100e20 0x7f94b00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94a7fff640 1 -- 172.21.15.62:0/3225648357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b0102ef0 msgr2=0x7f94b00fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94a7fff640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b0102ef0 0x7f94b00fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94a7fff640 1 -- 172.21.15.62:0/3225648357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0100e20 msgr2=0x7f94b00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94a7fff640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0100e20 0x7f94b00fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94a7fff640 1 -- 172.21.15.62:0/3225648357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b01a6860 con 0x7f94b0102570 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.255+0000 7f94a7fff640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b0102570 0x7f94b00fd1c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f949800c920 tx=0x7f949800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.256+0000 7f94adffb640 1 -- 172.21.15.62:0/3225648357 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498007c70 con 0x7f94b0102570 2024-09-11T23:50:31.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.256+0000 7f94adffb640 1 -- 172.21.15.62:0/3225648357 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9498007e10 con 0x7f94b0102570 2024-09-11T23:50:31.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.256+0000 7f94b6502640 1 -- 172.21.15.62:0/3225648357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b01a6b50 con 0x7f94b0102570 2024-09-11T23:50:31.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.256+0000 7f94adffb640 1 -- 172.21.15.62:0/3225648357 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498018890 con 0x7f94b0102570 2024-09-11T23:50:31.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.256+0000 7f94b6502640 1 -- 172.21.15.62:0/3225648357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b01a70e0 con 0x7f94b0102570 2024-09-11T23:50:31.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.257+0000 7f94b6502640 1 -- 172.21.15.62:0/3225648357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b0101200 con 0x7f94b0102570 2024-09-11T23:50:31.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.258+0000 7f94adffb640 1 -- 172.21.15.62:0/3225648357 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9498018a70 con 0x7f94b0102570 2024-09-11T23:50:31.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.259+0000 7f94adffb640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f949c0a2c90 0x7f949c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:31.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.259+0000 7f94affff640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f949c0a2c90 0x7f949c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:31.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.259+0000 7f94adffb640 1 -- 172.21.15.62:0/3225648357 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f94980cf220 con 0x7f94b0102570 2024-09-11T23:50:31.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.259+0000 7f94affff640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f949c0a2c90 0x7f949c0a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005fd0 tx=0x7f94a0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:31.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.263+0000 7f94adffb640 1 -- 172.21.15.62:0/3225648357 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9498014030 con 0x7f94b0102570 2024-09-11T23:50:31.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:30 smithi120 ceph-mon[35182]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.471+0000 7f94b6502640 1 -- 172.21.15.62:0/3225648357 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f94b006a7c0 con 0x7f949c0a2c90 2024-09-11T23:50:31.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.476+0000 7f94adffb640 1 -- 172.21.15.62:0/3225648357 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f94b006a7c0 con 0x7f949c0a2c90 2024-09-11T23:50:31.475 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/qa/sv_1/324e6219-dcd5-475c-bde4-eff522131966 2024-09-11T23:50:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 -- 172.21.15.62:0/3225648357 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f949c0a2c90 msgr2=0x7f949c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f949c0a2c90 0x7f949c0a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005fd0 tx=0x7f94a0005850 comp rx=0 tx=0).stop 2024-09-11T23:50:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 -- 172.21.15.62:0/3225648357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b0102570 msgr2=0x7f94b00fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b0102570 0x7f94b00fd1c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f949800c920 tx=0x7f949800cde0 comp rx=0 tx=0).stop 2024-09-11T23:50:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 -- 172.21.15.62:0/3225648357 shutdown_connections 2024-09-11T23:50:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f949c0a2c90 0x7f949c0a5150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94b0102ef0 0x7f94b00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b0102570 0x7f94b00fd1c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 --2- 172.21.15.62:0/3225648357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0100e20 0x7f94b00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 -- 172.21.15.62:0/3225648357 >> 172.21.15.62:0/3225648357 conn(0x7f94b00fc7d0 msgr2=0x7f94b006a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 -- 172.21.15.62:0/3225648357 shutdown_connections 2024-09-11T23:50:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:31.478+0000 7f94b6502640 1 -- 172.21.15.62:0/3225648357 wait complete. 2024-09-11T23:50:32.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:31 smithi062 ceph-mon[30958]: pgmap v290: 97 pgs: 97 active+clean; 602 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-11T23:50:32.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:31 smithi062 ceph-mon[30958]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:32.075 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:50:32.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:31 smithi083 ceph-mon[34805]: pgmap v290: 97 pgs: 97 active+clean; 602 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-11T23:50:32.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:31 smithi083 ceph-mon[34805]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:32.331 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:32.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:31 smithi120 ceph-mon[35182]: pgmap v290: 97 pgs: 97 active+clean; 602 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-11T23:50:32.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:31 smithi120 ceph-mon[35182]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:50:32.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.807+0000 7f5790019640 1 -- 172.21.15.62:0/100291032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5788102d40 msgr2=0x7f5788103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:32.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.807+0000 7f5790019640 1 --2- 172.21.15.62:0/100291032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5788102d40 0x7f5788103120 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5784009900 tx=0x7f578402a480 comp rx=0 tx=0).stop 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.807+0000 7f5790019640 1 -- 172.21.15.62:0/100291032 shutdown_connections 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.807+0000 7f5790019640 1 --2- 172.21.15.62:0/100291032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f578810c170 0x7f578810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.807+0000 7f5790019640 1 --2- 172.21.15.62:0/100291032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5788103660 0x7f578810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.807+0000 7f5790019640 1 --2- 172.21.15.62:0/100291032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5788102d40 0x7f5788103120 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.807+0000 7f5790019640 1 -- 172.21.15.62:0/100291032 >> 172.21.15.62:0/100291032 conn(0x7f57880fc7a0 msgr2=0x7f57880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.807+0000 7f5790019640 1 -- 172.21.15.62:0/100291032 shutdown_connections 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.807+0000 7f5790019640 1 -- 172.21.15.62:0/100291032 wait complete. 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f5790019640 1 Processor -- start 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f5790019640 1 -- start start 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f5790019640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5788102d40 0x7f5788198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f5790019640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5788103660 0x7f5788198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f5790019640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f578810c170 0x7f578819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:32.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f5790019640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5788111f70 con 0x7f5788102d40 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f5790019640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5788111df0 con 0x7f5788103660 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f5790019640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57881120f0 con 0x7f578810c170 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f578ddb5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5788102d40 0x7f5788198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f578ddb5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5788102d40 0x7f5788198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51986/0 (socket says 172.21.15.62:51986) 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f578ddb5640 1 -- 172.21.15.62:0/3752050124 learned_addr learned my addr 172.21.15.62:0/3752050124 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.808+0000 7f578d5b4640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5788103660 0x7f5788198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f578e5b6640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f578810c170 0x7f578819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f578ddb5640 1 -- 172.21.15.62:0/3752050124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f578810c170 msgr2=0x7f578819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f578ddb5640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f578810c170 0x7f578819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f578ddb5640 1 -- 172.21.15.62:0/3752050124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5788103660 msgr2=0x7f5788198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f578ddb5640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5788103660 0x7f5788198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f578ddb5640 1 -- 172.21.15.62:0/3752050124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578819d400 con 0x7f5788102d40 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f578e5b6640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f578810c170 0x7f578819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f578ddb5640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5788102d40 0x7f5788198450 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5784006010 tx=0x7f57840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f577effd640 1 -- 172.21.15.62:0/3752050124 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578403f030 con 0x7f5788102d40 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f5790019640 1 -- 172.21.15.62:0/3752050124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578819d690 con 0x7f5788102d40 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f577effd640 1 -- 172.21.15.62:0/3752050124 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5784003ce0 con 0x7f5788102d40 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f577effd640 1 -- 172.21.15.62:0/3752050124 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784035070 con 0x7f5788102d40 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.809+0000 7f5790019640 1 -- 172.21.15.62:0/3752050124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57881a4fd0 con 0x7f5788102d40 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.810+0000 7f5790019640 1 -- 172.21.15.62:0/3752050124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f578806b070 con 0x7f5788102d40 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.811+0000 7f577effd640 1 -- 172.21.15.62:0/3752050124 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5784042030 con 0x7f5788102d40 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.812+0000 7f577effd640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57740a2c10 0x7f57740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.812+0000 7f578d5b4640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57740a2c10 0x7f57740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.812+0000 7f577effd640 1 -- 172.21.15.62:0/3752050124 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f57840ef1e0 con 0x7f5788102d40 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.812+0000 7f578d5b4640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57740a2c10 0x7f57740a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f578810d5b0 tx=0x7f577800b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:32.816+0000 7f577effd640 1 -- 172.21.15.62:0/3752050124 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57840bc250 con 0x7f5788102d40 2024-09-11T23:50:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.029+0000 7f5790019640 1 -- 172.21.15.62:0/3752050124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5788107210 con 0x7f5788102d40 2024-09-11T23:50:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.030+0000 7f577effd640 1 -- 172.21.15.62:0/3752050124 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11909 (secure 0 0 0) 0x7f57840bbb50 con 0x7f5788102d40 2024-09-11T23:50:33.030 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:33.030 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"btime":"2024-09-11T23:50:22:864278+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:50:22.553275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:50:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2024-09-11T23:50:33.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.031+0000 7f5790019640 1 -- 172.21.15.62:0/3752050124 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57740a2c10 msgr2=0x7f57740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:33.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.031+0000 7f5790019640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57740a2c10 0x7f57740a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f578810d5b0 tx=0x7f577800b2e0 comp rx=0 tx=0).stop 2024-09-11T23:50:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.032+0000 7f5790019640 1 -- 172.21.15.62:0/3752050124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5788102d40 msgr2=0x7f5788198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.032+0000 7f5790019640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5788102d40 0x7f5788198450 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5784006010 tx=0x7f57840042f0 comp rx=0 tx=0).stop 2024-09-11T23:50:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.032+0000 7f5790019640 1 -- 172.21.15.62:0/3752050124 shutdown_connections 2024-09-11T23:50:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.032+0000 7f5790019640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f57740a2c10 0x7f57740a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.032+0000 7f5790019640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f578810c170 0x7f578819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.032+0000 7f5790019640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5788103660 0x7f5788198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.032+0000 7f5790019640 1 --2- 172.21.15.62:0/3752050124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5788102d40 0x7f5788198450 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.032+0000 7f5790019640 1 -- 172.21.15.62:0/3752050124 >> 172.21.15.62:0/3752050124 conn(0x7f57880fc7a0 msgr2=0x7f57880fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.032+0000 7f5790019640 1 -- 172.21.15.62:0/3752050124 shutdown_connections 2024-09-11T23:50:33.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:33.032+0000 7f5790019640 1 -- 172.21.15.62:0/3752050124 wait complete. 2024-09-11T23:50:33.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:50:34.083 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:34.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:33 smithi083 ceph-mon[34805]: pgmap v291: 97 pgs: 97 active+clean; 602 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-11T23:50:34.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:33 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3752050124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:50:34.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:33 smithi062 ceph-mon[30958]: pgmap v291: 97 pgs: 97 active+clean; 602 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-11T23:50:34.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:33 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3752050124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:50:34.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:33 smithi120 ceph-mon[35182]: pgmap v291: 97 pgs: 97 active+clean; 602 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-11T23:50:34.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:33 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3752050124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:50:34.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.528+0000 7f6056f58640 1 -- 172.21.15.62:0/2182437590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6050100a90 msgr2=0x7f6050100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:34.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.528+0000 7f6056f58640 1 --2- 172.21.15.62:0/2182437590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6050100a90 0x7f6050100ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f603c009930 tx=0x7f603c02a420 comp rx=0 tx=0).stop 2024-09-11T23:50:34.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.528+0000 7f6056f58640 1 -- 172.21.15.62:0/2182437590 shutdown_connections 2024-09-11T23:50:34.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.528+0000 7f6056f58640 1 --2- 172.21.15.62:0/2182437590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6050101430 0x7f605010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:34.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.528+0000 7f6056f58640 1 --2- 172.21.15.62:0/2182437590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6050100a90 0x7f6050100ef0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.528+0000 7f6056f58640 1 --2- 172.21.15.62:0/2182437590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6050106a90 0x7f6050106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.528+0000 7f6056f58640 1 -- 172.21.15.62:0/2182437590 >> 172.21.15.62:0/2182437590 conn(0x7f60500fc7d0 msgr2=0x7f60500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.528+0000 7f6056f58640 1 -- 172.21.15.62:0/2182437590 shutdown_connections 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.528+0000 7f6056f58640 1 -- 172.21.15.62:0/2182437590 wait complete. 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f6056f58640 1 Processor -- start 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f6056f58640 1 -- start start 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f6056f58640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6050100a90 0x7f6050199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f6056f58640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6050101430 0x7f6050199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f6056f58640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6050106a90 0x7f60501a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f6056f58640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6050110f30 con 0x7f6050106a90 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f6056f58640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6050110db0 con 0x7f6050101430 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f6056f58640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60501110b0 con 0x7f6050100a90 2024-09-11T23:50:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f60554f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6050106a90 0x7f60501a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f60554f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6050106a90 0x7f60501a44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52008/0 (socket says 172.21.15.62:52008) 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f60554f5640 1 -- 172.21.15.62:0/1779984556 learned_addr learned my addr 172.21.15.62:0/1779984556 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f604ffff640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6050101430 0x7f6050199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f6054cf4640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6050100a90 0x7f6050199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f60554f5640 1 -- 172.21.15.62:0/1779984556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6050100a90 msgr2=0x7f6050199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f60554f5640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6050100a90 0x7f6050199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f60554f5640 1 -- 172.21.15.62:0/1779984556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6050101430 msgr2=0x7f6050199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f60554f5640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6050101430 0x7f6050199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f60554f5640 1 -- 172.21.15.62:0/1779984556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60501a4a00 con 0x7f6050106a90 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f6054cf4640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6050100a90 0x7f6050199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.529+0000 7f604ffff640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6050101430 0x7f6050199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:50:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.530+0000 7f60554f5640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6050106a90 0x7f60501a44c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f604800b970 tx=0x7f604800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.530+0000 7f604dffb640 1 -- 172.21.15.62:0/1779984556 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048019070 con 0x7f6050106a90 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.530+0000 7f6056f58640 1 -- 172.21.15.62:0/1779984556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60501a4bd0 con 0x7f6050106a90 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.530+0000 7f604dffb640 1 -- 172.21.15.62:0/1779984556 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f604800cb40 con 0x7f6050106a90 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.530+0000 7f604dffb640 1 -- 172.21.15.62:0/1779984556 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048007650 con 0x7f6050106a90 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.530+0000 7f6056f58640 1 -- 172.21.15.62:0/1779984556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60501a5090 con 0x7f6050106a90 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.531+0000 7f6056f58640 1 -- 172.21.15.62:0/1779984556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6050100f70 con 0x7f6050106a90 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.532+0000 7f604dffb640 1 -- 172.21.15.62:0/1779984556 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f604801c070 con 0x7f6050106a90 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.533+0000 7f604dffb640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f60400a2c10 0x7f60400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.533+0000 7f6054cf4640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f60400a2c10 0x7f60400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.533+0000 7f604dffb640 1 -- 172.21.15.62:0/1779984556 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f60480ce560 con 0x7f6050106a90 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.533+0000 7f6054cf4640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f60400a2c10 0x7f60400a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6044006fd0 tx=0x7f6044008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.537+0000 7f604dffb640 1 -- 172.21.15.62:0/1779984556 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f604809b620 con 0x7f6050106a90 2024-09-11T23:50:34.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.715+0000 7f6056f58640 1 -- 172.21.15.62:0/1779984556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f605002d7c0 con 0x7f6050106a90 2024-09-11T23:50:34.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.716+0000 7f604dffb640 1 -- 172.21.15.62:0/1779984556 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22244 (secure 0 0 0) 0x7f604809af20 con 0x7f6050106a90 2024-09-11T23:50:34.717 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:34.717 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:50:11.406572+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:50:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.718+0000 7f6056f58640 1 -- 172.21.15.62:0/1779984556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f60400a2c10 msgr2=0x7f60400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.718+0000 7f6056f58640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f60400a2c10 0x7f60400a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6044006fd0 tx=0x7f6044008040 comp rx=0 tx=0).stop 2024-09-11T23:50:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.718+0000 7f6056f58640 1 -- 172.21.15.62:0/1779984556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6050106a90 msgr2=0x7f60501a44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.718+0000 7f6056f58640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6050106a90 0x7f60501a44c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f604800b970 tx=0x7f604800be30 comp rx=0 tx=0).stop 2024-09-11T23:50:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.719+0000 7f6056f58640 1 -- 172.21.15.62:0/1779984556 shutdown_connections 2024-09-11T23:50:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.719+0000 7f6056f58640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f60400a2c10 0x7f60400a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.719+0000 7f6056f58640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6050106a90 0x7f60501a44c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.719+0000 7f6056f58640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6050101430 0x7f6050199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.719+0000 7f6056f58640 1 --2- 172.21.15.62:0/1779984556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6050100a90 0x7f6050199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:34.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.719+0000 7f6056f58640 1 -- 172.21.15.62:0/1779984556 >> 172.21.15.62:0/1779984556 conn(0x7f60500fc7d0 msgr2=0x7f60500fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:34.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.719+0000 7f6056f58640 1 -- 172.21.15.62:0/1779984556 shutdown_connections 2024-09-11T23:50:34.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:34.719+0000 7f6056f58640 1 -- 172.21.15.62:0/1779984556 wait complete. 2024-09-11T23:50:35.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:34 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1779984556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:35.209 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:34 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1779984556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:35.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:34 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1779984556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:35.484 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:50:35.484 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:50:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:36.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:35 smithi120 ceph-mon[35182]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-11T23:50:36.136 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:35 smithi062 ceph-mon[30958]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-11T23:50:36.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:35 smithi083 ceph-mon[34805]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-11T23:50:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/2584627639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b0410c180 msgr2=0x7f3b0410e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 --2- 172.21.15.62:0/2584627639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b0410c180 0x7f3b0410e620 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009ca0 tx=0x7f3afc02a950 comp rx=0 tx=0).stop 2024-09-11T23:50:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/2584627639 shutdown_connections 2024-09-11T23:50:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 --2- 172.21.15.62:0/2584627639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b0410c180 0x7f3b0410e620 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 --2- 172.21.15.62:0/2584627639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04103670 0x7f3b0410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 --2- 172.21.15.62:0/2584627639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b04102d50 0x7f3b04103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/2584627639 >> 172.21.15.62:0/2584627639 conn(0x7f3b040fc7d0 msgr2=0x7f3b040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/2584627639 shutdown_connections 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/2584627639 wait complete. 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 Processor -- start 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.219+0000 7f3b0a2ca640 1 -- start start 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b0a2ca640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b04102d50 0x7f3b041009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b0a2ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04103670 0x7f3b04100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b0a2ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b0410c180 0x7f3b04101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b0a2ca640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3b04111fe0 con 0x7f3b04102d50 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b0a2ca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b04111e60 con 0x7f3b04103670 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b0a2ca640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b04112160 con 0x7f3b0410c180 2024-09-11T23:50:36.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b04102d50 0x7f3b041009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b04102d50 0x7f3b041009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52020/0 (socket says 172.21.15.62:52020) 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b03fff640 1 -- 172.21.15.62:0/1308084874 learned_addr learned my addr 172.21.15.62:0/1308084874 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b08867640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b0410c180 0x7f3b04101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b03fff640 1 -- 172.21.15.62:0/1308084874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b0410c180 msgr2=0x7f3b04101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b037fe640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04103670 0x7f3b04100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b03fff640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b0410c180 0x7f3b04101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b03fff640 1 -- 172.21.15.62:0/1308084874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04103670 msgr2=0x7f3b04100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b03fff640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04103670 0x7f3b04100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b03fff640 1 -- 172.21.15.62:0/1308084874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b041a6960 con 0x7f3b04102d50 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b08867640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b0410c180 0x7f3b04101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b037fe640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04103670 0x7f3b04100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:36.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.220+0000 7f3b03fff640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b04102d50 0x7f3b041009b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3af400d770 tx=0x7f3af400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.221+0000 7f3b017fa640 1 -- 172.21.15.62:0/1308084874 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4017070 con 0x7f3b04102d50 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.221+0000 7f3b017fa640 1 -- 172.21.15.62:0/1308084874 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af4005500 con 0x7f3b04102d50 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.221+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/1308084874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b041a6c50 con 0x7f3b04102d50 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.221+0000 7f3b017fa640 1 -- 172.21.15.62:0/1308084874 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4004e00 con 0x7f3b04102d50 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.221+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/1308084874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b041a7140 con 0x7f3b04102d50 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.222+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/1308084874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b04103130 con 0x7f3b04102d50 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.223+0000 7f3b017fa640 1 -- 172.21.15.62:0/1308084874 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3af4004fa0 con 0x7f3b04102d50 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.223+0000 7f3b017fa640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3aec0a2c10 0x7f3aec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.224+0000 7f3b037fe640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3aec0a2c10 0x7f3aec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.224+0000 7f3b017fa640 1 -- 172.21.15.62:0/1308084874 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f3af40ceb50 con 0x7f3b04102d50 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.224+0000 7f3b037fe640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3aec0a2c10 0x7f3aec0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3af8006fd0 tx=0x7f3af8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:36.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.228+0000 7f3b017fa640 1 -- 172.21.15.62:0/1308084874 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3af409bb00 con 0x7f3b04102d50 2024-09-11T23:50:36.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.400+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/1308084874 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b040feaf0 con 0x7f3aec0a2c10 2024-09-11T23:50:36.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.407+0000 7f3b017fa640 1 -- 172.21.15.62:0/1308084874 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258864 (secure 0 0 0) 0x7f3b040feaf0 con 0x7f3aec0a2c10 2024-09-11T23:50:36.410 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:36.411 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:50:36.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/1308084874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3aec0a2c10 msgr2=0x7f3aec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3aec0a2c10 0x7f3aec0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3af8006fd0 tx=0x7f3af8008040 comp rx=0 tx=0).stop 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/1308084874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b04102d50 msgr2=0x7f3b041009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b04102d50 0x7f3b041009b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3af400d770 tx=0x7f3af400dc30 comp rx=0 tx=0).stop 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/1308084874 shutdown_connections 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f3aec0a2c10 0x7f3aec0a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b0410c180 0x7f3b04101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04103670 0x7f3b04100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 --2- 172.21.15.62:0/1308084874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b04102d50 0x7f3b041009b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/1308084874 >> 172.21.15.62:0/1308084874 conn(0x7f3b040fc7d0 msgr2=0x7f3b040fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/1308084874 shutdown_connections 2024-09-11T23:50:36.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:36.410+0000 7f3b0a2ca640 1 -- 172.21.15.62:0/1308084874 wait complete. 2024-09-11T23:50:37.060 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:36 smithi062 ceph-mon[30958]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:50:37.091 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-11T23:50:34.805642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641407,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":235,"num_read_kb":223,"num_write":333,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347480,"kb_used_data":10712,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594344,"statfs":{"total":1151940427776,"available":1151584608256,"internally_reserved":0,"allocated":10969088,"data_stored":6955620,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47111,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":93,"num_read_kb":98,"num_write":61,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002994"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830856+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:50:11.830856+0000","last_peered":"2024-09-11T23:50:11.830856+0000","last_clean":"2024-09-11T23:50:11.830856+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:50:11.830856+0000","last_undegraded":"2024-09-11T23:50:11.830856+0000","last_fullsized":"2024-09-11T23:50:11.830856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880987+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:50:11.880987+0000","last_peered":"2024-09-11T23:50:11.880987+0000","last_clean":"2024-09-11T23:50:11.880987+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:50:11.880987+0000","last_undegraded":"2024-09-11T23:50:11.880987+0000","last_fullsized":"2024-09-11T23:50:11.880987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880650+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:50:11.880650+0000","last_peered":"2024-09-11T23:50:11.880650+0000","last_clean":"2024-09-11T23:50:11.880650+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:50:11.880650+0000","last_undegraded":"2024-09-11T23:50:11.880650+0000","last_fullsized":"2024-09-11T23:50:11.880650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878400+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:50:11.878400+0000","last_peered":"2024-09-11T23:50:11.878400+0000","last_clean":"2024-09-11T23:50:11.878400+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:50:11.878400+0000","last_undegraded":"2024-09-11T23:50:11.878400+0000","last_fullsized":"2024-09-11T23:50:11.878400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876576+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:50:11.876576+0000","last_peered":"2024-09-11T23:50:11.876576+0000","last_clean":"2024-09-11T23:50:11.876576+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:50:11.876576+0000","last_undegraded":"2024-09-11T23:50:11.876576+0000","last_fullsized":"2024-09-11T23:50:11.876576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.828168+0000","last_change":"2024-09-11T23:50:10.842560+0000","last_active":"2024-09-11T23:50:11.828168+0000","last_peered":"2024-09-11T23:50:11.828168+0000","last_clean":"2024-09-11T23:50:11.828168+0000","last_became_active":"2024-09-11T23:50:10.842259+0000","last_became_peered":"2024-09-11T23:50:10.842259+0000","last_unstale":"2024-09-11T23:50:11.828168+0000","last_undegraded":"2024-09-11T23:50:11.828168+0000","last_fullsized":"2024-09-11T23:50:11.828168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:36.538402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881133+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:50:11.881133+0000","last_peered":"2024-09-11T23:50:11.881133+0000","last_clean":"2024-09-11T23:50:11.881133+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:50:11.881133+0000","last_undegraded":"2024-09-11T23:50:11.881133+0000","last_fullsized":"2024-09-11T23:50:11.881133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877161+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:50:11.877161+0000","last_peered":"2024-09-11T23:50:11.877161+0000","last_clean":"2024-09-11T23:50:11.877161+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:50:11.877161+0000","last_undegraded":"2024-09-11T23:50:11.877161+0000","last_fullsized":"2024-09-11T23:50:11.877161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878788+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:50:11.878788+0000","last_peered":"2024-09-11T23:50:11.878788+0000","last_clean":"2024-09-11T23:50:11.878788+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:50:11.878788+0000","last_undegraded":"2024-09-11T23:50:11.878788+0000","last_fullsized":"2024-09-11T23:50:11.878788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.828525+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:50:11.828525+0000","last_peered":"2024-09-11T23:50:11.828525+0000","last_clean":"2024-09-11T23:50:11.828525+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:50:11.828525+0000","last_undegraded":"2024-09-11T23:50:11.828525+0000","last_fullsized":"2024-09-11T23:50:11.828525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881588+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:50:11.881588+0000","last_peered":"2024-09-11T23:50:11.881588+0000","last_clean":"2024-09-11T23:50:11.881588+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:50:11.881588+0000","last_undegraded":"2024-09-11T23:50:11.881588+0000","last_fullsized":"2024-09-11T23:50:11.881588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878548+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:50:11.878548+0000","last_peered":"2024-09-11T23:50:11.878548+0000","last_clean":"2024-09-11T23:50:11.878548+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:50:11.878548+0000","last_undegraded":"2024-09-11T23:50:11.878548+0000","last_fullsized":"2024-09-11T23:50:11.878548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827687+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:50:11.827687+0000","last_peered":"2024-09-11T23:50:11.827687+0000","last_clean":"2024-09-11T23:50:11.827687+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:50:11.827687+0000","last_undegraded":"2024-09-11T23:50:11.827687+0000","last_fullsized":"2024-09-11T23:50:11.827687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880393+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:50:11.880393+0000","last_peered":"2024-09-11T23:50:11.880393+0000","last_clean":"2024-09-11T23:50:11.880393+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:50:11.880393+0000","last_undegraded":"2024-09-11T23:50:11.880393+0000","last_fullsized":"2024-09-11T23:50:11.880393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880060+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:50:11.880060+0000","last_peered":"2024-09-11T23:50:11.880060+0000","last_clean":"2024-09-11T23:50:11.880060+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:50:11.880060+0000","last_undegraded":"2024-09-11T23:50:11.880060+0000","last_fullsized":"2024-09-11T23:50:11.880060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877177+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:50:11.877177+0000","last_peered":"2024-09-11T23:50:11.877177+0000","last_clean":"2024-09-11T23:50:11.877177+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:50:11.877177+0000","last_undegraded":"2024-09-11T23:50:11.877177+0000","last_fullsized":"2024-09-11T23:50:11.877177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830826+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:50:11.830826+0000","last_peered":"2024-09-11T23:50:11.830826+0000","last_clean":"2024-09-11T23:50:11.830826+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:50:11.830826+0000","last_undegraded":"2024-09-11T23:50:11.830826+0000","last_fullsized":"2024-09-11T23:50:11.830826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876998+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:50:11.876998+0000","last_peered":"2024-09-11T23:50:11.876998+0000","last_clean":"2024-09-11T23:50:11.876998+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:50:11.876998+0000","last_undegraded":"2024-09-11T23:50:11.876998+0000","last_fullsized":"2024-09-11T23:50:11.876998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876734+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:50:11.876734+0000","last_peered":"2024-09-11T23:50:11.876734+0000","last_clean":"2024-09-11T23:50:11.876734+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:50:11.876734+0000","last_undegraded":"2024-09-11T23:50:11.876734+0000","last_fullsized":"2024-09-11T23:50:11.876734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830673+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:50:11.830673+0000","last_peered":"2024-09-11T23:50:11.830673+0000","last_clean":"2024-09-11T23:50:11.830673+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:50:11.830673+0000","last_undegraded":"2024-09-11T23:50:11.830673+0000","last_fullsized":"2024-09-11T23:50:11.830673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827752+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:50:11.827752+0000","last_peered":"2024-09-11T23:50:11.827752+0000","last_clean":"2024-09-11T23:50:11.827752+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:50:11.827752+0000","last_undegraded":"2024-09-11T23:50:11.827752+0000","last_fullsized":"2024-09-11T23:50:11.827752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877072+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:50:11.877072+0000","last_peered":"2024-09-11T23:50:11.877072+0000","last_clean":"2024-09-11T23:50:11.877072+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:50:11.877072+0000","last_undegraded":"2024-09-11T23:50:11.877072+0000","last_fullsized":"2024-09-11T23:50:11.877072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878562+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:50:11.878562+0000","last_peered":"2024-09-11T23:50:11.878562+0000","last_clean":"2024-09-11T23:50:11.878562+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:50:11.878562+0000","last_undegraded":"2024-09-11T23:50:11.878562+0000","last_fullsized":"2024-09-11T23:50:11.878562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882045+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:50:11.882045+0000","last_peered":"2024-09-11T23:50:11.882045+0000","last_clean":"2024-09-11T23:50:11.882045+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:50:11.882045+0000","last_undegraded":"2024-09-11T23:50:11.882045+0000","last_fullsized":"2024-09-11T23:50:11.882045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880422+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:50:11.880422+0000","last_peered":"2024-09-11T23:50:11.880422+0000","last_clean":"2024-09-11T23:50:11.880422+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:50:11.880422+0000","last_undegraded":"2024-09-11T23:50:11.880422+0000","last_fullsized":"2024-09-11T23:50:11.880422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877065+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:50:11.877065+0000","last_peered":"2024-09-11T23:50:11.877065+0000","last_clean":"2024-09-11T23:50:11.877065+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:50:11.877065+0000","last_undegraded":"2024-09-11T23:50:11.877065+0000","last_fullsized":"2024-09-11T23:50:11.877065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830611+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:50:11.830611+0000","last_peered":"2024-09-11T23:50:11.830611+0000","last_clean":"2024-09-11T23:50:11.830611+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:50:11.830611+0000","last_undegraded":"2024-09-11T23:50:11.830611+0000","last_fullsized":"2024-09-11T23:50:11.830611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827672+0000","last_change":"2024-09-11T23:50:10.845202+0000","last_active":"2024-09-11T23:50:11.827672+0000","last_peered":"2024-09-11T23:50:11.827672+0000","last_clean":"2024-09-11T23:50:11.827672+0000","last_became_active":"2024-09-11T23:50:10.844892+0000","last_became_peered":"2024-09-11T23:50:10.844892+0000","last_unstale":"2024-09-11T23:50:11.827672+0000","last_undegraded":"2024-09-11T23:50:11.827672+0000","last_fullsized":"2024-09-11T23:50:11.827672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:09.744773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880722+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:50:11.880722+0000","last_peered":"2024-09-11T23:50:11.880722+0000","last_clean":"2024-09-11T23:50:11.880722+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:50:11.880722+0000","last_undegraded":"2024-09-11T23:50:11.880722+0000","last_fullsized":"2024-09-11T23:50:11.880722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876440+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:50:11.876440+0000","last_peered":"2024-09-11T23:50:11.876440+0000","last_clean":"2024-09-11T23:50:11.876440+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:50:11.876440+0000","last_undegraded":"2024-09-11T23:50:11.876440+0000","last_fullsized":"2024-09-11T23:50:11.876440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877202+0000","last_change":"2024-09-11T23:50:10.843592+0000","last_active":"2024-09-11T23:50:11.877202+0000","last_peered":"2024-09-11T23:50:11.877202+0000","last_clean":"2024-09-11T23:50:11.877202+0000","last_became_active":"2024-09-11T23:50:10.843259+0000","last_became_peered":"2024-09-11T23:50:10.843259+0000","last_unstale":"2024-09-11T23:50:11.877202+0000","last_undegraded":"2024-09-11T23:50:11.877202+0000","last_fullsized":"2024-09-11T23:50:11.877202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:05.557977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830450+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:50:11.830450+0000","last_peered":"2024-09-11T23:50:11.830450+0000","last_clean":"2024-09-11T23:50:11.830450+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:50:11.830450+0000","last_undegraded":"2024-09-11T23:50:11.830450+0000","last_fullsized":"2024-09-11T23:50:11.830450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876539+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:50:11.876539+0000","last_peered":"2024-09-11T23:50:11.876539+0000","last_clean":"2024-09-11T23:50:11.876539+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:50:11.876539+0000","last_undegraded":"2024-09-11T23:50:11.876539+0000","last_fullsized":"2024-09-11T23:50:11.876539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:28.885041+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:50:28.885041+0000","last_peered":"2024-09-11T23:50:28.885041+0000","last_clean":"2024-09-11T23:50:28.885041+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:50:28.885041+0000","last_undegraded":"2024-09-11T23:50:28.885041+0000","last_fullsized":"2024-09-11T23:50:28.885041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880098+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:50:11.880098+0000","last_peered":"2024-09-11T23:50:11.880098+0000","last_clean":"2024-09-11T23:50:11.880098+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:50:11.880098+0000","last_undegraded":"2024-09-11T23:50:11.880098+0000","last_fullsized":"2024-09-11T23:50:11.880098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.866074+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:50:19.866074+0000","last_peered":"2024-09-11T23:50:19.866074+0000","last_clean":"2024-09-11T23:50:19.866074+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:50:19.866074+0000","last_undegraded":"2024-09-11T23:50:19.866074+0000","last_fullsized":"2024-09-11T23:50:19.866074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.866815+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:50:19.866815+0000","last_peered":"2024-09-11T23:50:19.866815+0000","last_clean":"2024-09-11T23:50:19.866815+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:50:19.866815+0000","last_undegraded":"2024-09-11T23:50:19.866815+0000","last_fullsized":"2024-09-11T23:50:19.866815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877320+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:50:11.877320+0000","last_peered":"2024-09-11T23:50:11.877320+0000","last_clean":"2024-09-11T23:50:11.877320+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:50:11.877320+0000","last_undegraded":"2024-09-11T23:50:11.877320+0000","last_fullsized":"2024-09-11T23:50:11.877320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882426+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:50:11.882426+0000","last_peered":"2024-09-11T23:50:11.882426+0000","last_clean":"2024-09-11T23:50:11.882426+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:50:11.882426+0000","last_undegraded":"2024-09-11T23:50:11.882426+0000","last_fullsized":"2024-09-11T23:50:11.882426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877143+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:50:11.877143+0000","last_peered":"2024-09-11T23:50:11.877143+0000","last_clean":"2024-09-11T23:50:11.877143+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:50:11.877143+0000","last_undegraded":"2024-09-11T23:50:11.877143+0000","last_fullsized":"2024-09-11T23:50:11.877143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:31.888120+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:50:31.888120+0000","last_peered":"2024-09-11T23:50:31.888120+0000","last_clean":"2024-09-11T23:50:31.888120+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:50:31.888120+0000","last_undegraded":"2024-09-11T23:50:31.888120+0000","last_fullsized":"2024-09-11T23:50:31.888120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878635+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:50:11.878635+0000","last_peered":"2024-09-11T23:50:11.878635+0000","last_clean":"2024-09-11T23:50:11.878635+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:50:11.878635+0000","last_undegraded":"2024-09-11T23:50:11.878635+0000","last_fullsized":"2024-09-11T23:50:11.878635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862666+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:50:18.862666+0000","last_peered":"2024-09-11T23:50:18.862666+0000","last_clean":"2024-09-11T23:50:18.862666+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:50:18.862666+0000","last_undegraded":"2024-09-11T23:50:18.862666+0000","last_fullsized":"2024-09-11T23:50:18.862666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881420+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:50:11.881420+0000","last_peered":"2024-09-11T23:50:11.881420+0000","last_clean":"2024-09-11T23:50:11.881420+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:50:11.881420+0000","last_undegraded":"2024-09-11T23:50:11.881420+0000","last_fullsized":"2024-09-11T23:50:11.881420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861345+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:50:18.861345+0000","last_peered":"2024-09-11T23:50:18.861345+0000","last_clean":"2024-09-11T23:50:18.861345+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:50:18.861345+0000","last_undegraded":"2024-09-11T23:50:18.861345+0000","last_fullsized":"2024-09-11T23:50:18.861345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877016+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:50:11.877016+0000","last_peered":"2024-09-11T23:50:11.877016+0000","last_clean":"2024-09-11T23:50:11.877016+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:50:11.877016+0000","last_undegraded":"2024-09-11T23:50:11.877016+0000","last_fullsized":"2024-09-11T23:50:11.877016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'22","reported_seq":72,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:32.893157+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:50:32.893157+0000","last_peered":"2024-09-11T23:50:32.893157+0000","last_clean":"2024-09-11T23:50:32.893157+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:50:32.893157+0000","last_undegraded":"2024-09-11T23:50:32.893157+0000","last_fullsized":"2024-09-11T23:50:32.893157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":61,"num_write":23,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877925+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:50:11.877925+0000","last_peered":"2024-09-11T23:50:11.877925+0000","last_clean":"2024-09-11T23:50:11.877925+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:50:11.877925+0000","last_undegraded":"2024-09-11T23:50:11.877925+0000","last_fullsized":"2024-09-11T23:50:11.877925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.866530+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:50:19.866530+0000","last_peered":"2024-09-11T23:50:19.866530+0000","last_clean":"2024-09-11T23:50:19.866530+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:50:19.866530+0000","last_undegraded":"2024-09-11T23:50:19.866530+0000","last_fullsized":"2024-09-11T23:50:19.866530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827276+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:50:11.827276+0000","last_peered":"2024-09-11T23:50:11.827276+0000","last_clean":"2024-09-11T23:50:11.827276+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:50:11.827276+0000","last_undegraded":"2024-09-11T23:50:11.827276+0000","last_fullsized":"2024-09-11T23:50:11.827276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862298+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:50:18.862298+0000","last_peered":"2024-09-11T23:50:18.862298+0000","last_clean":"2024-09-11T23:50:18.862298+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:50:18.862298+0000","last_undegraded":"2024-09-11T23:50:18.862298+0000","last_fullsized":"2024-09-11T23:50:18.862298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881417+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:50:11.881417+0000","last_peered":"2024-09-11T23:50:11.881417+0000","last_clean":"2024-09-11T23:50:11.881417+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:50:11.881417+0000","last_undegraded":"2024-09-11T23:50:11.881417+0000","last_fullsized":"2024-09-11T23:50:11.881417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:32.889902+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:50:32.889902+0000","last_peered":"2024-09-11T23:50:32.889902+0000","last_clean":"2024-09-11T23:50:32.889902+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:50:32.889902+0000","last_undegraded":"2024-09-11T23:50:32.889902+0000","last_fullsized":"2024-09-11T23:50:32.889902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880994+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:50:11.880994+0000","last_peered":"2024-09-11T23:50:11.880994+0000","last_clean":"2024-09-11T23:50:11.880994+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:50:11.880994+0000","last_undegraded":"2024-09-11T23:50:11.880994+0000","last_fullsized":"2024-09-11T23:50:11.880994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868544+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:50:19.868544+0000","last_peered":"2024-09-11T23:50:19.868544+0000","last_clean":"2024-09-11T23:50:19.868544+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:50:19.868544+0000","last_undegraded":"2024-09-11T23:50:19.868544+0000","last_fullsized":"2024-09-11T23:50:19.868544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879800+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:50:11.879800+0000","last_peered":"2024-09-11T23:50:11.879800+0000","last_clean":"2024-09-11T23:50:11.879800+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:50:11.879800+0000","last_undegraded":"2024-09-11T23:50:11.879800+0000","last_fullsized":"2024-09-11T23:50:11.879800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:20.871408+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:50:20.871408+0000","last_peered":"2024-09-11T23:50:20.871408+0000","last_clean":"2024-09-11T23:50:20.871408+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:50:20.871408+0000","last_undegraded":"2024-09-11T23:50:20.871408+0000","last_fullsized":"2024-09-11T23:50:20.871408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830599+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:50:11.830599+0000","last_peered":"2024-09-11T23:50:11.830599+0000","last_clean":"2024-09-11T23:50:11.830599+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:50:11.830599+0000","last_undegraded":"2024-09-11T23:50:11.830599+0000","last_fullsized":"2024-09-11T23:50:11.830599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877724+0000","last_change":"2024-09-11T23:50:08.766524+0000","last_active":"2024-09-11T23:50:11.877724+0000","last_peered":"2024-09-11T23:50:11.877724+0000","last_clean":"2024-09-11T23:50:11.877724+0000","last_became_active":"2024-09-11T23:50:08.766120+0000","last_became_peered":"2024-09-11T23:50:08.766120+0000","last_unstale":"2024-09-11T23:50:11.877724+0000","last_undegraded":"2024-09-11T23:50:11.877724+0000","last_fullsized":"2024-09-11T23:50:11.877724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:51:49.261871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881682+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:50:11.881682+0000","last_peered":"2024-09-11T23:50:11.881682+0000","last_clean":"2024-09-11T23:50:11.881682+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:50:11.881682+0000","last_undegraded":"2024-09-11T23:50:11.881682+0000","last_fullsized":"2024-09-11T23:50:11.881682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.886203+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:50:11.886203+0000","last_peered":"2024-09-11T23:50:11.886203+0000","last_clean":"2024-09-11T23:50:11.886203+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:50:11.886203+0000","last_undegraded":"2024-09-11T23:50:11.886203+0000","last_fullsized":"2024-09-11T23:50:11.886203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881405+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:50:11.881405+0000","last_peered":"2024-09-11T23:50:11.881405+0000","last_clean":"2024-09-11T23:50:11.881405+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:50:11.881405+0000","last_undegraded":"2024-09-11T23:50:11.881405+0000","last_fullsized":"2024-09-11T23:50:11.881405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.867894+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:50:19.867894+0000","last_peered":"2024-09-11T23:50:19.867894+0000","last_clean":"2024-09-11T23:50:19.867894+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:50:19.867894+0000","last_undegraded":"2024-09-11T23:50:19.867894+0000","last_fullsized":"2024-09-11T23:50:19.867894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881586+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:50:11.881586+0000","last_peered":"2024-09-11T23:50:11.881586+0000","last_clean":"2024-09-11T23:50:11.881586+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:50:11.881586+0000","last_undegraded":"2024-09-11T23:50:11.881586+0000","last_fullsized":"2024-09-11T23:50:11.881586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881179+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:50:11.881179+0000","last_peered":"2024-09-11T23:50:11.881179+0000","last_clean":"2024-09-11T23:50:11.881179+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:50:11.881179+0000","last_undegraded":"2024-09-11T23:50:11.881179+0000","last_fullsized":"2024-09-11T23:50:11.881179+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:20.871596+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:50:20.871596+0000","last_peered":"2024-09-11T23:50:20.871596+0000","last_clean":"2024-09-11T23:50:20.871596+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:50:20.871596+0000","last_undegraded":"2024-09-11T23:50:20.871596+0000","last_fullsized":"2024-09-11T23:50:20.871596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879540+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:50:11.879540+0000","last_peered":"2024-09-11T23:50:11.879540+0000","last_clean":"2024-09-11T23:50:11.879540+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:50:11.879540+0000","last_undegraded":"2024-09-11T23:50:11.879540+0000","last_fullsized":"2024-09-11T23:50:11.879540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868224+0000","last_change":"2024-09-11T23:50:08.767568+0000","last_active":"2024-09-11T23:50:19.868224+0000","last_peered":"2024-09-11T23:50:19.868224+0000","last_clean":"2024-09-11T23:50:19.868224+0000","last_became_active":"2024-09-11T23:50:08.767217+0000","last_became_peered":"2024-09-11T23:50:08.767217+0000","last_unstale":"2024-09-11T23:50:19.868224+0000","last_undegraded":"2024-09-11T23:50:19.868224+0000","last_fullsized":"2024-09-11T23:50:19.868224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:12.102018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827258+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:50:11.827258+0000","last_peered":"2024-09-11T23:50:11.827258+0000","last_clean":"2024-09-11T23:50:11.827258+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:50:11.827258+0000","last_undegraded":"2024-09-11T23:50:11.827258+0000","last_fullsized":"2024-09-11T23:50:11.827258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:29.885953+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:50:29.885953+0000","last_peered":"2024-09-11T23:50:29.885953+0000","last_clean":"2024-09-11T23:50:29.885953+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:50:29.885953+0000","last_undegraded":"2024-09-11T23:50:29.885953+0000","last_fullsized":"2024-09-11T23:50:29.885953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882510+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:50:11.882510+0000","last_peered":"2024-09-11T23:50:11.882510+0000","last_clean":"2024-09-11T23:50:11.882510+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:50:11.882510+0000","last_undegraded":"2024-09-11T23:50:11.882510+0000","last_fullsized":"2024-09-11T23:50:11.882510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881397+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:50:11.881397+0000","last_peered":"2024-09-11T23:50:11.881397+0000","last_clean":"2024-09-11T23:50:11.881397+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:50:11.881397+0000","last_undegraded":"2024-09-11T23:50:11.881397+0000","last_fullsized":"2024-09-11T23:50:11.881397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.828518+0000","last_change":"2024-09-11T23:50:10.846416+0000","last_active":"2024-09-11T23:50:11.828518+0000","last_peered":"2024-09-11T23:50:11.828518+0000","last_clean":"2024-09-11T23:50:11.828518+0000","last_became_active":"2024-09-11T23:50:10.843961+0000","last_became_peered":"2024-09-11T23:50:10.843961+0000","last_unstale":"2024-09-11T23:50:11.828518+0000","last_undegraded":"2024-09-11T23:50:11.828518+0000","last_fullsized":"2024-09-11T23:50:11.828518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:46.772458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877175+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:50:11.877175+0000","last_peered":"2024-09-11T23:50:11.877175+0000","last_clean":"2024-09-11T23:50:11.877175+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:50:11.877175+0000","last_undegraded":"2024-09-11T23:50:11.877175+0000","last_fullsized":"2024-09-11T23:50:11.877175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:13.236129+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:50:13.236129+0000","last_peered":"2024-09-11T23:50:13.236129+0000","last_clean":"2024-09-11T23:50:13.236129+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:50:13.236129+0000","last_undegraded":"2024-09-11T23:50:13.236129+0000","last_fullsized":"2024-09-11T23:50:13.236129+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861294+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:50:18.861294+0000","last_peered":"2024-09-11T23:50:18.861294+0000","last_clean":"2024-09-11T23:50:18.861294+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:50:18.861294+0000","last_undegraded":"2024-09-11T23:50:18.861294+0000","last_fullsized":"2024-09-11T23:50:18.861294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827648+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:50:11.827648+0000","last_peered":"2024-09-11T23:50:11.827648+0000","last_clean":"2024-09-11T23:50:11.827648+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:50:11.827648+0000","last_undegraded":"2024-09-11T23:50:11.827648+0000","last_fullsized":"2024-09-11T23:50:11.827648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:30.877290+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:50:30.877290+0000","last_peered":"2024-09-11T23:50:30.877290+0000","last_clean":"2024-09-11T23:50:30.877290+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:50:30.877290+0000","last_undegraded":"2024-09-11T23:50:30.877290+0000","last_fullsized":"2024-09-11T23:50:30.877290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880484+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:50:11.880484+0000","last_peered":"2024-09-11T23:50:11.880484+0000","last_clean":"2024-09-11T23:50:11.880484+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:50:11.880484+0000","last_undegraded":"2024-09-11T23:50:11.880484+0000","last_fullsized":"2024-09-11T23:50:11.880484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861080+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:50:18.861080+0000","last_peered":"2024-09-11T23:50:18.861080+0000","last_clean":"2024-09-11T23:50:18.861080+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:50:18.861080+0000","last_undegraded":"2024-09-11T23:50:18.861080+0000","last_fullsized":"2024-09-11T23:50:18.861080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881017+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:50:11.881017+0000","last_peered":"2024-09-11T23:50:11.881017+0000","last_clean":"2024-09-11T23:50:11.881017+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:50:11.881017+0000","last_undegraded":"2024-09-11T23:50:11.881017+0000","last_fullsized":"2024-09-11T23:50:11.881017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862125+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:50:18.862125+0000","last_peered":"2024-09-11T23:50:18.862125+0000","last_clean":"2024-09-11T23:50:18.862125+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:50:18.862125+0000","last_undegraded":"2024-09-11T23:50:18.862125+0000","last_fullsized":"2024-09-11T23:50:18.862125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876349+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:50:11.876349+0000","last_peered":"2024-09-11T23:50:11.876349+0000","last_clean":"2024-09-11T23:50:11.876349+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:50:11.876349+0000","last_undegraded":"2024-09-11T23:50:11.876349+0000","last_fullsized":"2024-09-11T23:50:11.876349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861563+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:50:18.861563+0000","last_peered":"2024-09-11T23:50:18.861563+0000","last_clean":"2024-09-11T23:50:18.861563+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:50:18.861563+0000","last_undegraded":"2024-09-11T23:50:18.861563+0000","last_fullsized":"2024-09-11T23:50:18.861563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830666+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:50:11.830666+0000","last_peered":"2024-09-11T23:50:11.830666+0000","last_clean":"2024-09-11T23:50:11.830666+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:50:11.830666+0000","last_undegraded":"2024-09-11T23:50:11.830666+0000","last_fullsized":"2024-09-11T23:50:11.830666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876431+0000","last_change":"2024-09-11T23:50:08.766530+0000","last_active":"2024-09-11T23:50:11.876431+0000","last_peered":"2024-09-11T23:50:11.876431+0000","last_clean":"2024-09-11T23:50:11.876431+0000","last_became_active":"2024-09-11T23:50:08.766014+0000","last_became_peered":"2024-09-11T23:50:08.766014+0000","last_unstale":"2024-09-11T23:50:11.876431+0000","last_undegraded":"2024-09-11T23:50:11.876431+0000","last_fullsized":"2024-09-11T23:50:11.876431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:38.056709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878313+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:50:11.878313+0000","last_peered":"2024-09-11T23:50:11.878313+0000","last_clean":"2024-09-11T23:50:11.878313+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:50:11.878313+0000","last_undegraded":"2024-09-11T23:50:11.878313+0000","last_fullsized":"2024-09-11T23:50:11.878313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.863430+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:50:18.863430+0000","last_peered":"2024-09-11T23:50:18.863430+0000","last_clean":"2024-09-11T23:50:18.863430+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:50:18.863430+0000","last_undegraded":"2024-09-11T23:50:18.863430+0000","last_fullsized":"2024-09-11T23:50:18.863430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881169+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:50:11.881169+0000","last_peered":"2024-09-11T23:50:11.881169+0000","last_clean":"2024-09-11T23:50:11.881169+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:50:11.881169+0000","last_undegraded":"2024-09-11T23:50:11.881169+0000","last_fullsized":"2024-09-11T23:50:11.881169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881434+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:50:11.881434+0000","last_peered":"2024-09-11T23:50:11.881434+0000","last_clean":"2024-09-11T23:50:11.881434+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:50:11.881434+0000","last_undegraded":"2024-09-11T23:50:11.881434+0000","last_fullsized":"2024-09-11T23:50:11.881434+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.885419+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:50:11.885419+0000","last_peered":"2024-09-11T23:50:11.885419+0000","last_clean":"2024-09-11T23:50:11.885419+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:50:11.885419+0000","last_undegraded":"2024-09-11T23:50:11.885419+0000","last_fullsized":"2024-09-11T23:50:11.885419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876830+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:50:11.876830+0000","last_peered":"2024-09-11T23:50:11.876830+0000","last_clean":"2024-09-11T23:50:11.876830+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:50:11.876830+0000","last_undegraded":"2024-09-11T23:50:11.876830+0000","last_fullsized":"2024-09-11T23:50:11.876830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.867614+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:50:19.867614+0000","last_peered":"2024-09-11T23:50:19.867614+0000","last_clean":"2024-09-11T23:50:19.867614+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:50:19.867614+0000","last_undegraded":"2024-09-11T23:50:19.867614+0000","last_fullsized":"2024-09-11T23:50:19.867614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881551+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:50:11.881551+0000","last_peered":"2024-09-11T23:50:11.881551+0000","last_clean":"2024-09-11T23:50:11.881551+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:50:11.881551+0000","last_undegraded":"2024-09-11T23:50:11.881551+0000","last_fullsized":"2024-09-11T23:50:11.881551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:12.424274+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:50:12.424274+0000","last_peered":"2024-09-11T23:50:12.424274+0000","last_clean":"2024-09-11T23:50:12.424274+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:50:12.424274+0000","last_undegraded":"2024-09-11T23:50:12.424274+0000","last_fullsized":"2024-09-11T23:50:12.424274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879978+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:50:11.879978+0000","last_peered":"2024-09-11T23:50:11.879978+0000","last_clean":"2024-09-11T23:50:11.879978+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:50:11.879978+0000","last_undegraded":"2024-09-11T23:50:11.879978+0000","last_fullsized":"2024-09-11T23:50:11.879978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868424+0000","last_change":"2024-09-11T23:50:08.764272+0000","last_active":"2024-09-11T23:50:19.868424+0000","last_peered":"2024-09-11T23:50:19.868424+0000","last_clean":"2024-09-11T23:50:19.868424+0000","last_became_active":"2024-09-11T23:50:08.763978+0000","last_became_peered":"2024-09-11T23:50:08.763978+0000","last_unstale":"2024-09-11T23:50:19.868424+0000","last_undegraded":"2024-09-11T23:50:19.868424+0000","last_fullsized":"2024-09-11T23:50:19.868424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:50.257629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50805,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":99,"num_read_kb":104,"num_write":150,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":152415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281926,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":419875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1058551,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1011963,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":419500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1011137,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":420330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":419811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":467445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":466095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":420771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":46802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":48152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:50:37.095 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:50:37.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:36 smithi083 ceph-mon[34805]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:50:37.353 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:37.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:36 smithi120 ceph-mon[35182]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:50:37.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.808+0000 7f512bb7e640 1 -- 172.21.15.62:0/2111070333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5124074eb0 msgr2=0x7f5124112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:37.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.808+0000 7f512bb7e640 1 --2- 172.21.15.62:0/2111070333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5124074eb0 0x7f5124112650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5114009900 tx=0x7f511402a480 comp rx=0 tx=0).stop 2024-09-11T23:50:37.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.808+0000 7f512bb7e640 1 -- 172.21.15.62:0/2111070333 shutdown_connections 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.808+0000 7f512bb7e640 1 --2- 172.21.15.62:0/2111070333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5124112b90 0x7f5124114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.808+0000 7f512bb7e640 1 --2- 172.21.15.62:0/2111070333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5124074eb0 0x7f5124112650 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.808+0000 7f512bb7e640 1 --2- 172.21.15.62:0/2111070333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124074590 0x7f5124074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.808+0000 7f512bb7e640 1 -- 172.21.15.62:0/2111070333 >> 172.21.15.62:0/2111070333 conn(0x7f51240fd8c0 msgr2=0x7f51240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.808+0000 7f512bb7e640 1 -- 172.21.15.62:0/2111070333 shutdown_connections 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.808+0000 7f512bb7e640 1 -- 172.21.15.62:0/2111070333 wait complete. 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512bb7e640 1 Processor -- start 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512bb7e640 1 -- start start 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512bb7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5124074590 0x7f51241a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512bb7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5124074eb0 0x7f51241a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512bb7e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124112b90 0x7f512419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512bb7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5124118560 con 0x7f5124074590 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512bb7e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51241183e0 con 0x7f5124112b90 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512bb7e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51241186e0 con 0x7f5124074eb0 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512991a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5124074590 0x7f51241a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512991a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5124074590 0x7f51241a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52036/0 (socket says 172.21.15.62:52036) 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.809+0000 7f512991a640 1 -- 172.21.15.62:0/3932434570 learned_addr learned my addr 172.21.15.62:0/3932434570 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f512a11b640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124112b90 0x7f512419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f512991a640 1 -- 172.21.15.62:0/3932434570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5124074eb0 msgr2=0x7f51241a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f512991a640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5124074eb0 0x7f51241a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f512991a640 1 -- 172.21.15.62:0/3932434570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124112b90 msgr2=0x7f512419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:37.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f512991a640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124112b90 0x7f512419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:37.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f512991a640 1 -- 172.21.15.62:0/3932434570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512419f000 con 0x7f5124074590 2024-09-11T23:50:37.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f512a11b640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124112b90 0x7f512419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:37.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f512991a640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5124074590 0x7f51241a4710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f512000e9f0 tx=0x7f512000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:37.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f511affd640 1 -- 172.21.15.62:0/3932434570 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512000cd80 con 0x7f5124074590 2024-09-11T23:50:37.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f512bb7e640 1 -- 172.21.15.62:0/3932434570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512419f2f0 con 0x7f5124074590 2024-09-11T23:50:37.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f511affd640 1 -- 172.21.15.62:0/3932434570 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5120010910 con 0x7f5124074590 2024-09-11T23:50:37.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f511affd640 1 -- 172.21.15.62:0/3932434570 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51200188f0 con 0x7f5124074590 2024-09-11T23:50:37.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.810+0000 7f512bb7e640 1 -- 172.21.15.62:0/3932434570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51241ab5e0 con 0x7f5124074590 2024-09-11T23:50:37.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.811+0000 7f512bb7e640 1 -- 172.21.15.62:0/3932434570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5124074970 con 0x7f5124074590 2024-09-11T23:50:37.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.812+0000 7f511affd640 1 -- 172.21.15.62:0/3932434570 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5120010ac0 con 0x7f5124074590 2024-09-11T23:50:37.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.813+0000 7f511affd640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f51100a2c10 0x7f51100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:37.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.813+0000 7f5129119640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f51100a2c10 0x7f51100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:37.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.813+0000 7f511affd640 1 -- 172.21.15.62:0/3932434570 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f51200cf940 con 0x7f5124074590 2024-09-11T23:50:37.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.813+0000 7f5129119640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f51100a2c10 0x7f51100a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f511402a9a0 tx=0x7f5114000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:37.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.816+0000 7f511affd640 1 -- 172.21.15.62:0/3932434570 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f512009c8f0 con 0x7f5124074590 2024-09-11T23:50:37.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.990+0000 7f512bb7e640 1 -- 172.21.15.62:0/3932434570 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51240ff720 con 0x7f51100a2c10 2024-09-11T23:50:37.990 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:37 smithi062 ceph-mon[30958]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-11T23:50:37.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.997+0000 7f511affd640 1 -- 172.21.15.62:0/3932434570 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258861 (secure 0 0 0) 0x7f51240ff720 con 0x7f51100a2c10 2024-09-11T23:50:37.999 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:37.999 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:50:38.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.999+0000 7f512bb7e640 1 -- 172.21.15.62:0/3932434570 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f51100a2c10 msgr2=0x7f51100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:38.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.999+0000 7f512bb7e640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f51100a2c10 0x7f51100a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f511402a9a0 tx=0x7f5114000bc0 comp rx=0 tx=0).stop 2024-09-11T23:50:38.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.999+0000 7f512bb7e640 1 -- 172.21.15.62:0/3932434570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5124074590 msgr2=0x7f51241a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:38.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:37.999+0000 7f512bb7e640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5124074590 0x7f51241a4710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f512000e9f0 tx=0x7f512000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:50:38.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:38.000+0000 7f512bb7e640 1 -- 172.21.15.62:0/3932434570 shutdown_connections 2024-09-11T23:50:38.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:38.000+0000 7f512bb7e640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f51100a2c10 0x7f51100a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:38.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:38.000+0000 7f512bb7e640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124112b90 0x7f512419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:38.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:38.000+0000 7f512bb7e640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5124074eb0 0x7f51241a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:38.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:38.000+0000 7f512bb7e640 1 --2- 172.21.15.62:0/3932434570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5124074590 0x7f51241a4710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:38.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:38.000+0000 7f512bb7e640 1 -- 172.21.15.62:0/3932434570 >> 172.21.15.62:0/3932434570 conn(0x7f51240fd8c0 msgr2=0x7f51240ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:38.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:38.000+0000 7f512bb7e640 1 -- 172.21.15.62:0/3932434570 shutdown_connections 2024-09-11T23:50:38.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:38.000+0000 7f512bb7e640 1 -- 172.21.15.62:0/3932434570 wait complete. 2024-09-11T23:50:38.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:37 smithi083 ceph-mon[34805]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-11T23:50:38.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:37 smithi120 ceph-mon[35182]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-11T23:50:38.738 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-11T23:50:36.806081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641407,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":245,"num_read_kb":228,"num_write":333,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347484,"kb_used_data":10716,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594340,"statfs":{"total":1151940427776,"available":1151584604160,"internally_reserved":0,"allocated":10973184,"data_stored":6955737,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45817,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":86,"num_write":28,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002953"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830856+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:50:11.830856+0000","last_peered":"2024-09-11T23:50:11.830856+0000","last_clean":"2024-09-11T23:50:11.830856+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:50:11.830856+0000","last_undegraded":"2024-09-11T23:50:11.830856+0000","last_fullsized":"2024-09-11T23:50:11.830856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880987+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:50:11.880987+0000","last_peered":"2024-09-11T23:50:11.880987+0000","last_clean":"2024-09-11T23:50:11.880987+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:50:11.880987+0000","last_undegraded":"2024-09-11T23:50:11.880987+0000","last_fullsized":"2024-09-11T23:50:11.880987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880650+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:50:11.880650+0000","last_peered":"2024-09-11T23:50:11.880650+0000","last_clean":"2024-09-11T23:50:11.880650+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:50:11.880650+0000","last_undegraded":"2024-09-11T23:50:11.880650+0000","last_fullsized":"2024-09-11T23:50:11.880650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878400+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:50:11.878400+0000","last_peered":"2024-09-11T23:50:11.878400+0000","last_clean":"2024-09-11T23:50:11.878400+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:50:11.878400+0000","last_undegraded":"2024-09-11T23:50:11.878400+0000","last_fullsized":"2024-09-11T23:50:11.878400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876576+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:50:11.876576+0000","last_peered":"2024-09-11T23:50:11.876576+0000","last_clean":"2024-09-11T23:50:11.876576+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:50:11.876576+0000","last_undegraded":"2024-09-11T23:50:11.876576+0000","last_fullsized":"2024-09-11T23:50:11.876576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.828168+0000","last_change":"2024-09-11T23:50:10.842560+0000","last_active":"2024-09-11T23:50:11.828168+0000","last_peered":"2024-09-11T23:50:11.828168+0000","last_clean":"2024-09-11T23:50:11.828168+0000","last_became_active":"2024-09-11T23:50:10.842259+0000","last_became_peered":"2024-09-11T23:50:10.842259+0000","last_unstale":"2024-09-11T23:50:11.828168+0000","last_undegraded":"2024-09-11T23:50:11.828168+0000","last_fullsized":"2024-09-11T23:50:11.828168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:36.538402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881133+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:50:11.881133+0000","last_peered":"2024-09-11T23:50:11.881133+0000","last_clean":"2024-09-11T23:50:11.881133+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:50:11.881133+0000","last_undegraded":"2024-09-11T23:50:11.881133+0000","last_fullsized":"2024-09-11T23:50:11.881133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877161+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:50:11.877161+0000","last_peered":"2024-09-11T23:50:11.877161+0000","last_clean":"2024-09-11T23:50:11.877161+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:50:11.877161+0000","last_undegraded":"2024-09-11T23:50:11.877161+0000","last_fullsized":"2024-09-11T23:50:11.877161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878788+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:50:11.878788+0000","last_peered":"2024-09-11T23:50:11.878788+0000","last_clean":"2024-09-11T23:50:11.878788+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:50:11.878788+0000","last_undegraded":"2024-09-11T23:50:11.878788+0000","last_fullsized":"2024-09-11T23:50:11.878788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.828525+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:50:11.828525+0000","last_peered":"2024-09-11T23:50:11.828525+0000","last_clean":"2024-09-11T23:50:11.828525+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:50:11.828525+0000","last_undegraded":"2024-09-11T23:50:11.828525+0000","last_fullsized":"2024-09-11T23:50:11.828525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881588+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:50:11.881588+0000","last_peered":"2024-09-11T23:50:11.881588+0000","last_clean":"2024-09-11T23:50:11.881588+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:50:11.881588+0000","last_undegraded":"2024-09-11T23:50:11.881588+0000","last_fullsized":"2024-09-11T23:50:11.881588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878548+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:50:11.878548+0000","last_peered":"2024-09-11T23:50:11.878548+0000","last_clean":"2024-09-11T23:50:11.878548+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:50:11.878548+0000","last_undegraded":"2024-09-11T23:50:11.878548+0000","last_fullsized":"2024-09-11T23:50:11.878548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827687+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:50:11.827687+0000","last_peered":"2024-09-11T23:50:11.827687+0000","last_clean":"2024-09-11T23:50:11.827687+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:50:11.827687+0000","last_undegraded":"2024-09-11T23:50:11.827687+0000","last_fullsized":"2024-09-11T23:50:11.827687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880393+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:50:11.880393+0000","last_peered":"2024-09-11T23:50:11.880393+0000","last_clean":"2024-09-11T23:50:11.880393+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:50:11.880393+0000","last_undegraded":"2024-09-11T23:50:11.880393+0000","last_fullsized":"2024-09-11T23:50:11.880393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880060+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:50:11.880060+0000","last_peered":"2024-09-11T23:50:11.880060+0000","last_clean":"2024-09-11T23:50:11.880060+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:50:11.880060+0000","last_undegraded":"2024-09-11T23:50:11.880060+0000","last_fullsized":"2024-09-11T23:50:11.880060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877177+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:50:11.877177+0000","last_peered":"2024-09-11T23:50:11.877177+0000","last_clean":"2024-09-11T23:50:11.877177+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:50:11.877177+0000","last_undegraded":"2024-09-11T23:50:11.877177+0000","last_fullsized":"2024-09-11T23:50:11.877177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830826+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:50:11.830826+0000","last_peered":"2024-09-11T23:50:11.830826+0000","last_clean":"2024-09-11T23:50:11.830826+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:50:11.830826+0000","last_undegraded":"2024-09-11T23:50:11.830826+0000","last_fullsized":"2024-09-11T23:50:11.830826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876998+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:50:11.876998+0000","last_peered":"2024-09-11T23:50:11.876998+0000","last_clean":"2024-09-11T23:50:11.876998+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:50:11.876998+0000","last_undegraded":"2024-09-11T23:50:11.876998+0000","last_fullsized":"2024-09-11T23:50:11.876998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876734+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:50:11.876734+0000","last_peered":"2024-09-11T23:50:11.876734+0000","last_clean":"2024-09-11T23:50:11.876734+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:50:11.876734+0000","last_undegraded":"2024-09-11T23:50:11.876734+0000","last_fullsized":"2024-09-11T23:50:11.876734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830673+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:50:11.830673+0000","last_peered":"2024-09-11T23:50:11.830673+0000","last_clean":"2024-09-11T23:50:11.830673+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:50:11.830673+0000","last_undegraded":"2024-09-11T23:50:11.830673+0000","last_fullsized":"2024-09-11T23:50:11.830673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827752+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:50:11.827752+0000","last_peered":"2024-09-11T23:50:11.827752+0000","last_clean":"2024-09-11T23:50:11.827752+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:50:11.827752+0000","last_undegraded":"2024-09-11T23:50:11.827752+0000","last_fullsized":"2024-09-11T23:50:11.827752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877072+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:50:11.877072+0000","last_peered":"2024-09-11T23:50:11.877072+0000","last_clean":"2024-09-11T23:50:11.877072+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:50:11.877072+0000","last_undegraded":"2024-09-11T23:50:11.877072+0000","last_fullsized":"2024-09-11T23:50:11.877072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878562+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:50:11.878562+0000","last_peered":"2024-09-11T23:50:11.878562+0000","last_clean":"2024-09-11T23:50:11.878562+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:50:11.878562+0000","last_undegraded":"2024-09-11T23:50:11.878562+0000","last_fullsized":"2024-09-11T23:50:11.878562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882045+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:50:11.882045+0000","last_peered":"2024-09-11T23:50:11.882045+0000","last_clean":"2024-09-11T23:50:11.882045+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:50:11.882045+0000","last_undegraded":"2024-09-11T23:50:11.882045+0000","last_fullsized":"2024-09-11T23:50:11.882045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880422+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:50:11.880422+0000","last_peered":"2024-09-11T23:50:11.880422+0000","last_clean":"2024-09-11T23:50:11.880422+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:50:11.880422+0000","last_undegraded":"2024-09-11T23:50:11.880422+0000","last_fullsized":"2024-09-11T23:50:11.880422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877065+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:50:11.877065+0000","last_peered":"2024-09-11T23:50:11.877065+0000","last_clean":"2024-09-11T23:50:11.877065+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:50:11.877065+0000","last_undegraded":"2024-09-11T23:50:11.877065+0000","last_fullsized":"2024-09-11T23:50:11.877065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830611+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:50:11.830611+0000","last_peered":"2024-09-11T23:50:11.830611+0000","last_clean":"2024-09-11T23:50:11.830611+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:50:11.830611+0000","last_undegraded":"2024-09-11T23:50:11.830611+0000","last_fullsized":"2024-09-11T23:50:11.830611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827672+0000","last_change":"2024-09-11T23:50:10.845202+0000","last_active":"2024-09-11T23:50:11.827672+0000","last_peered":"2024-09-11T23:50:11.827672+0000","last_clean":"2024-09-11T23:50:11.827672+0000","last_became_active":"2024-09-11T23:50:10.844892+0000","last_became_peered":"2024-09-11T23:50:10.844892+0000","last_unstale":"2024-09-11T23:50:11.827672+0000","last_undegraded":"2024-09-11T23:50:11.827672+0000","last_fullsized":"2024-09-11T23:50:11.827672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:09.744773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880722+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:50:11.880722+0000","last_peered":"2024-09-11T23:50:11.880722+0000","last_clean":"2024-09-11T23:50:11.880722+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:50:11.880722+0000","last_undegraded":"2024-09-11T23:50:11.880722+0000","last_fullsized":"2024-09-11T23:50:11.880722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876440+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:50:11.876440+0000","last_peered":"2024-09-11T23:50:11.876440+0000","last_clean":"2024-09-11T23:50:11.876440+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:50:11.876440+0000","last_undegraded":"2024-09-11T23:50:11.876440+0000","last_fullsized":"2024-09-11T23:50:11.876440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877202+0000","last_change":"2024-09-11T23:50:10.843592+0000","last_active":"2024-09-11T23:50:11.877202+0000","last_peered":"2024-09-11T23:50:11.877202+0000","last_clean":"2024-09-11T23:50:11.877202+0000","last_became_active":"2024-09-11T23:50:10.843259+0000","last_became_peered":"2024-09-11T23:50:10.843259+0000","last_unstale":"2024-09-11T23:50:11.877202+0000","last_undegraded":"2024-09-11T23:50:11.877202+0000","last_fullsized":"2024-09-11T23:50:11.877202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:05.557977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830450+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:50:11.830450+0000","last_peered":"2024-09-11T23:50:11.830450+0000","last_clean":"2024-09-11T23:50:11.830450+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:50:11.830450+0000","last_undegraded":"2024-09-11T23:50:11.830450+0000","last_fullsized":"2024-09-11T23:50:11.830450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876539+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:50:11.876539+0000","last_peered":"2024-09-11T23:50:11.876539+0000","last_clean":"2024-09-11T23:50:11.876539+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:50:11.876539+0000","last_undegraded":"2024-09-11T23:50:11.876539+0000","last_fullsized":"2024-09-11T23:50:11.876539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:33.891543+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:50:33.891543+0000","last_peered":"2024-09-11T23:50:33.891543+0000","last_clean":"2024-09-11T23:50:33.891543+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:50:33.891543+0000","last_undegraded":"2024-09-11T23:50:33.891543+0000","last_fullsized":"2024-09-11T23:50:33.891543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880098+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:50:11.880098+0000","last_peered":"2024-09-11T23:50:11.880098+0000","last_clean":"2024-09-11T23:50:11.880098+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:50:11.880098+0000","last_undegraded":"2024-09-11T23:50:11.880098+0000","last_fullsized":"2024-09-11T23:50:11.880098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.866074+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:50:19.866074+0000","last_peered":"2024-09-11T23:50:19.866074+0000","last_clean":"2024-09-11T23:50:19.866074+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:50:19.866074+0000","last_undegraded":"2024-09-11T23:50:19.866074+0000","last_fullsized":"2024-09-11T23:50:19.866074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.866815+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:50:19.866815+0000","last_peered":"2024-09-11T23:50:19.866815+0000","last_clean":"2024-09-11T23:50:19.866815+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:50:19.866815+0000","last_undegraded":"2024-09-11T23:50:19.866815+0000","last_fullsized":"2024-09-11T23:50:19.866815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877320+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:50:11.877320+0000","last_peered":"2024-09-11T23:50:11.877320+0000","last_clean":"2024-09-11T23:50:11.877320+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:50:11.877320+0000","last_undegraded":"2024-09-11T23:50:11.877320+0000","last_fullsized":"2024-09-11T23:50:11.877320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882426+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:50:11.882426+0000","last_peered":"2024-09-11T23:50:11.882426+0000","last_clean":"2024-09-11T23:50:11.882426+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:50:11.882426+0000","last_undegraded":"2024-09-11T23:50:11.882426+0000","last_fullsized":"2024-09-11T23:50:11.882426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877143+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:50:11.877143+0000","last_peered":"2024-09-11T23:50:11.877143+0000","last_clean":"2024-09-11T23:50:11.877143+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:50:11.877143+0000","last_undegraded":"2024-09-11T23:50:11.877143+0000","last_fullsized":"2024-09-11T23:50:11.877143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:31.888120+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:50:31.888120+0000","last_peered":"2024-09-11T23:50:31.888120+0000","last_clean":"2024-09-11T23:50:31.888120+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:50:31.888120+0000","last_undegraded":"2024-09-11T23:50:31.888120+0000","last_fullsized":"2024-09-11T23:50:31.888120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878635+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:50:11.878635+0000","last_peered":"2024-09-11T23:50:11.878635+0000","last_clean":"2024-09-11T23:50:11.878635+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:50:11.878635+0000","last_undegraded":"2024-09-11T23:50:11.878635+0000","last_fullsized":"2024-09-11T23:50:11.878635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862666+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:50:18.862666+0000","last_peered":"2024-09-11T23:50:18.862666+0000","last_clean":"2024-09-11T23:50:18.862666+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:50:18.862666+0000","last_undegraded":"2024-09-11T23:50:18.862666+0000","last_fullsized":"2024-09-11T23:50:18.862666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881420+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:50:11.881420+0000","last_peered":"2024-09-11T23:50:11.881420+0000","last_clean":"2024-09-11T23:50:11.881420+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:50:11.881420+0000","last_undegraded":"2024-09-11T23:50:11.881420+0000","last_fullsized":"2024-09-11T23:50:11.881420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861345+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:50:18.861345+0000","last_peered":"2024-09-11T23:50:18.861345+0000","last_clean":"2024-09-11T23:50:18.861345+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:50:18.861345+0000","last_undegraded":"2024-09-11T23:50:18.861345+0000","last_fullsized":"2024-09-11T23:50:18.861345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877016+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:50:11.877016+0000","last_peered":"2024-09-11T23:50:11.877016+0000","last_clean":"2024-09-11T23:50:11.877016+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:50:11.877016+0000","last_undegraded":"2024-09-11T23:50:11.877016+0000","last_fullsized":"2024-09-11T23:50:11.877016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'22","reported_seq":72,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:32.893157+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:50:32.893157+0000","last_peered":"2024-09-11T23:50:32.893157+0000","last_clean":"2024-09-11T23:50:32.893157+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:50:32.893157+0000","last_undegraded":"2024-09-11T23:50:32.893157+0000","last_fullsized":"2024-09-11T23:50:32.893157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":61,"num_write":23,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877925+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:50:11.877925+0000","last_peered":"2024-09-11T23:50:11.877925+0000","last_clean":"2024-09-11T23:50:11.877925+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:50:11.877925+0000","last_undegraded":"2024-09-11T23:50:11.877925+0000","last_fullsized":"2024-09-11T23:50:11.877925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.866530+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:50:19.866530+0000","last_peered":"2024-09-11T23:50:19.866530+0000","last_clean":"2024-09-11T23:50:19.866530+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:50:19.866530+0000","last_undegraded":"2024-09-11T23:50:19.866530+0000","last_fullsized":"2024-09-11T23:50:19.866530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827276+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:50:11.827276+0000","last_peered":"2024-09-11T23:50:11.827276+0000","last_clean":"2024-09-11T23:50:11.827276+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:50:11.827276+0000","last_undegraded":"2024-09-11T23:50:11.827276+0000","last_fullsized":"2024-09-11T23:50:11.827276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862298+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:50:18.862298+0000","last_peered":"2024-09-11T23:50:18.862298+0000","last_clean":"2024-09-11T23:50:18.862298+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:50:18.862298+0000","last_undegraded":"2024-09-11T23:50:18.862298+0000","last_fullsized":"2024-09-11T23:50:18.862298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881417+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:50:11.881417+0000","last_peered":"2024-09-11T23:50:11.881417+0000","last_clean":"2024-09-11T23:50:11.881417+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:50:11.881417+0000","last_undegraded":"2024-09-11T23:50:11.881417+0000","last_fullsized":"2024-09-11T23:50:11.881417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:32.889902+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:50:32.889902+0000","last_peered":"2024-09-11T23:50:32.889902+0000","last_clean":"2024-09-11T23:50:32.889902+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:50:32.889902+0000","last_undegraded":"2024-09-11T23:50:32.889902+0000","last_fullsized":"2024-09-11T23:50:32.889902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880994+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:50:11.880994+0000","last_peered":"2024-09-11T23:50:11.880994+0000","last_clean":"2024-09-11T23:50:11.880994+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:50:11.880994+0000","last_undegraded":"2024-09-11T23:50:11.880994+0000","last_fullsized":"2024-09-11T23:50:11.880994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868544+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:50:19.868544+0000","last_peered":"2024-09-11T23:50:19.868544+0000","last_clean":"2024-09-11T23:50:19.868544+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:50:19.868544+0000","last_undegraded":"2024-09-11T23:50:19.868544+0000","last_fullsized":"2024-09-11T23:50:19.868544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879800+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:50:11.879800+0000","last_peered":"2024-09-11T23:50:11.879800+0000","last_clean":"2024-09-11T23:50:11.879800+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:50:11.879800+0000","last_undegraded":"2024-09-11T23:50:11.879800+0000","last_fullsized":"2024-09-11T23:50:11.879800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:20.871408+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:50:20.871408+0000","last_peered":"2024-09-11T23:50:20.871408+0000","last_clean":"2024-09-11T23:50:20.871408+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:50:20.871408+0000","last_undegraded":"2024-09-11T23:50:20.871408+0000","last_fullsized":"2024-09-11T23:50:20.871408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830599+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:50:11.830599+0000","last_peered":"2024-09-11T23:50:11.830599+0000","last_clean":"2024-09-11T23:50:11.830599+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:50:11.830599+0000","last_undegraded":"2024-09-11T23:50:11.830599+0000","last_fullsized":"2024-09-11T23:50:11.830599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877724+0000","last_change":"2024-09-11T23:50:08.766524+0000","last_active":"2024-09-11T23:50:11.877724+0000","last_peered":"2024-09-11T23:50:11.877724+0000","last_clean":"2024-09-11T23:50:11.877724+0000","last_became_active":"2024-09-11T23:50:08.766120+0000","last_became_peered":"2024-09-11T23:50:08.766120+0000","last_unstale":"2024-09-11T23:50:11.877724+0000","last_undegraded":"2024-09-11T23:50:11.877724+0000","last_fullsized":"2024-09-11T23:50:11.877724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:51:49.261871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881682+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:50:11.881682+0000","last_peered":"2024-09-11T23:50:11.881682+0000","last_clean":"2024-09-11T23:50:11.881682+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:50:11.881682+0000","last_undegraded":"2024-09-11T23:50:11.881682+0000","last_fullsized":"2024-09-11T23:50:11.881682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.886203+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:50:11.886203+0000","last_peered":"2024-09-11T23:50:11.886203+0000","last_clean":"2024-09-11T23:50:11.886203+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:50:11.886203+0000","last_undegraded":"2024-09-11T23:50:11.886203+0000","last_fullsized":"2024-09-11T23:50:11.886203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881405+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:50:11.881405+0000","last_peered":"2024-09-11T23:50:11.881405+0000","last_clean":"2024-09-11T23:50:11.881405+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:50:11.881405+0000","last_undegraded":"2024-09-11T23:50:11.881405+0000","last_fullsized":"2024-09-11T23:50:11.881405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.867894+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:50:19.867894+0000","last_peered":"2024-09-11T23:50:19.867894+0000","last_clean":"2024-09-11T23:50:19.867894+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:50:19.867894+0000","last_undegraded":"2024-09-11T23:50:19.867894+0000","last_fullsized":"2024-09-11T23:50:19.867894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881586+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:50:11.881586+0000","last_peered":"2024-09-11T23:50:11.881586+0000","last_clean":"2024-09-11T23:50:11.881586+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:50:11.881586+0000","last_undegraded":"2024-09-11T23:50:11.881586+0000","last_fullsized":"2024-09-11T23:50:11.881586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881179+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:50:11.881179+0000","last_peered":"2024-09-11T23:50:11.881179+0000","last_clean":"2024-09-11T23:50:11.881179+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:50:11.881179+0000","last_undegraded":"2024-09-11T23:50:11.881179+0000","last_fullsized":"2024-09-11T23:50:11.881179+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:20.871596+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:50:20.871596+0000","last_peered":"2024-09-11T23:50:20.871596+0000","last_clean":"2024-09-11T23:50:20.871596+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:50:20.871596+0000","last_undegraded":"2024-09-11T23:50:20.871596+0000","last_fullsized":"2024-09-11T23:50:20.871596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879540+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:50:11.879540+0000","last_peered":"2024-09-11T23:50:11.879540+0000","last_clean":"2024-09-11T23:50:11.879540+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:50:11.879540+0000","last_undegraded":"2024-09-11T23:50:11.879540+0000","last_fullsized":"2024-09-11T23:50:11.879540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868224+0000","last_change":"2024-09-11T23:50:08.767568+0000","last_active":"2024-09-11T23:50:19.868224+0000","last_peered":"2024-09-11T23:50:19.868224+0000","last_clean":"2024-09-11T23:50:19.868224+0000","last_became_active":"2024-09-11T23:50:08.767217+0000","last_became_peered":"2024-09-11T23:50:08.767217+0000","last_unstale":"2024-09-11T23:50:19.868224+0000","last_undegraded":"2024-09-11T23:50:19.868224+0000","last_fullsized":"2024-09-11T23:50:19.868224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:12.102018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827258+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:50:11.827258+0000","last_peered":"2024-09-11T23:50:11.827258+0000","last_clean":"2024-09-11T23:50:11.827258+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:50:11.827258+0000","last_undegraded":"2024-09-11T23:50:11.827258+0000","last_fullsized":"2024-09-11T23:50:11.827258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:34.892173+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:50:34.892173+0000","last_peered":"2024-09-11T23:50:34.892173+0000","last_clean":"2024-09-11T23:50:34.892173+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:50:34.892173+0000","last_undegraded":"2024-09-11T23:50:34.892173+0000","last_fullsized":"2024-09-11T23:50:34.892173+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882510+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:50:11.882510+0000","last_peered":"2024-09-11T23:50:11.882510+0000","last_clean":"2024-09-11T23:50:11.882510+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:50:11.882510+0000","last_undegraded":"2024-09-11T23:50:11.882510+0000","last_fullsized":"2024-09-11T23:50:11.882510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881397+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:50:11.881397+0000","last_peered":"2024-09-11T23:50:11.881397+0000","last_clean":"2024-09-11T23:50:11.881397+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:50:11.881397+0000","last_undegraded":"2024-09-11T23:50:11.881397+0000","last_fullsized":"2024-09-11T23:50:11.881397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.828518+0000","last_change":"2024-09-11T23:50:10.846416+0000","last_active":"2024-09-11T23:50:11.828518+0000","last_peered":"2024-09-11T23:50:11.828518+0000","last_clean":"2024-09-11T23:50:11.828518+0000","last_became_active":"2024-09-11T23:50:10.843961+0000","last_became_peered":"2024-09-11T23:50:10.843961+0000","last_unstale":"2024-09-11T23:50:11.828518+0000","last_undegraded":"2024-09-11T23:50:11.828518+0000","last_fullsized":"2024-09-11T23:50:11.828518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:46.772458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877175+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:50:11.877175+0000","last_peered":"2024-09-11T23:50:11.877175+0000","last_clean":"2024-09-11T23:50:11.877175+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:50:11.877175+0000","last_undegraded":"2024-09-11T23:50:11.877175+0000","last_fullsized":"2024-09-11T23:50:11.877175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:13.236129+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:50:13.236129+0000","last_peered":"2024-09-11T23:50:13.236129+0000","last_clean":"2024-09-11T23:50:13.236129+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:50:13.236129+0000","last_undegraded":"2024-09-11T23:50:13.236129+0000","last_fullsized":"2024-09-11T23:50:13.236129+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861294+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:50:18.861294+0000","last_peered":"2024-09-11T23:50:18.861294+0000","last_clean":"2024-09-11T23:50:18.861294+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:50:18.861294+0000","last_undegraded":"2024-09-11T23:50:18.861294+0000","last_fullsized":"2024-09-11T23:50:18.861294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827648+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:50:11.827648+0000","last_peered":"2024-09-11T23:50:11.827648+0000","last_clean":"2024-09-11T23:50:11.827648+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:50:11.827648+0000","last_undegraded":"2024-09-11T23:50:11.827648+0000","last_fullsized":"2024-09-11T23:50:11.827648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:30.877290+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:50:30.877290+0000","last_peered":"2024-09-11T23:50:30.877290+0000","last_clean":"2024-09-11T23:50:30.877290+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:50:30.877290+0000","last_undegraded":"2024-09-11T23:50:30.877290+0000","last_fullsized":"2024-09-11T23:50:30.877290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880484+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:50:11.880484+0000","last_peered":"2024-09-11T23:50:11.880484+0000","last_clean":"2024-09-11T23:50:11.880484+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:50:11.880484+0000","last_undegraded":"2024-09-11T23:50:11.880484+0000","last_fullsized":"2024-09-11T23:50:11.880484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861080+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:50:18.861080+0000","last_peered":"2024-09-11T23:50:18.861080+0000","last_clean":"2024-09-11T23:50:18.861080+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:50:18.861080+0000","last_undegraded":"2024-09-11T23:50:18.861080+0000","last_fullsized":"2024-09-11T23:50:18.861080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881017+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:50:11.881017+0000","last_peered":"2024-09-11T23:50:11.881017+0000","last_clean":"2024-09-11T23:50:11.881017+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:50:11.881017+0000","last_undegraded":"2024-09-11T23:50:11.881017+0000","last_fullsized":"2024-09-11T23:50:11.881017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862125+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:50:18.862125+0000","last_peered":"2024-09-11T23:50:18.862125+0000","last_clean":"2024-09-11T23:50:18.862125+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:50:18.862125+0000","last_undegraded":"2024-09-11T23:50:18.862125+0000","last_fullsized":"2024-09-11T23:50:18.862125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876349+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:50:11.876349+0000","last_peered":"2024-09-11T23:50:11.876349+0000","last_clean":"2024-09-11T23:50:11.876349+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:50:11.876349+0000","last_undegraded":"2024-09-11T23:50:11.876349+0000","last_fullsized":"2024-09-11T23:50:11.876349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861563+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:50:18.861563+0000","last_peered":"2024-09-11T23:50:18.861563+0000","last_clean":"2024-09-11T23:50:18.861563+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:50:18.861563+0000","last_undegraded":"2024-09-11T23:50:18.861563+0000","last_fullsized":"2024-09-11T23:50:18.861563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830666+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:50:11.830666+0000","last_peered":"2024-09-11T23:50:11.830666+0000","last_clean":"2024-09-11T23:50:11.830666+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:50:11.830666+0000","last_undegraded":"2024-09-11T23:50:11.830666+0000","last_fullsized":"2024-09-11T23:50:11.830666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876431+0000","last_change":"2024-09-11T23:50:08.766530+0000","last_active":"2024-09-11T23:50:11.876431+0000","last_peered":"2024-09-11T23:50:11.876431+0000","last_clean":"2024-09-11T23:50:11.876431+0000","last_became_active":"2024-09-11T23:50:08.766014+0000","last_became_peered":"2024-09-11T23:50:08.766014+0000","last_unstale":"2024-09-11T23:50:11.876431+0000","last_undegraded":"2024-09-11T23:50:11.876431+0000","last_fullsized":"2024-09-11T23:50:11.876431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:38.056709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878313+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:50:11.878313+0000","last_peered":"2024-09-11T23:50:11.878313+0000","last_clean":"2024-09-11T23:50:11.878313+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:50:11.878313+0000","last_undegraded":"2024-09-11T23:50:11.878313+0000","last_fullsized":"2024-09-11T23:50:11.878313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.863430+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:50:18.863430+0000","last_peered":"2024-09-11T23:50:18.863430+0000","last_clean":"2024-09-11T23:50:18.863430+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:50:18.863430+0000","last_undegraded":"2024-09-11T23:50:18.863430+0000","last_fullsized":"2024-09-11T23:50:18.863430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881169+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:50:11.881169+0000","last_peered":"2024-09-11T23:50:11.881169+0000","last_clean":"2024-09-11T23:50:11.881169+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:50:11.881169+0000","last_undegraded":"2024-09-11T23:50:11.881169+0000","last_fullsized":"2024-09-11T23:50:11.881169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881434+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:50:11.881434+0000","last_peered":"2024-09-11T23:50:11.881434+0000","last_clean":"2024-09-11T23:50:11.881434+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:50:11.881434+0000","last_undegraded":"2024-09-11T23:50:11.881434+0000","last_fullsized":"2024-09-11T23:50:11.881434+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.885419+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:50:11.885419+0000","last_peered":"2024-09-11T23:50:11.885419+0000","last_clean":"2024-09-11T23:50:11.885419+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:50:11.885419+0000","last_undegraded":"2024-09-11T23:50:11.885419+0000","last_fullsized":"2024-09-11T23:50:11.885419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876830+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:50:11.876830+0000","last_peered":"2024-09-11T23:50:11.876830+0000","last_clean":"2024-09-11T23:50:11.876830+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:50:11.876830+0000","last_undegraded":"2024-09-11T23:50:11.876830+0000","last_fullsized":"2024-09-11T23:50:11.876830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.867614+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:50:19.867614+0000","last_peered":"2024-09-11T23:50:19.867614+0000","last_clean":"2024-09-11T23:50:19.867614+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:50:19.867614+0000","last_undegraded":"2024-09-11T23:50:19.867614+0000","last_fullsized":"2024-09-11T23:50:19.867614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881551+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:50:11.881551+0000","last_peered":"2024-09-11T23:50:11.881551+0000","last_clean":"2024-09-11T23:50:11.881551+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:50:11.881551+0000","last_undegraded":"2024-09-11T23:50:11.881551+0000","last_fullsized":"2024-09-11T23:50:11.881551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:12.424274+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:50:12.424274+0000","last_peered":"2024-09-11T23:50:12.424274+0000","last_clean":"2024-09-11T23:50:12.424274+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:50:12.424274+0000","last_undegraded":"2024-09-11T23:50:12.424274+0000","last_fullsized":"2024-09-11T23:50:12.424274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879978+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:50:11.879978+0000","last_peered":"2024-09-11T23:50:11.879978+0000","last_clean":"2024-09-11T23:50:11.879978+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:50:11.879978+0000","last_undegraded":"2024-09-11T23:50:11.879978+0000","last_fullsized":"2024-09-11T23:50:11.879978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868424+0000","last_change":"2024-09-11T23:50:08.764272+0000","last_active":"2024-09-11T23:50:19.868424+0000","last_peered":"2024-09-11T23:50:19.868424+0000","last_clean":"2024-09-11T23:50:19.868424+0000","last_became_active":"2024-09-11T23:50:08.763978+0000","last_became_peered":"2024-09-11T23:50:08.763978+0000","last_unstale":"2024-09-11T23:50:19.868424+0000","last_undegraded":"2024-09-11T23:50:19.868424+0000","last_fullsized":"2024-09-11T23:50:19.868424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:50.257629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50805,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":150,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":152415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281926,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":419875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1058551,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397543,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1011963,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561068,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":419500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":1011254,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":420330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":419811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":467445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":466095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":420771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":46802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":48152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:50:38.743 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:50:38.744 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T23:50:38.744 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T23:50:39.001 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:39.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:38 smithi062 ceph-mon[30958]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:50:39.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:38 smithi120 ceph-mon[35182]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:50:39.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:38 smithi083 ceph-mon[34805]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:50:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.461+0000 7fb9b92fe640 1 -- 172.21.15.62:0/1292217674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b41037c0 msgr2=0x7fb9b410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.461+0000 7fb9b92fe640 1 --2- 172.21.15.62:0/1292217674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b41037c0 0x7fb9b410a050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009ca0 tx=0x7fb9ac02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:50:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.461+0000 7fb9b92fe640 1 -- 172.21.15.62:0/1292217674 shutdown_connections 2024-09-11T23:50:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.461+0000 7fb9b92fe640 1 --2- 172.21.15.62:0/1292217674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b41037c0 0x7fb9b410a050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.461+0000 7fb9b92fe640 1 --2- 172.21.15.62:0/1292217674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b4102e00 0x7fb9b4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.461+0000 7fb9b92fe640 1 --2- 172.21.15.62:0/1292217674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9b4101c00 0x7fb9b4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.461+0000 7fb9b92fe640 1 -- 172.21.15.62:0/1292217674 >> 172.21.15.62:0/1292217674 conn(0x7fb9b40fd390 msgr2=0x7fb9b40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.462+0000 7fb9b92fe640 1 -- 172.21.15.62:0/1292217674 shutdown_connections 2024-09-11T23:50:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.462+0000 7fb9b92fe640 1 -- 172.21.15.62:0/1292217674 wait complete. 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.462+0000 7fb9b92fe640 1 Processor -- start 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.462+0000 7fb9b92fe640 1 -- start start 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.462+0000 7fb9b92fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b4101c00 0x7fb9b4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.462+0000 7fb9b92fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b4102e00 0x7fb9b4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.462+0000 7fb9b92fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9b41037c0 0x7fb9b419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.462+0000 7fb9b92fe640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b410d900 con 0x7fb9b4101c00 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.462+0000 7fb9b92fe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b410d780 con 0x7fb9b4102e00 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.462+0000 7fb9b92fe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b410da80 con 0x7fb9b41037c0 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b4101c00 0x7fb9b4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b4101c00 0x7fb9b4197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52052/0 (socket says 172.21.15.62:52052) 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b2ffd640 1 -- 172.21.15.62:0/2476652285 learned_addr learned my addr 172.21.15.62:0/2476652285 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:39.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b37fe640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9b41037c0 0x7fb9b419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b27fc640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b4102e00 0x7fb9b4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b2ffd640 1 -- 172.21.15.62:0/2476652285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9b41037c0 msgr2=0x7fb9b419f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b2ffd640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9b41037c0 0x7fb9b419f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b2ffd640 1 -- 172.21.15.62:0/2476652285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b4102e00 msgr2=0x7fb9b4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b2ffd640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b4102e00 0x7fb9b4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b2ffd640 1 -- 172.21.15.62:0/2476652285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b419fb80 con 0x7fb9b4101c00 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b2ffd640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b4101c00 0x7fb9b4197f50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400d950 tx=0x7fb9a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb99bfff640 1 -- 172.21.15.62:0/2476652285 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4017070 con 0x7fb9b4101c00 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b92fe640 1 -- 172.21.15.62:0/2476652285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b419fe70 con 0x7fb9b4101c00 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb99bfff640 1 -- 172.21.15.62:0/2476652285 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a4009c30 con 0x7fb9b4101c00 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb99bfff640 1 -- 172.21.15.62:0/2476652285 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a400f600 con 0x7fb9b4101c00 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.463+0000 7fb9b92fe640 1 -- 172.21.15.62:0/2476652285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b41a0340 con 0x7fb9b4101c00 2024-09-11T23:50:39.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.464+0000 7fb9b92fe640 1 -- 172.21.15.62:0/2476652285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b4108fa0 con 0x7fb9b4101c00 2024-09-11T23:50:39.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.466+0000 7fb99bfff640 1 -- 172.21.15.62:0/2476652285 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb9a401a070 con 0x7fb9b4101c00 2024-09-11T23:50:39.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.466+0000 7fb99bfff640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb99c0a2c40 0x7fb99c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:39.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.466+0000 7fb9b27fc640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb99c0a2c40 0x7fb99c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:39.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.467+0000 7fb99bfff640 1 -- 172.21.15.62:0/2476652285 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb9a40cea50 con 0x7fb9b4101c00 2024-09-11T23:50:39.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.467+0000 7fb9b27fc640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb99c0a2c40 0x7fb99c0a5100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4199470 tx=0x7fb9a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:39.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.469+0000 7fb99bfff640 1 -- 172.21.15.62:0/2476652285 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9a409ba00 con 0x7fb9b4101c00 2024-09-11T23:50:39.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.713+0000 7fb9b92fe640 1 -- 172.21.15.62:0/2476652285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb9b402d7c0 con 0x7fb9b4101c00 2024-09-11T23:50:39.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.715+0000 7fb99bfff640 1 -- 172.21.15.62:0/2476652285 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb9a409b300 con 0x7fb9b4101c00 2024-09-11T23:50:39.716 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-09-11T23:50:39.716 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQDPLOJmi2WQKhAAAUYxyjaYZ+VdR+nunx9S2A== 2024-09-11T23:50:39.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.717+0000 7fb9b92fe640 1 -- 172.21.15.62:0/2476652285 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb99c0a2c40 msgr2=0x7fb99c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:39.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.717+0000 7fb9b92fe640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb99c0a2c40 0x7fb99c0a5100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4199470 tx=0x7fb9a8008040 comp rx=0 tx=0).stop 2024-09-11T23:50:39.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.717+0000 7fb9b92fe640 1 -- 172.21.15.62:0/2476652285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b4101c00 msgr2=0x7fb9b4197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:39.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.717+0000 7fb9b92fe640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b4101c00 0x7fb9b4197f50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400d950 tx=0x7fb9a400de10 comp rx=0 tx=0).stop 2024-09-11T23:50:39.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.717+0000 7fb9b92fe640 1 -- 172.21.15.62:0/2476652285 shutdown_connections 2024-09-11T23:50:39.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.717+0000 7fb9b92fe640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb99c0a2c40 0x7fb99c0a5100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:39.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.717+0000 7fb9b92fe640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9b41037c0 0x7fb9b419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:39.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.717+0000 7fb9b92fe640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9b4102e00 0x7fb9b4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:39.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.717+0000 7fb9b92fe640 1 --2- 172.21.15.62:0/2476652285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b4101c00 0x7fb9b4197f50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:39.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.717+0000 7fb9b92fe640 1 -- 172.21.15.62:0/2476652285 >> 172.21.15.62:0/2476652285 conn(0x7fb9b40fd390 msgr2=0x7fb9b4106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:39.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.718+0000 7fb9b92fe640 1 -- 172.21.15.62:0/2476652285 shutdown_connections 2024-09-11T23:50:39.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:39.718+0000 7fb9b92fe640 1 -- 172.21.15.62:0/2476652285 wait complete. 2024-09-11T23:50:39.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:39 smithi062 ceph-mon[30958]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-09-11T23:50:39.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:39 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2476652285' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:50:39.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:39 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2476652285' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:50:40.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:39 smithi120 ceph-mon[35182]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-09-11T23:50:40.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:39 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2476652285' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:50:40.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:39 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2476652285' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:50:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:39 smithi083 ceph-mon[34805]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-09-11T23:50:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:39 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2476652285' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:50:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:39 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2476652285' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:50:40.506 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:50:40.507 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T23:50:40.507 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T23:50:40.545 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T23:50:40.545 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T23:50:40.545 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T23:50:40.840 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:41.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.335+0000 7fa498e0b640 1 -- 172.21.15.62:0/81217359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa494100480 msgr2=0x7fa494100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.335+0000 7fa498e0b640 1 --2- 172.21.15.62:0/81217359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa494100480 0x7fa494100860 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa484009ca0 tx=0x7fa48402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.336+0000 7fa498e0b640 1 -- 172.21.15.62:0/81217359 shutdown_connections 2024-09-11T23:50:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.336+0000 7fa498e0b640 1 --2- 172.21.15.62:0/81217359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4941055b0 0x7fa49410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.336+0000 7fa498e0b640 1 --2- 172.21.15.62:0/81217359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa494100e30 0x7fa494104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.336+0000 7fa498e0b640 1 --2- 172.21.15.62:0/81217359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa494100480 0x7fa494100860 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.336+0000 7fa498e0b640 1 -- 172.21.15.62:0/81217359 >> 172.21.15.62:0/81217359 conn(0x7fa494077210 msgr2=0x7fa4940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.336+0000 7fa498e0b640 1 -- 172.21.15.62:0/81217359 shutdown_connections 2024-09-11T23:50:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.336+0000 7fa498e0b640 1 -- 172.21.15.62:0/81217359 wait complete. 2024-09-11T23:50:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.336+0000 7fa498e0b640 1 Processor -- start 2024-09-11T23:50:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.336+0000 7fa498e0b640 1 -- start start 2024-09-11T23:50:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa498e0b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa494100480 0x7fa494198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa498e0b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa494100e30 0x7fa494198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa498e0b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4941055b0 0x7fa49419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa498e0b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941105b0 con 0x7fa494100480 2024-09-11T23:50:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa498e0b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa494110430 con 0x7fa4941055b0 2024-09-11T23:50:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa498e0b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa494110730 con 0x7fa494100e30 2024-09-11T23:50:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa49259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa494100480 0x7fa494198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa49259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa494100480 0x7fa494198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42558/0 (socket says 172.21.15.62:42558) 2024-09-11T23:50:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa49259c640 1 -- 172.21.15.62:0/4111522341 learned_addr learned my addr 172.21.15.62:0/4111522341 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa491d9b640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa494100e30 0x7fa494198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa492d9d640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4941055b0 0x7fa49419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa49259c640 1 -- 172.21.15.62:0/4111522341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa494100e30 msgr2=0x7fa494198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa49259c640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa494100e30 0x7fa494198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa49259c640 1 -- 172.21.15.62:0/4111522341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4941055b0 msgr2=0x7fa49419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa49259c640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4941055b0 0x7fa49419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa49259c640 1 -- 172.21.15.62:0/4111522341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49419d3b0 con 0x7fa494100480 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa492d9d640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4941055b0 0x7fa49419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa49259c640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa494100480 0x7fa494198400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa4840097b0 tx=0x7fa484031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa4837fe640 1 -- 172.21.15.62:0/4111522341 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48402fba0 con 0x7fa494100480 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa4837fe640 1 -- 172.21.15.62:0/4111522341 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48402fd40 con 0x7fa494100480 2024-09-11T23:50:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa498e0b640 1 -- 172.21.15.62:0/4111522341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49419d640 con 0x7fa494100480 2024-09-11T23:50:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa4837fe640 1 -- 172.21.15.62:0/4111522341 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48403b930 con 0x7fa494100480 2024-09-11T23:50:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.337+0000 7fa498e0b640 1 -- 172.21.15.62:0/4111522341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4941a4f80 con 0x7fa494100480 2024-09-11T23:50:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.339+0000 7fa498e0b640 1 -- 172.21.15.62:0/4111522341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa494100860 con 0x7fa494100480 2024-09-11T23:50:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.339+0000 7fa4837fe640 1 -- 172.21.15.62:0/4111522341 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa484030cc0 con 0x7fa494100480 2024-09-11T23:50:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.340+0000 7fa4837fe640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa47c0a2c10 0x7fa47c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.340+0000 7fa491d9b640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa47c0a2c10 0x7fa47c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.340+0000 7fa4837fe640 1 -- 172.21.15.62:0/4111522341 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fa4840f06b0 con 0x7fa494100480 2024-09-11T23:50:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.341+0000 7fa491d9b640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa47c0a2c10 0x7fa47c0a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4940feda0 tx=0x7fa48800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.344+0000 7fa4837fe640 1 -- 172.21.15.62:0/4111522341 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4840bd770 con 0x7fa494100480 2024-09-11T23:50:41.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.568+0000 7fa498e0b640 1 -- 172.21.15.62:0/4111522341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa4941a5150 con 0x7fa494100480 2024-09-11T23:50:41.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.573+0000 7fa4837fe640 1 -- 172.21.15.62:0/4111522341 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7fa4840098b0 con 0x7fa494100480 2024-09-11T23:50:41.575 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.575+0000 7fa498e0b640 1 -- 172.21.15.62:0/4111522341 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa47c0a2c10 msgr2=0x7fa47c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.575+0000 7fa498e0b640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa47c0a2c10 0x7fa47c0a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4940feda0 tx=0x7fa48800a4c0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.575+0000 7fa498e0b640 1 -- 172.21.15.62:0/4111522341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa494100480 msgr2=0x7fa494198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.575+0000 7fa498e0b640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa494100480 0x7fa494198400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa4840097b0 tx=0x7fa484031b10 comp rx=0 tx=0).stop 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.576+0000 7fa498e0b640 1 -- 172.21.15.62:0/4111522341 shutdown_connections 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.576+0000 7fa498e0b640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa47c0a2c10 0x7fa47c0a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.576+0000 7fa498e0b640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4941055b0 0x7fa49419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.576+0000 7fa498e0b640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa494100e30 0x7fa494198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.576+0000 7fa498e0b640 1 --2- 172.21.15.62:0/4111522341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa494100480 0x7fa494198400 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.576+0000 7fa498e0b640 1 -- 172.21.15.62:0/4111522341 >> 172.21.15.62:0/4111522341 conn(0x7fa494077210 msgr2=0x7fa4940fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.576+0000 7fa498e0b640 1 -- 172.21.15.62:0/4111522341 shutdown_connections 2024-09-11T23:50:41.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:41.576+0000 7fa498e0b640 1 -- 172.21.15.62:0/4111522341 wait complete. 2024-09-11T23:50:42.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:41 smithi062 ceph-mon[30958]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-11T23:50:42.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:41 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/4111522341' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:50:42.224 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3106394779},{"type":"v1","addr":"172.21.15.62:6801","nonce":3106394779}]},"active_addr":"172.21.15.62:6801/3106394779","active_change":"2024-09-11T23:43:12.720394+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.62:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":133064408}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":2894445674}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":518295197}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":908998551}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":1815817967}]}]} 2024-09-11T23:50:42.229 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T23:50:42.229 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T23:50:42.229 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:50:42.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:41 smithi120 ceph-mon[35182]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-11T23:50:42.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:41 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/4111522341' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:50:42.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:41 smithi083 ceph-mon[34805]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-11T23:50:42.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:41 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/4111522341' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:50:42.487 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:42.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.969+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2962841788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2e4100e20 msgr2=0x7fd2e4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:42.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.969+0000 7fd2e8ada640 1 --2- 172.21.15.62:0/2962841788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2e4100e20 0x7fd2e4101200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009900 tx=0x7fd2d402a480 comp rx=0 tx=0).stop 2024-09-11T23:50:42.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.969+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2962841788 shutdown_connections 2024-09-11T23:50:42.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.969+0000 7fd2e8ada640 1 --2- 172.21.15.62:0/2962841788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e4102ef0 0x7fd2e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:42.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.969+0000 7fd2e8ada640 1 --2- 172.21.15.62:0/2962841788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2e4102570 0x7fd2e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:42.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.969+0000 7fd2e8ada640 1 --2- 172.21.15.62:0/2962841788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2e4100e20 0x7fd2e4101200 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:42.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.969+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2962841788 >> 172.21.15.62:0/2962841788 conn(0x7fd2e40fc7d0 msgr2=0x7fd2e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:42.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.969+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2962841788 shutdown_connections 2024-09-11T23:50:42.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.969+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2962841788 wait complete. 2024-09-11T23:50:42.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e8ada640 1 Processor -- start 2024-09-11T23:50:42.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e8ada640 1 -- start start 2024-09-11T23:50:42.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e8ada640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e4100e20 0x7fd2e4103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:42.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e8ada640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2e4102570 0x7fd2e4103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:42.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e8ada640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2e4102ef0 0x7fd2e41041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:42.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e8ada640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4110f80 con 0x7fd2e4102ef0 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e8ada640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4110e00 con 0x7fd2e4100e20 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e8ada640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4111100 con 0x7fd2e4102570 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2e4102ef0 0x7fd2e41041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2e4102ef0 0x7fd2e41041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42590/0 (socket says 172.21.15.62:42590) 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e2d9d640 1 -- 172.21.15.62:0/2842060801 learned_addr learned my addr 172.21.15.62:0/2842060801 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e259c640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e4100e20 0x7fd2e4103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2dbfff640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2e4102570 0x7fd2e4103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e2d9d640 1 -- 172.21.15.62:0/2842060801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2e4102570 msgr2=0x7fd2e4103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e2d9d640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2e4102570 0x7fd2e4103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e2d9d640 1 -- 172.21.15.62:0/2842060801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e4100e20 msgr2=0x7fd2e4103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e2d9d640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e4100e20 0x7fd2e4103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2e2d9d640 1 -- 172.21.15.62:0/2842060801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e41a69b0 con 0x7fd2e4102ef0 2024-09-11T23:50:42.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.970+0000 7fd2dbfff640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2e4102570 0x7fd2e4103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:50:42.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.971+0000 7fd2e2d9d640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2e4102ef0 0x7fd2e41041d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00eaa0 tx=0x7fd2dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:42.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.971+0000 7fd2db7fe640 1 -- 172.21.15.62:0/2842060801 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00cdb0 con 0x7fd2e4102ef0 2024-09-11T23:50:42.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.971+0000 7fd2db7fe640 1 -- 172.21.15.62:0/2842060801 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc005020 con 0x7fd2e4102ef0 2024-09-11T23:50:42.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.971+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2842060801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e41a6ca0 con 0x7fd2e4102ef0 2024-09-11T23:50:42.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.971+0000 7fd2db7fe640 1 -- 172.21.15.62:0/2842060801 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc0116b0 con 0x7fd2e4102ef0 2024-09-11T23:50:42.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.971+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2842060801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a7190 con 0x7fd2e4102ef0 2024-09-11T23:50:42.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.972+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2842060801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e4101200 con 0x7fd2e4102ef0 2024-09-11T23:50:42.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.973+0000 7fd2db7fe640 1 -- 172.21.15.62:0/2842060801 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd2dc005560 con 0x7fd2e4102ef0 2024-09-11T23:50:42.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.974+0000 7fd2db7fe640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd2d00a2c10 0x7fd2d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:42.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.974+0000 7fd2e259c640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd2d00a2c10 0x7fd2d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:42.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.974+0000 7fd2db7fe640 1 -- 172.21.15.62:0/2842060801 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fd2dc0cf230 con 0x7fd2e4102ef0 2024-09-11T23:50:42.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.974+0000 7fd2e259c640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd2d00a2c10 0x7fd2d00a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009230 tx=0x7fd2d40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:42.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:42.978+0000 7fd2db7fe640 1 -- 172.21.15.62:0/2842060801 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2dc016030 con 0x7fd2e4102ef0 2024-09-11T23:50:43.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.155+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2842060801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2e4104c80 con 0x7fd2e4102ef0 2024-09-11T23:50:43.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.157+0000 7fd2db7fe640 1 -- 172.21.15.62:0/2842060801 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22244 (secure 0 0 0) 0x7fd2e4104c80 con 0x7fd2e4102ef0 2024-09-11T23:50:43.156 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:43.156 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:50:11.406572+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:50:43.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2842060801 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd2d00a2c10 msgr2=0x7fd2d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:43.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd2d00a2c10 0x7fd2d00a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009230 tx=0x7fd2d40058d0 comp rx=0 tx=0).stop 2024-09-11T23:50:43.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2842060801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2e4102ef0 msgr2=0x7fd2e41041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:43.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2e4102ef0 0x7fd2e41041d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00eaa0 tx=0x7fd2dc00ef60 comp rx=0 tx=0).stop 2024-09-11T23:50:43.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2842060801 shutdown_connections 2024-09-11T23:50:43.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd2d00a2c10 0x7fd2d00a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:43.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2e4102ef0 0x7fd2e41041d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:43.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2e4102570 0x7fd2e4103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:43.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 --2- 172.21.15.62:0/2842060801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e4100e20 0x7fd2e4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:43.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2842060801 >> 172.21.15.62:0/2842060801 conn(0x7fd2e40fc7d0 msgr2=0x7fd2e406a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:43.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2842060801 shutdown_connections 2024-09-11T23:50:43.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:43.159+0000 7fd2e8ada640 1 -- 172.21.15.62:0/2842060801 wait complete. 2024-09-11T23:50:43.903 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T23:50:43.903 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:50:44.160 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:44.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:43 smithi062 ceph-mon[30958]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-11T23:50:44.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:43 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2842060801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:44.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:43 smithi120 ceph-mon[35182]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-11T23:50:44.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:43 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2842060801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:44.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:43 smithi083 ceph-mon[34805]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-11T23:50:44.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:43 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2842060801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:44.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.619+0000 7f5d2341f640 1 -- 172.21.15.62:0/1587434733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1c102570 msgr2=0x7f5d1c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.619+0000 7f5d2341f640 1 --2- 172.21.15.62:0/1587434733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1c102570 0x7f5d1c1029b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009900 tx=0x7f5d0c02a480 comp rx=0 tx=0).stop 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.619+0000 7f5d2341f640 1 -- 172.21.15.62:0/1587434733 shutdown_connections 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.619+0000 7f5d2341f640 1 --2- 172.21.15.62:0/1587434733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d1c102ef0 0x7f5d1c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.619+0000 7f5d2341f640 1 --2- 172.21.15.62:0/1587434733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1c102570 0x7f5d1c1029b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.619+0000 7f5d2341f640 1 --2- 172.21.15.62:0/1587434733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d1c100e20 0x7f5d1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.619+0000 7f5d2341f640 1 -- 172.21.15.62:0/1587434733 >> 172.21.15.62:0/1587434733 conn(0x7f5d1c0fc7d0 msgr2=0x7f5d1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.619+0000 7f5d2341f640 1 -- 172.21.15.62:0/1587434733 shutdown_connections 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.619+0000 7f5d2341f640 1 -- 172.21.15.62:0/1587434733 wait complete. 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d2341f640 1 Processor -- start 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d2341f640 1 -- start start 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d2341f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1c100e20 0x7f5d1c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:44.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d2341f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d1c102570 0x7f5d1c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d2341f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d1c102ef0 0x7f5d1c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d2341f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c110fa0 con 0x7f5d1c100e20 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d2341f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c110e20 con 0x7f5d1c102ef0 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d2341f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c111120 con 0x7f5d1c102570 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d211bb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1c100e20 0x7f5d1c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d211bb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1c100e20 0x7f5d1c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42608/0 (socket says 172.21.15.62:42608) 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d211bb640 1 -- 172.21.15.62:0/293016045 learned_addr learned my addr 172.21.15.62:0/293016045 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.620+0000 7f5d219bc640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d1c102ef0 0x7f5d1c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d209ba640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d1c102570 0x7f5d1c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d211bb640 1 -- 172.21.15.62:0/293016045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d1c102570 msgr2=0x7f5d1c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d211bb640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d1c102570 0x7f5d1c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:44.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d211bb640 1 -- 172.21.15.62:0/293016045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d1c102ef0 msgr2=0x7f5d1c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d211bb640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d1c102ef0 0x7f5d1c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d211bb640 1 -- 172.21.15.62:0/293016045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1c19f6c0 con 0x7f5d1c100e20 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d211bb640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1c100e20 0x7f5d1c19a710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800d950 tx=0x7f5d1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d127fc640 1 -- 172.21.15.62:0/293016045 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18017070 con 0x7f5d1c100e20 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d2341f640 1 -- 172.21.15.62:0/293016045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1c19f9b0 con 0x7f5d1c100e20 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d127fc640 1 -- 172.21.15.62:0/293016045 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d18009c30 con 0x7f5d1c100e20 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d127fc640 1 -- 172.21.15.62:0/293016045 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1800f6b0 con 0x7f5d1c100e20 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.621+0000 7f5d2341f640 1 -- 172.21.15.62:0/293016045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d1c1a7290 con 0x7f5d1c100e20 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.622+0000 7f5d2341f640 1 -- 172.21.15.62:0/293016045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d1c101200 con 0x7f5d1c100e20 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.623+0000 7f5d127fc640 1 -- 172.21.15.62:0/293016045 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5d180109d0 con 0x7f5d1c100e20 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.624+0000 7f5d127fc640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5d080a2c10 0x7f5d080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:44.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.624+0000 7f5d209ba640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5d080a2c10 0x7f5d080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:44.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.624+0000 7f5d127fc640 1 -- 172.21.15.62:0/293016045 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f5d180ced50 con 0x7f5d1c100e20 2024-09-11T23:50:44.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.625+0000 7f5d209ba640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5d080a2c10 0x7f5d080a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c104bd0 tx=0x7f5d0c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:44.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.628+0000 7f5d127fc640 1 -- 172.21.15.62:0/293016045 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d18009de0 con 0x7f5d1c100e20 2024-09-11T23:50:44.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.804+0000 7f5d2341f640 1 -- 172.21.15.62:0/293016045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d1c1a74c0 con 0x7f5d1c100e20 2024-09-11T23:50:44.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.806+0000 7f5d127fc640 1 -- 172.21.15.62:0/293016045 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22244 (secure 0 0 0) 0x7f5d1809bc00 con 0x7f5d1c100e20 2024-09-11T23:50:44.805 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:50:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:50:11.406572+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:50:44.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 -- 172.21.15.62:0/293016045 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5d080a2c10 msgr2=0x7f5d080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:44.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5d080a2c10 0x7f5d080a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c104bd0 tx=0x7f5d0c005a20 comp rx=0 tx=0).stop 2024-09-11T23:50:44.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 -- 172.21.15.62:0/293016045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1c100e20 msgr2=0x7f5d1c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:44.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1c100e20 0x7f5d1c19a710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800d950 tx=0x7f5d1800de10 comp rx=0 tx=0).stop 2024-09-11T23:50:44.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 -- 172.21.15.62:0/293016045 shutdown_connections 2024-09-11T23:50:44.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5d080a2c10 0x7f5d080a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:44.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d1c102ef0 0x7f5d1c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d1c102570 0x7f5d1c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 --2- 172.21.15.62:0/293016045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1c100e20 0x7f5d1c19a710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 -- 172.21.15.62:0/293016045 >> 172.21.15.62:0/293016045 conn(0x7f5d1c0fc7d0 msgr2=0x7f5d1c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 -- 172.21.15.62:0/293016045 shutdown_connections 2024-09-11T23:50:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:44.808+0000 7f5d2341f640 1 -- 172.21.15.62:0/293016045 wait complete. 2024-09-11T23:50:45.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:44 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/293016045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:45.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:44 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/293016045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:45.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:44 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/293016045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:50:45.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T23:50:45.548 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T23:50:45.548 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T23:50:45.549 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T23:50:45.549 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T23:50:45.550 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T23:50:45.550 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T23:50:45.551 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T23:50:45.551 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T23:50:45.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T23:50:45.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T23:50:45.553 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T23:50:45.943 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:45 smithi062 ceph-mon[30958]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-11T23:50:46.246 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:46.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:45 smithi120 ceph-mon[35182]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-11T23:50:46.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:45 smithi083 ceph-mon[34805]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-11T23:50:46.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.855+0000 7f7b5d77d640 1 -- 172.21.15.62:0/3516169034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58103660 msgr2=0x7f7b5810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.855+0000 7f7b5d77d640 1 --2- 172.21.15.62:0/3516169034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58103660 0x7f7b5810bb50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009900 tx=0x7f7b4c02a480 comp rx=0 tx=0).stop 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.855+0000 7f7b5d77d640 1 -- 172.21.15.62:0/3516169034 shutdown_connections 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.855+0000 7f7b5d77d640 1 --2- 172.21.15.62:0/3516169034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5810c170 0x7f7b5810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.855+0000 7f7b5d77d640 1 --2- 172.21.15.62:0/3516169034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58103660 0x7f7b5810bb50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.855+0000 7f7b5d77d640 1 --2- 172.21.15.62:0/3516169034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58102d40 0x7f7b58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.855+0000 7f7b5d77d640 1 -- 172.21.15.62:0/3516169034 >> 172.21.15.62:0/3516169034 conn(0x7f7b580fc7a0 msgr2=0x7f7b580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.855+0000 7f7b5d77d640 1 -- 172.21.15.62:0/3516169034 shutdown_connections 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.855+0000 7f7b5d77d640 1 -- 172.21.15.62:0/3516169034 wait complete. 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.856+0000 7f7b5d77d640 1 Processor -- start 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.856+0000 7f7b5d77d640 1 -- start start 2024-09-11T23:50:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b5d77d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58102d40 0x7f7b58198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b5d77d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58103660 0x7f7b58198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b5d77d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5810c170 0x7f7b5819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b5d77d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58111fb0 con 0x7f7b58103660 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b5d77d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58111e30 con 0x7f7b5810c170 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b5d77d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58112130 con 0x7f7b58102d40 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b567fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58103660 0x7f7b58198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b567fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58103660 0x7f7b58198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42612/0 (socket says 172.21.15.62:42612) 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b567fc640 1 -- 172.21.15.62:0/1060188874 learned_addr learned my addr 172.21.15.62:0/1060188874 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b56ffd640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58102d40 0x7f7b58198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b577fe640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5810c170 0x7f7b5819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b567fc640 1 -- 172.21.15.62:0/1060188874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58102d40 msgr2=0x7f7b58198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b567fc640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58102d40 0x7f7b58198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b567fc640 1 -- 172.21.15.62:0/1060188874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5810c170 msgr2=0x7f7b5819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b567fc640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5810c170 0x7f7b5819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b567fc640 1 -- 172.21.15.62:0/1060188874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5819d5c0 con 0x7f7b58103660 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b577fe640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5810c170 0x7f7b5819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b567fc640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58103660 0x7f7b58198b50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005ab0 tx=0x7f7b4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b3ffff640 1 -- 172.21.15.62:0/1060188874 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c03f030 con 0x7f7b58103660 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5819d850 con 0x7f7b58103660 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b3ffff640 1 -- 172.21.15.62:0/1060188874 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4c003ce0 con 0x7f7b58103660 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.857+0000 7f7b3ffff640 1 -- 172.21.15.62:0/1060188874 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c035070 con 0x7f7b58103660 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.858+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b581a5190 con 0x7f7b58103660 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.858+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b58062900 con 0x7f7b58103660 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.860+0000 7f7b3ffff640 1 -- 172.21.15.62:0/1060188874 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7b4c042030 con 0x7f7b58103660 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.860+0000 7f7b3ffff640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7b400a2c10 0x7f7b400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.860+0000 7f7b56ffd640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7b400a2c10 0x7f7b400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.860+0000 7f7b3ffff640 1 -- 172.21.15.62:0/1060188874 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f7b4c0ef4a0 con 0x7f7b58103660 2024-09-11T23:50:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.860+0000 7f7b3ffff640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] conn(0x7f7b400ac460 0x7f7b400ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.861+0000 7f7b3ffff640 1 -- 172.21.15.62:0/1060188874 --> [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b4c003b30 con 0x7f7b400ac460 2024-09-11T23:50:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.861+0000 7f7b3ffff640 1 -- 172.21.15.62:0/1060188874 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f7b4c0f6050 con 0x7f7b58103660 2024-09-11T23:50:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.861+0000 7f7b56ffd640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7b400a2c10 0x7f7b400a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009990 tx=0x7f7b48009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.861+0000 7f7b577fe640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] conn(0x7f7b400ac460 0x7f7b400ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.861+0000 7f7b577fe640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] conn(0x7f7b400ac460 0x7f7b400ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.862+0000 7f7b3ffff640 1 -- 172.21.15.62:0/1060188874 <== osd.2 v2:172.21.15.120:6800/3994166921 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b4c003b30 con 0x7f7b400ac460 2024-09-11T23:50:46.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.876+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 --> [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b58062d00 con 0x7f7b400ac460 2024-09-11T23:50:46.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.876+0000 7f7b3ffff640 1 -- 172.21.15.62:0/1060188874 <== osd.2 v2:172.21.15.120:6800/3994166921 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7b58062d00 con 0x7f7b400ac460 2024-09-11T23:50:46.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 >> [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] conn(0x7f7b400ac460 msgr2=0x7f7b400ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:46.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b5d77d640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] conn(0x7f7b400ac460 0x7f7b400ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:46.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7b400a2c10 msgr2=0x7f7b400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:46.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b5d77d640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7b400a2c10 0x7f7b400a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009990 tx=0x7f7b48009110 comp rx=0 tx=0).stop 2024-09-11T23:50:46.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58103660 msgr2=0x7f7b58198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:46.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b5d77d640 1 --2- 172.21.15.62:0/1060188874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b58103660 0x7f7b58198b50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005ab0 tx=0x7f7b4c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:50:46.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b577fe640 1 -- 172.21.15.62:0/1060188874 reap_dead start 2024-09-11T23:50:46.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 shutdown_connections 2024-09-11T23:50:46.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 >> 172.21.15.62:0/1060188874 conn(0x7f7b580fc7a0 msgr2=0x7f7b580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:46.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 shutdown_connections 2024-09-11T23:50:46.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:46.877+0000 7f7b5d77d640 1 -- 172.21.15.62:0/1060188874 wait complete. 2024-09-11T23:50:47.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.493+0000 7f9289526640 1 -- 172.21.15.62:0/2879077055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9284102570 msgr2=0x7f92841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:47.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.493+0000 7f9289526640 1 --2- 172.21.15.62:0/2879077055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9284102570 0x7f92841029b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f926c009900 tx=0x7f926c02a480 comp rx=0 tx=0).stop 2024-09-11T23:50:47.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.493+0000 7f9289526640 1 -- 172.21.15.62:0/2879077055 shutdown_connections 2024-09-11T23:50:47.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.493+0000 7f9289526640 1 --2- 172.21.15.62:0/2879077055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284102ef0 0x7f928410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:47.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.493+0000 7f9289526640 1 --2- 172.21.15.62:0/2879077055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9284102570 0x7f92841029b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:47.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.493+0000 7f9289526640 1 --2- 172.21.15.62:0/2879077055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9284100e20 0x7f9284101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:47.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.493+0000 7f9289526640 1 -- 172.21.15.62:0/2879077055 >> 172.21.15.62:0/2879077055 conn(0x7f92840fc7d0 msgr2=0x7f92840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:47.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.493+0000 7f9289526640 1 -- 172.21.15.62:0/2879077055 shutdown_connections 2024-09-11T23:50:47.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.493+0000 7f9289526640 1 -- 172.21.15.62:0/2879077055 wait complete. 2024-09-11T23:50:47.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f9289526640 1 Processor -- start 2024-09-11T23:50:47.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f9289526640 1 -- start start 2024-09-11T23:50:47.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f9289526640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284100e20 0x7f92841994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:47.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f9289526640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9284102570 0x7f9284199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:47.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f9289526640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9284102ef0 0x7f92841a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:47.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f9289526640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9284111070 con 0x7f9284102570 2024-09-11T23:50:47.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f9289526640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9284110ef0 con 0x7f9284102ef0 2024-09-11T23:50:47.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f9289526640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92841111f0 con 0x7f9284100e20 2024-09-11T23:50:47.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f92827fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9284102570 0x7f9284199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:47.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f92827fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9284102570 0x7f9284199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42628/0 (socket says 172.21.15.62:42628) 2024-09-11T23:50:47.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.494+0000 7f92827fc640 1 -- 172.21.15.62:0/454298409 learned_addr learned my addr 172.21.15.62:0/454298409 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:47.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f92837fe640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9284102ef0 0x7f92841a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:47.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f9282ffd640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284100e20 0x7f92841994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:47.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f92827fc640 1 -- 172.21.15.62:0/454298409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284100e20 msgr2=0x7f92841994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:47.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f92827fc640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284100e20 0x7f92841994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:47.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f92827fc640 1 -- 172.21.15.62:0/454298409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9284102ef0 msgr2=0x7f92841a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:47.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f92827fc640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9284102ef0 0x7f92841a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:47.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f92827fc640 1 -- 172.21.15.62:0/454298409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92841a4aa0 con 0x7f9284102570 2024-09-11T23:50:47.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f92827fc640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9284102570 0x7f9284199a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f926c006010 tx=0x7f926c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:47.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f927bfff640 1 -- 172.21.15.62:0/454298409 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c03f070 con 0x7f9284102570 2024-09-11T23:50:47.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f927bfff640 1 -- 172.21.15.62:0/454298409 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f926c003e30 con 0x7f9284102570 2024-09-11T23:50:47.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92841a4c70 con 0x7f9284102570 2024-09-11T23:50:47.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f927bfff640 1 -- 172.21.15.62:0/454298409 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c035030 con 0x7f9284102570 2024-09-11T23:50:47.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.495+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92841a5120 con 0x7f9284102570 2024-09-11T23:50:47.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.496+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9284062900 con 0x7f9284102570 2024-09-11T23:50:47.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.497+0000 7f927bfff640 1 -- 172.21.15.62:0/454298409 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f926c042070 con 0x7f9284102570 2024-09-11T23:50:47.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.498+0000 7f927bfff640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f92700a2c90 0x7f92700a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:47.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.498+0000 7f9282ffd640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f92700a2c90 0x7f92700a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:47.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.498+0000 7f927bfff640 1 -- 172.21.15.62:0/454298409 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f926c0efb50 con 0x7f9284102570 2024-09-11T23:50:47.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.498+0000 7f927bfff640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] conn(0x7f92700ac4e0 0x7f92700ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:47.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.498+0000 7f927bfff640 1 -- 172.21.15.62:0/454298409 --> [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f926c00b0b0 con 0x7f92700ac4e0 2024-09-11T23:50:47.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.498+0000 7f927bfff640 1 -- 172.21.15.62:0/454298409 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f926c0f0030 con 0x7f9284102570 2024-09-11T23:50:47.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.499+0000 7f9282ffd640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f92700a2c90 0x7f92700a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f927c009f20 tx=0x7f927c0094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:47.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.499+0000 7f92837fe640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] conn(0x7f92700ac4e0 0x7f92700ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:47.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.499+0000 7f92837fe640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] conn(0x7f92700ac4e0 0x7f92700ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:47.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.500+0000 7f927bfff640 1 -- 172.21.15.62:0/454298409 <== osd.5 v2:172.21.15.120:6808/186624354 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f926c00b0b0 con 0x7f92700ac4e0 2024-09-11T23:50:47.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.514+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 --> [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9284062d80 con 0x7f92700ac4e0 2024-09-11T23:50:47.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.514+0000 7f927bfff640 1 -- 172.21.15.62:0/454298409 <== osd.5 v2:172.21.15.120:6808/186624354 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9284062d80 con 0x7f92700ac4e0 2024-09-11T23:50:47.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.514+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 >> [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] conn(0x7f92700ac4e0 msgr2=0x7f92700ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.514+0000 7f9289526640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] conn(0x7f92700ac4e0 0x7f92700ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.514+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f92700a2c90 msgr2=0x7f92700a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.514+0000 7f9289526640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f92700a2c90 0x7f92700a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f927c009f20 tx=0x7f927c0094f0 comp rx=0 tx=0).stop 2024-09-11T23:50:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.514+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9284102570 msgr2=0x7f9284199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.514+0000 7f9289526640 1 --2- 172.21.15.62:0/454298409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9284102570 0x7f9284199a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f926c006010 tx=0x7f926c003c10 comp rx=0 tx=0).stop 2024-09-11T23:50:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.514+0000 7f92837fe640 1 -- 172.21.15.62:0/454298409 reap_dead start 2024-09-11T23:50:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.515+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 shutdown_connections 2024-09-11T23:50:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.515+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 >> 172.21.15.62:0/454298409 conn(0x7f92840fc7d0 msgr2=0x7f92840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.515+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 shutdown_connections 2024-09-11T23:50:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:47.515+0000 7f9289526640 1 -- 172.21.15.62:0/454298409 wait complete. 2024-09-11T23:50:48.035 INFO:teuthology.orchestra.run.smithi062.stdout:77309411394 2024-09-11T23:50:48.036 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T23:50:48.048 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:48.049 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:48.052 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:48.052 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:48.055 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:48.055 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:48.059 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:48.059 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:48.066 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:48.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:47 smithi062 ceph-mon[30958]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-11T23:50:48.241 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:48.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:47 smithi120 ceph-mon[35182]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-11T23:50:48.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:47 smithi083 ceph-mon[34805]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-11T23:50:48.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.901+0000 7f1545421640 1 -- 172.21.15.62:0/168938204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540102570 msgr2=0x7f15401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:48.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.901+0000 7f1545421640 1 --2- 172.21.15.62:0/168938204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540102570 0x7f15401029b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1528009900 tx=0x7f152802a480 comp rx=0 tx=0).stop 2024-09-11T23:50:48.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.902+0000 7f1545421640 1 -- 172.21.15.62:0/168938204 shutdown_connections 2024-09-11T23:50:48.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.902+0000 7f1545421640 1 --2- 172.21.15.62:0/168938204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1540102ef0 0x7f154010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:48.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.902+0000 7f1545421640 1 --2- 172.21.15.62:0/168938204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540102570 0x7f15401029b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:48.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.902+0000 7f1545421640 1 --2- 172.21.15.62:0/168938204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1540100e20 0x7f1540101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:48.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.902+0000 7f1545421640 1 -- 172.21.15.62:0/168938204 >> 172.21.15.62:0/168938204 conn(0x7f15400fc7d0 msgr2=0x7f15400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:48.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.902+0000 7f1545421640 1 -- 172.21.15.62:0/168938204 shutdown_connections 2024-09-11T23:50:48.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.902+0000 7f1545421640 1 -- 172.21.15.62:0/168938204 wait complete. 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.902+0000 7f1545421640 1 Processor -- start 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.902+0000 7f1545421640 1 -- start start 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f1545421640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540100e20 0x7f1540198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f1545421640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1540102570 0x7f1540198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f1545421640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1540102ef0 0x7f154019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f1545421640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1540110f80 con 0x7f1540100e20 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f1545421640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1540110e00 con 0x7f1540102570 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f1545421640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1540111100 con 0x7f1540102ef0 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540100e20 0x7f1540198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540100e20 0x7f1540198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42648/0 (socket says 172.21.15.62:42648) 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153effd640 1 -- 172.21.15.62:0/2714467332 learned_addr learned my addr 172.21.15.62:0/2714467332 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153e7fc640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1540102570 0x7f1540198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:48.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153f7fe640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1540102ef0 0x7f154019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153effd640 1 -- 172.21.15.62:0/2714467332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1540102ef0 msgr2=0x7f154019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153effd640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1540102ef0 0x7f154019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153effd640 1 -- 172.21.15.62:0/2714467332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1540102570 msgr2=0x7f1540198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153effd640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1540102570 0x7f1540198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153effd640 1 -- 172.21.15.62:0/2714467332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154019d510 con 0x7f1540100e20 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153e7fc640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1540102570 0x7f1540198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153f7fe640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1540102ef0 0x7f154019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f153effd640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540100e20 0x7f1540198560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f153000ebc0 tx=0x7f153000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f1537fff640 1 -- 172.21.15.62:0/2714467332 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530010630 con 0x7f1540100e20 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f1537fff640 1 -- 172.21.15.62:0/2714467332 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1530010cb0 con 0x7f1540100e20 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.903+0000 7f1537fff640 1 -- 172.21.15.62:0/2714467332 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530018bd0 con 0x7f1540100e20 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.904+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154019d800 con 0x7f1540100e20 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.904+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15401a50e0 con 0x7f1540100e20 2024-09-11T23:50:48.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.904+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1540062980 con 0x7f1540100e20 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.906+0000 7f1537fff640 1 -- 172.21.15.62:0/2714467332 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f15300107d0 con 0x7f1540100e20 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.906+0000 7f1537fff640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f152c0a2c10 0x7f152c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.906+0000 7f153e7fc640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f152c0a2c10 0x7f152c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.906+0000 7f1537fff640 1 -- 172.21.15.62:0/2714467332 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f15300cf7a0 con 0x7f1540100e20 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.907+0000 7f1537fff640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] conn(0x7f152c0ac460 0x7f152c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.907+0000 7f1537fff640 1 -- 172.21.15.62:0/2714467332 --> [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f152c0aef30 con 0x7f152c0ac460 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.907+0000 7f1537fff640 1 -- 172.21.15.62:0/2714467332 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f15300d42c0 con 0x7f1540100e20 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.907+0000 7f153f7fe640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] conn(0x7f152c0ac460 0x7f152c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.907+0000 7f153e7fc640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f152c0a2c10 0x7f152c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1528000c00 tx=0x7f1528004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.907+0000 7f153f7fe640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] conn(0x7f152c0ac460 0x7f152c0ae880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:48.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.908+0000 7f1537fff640 1 -- 172.21.15.62:0/2714467332 <== osd.3 v2:172.21.15.62:6810/1879962337 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f152c0aef30 con 0x7f152c0ac460 2024-09-11T23:50:48.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.921+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 --> [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1540062e00 con 0x7f152c0ac460 2024-09-11T23:50:48.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.922+0000 7f1537fff640 1 -- 172.21.15.62:0/2714467332 <== osd.3 v2:172.21.15.62:6810/1879962337 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1540062e00 con 0x7f152c0ac460 2024-09-11T23:50:48.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.922+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] conn(0x7f152c0ac460 msgr2=0x7f152c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:48.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.922+0000 7f1545421640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] conn(0x7f152c0ac460 0x7f152c0ae880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:48.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.922+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f152c0a2c10 msgr2=0x7f152c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:48.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.922+0000 7f1545421640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f152c0a2c10 0x7f152c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1528000c00 tx=0x7f1528004100 comp rx=0 tx=0).stop 2024-09-11T23:50:48.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.922+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540100e20 msgr2=0x7f1540198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:48.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.922+0000 7f1545421640 1 --2- 172.21.15.62:0/2714467332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1540100e20 0x7f1540198560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f153000ebc0 tx=0x7f153000c6a0 comp rx=0 tx=0).stop 2024-09-11T23:50:48.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.922+0000 7f153f7fe640 1 -- 172.21.15.62:0/2714467332 reap_dead start 2024-09-11T23:50:48.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.922+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 shutdown_connections 2024-09-11T23:50:48.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.922+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 >> 172.21.15.62:0/2714467332 conn(0x7f15400fc7d0 msgr2=0x7f154006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:48.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.923+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 shutdown_connections 2024-09-11T23:50:48.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:48.923+0000 7f1545421640 1 -- 172.21.15.62:0/2714467332 wait complete. 2024-09-11T23:50:50.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:49 smithi062 ceph-mon[30958]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-11T23:50:50.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:49 smithi120 ceph-mon[35182]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-11T23:50:50.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:49 smithi083 ceph-mon[34805]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-11T23:50:50.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.657+0000 7fa287c5f640 1 -- 172.21.15.62:0/1217724499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa280106f10 msgr2=0x7fa28010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.657+0000 7fa287c5f640 1 --2- 172.21.15.62:0/1217724499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa280106f10 0x7fa28010ea90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa278009ca0 tx=0x7fa27802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.657+0000 7fa287c5f640 1 -- 172.21.15.62:0/1217724499 shutdown_connections 2024-09-11T23:50:50.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.657+0000 7fa287c5f640 1 --2- 172.21.15.62:0/1217724499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa280106f10 0x7fa28010ea90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.657+0000 7fa287c5f640 1 --2- 172.21.15.62:0/1217724499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa280102790 0x7fa2801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.657+0000 7fa287c5f640 1 --2- 172.21.15.62:0/1217724499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa280101de0 0x7fa2801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.657+0000 7fa287c5f640 1 -- 172.21.15.62:0/1217724499 >> 172.21.15.62:0/1217724499 conn(0x7fa2800fd880 msgr2=0x7fa2800ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.657+0000 7fa287c5f640 1 -- 172.21.15.62:0/1217724499 shutdown_connections 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.657+0000 7fa287c5f640 1 -- 172.21.15.62:0/1217724499 wait complete. 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa287c5f640 1 Processor -- start 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa287c5f640 1 -- start start 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa287c5f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa280101de0 0x7fa28019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa287c5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa280102790 0x7fa28019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa287c5f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa280106f10 0x7fa28019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa287c5f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa280111fc0 con 0x7fa280101de0 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa287c5f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa280111e40 con 0x7fa280102790 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa287c5f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa280112140 con 0x7fa280106f10 2024-09-11T23:50:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa2859fb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa280101de0 0x7fa28019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa2859fb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa280101de0 0x7fa28019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42676/0 (socket says 172.21.15.62:42676) 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa2859fb640 1 -- 172.21.15.62:0/1789296073 learned_addr learned my addr 172.21.15.62:0/1789296073 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.658+0000 7fa2851fa640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa280102790 0x7fa28019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa2861fc640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa280106f10 0x7fa28019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa2859fb640 1 -- 172.21.15.62:0/1789296073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa280106f10 msgr2=0x7fa28019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa2859fb640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa280106f10 0x7fa28019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa2859fb640 1 -- 172.21.15.62:0/1789296073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa280102790 msgr2=0x7fa28019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa2859fb640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa280102790 0x7fa28019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa2859fb640 1 -- 172.21.15.62:0/1789296073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28019f620 con 0x7fa280101de0 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa2859fb640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa280101de0 0x7fa28019a670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00c970 tx=0x7fa27c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa276ffd640 1 -- 172.21.15.62:0/1789296073 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c007bb0 con 0x7fa280101de0 2024-09-11T23:50:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28019f910 con 0x7fa280101de0 2024-09-11T23:50:50.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa276ffd640 1 -- 172.21.15.62:0/1789296073 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa27c007d50 con 0x7fa280101de0 2024-09-11T23:50:50.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa276ffd640 1 -- 172.21.15.62:0/1789296073 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00f450 con 0x7fa280101de0 2024-09-11T23:50:50.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.659+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2801a71f0 con 0x7fa280101de0 2024-09-11T23:50:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.660+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa264001030 con 0x7fa280101de0 2024-09-11T23:50:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.661+0000 7fa276ffd640 1 -- 172.21.15.62:0/1789296073 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa27c0046c0 con 0x7fa280101de0 2024-09-11T23:50:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.662+0000 7fa276ffd640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa26c0a2c10 0x7fa26c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.662+0000 7fa2851fa640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa26c0a2c10 0x7fa26c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.662+0000 7fa276ffd640 1 -- 172.21.15.62:0/1789296073 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fa27c0cf1c0 con 0x7fa280101de0 2024-09-11T23:50:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.662+0000 7fa276ffd640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] conn(0x7fa26c0ac460 0x7fa26c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.662+0000 7fa276ffd640 1 -- 172.21.15.62:0/1789296073 --> [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa27c012ec0 con 0x7fa26c0ac460 2024-09-11T23:50:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.662+0000 7fa276ffd640 1 -- 172.21.15.62:0/1789296073 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fa27c0cf570 con 0x7fa280101de0 2024-09-11T23:50:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.663+0000 7fa2861fc640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] conn(0x7fa26c0ac460 0x7fa26c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.663+0000 7fa2851fa640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa26c0a2c10 0x7fa26c0a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa2800fdef0 tx=0x7fa27000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:50.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.663+0000 7fa2861fc640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] conn(0x7fa26c0ac460 0x7fa26c0ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:50.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.664+0000 7fa276ffd640 1 -- 172.21.15.62:0/1789296073 <== osd.11 v2:172.21.15.120:6824/4266702819 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa27c012ec0 con 0x7fa26c0ac460 2024-09-11T23:50:50.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.685+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 --> [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa264002dd0 con 0x7fa26c0ac460 2024-09-11T23:50:50.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.685+0000 7fa276ffd640 1 -- 172.21.15.62:0/1789296073 <== osd.11 v2:172.21.15.120:6824/4266702819 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa264002dd0 con 0x7fa26c0ac460 2024-09-11T23:50:50.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 >> [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] conn(0x7fa26c0ac460 msgr2=0x7fa26c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa287c5f640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] conn(0x7fa26c0ac460 0x7fa26c0ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa26c0a2c10 msgr2=0x7fa26c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa287c5f640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa26c0a2c10 0x7fa26c0a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa2800fdef0 tx=0x7fa27000b390 comp rx=0 tx=0).stop 2024-09-11T23:50:50.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa280101de0 msgr2=0x7fa28019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa287c5f640 1 --2- 172.21.15.62:0/1789296073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa280101de0 0x7fa28019a670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00c970 tx=0x7fa27c00ce30 comp rx=0 tx=0).stop 2024-09-11T23:50:50.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa2861fc640 1 -- 172.21.15.62:0/1789296073 reap_dead start 2024-09-11T23:50:50.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 shutdown_connections 2024-09-11T23:50:50.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 >> 172.21.15.62:0/1789296073 conn(0x7fa2800fd880 msgr2=0x7fa2800feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:50.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 shutdown_connections 2024-09-11T23:50:50.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.686+0000 7fa287c5f640 1 -- 172.21.15.62:0/1789296073 wait complete. 2024-09-11T23:50:50.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.872+0000 7f4f056e0640 1 -- 172.21.15.62:0/4118105794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00102ef0 msgr2=0x7f4f0010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.872+0000 7f4f056e0640 1 --2- 172.21.15.62:0/4118105794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00102ef0 0x7f4f0010da40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009ca0 tx=0x7f4ef802a950 comp rx=0 tx=0).stop 2024-09-11T23:50:50.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.872+0000 7f4f056e0640 1 -- 172.21.15.62:0/4118105794 shutdown_connections 2024-09-11T23:50:50.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.872+0000 7f4f056e0640 1 --2- 172.21.15.62:0/4118105794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00102ef0 0x7f4f0010da40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.872+0000 7f4f056e0640 1 --2- 172.21.15.62:0/4118105794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00102570 0x7f4f001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.872+0000 7f4f056e0640 1 --2- 172.21.15.62:0/4118105794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00100e20 0x7f4f00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.872+0000 7f4f056e0640 1 -- 172.21.15.62:0/4118105794 >> 172.21.15.62:0/4118105794 conn(0x7f4f000fc7d0 msgr2=0x7f4f000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.872+0000 7f4f056e0640 1 -- 172.21.15.62:0/4118105794 shutdown_connections 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.872+0000 7f4f056e0640 1 -- 172.21.15.62:0/4118105794 wait complete. 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.873+0000 7f4f056e0640 1 Processor -- start 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.873+0000 7f4f056e0640 1 -- start start 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.873+0000 7f4f056e0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f00198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.873+0000 7f4f056e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00102570 0x7f4f001989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.873+0000 7f4f056e0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00102ef0 0x7f4f0019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.873+0000 7f4f056e0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00110f80 con 0x7f4f00100e20 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.873+0000 7f4f056e0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00110e00 con 0x7f4f00102570 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.873+0000 7f4f056e0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00111100 con 0x7f4f00102ef0 2024-09-11T23:50:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.873+0000 7f4efeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f00198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4efeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f00198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42698/0 (socket says 172.21.15.62:42698) 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4efeffd640 1 -- 172.21.15.62:0/470122787 learned_addr learned my addr 172.21.15.62:0/470122787 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4efe7fc640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00102570 0x7f4f001989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4eff7fe640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00102ef0 0x7f4f0019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4efeffd640 1 -- 172.21.15.62:0/470122787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00102ef0 msgr2=0x7f4f0019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4efeffd640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00102ef0 0x7f4f0019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4efeffd640 1 -- 172.21.15.62:0/470122787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00102570 msgr2=0x7f4f001989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4efeffd640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00102570 0x7f4f001989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4efeffd640 1 -- 172.21.15.62:0/470122787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0019d410 con 0x7f4f00100e20 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4efeffd640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f00198460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000d950 tx=0x7f4ef000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4ee7fff640 1 -- 172.21.15.62:0/470122787 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0017030 con 0x7f4f00100e20 2024-09-11T23:50:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0019d700 con 0x7f4f00100e20 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4ee7fff640 1 -- 172.21.15.62:0/470122787 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef0004020 con 0x7f4f00100e20 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4ee7fff640 1 -- 172.21.15.62:0/470122787 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00047a0 con 0x7f4f00100e20 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.874+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f001a4fe0 con 0x7f4f00100e20 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.875+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f00062980 con 0x7f4f00100e20 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.877+0000 7f4ee7fff640 1 -- 172.21.15.62:0/470122787 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4ef00041d0 con 0x7f4f00100e20 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.877+0000 7f4ee7fff640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4ee80a2c10 0x7f4ee80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.877+0000 7f4efe7fc640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4ee80a2c10 0x7f4ee80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.877+0000 7f4ee7fff640 1 -- 172.21.15.62:0/470122787 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f4ef00cf270 con 0x7f4f00100e20 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.878+0000 7f4ee7fff640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] conn(0x7f4ee80ac460 0x7f4ee80ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.878+0000 7f4ee7fff640 1 -- 172.21.15.62:0/470122787 --> [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ef0021e70 con 0x7f4ee80ac460 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.878+0000 7f4ee7fff640 1 -- 172.21.15.62:0/470122787 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f4ef00cf620 con 0x7f4f00100e20 2024-09-11T23:50:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.878+0000 7f4eff7fe640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] conn(0x7f4ee80ac460 0x7f4ee80ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:50.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.878+0000 7f4efe7fc640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4ee80a2c10 0x7f4ee80a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f000ff0e0 tx=0x7f4ef4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:50.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.878+0000 7f4eff7fe640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] conn(0x7f4ee80ac460 0x7f4ee80ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:50.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.879+0000 7f4ee7fff640 1 -- 172.21.15.62:0/470122787 <== osd.9 v2:172.21.15.62:6826/343213579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ef0021e70 con 0x7f4ee80ac460 2024-09-11T23:50:50.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.893+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 --> [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f00062d80 con 0x7f4ee80ac460 2024-09-11T23:50:50.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.893+0000 7f4ee7fff640 1 -- 172.21.15.62:0/470122787 <== osd.9 v2:172.21.15.62:6826/343213579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4f00062d80 con 0x7f4ee80ac460 2024-09-11T23:50:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.893+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] conn(0x7f4ee80ac460 msgr2=0x7f4ee80ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.893+0000 7f4f056e0640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] conn(0x7f4ee80ac460 0x7f4ee80ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.894+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4ee80a2c10 msgr2=0x7f4ee80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.894+0000 7f4f056e0640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4ee80a2c10 0x7f4ee80a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f000ff0e0 tx=0x7f4ef4005f90 comp rx=0 tx=0).stop 2024-09-11T23:50:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.894+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 msgr2=0x7f4f00198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.894+0000 7f4f056e0640 1 --2- 172.21.15.62:0/470122787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f00198460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000d950 tx=0x7f4ef000de10 comp rx=0 tx=0).stop 2024-09-11T23:50:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.894+0000 7f4eff7fe640 1 -- 172.21.15.62:0/470122787 reap_dead start 2024-09-11T23:50:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.894+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 shutdown_connections 2024-09-11T23:50:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.894+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 >> 172.21.15.62:0/470122787 conn(0x7f4f000fc7d0 msgr2=0x7f4f0006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.894+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 shutdown_connections 2024-09-11T23:50:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:50.894+0000 7f4f056e0640 1 -- 172.21.15.62:0/470122787 wait complete. 2024-09-11T23:50:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.967+0000 7f25503d1640 1 -- 172.21.15.62:0/2500907781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2548102d90 msgr2=0x7f2548103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.967+0000 7f25503d1640 1 --2- 172.21.15.62:0/2500907781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2548102d90 0x7f2548103170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2544009930 tx=0x7f254402a420 comp rx=0 tx=0).stop 2024-09-11T23:50:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 -- 172.21.15.62:0/2500907781 shutdown_connections 2024-09-11T23:50:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 --2- 172.21.15.62:0/2500907781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548108290 0x7f254810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 --2- 172.21.15.62:0/2500907781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2548103740 0x7f2548107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 --2- 172.21.15.62:0/2500907781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2548102d90 0x7f2548103170 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 -- 172.21.15.62:0/2500907781 >> 172.21.15.62:0/2500907781 conn(0x7f25480fc7d0 msgr2=0x7f25480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 -- 172.21.15.62:0/2500907781 shutdown_connections 2024-09-11T23:50:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 -- 172.21.15.62:0/2500907781 wait complete. 2024-09-11T23:50:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 Processor -- start 2024-09-11T23:50:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 -- start start 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2548102d90 0x7f25481985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.968+0000 7f25503d1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2548103740 0x7f2548198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f25503d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548108290 0x7f254819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f25503d1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2548112090 con 0x7f2548103740 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f25503d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2548111f10 con 0x7f2548108290 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f25503d1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2548112210 con 0x7f2548102d90 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254d96c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2548103740 0x7f2548198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254d96c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2548103740 0x7f2548198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42664/0 (socket says 172.21.15.62:42664) 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254d96c640 1 -- 172.21.15.62:0/806926935 learned_addr learned my addr 172.21.15.62:0/806926935 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254e16d640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2548102d90 0x7f25481985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254e96e640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548108290 0x7f254819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254d96c640 1 -- 172.21.15.62:0/806926935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2548102d90 msgr2=0x7f25481985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254d96c640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2548102d90 0x7f25481985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254d96c640 1 -- 172.21.15.62:0/806926935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548108290 msgr2=0x7f254819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254d96c640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548108290 0x7f254819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254d96c640 1 -- 172.21.15.62:0/806926935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254819d550 con 0x7f2548103740 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f254d96c640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2548103740 0x7f2548198ae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f253800e9f0 tx=0x7f253800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f253f7fe640 1 -- 172.21.15.62:0/806926935 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253800ccc0 con 0x7f2548103740 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254819d840 con 0x7f2548103740 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f253f7fe640 1 -- 172.21.15.62:0/806926935 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2538005500 con 0x7f2548103740 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f253f7fe640 1 -- 172.21.15.62:0/806926935 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538010690 con 0x7f2548103740 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.969+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25481a5120 con 0x7f2548103740 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.970+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2548062980 con 0x7f2548103740 2024-09-11T23:50:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.972+0000 7f253f7fe640 1 -- 172.21.15.62:0/806926935 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f25380046c0 con 0x7f2548103740 2024-09-11T23:50:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.973+0000 7f253f7fe640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f25340a2c10 0x7f25340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.973+0000 7f253f7fe640 1 -- 172.21.15.62:0/806926935 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f25380cf880 con 0x7f2548103740 2024-09-11T23:50:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.973+0000 7f254e16d640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f25340a2c10 0x7f25340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.973+0000 7f253f7fe640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] conn(0x7f25340ac460 0x7f25340ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.973+0000 7f253f7fe640 1 -- 172.21.15.62:0/806926935 --> [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25340aef30 con 0x7f25340ac460 2024-09-11T23:50:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.973+0000 7f253f7fe640 1 -- 172.21.15.62:0/806926935 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f25380cfd60 con 0x7f2548103740 2024-09-11T23:50:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.974+0000 7f254e96e640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] conn(0x7f25340ac460 0x7f25340ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.974+0000 7f254e16d640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f25340a2c10 0x7f25340a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2544009900 tx=0x7f25440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.974+0000 7f254e96e640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] conn(0x7f25340ac460 0x7f25340ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.975+0000 7f253f7fe640 1 -- 172.21.15.62:0/806926935 <== osd.10 v2:172.21.15.83:6824/4134153557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25340aef30 con 0x7f25340ac460 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.989+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 --> [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2548062e00 con 0x7f25340ac460 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.989+0000 7f253f7fe640 1 -- 172.21.15.62:0/806926935 <== osd.10 v2:172.21.15.83:6824/4134153557 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2548062e00 con 0x7f25340ac460 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.989+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 >> [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] conn(0x7f25340ac460 msgr2=0x7f25340ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.989+0000 7f25503d1640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] conn(0x7f25340ac460 0x7f25340ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.990+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f25340a2c10 msgr2=0x7f25340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.990+0000 7f25503d1640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f25340a2c10 0x7f25340a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2544009900 tx=0x7f25440058d0 comp rx=0 tx=0).stop 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.990+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2548103740 msgr2=0x7f2548198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.990+0000 7f25503d1640 1 --2- 172.21.15.62:0/806926935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2548103740 0x7f2548198ae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f253800e9f0 tx=0x7f253800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.990+0000 7f254e96e640 1 -- 172.21.15.62:0/806926935 reap_dead start 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.990+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 shutdown_connections 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.990+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 >> 172.21.15.62:0/806926935 conn(0x7f25480fc7d0 msgr2=0x7f2548105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.990+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 shutdown_connections 2024-09-11T23:50:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:49.990+0000 7f25503d1640 1 -- 172.21.15.62:0/806926935 wait complete. 2024-09-11T23:50:52.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:51 smithi062 ceph-mon[30958]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-11T23:50:52.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 -- 172.21.15.62:0/1768130682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c0102d30 msgr2=0x7f79c0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 --2- 172.21.15.62:0/1768130682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c0102d30 0x7f79c0103110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009930 tx=0x7f79b802a440 comp rx=0 tx=0).stop 2024-09-11T23:50:52.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 -- 172.21.15.62:0/1768130682 shutdown_connections 2024-09-11T23:50:52.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 --2- 172.21.15.62:0/1768130682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c01082a0 0x7f79c010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 --2- 172.21.15.62:0/1768130682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c01036e0 0x7f79c0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 --2- 172.21.15.62:0/1768130682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c0102d30 0x7f79c0103110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 -- 172.21.15.62:0/1768130682 >> 172.21.15.62:0/1768130682 conn(0x7f79c00fc7d0 msgr2=0x7f79c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:52.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 -- 172.21.15.62:0/1768130682 shutdown_connections 2024-09-11T23:50:52.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 -- 172.21.15.62:0/1768130682 wait complete. 2024-09-11T23:50:52.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 Processor -- start 2024-09-11T23:50:52.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.223+0000 7f79c83f3640 1 -- start start 2024-09-11T23:50:52.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c83f3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c0102d30 0x7f79c01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c83f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c01036e0 0x7f79c0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c83f3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c01082a0 0x7f79c019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c83f3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f79c01120e0 con 0x7f79c01036e0 2024-09-11T23:50:52.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c83f3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79c0111f60 con 0x7f79c01082a0 2024-09-11T23:50:52.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c83f3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79c0112260 con 0x7f79c0102d30 2024-09-11T23:50:52.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c598e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c01036e0 0x7f79c0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c598e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c01036e0 0x7f79c0198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38918/0 (socket says 172.21.15.62:38918) 2024-09-11T23:50:52.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c598e640 1 -- 172.21.15.62:0/2885950219 learned_addr learned my addr 172.21.15.62:0/2885950219 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:52.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c618f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c0102d30 0x7f79c01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c6990640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c01082a0 0x7f79c019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c598e640 1 -- 172.21.15.62:0/2885950219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c0102d30 msgr2=0x7f79c01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c598e640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c0102d30 0x7f79c01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c598e640 1 -- 172.21.15.62:0/2885950219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c01082a0 msgr2=0x7f79c019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c598e640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c01082a0 0x7f79c019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c598e640 1 -- 172.21.15.62:0/2885950219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c019d550 con 0x7f79c01036e0 2024-09-11T23:50:52.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79c598e640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c01036e0 0x7f79c0198ae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79b4006fd0 tx=0x7f79b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:52.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.224+0000 7f79bf7fe640 1 -- 172.21.15.62:0/2885950219 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4011830 con 0x7f79c01036e0 2024-09-11T23:50:52.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.225+0000 7f79bf7fe640 1 -- 172.21.15.62:0/2885950219 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b4016070 con 0x7f79c01036e0 2024-09-11T23:50:52.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.225+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c019d840 con 0x7f79c01036e0 2024-09-11T23:50:52.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.225+0000 7f79bf7fe640 1 -- 172.21.15.62:0/2885950219 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b400f2e0 con 0x7f79c01036e0 2024-09-11T23:50:52.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.225+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c01a5120 con 0x7f79c01036e0 2024-09-11T23:50:52.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.226+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f79c0062980 con 0x7f79c01036e0 2024-09-11T23:50:52.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.227+0000 7f79bf7fe640 1 -- 172.21.15.62:0/2885950219 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f79b401b460 con 0x7f79c01036e0 2024-09-11T23:50:52.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.228+0000 7f79bf7fe640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f79a80a2c90 0x7f79a80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.228+0000 7f79c618f640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f79a80a2c90 0x7f79a80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.228+0000 7f79bf7fe640 1 -- 172.21.15.62:0/2885950219 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f79b40cef30 con 0x7f79c01036e0 2024-09-11T23:50:52.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.228+0000 7f79bf7fe640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] conn(0x7f79a80ac4e0 0x7f79a80ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.228+0000 7f79bf7fe640 1 -- 172.21.15.62:0/2885950219 --> [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79b401ee50 con 0x7f79a80ac4e0 2024-09-11T23:50:52.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.228+0000 7f79bf7fe640 1 -- 172.21.15.62:0/2885950219 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f79b40cf410 con 0x7f79c01036e0 2024-09-11T23:50:52.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.228+0000 7f79c6990640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] conn(0x7f79a80ac4e0 0x7f79a80ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.228+0000 7f79c618f640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f79a80a2c90 0x7f79a80a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009900 tx=0x7f79b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:52.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.228+0000 7f79c6990640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] conn(0x7f79a80ac4e0 0x7f79a80ae900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:52.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.229+0000 7f79bf7fe640 1 -- 172.21.15.62:0/2885950219 <== osd.0 v2:172.21.15.62:6802/3944758612 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79b401ee50 con 0x7f79a80ac4e0 2024-09-11T23:50:52.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 --> [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f79c0062db0 con 0x7f79a80ac4e0 2024-09-11T23:50:52.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79bf7fe640 1 -- 172.21.15.62:0/2885950219 <== osd.0 v2:172.21.15.62:6802/3944758612 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f79c0062db0 con 0x7f79a80ac4e0 2024-09-11T23:50:52.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] conn(0x7f79a80ac4e0 msgr2=0x7f79a80ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79c83f3640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] conn(0x7f79a80ac4e0 0x7f79a80ae900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f79a80a2c90 msgr2=0x7f79a80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79c83f3640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f79a80a2c90 0x7f79a80a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009900 tx=0x7f79b80058d0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c01036e0 msgr2=0x7f79c0198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79c83f3640 1 --2- 172.21.15.62:0/2885950219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c01036e0 0x7f79c0198ae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79b4006fd0 tx=0x7f79b400b8e0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79c6990640 1 -- 172.21.15.62:0/2885950219 reap_dead start 2024-09-11T23:50:52.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 shutdown_connections 2024-09-11T23:50:52.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.243+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 >> 172.21.15.62:0/2885950219 conn(0x7f79c00fc7d0 msgr2=0x7f79c0105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:52.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.244+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 shutdown_connections 2024-09-11T23:50:52.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.244+0000 7f79c83f3640 1 -- 172.21.15.62:0/2885950219 wait complete. 2024-09-11T23:50:52.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:51 smithi120 ceph-mon[35182]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-11T23:50:52.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 -- 172.21.15.62:0/4142295938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b0101420 msgr2=0x7f65b010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 --2- 172.21.15.62:0/4142295938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b0101420 0x7f65b010d950 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009930 tx=0x7f65a802a420 comp rx=0 tx=0).stop 2024-09-11T23:50:52.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 -- 172.21.15.62:0/4142295938 shutdown_connections 2024-09-11T23:50:52.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 --2- 172.21.15.62:0/4142295938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b0101420 0x7f65b010d950 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 --2- 172.21.15.62:0/4142295938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b0100a80 0x7f65b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 --2- 172.21.15.62:0/4142295938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65b0106a80 0x7f65b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 -- 172.21.15.62:0/4142295938 >> 172.21.15.62:0/4142295938 conn(0x7f65b00fc7a0 msgr2=0x7f65b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 -- 172.21.15.62:0/4142295938 shutdown_connections 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 -- 172.21.15.62:0/4142295938 wait complete. 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 Processor -- start 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.407+0000 7f65b7495640 1 -- start start 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b7495640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b0100a80 0x7f65b019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b7495640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b0101420 0x7f65b019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b7495640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65b0106a80 0x7f65b019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b7495640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65b0110e80 con 0x7f65b0100a80 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b7495640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65b0110d00 con 0x7f65b0101420 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b7495640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65b0111000 con 0x7f65b0106a80 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b5231640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b0100a80 0x7f65b019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b5231640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b0100a80 0x7f65b019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38936/0 (socket says 172.21.15.62:38936) 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b5231640 1 -- 172.21.15.62:0/3801007901 learned_addr learned my addr 172.21.15.62:0/3801007901 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b4a30640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b0101420 0x7f65b019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b5a32640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65b0106a80 0x7f65b019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b5231640 1 -- 172.21.15.62:0/3801007901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65b0106a80 msgr2=0x7f65b019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b5231640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65b0106a80 0x7f65b019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b5231640 1 -- 172.21.15.62:0/3801007901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b0101420 msgr2=0x7f65b019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b5231640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b0101420 0x7f65b019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b5231640 1 -- 172.21.15.62:0/3801007901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b019f650 con 0x7f65b0100a80 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b5231640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b0100a80 0x7f65b019a6a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f65a400d770 tx=0x7f65a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65a27fc640 1 -- 172.21.15.62:0/3801007901 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4017070 con 0x7f65b0100a80 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65a27fc640 1 -- 172.21.15.62:0/3801007901 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a4005500 con 0x7f65b0100a80 2024-09-11T23:50:52.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b019f940 con 0x7f65b0100a80 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.408+0000 7f65a27fc640 1 -- 172.21.15.62:0/3801007901 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4004e00 con 0x7f65b0100a80 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.409+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b01a7220 con 0x7f65b0100a80 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.409+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65b010d100 con 0x7f65b0100a80 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.411+0000 7f65a27fc640 1 -- 172.21.15.62:0/3801007901 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f65a4004fa0 con 0x7f65b0100a80 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.411+0000 7f65a27fc640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f659c0a2c10 0x7f659c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.411+0000 7f65b4a30640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f659c0a2c10 0x7f659c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.411+0000 7f65a27fc640 1 -- 172.21.15.62:0/3801007901 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f65a40cee20 con 0x7f65b0100a80 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.412+0000 7f65a27fc640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] conn(0x7f659c0ac460 0x7f659c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.412+0000 7f65a27fc640 1 -- 172.21.15.62:0/3801007901 --> [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65a4001d30 con 0x7f659c0ac460 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.412+0000 7f65a27fc640 1 -- 172.21.15.62:0/3801007901 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f65a40cf300 con 0x7f65b0100a80 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.412+0000 7f65b4a30640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f659c0a2c10 0x7f659c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f65b010c610 tx=0x7f65ac009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.412+0000 7f65b5a32640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] conn(0x7f659c0ac460 0x7f659c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.412+0000 7f65b5a32640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] conn(0x7f659c0ac460 0x7f659c0ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:52.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.413+0000 7f65a27fc640 1 -- 172.21.15.62:0/3801007901 <== osd.4 v2:172.21.15.83:6808/3966827611 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65a4001d30 con 0x7f659c0ac460 2024-09-11T23:50:52.420 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:51 smithi083 ceph-mon[34805]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-11T23:50:52.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.427+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 --> [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65b01021c0 con 0x7f659c0ac460 2024-09-11T23:50:52.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.427+0000 7f65a27fc640 1 -- 172.21.15.62:0/3801007901 <== osd.4 v2:172.21.15.83:6808/3966827611 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f65b01021c0 con 0x7f659c0ac460 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.427+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 >> [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] conn(0x7f659c0ac460 msgr2=0x7f659c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.427+0000 7f65b7495640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] conn(0x7f659c0ac460 0x7f659c0ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.427+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f659c0a2c10 msgr2=0x7f659c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.427+0000 7f65b7495640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f659c0a2c10 0x7f659c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f65b010c610 tx=0x7f65ac009760 comp rx=0 tx=0).stop 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.427+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b0100a80 msgr2=0x7f65b019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.427+0000 7f65b7495640 1 --2- 172.21.15.62:0/3801007901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b0100a80 0x7f65b019a6a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f65a400d770 tx=0x7f65a400dc30 comp rx=0 tx=0).stop 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.427+0000 7f65b5a32640 1 -- 172.21.15.62:0/3801007901 reap_dead start 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.427+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 shutdown_connections 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.428+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 >> 172.21.15.62:0/3801007901 conn(0x7f65b00fc7a0 msgr2=0x7f65b010c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.428+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 shutdown_connections 2024-09-11T23:50:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.428+0000 7f65b7495640 1 -- 172.21.15.62:0/3801007901 wait complete. 2024-09-11T23:50:52.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.954+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/4262262510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb84100a80 msgr2=0x7fbb84100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.954+0000 7fbb8a9e5640 1 --2- 172.21.15.62:0/4262262510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb84100a80 0x7fbb84100ee0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009930 tx=0x7fbb7802a420 comp rx=0 tx=0).stop 2024-09-11T23:50:52.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.954+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/4262262510 shutdown_connections 2024-09-11T23:50:52.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.954+0000 7fbb8a9e5640 1 --2- 172.21.15.62:0/4262262510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb84101420 0x7fbb8410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.954+0000 7fbb8a9e5640 1 --2- 172.21.15.62:0/4262262510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb84100a80 0x7fbb84100ee0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.954+0000 7fbb8a9e5640 1 --2- 172.21.15.62:0/4262262510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84106a80 0x7fbb84106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.954+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/4262262510 >> 172.21.15.62:0/4262262510 conn(0x7fbb840fc7a0 msgr2=0x7fbb840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:52.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.954+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/4262262510 shutdown_connections 2024-09-11T23:50:52.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.955+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/4262262510 wait complete. 2024-09-11T23:50:52.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.955+0000 7fbb8a9e5640 1 Processor -- start 2024-09-11T23:50:52.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.955+0000 7fbb8a9e5640 1 -- start start 2024-09-11T23:50:52.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.955+0000 7fbb8a9e5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb84100a80 0x7fbb84198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb8a9e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb84101420 0x7fbb841989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb8a9e5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84106a80 0x7fbb8419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb8a9e5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84110ea0 con 0x7fbb84100a80 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb8a9e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84110d20 con 0x7fbb84101420 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb8a9e5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84111020 con 0x7fbb84106a80 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb83fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb84100a80 0x7fbb84198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb83fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb84100a80 0x7fbb84198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38950/0 (socket says 172.21.15.62:38950) 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb83fff640 1 -- 172.21.15.62:0/2273489145 learned_addr learned my addr 172.21.15.62:0/2273489145 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb837fe640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb84101420 0x7fbb841989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb88f82640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84106a80 0x7fbb8419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb83fff640 1 -- 172.21.15.62:0/2273489145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84106a80 msgr2=0x7fbb8419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb83fff640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84106a80 0x7fbb8419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb83fff640 1 -- 172.21.15.62:0/2273489145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb84101420 msgr2=0x7fbb841989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb83fff640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb84101420 0x7fbb841989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb83fff640 1 -- 172.21.15.62:0/2273489145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8419d440 con 0x7fbb84100a80 2024-09-11T23:50:52.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb88f82640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84106a80 0x7fbb8419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:52.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb83fff640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb84100a80 0x7fbb84198490 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400b4a0 tx=0x7fbb7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:52.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.956+0000 7fbb817fa640 1 -- 172.21.15.62:0/2273489145 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74014070 con 0x7fbb84100a80 2024-09-11T23:50:52.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.957+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8419d730 con 0x7fbb84100a80 2024-09-11T23:50:52.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.957+0000 7fbb817fa640 1 -- 172.21.15.62:0/2273489145 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb74010af0 con 0x7fbb84100a80 2024-09-11T23:50:52.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.957+0000 7fbb817fa640 1 -- 172.21.15.62:0/2273489145 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74017d10 con 0x7fbb84100a80 2024-09-11T23:50:52.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.957+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb840fcc80 con 0x7fbb84100a80 2024-09-11T23:50:52.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.957+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb84062980 con 0x7fbb84100a80 2024-09-11T23:50:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.959+0000 7fbb817fa640 1 -- 172.21.15.62:0/2273489145 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbb74017070 con 0x7fbb84100a80 2024-09-11T23:50:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.959+0000 7fbb817fa640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb6c0a2c10 0x7fbb6c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.960+0000 7fbb837fe640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb6c0a2c10 0x7fbb6c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.960+0000 7fbb817fa640 1 -- 172.21.15.62:0/2273489145 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fbb740ce680 con 0x7fbb84100a80 2024-09-11T23:50:52.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.960+0000 7fbb817fa640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] conn(0x7fbb6c0ac460 0x7fbb6c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:52.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.960+0000 7fbb817fa640 1 -- 172.21.15.62:0/2273489145 --> [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb7401ae50 con 0x7fbb6c0ac460 2024-09-11T23:50:52.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.960+0000 7fbb817fa640 1 -- 172.21.15.62:0/2273489145 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fbb740972e0 con 0x7fbb84100a80 2024-09-11T23:50:52.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.960+0000 7fbb837fe640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb6c0a2c10 0x7fbb6c0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009900 tx=0x7fbb780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:52.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.960+0000 7fbb88f82640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] conn(0x7fbb6c0ac460 0x7fbb6c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:52.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.960+0000 7fbb88f82640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] conn(0x7fbb6c0ac460 0x7fbb6c0ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:52.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.961+0000 7fbb817fa640 1 -- 172.21.15.62:0/2273489145 <== osd.1 v2:172.21.15.83:6800/2108490342 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb7401ae50 con 0x7fbb6c0ac460 2024-09-11T23:50:52.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.975+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 --> [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb84062e00 con 0x7fbb6c0ac460 2024-09-11T23:50:52.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.975+0000 7fbb817fa640 1 -- 172.21.15.62:0/2273489145 <== osd.1 v2:172.21.15.83:6800/2108490342 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbb84062e00 con 0x7fbb6c0ac460 2024-09-11T23:50:52.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 >> [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] conn(0x7fbb6c0ac460 msgr2=0x7fbb6c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb8a9e5640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] conn(0x7fbb6c0ac460 0x7fbb6c0ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb6c0a2c10 msgr2=0x7fbb6c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb8a9e5640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb6c0a2c10 0x7fbb6c0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009900 tx=0x7fbb780058d0 comp rx=0 tx=0).stop 2024-09-11T23:50:52.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb84100a80 msgr2=0x7fbb84198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:52.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb8a9e5640 1 --2- 172.21.15.62:0/2273489145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb84100a80 0x7fbb84198490 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400b4a0 tx=0x7fbb7400b960 comp rx=0 tx=0).stop 2024-09-11T23:50:52.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb88f82640 1 -- 172.21.15.62:0/2273489145 reap_dead start 2024-09-11T23:50:52.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 shutdown_connections 2024-09-11T23:50:52.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 >> 172.21.15.62:0/2273489145 conn(0x7fbb840fc7a0 msgr2=0x7fbb8406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:52.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 shutdown_connections 2024-09-11T23:50:52.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:52.976+0000 7fbb8a9e5640 1 -- 172.21.15.62:0/2273489145 wait complete. 2024-09-11T23:50:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.113+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/1950898501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4dc103660 msgr2=0x7fb4dc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.113+0000 7fb4e2fa3640 1 --2- 172.21.15.62:0/1950898501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4dc103660 0x7fb4dc10bb50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009900 tx=0x7fb4c802a480 comp rx=0 tx=0).stop 2024-09-11T23:50:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.113+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/1950898501 shutdown_connections 2024-09-11T23:50:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.113+0000 7fb4e2fa3640 1 --2- 172.21.15.62:0/1950898501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4dc10c170 0x7fb4dc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.113+0000 7fb4e2fa3640 1 --2- 172.21.15.62:0/1950898501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4dc103660 0x7fb4dc10bb50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.113+0000 7fb4e2fa3640 1 --2- 172.21.15.62:0/1950898501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4dc102d40 0x7fb4dc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.113+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/1950898501 >> 172.21.15.62:0/1950898501 conn(0x7fb4dc0fc7a0 msgr2=0x7fb4dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.113+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/1950898501 shutdown_connections 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.113+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/1950898501 wait complete. 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e2fa3640 1 Processor -- start 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e2fa3640 1 -- start start 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e2fa3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4dc102d40 0x7fb4dc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e2fa3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4dc103660 0x7fb4dc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e2fa3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4dc10c170 0x7fb4dc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e2fa3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc111f70 con 0x7fb4dc10c170 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e2fa3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc111df0 con 0x7fb4dc102d40 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e2fa3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1120f0 con 0x7fb4dc103660 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e1540640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4dc10c170 0x7fb4dc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e1540640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4dc10c170 0x7fb4dc19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38974/0 (socket says 172.21.15.62:38974) 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e1540640 1 -- 172.21.15.62:0/2327512614 learned_addr learned my addr 172.21.15.62:0/2327512614 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4dbfff640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4dc103660 0x7fb4dc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e0d3f640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4dc102d40 0x7fb4dc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e1540640 1 -- 172.21.15.62:0/2327512614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4dc103660 msgr2=0x7fb4dc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e1540640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4dc103660 0x7fb4dc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e1540640 1 -- 172.21.15.62:0/2327512614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4dc102d40 msgr2=0x7fb4dc1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e1540640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4dc102d40 0x7fb4dc1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e1540640 1 -- 172.21.15.62:0/2327512614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc19d3a0 con 0x7fb4dc10c170 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.114+0000 7fb4e0d3f640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4dc102d40 0x7fb4dc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.115+0000 7fb4e1540640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4dc10c170 0x7fb4dc19ccc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400cad0 tx=0x7fb4d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.115+0000 7fb4d9ffb640 1 -- 172.21.15.62:0/2327512614 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400ae20 con 0x7fb4dc10c170 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.115+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4dc19d690 con 0x7fb4dc10c170 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.115+0000 7fb4d9ffb640 1 -- 172.21.15.62:0/2327512614 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d4005020 con 0x7fb4dc10c170 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.115+0000 7fb4d9ffb640 1 -- 172.21.15.62:0/2327512614 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400f6b0 con 0x7fb4dc10c170 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.115+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4dc1a4f70 con 0x7fb4dc10c170 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.116+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4dc062980 con 0x7fb4dc10c170 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.117+0000 7fb4d9ffb640 1 -- 172.21.15.62:0/2327512614 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb4d40051d0 con 0x7fb4dc10c170 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.118+0000 7fb4d9ffb640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb4cc0a2c10 0x7fb4cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.118+0000 7fb4e0d3f640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb4cc0a2c10 0x7fb4cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.118+0000 7fb4d9ffb640 1 -- 172.21.15.62:0/2327512614 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb4d40cf480 con 0x7fb4dc10c170 2024-09-11T23:50:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.118+0000 7fb4d9ffb640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] conn(0x7fb4cc0ac460 0x7fb4cc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.118+0000 7fb4d9ffb640 1 -- 172.21.15.62:0/2327512614 --> [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4d4012ec0 con 0x7fb4cc0ac460 2024-09-11T23:50:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.118+0000 7fb4d9ffb640 1 -- 172.21.15.62:0/2327512614 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fb4d40cf960 con 0x7fb4dc10c170 2024-09-11T23:50:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.118+0000 7fb4e0d3f640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb4cc0a2c10 0x7fb4cc0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000baa0 tx=0x7fb4d0005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.118+0000 7fb4dbfff640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] conn(0x7fb4cc0ac460 0x7fb4cc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.119+0000 7fb4dbfff640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] conn(0x7fb4cc0ac460 0x7fb4cc0ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.120+0000 7fb4d9ffb640 1 -- 172.21.15.62:0/2327512614 <== osd.8 v2:172.21.15.120:6816/1046771944 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4d4012ec0 con 0x7fb4cc0ac460 2024-09-11T23:50:53.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.134+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 --> [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4dc062e00 con 0x7fb4cc0ac460 2024-09-11T23:50:53.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.134+0000 7fb4d9ffb640 1 -- 172.21.15.62:0/2327512614 <== osd.8 v2:172.21.15.120:6816/1046771944 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb4dc062e00 con 0x7fb4cc0ac460 2024-09-11T23:50:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 >> [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] conn(0x7fb4cc0ac460 msgr2=0x7fb4cc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e2fa3640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] conn(0x7fb4cc0ac460 0x7fb4cc0ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb4cc0a2c10 msgr2=0x7fb4cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e2fa3640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb4cc0a2c10 0x7fb4cc0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000baa0 tx=0x7fb4d0005d90 comp rx=0 tx=0).stop 2024-09-11T23:50:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4dc10c170 msgr2=0x7fb4dc19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e2fa3640 1 --2- 172.21.15.62:0/2327512614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4dc10c170 0x7fb4dc19ccc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400cad0 tx=0x7fb4d400cf90 comp rx=0 tx=0).stop 2024-09-11T23:50:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e1540640 1 -- 172.21.15.62:0/2327512614 reap_dead start 2024-09-11T23:50:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 shutdown_connections 2024-09-11T23:50:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 >> 172.21.15.62:0/2327512614 conn(0x7fb4dc0fc7a0 msgr2=0x7fb4dc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 shutdown_connections 2024-09-11T23:50:53.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:53.135+0000 7fb4e2fa3640 1 -- 172.21.15.62:0/2327512614 wait complete. 2024-09-11T23:50:54.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:53 smithi062 ceph-mon[30958]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-11T23:50:54.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:53 smithi120 ceph-mon[35182]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-11T23:50:54.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:53 smithi083 ceph-mon[34805]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-11T23:50:54.446 INFO:teuthology.orchestra.run.smithi062.stdout:107374182462 2024-09-11T23:50:54.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T23:50:55.177 INFO:teuthology.orchestra.run.smithi062.stdout:55834574921 2024-09-11T23:50:55.178 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T23:50:55.479 INFO:teuthology.orchestra.run.smithi062.stdout:34359738446 2024-09-11T23:50:55.480 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T23:50:55.890 INFO:teuthology.orchestra.run.smithi062.stdout:158913790004 2024-09-11T23:50:55.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T23:50:56.037 INFO:teuthology.orchestra.run.smithi062.stdout:292057776153 2024-09-11T23:50:56.038 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T23:50:56.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:55 smithi062 ceph-mon[30958]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-11T23:50:56.219 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:55 smithi083 ceph-mon[34805]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-11T23:50:56.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:55 smithi120 ceph-mon[35182]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-11T23:50:56.461 INFO:teuthology.orchestra.run.smithi062.stdout:322122547220 2024-09-11T23:50:56.461 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T23:50:56.849 INFO:teuthology.orchestra.run.smithi062.stdout:270582939680 2024-09-11T23:50:56.849 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T23:50:56.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.772+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3116630656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1c103660 msgr2=0x7f7b1c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:56.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.772+0000 7f7b22f7b640 1 --2- 172.21.15.62:0/3116630656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1c103660 0x7f7b1c10bb50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009900 tx=0x7f7b0802a480 comp rx=0 tx=0).stop 2024-09-11T23:50:56.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.772+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3116630656 shutdown_connections 2024-09-11T23:50:56.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.772+0000 7f7b22f7b640 1 --2- 172.21.15.62:0/3116630656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b1c10c170 0x7f7b1c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.772+0000 7f7b22f7b640 1 --2- 172.21.15.62:0/3116630656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1c103660 0x7f7b1c10bb50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.772+0000 7f7b22f7b640 1 --2- 172.21.15.62:0/3116630656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c102d40 0x7f7b1c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.772+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3116630656 >> 172.21.15.62:0/3116630656 conn(0x7f7b1c0fc7a0 msgr2=0x7f7b1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.772+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3116630656 shutdown_connections 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.772+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3116630656 wait complete. 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.772+0000 7f7b22f7b640 1 Processor -- start 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b22f7b640 1 -- start start 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b22f7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c102d40 0x7f7b1c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b22f7b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1c103660 0x7f7b1c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b22f7b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b1c10c170 0x7f7b1c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b22f7b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c111f70 con 0x7f7b1c103660 2024-09-11T23:50:56.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b22f7b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c111df0 con 0x7f7b1c10c170 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b22f7b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1120f0 con 0x7f7b1c102d40 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b1bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1c103660 0x7f7b1c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b1bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1c103660 0x7f7b1c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39012/0 (socket says 172.21.15.62:39012) 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b1bfff640 1 -- 172.21.15.62:0/3127092330 learned_addr learned my addr 172.21.15.62:0/3127092330 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b20d17640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c102d40 0x7f7b1c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b21518640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b1c10c170 0x7f7b1c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b1bfff640 1 -- 172.21.15.62:0/3127092330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c102d40 msgr2=0x7f7b1c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b1bfff640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c102d40 0x7f7b1c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b1bfff640 1 -- 172.21.15.62:0/3127092330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b1c10c170 msgr2=0x7f7b1c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b1bfff640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b1c10c170 0x7f7b1c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b1bfff640 1 -- 172.21.15.62:0/3127092330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1c1a6930 con 0x7f7b1c103660 2024-09-11T23:50:56.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b20d17640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c102d40 0x7f7b1c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:50:56.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.773+0000 7f7b1bfff640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1c103660 0x7f7b1c100ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7b08006010 tx=0x7f7b08004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:56.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.774+0000 7f7b19ffb640 1 -- 172.21.15.62:0/3127092330 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0803f030 con 0x7f7b1c103660 2024-09-11T23:50:56.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.774+0000 7f7b19ffb640 1 -- 172.21.15.62:0/3127092330 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b08004440 con 0x7f7b1c103660 2024-09-11T23:50:56.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.774+0000 7f7b19ffb640 1 -- 172.21.15.62:0/3127092330 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08035070 con 0x7f7b1c103660 2024-09-11T23:50:56.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.774+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1c1a6bc0 con 0x7f7b1c103660 2024-09-11T23:50:56.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.774+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b1c1a70f0 con 0x7f7b1c103660 2024-09-11T23:50:56.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.775+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b1c0628c0 con 0x7f7b1c103660 2024-09-11T23:50:56.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.776+0000 7f7b19ffb640 1 -- 172.21.15.62:0/3127092330 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7b08042030 con 0x7f7b1c103660 2024-09-11T23:50:56.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.777+0000 7f7b19ffb640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7b0c0a2c90 0x7f7b0c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:56.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.777+0000 7f7b20d17640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7b0c0a2c90 0x7f7b0c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:56.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.777+0000 7f7b19ffb640 1 -- 172.21.15.62:0/3127092330 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f7b080efaa0 con 0x7f7b1c103660 2024-09-11T23:50:56.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.777+0000 7f7b19ffb640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] conn(0x7f7b0c0ac4e0 0x7f7b0c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:56.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.777+0000 7f7b19ffb640 1 -- 172.21.15.62:0/3127092330 --> [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b0c0aefb0 con 0x7f7b0c0ac4e0 2024-09-11T23:50:56.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.777+0000 7f7b19ffb640 1 -- 172.21.15.62:0/3127092330 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f7b080eff80 con 0x7f7b1c103660 2024-09-11T23:50:56.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.777+0000 7f7b21518640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] conn(0x7f7b0c0ac4e0 0x7f7b0c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:56.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.777+0000 7f7b20d17640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7b0c0a2c90 0x7f7b0c0a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009c20 tx=0x7f7b10005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:56.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.778+0000 7f7b21518640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] conn(0x7f7b0c0ac4e0 0x7f7b0c0ae900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:56.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.778+0000 7f7b19ffb640 1 -- 172.21.15.62:0/3127092330 <== osd.7 v2:172.21.15.83:6816/3138900766 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b0c0aefb0 con 0x7f7b0c0ac4e0 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.792+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 --> [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b1c062cc0 con 0x7f7b0c0ac4e0 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b19ffb640 1 -- 172.21.15.62:0/3127092330 <== osd.7 v2:172.21.15.83:6816/3138900766 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7b1c062cc0 con 0x7f7b0c0ac4e0 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 >> [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] conn(0x7f7b0c0ac4e0 msgr2=0x7f7b0c0ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b22f7b640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] conn(0x7f7b0c0ac4e0 0x7f7b0c0ae900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7b0c0a2c90 msgr2=0x7f7b0c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b22f7b640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7b0c0a2c90 0x7f7b0c0a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009c20 tx=0x7f7b10005f90 comp rx=0 tx=0).stop 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1c103660 msgr2=0x7f7b1c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b22f7b640 1 --2- 172.21.15.62:0/3127092330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b1c103660 0x7f7b1c100ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7b08006010 tx=0x7f7b08004040 comp rx=0 tx=0).stop 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b21518640 1 -- 172.21.15.62:0/3127092330 reap_dead start 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 shutdown_connections 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 >> 172.21.15.62:0/3127092330 conn(0x7f7b1c0fc7a0 msgr2=0x7f7b1c0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 shutdown_connections 2024-09-11T23:50:56.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.793+0000 7f7b22f7b640 1 -- 172.21.15.62:0/3127092330 wait complete. 2024-09-11T23:50:57.535 INFO:teuthology.orchestra.run.smithi062.stdout:210453397547 2024-09-11T23:50:57.536 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T23:50:57.921 INFO:teuthology.orchestra.run.smithi062.stdout:240518168613 2024-09-11T23:50:57.922 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T23:50:58.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:57 smithi062 ceph-mon[30958]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:50:58.363 INFO:teuthology.orchestra.run.smithi062.stdout:137438953530 2024-09-11T23:50:58.364 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T23:50:58.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:57 smithi120 ceph-mon[35182]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:50:58.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:57 smithi083 ceph-mon[34805]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:50:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.452+0000 7fce7b712640 1 -- 172.21.15.62:0/1825925495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce74101420 msgr2=0x7fce7410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.452+0000 7fce7b712640 1 --2- 172.21.15.62:0/1825925495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce74101420 0x7fce7410d950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009cd0 tx=0x7fce6c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:50:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.452+0000 7fce7b712640 1 -- 172.21.15.62:0/1825925495 shutdown_connections 2024-09-11T23:50:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.452+0000 7fce7b712640 1 --2- 172.21.15.62:0/1825925495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce74101420 0x7fce7410d950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.452+0000 7fce7b712640 1 --2- 172.21.15.62:0/1825925495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce74100a80 0x7fce74100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.452+0000 7fce7b712640 1 --2- 172.21.15.62:0/1825925495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74106a80 0x7fce74106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.452+0000 7fce7b712640 1 -- 172.21.15.62:0/1825925495 >> 172.21.15.62:0/1825925495 conn(0x7fce740fc7a0 msgr2=0x7fce740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.452+0000 7fce7b712640 1 -- 172.21.15.62:0/1825925495 shutdown_connections 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.452+0000 7fce7b712640 1 -- 172.21.15.62:0/1825925495 wait complete. 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.453+0000 7fce7b712640 1 Processor -- start 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.453+0000 7fce7b712640 1 -- start start 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.453+0000 7fce7b712640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce74100a80 0x7fce741972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce7b712640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce74101420 0x7fce74197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce7b712640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74106a80 0x7fce7419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce7b712640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce74110ea0 con 0x7fce74100a80 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce7b712640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fce74110d20 con 0x7fce74101420 2024-09-11T23:50:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce7b712640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce74111020 con 0x7fce74106a80 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce794ae640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce74100a80 0x7fce741972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce794ae640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce74100a80 0x7fce741972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39000/0 (socket says 172.21.15.62:39000) 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce794ae640 1 -- 172.21.15.62:0/950025055 learned_addr learned my addr 172.21.15.62:0/950025055 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce78cad640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce74101420 0x7fce74197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce79caf640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74106a80 0x7fce7419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce794ae640 1 -- 172.21.15.62:0/950025055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74106a80 msgr2=0x7fce7419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce794ae640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74106a80 0x7fce7419bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce794ae640 1 -- 172.21.15.62:0/950025055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce74101420 msgr2=0x7fce74197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce794ae640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce74101420 0x7fce74197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce794ae640 1 -- 172.21.15.62:0/950025055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7419c270 con 0x7fce74100a80 2024-09-11T23:50:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce794ae640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce74100a80 0x7fce741972c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce7000b4a0 tx=0x7fce7000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce6a7fc640 1 -- 172.21.15.62:0/950025055 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70015020 con 0x7fce74100a80 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce6a7fc640 1 -- 172.21.15.62:0/950025055 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce70011d10 con 0x7fce74100a80 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce6a7fc640 1 -- 172.21.15.62:0/950025055 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7000f2e0 con 0x7fce74100a80 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7419c560 con 0x7fce74100a80 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.454+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce741a3e40 con 0x7fce74100a80 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.455+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce740fd740 con 0x7fce74100a80 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.457+0000 7fce6a7fc640 1 -- 172.21.15.62:0/950025055 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fce70018020 con 0x7fce74100a80 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.457+0000 7fce6a7fc640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fce600a2c90 0x7fce600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.457+0000 7fce78cad640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fce600a2c90 0x7fce600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.458+0000 7fce6a7fc640 1 -- 172.21.15.62:0/950025055 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fce700ced40 con 0x7fce74100a80 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.458+0000 7fce6a7fc640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] conn(0x7fce600ac4e0 0x7fce600ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.458+0000 7fce6a7fc640 1 -- 172.21.15.62:0/950025055 --> [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce70017e50 con 0x7fce600ac4e0 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.458+0000 7fce6a7fc640 1 -- 172.21.15.62:0/950025055 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fce700cf220 con 0x7fce74100a80 2024-09-11T23:50:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.458+0000 7fce79caf640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] conn(0x7fce600ac4e0 0x7fce600ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.458+0000 7fce78cad640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fce600a2c90 0x7fce600a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fce740fec80 tx=0x7fce6400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.458+0000 7fce79caf640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] conn(0x7fce600ac4e0 0x7fce600ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.459+0000 7fce6a7fc640 1 -- 172.21.15.62:0/950025055 <== osd.6 v2:172.21.15.62:6818/461634356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce70017e50 con 0x7fce600ac4e0 2024-09-11T23:50:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.473+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 --> [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce741021c0 con 0x7fce600ac4e0 2024-09-11T23:50:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.473+0000 7fce6a7fc640 1 -- 172.21.15.62:0/950025055 <== osd.6 v2:172.21.15.62:6818/461634356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fce741021c0 con 0x7fce600ac4e0 2024-09-11T23:50:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] conn(0x7fce600ac4e0 msgr2=0x7fce600ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce7b712640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] conn(0x7fce600ac4e0 0x7fce600ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fce600a2c90 msgr2=0x7fce600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce7b712640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fce600a2c90 0x7fce600a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fce740fec80 tx=0x7fce6400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:50:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce74100a80 msgr2=0x7fce741972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce7b712640 1 --2- 172.21.15.62:0/950025055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce74100a80 0x7fce741972c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce7000b4a0 tx=0x7fce7000b8a0 comp rx=0 tx=0).stop 2024-09-11T23:50:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce79caf640 1 -- 172.21.15.62:0/950025055 reap_dead start 2024-09-11T23:50:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 shutdown_connections 2024-09-11T23:50:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 >> 172.21.15.62:0/950025055 conn(0x7fce740fc7a0 msgr2=0x7fce740fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 shutdown_connections 2024-09-11T23:50:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:54.474+0000 7fce7b712640 1 -- 172.21.15.62:0/950025055 wait complete. 2024-09-11T23:50:58.782 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:59.212 INFO:teuthology.orchestra.run.smithi062.stdout:188978561072 2024-09-11T23:50:59.212 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T23:50:59.320 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.860+0000 7fdc4596b640 1 -- 172.21.15.62:0/1215881870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc40102790 msgr2=0x7fdc401067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:59.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.860+0000 7fdc4596b640 1 --2- 172.21.15.62:0/1215881870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc40102790 0x7fdc401067e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009900 tx=0x7fdc3402a410 comp rx=0 tx=0).stop 2024-09-11T23:50:59.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.860+0000 7fdc4596b640 1 -- 172.21.15.62:0/1215881870 shutdown_connections 2024-09-11T23:50:59.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.860+0000 7fdc4596b640 1 --2- 172.21.15.62:0/1215881870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc40106f10 0x7fdc4010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:59.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.860+0000 7fdc4596b640 1 --2- 172.21.15.62:0/1215881870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc40102790 0x7fdc401067e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:59.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.860+0000 7fdc4596b640 1 --2- 172.21.15.62:0/1215881870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc40101de0 0x7fdc401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:59.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.860+0000 7fdc4596b640 1 -- 172.21.15.62:0/1215881870 >> 172.21.15.62:0/1215881870 conn(0x7fdc400fd8c0 msgr2=0x7fdc400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:50:59.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.861+0000 7fdc4596b640 1 -- 172.21.15.62:0/1215881870 shutdown_connections 2024-09-11T23:50:59.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.861+0000 7fdc4596b640 1 -- 172.21.15.62:0/1215881870 wait complete. 2024-09-11T23:50:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.861+0000 7fdc4596b640 1 Processor -- start 2024-09-11T23:50:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.861+0000 7fdc4596b640 1 -- start start 2024-09-11T23:50:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc4596b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc40101de0 0x7fdc40198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc4596b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc40102790 0x7fdc401989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc4596b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc40106f10 0x7fdc4019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc4596b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40111f70 con 0x7fdc40106f10 2024-09-11T23:50:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc4596b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40111df0 con 0x7fdc40101de0 2024-09-11T23:50:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc4596b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401120f0 con 0x7fdc40102790 2024-09-11T23:50:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc40106f10 0x7fdc4019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc40106f10 0x7fdc4019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39026/0 (socket says 172.21.15.62:39026) 2024-09-11T23:50:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3f7fe640 1 -- 172.21.15.62:0/3433434593 learned_addr learned my addr 172.21.15.62:0/3433434593 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:50:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3effd640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc40101de0 0x7fdc40198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3e7fc640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc40102790 0x7fdc401989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3f7fe640 1 -- 172.21.15.62:0/3433434593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc40102790 msgr2=0x7fdc401989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3f7fe640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc40102790 0x7fdc401989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3f7fe640 1 -- 172.21.15.62:0/3433434593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc40101de0 msgr2=0x7fdc40198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:50:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3f7fe640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc40101de0 0x7fdc40198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:50:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3f7fe640 1 -- 172.21.15.62:0/3433434593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4019d420 con 0x7fdc40106f10 2024-09-11T23:50:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.862+0000 7fdc3f7fe640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc40106f10 0x7fdc4019cd40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009fd0 tx=0x7fdc3800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.863+0000 7fdc44969640 1 -- 172.21.15.62:0/3433434593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38017070 con 0x7fdc40106f10 2024-09-11T23:50:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.863+0000 7fdc44969640 1 -- 172.21.15.62:0/3433434593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc38003e50 con 0x7fdc40106f10 2024-09-11T23:50:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.863+0000 7fdc4596b640 1 -- 172.21.15.62:0/3433434593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4019d710 con 0x7fdc40106f10 2024-09-11T23:50:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.863+0000 7fdc44969640 1 -- 172.21.15.62:0/3433434593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc380106c0 con 0x7fdc40106f10 2024-09-11T23:50:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.863+0000 7fdc4596b640 1 -- 172.21.15.62:0/3433434593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc401a4ff0 con 0x7fdc40106f10 2024-09-11T23:50:59.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.864+0000 7fdc4596b640 1 -- 172.21.15.62:0/3433434593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc401032c0 con 0x7fdc40106f10 2024-09-11T23:50:59.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.865+0000 7fdc44969640 1 -- 172.21.15.62:0/3433434593 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdc38003970 con 0x7fdc40106f10 2024-09-11T23:50:59.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.865+0000 7fdc44969640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdc280a2c10 0x7fdc280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:50:59.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.866+0000 7fdc3effd640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdc280a2c10 0x7fdc280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:50:59.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.866+0000 7fdc44969640 1 -- 172.21.15.62:0/3433434593 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fdc380ceca0 con 0x7fdc40106f10 2024-09-11T23:50:59.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.866+0000 7fdc3effd640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdc280a2c10 0x7fdc280a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc30005d00 tx=0x7fdc3000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:50:59.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:50:59.870+0000 7fdc44969640 1 -- 172.21.15.62:0/3433434593 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc380d3070 con 0x7fdc40106f10 2024-09-11T23:50:59.930 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:59.930 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:59.931 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:59.932 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:59.938 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:59.938 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:50:59.939 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:00.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:50:59 smithi062 ceph-mon[30958]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:00.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.090+0000 7fdc4596b640 1 -- 172.21.15.62:0/3433434593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fdc4010a180 con 0x7fdc40106f10 2024-09-11T23:51:00.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.090+0000 7fdc44969640 1 -- 172.21.15.62:0/3433434593 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdc3809b600 con 0x7fdc40106f10 2024-09-11T23:51:00.090 INFO:teuthology.orchestra.run.smithi062.stdout:270582939682 2024-09-11T23:51:00.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.092+0000 7fdc4596b640 1 -- 172.21.15.62:0/3433434593 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdc280a2c10 msgr2=0x7fdc280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:00.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.092+0000 7fdc4596b640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdc280a2c10 0x7fdc280a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc30005d00 tx=0x7fdc3000b410 comp rx=0 tx=0).stop 2024-09-11T23:51:00.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.092+0000 7fdc4596b640 1 -- 172.21.15.62:0/3433434593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc40106f10 msgr2=0x7fdc4019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:00.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.092+0000 7fdc4596b640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc40106f10 0x7fdc4019cd40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009fd0 tx=0x7fdc3800b260 comp rx=0 tx=0).stop 2024-09-11T23:51:00.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.092+0000 7fdc4596b640 1 -- 172.21.15.62:0/3433434593 shutdown_connections 2024-09-11T23:51:00.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.092+0000 7fdc4596b640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fdc280a2c10 0x7fdc280a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:00.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.092+0000 7fdc4596b640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc40106f10 0x7fdc4019cd40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:00.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.092+0000 7fdc4596b640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc40102790 0x7fdc401989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:00.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.092+0000 7fdc4596b640 1 --2- 172.21.15.62:0/3433434593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc40101de0 0x7fdc40198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:00.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.092+0000 7fdc4596b640 1 -- 172.21.15.62:0/3433434593 >> 172.21.15.62:0/3433434593 conn(0x7fdc400fd8c0 msgr2=0x7fdc400fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:00.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.093+0000 7fdc4596b640 1 -- 172.21.15.62:0/3433434593 shutdown_connections 2024-09-11T23:51:00.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.093+0000 7fdc4596b640 1 -- 172.21.15.62:0/3433434593 wait complete. 2024-09-11T23:51:00.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:50:59 smithi120 ceph-mon[35182]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:00.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:50:59 smithi083 ceph-mon[34805]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.896+0000 7f4e00a5c640 1 -- 172.21.15.62:0/677691603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dfc100e20 msgr2=0x7f4dfc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.896+0000 7f4e00a5c640 1 --2- 172.21.15.62:0/677691603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dfc100e20 0x7f4dfc101200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009900 tx=0x7f4dec02a480 comp rx=0 tx=0).stop 2024-09-11T23:51:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.896+0000 7f4e00a5c640 1 -- 172.21.15.62:0/677691603 shutdown_connections 2024-09-11T23:51:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.896+0000 7f4e00a5c640 1 --2- 172.21.15.62:0/677691603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dfc102ef0 0x7f4dfc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.896+0000 7f4e00a5c640 1 --2- 172.21.15.62:0/677691603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dfc102570 0x7f4dfc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.896+0000 7f4e00a5c640 1 --2- 172.21.15.62:0/677691603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dfc100e20 0x7f4dfc101200 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.896+0000 7f4e00a5c640 1 -- 172.21.15.62:0/677691603 >> 172.21.15.62:0/677691603 conn(0x7f4dfc0fc7d0 msgr2=0x7f4dfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.896+0000 7f4e00a5c640 1 -- 172.21.15.62:0/677691603 shutdown_connections 2024-09-11T23:51:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.896+0000 7f4e00a5c640 1 -- 172.21.15.62:0/677691603 wait complete. 2024-09-11T23:51:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.897+0000 7f4e00a5c640 1 Processor -- start 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.897+0000 7f4e00a5c640 1 -- start start 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.897+0000 7f4e00a5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dfc100e20 0x7f4dfc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.897+0000 7f4e00a5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dfc102570 0x7f4dfc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4e00a5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dfc102ef0 0x7f4dfc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4e00a5c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc110f50 con 0x7f4dfc102ef0 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4e00a5c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc110dd0 con 0x7f4dfc102570 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4e00a5c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1110d0 con 0x7f4dfc100e20 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4df3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dfc102570 0x7f4dfc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4dfad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dfc102ef0 0x7f4dfc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4df3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dfc102570 0x7f4dfc0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:38706/0 (socket says 172.21.15.62:38706) 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4df3fff640 1 -- 172.21.15.62:0/2744652097 learned_addr learned my addr 172.21.15.62:0/2744652097 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4dfa59c640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dfc100e20 0x7f4dfc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:00.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4df3fff640 1 -- 172.21.15.62:0/2744652097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dfc100e20 msgr2=0x7f4dfc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:00.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4df3fff640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dfc100e20 0x7f4dfc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:00.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4df3fff640 1 -- 172.21.15.62:0/2744652097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dfc102ef0 msgr2=0x7f4dfc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:00.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4df3fff640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dfc102ef0 0x7f4dfc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:00.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4df3fff640 1 -- 172.21.15.62:0/2744652097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc1a6a20 con 0x7f4dfc102570 2024-09-11T23:51:00.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4dfa59c640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dfc100e20 0x7f4dfc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:00.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4dfad9d640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dfc102ef0 0x7f4dfc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:51:00.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4df3fff640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dfc102570 0x7f4dfc0fd1c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4de400d950 tx=0x7f4de400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:00.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.898+0000 7f4df37fe640 1 -- 172.21.15.62:0/2744652097 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4017030 con 0x7f4dfc102570 2024-09-11T23:51:00.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.899+0000 7f4e00a5c640 1 -- 172.21.15.62:0/2744652097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc1a6d10 con 0x7f4dfc102570 2024-09-11T23:51:00.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.899+0000 7f4df37fe640 1 -- 172.21.15.62:0/2744652097 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de4004020 con 0x7f4dfc102570 2024-09-11T23:51:00.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.899+0000 7f4df37fe640 1 -- 172.21.15.62:0/2744652097 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de40047a0 con 0x7f4dfc102570 2024-09-11T23:51:00.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.899+0000 7f4e00a5c640 1 -- 172.21.15.62:0/2744652097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc1a71e0 con 0x7f4dfc102570 2024-09-11T23:51:00.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.900+0000 7f4e00a5c640 1 -- 172.21.15.62:0/2744652097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dfc101200 con 0x7f4dfc102570 2024-09-11T23:51:00.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.901+0000 7f4df37fe640 1 -- 172.21.15.62:0/2744652097 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4de40041d0 con 0x7f4dfc102570 2024-09-11T23:51:00.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.902+0000 7f4df37fe640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4de80a2c10 0x7f4de80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:00.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.902+0000 7f4dfa59c640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4de80a2c10 0x7f4de80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:00.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.902+0000 7f4df37fe640 1 -- 172.21.15.62:0/2744652097 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f4de40cf1a0 con 0x7f4dfc102570 2024-09-11T23:51:00.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.902+0000 7f4dfa59c640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4de80a2c10 0x7f4de80a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4dec005ab0 tx=0x7f4dec005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:00.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.905+0000 7f4df37fe640 1 -- 172.21.15.62:0/2744652097 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4de409c050 con 0x7f4dfc102570 2024-09-11T23:51:01.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.089+0000 7f4e00a5c640 1 -- 172.21.15.62:0/2744652097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4dfc0fe1b0 con 0x7f4dfc102570 2024-09-11T23:51:01.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.090+0000 7f4df37fe640 1 -- 172.21.15.62:0/2744652097 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4dfc0fe1b0 con 0x7f4dfc102570 2024-09-11T23:51:01.090 INFO:teuthology.orchestra.run.smithi062.stdout:77309411397 2024-09-11T23:51:01.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 -- 172.21.15.62:0/2744652097 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4de80a2c10 msgr2=0x7f4de80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:01.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4de80a2c10 0x7f4de80a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4dec005ab0 tx=0x7f4dec005a20 comp rx=0 tx=0).stop 2024-09-11T23:51:01.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 -- 172.21.15.62:0/2744652097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dfc102570 msgr2=0x7f4dfc0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:01.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dfc102570 0x7f4dfc0fd1c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4de400d950 tx=0x7f4de400de10 comp rx=0 tx=0).stop 2024-09-11T23:51:01.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 -- 172.21.15.62:0/2744652097 shutdown_connections 2024-09-11T23:51:01.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4de80a2c10 0x7f4de80a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dfc102ef0 0x7f4dfc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dfc102570 0x7f4dfc0fd1c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 --2- 172.21.15.62:0/2744652097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dfc100e20 0x7f4dfc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 -- 172.21.15.62:0/2744652097 >> 172.21.15.62:0/2744652097 conn(0x7f4dfc0fc7d0 msgr2=0x7f4dfc06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:01.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 -- 172.21.15.62:0/2744652097 shutdown_connections 2024-09-11T23:51:01.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:01.092+0000 7f4e00a5c640 1 -- 172.21.15.62:0/2744652097 wait complete. 2024-09-11T23:51:01.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:00 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3433434593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:51:01.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:00 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3433434593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:51:01.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:00 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3433434593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:51:01.548 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939680 got 270582939682 for osd.9 2024-09-11T23:51:01.548 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:01.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.787+0000 7f54c9f0e640 1 -- 172.21.15.62:0/1346332098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54c4101de0 msgr2=0x7f54c41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:01.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.787+0000 7f54c9f0e640 1 --2- 172.21.15.62:0/1346332098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54c4101de0 0x7f54c41021c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009900 tx=0x7f54b402a410 comp rx=0 tx=0).stop 2024-09-11T23:51:01.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.787+0000 7f54c9f0e640 1 -- 172.21.15.62:0/1346332098 shutdown_connections 2024-09-11T23:51:01.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.787+0000 7f54c9f0e640 1 --2- 172.21.15.62:0/1346332098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54c4106f10 0x7f54c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.787+0000 7f54c9f0e640 1 --2- 172.21.15.62:0/1346332098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102790 0x7f54c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.787+0000 7f54c9f0e640 1 --2- 172.21.15.62:0/1346332098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54c4101de0 0x7f54c41021c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.787+0000 7f54c9f0e640 1 -- 172.21.15.62:0/1346332098 >> 172.21.15.62:0/1346332098 conn(0x7f54c40fd8c0 msgr2=0x7f54c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:01.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.787+0000 7f54c9f0e640 1 -- 172.21.15.62:0/1346332098 shutdown_connections 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.787+0000 7f54c9f0e640 1 -- 172.21.15.62:0/1346332098 wait complete. 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c9f0e640 1 Processor -- start 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c9f0e640 1 -- start start 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c9f0e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54c4101de0 0x7f54c4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c9f0e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54c4102790 0x7f54c4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c9f0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4106f10 0x7f54c419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c9f0e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4111f90 con 0x7f54c4102790 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c9f0e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4111e10 con 0x7f54c4106f10 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c9f0e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4112110 con 0x7f54c4101de0 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54c4102790 0x7f54c4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54c4102790 0x7f54c4198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39040/0 (socket says 172.21.15.62:39040) 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c2ffd640 1 -- 172.21.15.62:0/910509624 learned_addr learned my addr 172.21.15.62:0/910509624 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c37fe640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54c4101de0 0x7f54c4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c3fff640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4106f10 0x7f54c419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c2ffd640 1 -- 172.21.15.62:0/910509624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54c4101de0 msgr2=0x7f54c4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c2ffd640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54c4101de0 0x7f54c4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c2ffd640 1 -- 172.21.15.62:0/910509624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4106f10 msgr2=0x7f54c419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.788+0000 7f54c2ffd640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4106f10 0x7f54c419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.789+0000 7f54c2ffd640 1 -- 172.21.15.62:0/910509624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c419d4c0 con 0x7f54c4102790 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.789+0000 7f54c37fe640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54c4101de0 0x7f54c4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.789+0000 7f54c2ffd640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54c4102790 0x7f54c4198a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c890 tx=0x7f54b800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.789+0000 7f54c0ff9640 1 -- 172.21.15.62:0/910509624 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8004220 con 0x7f54c4102790 2024-09-11T23:51:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.789+0000 7f54c9f0e640 1 -- 172.21.15.62:0/910509624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c419d7b0 con 0x7f54c4102790 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.789+0000 7f54c0ff9640 1 -- 172.21.15.62:0/910509624 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b80043c0 con 0x7f54c4102790 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.789+0000 7f54c0ff9640 1 -- 172.21.15.62:0/910509624 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80049e0 con 0x7f54c4102790 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.789+0000 7f54c9f0e640 1 -- 172.21.15.62:0/910509624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c41a5090 con 0x7f54c4102790 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.790+0000 7f54c9f0e640 1 -- 172.21.15.62:0/910509624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c41021c0 con 0x7f54c4102790 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.791+0000 7f54c0ff9640 1 -- 172.21.15.62:0/910509624 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f54b80079b0 con 0x7f54c4102790 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.792+0000 7f54c0ff9640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f54ac0a2c10 0x7f54ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.792+0000 7f54c37fe640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f54ac0a2c10 0x7f54ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.792+0000 7f54c0ff9640 1 -- 172.21.15.62:0/910509624 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f54b80cf180 con 0x7f54c4102790 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.793+0000 7f54c37fe640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f54ac0a2c10 0x7f54ac0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54b4005ab0 tx=0x7f54b4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.795+0000 7f54c0ff9640 1 -- 172.21.15.62:0/910509624 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54b8014030 con 0x7f54c4102790 2024-09-11T23:51:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.980+0000 7f54c9f0e640 1 -- 172.21.15.62:0/910509624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f54c4199870 con 0x7f54c4102790 2024-09-11T23:51:01.842 INFO:teuthology.orchestra.run.smithi062.stdout:292057776155 2024-09-11T23:51:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.980+0000 7f54c0ff9640 1 -- 172.21.15.62:0/910509624 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f54b809c130 con 0x7f54c4102790 2024-09-11T23:51:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 -- 172.21.15.62:0/910509624 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f54ac0a2c10 msgr2=0x7f54ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f54ac0a2c10 0x7f54ac0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54b4005ab0 tx=0x7f54b4005a20 comp rx=0 tx=0).stop 2024-09-11T23:51:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 -- 172.21.15.62:0/910509624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54c4102790 msgr2=0x7f54c4198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54c4102790 0x7f54c4198a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c890 tx=0x7f54b800cd50 comp rx=0 tx=0).stop 2024-09-11T23:51:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 -- 172.21.15.62:0/910509624 shutdown_connections 2024-09-11T23:51:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f54ac0a2c10 0x7f54ac0a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4106f10 0x7f54c419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54c4102790 0x7f54c4198a50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 --2- 172.21.15.62:0/910509624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54c4101de0 0x7f54c4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:01.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 -- 172.21.15.62:0/910509624 >> 172.21.15.62:0/910509624 conn(0x7f54c40fd8c0 msgr2=0x7f54c40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:01.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 -- 172.21.15.62:0/910509624 shutdown_connections 2024-09-11T23:51:01.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:00.982+0000 7f54c9f0e640 1 -- 172.21.15.62:0/910509624 wait complete. 2024-09-11T23:51:02.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:01 smithi062 ceph-mon[30958]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:02.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:01 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/910509624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:51:02.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:01 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2744652097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:51:02.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:01 smithi120 ceph-mon[35182]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:02.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:01 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/910509624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:51:02.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:01 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2744652097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:51:02.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:01 smithi083 ceph-mon[34805]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:02.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:01 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/910509624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:51:02.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:01 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2744652097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:51:02.472 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776153 got 292057776155 for osd.10 2024-09-11T23:51:02.473 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:02.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.639+0000 7fb064bc8640 1 -- 172.21.15.62:0/2654887239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb060100e20 msgr2=0x7fb060101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:02.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.639+0000 7fb064bc8640 1 --2- 172.21.15.62:0/2654887239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb060100e20 0x7fb060101200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb050009ca0 tx=0x7fb05002a950 comp rx=0 tx=0).stop 2024-09-11T23:51:02.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.640+0000 7fb064bc8640 1 -- 172.21.15.62:0/2654887239 shutdown_connections 2024-09-11T23:51:02.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.640+0000 7fb064bc8640 1 --2- 172.21.15.62:0/2654887239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb060102ef0 0x7fb06010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:02.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.640+0000 7fb064bc8640 1 --2- 172.21.15.62:0/2654887239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb060102570 0x7fb0601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:02.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.640+0000 7fb064bc8640 1 --2- 172.21.15.62:0/2654887239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb060100e20 0x7fb060101200 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:02.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.640+0000 7fb064bc8640 1 -- 172.21.15.62:0/2654887239 >> 172.21.15.62:0/2654887239 conn(0x7fb0600fc7d0 msgr2=0x7fb0600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:02.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.640+0000 7fb064bc8640 1 -- 172.21.15.62:0/2654887239 shutdown_connections 2024-09-11T23:51:02.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.640+0000 7fb064bc8640 1 -- 172.21.15.62:0/2654887239 wait complete. 2024-09-11T23:51:02.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.640+0000 7fb064bc8640 1 Processor -- start 2024-09-11T23:51:02.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.640+0000 7fb064bc8640 1 -- start start 2024-09-11T23:51:02.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb064bc8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb060100e20 0x7fb060197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:02.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb064bc8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb060102570 0x7fb060197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:02.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb064bc8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb060102ef0 0x7fb06019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:02.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb064bc8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb060110f50 con 0x7fb060100e20 2024-09-11T23:51:02.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb064bc8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb060110dd0 con 0x7fb060102570 2024-09-11T23:51:02.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb064bc8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601110d0 con 0x7fb060102ef0 2024-09-11T23:51:02.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb060100e20 0x7fb060197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:02.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb060100e20 0x7fb060197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57828/0 (socket says 172.21.15.62:57828) 2024-09-11T23:51:02.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05e59c640 1 -- 172.21.15.62:0/128900478 learned_addr learned my addr 172.21.15.62:0/128900478 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:02.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05ed9d640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb060102ef0 0x7fb06019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:02.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05dd9b640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb060102570 0x7fb060197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:02.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05e59c640 1 -- 172.21.15.62:0/128900478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb060102ef0 msgr2=0x7fb06019bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:02.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05e59c640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb060102ef0 0x7fb06019bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:02.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05e59c640 1 -- 172.21.15.62:0/128900478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb060102570 msgr2=0x7fb060197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:02.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05e59c640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb060102570 0x7fb060197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:02.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05e59c640 1 -- 172.21.15.62:0/128900478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06019c2c0 con 0x7fb060100e20 2024-09-11T23:51:02.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05dd9b640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb060102570 0x7fb060197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:02.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.641+0000 7fb05ed9d640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb060102ef0 0x7fb06019bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:02.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.642+0000 7fb05e59c640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb060100e20 0x7fb060197310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb050005cc0 tx=0x7fb05002f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:02.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.642+0000 7fb0577fe640 1 -- 172.21.15.62:0/128900478 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050034040 con 0x7fb060100e20 2024-09-11T23:51:02.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.642+0000 7fb064bc8640 1 -- 172.21.15.62:0/128900478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06019c550 con 0x7fb060100e20 2024-09-11T23:51:02.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.642+0000 7fb0577fe640 1 -- 172.21.15.62:0/128900478 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050003e40 con 0x7fb060100e20 2024-09-11T23:51:02.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.642+0000 7fb0577fe640 1 -- 172.21.15.62:0/128900478 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050045480 con 0x7fb060100e20 2024-09-11T23:51:02.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.642+0000 7fb064bc8640 1 -- 172.21.15.62:0/128900478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb060069f70 con 0x7fb060100e20 2024-09-11T23:51:02.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.643+0000 7fb064bc8640 1 -- 172.21.15.62:0/128900478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0601021b0 con 0x7fb060100e20 2024-09-11T23:51:02.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.645+0000 7fb0577fe640 1 -- 172.21.15.62:0/128900478 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb050037020 con 0x7fb060100e20 2024-09-11T23:51:02.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.645+0000 7fb0577fe640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb04c0a2c10 0x7fb04c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:02.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.645+0000 7fb05dd9b640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb04c0a2c10 0x7fb04c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:02.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.645+0000 7fb0577fe640 1 -- 172.21.15.62:0/128900478 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb0500f03e0 con 0x7fb060100e20 2024-09-11T23:51:02.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.646+0000 7fb05dd9b640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb04c0a2c10 0x7fb04c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb0600fec10 tx=0x7fb048005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:02.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.649+0000 7fb0577fe640 1 -- 172.21.15.62:0/128900478 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0500bd4a0 con 0x7fb060100e20 2024-09-11T23:51:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.830+0000 7fb064bc8640 1 -- 172.21.15.62:0/128900478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb060198670 con 0x7fb060100e20 2024-09-11T23:51:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.831+0000 7fb0577fe640 1 -- 172.21.15.62:0/128900478 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb0500bcda0 con 0x7fb060100e20 2024-09-11T23:51:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:137438953532 2024-09-11T23:51:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 -- 172.21.15.62:0/128900478 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb04c0a2c10 msgr2=0x7fb04c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb04c0a2c10 0x7fb04c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb0600fec10 tx=0x7fb048005d90 comp rx=0 tx=0).stop 2024-09-11T23:51:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 -- 172.21.15.62:0/128900478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb060100e20 msgr2=0x7fb060197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb060100e20 0x7fb060197310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb050005cc0 tx=0x7fb05002f6c0 comp rx=0 tx=0).stop 2024-09-11T23:51:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 -- 172.21.15.62:0/128900478 shutdown_connections 2024-09-11T23:51:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb04c0a2c10 0x7fb04c0a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb060102ef0 0x7fb06019bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb060102570 0x7fb060197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 --2- 172.21.15.62:0/128900478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb060100e20 0x7fb060197310 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 -- 172.21.15.62:0/128900478 >> 172.21.15.62:0/128900478 conn(0x7fb0600fc7d0 msgr2=0x7fb0601067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 -- 172.21.15.62:0/128900478 shutdown_connections 2024-09-11T23:51:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:02.833+0000 7fb064bc8640 1 -- 172.21.15.62:0/128900478 wait complete. 2024-09-11T23:51:03.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:02 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/128900478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:51:03.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:02 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/128900478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:51:03.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:02 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/128900478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:51:03.501 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953530 got 137438953532 for osd.4 2024-09-11T23:51:03.501 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:04.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:03 smithi062 ceph-mon[30958]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-11T23:51:04.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:03 smithi120 ceph-mon[35182]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-11T23:51:04.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:03 smithi083 ceph-mon[34805]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-11T23:51:04.449 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411397 for osd.2 2024-09-11T23:51:04.449 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:04.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.730+0000 7f92088a1640 1 -- 172.21.15.62:0/2777330293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9204102570 msgr2=0x7f92041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:04.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.730+0000 7f92088a1640 1 --2- 172.21.15.62:0/2777330293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9204102570 0x7f92041029b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009900 tx=0x7f91f802a480 comp rx=0 tx=0).stop 2024-09-11T23:51:04.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.730+0000 7f92088a1640 1 -- 172.21.15.62:0/2777330293 shutdown_connections 2024-09-11T23:51:04.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.730+0000 7f92088a1640 1 --2- 172.21.15.62:0/2777330293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9204102ef0 0x7f920410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.730+0000 7f92088a1640 1 --2- 172.21.15.62:0/2777330293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9204102570 0x7f92041029b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.730+0000 7f92088a1640 1 --2- 172.21.15.62:0/2777330293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9204100e20 0x7f9204101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.730+0000 7f92088a1640 1 -- 172.21.15.62:0/2777330293 >> 172.21.15.62:0/2777330293 conn(0x7f92040fc7d0 msgr2=0x7f92040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:04.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.730+0000 7f92088a1640 1 -- 172.21.15.62:0/2777330293 shutdown_connections 2024-09-11T23:51:04.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.730+0000 7f92088a1640 1 -- 172.21.15.62:0/2777330293 wait complete. 2024-09-11T23:51:04.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f92088a1640 1 Processor -- start 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f92088a1640 1 -- start start 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f92088a1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9204100e20 0x7f92040fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f92088a1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9204102570 0x7f92040fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f92088a1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9204102ef0 0x7f92040fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f92088a1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9204110f50 con 0x7f9204102570 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f92088a1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9204110dd0 con 0x7f9204102ef0 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f92088a1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92041110d0 con 0x7f9204100e20 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f9201d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9204102570 0x7f92040fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f9201d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9204102570 0x7f92040fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57884/0 (socket says 172.21.15.62:57884) 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.731+0000 7f9201d9b640 1 -- 172.21.15.62:0/2114066207 learned_addr learned my addr 172.21.15.62:0/2114066207 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f9202d9d640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9204102ef0 0x7f92040fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f920259c640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9204100e20 0x7f92040fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f9201d9b640 1 -- 172.21.15.62:0/2114066207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9204100e20 msgr2=0x7f92040fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f9201d9b640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9204100e20 0x7f92040fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f9201d9b640 1 -- 172.21.15.62:0/2114066207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9204102ef0 msgr2=0x7f92040fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f9201d9b640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9204102ef0 0x7f92040fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f9201d9b640 1 -- 172.21.15.62:0/2114066207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92041a6980 con 0x7f9204102570 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f9201d9b640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9204102570 0x7f92040fd1c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f91f8006010 tx=0x7f91f8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f91f37fe640 1 -- 172.21.15.62:0/2114066207 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f803f070 con 0x7f9204102570 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f92088a1640 1 -- 172.21.15.62:0/2114066207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92041a6c10 con 0x7f9204102570 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f91f37fe640 1 -- 172.21.15.62:0/2114066207 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8031c30 con 0x7f9204102570 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f91f37fe640 1 -- 172.21.15.62:0/2114066207 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8035030 con 0x7f9204102570 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.732+0000 7f92088a1640 1 -- 172.21.15.62:0/2114066207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92041a7140 con 0x7f9204102570 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.733+0000 7f92088a1640 1 -- 172.21.15.62:0/2114066207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9204101200 con 0x7f9204102570 2024-09-11T23:51:04.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.735+0000 7f91f37fe640 1 -- 172.21.15.62:0/2114066207 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f91f8042070 con 0x7f9204102570 2024-09-11T23:51:04.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.735+0000 7f91f37fe640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91ec0a2c10 0x7f91ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:04.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.735+0000 7f920259c640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91ec0a2c10 0x7f91ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:04.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.735+0000 7f91f37fe640 1 -- 172.21.15.62:0/2114066207 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f91f80efc80 con 0x7f9204102570 2024-09-11T23:51:04.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.736+0000 7f920259c640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91ec0a2c10 0x7f91ec0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f91f40058c0 tx=0x7f91f400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:04.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.738+0000 7f91f37fe640 1 -- 172.21.15.62:0/2114066207 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91f8031de0 con 0x7f9204102570 2024-09-11T23:51:04.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.921+0000 7f92088a1640 1 -- 172.21.15.62:0/2114066207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f920410a0a0 con 0x7f9204102570 2024-09-11T23:51:04.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.921+0000 7f91f37fe640 1 -- 172.21.15.62:0/2114066207 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f91f80bccf0 con 0x7f9204102570 2024-09-11T23:51:04.921 INFO:teuthology.orchestra.run.smithi062.stdout:55834574923 2024-09-11T23:51:04.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 -- 172.21.15.62:0/2114066207 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91ec0a2c10 msgr2=0x7f91ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91ec0a2c10 0x7f91ec0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f91f40058c0 tx=0x7f91f400a4c0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 -- 172.21.15.62:0/2114066207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9204102570 msgr2=0x7f92040fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9204102570 0x7f92040fd1c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f91f8006010 tx=0x7f91f8003c10 comp rx=0 tx=0).stop 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 -- 172.21.15.62:0/2114066207 shutdown_connections 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91ec0a2c10 0x7f91ec0a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9204102ef0 0x7f92040fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9204102570 0x7f92040fd1c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 --2- 172.21.15.62:0/2114066207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9204100e20 0x7f92040fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 -- 172.21.15.62:0/2114066207 >> 172.21.15.62:0/2114066207 conn(0x7f92040fc7d0 msgr2=0x7f920406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 -- 172.21.15.62:0/2114066207 shutdown_connections 2024-09-11T23:51:04.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.925+0000 7f92088a1640 1 -- 172.21.15.62:0/2114066207 wait complete. 2024-09-11T23:51:04.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.960+0000 7f8a9a7df640 1 -- 172.21.15.62:0/3602524822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a94102d30 msgr2=0x7f8a94103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:04.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.960+0000 7f8a9a7df640 1 --2- 172.21.15.62:0/3602524822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a94102d30 0x7f8a94103110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009930 tx=0x7f8a8402a420 comp rx=0 tx=0).stop 2024-09-11T23:51:04.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.960+0000 7f8a9a7df640 1 -- 172.21.15.62:0/3602524822 shutdown_connections 2024-09-11T23:51:04.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.960+0000 7f8a9a7df640 1 --2- 172.21.15.62:0/3602524822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a941082a0 0x7f8a9410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.960+0000 7f8a9a7df640 1 --2- 172.21.15.62:0/3602524822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a941036e0 0x7f8a94107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.960+0000 7f8a9a7df640 1 --2- 172.21.15.62:0/3602524822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a94102d30 0x7f8a94103110 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.960+0000 7f8a9a7df640 1 -- 172.21.15.62:0/3602524822 >> 172.21.15.62:0/3602524822 conn(0x7f8a940fc7d0 msgr2=0x7f8a940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:04.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.960+0000 7f8a9a7df640 1 -- 172.21.15.62:0/3602524822 shutdown_connections 2024-09-11T23:51:04.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.960+0000 7f8a9a7df640 1 -- 172.21.15.62:0/3602524822 wait complete. 2024-09-11T23:51:04.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.961+0000 7f8a9a7df640 1 Processor -- start 2024-09-11T23:51:04.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.961+0000 7f8a9a7df640 1 -- start start 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.961+0000 7f8a9a7df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a94102d30 0x7f8a94100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.961+0000 7f8a9a7df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a941036e0 0x7f8a94100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.961+0000 7f8a9a7df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a941082a0 0x7f8a94101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.961+0000 7f8a9a7df640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94112100 con 0x7f8a941082a0 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.961+0000 7f8a9a7df640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94111f80 con 0x7f8a941036e0 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.961+0000 7f8a9a7df640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94112280 con 0x7f8a94102d30 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.961+0000 7f8a98d7c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a941082a0 0x7f8a94101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a98d7c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a941082a0 0x7f8a94101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57908/0 (socket says 172.21.15.62:57908) 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a98d7c640 1 -- 172.21.15.62:0/1092714618 learned_addr learned my addr 172.21.15.62:0/1092714618 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a93fff640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a94102d30 0x7f8a94100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a937fe640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a941036e0 0x7f8a94100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a98d7c640 1 -- 172.21.15.62:0/1092714618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a94102d30 msgr2=0x7f8a94100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a98d7c640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a94102d30 0x7f8a94100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a98d7c640 1 -- 172.21.15.62:0/1092714618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a941036e0 msgr2=0x7f8a94100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a98d7c640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a941036e0 0x7f8a94100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a98d7c640 1 -- 172.21.15.62:0/1092714618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a941a6aa0 con 0x7f8a941082a0 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a937fe640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a941036e0 0x7f8a94100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a98d7c640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a941082a0 0x7f8a94101410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00b970 tx=0x7f8a8c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a917fa640 1 -- 172.21.15.62:0/1092714618 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c019070 con 0x7f8a941082a0 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a917fa640 1 -- 172.21.15.62:0/1092714618 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a8c00cb40 con 0x7f8a941082a0 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a9a7df640 1 -- 172.21.15.62:0/1092714618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a941a6d90 con 0x7f8a941082a0 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a917fa640 1 -- 172.21.15.62:0/1092714618 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c007650 con 0x7f8a941082a0 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.962+0000 7f8a9a7df640 1 -- 172.21.15.62:0/1092714618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a941a7280 con 0x7f8a941082a0 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.963+0000 7f8a9a7df640 1 -- 172.21.15.62:0/1092714618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a94103110 con 0x7f8a941082a0 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.965+0000 7f8a917fa640 1 -- 172.21.15.62:0/1092714618 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8a8c01c070 con 0x7f8a941082a0 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.965+0000 7f8a917fa640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a7c0a2c10 0x7f8a7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:04.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.965+0000 7f8a93fff640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a7c0a2c10 0x7f8a7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:04.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.965+0000 7f8a917fa640 1 -- 172.21.15.62:0/1092714618 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f8a8c0ce920 con 0x7f8a941082a0 2024-09-11T23:51:04.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.966+0000 7f8a93fff640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a7c0a2c10 0x7f8a7c0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8a84005940 tx=0x7f8a84005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:04.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.969+0000 7f8a917fa640 1 -- 172.21.15.62:0/1092714618 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a8c09b390 con 0x7f8a941082a0 2024-09-11T23:51:05.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.150+0000 7f8a9a7df640 1 -- 172.21.15.62:0/1092714618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8a9410a2b0 con 0x7f8a941082a0 2024-09-11T23:51:05.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.151+0000 7f8a917fa640 1 -- 172.21.15.62:0/1092714618 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8a8c09f690 con 0x7f8a941082a0 2024-09-11T23:51:05.151 INFO:teuthology.orchestra.run.smithi062.stdout:158913790007 2024-09-11T23:51:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 -- 172.21.15.62:0/1092714618 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a7c0a2c10 msgr2=0x7f8a7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a7c0a2c10 0x7f8a7c0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8a84005940 tx=0x7f8a84005fd0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 -- 172.21.15.62:0/1092714618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a941082a0 msgr2=0x7f8a94101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a941082a0 0x7f8a94101410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00b970 tx=0x7f8a8c00be30 comp rx=0 tx=0).stop 2024-09-11T23:51:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 -- 172.21.15.62:0/1092714618 shutdown_connections 2024-09-11T23:51:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a7c0a2c10 0x7f8a7c0a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a941082a0 0x7f8a94101410 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a941036e0 0x7f8a94100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 --2- 172.21.15.62:0/1092714618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a94102d30 0x7f8a94100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 -- 172.21.15.62:0/1092714618 >> 172.21.15.62:0/1092714618 conn(0x7f8a940fc7d0 msgr2=0x7f8a94106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 -- 172.21.15.62:0/1092714618 shutdown_connections 2024-09-11T23:51:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:05.153+0000 7f8a9a7df640 1 -- 172.21.15.62:0/1092714618 wait complete. 2024-09-11T23:51:05.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:04 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2769980532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:51:05.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:04 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2114066207' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:51:05.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:04 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2769980532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:51:05.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:04 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2114066207' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:51:05.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:04 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2769980532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:51:05.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:04 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2114066207' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:51:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.630+0000 7f9215bbb640 1 -- 172.21.15.62:0/3034929124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9210100ca0 msgr2=0x7f9210106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.630+0000 7f9215bbb640 1 --2- 172.21.15.62:0/3034929124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9210100ca0 0x7f9210106840 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9204009930 tx=0x7f920402a420 comp rx=0 tx=0).stop 2024-09-11T23:51:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.630+0000 7f9215bbb640 1 -- 172.21.15.62:0/3034929124 shutdown_connections 2024-09-11T23:51:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.630+0000 7f9215bbb640 1 --2- 172.21.15.62:0/3034929124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9210106f50 0x7f921010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.631+0000 7f9215bbb640 1 --2- 172.21.15.62:0/3034929124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9210100ca0 0x7f9210106840 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.631+0000 7f9215bbb640 1 --2- 172.21.15.62:0/3034929124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92101002f0 0x7f92101006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.631+0000 7f9215bbb640 1 -- 172.21.15.62:0/3034929124 >> 172.21.15.62:0/3034929124 conn(0x7f9210077210 msgr2=0x7f92100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.631+0000 7f9215bbb640 1 -- 172.21.15.62:0/3034929124 shutdown_connections 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.631+0000 7f9215bbb640 1 -- 172.21.15.62:0/3034929124 wait complete. 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.631+0000 7f9215bbb640 1 Processor -- start 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.631+0000 7f9215bbb640 1 -- start start 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f9215bbb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92101002f0 0x7f9210101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f9215bbb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9210100ca0 0x7f92101017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f9215bbb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9210106f50 0x7f9210101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f9215bbb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9210112060 con 0x7f9210106f50 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f9215bbb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9210111ee0 con 0x7f9210100ca0 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f9215bbb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92101121e0 con 0x7f92101002f0 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9210106f50 0x7f9210101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9210106f50 0x7f9210101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57858/0 (socket says 172.21.15.62:57858) 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920ffff640 1 -- 172.21.15.62:0/2769980532 learned_addr learned my addr 172.21.15.62:0/2769980532 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920f7fe640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92101002f0 0x7f9210101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920effd640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9210100ca0 0x7f92101017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920ffff640 1 -- 172.21.15.62:0/2769980532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92101002f0 msgr2=0x7f9210101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920ffff640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92101002f0 0x7f9210101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920ffff640 1 -- 172.21.15.62:0/2769980532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9210100ca0 msgr2=0x7f92101017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920ffff640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9210100ca0 0x7f92101017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920ffff640 1 -- 172.21.15.62:0/2769980532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92101a6b40 con 0x7f9210106f50 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920f7fe640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92101002f0 0x7f9210101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920ffff640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9210106f50 0x7f9210101cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f920800b970 tx=0x7f920800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920cff9640 1 -- 172.21.15.62:0/2769980532 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208019070 con 0x7f9210106f50 2024-09-11T23:51:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920cff9640 1 -- 172.21.15.62:0/2769980532 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f920800cb40 con 0x7f9210106f50 2024-09-11T23:51:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f9215bbb640 1 -- 172.21.15.62:0/2769980532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92101a6e30 con 0x7f9210106f50 2024-09-11T23:51:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f920cff9640 1 -- 172.21.15.62:0/2769980532 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208007650 con 0x7f9210106f50 2024-09-11T23:51:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.632+0000 7f9215bbb640 1 -- 172.21.15.62:0/2769980532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92101a7340 con 0x7f9210106f50 2024-09-11T23:51:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.633+0000 7f9215bbb640 1 -- 172.21.15.62:0/2769980532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9210100750 con 0x7f9210106f50 2024-09-11T23:51:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.635+0000 7f920cff9640 1 -- 172.21.15.62:0/2769980532 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f920801c070 con 0x7f9210106f50 2024-09-11T23:51:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.636+0000 7f920cff9640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91f80a2c90 0x7f91f80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.636+0000 7f920f7fe640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91f80a2c90 0x7f91f80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.636+0000 7f920cff9640 1 -- 172.21.15.62:0/2769980532 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f92080ce470 con 0x7f9210106f50 2024-09-11T23:51:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.636+0000 7f920f7fe640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91f80a2c90 0x7f91f80a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9200006fd0 tx=0x7f9200008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.640+0000 7f920cff9640 1 -- 172.21.15.62:0/2769980532 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f920809b530 con 0x7f9210106f50 2024-09-11T23:51:05.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.823+0000 7f9215bbb640 1 -- 172.21.15.62:0/2769980532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f921010a120 con 0x7f9210106f50 2024-09-11T23:51:05.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.823+0000 7f920cff9640 1 -- 172.21.15.62:0/2769980532 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f920809ae30 con 0x7f9210106f50 2024-09-11T23:51:05.501 INFO:teuthology.orchestra.run.smithi062.stdout:240518168615 2024-09-11T23:51:05.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.825+0000 7f9215bbb640 1 -- 172.21.15.62:0/2769980532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91f80a2c90 msgr2=0x7f91f80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:05.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.825+0000 7f9215bbb640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91f80a2c90 0x7f91f80a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9200006fd0 tx=0x7f9200008040 comp rx=0 tx=0).stop 2024-09-11T23:51:05.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.825+0000 7f9215bbb640 1 -- 172.21.15.62:0/2769980532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9210106f50 msgr2=0x7f9210101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:05.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.825+0000 7f9215bbb640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9210106f50 0x7f9210101cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f920800b970 tx=0x7f920800be30 comp rx=0 tx=0).stop 2024-09-11T23:51:05.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.826+0000 7f9215bbb640 1 -- 172.21.15.62:0/2769980532 shutdown_connections 2024-09-11T23:51:05.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.826+0000 7f9215bbb640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f91f80a2c90 0x7f91f80a5150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.826+0000 7f9215bbb640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9210106f50 0x7f9210101cf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.826+0000 7f9215bbb640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9210100ca0 0x7f92101017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.826+0000 7f9215bbb640 1 --2- 172.21.15.62:0/2769980532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92101002f0 0x7f9210101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:05.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.826+0000 7f9215bbb640 1 -- 172.21.15.62:0/2769980532 >> 172.21.15.62:0/2769980532 conn(0x7f9210077210 msgr2=0x7f92101053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:05.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.826+0000 7f9215bbb640 1 -- 172.21.15.62:0/2769980532 shutdown_connections 2024-09-11T23:51:05.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:04.826+0000 7f9215bbb640 1 -- 172.21.15.62:0/2769980532 wait complete. 2024-09-11T23:51:05.927 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:05.927 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:05.928 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:06.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:06 smithi062 ceph-mon[30958]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:51:06.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:06 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1092714618' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:51:06.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:06 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:51:06.321 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790004 got 158913790007 for osd.5 2024-09-11T23:51:06.322 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:06.328 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:06 smithi120 ceph-mon[35182]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:51:06.328 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:06 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1092714618' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:51:06.329 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:06 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:51:06.387 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:06 smithi083 ceph-mon[34805]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:51:06.387 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:06 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1092714618' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:51:06.387 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:06 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:51:06.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.745+0000 7f2d28d06640 1 -- 172.21.15.62:0/1726884788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24106f10 msgr2=0x7f2d2410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.745+0000 7f2d28d06640 1 --2- 172.21.15.62:0/1726884788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24106f10 0x7f2d2410ea90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009ca0 tx=0x7f2d1c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.746+0000 7f2d28d06640 1 -- 172.21.15.62:0/1726884788 shutdown_connections 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.746+0000 7f2d28d06640 1 --2- 172.21.15.62:0/1726884788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24106f10 0x7f2d2410ea90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.746+0000 7f2d28d06640 1 --2- 172.21.15.62:0/1726884788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d24102790 0x7f2d241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.746+0000 7f2d28d06640 1 --2- 172.21.15.62:0/1726884788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d24101de0 0x7f2d241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.746+0000 7f2d28d06640 1 -- 172.21.15.62:0/1726884788 >> 172.21.15.62:0/1726884788 conn(0x7f2d240fd8c0 msgr2=0x7f2d240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.746+0000 7f2d28d06640 1 -- 172.21.15.62:0/1726884788 shutdown_connections 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.746+0000 7f2d28d06640 1 -- 172.21.15.62:0/1726884788 wait complete. 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.746+0000 7f2d28d06640 1 Processor -- start 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.746+0000 7f2d28d06640 1 -- start start 2024-09-11T23:51:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d28d06640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d24101de0 0x7f2d240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d28d06640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24102790 0x7f2d24074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d28d06640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d24106f10 0x7f2d24074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d28d06640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24111f70 con 0x7f2d24102790 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d28d06640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24111df0 con 0x7f2d24106f10 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d28d06640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241120f0 con 0x7f2d24101de0 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d21d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24102790 0x7f2d24074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d21d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24102790 0x7f2d24074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57926/0 (socket says 172.21.15.62:57926) 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d21d9b640 1 -- 172.21.15.62:0/2307231580 learned_addr learned my addr 172.21.15.62:0/2307231580 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d2259c640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d24101de0 0x7f2d240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d22d9d640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d24106f10 0x7f2d24074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d21d9b640 1 -- 172.21.15.62:0/2307231580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d24101de0 msgr2=0x7f2d240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d21d9b640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d24101de0 0x7f2d240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d21d9b640 1 -- 172.21.15.62:0/2307231580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d24106f10 msgr2=0x7f2d24074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d21d9b640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d24106f10 0x7f2d24074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d21d9b640 1 -- 172.21.15.62:0/2307231580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d241a6870 con 0x7f2d24102790 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d2259c640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d24101de0 0x7f2d240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.747+0000 7f2d21d9b640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24102790 0x7f2d24074a30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800e9f0 tx=0x7f2d1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.748+0000 7f2d137fe640 1 -- 172.21.15.62:0/2307231580 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800cd80 con 0x7f2d24102790 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.748+0000 7f2d137fe640 1 -- 172.21.15.62:0/2307231580 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d18005500 con 0x7f2d24102790 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.748+0000 7f2d28d06640 1 -- 172.21.15.62:0/2307231580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d241a6b60 con 0x7f2d24102790 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.748+0000 7f2d137fe640 1 -- 172.21.15.62:0/2307231580 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18010690 con 0x7f2d24102790 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.748+0000 7f2d28d06640 1 -- 172.21.15.62:0/2307231580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241a7050 con 0x7f2d24102790 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.749+0000 7f2d28d06640 1 -- 172.21.15.62:0/2307231580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d241021c0 con 0x7f2d24102790 2024-09-11T23:51:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.750+0000 7f2d137fe640 1 -- 172.21.15.62:0/2307231580 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2d180046c0 con 0x7f2d24102790 2024-09-11T23:51:06.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.750+0000 7f2d137fe640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2d0c0a2c10 0x7f2d0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:06.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.751+0000 7f2d2259c640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2d0c0a2c10 0x7f2d0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:06.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.751+0000 7f2d137fe640 1 -- 172.21.15.62:0/2307231580 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f2d180cf3d0 con 0x7f2d24102790 2024-09-11T23:51:06.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.751+0000 7f2d2259c640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2d0c0a2c10 0x7f2d0c0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009fc0 tx=0x7f2d14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:06.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.755+0000 7f2d137fe640 1 -- 172.21.15.62:0/2307231580 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d1809c380 con 0x7f2d24102790 2024-09-11T23:51:06.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.943+0000 7f2d28d06640 1 -- 172.21.15.62:0/2307231580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f2d24109b60 con 0x7f2d24102790 2024-09-11T23:51:06.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.944+0000 7f2d137fe640 1 -- 172.21.15.62:0/2307231580 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2d1809bc80 con 0x7f2d24102790 2024-09-11T23:51:06.944 INFO:teuthology.orchestra.run.smithi062.stdout:322122547222 2024-09-11T23:51:06.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 -- 172.21.15.62:0/2307231580 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2d0c0a2c10 msgr2=0x7f2d0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:06.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2d0c0a2c10 0x7f2d0c0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009fc0 tx=0x7f2d14005f90 comp rx=0 tx=0).stop 2024-09-11T23:51:06.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 -- 172.21.15.62:0/2307231580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24102790 msgr2=0x7f2d24074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:06.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24102790 0x7f2d24074a30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800e9f0 tx=0x7f2d1800edf0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 -- 172.21.15.62:0/2307231580 shutdown_connections 2024-09-11T23:51:06.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2d0c0a2c10 0x7f2d0c0a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d24106f10 0x7f2d24074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d24102790 0x7f2d24074a30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 --2- 172.21.15.62:0/2307231580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d24101de0 0x7f2d240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:06.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 -- 172.21.15.62:0/2307231580 >> 172.21.15.62:0/2307231580 conn(0x7f2d240fd8c0 msgr2=0x7f2d240fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:06.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 -- 172.21.15.62:0/2307231580 shutdown_connections 2024-09-11T23:51:06.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.947+0000 7f2d28d06640 1 -- 172.21.15.62:0/2307231580 wait complete. 2024-09-11T23:51:07.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 -- 172.21.15.62:0/4104982384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff230074510 msgr2=0x7ff2300748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:07.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 --2- 172.21.15.62:0/4104982384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff230074510 0x7ff2300748f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009860 tx=0x7ff22c02a3d0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 -- 172.21.15.62:0/4104982384 shutdown_connections 2024-09-11T23:51:07.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 --2- 172.21.15.62:0/4104982384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff23010e440 0x7ff230110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 --2- 172.21.15.62:0/4104982384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff230074e30 0x7ff23010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 --2- 172.21.15.62:0/4104982384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff230074510 0x7ff2300748f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 -- 172.21.15.62:0/4104982384 >> 172.21.15.62:0/4104982384 conn(0x7ff2300fd8c0 msgr2=0x7ff2300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 -- 172.21.15.62:0/4104982384 shutdown_connections 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 -- 172.21.15.62:0/4104982384 wait complete. 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 Processor -- start 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.977+0000 7ff238407640 1 -- start start 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff238407640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff230074510 0x7ff23019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff238407640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff230074e30 0x7ff23019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff238407640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff23010e440 0x7ff23019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff238407640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff230114170 con 0x7ff230074e30 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff238407640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff230113ff0 con 0x7ff23010e440 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff238407640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2301142f0 con 0x7ff230074510 2024-09-11T23:51:07.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2359a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff230074e30 0x7ff23019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2359a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff230074e30 0x7ff23019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57934/0 (socket says 172.21.15.62:57934) 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2359a2640 1 -- 172.21.15.62:0/3913602200 learned_addr learned my addr 172.21.15.62:0/3913602200 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2361a3640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff230074510 0x7ff23019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2369a4640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff23010e440 0x7ff23019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2359a2640 1 -- 172.21.15.62:0/3913602200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff230074510 msgr2=0x7ff23019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2359a2640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff230074510 0x7ff23019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2359a2640 1 -- 172.21.15.62:0/3913602200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff23010e440 msgr2=0x7ff23019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2359a2640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff23010e440 0x7ff23019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2359a2640 1 -- 172.21.15.62:0/3913602200 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23019f6d0 con 0x7ff230074e30 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2359a2640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff230074e30 0x7ff23019ac60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff22000e9f0 tx=0x7ff22000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2277fe640 1 -- 172.21.15.62:0/3913602200 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22000cd80 con 0x7ff230074e30 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff238407640 1 -- 172.21.15.62:0/3913602200 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23019f9c0 con 0x7ff230074e30 2024-09-11T23:51:07.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2277fe640 1 -- 172.21.15.62:0/3913602200 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff220005500 con 0x7ff230074e30 2024-09-11T23:51:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.978+0000 7ff2277fe640 1 -- 172.21.15.62:0/3913602200 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220010690 con 0x7ff230074e30 2024-09-11T23:51:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.979+0000 7ff238407640 1 -- 172.21.15.62:0/3913602200 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2301a72a0 con 0x7ff230074e30 2024-09-11T23:51:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.980+0000 7ff238407640 1 -- 172.21.15.62:0/3913602200 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2300748f0 con 0x7ff230074e30 2024-09-11T23:51:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.981+0000 7ff2277fe640 1 -- 172.21.15.62:0/3913602200 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff2200046c0 con 0x7ff230074e30 2024-09-11T23:51:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.981+0000 7ff2277fe640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff21c0a2c10 0x7ff21c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.982+0000 7ff2361a3640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff21c0a2c10 0x7ff21c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.982+0000 7ff2277fe640 1 -- 172.21.15.62:0/3913602200 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ff2200cf470 con 0x7ff230074e30 2024-09-11T23:51:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.982+0000 7ff2361a3640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff21c0a2c10 0x7ff21c0a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009830 tx=0x7ff22c0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:07.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:06.986+0000 7ff2277fe640 1 -- 172.21.15.62:0/3913602200 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff22009c420 con 0x7ff230074e30 2024-09-11T23:51:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.168+0000 7ff238407640 1 -- 172.21.15.62:0/3913602200 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff2301095c0 con 0x7ff230074e30 2024-09-11T23:51:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.168+0000 7ff2277fe640 1 -- 172.21.15.62:0/3913602200 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff22009bd20 con 0x7ff230074e30 2024-09-11T23:51:07.168 INFO:teuthology.orchestra.run.smithi062.stdout:210453397550 2024-09-11T23:51:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.170+0000 7ff238407640 1 -- 172.21.15.62:0/3913602200 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff21c0a2c10 msgr2=0x7ff21c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.170+0000 7ff238407640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff21c0a2c10 0x7ff21c0a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009830 tx=0x7ff22c0094d0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.170+0000 7ff238407640 1 -- 172.21.15.62:0/3913602200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff230074e30 msgr2=0x7ff23019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.170+0000 7ff238407640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff230074e30 0x7ff23019ac60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff22000e9f0 tx=0x7ff22000edf0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.170+0000 7ff238407640 1 -- 172.21.15.62:0/3913602200 shutdown_connections 2024-09-11T23:51:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.170+0000 7ff238407640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff21c0a2c10 0x7ff21c0a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.170+0000 7ff238407640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff23010e440 0x7ff23019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.170+0000 7ff238407640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff230074e30 0x7ff23019ac60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.170+0000 7ff238407640 1 --2- 172.21.15.62:0/3913602200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff230074510 0x7ff23019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.170+0000 7ff238407640 1 -- 172.21.15.62:0/3913602200 >> 172.21.15.62:0/3913602200 conn(0x7ff2300fd8c0 msgr2=0x7ff2300ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.171+0000 7ff238407640 1 -- 172.21.15.62:0/3913602200 shutdown_connections 2024-09-11T23:51:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:07.171+0000 7ff238407640 1 -- 172.21.15.62:0/3913602200 wait complete. 2024-09-11T23:51:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:07 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2307231580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:51:07.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:07 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2307231580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:51:07.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:07 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2307231580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:51:08.117 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574923 for osd.1 2024-09-11T23:51:08.117 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:08.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:08 smithi120 ceph-mon[35182]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-11T23:51:08.407 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.407 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.407 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.407 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:08 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.407 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:08 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3913602200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:51:08.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:08 smithi062 ceph-mon[30958]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-11T23:51:08.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:08 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:08 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3913602200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:51:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:08 smithi083 ceph-mon[34805]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-11T23:51:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:08 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:08.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:08 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3913602200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:51:08.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.493+0000 7f5262d2b640 1 -- 172.21.15.62:0/2702008761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c101de0 msgr2=0x7f525c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:08.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.493+0000 7f5262d2b640 1 --2- 172.21.15.62:0/2702008761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c101de0 0x7f525c1021c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5250009900 tx=0x7f525002a410 comp rx=0 tx=0).stop 2024-09-11T23:51:08.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.493+0000 7f5262d2b640 1 -- 172.21.15.62:0/2702008761 shutdown_connections 2024-09-11T23:51:08.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.493+0000 7f5262d2b640 1 --2- 172.21.15.62:0/2702008761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f525c106f10 0x7f525c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:08.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.493+0000 7f5262d2b640 1 --2- 172.21.15.62:0/2702008761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f525c102790 0x7f525c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:08.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.493+0000 7f5262d2b640 1 --2- 172.21.15.62:0/2702008761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c101de0 0x7f525c1021c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:08.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.493+0000 7f5262d2b640 1 -- 172.21.15.62:0/2702008761 >> 172.21.15.62:0/2702008761 conn(0x7f525c0fd8c0 msgr2=0x7f525c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:08.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.493+0000 7f5262d2b640 1 -- 172.21.15.62:0/2702008761 shutdown_connections 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.493+0000 7f5262d2b640 1 -- 172.21.15.62:0/2702008761 wait complete. 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f5262d2b640 1 Processor -- start 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f5262d2b640 1 -- start start 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f5262d2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f525c101de0 0x7f525c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f5262d2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f525c102790 0x7f525c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f5262d2b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c106f10 0x7f525c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f5262d2b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f525c111f80 con 0x7f525c106f10 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f5262d2b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f525c111e00 con 0x7f525c101de0 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f5262d2b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f525c112100 con 0x7f525c102790 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f52612c8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c106f10 0x7f525c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f52612c8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c106f10 0x7f525c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57938/0 (socket says 172.21.15.62:57938) 2024-09-11T23:51:08.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f52612c8640 1 -- 172.21.15.62:0/2972901100 learned_addr learned my addr 172.21.15.62:0/2972901100 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f525bfff640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f525c102790 0x7f525c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f5260ac7640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f525c101de0 0x7f525c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f52612c8640 1 -- 172.21.15.62:0/2972901100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f525c102790 msgr2=0x7f525c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f52612c8640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f525c102790 0x7f525c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f52612c8640 1 -- 172.21.15.62:0/2972901100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f525c101de0 msgr2=0x7f525c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f52612c8640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f525c101de0 0x7f525c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f52612c8640 1 -- 172.21.15.62:0/2972901100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c19d390 con 0x7f525c106f10 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.494+0000 7f525bfff640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f525c102790 0x7f525c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.495+0000 7f52612c8640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c106f10 0x7f525c19ccb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5254009fd0 tx=0x7f525400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.495+0000 7f5259ffb640 1 -- 172.21.15.62:0/2972901100 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254017070 con 0x7f525c106f10 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.495+0000 7f5259ffb640 1 -- 172.21.15.62:0/2972901100 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5254003e50 con 0x7f525c106f10 2024-09-11T23:51:08.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.495+0000 7f5262d2b640 1 -- 172.21.15.62:0/2972901100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525c19d680 con 0x7f525c106f10 2024-09-11T23:51:08.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.495+0000 7f5259ffb640 1 -- 172.21.15.62:0/2972901100 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52540106c0 con 0x7f525c106f10 2024-09-11T23:51:08.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.495+0000 7f5262d2b640 1 -- 172.21.15.62:0/2972901100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525c1a4f60 con 0x7f525c106f10 2024-09-11T23:51:08.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.496+0000 7f5262d2b640 1 -- 172.21.15.62:0/2972901100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525c1021c0 con 0x7f525c106f10 2024-09-11T23:51:08.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.497+0000 7f5259ffb640 1 -- 172.21.15.62:0/2972901100 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5254003970 con 0x7f525c106f10 2024-09-11T23:51:08.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.498+0000 7f5259ffb640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f524c0a2c90 0x7f524c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:08.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.498+0000 7f5260ac7640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f524c0a2c90 0x7f524c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:08.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.498+0000 7f5259ffb640 1 -- 172.21.15.62:0/2972901100 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f52540ced70 con 0x7f525c106f10 2024-09-11T23:51:08.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.499+0000 7f5260ac7640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f524c0a2c90 0x7f524c0a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f525002a930 tx=0x7f5250005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:08.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.502+0000 7f5259ffb640 1 -- 172.21.15.62:0/2972901100 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f525409bd20 con 0x7f525c106f10 2024-09-11T23:51:08.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.681+0000 7f5262d2b640 1 -- 172.21.15.62:0/2972901100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f525c10a150 con 0x7f525c106f10 2024-09-11T23:51:08.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.681+0000 7f5259ffb640 1 -- 172.21.15.62:0/2972901100 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f525409b620 con 0x7f525c106f10 2024-09-11T23:51:08.681 INFO:teuthology.orchestra.run.smithi062.stdout:34359738449 2024-09-11T23:51:08.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 -- 172.21.15.62:0/2972901100 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f524c0a2c90 msgr2=0x7f524c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:08.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f524c0a2c90 0x7f524c0a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f525002a930 tx=0x7f5250005fd0 comp rx=0 tx=0).stop 2024-09-11T23:51:08.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 -- 172.21.15.62:0/2972901100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c106f10 msgr2=0x7f525c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:08.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c106f10 0x7f525c19ccb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5254009fd0 tx=0x7f525400b260 comp rx=0 tx=0).stop 2024-09-11T23:51:08.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 -- 172.21.15.62:0/2972901100 shutdown_connections 2024-09-11T23:51:08.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f524c0a2c90 0x7f524c0a5150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:08.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c106f10 0x7f525c19ccb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:08.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f525c102790 0x7f525c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:08.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 --2- 172.21.15.62:0/2972901100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f525c101de0 0x7f525c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:08.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 -- 172.21.15.62:0/2972901100 >> 172.21.15.62:0/2972901100 conn(0x7f525c0fd8c0 msgr2=0x7f525c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:08.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 -- 172.21.15.62:0/2972901100 shutdown_connections 2024-09-11T23:51:08.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:08.683+0000 7f5262d2b640 1 -- 172.21.15.62:0/2972901100 wait complete. 2024-09-11T23:51:08.965 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168613 got 240518168615 for osd.8 2024-09-11T23:51:08.965 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.129+0000 7f1799163640 1 -- 172.21.15.62:0/760745942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1794102dd0 msgr2=0x7f17941031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.129+0000 7f1799163640 1 --2- 172.21.15.62:0/760745942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1794102dd0 0x7f17941031b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1784009930 tx=0x7f178402a440 comp rx=0 tx=0).stop 2024-09-11T23:51:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.129+0000 7f1799163640 1 -- 172.21.15.62:0/760745942 shutdown_connections 2024-09-11T23:51:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.129+0000 7f1799163640 1 --2- 172.21.15.62:0/760745942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1794108300 0x7f179410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.129+0000 7f1799163640 1 --2- 172.21.15.62:0/760745942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1794103780 0x7f1794107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.129+0000 7f1799163640 1 --2- 172.21.15.62:0/760745942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1794102dd0 0x7f17941031b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.129+0000 7f1799163640 1 -- 172.21.15.62:0/760745942 >> 172.21.15.62:0/760745942 conn(0x7f17940fc7d0 msgr2=0x7f17940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.129+0000 7f1799163640 1 -- 172.21.15.62:0/760745942 shutdown_connections 2024-09-11T23:51:09.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.129+0000 7f1799163640 1 -- 172.21.15.62:0/760745942 wait complete. 2024-09-11T23:51:09.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f1799163640 1 Processor -- start 2024-09-11T23:51:09.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f1799163640 1 -- start start 2024-09-11T23:51:09.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f1799163640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1794102dd0 0x7f1794198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:09.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f1799163640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1794103780 0x7f1794198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:09.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f1799163640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1794108300 0x7f179419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f1799163640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f17941120f0 con 0x7f1794108300 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f1799163640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1794111f70 con 0x7f1794103780 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f1799163640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1794112270 con 0x7f1794102dd0 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f179359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1794108300 0x7f179419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f179359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1794108300 0x7f179419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57946/0 (socket says 172.21.15.62:57946) 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f179359e640 1 -- 172.21.15.62:0/1493048516 learned_addr learned my addr 172.21.15.62:0/1493048516 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f179259c640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1794103780 0x7f1794198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.130+0000 7f1792d9d640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1794102dd0 0x7f1794198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f179359e640 1 -- 172.21.15.62:0/1493048516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1794102dd0 msgr2=0x7f1794198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f179359e640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1794102dd0 0x7f1794198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f179359e640 1 -- 172.21.15.62:0/1493048516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1794103780 msgr2=0x7f1794198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f179359e640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1794103780 0x7f1794198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f179359e640 1 -- 172.21.15.62:0/1493048516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179419d540 con 0x7f1794108300 2024-09-11T23:51:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f179359e640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1794108300 0x7f179419ce60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f178c00cce0 tx=0x7f178c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f177bfff640 1 -- 172.21.15.62:0/1493048516 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c00f450 con 0x7f1794108300 2024-09-11T23:51:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f1799163640 1 -- 172.21.15.62:0/1493048516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179419d830 con 0x7f1794108300 2024-09-11T23:51:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f177bfff640 1 -- 172.21.15.62:0/1493048516 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c00fad0 con 0x7f1794108300 2024-09-11T23:51:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f177bfff640 1 -- 172.21.15.62:0/1493048516 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c018b60 con 0x7f1794108300 2024-09-11T23:51:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.131+0000 7f1799163640 1 -- 172.21.15.62:0/1493048516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17941a5110 con 0x7f1794108300 2024-09-11T23:51:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.132+0000 7f1799163640 1 -- 172.21.15.62:0/1493048516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179406a7b0 con 0x7f1794108300 2024-09-11T23:51:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.134+0000 7f177bfff640 1 -- 172.21.15.62:0/1493048516 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f178c00fc80 con 0x7f1794108300 2024-09-11T23:51:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.134+0000 7f177bfff640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f177c0a2c10 0x7f177c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.134+0000 7f1792d9d640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f177c0a2c10 0x7f177c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.134+0000 7f177bfff640 1 -- 172.21.15.62:0/1493048516 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f178c0cf880 con 0x7f1794108300 2024-09-11T23:51:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.135+0000 7f1792d9d640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f177c0a2c10 0x7f177c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f17840058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:09.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.137+0000 7f177bfff640 1 -- 172.21.15.62:0/1493048516 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f178c014030 con 0x7f1794108300 2024-09-11T23:51:09.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:09 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2972901100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:51:09.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.316+0000 7f1799163640 1 -- 172.21.15.62:0/1493048516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f17941031b0 con 0x7f1794108300 2024-09-11T23:51:09.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.317+0000 7f177bfff640 1 -- 172.21.15.62:0/1493048516 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f17941031b0 con 0x7f1794108300 2024-09-11T23:51:09.317 INFO:teuthology.orchestra.run.smithi062.stdout:188978561075 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 -- 172.21.15.62:0/1493048516 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f177c0a2c10 msgr2=0x7f177c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f177c0a2c10 0x7f177c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f17840058d0 comp rx=0 tx=0).stop 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 -- 172.21.15.62:0/1493048516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1794108300 msgr2=0x7f179419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1794108300 0x7f179419ce60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f178c00cce0 tx=0x7f178c00a620 comp rx=0 tx=0).stop 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 -- 172.21.15.62:0/1493048516 shutdown_connections 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f177c0a2c10 0x7f177c0a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1794108300 0x7f179419ce60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1794103780 0x7f1794198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 --2- 172.21.15.62:0/1493048516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1794102dd0 0x7f1794198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 -- 172.21.15.62:0/1493048516 >> 172.21.15.62:0/1493048516 conn(0x7f17940fc7d0 msgr2=0x7f1794105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 -- 172.21.15.62:0/1493048516 shutdown_connections 2024-09-11T23:51:09.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:09.319+0000 7f1799163640 1 -- 172.21.15.62:0/1493048516 wait complete. 2024-09-11T23:51:09.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:09 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2972901100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:51:09.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:09 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2972901100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:51:09.730 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738449 for osd.0 2024-09-11T23:51:09.730 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:10.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:10 smithi120 ceph-mon[35182]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:51:10.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:10 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1493048516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:51:10.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:10 smithi083 ceph-mon[34805]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:51:10.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:10 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1493048516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:51:10.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:10 smithi062 ceph-mon[30958]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:51:10.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:10 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1493048516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:51:10.567 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547220 got 322122547222 for osd.11 2024-09-11T23:51:10.567 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:11.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397547 got 210453397550 for osd.7 2024-09-11T23:51:11.040 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:11.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561072 got 188978561075 for osd.6 2024-09-11T23:51:11.421 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.658+0000 7ff293d0b640 1 -- 172.21.15.62:0/465832094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff28c100a80 msgr2=0x7ff28c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.658+0000 7ff293d0b640 1 --2- 172.21.15.62:0/465832094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff28c100a80 0x7ff28c100ee0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009930 tx=0x7ff27c02a420 comp rx=0 tx=0).stop 2024-09-11T23:51:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.658+0000 7ff293d0b640 1 -- 172.21.15.62:0/465832094 shutdown_connections 2024-09-11T23:51:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.658+0000 7ff293d0b640 1 --2- 172.21.15.62:0/465832094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c101420 0x7ff28c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.658+0000 7ff293d0b640 1 --2- 172.21.15.62:0/465832094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff28c100a80 0x7ff28c100ee0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.658+0000 7ff293d0b640 1 --2- 172.21.15.62:0/465832094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28c106a80 0x7ff28c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.658+0000 7ff293d0b640 1 -- 172.21.15.62:0/465832094 >> 172.21.15.62:0/465832094 conn(0x7ff28c0fc7a0 msgr2=0x7ff28c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.658+0000 7ff293d0b640 1 -- 172.21.15.62:0/465832094 shutdown_connections 2024-09-11T23:51:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.658+0000 7ff293d0b640 1 -- 172.21.15.62:0/465832094 wait complete. 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff293d0b640 1 Processor -- start 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff293d0b640 1 -- start start 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff293d0b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c100a80 0x7ff28c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff293d0b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28c101420 0x7ff28c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff293d0b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff28c106a80 0x7ff28c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff293d0b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c110e80 con 0x7ff28c106a80 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff293d0b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c110d00 con 0x7ff28c100a80 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff293d0b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c111000 con 0x7ff28c101420 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff2922a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff28c106a80 0x7ff28c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff2922a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff28c106a80 0x7ff28c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55318/0 (socket says 172.21.15.62:55318) 2024-09-11T23:51:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff2922a8640 1 -- 172.21.15.62:0/4122317747 learned_addr learned my addr 172.21.15.62:0/4122317747 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.659+0000 7ff291aa7640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c100a80 0x7ff28c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff2912a6640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28c101420 0x7ff28c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff2922a8640 1 -- 172.21.15.62:0/4122317747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28c101420 msgr2=0x7ff28c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff2922a8640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28c101420 0x7ff28c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff2922a8640 1 -- 172.21.15.62:0/4122317747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c100a80 msgr2=0x7ff28c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff2922a8640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c100a80 0x7ff28c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff2922a8640 1 -- 172.21.15.62:0/4122317747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c19d420 con 0x7ff28c106a80 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff2922a8640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff28c106a80 0x7ff28c19cd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff28400eaa0 tx=0x7ff28400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff282ffd640 1 -- 172.21.15.62:0/4122317747 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28400cdb0 con 0x7ff28c106a80 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff282ffd640 1 -- 172.21.15.62:0/4122317747 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff284005020 con 0x7ff28c106a80 2024-09-11T23:51:11.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff293d0b640 1 -- 172.21.15.62:0/4122317747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c19d710 con 0x7ff28c106a80 2024-09-11T23:51:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff282ffd640 1 -- 172.21.15.62:0/4122317747 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2840116b0 con 0x7ff28c106a80 2024-09-11T23:51:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.660+0000 7ff293d0b640 1 -- 172.21.15.62:0/4122317747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c1a4ff0 con 0x7ff28c106a80 2024-09-11T23:51:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.661+0000 7ff293d0b640 1 -- 172.21.15.62:0/4122317747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c100ee0 con 0x7ff28c106a80 2024-09-11T23:51:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.663+0000 7ff282ffd640 1 -- 172.21.15.62:0/4122317747 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff284005560 con 0x7ff28c106a80 2024-09-11T23:51:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.664+0000 7ff282ffd640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2780a2c90 0x7ff2780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.664+0000 7ff291aa7640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2780a2c90 0x7ff2780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.664+0000 7ff282ffd640 1 -- 172.21.15.62:0/4122317747 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ff2840cf4c0 con 0x7ff28c106a80 2024-09-11T23:51:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.664+0000 7ff291aa7640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2780a2c90 0x7ff2780a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff288005fd0 tx=0x7ff28800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.667+0000 7ff282ffd640 1 -- 172.21.15.62:0/4122317747 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff284016030 con 0x7ff28c106a80 2024-09-11T23:51:11.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.859+0000 7ff293d0b640 1 -- 172.21.15.62:0/4122317747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff28c1997d0 con 0x7ff28c106a80 2024-09-11T23:51:11.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.859+0000 7ff282ffd640 1 -- 172.21.15.62:0/4122317747 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff28c1997d0 con 0x7ff28c106a80 2024-09-11T23:51:11.859 INFO:teuthology.orchestra.run.smithi062.stdout:107374182466 2024-09-11T23:51:11.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.861+0000 7ff293d0b640 1 -- 172.21.15.62:0/4122317747 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2780a2c90 msgr2=0x7ff2780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.861+0000 7ff293d0b640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2780a2c90 0x7ff2780a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff288005fd0 tx=0x7ff28800b390 comp rx=0 tx=0).stop 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.861+0000 7ff293d0b640 1 -- 172.21.15.62:0/4122317747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff28c106a80 msgr2=0x7ff28c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.861+0000 7ff293d0b640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff28c106a80 0x7ff28c19cd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff28400eaa0 tx=0x7ff28400ef60 comp rx=0 tx=0).stop 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.862+0000 7ff293d0b640 1 -- 172.21.15.62:0/4122317747 shutdown_connections 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.862+0000 7ff293d0b640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7ff2780a2c90 0x7ff2780a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.862+0000 7ff293d0b640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff28c106a80 0x7ff28c19cd40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.862+0000 7ff293d0b640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28c101420 0x7ff28c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.862+0000 7ff293d0b640 1 --2- 172.21.15.62:0/4122317747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c100a80 0x7ff28c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.862+0000 7ff293d0b640 1 -- 172.21.15.62:0/4122317747 >> 172.21.15.62:0/4122317747 conn(0x7ff28c0fc7a0 msgr2=0x7ff28c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.862+0000 7ff293d0b640 1 -- 172.21.15.62:0/4122317747 shutdown_connections 2024-09-11T23:51:11.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:11.862+0000 7ff293d0b640 1 -- 172.21.15.62:0/4122317747 wait complete. 2024-09-11T23:51:12.288 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:12 smithi062 ceph-mon[30958]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:12.288 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:12 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/4122317747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:51:12.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:12 smithi120 ceph-mon[35182]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:12.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:12 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/4122317747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:51:12.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:12 smithi083 ceph-mon[34805]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:12.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:12 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/4122317747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:51:12.589 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182466 for osd.3 2024-09-11T23:51:12.590 DEBUG:teuthology.parallel:result is None 2024-09-11T23:51:12.590 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:51:12.590 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:51:13.090 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:13.356 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-11T23:51:13.356 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T23:51:13.356 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-11T23:51:13.356 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-11T23:51:13.356 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-11T23:51:13.356 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-11T23:51:13.356 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-11T23:51:13.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:13 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-11T23:51:13.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-11T23:51:13.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T23:51:13.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-11T23:51:13.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-11T23:51:13.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-11T23:51:13.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-11T23:51:13.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-11T23:51:13.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:13 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-11T23:51:13.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-11T23:51:13.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T23:51:13.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-11T23:51:13.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-11T23:51:13.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-11T23:51:13.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-11T23:51:13.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-11T23:51:13.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:13 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-11T23:51:13.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.577+0000 7f95ead5a640 1 -- 172.21.15.62:0/2089914715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95e410c170 msgr2=0x7f95e410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:13.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.577+0000 7f95ead5a640 1 --2- 172.21.15.62:0/2089914715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95e410c170 0x7f95e410e610 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009ca0 tx=0x7f95dc02a950 comp rx=0 tx=0).stop 2024-09-11T23:51:13.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.577+0000 7f95ead5a640 1 -- 172.21.15.62:0/2089914715 shutdown_connections 2024-09-11T23:51:13.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.577+0000 7f95ead5a640 1 --2- 172.21.15.62:0/2089914715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95e410c170 0x7f95e410e610 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.577+0000 7f95ead5a640 1 --2- 172.21.15.62:0/2089914715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e4103660 0x7f95e410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.577+0000 7f95ead5a640 1 --2- 172.21.15.62:0/2089914715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e4102d40 0x7f95e4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.577+0000 7f95ead5a640 1 -- 172.21.15.62:0/2089914715 >> 172.21.15.62:0/2089914715 conn(0x7f95e40fc7a0 msgr2=0x7f95e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.577+0000 7f95ead5a640 1 -- 172.21.15.62:0/2089914715 shutdown_connections 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95ead5a640 1 -- 172.21.15.62:0/2089914715 wait complete. 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95ead5a640 1 Processor -- start 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95ead5a640 1 -- start start 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95ead5a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95e4102d40 0x7f95e41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95ead5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e4103660 0x7f95e4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95ead5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e410c170 0x7f95e4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95ead5a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4111f50 con 0x7f95e4102d40 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95ead5a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4111dd0 con 0x7f95e410c170 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95ead5a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41120d0 con 0x7f95e4103660 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95e8af6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95e4102d40 0x7f95e41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95e8af6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95e4102d40 0x7f95e41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55334/0 (socket says 172.21.15.62:55334) 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.578+0000 7f95e8af6640 1 -- 172.21.15.62:0/640978321 learned_addr learned my addr 172.21.15.62:0/640978321 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e3fff640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e4103660 0x7f95e4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e92f7640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e410c170 0x7f95e4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e8af6640 1 -- 172.21.15.62:0/640978321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e4103660 msgr2=0x7f95e4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e8af6640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e4103660 0x7f95e4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e8af6640 1 -- 172.21.15.62:0/640978321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e410c170 msgr2=0x7f95e4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e8af6640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e410c170 0x7f95e4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e8af6640 1 -- 172.21.15.62:0/640978321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e41a6920 con 0x7f95e4102d40 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e3fff640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e4103660 0x7f95e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e8af6640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95e4102d40 0x7f95e41009a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f95d400d950 tx=0x7f95d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e1ffb640 1 -- 172.21.15.62:0/640978321 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4017030 con 0x7f95e4102d40 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95ead5a640 1 -- 172.21.15.62:0/640978321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e41a6c10 con 0x7f95e4102d40 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e1ffb640 1 -- 172.21.15.62:0/640978321 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d4004020 con 0x7f95e4102d40 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95e1ffb640 1 -- 172.21.15.62:0/640978321 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d40047a0 con 0x7f95e4102d40 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.579+0000 7f95ead5a640 1 -- 172.21.15.62:0/640978321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e41a71a0 con 0x7f95e4102d40 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.580+0000 7f95ead5a640 1 -- 172.21.15.62:0/640978321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e4103120 con 0x7f95e4102d40 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.581+0000 7f95e1ffb640 1 -- 172.21.15.62:0/640978321 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f95d40041d0 con 0x7f95e4102d40 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.582+0000 7f95e1ffb640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f95cc0a2c10 0x7f95cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.582+0000 7f95e3fff640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f95cc0a2c10 0x7f95cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.582+0000 7f95e1ffb640 1 -- 172.21.15.62:0/640978321 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f95d40cec50 con 0x7f95e4102d40 2024-09-11T23:51:13.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.582+0000 7f95e3fff640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f95cc0a2c10 0x7f95cc0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f95d8005d70 tx=0x7f95d8005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:13.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.586+0000 7f95e1ffb640 1 -- 172.21.15.62:0/640978321 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95d409ba60 con 0x7f95e4102d40 2024-09-11T23:51:13.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.766+0000 7f95ead5a640 1 -- 172.21.15.62:0/640978321 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95e4062630 con 0x7f95cc0a2c10 2024-09-11T23:51:13.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.773+0000 7f95e1ffb640 1 -- 172.21.15.62:0/640978321 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258864 (secure 0 0 0) 0x7f95e4062630 con 0x7f95cc0a2c10 2024-09-11T23:51:13.774 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:13.779 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:51:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 -- 172.21.15.62:0/640978321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f95cc0a2c10 msgr2=0x7f95cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f95cc0a2c10 0x7f95cc0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f95d8005d70 tx=0x7f95d8005d00 comp rx=0 tx=0).stop 2024-09-11T23:51:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 -- 172.21.15.62:0/640978321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95e4102d40 msgr2=0x7f95e41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95e4102d40 0x7f95e41009a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f95d400d950 tx=0x7f95d400de10 comp rx=0 tx=0).stop 2024-09-11T23:51:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 -- 172.21.15.62:0/640978321 shutdown_connections 2024-09-11T23:51:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f95cc0a2c10 0x7f95cc0a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e410c170 0x7f95e4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95e4103660 0x7f95e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 --2- 172.21.15.62:0/640978321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95e4102d40 0x7f95e41009a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 -- 172.21.15.62:0/640978321 >> 172.21.15.62:0/640978321 conn(0x7f95e40fc7a0 msgr2=0x7f95e40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 -- 172.21.15.62:0/640978321 shutdown_connections 2024-09-11T23:51:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:13.776+0000 7f95ead5a640 1 -- 172.21.15.62:0/640978321 wait complete. 2024-09-11T23:51:14.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:14.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-11T23:51:14.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T23:51:14.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-11T23:51:14.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-11T23:51:14.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-11T23:51:14.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-11T23:51:14.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-11T23:51:14.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-11T23:51:14.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:51:14.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:14.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:14.372 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:14 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:14.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:14.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-11T23:51:14.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T23:51:14.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-11T23:51:14.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-11T23:51:14.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-11T23:51:14.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-11T23:51:14.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-11T23:51:14.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-11T23:51:14.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:51:14.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:14.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:14.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:14 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:14.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:14.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:14 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:14.509 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-11T23:51:13.102017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655777,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":458,"num_read_kb":348,"num_write":346,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347520,"kb_used_data":10752,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594304,"statfs":{"total":1151940427776,"available":1151584567296,"internally_reserved":0,"allocated":11010048,"data_stored":6998847,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.289022"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830856+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:50:11.830856+0000","last_peered":"2024-09-11T23:50:11.830856+0000","last_clean":"2024-09-11T23:50:11.830856+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:50:11.830856+0000","last_undegraded":"2024-09-11T23:50:11.830856+0000","last_fullsized":"2024-09-11T23:50:11.830856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880987+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:50:11.880987+0000","last_peered":"2024-09-11T23:50:11.880987+0000","last_clean":"2024-09-11T23:50:11.880987+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:50:11.880987+0000","last_undegraded":"2024-09-11T23:50:11.880987+0000","last_fullsized":"2024-09-11T23:50:11.880987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880650+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:50:11.880650+0000","last_peered":"2024-09-11T23:50:11.880650+0000","last_clean":"2024-09-11T23:50:11.880650+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:50:11.880650+0000","last_undegraded":"2024-09-11T23:50:11.880650+0000","last_fullsized":"2024-09-11T23:50:11.880650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878400+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:50:11.878400+0000","last_peered":"2024-09-11T23:50:11.878400+0000","last_clean":"2024-09-11T23:50:11.878400+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:50:11.878400+0000","last_undegraded":"2024-09-11T23:50:11.878400+0000","last_fullsized":"2024-09-11T23:50:11.878400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876576+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:50:11.876576+0000","last_peered":"2024-09-11T23:50:11.876576+0000","last_clean":"2024-09-11T23:50:11.876576+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:50:11.876576+0000","last_undegraded":"2024-09-11T23:50:11.876576+0000","last_fullsized":"2024-09-11T23:50:11.876576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.828168+0000","last_change":"2024-09-11T23:50:10.842560+0000","last_active":"2024-09-11T23:50:11.828168+0000","last_peered":"2024-09-11T23:50:11.828168+0000","last_clean":"2024-09-11T23:50:11.828168+0000","last_became_active":"2024-09-11T23:50:10.842259+0000","last_became_peered":"2024-09-11T23:50:10.842259+0000","last_unstale":"2024-09-11T23:50:11.828168+0000","last_undegraded":"2024-09-11T23:50:11.828168+0000","last_fullsized":"2024-09-11T23:50:11.828168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:36.538402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881133+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:50:11.881133+0000","last_peered":"2024-09-11T23:50:11.881133+0000","last_clean":"2024-09-11T23:50:11.881133+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:50:11.881133+0000","last_undegraded":"2024-09-11T23:50:11.881133+0000","last_fullsized":"2024-09-11T23:50:11.881133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877161+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:50:11.877161+0000","last_peered":"2024-09-11T23:50:11.877161+0000","last_clean":"2024-09-11T23:50:11.877161+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:50:11.877161+0000","last_undegraded":"2024-09-11T23:50:11.877161+0000","last_fullsized":"2024-09-11T23:50:11.877161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878788+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:50:11.878788+0000","last_peered":"2024-09-11T23:50:11.878788+0000","last_clean":"2024-09-11T23:50:11.878788+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:50:11.878788+0000","last_undegraded":"2024-09-11T23:50:11.878788+0000","last_fullsized":"2024-09-11T23:50:11.878788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.828525+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:50:11.828525+0000","last_peered":"2024-09-11T23:50:11.828525+0000","last_clean":"2024-09-11T23:50:11.828525+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:50:11.828525+0000","last_undegraded":"2024-09-11T23:50:11.828525+0000","last_fullsized":"2024-09-11T23:50:11.828525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881588+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:50:11.881588+0000","last_peered":"2024-09-11T23:50:11.881588+0000","last_clean":"2024-09-11T23:50:11.881588+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:50:11.881588+0000","last_undegraded":"2024-09-11T23:50:11.881588+0000","last_fullsized":"2024-09-11T23:50:11.881588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878548+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:50:11.878548+0000","last_peered":"2024-09-11T23:50:11.878548+0000","last_clean":"2024-09-11T23:50:11.878548+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:50:11.878548+0000","last_undegraded":"2024-09-11T23:50:11.878548+0000","last_fullsized":"2024-09-11T23:50:11.878548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827687+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:50:11.827687+0000","last_peered":"2024-09-11T23:50:11.827687+0000","last_clean":"2024-09-11T23:50:11.827687+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:50:11.827687+0000","last_undegraded":"2024-09-11T23:50:11.827687+0000","last_fullsized":"2024-09-11T23:50:11.827687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880393+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:50:11.880393+0000","last_peered":"2024-09-11T23:50:11.880393+0000","last_clean":"2024-09-11T23:50:11.880393+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:50:11.880393+0000","last_undegraded":"2024-09-11T23:50:11.880393+0000","last_fullsized":"2024-09-11T23:50:11.880393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880060+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:50:11.880060+0000","last_peered":"2024-09-11T23:50:11.880060+0000","last_clean":"2024-09-11T23:50:11.880060+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:50:11.880060+0000","last_undegraded":"2024-09-11T23:50:11.880060+0000","last_fullsized":"2024-09-11T23:50:11.880060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877177+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:50:11.877177+0000","last_peered":"2024-09-11T23:50:11.877177+0000","last_clean":"2024-09-11T23:50:11.877177+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:50:11.877177+0000","last_undegraded":"2024-09-11T23:50:11.877177+0000","last_fullsized":"2024-09-11T23:50:11.877177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830826+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:50:11.830826+0000","last_peered":"2024-09-11T23:50:11.830826+0000","last_clean":"2024-09-11T23:50:11.830826+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:50:11.830826+0000","last_undegraded":"2024-09-11T23:50:11.830826+0000","last_fullsized":"2024-09-11T23:50:11.830826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876998+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:50:11.876998+0000","last_peered":"2024-09-11T23:50:11.876998+0000","last_clean":"2024-09-11T23:50:11.876998+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:50:11.876998+0000","last_undegraded":"2024-09-11T23:50:11.876998+0000","last_fullsized":"2024-09-11T23:50:11.876998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876734+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:50:11.876734+0000","last_peered":"2024-09-11T23:50:11.876734+0000","last_clean":"2024-09-11T23:50:11.876734+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:50:11.876734+0000","last_undegraded":"2024-09-11T23:50:11.876734+0000","last_fullsized":"2024-09-11T23:50:11.876734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830673+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:50:11.830673+0000","last_peered":"2024-09-11T23:50:11.830673+0000","last_clean":"2024-09-11T23:50:11.830673+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:50:11.830673+0000","last_undegraded":"2024-09-11T23:50:11.830673+0000","last_fullsized":"2024-09-11T23:50:11.830673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827752+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:50:11.827752+0000","last_peered":"2024-09-11T23:50:11.827752+0000","last_clean":"2024-09-11T23:50:11.827752+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:50:11.827752+0000","last_undegraded":"2024-09-11T23:50:11.827752+0000","last_fullsized":"2024-09-11T23:50:11.827752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877072+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:50:11.877072+0000","last_peered":"2024-09-11T23:50:11.877072+0000","last_clean":"2024-09-11T23:50:11.877072+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:50:11.877072+0000","last_undegraded":"2024-09-11T23:50:11.877072+0000","last_fullsized":"2024-09-11T23:50:11.877072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878562+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:50:11.878562+0000","last_peered":"2024-09-11T23:50:11.878562+0000","last_clean":"2024-09-11T23:50:11.878562+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:50:11.878562+0000","last_undegraded":"2024-09-11T23:50:11.878562+0000","last_fullsized":"2024-09-11T23:50:11.878562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882045+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:50:11.882045+0000","last_peered":"2024-09-11T23:50:11.882045+0000","last_clean":"2024-09-11T23:50:11.882045+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:50:11.882045+0000","last_undegraded":"2024-09-11T23:50:11.882045+0000","last_fullsized":"2024-09-11T23:50:11.882045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880422+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:50:11.880422+0000","last_peered":"2024-09-11T23:50:11.880422+0000","last_clean":"2024-09-11T23:50:11.880422+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:50:11.880422+0000","last_undegraded":"2024-09-11T23:50:11.880422+0000","last_fullsized":"2024-09-11T23:50:11.880422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877065+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:50:11.877065+0000","last_peered":"2024-09-11T23:50:11.877065+0000","last_clean":"2024-09-11T23:50:11.877065+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:50:11.877065+0000","last_undegraded":"2024-09-11T23:50:11.877065+0000","last_fullsized":"2024-09-11T23:50:11.877065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830611+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:50:11.830611+0000","last_peered":"2024-09-11T23:50:11.830611+0000","last_clean":"2024-09-11T23:50:11.830611+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:50:11.830611+0000","last_undegraded":"2024-09-11T23:50:11.830611+0000","last_fullsized":"2024-09-11T23:50:11.830611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827672+0000","last_change":"2024-09-11T23:50:10.845202+0000","last_active":"2024-09-11T23:50:11.827672+0000","last_peered":"2024-09-11T23:50:11.827672+0000","last_clean":"2024-09-11T23:50:11.827672+0000","last_became_active":"2024-09-11T23:50:10.844892+0000","last_became_peered":"2024-09-11T23:50:10.844892+0000","last_unstale":"2024-09-11T23:50:11.827672+0000","last_undegraded":"2024-09-11T23:50:11.827672+0000","last_fullsized":"2024-09-11T23:50:11.827672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:09.744773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880722+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:50:11.880722+0000","last_peered":"2024-09-11T23:50:11.880722+0000","last_clean":"2024-09-11T23:50:11.880722+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:50:11.880722+0000","last_undegraded":"2024-09-11T23:50:11.880722+0000","last_fullsized":"2024-09-11T23:50:11.880722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876440+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:50:11.876440+0000","last_peered":"2024-09-11T23:50:11.876440+0000","last_clean":"2024-09-11T23:50:11.876440+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:50:11.876440+0000","last_undegraded":"2024-09-11T23:50:11.876440+0000","last_fullsized":"2024-09-11T23:50:11.876440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877202+0000","last_change":"2024-09-11T23:50:10.843592+0000","last_active":"2024-09-11T23:50:11.877202+0000","last_peered":"2024-09-11T23:50:11.877202+0000","last_clean":"2024-09-11T23:50:11.877202+0000","last_became_active":"2024-09-11T23:50:10.843259+0000","last_became_peered":"2024-09-11T23:50:10.843259+0000","last_unstale":"2024-09-11T23:50:11.877202+0000","last_undegraded":"2024-09-11T23:50:11.877202+0000","last_fullsized":"2024-09-11T23:50:11.877202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:05.557977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830450+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:50:11.830450+0000","last_peered":"2024-09-11T23:50:11.830450+0000","last_clean":"2024-09-11T23:50:11.830450+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:50:11.830450+0000","last_undegraded":"2024-09-11T23:50:11.830450+0000","last_fullsized":"2024-09-11T23:50:11.830450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876539+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:50:11.876539+0000","last_peered":"2024-09-11T23:50:11.876539+0000","last_clean":"2024-09-11T23:50:11.876539+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:50:11.876539+0000","last_undegraded":"2024-09-11T23:50:11.876539+0000","last_fullsized":"2024-09-11T23:50:11.876539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:51:08.938762+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:51:08.938762+0000","last_peered":"2024-09-11T23:51:08.938762+0000","last_clean":"2024-09-11T23:51:08.938762+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:51:08.938762+0000","last_undegraded":"2024-09-11T23:51:08.938762+0000","last_fullsized":"2024-09-11T23:51:08.938762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880098+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:50:11.880098+0000","last_peered":"2024-09-11T23:50:11.880098+0000","last_clean":"2024-09-11T23:50:11.880098+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:50:11.880098+0000","last_undegraded":"2024-09-11T23:50:11.880098+0000","last_fullsized":"2024-09-11T23:50:11.880098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.866074+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:50:19.866074+0000","last_peered":"2024-09-11T23:50:19.866074+0000","last_clean":"2024-09-11T23:50:19.866074+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:50:19.866074+0000","last_undegraded":"2024-09-11T23:50:19.866074+0000","last_fullsized":"2024-09-11T23:50:19.866074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.866815+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:50:19.866815+0000","last_peered":"2024-09-11T23:50:19.866815+0000","last_clean":"2024-09-11T23:50:19.866815+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:50:19.866815+0000","last_undegraded":"2024-09-11T23:50:19.866815+0000","last_fullsized":"2024-09-11T23:50:19.866815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877320+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:50:11.877320+0000","last_peered":"2024-09-11T23:50:11.877320+0000","last_clean":"2024-09-11T23:50:11.877320+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:50:11.877320+0000","last_undegraded":"2024-09-11T23:50:11.877320+0000","last_fullsized":"2024-09-11T23:50:11.877320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882426+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:50:11.882426+0000","last_peered":"2024-09-11T23:50:11.882426+0000","last_clean":"2024-09-11T23:50:11.882426+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:50:11.882426+0000","last_undegraded":"2024-09-11T23:50:11.882426+0000","last_fullsized":"2024-09-11T23:50:11.882426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877143+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:50:11.877143+0000","last_peered":"2024-09-11T23:50:11.877143+0000","last_clean":"2024-09-11T23:50:11.877143+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:50:11.877143+0000","last_undegraded":"2024-09-11T23:50:11.877143+0000","last_fullsized":"2024-09-11T23:50:11.877143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":69,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:51:06.935503+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:51:06.935503+0000","last_peered":"2024-09-11T23:51:06.935503+0000","last_clean":"2024-09-11T23:51:06.935503+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:51:06.935503+0000","last_undegraded":"2024-09-11T23:51:06.935503+0000","last_fullsized":"2024-09-11T23:51:06.935503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878635+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:50:11.878635+0000","last_peered":"2024-09-11T23:50:11.878635+0000","last_clean":"2024-09-11T23:50:11.878635+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:50:11.878635+0000","last_undegraded":"2024-09-11T23:50:11.878635+0000","last_fullsized":"2024-09-11T23:50:11.878635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862666+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:50:18.862666+0000","last_peered":"2024-09-11T23:50:18.862666+0000","last_clean":"2024-09-11T23:50:18.862666+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:50:18.862666+0000","last_undegraded":"2024-09-11T23:50:18.862666+0000","last_fullsized":"2024-09-11T23:50:18.862666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881420+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:50:11.881420+0000","last_peered":"2024-09-11T23:50:11.881420+0000","last_clean":"2024-09-11T23:50:11.881420+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:50:11.881420+0000","last_undegraded":"2024-09-11T23:50:11.881420+0000","last_fullsized":"2024-09-11T23:50:11.881420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861345+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:50:18.861345+0000","last_peered":"2024-09-11T23:50:18.861345+0000","last_clean":"2024-09-11T23:50:18.861345+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:50:18.861345+0000","last_undegraded":"2024-09-11T23:50:18.861345+0000","last_fullsized":"2024-09-11T23:50:18.861345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877016+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:50:11.877016+0000","last_peered":"2024-09-11T23:50:11.877016+0000","last_clean":"2024-09-11T23:50:11.877016+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:50:11.877016+0000","last_undegraded":"2024-09-11T23:50:11.877016+0000","last_fullsized":"2024-09-11T23:50:11.877016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":152,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:51:07.940841+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:51:07.940841+0000","last_peered":"2024-09-11T23:51:07.940841+0000","last_clean":"2024-09-11T23:51:07.940841+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:51:07.940841+0000","last_undegraded":"2024-09-11T23:51:07.940841+0000","last_fullsized":"2024-09-11T23:51:07.940841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":111,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877925+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:50:11.877925+0000","last_peered":"2024-09-11T23:50:11.877925+0000","last_clean":"2024-09-11T23:50:11.877925+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:50:11.877925+0000","last_undegraded":"2024-09-11T23:50:11.877925+0000","last_fullsized":"2024-09-11T23:50:11.877925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.866530+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:50:19.866530+0000","last_peered":"2024-09-11T23:50:19.866530+0000","last_clean":"2024-09-11T23:50:19.866530+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:50:19.866530+0000","last_undegraded":"2024-09-11T23:50:19.866530+0000","last_fullsized":"2024-09-11T23:50:19.866530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827276+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:50:11.827276+0000","last_peered":"2024-09-11T23:50:11.827276+0000","last_clean":"2024-09-11T23:50:11.827276+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:50:11.827276+0000","last_undegraded":"2024-09-11T23:50:11.827276+0000","last_fullsized":"2024-09-11T23:50:11.827276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862298+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:50:18.862298+0000","last_peered":"2024-09-11T23:50:18.862298+0000","last_clean":"2024-09-11T23:50:18.862298+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:50:18.862298+0000","last_undegraded":"2024-09-11T23:50:18.862298+0000","last_fullsized":"2024-09-11T23:50:18.862298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881417+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:50:11.881417+0000","last_peered":"2024-09-11T23:50:11.881417+0000","last_clean":"2024-09-11T23:50:11.881417+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:50:11.881417+0000","last_undegraded":"2024-09-11T23:50:11.881417+0000","last_fullsized":"2024-09-11T23:50:11.881417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:51:07.937139+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:51:07.937139+0000","last_peered":"2024-09-11T23:51:07.937139+0000","last_clean":"2024-09-11T23:51:07.937139+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:51:07.937139+0000","last_undegraded":"2024-09-11T23:51:07.937139+0000","last_fullsized":"2024-09-11T23:51:07.937139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880994+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:50:11.880994+0000","last_peered":"2024-09-11T23:50:11.880994+0000","last_clean":"2024-09-11T23:50:11.880994+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:50:11.880994+0000","last_undegraded":"2024-09-11T23:50:11.880994+0000","last_fullsized":"2024-09-11T23:50:11.880994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868544+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:50:19.868544+0000","last_peered":"2024-09-11T23:50:19.868544+0000","last_clean":"2024-09-11T23:50:19.868544+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:50:19.868544+0000","last_undegraded":"2024-09-11T23:50:19.868544+0000","last_fullsized":"2024-09-11T23:50:19.868544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879800+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:50:11.879800+0000","last_peered":"2024-09-11T23:50:11.879800+0000","last_clean":"2024-09-11T23:50:11.879800+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:50:11.879800+0000","last_undegraded":"2024-09-11T23:50:11.879800+0000","last_fullsized":"2024-09-11T23:50:11.879800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:20.871408+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:50:20.871408+0000","last_peered":"2024-09-11T23:50:20.871408+0000","last_clean":"2024-09-11T23:50:20.871408+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:50:20.871408+0000","last_undegraded":"2024-09-11T23:50:20.871408+0000","last_fullsized":"2024-09-11T23:50:20.871408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830599+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:50:11.830599+0000","last_peered":"2024-09-11T23:50:11.830599+0000","last_clean":"2024-09-11T23:50:11.830599+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:50:11.830599+0000","last_undegraded":"2024-09-11T23:50:11.830599+0000","last_fullsized":"2024-09-11T23:50:11.830599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877724+0000","last_change":"2024-09-11T23:50:08.766524+0000","last_active":"2024-09-11T23:50:11.877724+0000","last_peered":"2024-09-11T23:50:11.877724+0000","last_clean":"2024-09-11T23:50:11.877724+0000","last_became_active":"2024-09-11T23:50:08.766120+0000","last_became_peered":"2024-09-11T23:50:08.766120+0000","last_unstale":"2024-09-11T23:50:11.877724+0000","last_undegraded":"2024-09-11T23:50:11.877724+0000","last_fullsized":"2024-09-11T23:50:11.877724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:51:49.261871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881682+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:50:11.881682+0000","last_peered":"2024-09-11T23:50:11.881682+0000","last_clean":"2024-09-11T23:50:11.881682+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:50:11.881682+0000","last_undegraded":"2024-09-11T23:50:11.881682+0000","last_fullsized":"2024-09-11T23:50:11.881682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.886203+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:50:11.886203+0000","last_peered":"2024-09-11T23:50:11.886203+0000","last_clean":"2024-09-11T23:50:11.886203+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:50:11.886203+0000","last_undegraded":"2024-09-11T23:50:11.886203+0000","last_fullsized":"2024-09-11T23:50:11.886203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881405+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:50:11.881405+0000","last_peered":"2024-09-11T23:50:11.881405+0000","last_clean":"2024-09-11T23:50:11.881405+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:50:11.881405+0000","last_undegraded":"2024-09-11T23:50:11.881405+0000","last_fullsized":"2024-09-11T23:50:11.881405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.867894+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:50:19.867894+0000","last_peered":"2024-09-11T23:50:19.867894+0000","last_clean":"2024-09-11T23:50:19.867894+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:50:19.867894+0000","last_undegraded":"2024-09-11T23:50:19.867894+0000","last_fullsized":"2024-09-11T23:50:19.867894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881586+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:50:11.881586+0000","last_peered":"2024-09-11T23:50:11.881586+0000","last_clean":"2024-09-11T23:50:11.881586+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:50:11.881586+0000","last_undegraded":"2024-09-11T23:50:11.881586+0000","last_fullsized":"2024-09-11T23:50:11.881586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881179+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:50:11.881179+0000","last_peered":"2024-09-11T23:50:11.881179+0000","last_clean":"2024-09-11T23:50:11.881179+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:50:11.881179+0000","last_undegraded":"2024-09-11T23:50:11.881179+0000","last_fullsized":"2024-09-11T23:50:11.881179+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:20.871596+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:50:20.871596+0000","last_peered":"2024-09-11T23:50:20.871596+0000","last_clean":"2024-09-11T23:50:20.871596+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:50:20.871596+0000","last_undegraded":"2024-09-11T23:50:20.871596+0000","last_fullsized":"2024-09-11T23:50:20.871596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879540+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:50:11.879540+0000","last_peered":"2024-09-11T23:50:11.879540+0000","last_clean":"2024-09-11T23:50:11.879540+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:50:11.879540+0000","last_undegraded":"2024-09-11T23:50:11.879540+0000","last_fullsized":"2024-09-11T23:50:11.879540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868224+0000","last_change":"2024-09-11T23:50:08.767568+0000","last_active":"2024-09-11T23:50:19.868224+0000","last_peered":"2024-09-11T23:50:19.868224+0000","last_clean":"2024-09-11T23:50:19.868224+0000","last_became_active":"2024-09-11T23:50:08.767217+0000","last_became_peered":"2024-09-11T23:50:08.767217+0000","last_unstale":"2024-09-11T23:50:19.868224+0000","last_undegraded":"2024-09-11T23:50:19.868224+0000","last_fullsized":"2024-09-11T23:50:19.868224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:12.102018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827258+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:50:11.827258+0000","last_peered":"2024-09-11T23:50:11.827258+0000","last_clean":"2024-09-11T23:50:11.827258+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:50:11.827258+0000","last_undegraded":"2024-09-11T23:50:11.827258+0000","last_fullsized":"2024-09-11T23:50:11.827258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":73,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:51:09.939270+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:51:09.939270+0000","last_peered":"2024-09-11T23:51:09.939270+0000","last_clean":"2024-09-11T23:51:09.939270+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:51:09.939270+0000","last_undegraded":"2024-09-11T23:51:09.939270+0000","last_fullsized":"2024-09-11T23:51:09.939270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882510+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:50:11.882510+0000","last_peered":"2024-09-11T23:50:11.882510+0000","last_clean":"2024-09-11T23:50:11.882510+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:50:11.882510+0000","last_undegraded":"2024-09-11T23:50:11.882510+0000","last_fullsized":"2024-09-11T23:50:11.882510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881397+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:50:11.881397+0000","last_peered":"2024-09-11T23:50:11.881397+0000","last_clean":"2024-09-11T23:50:11.881397+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:50:11.881397+0000","last_undegraded":"2024-09-11T23:50:11.881397+0000","last_fullsized":"2024-09-11T23:50:11.881397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.828518+0000","last_change":"2024-09-11T23:50:10.846416+0000","last_active":"2024-09-11T23:50:11.828518+0000","last_peered":"2024-09-11T23:50:11.828518+0000","last_clean":"2024-09-11T23:50:11.828518+0000","last_became_active":"2024-09-11T23:50:10.843961+0000","last_became_peered":"2024-09-11T23:50:10.843961+0000","last_unstale":"2024-09-11T23:50:11.828518+0000","last_undegraded":"2024-09-11T23:50:11.828518+0000","last_fullsized":"2024-09-11T23:50:11.828518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:46.772458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877175+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:50:11.877175+0000","last_peered":"2024-09-11T23:50:11.877175+0000","last_clean":"2024-09-11T23:50:11.877175+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:50:11.877175+0000","last_undegraded":"2024-09-11T23:50:11.877175+0000","last_fullsized":"2024-09-11T23:50:11.877175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:13.236129+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:50:13.236129+0000","last_peered":"2024-09-11T23:50:13.236129+0000","last_clean":"2024-09-11T23:50:13.236129+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:50:13.236129+0000","last_undegraded":"2024-09-11T23:50:13.236129+0000","last_fullsized":"2024-09-11T23:50:13.236129+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861294+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:50:18.861294+0000","last_peered":"2024-09-11T23:50:18.861294+0000","last_clean":"2024-09-11T23:50:18.861294+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:50:18.861294+0000","last_undegraded":"2024-09-11T23:50:18.861294+0000","last_fullsized":"2024-09-11T23:50:18.861294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827648+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:50:11.827648+0000","last_peered":"2024-09-11T23:50:11.827648+0000","last_clean":"2024-09-11T23:50:11.827648+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:50:11.827648+0000","last_undegraded":"2024-09-11T23:50:11.827648+0000","last_fullsized":"2024-09-11T23:50:11.827648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'11","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:35.878259+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:50:35.878259+0000","last_peered":"2024-09-11T23:50:35.878259+0000","last_clean":"2024-09-11T23:50:35.878259+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:50:35.878259+0000","last_undegraded":"2024-09-11T23:50:35.878259+0000","last_fullsized":"2024-09-11T23:50:35.878259+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880484+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:50:11.880484+0000","last_peered":"2024-09-11T23:50:11.880484+0000","last_clean":"2024-09-11T23:50:11.880484+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:50:11.880484+0000","last_undegraded":"2024-09-11T23:50:11.880484+0000","last_fullsized":"2024-09-11T23:50:11.880484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861080+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:50:18.861080+0000","last_peered":"2024-09-11T23:50:18.861080+0000","last_clean":"2024-09-11T23:50:18.861080+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:50:18.861080+0000","last_undegraded":"2024-09-11T23:50:18.861080+0000","last_fullsized":"2024-09-11T23:50:18.861080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881017+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:50:11.881017+0000","last_peered":"2024-09-11T23:50:11.881017+0000","last_clean":"2024-09-11T23:50:11.881017+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:50:11.881017+0000","last_undegraded":"2024-09-11T23:50:11.881017+0000","last_fullsized":"2024-09-11T23:50:11.881017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862125+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:50:18.862125+0000","last_peered":"2024-09-11T23:50:18.862125+0000","last_clean":"2024-09-11T23:50:18.862125+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:50:18.862125+0000","last_undegraded":"2024-09-11T23:50:18.862125+0000","last_fullsized":"2024-09-11T23:50:18.862125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876349+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:50:11.876349+0000","last_peered":"2024-09-11T23:50:11.876349+0000","last_clean":"2024-09-11T23:50:11.876349+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:50:11.876349+0000","last_undegraded":"2024-09-11T23:50:11.876349+0000","last_fullsized":"2024-09-11T23:50:11.876349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861563+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:50:18.861563+0000","last_peered":"2024-09-11T23:50:18.861563+0000","last_clean":"2024-09-11T23:50:18.861563+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:50:18.861563+0000","last_undegraded":"2024-09-11T23:50:18.861563+0000","last_fullsized":"2024-09-11T23:50:18.861563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830666+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:50:11.830666+0000","last_peered":"2024-09-11T23:50:11.830666+0000","last_clean":"2024-09-11T23:50:11.830666+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:50:11.830666+0000","last_undegraded":"2024-09-11T23:50:11.830666+0000","last_fullsized":"2024-09-11T23:50:11.830666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876431+0000","last_change":"2024-09-11T23:50:08.766530+0000","last_active":"2024-09-11T23:50:11.876431+0000","last_peered":"2024-09-11T23:50:11.876431+0000","last_clean":"2024-09-11T23:50:11.876431+0000","last_became_active":"2024-09-11T23:50:08.766014+0000","last_became_peered":"2024-09-11T23:50:08.766014+0000","last_unstale":"2024-09-11T23:50:11.876431+0000","last_undegraded":"2024-09-11T23:50:11.876431+0000","last_fullsized":"2024-09-11T23:50:11.876431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:38.056709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878313+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:50:11.878313+0000","last_peered":"2024-09-11T23:50:11.878313+0000","last_clean":"2024-09-11T23:50:11.878313+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:50:11.878313+0000","last_undegraded":"2024-09-11T23:50:11.878313+0000","last_fullsized":"2024-09-11T23:50:11.878313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.863430+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:50:18.863430+0000","last_peered":"2024-09-11T23:50:18.863430+0000","last_clean":"2024-09-11T23:50:18.863430+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:50:18.863430+0000","last_undegraded":"2024-09-11T23:50:18.863430+0000","last_fullsized":"2024-09-11T23:50:18.863430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881169+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:50:11.881169+0000","last_peered":"2024-09-11T23:50:11.881169+0000","last_clean":"2024-09-11T23:50:11.881169+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:50:11.881169+0000","last_undegraded":"2024-09-11T23:50:11.881169+0000","last_fullsized":"2024-09-11T23:50:11.881169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881434+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:50:11.881434+0000","last_peered":"2024-09-11T23:50:11.881434+0000","last_clean":"2024-09-11T23:50:11.881434+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:50:11.881434+0000","last_undegraded":"2024-09-11T23:50:11.881434+0000","last_fullsized":"2024-09-11T23:50:11.881434+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.885419+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:50:11.885419+0000","last_peered":"2024-09-11T23:50:11.885419+0000","last_clean":"2024-09-11T23:50:11.885419+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:50:11.885419+0000","last_undegraded":"2024-09-11T23:50:11.885419+0000","last_fullsized":"2024-09-11T23:50:11.885419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876830+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:50:11.876830+0000","last_peered":"2024-09-11T23:50:11.876830+0000","last_clean":"2024-09-11T23:50:11.876830+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:50:11.876830+0000","last_undegraded":"2024-09-11T23:50:11.876830+0000","last_fullsized":"2024-09-11T23:50:11.876830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.867614+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:50:19.867614+0000","last_peered":"2024-09-11T23:50:19.867614+0000","last_clean":"2024-09-11T23:50:19.867614+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:50:19.867614+0000","last_undegraded":"2024-09-11T23:50:19.867614+0000","last_fullsized":"2024-09-11T23:50:19.867614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881551+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:50:11.881551+0000","last_peered":"2024-09-11T23:50:11.881551+0000","last_clean":"2024-09-11T23:50:11.881551+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:50:11.881551+0000","last_undegraded":"2024-09-11T23:50:11.881551+0000","last_fullsized":"2024-09-11T23:50:11.881551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:12.424274+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:50:12.424274+0000","last_peered":"2024-09-11T23:50:12.424274+0000","last_clean":"2024-09-11T23:50:12.424274+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:50:12.424274+0000","last_undegraded":"2024-09-11T23:50:12.424274+0000","last_fullsized":"2024-09-11T23:50:12.424274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879978+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:50:11.879978+0000","last_peered":"2024-09-11T23:50:11.879978+0000","last_clean":"2024-09-11T23:50:11.879978+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:50:11.879978+0000","last_undegraded":"2024-09-11T23:50:11.879978+0000","last_fullsized":"2024-09-11T23:50:11.879978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868424+0000","last_change":"2024-09-11T23:50:08.764272+0000","last_active":"2024-09-11T23:50:19.868424+0000","last_peered":"2024-09-11T23:50:19.868424+0000","last_clean":"2024-09-11T23:50:19.868424+0000","last_became_active":"2024-09-11T23:50:08.763978+0000","last_became_peered":"2024-09-11T23:50:08.763978+0000","last_unstale":"2024-09-11T23:50:19.868424+0000","last_undegraded":"2024-09-11T23:50:19.868424+0000","last_fullsized":"2024-09-11T23:50:19.868424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:50.257629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65175,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":322,"num_read_kb":229,"num_write":163,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281926,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":419875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1072921,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939684,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1011963,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561076,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":419500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":1011254,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":420330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":419811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":481815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":480465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":420771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:51:14.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:51:14.783 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:15 smithi062 ceph-mon[30958]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:51:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:15 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:51:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:15 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:51:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:15 smithi062 ceph-mon[30958]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:15.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.253+0000 7f4026fc6640 1 -- 172.21.15.62:0/550959742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4020101410 msgr2=0x7f402010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:15.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.253+0000 7f4026fc6640 1 --2- 172.21.15.62:0/550959742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4020101410 0x7f402010d940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4018009cd0 tx=0x7f401802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.253+0000 7f4026fc6640 1 -- 172.21.15.62:0/550959742 shutdown_connections 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.253+0000 7f4026fc6640 1 --2- 172.21.15.62:0/550959742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4020101410 0x7f402010d940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.253+0000 7f4026fc6640 1 --2- 172.21.15.62:0/550959742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4020100a70 0x7f4020100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.253+0000 7f4026fc6640 1 --2- 172.21.15.62:0/550959742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4020106a70 0x7f4020106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.253+0000 7f4026fc6640 1 -- 172.21.15.62:0/550959742 >> 172.21.15.62:0/550959742 conn(0x7f40200fc790 msgr2=0x7f40200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.253+0000 7f4026fc6640 1 -- 172.21.15.62:0/550959742 shutdown_connections 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.253+0000 7f4026fc6640 1 -- 172.21.15.62:0/550959742 wait complete. 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4026fc6640 1 Processor -- start 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4026fc6640 1 -- start start 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4026fc6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4020100a70 0x7f40201970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4026fc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4020101410 0x7f40201975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4026fc6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4020106a70 0x7f4020197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4026fc6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4020110ec0 con 0x7f4020100a70 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4026fc6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4020110d40 con 0x7f4020106a70 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4026fc6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4020111040 con 0x7f4020101410 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4024d62640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4020100a70 0x7f40201970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4024d62640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4020100a70 0x7f40201970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55356/0 (socket says 172.21.15.62:55356) 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4024d62640 1 -- 172.21.15.62:0/3896285155 learned_addr learned my addr 172.21.15.62:0/3896285155 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4025563640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4020106a70 0x7f4020197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4017fff640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4020101410 0x7f40201975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4024d62640 1 -- 172.21.15.62:0/3896285155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4020101410 msgr2=0x7f40201975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4024d62640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4020101410 0x7f40201975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4024d62640 1 -- 172.21.15.62:0/3896285155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4020106a70 msgr2=0x7f4020197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4024d62640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4020106a70 0x7f4020197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.254+0000 7f4024d62640 1 -- 172.21.15.62:0/3896285155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40201a6910 con 0x7f4020100a70 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.255+0000 7f4024d62640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4020100a70 0x7f40201970a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f401c00c970 tx=0x7f401c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.255+0000 7f4015ffb640 1 -- 172.21.15.62:0/3896285155 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c007bb0 con 0x7f4020100a70 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.255+0000 7f4015ffb640 1 -- 172.21.15.62:0/3896285155 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f401c007d50 con 0x7f4020100a70 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.255+0000 7f4026fc6640 1 -- 172.21.15.62:0/3896285155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40201a6c00 con 0x7f4020100a70 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.255+0000 7f4015ffb640 1 -- 172.21.15.62:0/3896285155 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c00f450 con 0x7f4020100a70 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.255+0000 7f4026fc6640 1 -- 172.21.15.62:0/3896285155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40201a7140 con 0x7f4020100a70 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.256+0000 7f4026fc6640 1 -- 172.21.15.62:0/3896285155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4020100ed0 con 0x7f4020100a70 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.257+0000 7f4015ffb640 1 -- 172.21.15.62:0/3896285155 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f401c005020 con 0x7f4020100a70 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.258+0000 7f4015ffb640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f40080a2c90 0x7f40080a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.258+0000 7f4017fff640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f40080a2c90 0x7f40080a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.258+0000 7f4015ffb640 1 -- 172.21.15.62:0/3896285155 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f401c0ceee0 con 0x7f4020100a70 2024-09-11T23:51:15.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.258+0000 7f4017fff640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f40080a2c90 0x7f40080a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4010005fd0 tx=0x7f4010005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:15.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.262+0000 7f4015ffb640 1 -- 172.21.15.62:0/3896285155 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f401c014030 con 0x7f4020100a70 2024-09-11T23:51:15.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:15 smithi120 ceph-mon[35182]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:51:15.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:15 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:51:15.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:15 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:51:15.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:15 smithi120 ceph-mon[35182]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:15.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:15 smithi083 ceph-mon[34805]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:51:15.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:15 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:51:15.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:15 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:51:15.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:15 smithi083 ceph-mon[34805]: from='mgr.14150 172.21.15.62:0/672488100' entity='mgr.x' 2024-09-11T23:51:15.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.433+0000 7f4026fc6640 1 -- 172.21.15.62:0/3896285155 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4020062630 con 0x7f40080a2c90 2024-09-11T23:51:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.440+0000 7f4015ffb640 1 -- 172.21.15.62:0/3896285155 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258715 (secure 0 0 0) 0x7f4020062630 con 0x7f40080a2c90 2024-09-11T23:51:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:15.443 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:51:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 -- 172.21.15.62:0/3896285155 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f40080a2c90 msgr2=0x7f40080a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f40080a2c90 0x7f40080a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4010005fd0 tx=0x7f4010005850 comp rx=0 tx=0).stop 2024-09-11T23:51:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 -- 172.21.15.62:0/3896285155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4020100a70 msgr2=0x7f40201970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4020100a70 0x7f40201970a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f401c00c970 tx=0x7f401c00ce30 comp rx=0 tx=0).stop 2024-09-11T23:51:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 -- 172.21.15.62:0/3896285155 shutdown_connections 2024-09-11T23:51:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f40080a2c90 0x7f40080a5150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4020106a70 0x7f4020197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4020101410 0x7f40201975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 --2- 172.21.15.62:0/3896285155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4020100a70 0x7f40201970a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 -- 172.21.15.62:0/3896285155 >> 172.21.15.62:0/3896285155 conn(0x7f40200fc790 msgr2=0x7f40200fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 -- 172.21.15.62:0/3896285155 shutdown_connections 2024-09-11T23:51:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:15.443+0000 7f4026fc6640 1 -- 172.21.15.62:0/3896285155 wait complete. 2024-09-11T23:51:16.085 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-11T23:51:14.815872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655777,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":478,"num_read_kb":358,"num_write":346,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347600,"kb_used_data":10812,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594224,"statfs":{"total":1151940427776,"available":1151584485376,"internally_reserved":0,"allocated":11071488,"data_stored":7042062,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":25,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002025"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830856+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:50:11.830856+0000","last_peered":"2024-09-11T23:50:11.830856+0000","last_clean":"2024-09-11T23:50:11.830856+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:50:11.830856+0000","last_undegraded":"2024-09-11T23:50:11.830856+0000","last_fullsized":"2024-09-11T23:50:11.830856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105884+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:51:13.105884+0000","last_peered":"2024-09-11T23:51:13.105884+0000","last_clean":"2024-09-11T23:51:13.105884+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:51:13.105884+0000","last_undegraded":"2024-09-11T23:51:13.105884+0000","last_fullsized":"2024-09-11T23:51:13.105884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106685+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:51:13.106685+0000","last_peered":"2024-09-11T23:51:13.106685+0000","last_clean":"2024-09-11T23:51:13.106685+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:51:13.106685+0000","last_undegraded":"2024-09-11T23:51:13.106685+0000","last_fullsized":"2024-09-11T23:51:13.106685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.103950+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:51:13.103950+0000","last_peered":"2024-09-11T23:51:13.103950+0000","last_clean":"2024-09-11T23:51:13.103950+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:51:13.103950+0000","last_undegraded":"2024-09-11T23:51:13.103950+0000","last_fullsized":"2024-09-11T23:51:13.103950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876576+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:50:11.876576+0000","last_peered":"2024-09-11T23:50:11.876576+0000","last_clean":"2024-09-11T23:50:11.876576+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:50:11.876576+0000","last_undegraded":"2024-09-11T23:50:11.876576+0000","last_fullsized":"2024-09-11T23:50:11.876576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T23:50:13.236075+0000","last_change":"2024-09-11T23:51:13.106355+0000","last_active":"2024-09-11T23:51:13.106355+0000","last_peered":"2024-09-11T23:50:13.236075+0000","last_clean":"2024-09-11T23:50:13.236075+0000","last_became_active":"2024-09-11T23:50:10.842259+0000","last_became_peered":"2024-09-11T23:50:10.842259+0000","last_unstale":"2024-09-11T23:50:13.236075+0000","last_undegraded":"2024-09-11T23:50:13.236075+0000","last_fullsized":"2024-09-11T23:50:13.236075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,8,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881133+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:50:11.881133+0000","last_peered":"2024-09-11T23:50:11.881133+0000","last_clean":"2024-09-11T23:50:11.881133+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:50:11.881133+0000","last_undegraded":"2024-09-11T23:50:11.881133+0000","last_fullsized":"2024-09-11T23:50:11.881133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877161+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:50:11.877161+0000","last_peered":"2024-09-11T23:50:11.877161+0000","last_clean":"2024-09-11T23:50:11.877161+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:50:11.877161+0000","last_undegraded":"2024-09-11T23:50:11.877161+0000","last_fullsized":"2024-09-11T23:50:11.877161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110723+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:51:13.110723+0000","last_peered":"2024-09-11T23:51:13.110723+0000","last_clean":"2024-09-11T23:51:13.110723+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:51:13.110723+0000","last_undegraded":"2024-09-11T23:51:13.110723+0000","last_fullsized":"2024-09-11T23:51:13.110723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106297+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:51:13.106297+0000","last_peered":"2024-09-11T23:51:13.106297+0000","last_clean":"2024-09-11T23:51:13.106297+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:51:13.106297+0000","last_undegraded":"2024-09-11T23:51:13.106297+0000","last_fullsized":"2024-09-11T23:51:13.106297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881588+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:50:11.881588+0000","last_peered":"2024-09-11T23:50:11.881588+0000","last_clean":"2024-09-11T23:50:11.881588+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:50:11.881588+0000","last_undegraded":"2024-09-11T23:50:11.881588+0000","last_fullsized":"2024-09-11T23:50:11.881588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.104124+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:51:13.104124+0000","last_peered":"2024-09-11T23:51:13.104124+0000","last_clean":"2024-09-11T23:51:13.104124+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:51:13.104124+0000","last_undegraded":"2024-09-11T23:51:13.104124+0000","last_fullsized":"2024-09-11T23:51:13.104124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827687+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:50:11.827687+0000","last_peered":"2024-09-11T23:50:11.827687+0000","last_clean":"2024-09-11T23:50:11.827687+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:50:11.827687+0000","last_undegraded":"2024-09-11T23:50:11.827687+0000","last_fullsized":"2024-09-11T23:50:11.827687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105956+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:51:13.105956+0000","last_peered":"2024-09-11T23:51:13.105956+0000","last_clean":"2024-09-11T23:51:13.105956+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:51:13.105956+0000","last_undegraded":"2024-09-11T23:51:13.105956+0000","last_fullsized":"2024-09-11T23:51:13.105956+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105912+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:51:13.105912+0000","last_peered":"2024-09-11T23:51:13.105912+0000","last_clean":"2024-09-11T23:51:13.105912+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:51:13.105912+0000","last_undegraded":"2024-09-11T23:51:13.105912+0000","last_fullsized":"2024-09-11T23:51:13.105912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110093+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:51:13.110093+0000","last_peered":"2024-09-11T23:51:13.110093+0000","last_clean":"2024-09-11T23:51:13.110093+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:51:13.110093+0000","last_undegraded":"2024-09-11T23:51:13.110093+0000","last_fullsized":"2024-09-11T23:51:13.110093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830826+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:50:11.830826+0000","last_peered":"2024-09-11T23:50:11.830826+0000","last_clean":"2024-09-11T23:50:11.830826+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:50:11.830826+0000","last_undegraded":"2024-09-11T23:50:11.830826+0000","last_fullsized":"2024-09-11T23:50:11.830826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107885+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:51:13.107885+0000","last_peered":"2024-09-11T23:51:13.107885+0000","last_clean":"2024-09-11T23:51:13.107885+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:51:13.107885+0000","last_undegraded":"2024-09-11T23:51:13.107885+0000","last_fullsized":"2024-09-11T23:51:13.107885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107132+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:51:13.107132+0000","last_peered":"2024-09-11T23:51:13.107132+0000","last_clean":"2024-09-11T23:51:13.107132+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:51:13.107132+0000","last_undegraded":"2024-09-11T23:51:13.107132+0000","last_fullsized":"2024-09-11T23:51:13.107132+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830673+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:50:11.830673+0000","last_peered":"2024-09-11T23:50:11.830673+0000","last_clean":"2024-09-11T23:50:11.830673+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:50:11.830673+0000","last_undegraded":"2024-09-11T23:50:11.830673+0000","last_fullsized":"2024-09-11T23:50:11.830673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827752+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:50:11.827752+0000","last_peered":"2024-09-11T23:50:11.827752+0000","last_clean":"2024-09-11T23:50:11.827752+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:50:11.827752+0000","last_undegraded":"2024-09-11T23:50:11.827752+0000","last_fullsized":"2024-09-11T23:50:11.827752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110355+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:51:13.110355+0000","last_peered":"2024-09-11T23:51:13.110355+0000","last_clean":"2024-09-11T23:51:13.110355+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:51:13.110355+0000","last_undegraded":"2024-09-11T23:51:13.110355+0000","last_fullsized":"2024-09-11T23:51:13.110355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878562+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:50:11.878562+0000","last_peered":"2024-09-11T23:50:11.878562+0000","last_clean":"2024-09-11T23:50:11.878562+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:50:11.878562+0000","last_undegraded":"2024-09-11T23:50:11.878562+0000","last_fullsized":"2024-09-11T23:50:11.878562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882045+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:50:11.882045+0000","last_peered":"2024-09-11T23:50:11.882045+0000","last_clean":"2024-09-11T23:50:11.882045+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:50:11.882045+0000","last_undegraded":"2024-09-11T23:50:11.882045+0000","last_fullsized":"2024-09-11T23:50:11.882045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880422+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:50:11.880422+0000","last_peered":"2024-09-11T23:50:11.880422+0000","last_clean":"2024-09-11T23:50:11.880422+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:50:11.880422+0000","last_undegraded":"2024-09-11T23:50:11.880422+0000","last_fullsized":"2024-09-11T23:50:11.880422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107759+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:51:13.107759+0000","last_peered":"2024-09-11T23:51:13.107759+0000","last_clean":"2024-09-11T23:51:13.107759+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:51:13.107759+0000","last_undegraded":"2024-09-11T23:51:13.107759+0000","last_fullsized":"2024-09-11T23:51:13.107759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830611+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:50:11.830611+0000","last_peered":"2024-09-11T23:50:11.830611+0000","last_clean":"2024-09-11T23:50:11.830611+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:50:11.830611+0000","last_undegraded":"2024-09-11T23:50:11.830611+0000","last_fullsized":"2024-09-11T23:50:11.830611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827672+0000","last_change":"2024-09-11T23:50:10.845202+0000","last_active":"2024-09-11T23:50:11.827672+0000","last_peered":"2024-09-11T23:50:11.827672+0000","last_clean":"2024-09-11T23:50:11.827672+0000","last_became_active":"2024-09-11T23:50:10.844892+0000","last_became_peered":"2024-09-11T23:50:10.844892+0000","last_unstale":"2024-09-11T23:50:11.827672+0000","last_undegraded":"2024-09-11T23:50:11.827672+0000","last_fullsized":"2024-09-11T23:50:11.827672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:09.744773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106107+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:51:13.106107+0000","last_peered":"2024-09-11T23:51:13.106107+0000","last_clean":"2024-09-11T23:51:13.106107+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:51:13.106107+0000","last_undegraded":"2024-09-11T23:51:13.106107+0000","last_fullsized":"2024-09-11T23:51:13.106107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106485+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:51:13.106485+0000","last_peered":"2024-09-11T23:51:13.106485+0000","last_clean":"2024-09-11T23:51:13.106485+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:51:13.106485+0000","last_undegraded":"2024-09-11T23:51:13.106485+0000","last_fullsized":"2024-09-11T23:51:13.106485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T23:50:14.052022+0000","last_change":"2024-09-11T23:51:13.106565+0000","last_active":"2024-09-11T23:51:13.106565+0000","last_peered":"2024-09-11T23:50:14.052022+0000","last_clean":"2024-09-11T23:50:14.052022+0000","last_became_active":"2024-09-11T23:50:10.843259+0000","last_became_peered":"2024-09-11T23:50:10.843259+0000","last_unstale":"2024-09-11T23:50:14.052022+0000","last_undegraded":"2024-09-11T23:50:14.052022+0000","last_fullsized":"2024-09-11T23:50:14.052022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,9],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830450+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:50:11.830450+0000","last_peered":"2024-09-11T23:50:11.830450+0000","last_clean":"2024-09-11T23:50:11.830450+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:50:11.830450+0000","last_undegraded":"2024-09-11T23:50:11.830450+0000","last_fullsized":"2024-09-11T23:50:11.830450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876539+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:50:11.876539+0000","last_peered":"2024-09-11T23:50:11.876539+0000","last_clean":"2024-09-11T23:50:11.876539+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:50:11.876539+0000","last_undegraded":"2024-09-11T23:50:11.876539+0000","last_fullsized":"2024-09-11T23:50:11.876539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:51:08.938762+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:51:08.938762+0000","last_peered":"2024-09-11T23:51:08.938762+0000","last_clean":"2024-09-11T23:51:08.938762+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:51:08.938762+0000","last_undegraded":"2024-09-11T23:51:08.938762+0000","last_fullsized":"2024-09-11T23:51:08.938762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880098+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:50:11.880098+0000","last_peered":"2024-09-11T23:50:11.880098+0000","last_clean":"2024-09-11T23:50:11.880098+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:50:11.880098+0000","last_undegraded":"2024-09-11T23:50:11.880098+0000","last_fullsized":"2024-09-11T23:50:11.880098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106722+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:51:13.106722+0000","last_peered":"2024-09-11T23:51:13.106722+0000","last_clean":"2024-09-11T23:51:13.106722+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:51:13.106722+0000","last_undegraded":"2024-09-11T23:51:13.106722+0000","last_fullsized":"2024-09-11T23:51:13.106722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.866815+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:50:19.866815+0000","last_peered":"2024-09-11T23:50:19.866815+0000","last_clean":"2024-09-11T23:50:19.866815+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:50:19.866815+0000","last_undegraded":"2024-09-11T23:50:19.866815+0000","last_fullsized":"2024-09-11T23:50:19.866815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877320+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:50:11.877320+0000","last_peered":"2024-09-11T23:50:11.877320+0000","last_clean":"2024-09-11T23:50:11.877320+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:50:11.877320+0000","last_undegraded":"2024-09-11T23:50:11.877320+0000","last_fullsized":"2024-09-11T23:50:11.877320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882426+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:50:11.882426+0000","last_peered":"2024-09-11T23:50:11.882426+0000","last_clean":"2024-09-11T23:50:11.882426+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:50:11.882426+0000","last_undegraded":"2024-09-11T23:50:11.882426+0000","last_fullsized":"2024-09-11T23:50:11.882426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107022+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:51:13.107022+0000","last_peered":"2024-09-11T23:51:13.107022+0000","last_clean":"2024-09-11T23:51:13.107022+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:51:13.107022+0000","last_undegraded":"2024-09-11T23:51:13.107022+0000","last_fullsized":"2024-09-11T23:51:13.107022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106304+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:51:13.106304+0000","last_peered":"2024-09-11T23:51:13.106304+0000","last_clean":"2024-09-11T23:51:13.106304+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:51:13.106304+0000","last_undegraded":"2024-09-11T23:51:13.106304+0000","last_fullsized":"2024-09-11T23:51:13.106304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110528+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:51:13.110528+0000","last_peered":"2024-09-11T23:51:13.110528+0000","last_clean":"2024-09-11T23:51:13.110528+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:51:13.110528+0000","last_undegraded":"2024-09-11T23:51:13.110528+0000","last_fullsized":"2024-09-11T23:51:13.110528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862666+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:50:18.862666+0000","last_peered":"2024-09-11T23:50:18.862666+0000","last_clean":"2024-09-11T23:50:18.862666+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:50:18.862666+0000","last_undegraded":"2024-09-11T23:50:18.862666+0000","last_fullsized":"2024-09-11T23:50:18.862666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107122+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:51:13.107122+0000","last_peered":"2024-09-11T23:51:13.107122+0000","last_clean":"2024-09-11T23:51:13.107122+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:51:13.107122+0000","last_undegraded":"2024-09-11T23:51:13.107122+0000","last_fullsized":"2024-09-11T23:51:13.107122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861345+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:50:18.861345+0000","last_peered":"2024-09-11T23:50:18.861345+0000","last_clean":"2024-09-11T23:50:18.861345+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:50:18.861345+0000","last_undegraded":"2024-09-11T23:50:18.861345+0000","last_fullsized":"2024-09-11T23:50:18.861345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110340+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:51:13.110340+0000","last_peered":"2024-09-11T23:51:13.110340+0000","last_clean":"2024-09-11T23:51:13.110340+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:51:13.110340+0000","last_undegraded":"2024-09-11T23:51:13.110340+0000","last_fullsized":"2024-09-11T23:51:13.110340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":164,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106421+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:51:13.106421+0000","last_peered":"2024-09-11T23:51:13.106421+0000","last_clean":"2024-09-11T23:51:13.106421+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:51:13.106421+0000","last_undegraded":"2024-09-11T23:51:13.106421+0000","last_fullsized":"2024-09-11T23:51:13.106421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":115,"num_read_kb":116,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877925+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:50:11.877925+0000","last_peered":"2024-09-11T23:50:11.877925+0000","last_clean":"2024-09-11T23:50:11.877925+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:50:11.877925+0000","last_undegraded":"2024-09-11T23:50:11.877925+0000","last_fullsized":"2024-09-11T23:50:11.877925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107018+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:51:13.107018+0000","last_peered":"2024-09-11T23:51:13.107018+0000","last_clean":"2024-09-11T23:51:13.107018+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:51:13.107018+0000","last_undegraded":"2024-09-11T23:51:13.107018+0000","last_fullsized":"2024-09-11T23:51:13.107018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827276+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:50:11.827276+0000","last_peered":"2024-09-11T23:50:11.827276+0000","last_clean":"2024-09-11T23:50:11.827276+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:50:11.827276+0000","last_undegraded":"2024-09-11T23:50:11.827276+0000","last_fullsized":"2024-09-11T23:50:11.827276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.862298+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:50:18.862298+0000","last_peered":"2024-09-11T23:50:18.862298+0000","last_clean":"2024-09-11T23:50:18.862298+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:50:18.862298+0000","last_undegraded":"2024-09-11T23:50:18.862298+0000","last_fullsized":"2024-09-11T23:50:18.862298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881417+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:50:11.881417+0000","last_peered":"2024-09-11T23:50:11.881417+0000","last_clean":"2024-09-11T23:50:11.881417+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:50:11.881417+0000","last_undegraded":"2024-09-11T23:50:11.881417+0000","last_fullsized":"2024-09-11T23:50:11.881417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":81,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.103914+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:51:13.103914+0000","last_peered":"2024-09-11T23:51:13.103914+0000","last_clean":"2024-09-11T23:51:13.103914+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:51:13.103914+0000","last_undegraded":"2024-09-11T23:51:13.103914+0000","last_fullsized":"2024-09-11T23:51:13.103914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880994+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:50:11.880994+0000","last_peered":"2024-09-11T23:50:11.880994+0000","last_clean":"2024-09-11T23:50:11.880994+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:50:11.880994+0000","last_undegraded":"2024-09-11T23:50:11.880994+0000","last_fullsized":"2024-09-11T23:50:11.880994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868544+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:50:19.868544+0000","last_peered":"2024-09-11T23:50:19.868544+0000","last_clean":"2024-09-11T23:50:19.868544+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:50:19.868544+0000","last_undegraded":"2024-09-11T23:50:19.868544+0000","last_fullsized":"2024-09-11T23:50:19.868544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879800+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:50:11.879800+0000","last_peered":"2024-09-11T23:50:11.879800+0000","last_clean":"2024-09-11T23:50:11.879800+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:50:11.879800+0000","last_undegraded":"2024-09-11T23:50:11.879800+0000","last_fullsized":"2024-09-11T23:50:11.879800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:20.871408+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:50:20.871408+0000","last_peered":"2024-09-11T23:50:20.871408+0000","last_clean":"2024-09-11T23:50:20.871408+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:50:20.871408+0000","last_undegraded":"2024-09-11T23:50:20.871408+0000","last_fullsized":"2024-09-11T23:50:20.871408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830599+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:50:11.830599+0000","last_peered":"2024-09-11T23:50:11.830599+0000","last_clean":"2024-09-11T23:50:11.830599+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:50:11.830599+0000","last_undegraded":"2024-09-11T23:50:11.830599+0000","last_fullsized":"2024-09-11T23:50:11.830599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877724+0000","last_change":"2024-09-11T23:50:08.766524+0000","last_active":"2024-09-11T23:50:11.877724+0000","last_peered":"2024-09-11T23:50:11.877724+0000","last_clean":"2024-09-11T23:50:11.877724+0000","last_became_active":"2024-09-11T23:50:08.766120+0000","last_became_peered":"2024-09-11T23:50:08.766120+0000","last_unstale":"2024-09-11T23:50:11.877724+0000","last_undegraded":"2024-09-11T23:50:11.877724+0000","last_fullsized":"2024-09-11T23:50:11.877724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:51:49.261871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881682+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:50:11.881682+0000","last_peered":"2024-09-11T23:50:11.881682+0000","last_clean":"2024-09-11T23:50:11.881682+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:50:11.881682+0000","last_undegraded":"2024-09-11T23:50:11.881682+0000","last_fullsized":"2024-09-11T23:50:11.881682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.886203+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:50:11.886203+0000","last_peered":"2024-09-11T23:50:11.886203+0000","last_clean":"2024-09-11T23:50:11.886203+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:50:11.886203+0000","last_undegraded":"2024-09-11T23:50:11.886203+0000","last_fullsized":"2024-09-11T23:50:11.886203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881405+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:50:11.881405+0000","last_peered":"2024-09-11T23:50:11.881405+0000","last_clean":"2024-09-11T23:50:11.881405+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:50:11.881405+0000","last_undegraded":"2024-09-11T23:50:11.881405+0000","last_fullsized":"2024-09-11T23:50:11.881405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107580+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:51:13.107580+0000","last_peered":"2024-09-11T23:51:13.107580+0000","last_clean":"2024-09-11T23:51:13.107580+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:51:13.107580+0000","last_undegraded":"2024-09-11T23:51:13.107580+0000","last_fullsized":"2024-09-11T23:51:13.107580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881586+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:50:11.881586+0000","last_peered":"2024-09-11T23:50:11.881586+0000","last_clean":"2024-09-11T23:50:11.881586+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:50:11.881586+0000","last_undegraded":"2024-09-11T23:50:11.881586+0000","last_fullsized":"2024-09-11T23:50:11.881586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881179+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:50:11.881179+0000","last_peered":"2024-09-11T23:50:11.881179+0000","last_clean":"2024-09-11T23:50:11.881179+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:50:11.881179+0000","last_undegraded":"2024-09-11T23:50:11.881179+0000","last_fullsized":"2024-09-11T23:50:11.881179+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.104045+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:51:13.104045+0000","last_peered":"2024-09-11T23:51:13.104045+0000","last_clean":"2024-09-11T23:51:13.104045+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:51:13.104045+0000","last_undegraded":"2024-09-11T23:51:13.104045+0000","last_fullsized":"2024-09-11T23:51:13.104045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879540+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:50:11.879540+0000","last_peered":"2024-09-11T23:50:11.879540+0000","last_clean":"2024-09-11T23:50:11.879540+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:50:11.879540+0000","last_undegraded":"2024-09-11T23:50:11.879540+0000","last_fullsized":"2024-09-11T23:50:11.879540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.868224+0000","last_change":"2024-09-11T23:50:08.767568+0000","last_active":"2024-09-11T23:50:19.868224+0000","last_peered":"2024-09-11T23:50:19.868224+0000","last_clean":"2024-09-11T23:50:19.868224+0000","last_became_active":"2024-09-11T23:50:08.767217+0000","last_became_peered":"2024-09-11T23:50:08.767217+0000","last_unstale":"2024-09-11T23:50:19.868224+0000","last_undegraded":"2024-09-11T23:50:19.868224+0000","last_fullsized":"2024-09-11T23:50:19.868224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:12.102018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827258+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:50:11.827258+0000","last_peered":"2024-09-11T23:50:11.827258+0000","last_clean":"2024-09-11T23:50:11.827258+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:50:11.827258+0000","last_undegraded":"2024-09-11T23:50:11.827258+0000","last_fullsized":"2024-09-11T23:50:11.827258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":73,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:51:09.939270+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:51:09.939270+0000","last_peered":"2024-09-11T23:51:09.939270+0000","last_clean":"2024-09-11T23:51:09.939270+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:51:09.939270+0000","last_undegraded":"2024-09-11T23:51:09.939270+0000","last_fullsized":"2024-09-11T23:51:09.939270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.882510+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:50:11.882510+0000","last_peered":"2024-09-11T23:50:11.882510+0000","last_clean":"2024-09-11T23:50:11.882510+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:50:11.882510+0000","last_undegraded":"2024-09-11T23:50:11.882510+0000","last_fullsized":"2024-09-11T23:50:11.882510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881397+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:50:11.881397+0000","last_peered":"2024-09-11T23:50:11.881397+0000","last_clean":"2024-09-11T23:50:11.881397+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:50:11.881397+0000","last_undegraded":"2024-09-11T23:50:11.881397+0000","last_fullsized":"2024-09-11T23:50:11.881397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T23:50:13.236099+0000","last_change":"2024-09-11T23:51:13.106356+0000","last_active":"2024-09-11T23:51:13.106356+0000","last_peered":"2024-09-11T23:50:13.236099+0000","last_clean":"2024-09-11T23:50:13.236099+0000","last_became_active":"2024-09-11T23:50:10.843961+0000","last_became_peered":"2024-09-11T23:50:10.843961+0000","last_unstale":"2024-09-11T23:50:13.236099+0000","last_undegraded":"2024-09-11T23:50:13.236099+0000","last_fullsized":"2024-09-11T23:50:13.236099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106769+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:51:13.106769+0000","last_peered":"2024-09-11T23:51:13.106769+0000","last_clean":"2024-09-11T23:51:13.106769+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:51:13.106769+0000","last_undegraded":"2024-09-11T23:51:13.106769+0000","last_fullsized":"2024-09-11T23:51:13.106769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106131+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:51:13.106131+0000","last_peered":"2024-09-11T23:51:13.106131+0000","last_clean":"2024-09-11T23:51:13.106131+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:51:13.106131+0000","last_undegraded":"2024-09-11T23:51:13.106131+0000","last_fullsized":"2024-09-11T23:51:13.106131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861294+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:50:18.861294+0000","last_peered":"2024-09-11T23:50:18.861294+0000","last_clean":"2024-09-11T23:50:18.861294+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:50:18.861294+0000","last_undegraded":"2024-09-11T23:50:18.861294+0000","last_fullsized":"2024-09-11T23:50:18.861294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.827648+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:50:11.827648+0000","last_peered":"2024-09-11T23:50:11.827648+0000","last_clean":"2024-09-11T23:50:11.827648+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:50:11.827648+0000","last_undegraded":"2024-09-11T23:50:11.827648+0000","last_fullsized":"2024-09-11T23:50:11.827648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107995+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:51:13.107995+0000","last_peered":"2024-09-11T23:51:13.107995+0000","last_clean":"2024-09-11T23:51:13.107995+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:51:13.107995+0000","last_undegraded":"2024-09-11T23:51:13.107995+0000","last_fullsized":"2024-09-11T23:51:13.107995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880484+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:50:11.880484+0000","last_peered":"2024-09-11T23:50:11.880484+0000","last_clean":"2024-09-11T23:50:11.880484+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:50:11.880484+0000","last_undegraded":"2024-09-11T23:50:11.880484+0000","last_fullsized":"2024-09-11T23:50:11.880484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.104483+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:51:13.104483+0000","last_peered":"2024-09-11T23:51:13.104483+0000","last_clean":"2024-09-11T23:51:13.104483+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:51:13.104483+0000","last_undegraded":"2024-09-11T23:51:13.104483+0000","last_fullsized":"2024-09-11T23:51:13.104483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105995+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:51:13.105995+0000","last_peered":"2024-09-11T23:51:13.105995+0000","last_clean":"2024-09-11T23:51:13.105995+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:51:13.105995+0000","last_undegraded":"2024-09-11T23:51:13.105995+0000","last_fullsized":"2024-09-11T23:51:13.105995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110232+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:51:13.110232+0000","last_peered":"2024-09-11T23:51:13.110232+0000","last_clean":"2024-09-11T23:51:13.110232+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:51:13.110232+0000","last_undegraded":"2024-09-11T23:51:13.110232+0000","last_fullsized":"2024-09-11T23:51:13.110232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876349+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:50:11.876349+0000","last_peered":"2024-09-11T23:50:11.876349+0000","last_clean":"2024-09-11T23:50:11.876349+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:50:11.876349+0000","last_undegraded":"2024-09-11T23:50:11.876349+0000","last_fullsized":"2024-09-11T23:50:11.876349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861563+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:50:18.861563+0000","last_peered":"2024-09-11T23:50:18.861563+0000","last_clean":"2024-09-11T23:50:18.861563+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:50:18.861563+0000","last_undegraded":"2024-09-11T23:50:18.861563+0000","last_fullsized":"2024-09-11T23:50:18.861563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.830666+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:50:11.830666+0000","last_peered":"2024-09-11T23:50:11.830666+0000","last_clean":"2024-09-11T23:50:11.830666+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:50:11.830666+0000","last_undegraded":"2024-09-11T23:50:11.830666+0000","last_fullsized":"2024-09-11T23:50:11.830666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.876431+0000","last_change":"2024-09-11T23:50:08.766530+0000","last_active":"2024-09-11T23:50:11.876431+0000","last_peered":"2024-09-11T23:50:11.876431+0000","last_clean":"2024-09-11T23:50:11.876431+0000","last_became_active":"2024-09-11T23:50:08.766014+0000","last_became_peered":"2024-09-11T23:50:08.766014+0000","last_unstale":"2024-09-11T23:50:11.876431+0000","last_undegraded":"2024-09-11T23:50:11.876431+0000","last_fullsized":"2024-09-11T23:50:11.876431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:38.056709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878313+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:50:11.878313+0000","last_peered":"2024-09-11T23:50:11.878313+0000","last_clean":"2024-09-11T23:50:11.878313+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:50:11.878313+0000","last_undegraded":"2024-09-11T23:50:11.878313+0000","last_fullsized":"2024-09-11T23:50:11.878313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.863430+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:50:18.863430+0000","last_peered":"2024-09-11T23:50:18.863430+0000","last_clean":"2024-09-11T23:50:18.863430+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:50:18.863430+0000","last_undegraded":"2024-09-11T23:50:18.863430+0000","last_fullsized":"2024-09-11T23:50:18.863430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881169+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:50:11.881169+0000","last_peered":"2024-09-11T23:50:11.881169+0000","last_clean":"2024-09-11T23:50:11.881169+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:50:11.881169+0000","last_undegraded":"2024-09-11T23:50:11.881169+0000","last_fullsized":"2024-09-11T23:50:11.881169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.881434+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:50:11.881434+0000","last_peered":"2024-09-11T23:50:11.881434+0000","last_clean":"2024-09-11T23:50:11.881434+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:50:11.881434+0000","last_undegraded":"2024-09-11T23:50:11.881434+0000","last_fullsized":"2024-09-11T23:50:11.881434+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.885419+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:50:11.885419+0000","last_peered":"2024-09-11T23:50:11.885419+0000","last_clean":"2024-09-11T23:50:11.885419+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:50:11.885419+0000","last_undegraded":"2024-09-11T23:50:11.885419+0000","last_fullsized":"2024-09-11T23:50:11.885419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110326+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:51:13.110326+0000","last_peered":"2024-09-11T23:51:13.110326+0000","last_clean":"2024-09-11T23:51:13.110326+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:51:13.110326+0000","last_undegraded":"2024-09-11T23:51:13.110326+0000","last_fullsized":"2024-09-11T23:51:13.110326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:19.867614+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:50:19.867614+0000","last_peered":"2024-09-11T23:50:19.867614+0000","last_clean":"2024-09-11T23:50:19.867614+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:50:19.867614+0000","last_undegraded":"2024-09-11T23:50:19.867614+0000","last_fullsized":"2024-09-11T23:50:19.867614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107072+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:51:13.107072+0000","last_peered":"2024-09-11T23:51:13.107072+0000","last_clean":"2024-09-11T23:51:13.107072+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:51:13.107072+0000","last_undegraded":"2024-09-11T23:51:13.107072+0000","last_fullsized":"2024-09-11T23:51:13.107072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:12.424274+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:50:12.424274+0000","last_peered":"2024-09-11T23:50:12.424274+0000","last_clean":"2024-09-11T23:50:12.424274+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:50:12.424274+0000","last_undegraded":"2024-09-11T23:50:12.424274+0000","last_fullsized":"2024-09-11T23:50:12.424274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879978+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:50:11.879978+0000","last_peered":"2024-09-11T23:50:11.879978+0000","last_clean":"2024-09-11T23:50:11.879978+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:50:11.879978+0000","last_undegraded":"2024-09-11T23:50:11.879978+0000","last_fullsized":"2024-09-11T23:50:11.879978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":29,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T23:50:20.874695+0000","last_change":"2024-09-11T23:51:13.107286+0000","last_active":"2024-09-11T23:51:13.107286+0000","last_peered":"2024-09-11T23:50:20.874695+0000","last_clean":"2024-09-11T23:50:20.874695+0000","last_became_active":"2024-09-11T23:50:08.763978+0000","last_became_peered":"2024-09-11T23:50:08.763978+0000","last_unstale":"2024-09-11T23:50:20.874695+0000","last_undegraded":"2024-09-11T23:50:20.874695+0000","last_fullsized":"2024-09-11T23:50:20.874695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7,8],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65175,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":342,"num_read_kb":239,"num_write":163,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281926,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":419875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776158,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1036288,"data_stored":1081564,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1011963,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561076,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":419500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":1011254,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":892928,"data_stored":428973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":419811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":490458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":489108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":909312,"data_stored":429414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:51:16.089 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:51:16.349 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:16.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:16 smithi120 ceph-mon[35182]: pgmap v314: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-11T23:51:16.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:16 smithi120 ceph-mon[35182]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:16.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:16 smithi120 ceph-mon[35182]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T23:51:16.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:16 smithi083 ceph-mon[34805]: pgmap v314: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-11T23:51:16.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:16 smithi083 ceph-mon[34805]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:16.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:16 smithi083 ceph-mon[34805]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T23:51:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:16 smithi062 ceph-mon[30958]: pgmap v314: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-11T23:51:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:16 smithi062 ceph-mon[30958]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:16.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:16 smithi062 ceph-mon[30958]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.773+0000 7f694d4cd640 1 -- 172.21.15.62:0/3506894014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6948101de0 msgr2=0x7f69481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.773+0000 7f694d4cd640 1 --2- 172.21.15.62:0/3506894014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6948101de0 0x7f69481021c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6940009ca0 tx=0x7f694002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.773+0000 7f694d4cd640 1 -- 172.21.15.62:0/3506894014 shutdown_connections 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.773+0000 7f694d4cd640 1 --2- 172.21.15.62:0/3506894014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948106f10 0x7f694810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.773+0000 7f694d4cd640 1 --2- 172.21.15.62:0/3506894014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6948102790 0x7f69481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.773+0000 7f694d4cd640 1 --2- 172.21.15.62:0/3506894014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6948101de0 0x7f69481021c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.773+0000 7f694d4cd640 1 -- 172.21.15.62:0/3506894014 >> 172.21.15.62:0/3506894014 conn(0x7f69480fd8c0 msgr2=0x7f69480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.773+0000 7f694d4cd640 1 -- 172.21.15.62:0/3506894014 shutdown_connections 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.773+0000 7f694d4cd640 1 -- 172.21.15.62:0/3506894014 wait complete. 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.773+0000 7f694d4cd640 1 Processor -- start 2024-09-11T23:51:16.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f694d4cd640 1 -- start start 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f694d4cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6948101de0 0x7f694819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f694d4cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6948102790 0x7f694819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f694d4cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948106f10 0x7f694819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f694d4cd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6948111fc0 con 0x7f6948102790 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f694d4cd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6948111e40 con 0x7f6948106f10 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f694d4cd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6948112140 con 0x7f6948101de0 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f69467fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6948102790 0x7f694819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f69467fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6948102790 0x7f694819ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55372/0 (socket says 172.21.15.62:55372) 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f69467fc640 1 -- 172.21.15.62:0/2282917101 learned_addr learned my addr 172.21.15.62:0/2282917101 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f6946ffd640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6948101de0 0x7f694819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f69477fe640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948106f10 0x7f694819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:16.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f69467fc640 1 -- 172.21.15.62:0/2282917101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6948101de0 msgr2=0x7f694819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f69467fc640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6948101de0 0x7f694819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f69467fc640 1 -- 172.21.15.62:0/2282917101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948106f10 msgr2=0x7f694819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f69467fc640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948106f10 0x7f694819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f69467fc640 1 -- 172.21.15.62:0/2282917101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694819f6b0 con 0x7f6948102790 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.774+0000 7f69477fe640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948106f10 0x7f694819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.775+0000 7f69467fc640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6948102790 0x7f694819ac40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f693000d950 tx=0x7f693000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.775+0000 7f693ffff640 1 -- 172.21.15.62:0/2282917101 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930017070 con 0x7f6948102790 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.775+0000 7f693ffff640 1 -- 172.21.15.62:0/2282917101 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6930009c30 con 0x7f6948102790 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.775+0000 7f694d4cd640 1 -- 172.21.15.62:0/2282917101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694819f9a0 con 0x7f6948102790 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.775+0000 7f693ffff640 1 -- 172.21.15.62:0/2282917101 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693000f6b0 con 0x7f6948102790 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.775+0000 7f694d4cd640 1 -- 172.21.15.62:0/2282917101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69481a7280 con 0x7f6948102790 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.776+0000 7f694d4cd640 1 -- 172.21.15.62:0/2282917101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69481021c0 con 0x7f6948102790 2024-09-11T23:51:16.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.777+0000 7f693ffff640 1 -- 172.21.15.62:0/2282917101 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f69300109d0 con 0x7f6948102790 2024-09-11T23:51:16.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.778+0000 7f693ffff640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f69340a2c10 0x7f69340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:16.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.778+0000 7f6946ffd640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f69340a2c10 0x7f69340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:16.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.778+0000 7f693ffff640 1 -- 172.21.15.62:0/2282917101 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f69300cedf0 con 0x7f6948102790 2024-09-11T23:51:16.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.778+0000 7f6946ffd640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f69340a2c10 0x7f69340a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6940006010 tx=0x7f6940009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:16.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.781+0000 7f693ffff640 1 -- 172.21.15.62:0/2282917101 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6930009de0 con 0x7f6948102790 2024-09-11T23:51:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.954+0000 7f694d4cd640 1 -- 172.21.15.62:0/2282917101 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f694810de00 con 0x7f69340a2c10 2024-09-11T23:51:16.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.960+0000 7f693ffff640 1 -- 172.21.15.62:0/2282917101 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258723 (secure 0 0 0) 0x7f694810de00 con 0x7f69340a2c10 2024-09-11T23:51:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:16.963 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:51:16.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 -- 172.21.15.62:0/2282917101 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f69340a2c10 msgr2=0x7f69340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:16.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f69340a2c10 0x7f69340a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6940006010 tx=0x7f6940009820 comp rx=0 tx=0).stop 2024-09-11T23:51:16.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 -- 172.21.15.62:0/2282917101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6948102790 msgr2=0x7f694819ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:16.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6948102790 0x7f694819ac40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f693000d950 tx=0x7f693000de10 comp rx=0 tx=0).stop 2024-09-11T23:51:16.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 -- 172.21.15.62:0/2282917101 shutdown_connections 2024-09-11T23:51:16.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f69340a2c10 0x7f69340a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:16.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948106f10 0x7f694819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:16.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6948102790 0x7f694819ac40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:16.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 --2- 172.21.15.62:0/2282917101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6948101de0 0x7f694819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:16.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 -- 172.21.15.62:0/2282917101 >> 172.21.15.62:0/2282917101 conn(0x7f69480fd8c0 msgr2=0x7f69480fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:16.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 -- 172.21.15.62:0/2282917101 shutdown_connections 2024-09-11T23:51:16.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:16.964+0000 7f694d4cd640 1 -- 172.21.15.62:0/2282917101 wait complete. 2024-09-11T23:51:17.634 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-11T23:51:16.816348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655777,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":488,"num_read_kb":363,"num_write":346,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347744,"kb_used_data":10928,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594080,"statfs":{"total":1151940427776,"available":1151584337920,"internally_reserved":0,"allocated":11190272,"data_stored":7126950,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":20,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001880"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114410+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:51:14.114410+0000","last_peered":"2024-09-11T23:51:14.114410+0000","last_clean":"2024-09-11T23:51:14.114410+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:51:14.114410+0000","last_undegraded":"2024-09-11T23:51:14.114410+0000","last_fullsized":"2024-09-11T23:51:14.114410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105884+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:51:13.105884+0000","last_peered":"2024-09-11T23:51:13.105884+0000","last_clean":"2024-09-11T23:51:13.105884+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:51:13.105884+0000","last_undegraded":"2024-09-11T23:51:13.105884+0000","last_fullsized":"2024-09-11T23:51:13.105884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106685+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:51:13.106685+0000","last_peered":"2024-09-11T23:51:13.106685+0000","last_clean":"2024-09-11T23:51:13.106685+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:51:13.106685+0000","last_undegraded":"2024-09-11T23:51:13.106685+0000","last_fullsized":"2024-09-11T23:51:13.106685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.103950+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:51:13.103950+0000","last_peered":"2024-09-11T23:51:13.103950+0000","last_clean":"2024-09-11T23:51:13.103950+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:51:13.103950+0000","last_undegraded":"2024-09-11T23:51:13.103950+0000","last_fullsized":"2024-09-11T23:51:13.103950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116864+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:51:14.116864+0000","last_peered":"2024-09-11T23:51:14.116864+0000","last_clean":"2024-09-11T23:51:14.116864+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:51:14.116864+0000","last_undegraded":"2024-09-11T23:51:14.116864+0000","last_fullsized":"2024-09-11T23:51:14.116864+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T23:50:13.236075+0000","last_change":"2024-09-11T23:51:13.106355+0000","last_active":"2024-09-11T23:51:13.106355+0000","last_peered":"2024-09-11T23:50:13.236075+0000","last_clean":"2024-09-11T23:50:13.236075+0000","last_became_active":"2024-09-11T23:50:10.842259+0000","last_became_peered":"2024-09-11T23:50:10.842259+0000","last_unstale":"2024-09-11T23:50:13.236075+0000","last_undegraded":"2024-09-11T23:50:13.236075+0000","last_fullsized":"2024-09-11T23:50:13.236075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,8,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116252+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:51:14.116252+0000","last_peered":"2024-09-11T23:51:14.116252+0000","last_clean":"2024-09-11T23:51:14.116252+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:51:14.116252+0000","last_undegraded":"2024-09-11T23:51:14.116252+0000","last_fullsized":"2024-09-11T23:51:14.116252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877161+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:50:11.877161+0000","last_peered":"2024-09-11T23:50:11.877161+0000","last_clean":"2024-09-11T23:50:11.877161+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:50:11.877161+0000","last_undegraded":"2024-09-11T23:50:11.877161+0000","last_fullsized":"2024-09-11T23:50:11.877161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110723+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:51:13.110723+0000","last_peered":"2024-09-11T23:51:13.110723+0000","last_clean":"2024-09-11T23:51:13.110723+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:51:13.110723+0000","last_undegraded":"2024-09-11T23:51:13.110723+0000","last_fullsized":"2024-09-11T23:51:13.110723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106297+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:51:13.106297+0000","last_peered":"2024-09-11T23:51:13.106297+0000","last_clean":"2024-09-11T23:51:13.106297+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:51:13.106297+0000","last_undegraded":"2024-09-11T23:51:13.106297+0000","last_fullsized":"2024-09-11T23:51:13.106297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116742+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:51:14.116742+0000","last_peered":"2024-09-11T23:51:14.116742+0000","last_clean":"2024-09-11T23:51:14.116742+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:51:14.116742+0000","last_undegraded":"2024-09-11T23:51:14.116742+0000","last_fullsized":"2024-09-11T23:51:14.116742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.104124+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:51:13.104124+0000","last_peered":"2024-09-11T23:51:13.104124+0000","last_clean":"2024-09-11T23:51:13.104124+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:51:13.104124+0000","last_undegraded":"2024-09-11T23:51:13.104124+0000","last_fullsized":"2024-09-11T23:51:13.104124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114515+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:51:14.114515+0000","last_peered":"2024-09-11T23:51:14.114515+0000","last_clean":"2024-09-11T23:51:14.114515+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:51:14.114515+0000","last_undegraded":"2024-09-11T23:51:14.114515+0000","last_fullsized":"2024-09-11T23:51:14.114515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105956+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:51:13.105956+0000","last_peered":"2024-09-11T23:51:13.105956+0000","last_clean":"2024-09-11T23:51:13.105956+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:51:13.105956+0000","last_undegraded":"2024-09-11T23:51:13.105956+0000","last_fullsized":"2024-09-11T23:51:13.105956+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105912+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:51:13.105912+0000","last_peered":"2024-09-11T23:51:13.105912+0000","last_clean":"2024-09-11T23:51:13.105912+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:51:13.105912+0000","last_undegraded":"2024-09-11T23:51:13.105912+0000","last_fullsized":"2024-09-11T23:51:13.105912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110093+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:51:13.110093+0000","last_peered":"2024-09-11T23:51:13.110093+0000","last_clean":"2024-09-11T23:51:13.110093+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:51:13.110093+0000","last_undegraded":"2024-09-11T23:51:13.110093+0000","last_fullsized":"2024-09-11T23:51:13.110093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114431+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:51:14.114431+0000","last_peered":"2024-09-11T23:51:14.114431+0000","last_clean":"2024-09-11T23:51:14.114431+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:51:14.114431+0000","last_undegraded":"2024-09-11T23:51:14.114431+0000","last_fullsized":"2024-09-11T23:51:14.114431+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107885+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:51:13.107885+0000","last_peered":"2024-09-11T23:51:13.107885+0000","last_clean":"2024-09-11T23:51:13.107885+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:51:13.107885+0000","last_undegraded":"2024-09-11T23:51:13.107885+0000","last_fullsized":"2024-09-11T23:51:13.107885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107132+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:51:13.107132+0000","last_peered":"2024-09-11T23:51:13.107132+0000","last_clean":"2024-09-11T23:51:13.107132+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:51:13.107132+0000","last_undegraded":"2024-09-11T23:51:13.107132+0000","last_fullsized":"2024-09-11T23:51:13.107132+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.113934+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:51:14.113934+0000","last_peered":"2024-09-11T23:51:14.113934+0000","last_clean":"2024-09-11T23:51:14.113934+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:51:14.113934+0000","last_undegraded":"2024-09-11T23:51:14.113934+0000","last_fullsized":"2024-09-11T23:51:14.113934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114619+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:51:14.114619+0000","last_peered":"2024-09-11T23:51:14.114619+0000","last_clean":"2024-09-11T23:51:14.114619+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:51:14.114619+0000","last_undegraded":"2024-09-11T23:51:14.114619+0000","last_fullsized":"2024-09-11T23:51:14.114619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110355+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:51:13.110355+0000","last_peered":"2024-09-11T23:51:13.110355+0000","last_clean":"2024-09-11T23:51:13.110355+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:51:13.110355+0000","last_undegraded":"2024-09-11T23:51:13.110355+0000","last_fullsized":"2024-09-11T23:51:13.110355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878562+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:50:11.878562+0000","last_peered":"2024-09-11T23:50:11.878562+0000","last_clean":"2024-09-11T23:50:11.878562+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:50:11.878562+0000","last_undegraded":"2024-09-11T23:50:11.878562+0000","last_fullsized":"2024-09-11T23:50:11.878562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116635+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:51:14.116635+0000","last_peered":"2024-09-11T23:51:14.116635+0000","last_clean":"2024-09-11T23:51:14.116635+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:51:14.116635+0000","last_undegraded":"2024-09-11T23:51:14.116635+0000","last_fullsized":"2024-09-11T23:51:14.116635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880422+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:50:11.880422+0000","last_peered":"2024-09-11T23:50:11.880422+0000","last_clean":"2024-09-11T23:50:11.880422+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:50:11.880422+0000","last_undegraded":"2024-09-11T23:50:11.880422+0000","last_fullsized":"2024-09-11T23:50:11.880422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107759+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:51:13.107759+0000","last_peered":"2024-09-11T23:51:13.107759+0000","last_clean":"2024-09-11T23:51:13.107759+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:51:13.107759+0000","last_undegraded":"2024-09-11T23:51:13.107759+0000","last_fullsized":"2024-09-11T23:51:13.107759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114121+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:51:14.114121+0000","last_peered":"2024-09-11T23:51:14.114121+0000","last_clean":"2024-09-11T23:51:14.114121+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:51:14.114121+0000","last_undegraded":"2024-09-11T23:51:14.114121+0000","last_fullsized":"2024-09-11T23:51:14.114121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.410108+0000","last_change":"2024-09-11T23:51:14.410108+0000","last_active":"2024-09-11T23:51:14.410108+0000","last_peered":"2024-09-11T23:51:14.410108+0000","last_clean":"2024-09-11T23:51:14.410108+0000","last_became_active":"2024-09-11T23:51:14.409588+0000","last_became_peered":"2024-09-11T23:51:14.409588+0000","last_unstale":"2024-09-11T23:51:14.410108+0000","last_undegraded":"2024-09-11T23:51:14.410108+0000","last_fullsized":"2024-09-11T23:51:14.410108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:29.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106107+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:51:13.106107+0000","last_peered":"2024-09-11T23:51:13.106107+0000","last_clean":"2024-09-11T23:51:13.106107+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:51:13.106107+0000","last_undegraded":"2024-09-11T23:51:13.106107+0000","last_fullsized":"2024-09-11T23:51:13.106107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106485+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:51:13.106485+0000","last_peered":"2024-09-11T23:51:13.106485+0000","last_clean":"2024-09-11T23:51:13.106485+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:51:13.106485+0000","last_undegraded":"2024-09-11T23:51:13.106485+0000","last_fullsized":"2024-09-11T23:51:13.106485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T23:50:14.052022+0000","last_change":"2024-09-11T23:51:13.106565+0000","last_active":"2024-09-11T23:51:13.106565+0000","last_peered":"2024-09-11T23:50:14.052022+0000","last_clean":"2024-09-11T23:50:14.052022+0000","last_became_active":"2024-09-11T23:50:10.843259+0000","last_became_peered":"2024-09-11T23:50:10.843259+0000","last_unstale":"2024-09-11T23:50:14.052022+0000","last_undegraded":"2024-09-11T23:50:14.052022+0000","last_fullsized":"2024-09-11T23:50:14.052022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,9],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.113981+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:51:14.113981+0000","last_peered":"2024-09-11T23:51:14.113981+0000","last_clean":"2024-09-11T23:51:14.113981+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:51:14.113981+0000","last_undegraded":"2024-09-11T23:51:14.113981+0000","last_fullsized":"2024-09-11T23:51:14.113981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117356+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:51:14.117356+0000","last_peered":"2024-09-11T23:51:14.117356+0000","last_clean":"2024-09-11T23:51:14.117356+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:51:14.117356+0000","last_undegraded":"2024-09-11T23:51:14.117356+0000","last_fullsized":"2024-09-11T23:51:14.117356+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":84,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117335+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:51:14.117335+0000","last_peered":"2024-09-11T23:51:14.117335+0000","last_clean":"2024-09-11T23:51:14.117335+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:51:14.117335+0000","last_undegraded":"2024-09-11T23:51:14.117335+0000","last_fullsized":"2024-09-11T23:51:14.117335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116157+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:51:14.116157+0000","last_peered":"2024-09-11T23:51:14.116157+0000","last_clean":"2024-09-11T23:51:14.116157+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:51:14.116157+0000","last_undegraded":"2024-09-11T23:51:14.116157+0000","last_fullsized":"2024-09-11T23:51:14.116157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106722+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:51:13.106722+0000","last_peered":"2024-09-11T23:51:13.106722+0000","last_clean":"2024-09-11T23:51:13.106722+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:51:13.106722+0000","last_undegraded":"2024-09-11T23:51:13.106722+0000","last_fullsized":"2024-09-11T23:51:13.106722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114143+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:51:14.114143+0000","last_peered":"2024-09-11T23:51:14.114143+0000","last_clean":"2024-09-11T23:51:14.114143+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:51:14.114143+0000","last_undegraded":"2024-09-11T23:51:14.114143+0000","last_fullsized":"2024-09-11T23:51:14.114143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.877320+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:50:11.877320+0000","last_peered":"2024-09-11T23:50:11.877320+0000","last_clean":"2024-09-11T23:50:11.877320+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:50:11.877320+0000","last_undegraded":"2024-09-11T23:50:11.877320+0000","last_fullsized":"2024-09-11T23:50:11.877320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116877+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:51:14.116877+0000","last_peered":"2024-09-11T23:51:14.116877+0000","last_clean":"2024-09-11T23:51:14.116877+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:51:14.116877+0000","last_undegraded":"2024-09-11T23:51:14.116877+0000","last_fullsized":"2024-09-11T23:51:14.116877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107022+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:51:13.107022+0000","last_peered":"2024-09-11T23:51:13.107022+0000","last_clean":"2024-09-11T23:51:13.107022+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:51:13.107022+0000","last_undegraded":"2024-09-11T23:51:13.107022+0000","last_fullsized":"2024-09-11T23:51:13.107022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106304+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:51:13.106304+0000","last_peered":"2024-09-11T23:51:13.106304+0000","last_clean":"2024-09-11T23:51:13.106304+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:51:13.106304+0000","last_undegraded":"2024-09-11T23:51:13.106304+0000","last_fullsized":"2024-09-11T23:51:13.106304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110528+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:51:13.110528+0000","last_peered":"2024-09-11T23:51:13.110528+0000","last_clean":"2024-09-11T23:51:13.110528+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:51:13.110528+0000","last_undegraded":"2024-09-11T23:51:13.110528+0000","last_fullsized":"2024-09-11T23:51:13.110528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116551+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:51:14.116551+0000","last_peered":"2024-09-11T23:51:14.116551+0000","last_clean":"2024-09-11T23:51:14.116551+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:51:14.116551+0000","last_undegraded":"2024-09-11T23:51:14.116551+0000","last_fullsized":"2024-09-11T23:51:14.116551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107122+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:51:13.107122+0000","last_peered":"2024-09-11T23:51:13.107122+0000","last_clean":"2024-09-11T23:51:13.107122+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:51:13.107122+0000","last_undegraded":"2024-09-11T23:51:13.107122+0000","last_fullsized":"2024-09-11T23:51:13.107122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116331+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:51:14.116331+0000","last_peered":"2024-09-11T23:51:14.116331+0000","last_clean":"2024-09-11T23:51:14.116331+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:51:14.116331+0000","last_undegraded":"2024-09-11T23:51:14.116331+0000","last_fullsized":"2024-09-11T23:51:14.116331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110340+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:51:13.110340+0000","last_peered":"2024-09-11T23:51:13.110340+0000","last_clean":"2024-09-11T23:51:13.110340+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:51:13.110340+0000","last_undegraded":"2024-09-11T23:51:13.110340+0000","last_fullsized":"2024-09-11T23:51:13.110340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":164,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106421+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:51:13.106421+0000","last_peered":"2024-09-11T23:51:13.106421+0000","last_clean":"2024-09-11T23:51:13.106421+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:51:13.106421+0000","last_undegraded":"2024-09-11T23:51:13.106421+0000","last_fullsized":"2024-09-11T23:51:13.106421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":115,"num_read_kb":116,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117646+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:51:14.117646+0000","last_peered":"2024-09-11T23:51:14.117646+0000","last_clean":"2024-09-11T23:51:14.117646+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:51:14.117646+0000","last_undegraded":"2024-09-11T23:51:14.117646+0000","last_fullsized":"2024-09-11T23:51:14.117646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107018+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:51:13.107018+0000","last_peered":"2024-09-11T23:51:13.107018+0000","last_clean":"2024-09-11T23:51:13.107018+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:51:13.107018+0000","last_undegraded":"2024-09-11T23:51:13.107018+0000","last_fullsized":"2024-09-11T23:51:13.107018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114193+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:51:14.114193+0000","last_peered":"2024-09-11T23:51:14.114193+0000","last_clean":"2024-09-11T23:51:14.114193+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:51:14.114193+0000","last_undegraded":"2024-09-11T23:51:14.114193+0000","last_fullsized":"2024-09-11T23:51:14.114193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116707+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:51:14.116707+0000","last_peered":"2024-09-11T23:51:14.116707+0000","last_clean":"2024-09-11T23:51:14.116707+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:51:14.116707+0000","last_undegraded":"2024-09-11T23:51:14.116707+0000","last_fullsized":"2024-09-11T23:51:14.116707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116498+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:51:14.116498+0000","last_peered":"2024-09-11T23:51:14.116498+0000","last_clean":"2024-09-11T23:51:14.116498+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:51:14.116498+0000","last_undegraded":"2024-09-11T23:51:14.116498+0000","last_fullsized":"2024-09-11T23:51:14.116498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":81,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.103914+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:51:13.103914+0000","last_peered":"2024-09-11T23:51:13.103914+0000","last_clean":"2024-09-11T23:51:13.103914+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:51:13.103914+0000","last_undegraded":"2024-09-11T23:51:13.103914+0000","last_fullsized":"2024-09-11T23:51:13.103914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116150+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:51:14.116150+0000","last_peered":"2024-09-11T23:51:14.116150+0000","last_clean":"2024-09-11T23:51:14.116150+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:51:14.116150+0000","last_undegraded":"2024-09-11T23:51:14.116150+0000","last_fullsized":"2024-09-11T23:51:14.116150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117088+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:51:14.117088+0000","last_peered":"2024-09-11T23:51:14.117088+0000","last_clean":"2024-09-11T23:51:14.117088+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:51:14.117088+0000","last_undegraded":"2024-09-11T23:51:14.117088+0000","last_fullsized":"2024-09-11T23:51:14.117088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116158+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:51:14.116158+0000","last_peered":"2024-09-11T23:51:14.116158+0000","last_clean":"2024-09-11T23:51:14.116158+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:51:14.116158+0000","last_undegraded":"2024-09-11T23:51:14.116158+0000","last_fullsized":"2024-09-11T23:51:14.116158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117866+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:51:14.117866+0000","last_peered":"2024-09-11T23:51:14.117866+0000","last_clean":"2024-09-11T23:51:14.117866+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:51:14.117866+0000","last_undegraded":"2024-09-11T23:51:14.117866+0000","last_fullsized":"2024-09-11T23:51:14.117866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114175+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:51:14.114175+0000","last_peered":"2024-09-11T23:51:14.114175+0000","last_clean":"2024-09-11T23:51:14.114175+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:51:14.114175+0000","last_undegraded":"2024-09-11T23:51:14.114175+0000","last_fullsized":"2024-09-11T23:51:14.114175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.413526+0000","last_change":"2024-09-11T23:51:14.413526+0000","last_active":"2024-09-11T23:51:14.413526+0000","last_peered":"2024-09-11T23:51:14.413526+0000","last_clean":"2024-09-11T23:51:14.413526+0000","last_became_active":"2024-09-11T23:51:14.411027+0000","last_became_peered":"2024-09-11T23:51:14.411027+0000","last_unstale":"2024-09-11T23:51:14.413526+0000","last_undegraded":"2024-09-11T23:51:14.413526+0000","last_fullsized":"2024-09-11T23:51:14.413526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:55.020567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116641+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:51:14.116641+0000","last_peered":"2024-09-11T23:51:14.116641+0000","last_clean":"2024-09-11T23:51:14.116641+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:51:14.116641+0000","last_undegraded":"2024-09-11T23:51:14.116641+0000","last_fullsized":"2024-09-11T23:51:14.116641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116380+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:51:14.116380+0000","last_peered":"2024-09-11T23:51:14.116380+0000","last_clean":"2024-09-11T23:51:14.116380+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:51:14.116380+0000","last_undegraded":"2024-09-11T23:51:14.116380+0000","last_fullsized":"2024-09-11T23:51:14.116380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116489+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:51:14.116489+0000","last_peered":"2024-09-11T23:51:14.116489+0000","last_clean":"2024-09-11T23:51:14.116489+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:51:14.116489+0000","last_undegraded":"2024-09-11T23:51:14.116489+0000","last_fullsized":"2024-09-11T23:51:14.116489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107580+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:51:13.107580+0000","last_peered":"2024-09-11T23:51:13.107580+0000","last_clean":"2024-09-11T23:51:13.107580+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:51:13.107580+0000","last_undegraded":"2024-09-11T23:51:13.107580+0000","last_fullsized":"2024-09-11T23:51:13.107580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116847+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:51:14.116847+0000","last_peered":"2024-09-11T23:51:14.116847+0000","last_clean":"2024-09-11T23:51:14.116847+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:51:14.116847+0000","last_undegraded":"2024-09-11T23:51:14.116847+0000","last_fullsized":"2024-09-11T23:51:14.116847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116351+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:51:14.116351+0000","last_peered":"2024-09-11T23:51:14.116351+0000","last_clean":"2024-09-11T23:51:14.116351+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:51:14.116351+0000","last_undegraded":"2024-09-11T23:51:14.116351+0000","last_fullsized":"2024-09-11T23:51:14.116351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.104045+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:51:13.104045+0000","last_peered":"2024-09-11T23:51:13.104045+0000","last_clean":"2024-09-11T23:51:13.104045+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:51:13.104045+0000","last_undegraded":"2024-09-11T23:51:13.104045+0000","last_fullsized":"2024-09-11T23:51:13.104045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879540+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:50:11.879540+0000","last_peered":"2024-09-11T23:50:11.879540+0000","last_clean":"2024-09-11T23:50:11.879540+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:50:11.879540+0000","last_undegraded":"2024-09-11T23:50:11.879540+0000","last_fullsized":"2024-09-11T23:50:11.879540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.122022+0000","last_change":"2024-09-11T23:51:14.121480+0000","last_active":"2024-09-11T23:51:14.122022+0000","last_peered":"2024-09-11T23:51:14.122022+0000","last_clean":"2024-09-11T23:51:14.122022+0000","last_became_active":"2024-09-11T23:51:14.118435+0000","last_became_peered":"2024-09-11T23:51:14.118435+0000","last_unstale":"2024-09-11T23:51:14.122022+0000","last_undegraded":"2024-09-11T23:51:14.122022+0000","last_fullsized":"2024-09-11T23:51:14.122022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114117+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:51:14.114117+0000","last_peered":"2024-09-11T23:51:14.114117+0000","last_clean":"2024-09-11T23:51:14.114117+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:51:14.114117+0000","last_undegraded":"2024-09-11T23:51:14.114117+0000","last_fullsized":"2024-09-11T23:51:14.114117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.946369+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:51:14.946369+0000","last_peered":"2024-09-11T23:51:14.946369+0000","last_clean":"2024-09-11T23:51:14.946369+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:51:14.946369+0000","last_undegraded":"2024-09-11T23:51:14.946369+0000","last_fullsized":"2024-09-11T23:51:14.946369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116740+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:51:14.116740+0000","last_peered":"2024-09-11T23:51:14.116740+0000","last_clean":"2024-09-11T23:51:14.116740+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:51:14.116740+0000","last_undegraded":"2024-09-11T23:51:14.116740+0000","last_fullsized":"2024-09-11T23:51:14.116740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116523+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:51:14.116523+0000","last_peered":"2024-09-11T23:51:14.116523+0000","last_clean":"2024-09-11T23:51:14.116523+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:51:14.116523+0000","last_undegraded":"2024-09-11T23:51:14.116523+0000","last_fullsized":"2024-09-11T23:51:14.116523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T23:50:13.236099+0000","last_change":"2024-09-11T23:51:13.106356+0000","last_active":"2024-09-11T23:51:13.106356+0000","last_peered":"2024-09-11T23:50:13.236099+0000","last_clean":"2024-09-11T23:50:13.236099+0000","last_became_active":"2024-09-11T23:50:10.843961+0000","last_became_peered":"2024-09-11T23:50:10.843961+0000","last_unstale":"2024-09-11T23:50:13.236099+0000","last_undegraded":"2024-09-11T23:50:13.236099+0000","last_fullsized":"2024-09-11T23:50:13.236099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106769+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:51:13.106769+0000","last_peered":"2024-09-11T23:51:13.106769+0000","last_clean":"2024-09-11T23:51:13.106769+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:51:13.106769+0000","last_undegraded":"2024-09-11T23:51:13.106769+0000","last_fullsized":"2024-09-11T23:51:13.106769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106131+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:51:13.106131+0000","last_peered":"2024-09-11T23:51:13.106131+0000","last_clean":"2024-09-11T23:51:13.106131+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:51:13.106131+0000","last_undegraded":"2024-09-11T23:51:13.106131+0000","last_fullsized":"2024-09-11T23:51:13.106131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117793+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:51:14.117793+0000","last_peered":"2024-09-11T23:51:14.117793+0000","last_clean":"2024-09-11T23:51:14.117793+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:51:14.117793+0000","last_undegraded":"2024-09-11T23:51:14.117793+0000","last_fullsized":"2024-09-11T23:51:14.117793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.115193+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:51:14.115193+0000","last_peered":"2024-09-11T23:51:14.115193+0000","last_clean":"2024-09-11T23:51:14.115193+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:51:14.115193+0000","last_undegraded":"2024-09-11T23:51:14.115193+0000","last_fullsized":"2024-09-11T23:51:14.115193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107995+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:51:13.107995+0000","last_peered":"2024-09-11T23:51:13.107995+0000","last_clean":"2024-09-11T23:51:13.107995+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:51:13.107995+0000","last_undegraded":"2024-09-11T23:51:13.107995+0000","last_fullsized":"2024-09-11T23:51:13.107995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.880484+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:50:11.880484+0000","last_peered":"2024-09-11T23:50:11.880484+0000","last_clean":"2024-09-11T23:50:11.880484+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:50:11.880484+0000","last_undegraded":"2024-09-11T23:50:11.880484+0000","last_fullsized":"2024-09-11T23:50:11.880484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.104483+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:51:13.104483+0000","last_peered":"2024-09-11T23:51:13.104483+0000","last_clean":"2024-09-11T23:51:13.104483+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:51:13.104483+0000","last_undegraded":"2024-09-11T23:51:13.104483+0000","last_fullsized":"2024-09-11T23:51:13.104483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105995+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:51:13.105995+0000","last_peered":"2024-09-11T23:51:13.105995+0000","last_clean":"2024-09-11T23:51:13.105995+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:51:13.105995+0000","last_undegraded":"2024-09-11T23:51:13.105995+0000","last_fullsized":"2024-09-11T23:51:13.105995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110232+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:51:13.110232+0000","last_peered":"2024-09-11T23:51:13.110232+0000","last_clean":"2024-09-11T23:51:13.110232+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:51:13.110232+0000","last_undegraded":"2024-09-11T23:51:13.110232+0000","last_fullsized":"2024-09-11T23:51:13.110232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117114+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:51:14.117114+0000","last_peered":"2024-09-11T23:51:14.117114+0000","last_clean":"2024-09-11T23:51:14.117114+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:51:14.117114+0000","last_undegraded":"2024-09-11T23:51:14.117114+0000","last_fullsized":"2024-09-11T23:51:14.117114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:18.861563+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:50:18.861563+0000","last_peered":"2024-09-11T23:50:18.861563+0000","last_clean":"2024-09-11T23:50:18.861563+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:50:18.861563+0000","last_undegraded":"2024-09-11T23:50:18.861563+0000","last_fullsized":"2024-09-11T23:50:18.861563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114121+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:51:14.114121+0000","last_peered":"2024-09-11T23:51:14.114121+0000","last_clean":"2024-09-11T23:51:14.114121+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:51:14.114121+0000","last_undegraded":"2024-09-11T23:51:14.114121+0000","last_fullsized":"2024-09-11T23:51:14.114121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.124072+0000","last_change":"2024-09-11T23:51:14.123663+0000","last_active":"2024-09-11T23:51:14.124072+0000","last_peered":"2024-09-11T23:51:14.124072+0000","last_clean":"2024-09-11T23:51:14.124072+0000","last_became_active":"2024-09-11T23:51:14.118315+0000","last_became_peered":"2024-09-11T23:51:14.118315+0000","last_unstale":"2024-09-11T23:51:14.124072+0000","last_undegraded":"2024-09-11T23:51:14.124072+0000","last_fullsized":"2024-09-11T23:51:14.124072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:10.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.878313+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:50:11.878313+0000","last_peered":"2024-09-11T23:50:11.878313+0000","last_clean":"2024-09-11T23:50:11.878313+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:50:11.878313+0000","last_undegraded":"2024-09-11T23:50:11.878313+0000","last_fullsized":"2024-09-11T23:50:11.878313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116980+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:51:14.116980+0000","last_peered":"2024-09-11T23:51:14.116980+0000","last_clean":"2024-09-11T23:51:14.116980+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:51:14.116980+0000","last_undegraded":"2024-09-11T23:51:14.116980+0000","last_fullsized":"2024-09-11T23:51:14.116980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116368+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:51:14.116368+0000","last_peered":"2024-09-11T23:51:14.116368+0000","last_clean":"2024-09-11T23:51:14.116368+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:51:14.116368+0000","last_undegraded":"2024-09-11T23:51:14.116368+0000","last_fullsized":"2024-09-11T23:51:14.116368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116161+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:51:14.116161+0000","last_peered":"2024-09-11T23:51:14.116161+0000","last_clean":"2024-09-11T23:51:14.116161+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:51:14.116161+0000","last_undegraded":"2024-09-11T23:51:14.116161+0000","last_fullsized":"2024-09-11T23:51:14.116161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117388+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:51:14.117388+0000","last_peered":"2024-09-11T23:51:14.117388+0000","last_clean":"2024-09-11T23:51:14.117388+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:51:14.117388+0000","last_undegraded":"2024-09-11T23:51:14.117388+0000","last_fullsized":"2024-09-11T23:51:14.117388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.110326+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:51:13.110326+0000","last_peered":"2024-09-11T23:51:13.110326+0000","last_clean":"2024-09-11T23:51:13.110326+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:51:13.110326+0000","last_undegraded":"2024-09-11T23:51:13.110326+0000","last_fullsized":"2024-09-11T23:51:13.110326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117655+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:51:14.117655+0000","last_peered":"2024-09-11T23:51:14.117655+0000","last_clean":"2024-09-11T23:51:14.117655+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:51:14.117655+0000","last_undegraded":"2024-09-11T23:51:14.117655+0000","last_fullsized":"2024-09-11T23:51:14.117655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107072+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:51:13.107072+0000","last_peered":"2024-09-11T23:51:13.107072+0000","last_clean":"2024-09-11T23:51:13.107072+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:51:13.107072+0000","last_undegraded":"2024-09-11T23:51:13.107072+0000","last_fullsized":"2024-09-11T23:51:13.107072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:12.424274+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:50:12.424274+0000","last_peered":"2024-09-11T23:50:12.424274+0000","last_clean":"2024-09-11T23:50:12.424274+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:50:12.424274+0000","last_undegraded":"2024-09-11T23:50:12.424274+0000","last_fullsized":"2024-09-11T23:50:12.424274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:50:11.879978+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:50:11.879978+0000","last_peered":"2024-09-11T23:50:11.879978+0000","last_clean":"2024-09-11T23:50:11.879978+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:50:11.879978+0000","last_undegraded":"2024-09-11T23:50:11.879978+0000","last_fullsized":"2024-09-11T23:50:11.879978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":29,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T23:50:20.874695+0000","last_change":"2024-09-11T23:51:13.107286+0000","last_active":"2024-09-11T23:51:13.107286+0000","last_peered":"2024-09-11T23:50:20.874695+0000","last_clean":"2024-09-11T23:50:20.874695+0000","last_became_active":"2024-09-11T23:50:08.763978+0000","last_became_peered":"2024-09-11T23:50:08.763978+0000","last_unstale":"2024-09-11T23:50:20.874695+0000","last_undegraded":"2024-09-11T23:50:20.874695+0000","last_fullsized":"2024-09-11T23:50:20.874695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7,8],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65175,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":352,"num_read_kb":244,"num_write":163,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":194943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281926,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":436969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776158,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1036288,"data_stored":1081564,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168618,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":436543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":962560,"data_stored":1029057,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":436594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":1028348,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":892928,"data_stored":428973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":419811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":490458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":489108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":909312,"data_stored":429414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:51:17.638 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-11T23:51:17.639 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 81, 'state': 'peering', 'last_fresh': '2024-09-11T23:50:13.236075+0000', 'last_change': '2024-09-11T23:51:13.106355+0000', 'last_active': '2024-09-11T23:51:13.106355+0000', 'last_peered': '2024-09-11T23:50:13.236075+0000', 'last_clean': '2024-09-11T23:50:13.236075+0000', 'last_became_active': '2024-09-11T23:50:10.842259+0000', 'last_became_peered': '2024-09-11T23:50:10.842259+0000', 'last_unstale': '2024-09-11T23:50:13.236075+0000', 'last_undegraded': '2024-09-11T23:50:13.236075+0000', 'last_fullsized': '2024-09-11T23:50:13.236075+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 80, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T23:50:09.814327+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T23:50:09.814327+0000', 'last_clean_scrub_stamp': '2024-09-11T23:50:09.814327+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [10, 9, 8], 'acting': [10, 9, 8], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4, 8, 9], 'up_primary': 10, 'acting_primary': 10, 'purged_snaps': []} 2024-09-11T23:51:17.639 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-11T23:51:17.639 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 81, 'state': 'peering', 'last_fresh': '2024-09-11T23:50:14.052022+0000', 'last_change': '2024-09-11T23:51:13.106565+0000', 'last_active': '2024-09-11T23:51:13.106565+0000', 'last_peered': '2024-09-11T23:50:14.052022+0000', 'last_clean': '2024-09-11T23:50:14.052022+0000', 'last_became_active': '2024-09-11T23:50:10.843259+0000', 'last_became_peered': '2024-09-11T23:50:10.843259+0000', 'last_unstale': '2024-09-11T23:50:14.052022+0000', 'last_undegraded': '2024-09-11T23:50:14.052022+0000', 'last_fullsized': '2024-09-11T23:50:14.052022+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 80, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T23:50:09.814327+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T23:50:09.814327+0000', 'last_clean_scrub_stamp': '2024-09-11T23:50:09.814327+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 9, 4], 'acting': [2, 9, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4, 5, 9], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T23:51:17.639 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-11T23:51:17.639 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 81, 'state': 'peering', 'last_fresh': '2024-09-11T23:50:13.236099+0000', 'last_change': '2024-09-11T23:51:13.106356+0000', 'last_active': '2024-09-11T23:51:13.106356+0000', 'last_peered': '2024-09-11T23:50:13.236099+0000', 'last_clean': '2024-09-11T23:50:13.236099+0000', 'last_became_active': '2024-09-11T23:50:10.843961+0000', 'last_became_peered': '2024-09-11T23:50:10.843961+0000', 'last_unstale': '2024-09-11T23:50:13.236099+0000', 'last_undegraded': '2024-09-11T23:50:13.236099+0000', 'last_fullsized': '2024-09-11T23:50:13.236099+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 80, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T23:50:09.814327+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T23:50:09.814327+0000', 'last_clean_scrub_stamp': '2024-09-11T23:50:09.814327+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [10, 9, 0], 'acting': [10, 9, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 4, 9], 'up_primary': 10, 'acting_primary': 10, 'purged_snaps': []} 2024-09-11T23:51:17.639 INFO:tasks.cephadm.ceph_manager.ceph:PG 2.1f is not active+clean 2024-09-11T23:51:17.639 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '2.1f', 'version': "81'9", 'reported_seq': 29, 'reported_epoch': 81, 'state': 'peering', 'last_fresh': '2024-09-11T23:50:20.874695+0000', 'last_change': '2024-09-11T23:51:13.107286+0000', 'last_active': '2024-09-11T23:51:13.107286+0000', 'last_peered': '2024-09-11T23:50:20.874695+0000', 'last_clean': '2024-09-11T23:50:20.874695+0000', 'last_became_active': '2024-09-11T23:50:08.763978+0000', 'last_became_peered': '2024-09-11T23:50:08.763978+0000', 'last_unstale': '2024-09-11T23:50:20.874695+0000', 'last_undegraded': '2024-09-11T23:50:20.874695+0000', 'last_fullsized': '2024-09-11T23:50:20.874695+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 78, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T23:50:07.748464+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T23:50:07.748464+0000', 'last_clean_scrub_stamp': '2024-09-11T23:50:07.748464+0000', 'objects_scrubbed': 0, 'log_size': 9, 'log_dups_size': 0, 'ondisk_log_size': 9, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 582, 'num_objects': 4, 'num_object_clones': 0, 'num_object_copies': 12, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10, 'num_write_kb': 1, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 3, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 7, 1], 'acting': [9, 7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 7, 8], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2024-09-11T23:51:17.640 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-11T23:51:17.896 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:18.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:18 smithi062 ceph-mon[30958]: pgmap v315: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:51:18.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:18 smithi062 ceph-mon[30958]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:18.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:18 smithi120 ceph-mon[35182]: pgmap v315: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:51:18.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:18 smithi120 ceph-mon[35182]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:18.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.353+0000 7fa5d022e640 1 -- 172.21.15.62:0/3362389179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5c81082a0 msgr2=0x7fa5c810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:18.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.353+0000 7fa5d022e640 1 --2- 172.21.15.62:0/3362389179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5c81082a0 0x7fa5c810ec40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009cd0 tx=0x7fa5c002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.353+0000 7fa5d022e640 1 -- 172.21.15.62:0/3362389179 shutdown_connections 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.353+0000 7fa5d022e640 1 --2- 172.21.15.62:0/3362389179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5c81082a0 0x7fa5c810ec40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.353+0000 7fa5d022e640 1 --2- 172.21.15.62:0/3362389179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c81036e0 0x7fa5c8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.353+0000 7fa5d022e640 1 --2- 172.21.15.62:0/3362389179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5c8102d30 0x7fa5c8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.353+0000 7fa5d022e640 1 -- 172.21.15.62:0/3362389179 >> 172.21.15.62:0/3362389179 conn(0x7fa5c80fc7d0 msgr2=0x7fa5c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.353+0000 7fa5d022e640 1 -- 172.21.15.62:0/3362389179 shutdown_connections 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.353+0000 7fa5d022e640 1 -- 172.21.15.62:0/3362389179 wait complete. 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5d022e640 1 Processor -- start 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5d022e640 1 -- start start 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5d022e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c8102d30 0x7fa5c8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5d022e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5c81036e0 0x7fa5c8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5d022e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5c81082a0 0x7fa5c819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5d022e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81120a0 con 0x7fa5c81036e0 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5d022e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8111f20 con 0x7fa5c8102d30 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5d022e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8112220 con 0x7fa5c81082a0 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5cd7c9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5c81036e0 0x7fa5c8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5cd7c9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5c81036e0 0x7fa5c8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55394/0 (socket says 172.21.15.62:55394) 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.354+0000 7fa5cd7c9640 1 -- 172.21.15.62:0/3552262219 learned_addr learned my addr 172.21.15.62:0/3552262219 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5cdfca640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c8102d30 0x7fa5c8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5ce7cb640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5c81082a0 0x7fa5c819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5cd7c9640 1 -- 172.21.15.62:0/3552262219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5c81082a0 msgr2=0x7fa5c819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5cd7c9640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5c81082a0 0x7fa5c819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5cd7c9640 1 -- 172.21.15.62:0/3552262219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c8102d30 msgr2=0x7fa5c8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5cd7c9640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c8102d30 0x7fa5c8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5cd7c9640 1 -- 172.21.15.62:0/3552262219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c819d520 con 0x7fa5c81036e0 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5cd7c9640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5c81036e0 0x7fa5c8198ab0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800d740 tx=0x7fa5b800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5beffd640 1 -- 172.21.15.62:0/3552262219 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8017070 con 0x7fa5c81036e0 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5beffd640 1 -- 172.21.15.62:0/3552262219 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b800de50 con 0x7fa5c81036e0 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5d022e640 1 -- 172.21.15.62:0/3552262219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c819d810 con 0x7fa5c81036e0 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5beffd640 1 -- 172.21.15.62:0/3552262219 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b800f6b0 con 0x7fa5c81036e0 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.355+0000 7fa5d022e640 1 -- 172.21.15.62:0/3552262219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c81a50f0 con 0x7fa5c81036e0 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.356+0000 7fa5d022e640 1 -- 172.21.15.62:0/3552262219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c8103110 con 0x7fa5c81036e0 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.358+0000 7fa5beffd640 1 -- 172.21.15.62:0/3552262219 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa5b801a070 con 0x7fa5c81036e0 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.359+0000 7fa5beffd640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa5b40a2c10 0x7fa5b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.359+0000 7fa5cdfca640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa5b40a2c10 0x7fa5b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:18.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.359+0000 7fa5beffd640 1 -- 172.21.15.62:0/3552262219 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa5b80ceb30 con 0x7fa5c81036e0 2024-09-11T23:51:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.359+0000 7fa5cdfca640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa5b40a2c10 0x7fa5b40a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40058c0 tx=0x7fa5c400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.362+0000 7fa5beffd640 1 -- 172.21.15.62:0/3552262219 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5b809b940 con 0x7fa5c81036e0 2024-09-11T23:51:18.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:18 smithi083 ceph-mon[34805]: pgmap v315: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T23:51:18.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:18 smithi083 ceph-mon[34805]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:18.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.608+0000 7fa5d022e640 1 -- 172.21.15.62:0/3552262219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa5c81998d0 con 0x7fa5c81036e0 2024-09-11T23:51:18.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.608+0000 7fa5beffd640 1 -- 172.21.15.62:0/3552262219 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2563 (secure 0 0 0) 0x7fa5c81998d0 con 0x7fa5c81036e0 2024-09-11T23:51:18.608 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:18.608 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"56675698-7097-11ef-bcea-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 3 pgs peering","count":3},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":435,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1726098559,"num_in_osds":12,"osd_in_since":1726098535,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":93},{"state_name":"peering","count":4}],"num_pgs":97,"num_pools":3,"num_objects":64,"data_bytes":655777,"bytes_used":356089856,"bytes_avail":1151584337920,"bytes_total":1151940427776,"inactive_pgs_ratio":0.041237112134695053,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"read_bytes_sec":2559,"read_op_per_sec":5,"write_op_per_sec":0},"fsmap":{"epoch":24,"btime":"2024-09-11T23:50:22:864278+0000","id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24437},{"filesystem_id":1,"rank":1,"name":"h","status":"up:standby-replay","gid":24461},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24463},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24469},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24484},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24488}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.62:8443/"}},"servicemap":{"epoch":25,"modified":"2024-09-11T23:51:16.816524+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.610+0000 7fa5d022e640 1 -- 172.21.15.62:0/3552262219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa5b40a2c10 msgr2=0x7fa5b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa5b40a2c10 0x7fa5b40a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40058c0 tx=0x7fa5c400a250 comp rx=0 tx=0).stop 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 -- 172.21.15.62:0/3552262219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5c81036e0 msgr2=0x7fa5c8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5c81036e0 0x7fa5c8198ab0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800d740 tx=0x7fa5b800dc00 comp rx=0 tx=0).stop 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 -- 172.21.15.62:0/3552262219 shutdown_connections 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa5b40a2c10 0x7fa5b40a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5c81082a0 0x7fa5c819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5c81036e0 0x7fa5c8198ab0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 --2- 172.21.15.62:0/3552262219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c8102d30 0x7fa5c8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 -- 172.21.15.62:0/3552262219 >> 172.21.15.62:0/3552262219 conn(0x7fa5c80fc7d0 msgr2=0x7fa5c8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 -- 172.21.15.62:0/3552262219 shutdown_connections 2024-09-11T23:51:18.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:18.611+0000 7fa5d022e640 1 -- 172.21.15.62:0/3552262219 wait complete. 2024-09-11T23:51:19.279 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-11T23:51:19.279 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:51:19.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:19 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3552262219' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T23:51:19.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:19 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3552262219' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T23:51:19.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:19 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3552262219' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T23:51:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.022+0000 7fbb9c68b640 1 -- 172.21.15.62:0/1672828419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb9410c180 msgr2=0x7fbb9410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.022+0000 7fbb9c68b640 1 --2- 172.21.15.62:0/1672828419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb9410c180 0x7fbb9410e610 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009ca0 tx=0x7fbb8c02a950 comp rx=0 tx=0).stop 2024-09-11T23:51:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.022+0000 7fbb9c68b640 1 -- 172.21.15.62:0/1672828419 shutdown_connections 2024-09-11T23:51:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.022+0000 7fbb9c68b640 1 --2- 172.21.15.62:0/1672828419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb9410c180 0x7fbb9410e610 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.022+0000 7fbb9c68b640 1 --2- 172.21.15.62:0/1672828419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb94103700 0x7fbb9410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.022+0000 7fbb9c68b640 1 --2- 172.21.15.62:0/1672828419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94102de0 0x7fbb941031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.022+0000 7fbb9c68b640 1 -- 172.21.15.62:0/1672828419 >> 172.21.15.62:0/1672828419 conn(0x7fbb940fc7a0 msgr2=0x7fbb940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.022+0000 7fbb9c68b640 1 -- 172.21.15.62:0/1672828419 shutdown_connections 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.022+0000 7fbb9c68b640 1 -- 172.21.15.62:0/1672828419 wait complete. 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9c68b640 1 Processor -- start 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9c68b640 1 -- start start 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9c68b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb94102de0 0x7fbb94100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9c68b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb94103700 0x7fbb94100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9c68b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb9410c180 0x7fbb941014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9c68b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94111fd0 con 0x7fbb94102de0 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9c68b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94111e50 con 0x7fbb9410c180 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9c68b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94112150 con 0x7fbb94103700 2024-09-11T23:51:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9a427640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb94102de0 0x7fbb94100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9a427640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb94102de0 0x7fbb94100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55406/0 (socket says 172.21.15.62:55406) 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9a427640 1 -- 172.21.15.62:0/9217861 learned_addr learned my addr 172.21.15.62:0/9217861 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.023+0000 7fbb9ac28640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb9410c180 0x7fbb941014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb99c26640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb94103700 0x7fbb94100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb9a427640 1 -- 172.21.15.62:0/9217861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb94103700 msgr2=0x7fbb94100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb9a427640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb94103700 0x7fbb94100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb9a427640 1 -- 172.21.15.62:0/9217861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb9410c180 msgr2=0x7fbb941014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb9a427640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb9410c180 0x7fbb941014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb9a427640 1 -- 172.21.15.62:0/9217861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb941a6930 con 0x7fbb94102de0 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb9a427640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb94102de0 0x7fbb94100a40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000b4a0 tx=0x7fbb9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb8b7fe640 1 -- 172.21.15.62:0/9217861 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90014070 con 0x7fbb94102de0 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb9c68b640 1 -- 172.21.15.62:0/9217861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb941a6c20 con 0x7fbb94102de0 2024-09-11T23:51:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb8b7fe640 1 -- 172.21.15.62:0/9217861 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb90010af0 con 0x7fbb94102de0 2024-09-11T23:51:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb8b7fe640 1 -- 172.21.15.62:0/9217861 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90017d10 con 0x7fbb94102de0 2024-09-11T23:51:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.024+0000 7fbb9c68b640 1 -- 172.21.15.62:0/9217861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a7160 con 0x7fbb94102de0 2024-09-11T23:51:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.025+0000 7fbb9c68b640 1 -- 172.21.15.62:0/9217861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb941031c0 con 0x7fbb94102de0 2024-09-11T23:51:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.026+0000 7fbb8b7fe640 1 -- 172.21.15.62:0/9217861 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbb90017070 con 0x7fbb94102de0 2024-09-11T23:51:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.027+0000 7fbb8b7fe640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb800a2c10 0x7fbb800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.027+0000 7fbb99c26640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb800a2c10 0x7fbb800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.027+0000 7fbb8b7fe640 1 -- 172.21.15.62:0/9217861 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fbb900ce4d0 con 0x7fbb94102de0 2024-09-11T23:51:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.027+0000 7fbb99c26640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb800a2c10 0x7fbb800a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbb84005fd0 tx=0x7fbb84005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.031+0000 7fbb8b7fe640 1 -- 172.21.15.62:0/9217861 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb9009b3f0 con 0x7fbb94102de0 2024-09-11T23:51:20.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.204+0000 7fbb9c68b640 1 -- 172.21.15.62:0/9217861 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb94062630 con 0x7fbb800a2c10 2024-09-11T23:51:20.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:20 smithi062 ceph-mon[30958]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-09-11T23:51:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.211+0000 7fbb8b7fe640 1 -- 172.21.15.62:0/9217861 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258841 (secure 0 0 0) 0x7fbb94062630 con 0x7fbb800a2c10 2024-09-11T23:51:20.212 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:20.213 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.213+0000 7fbb9c68b640 1 -- 172.21.15.62:0/9217861 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb800a2c10 msgr2=0x7fbb800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.213+0000 7fbb9c68b640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb800a2c10 0x7fbb800a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbb84005fd0 tx=0x7fbb84005850 comp rx=0 tx=0).stop 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.213+0000 7fbb9c68b640 1 -- 172.21.15.62:0/9217861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb94102de0 msgr2=0x7fbb94100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.213+0000 7fbb9c68b640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb94102de0 0x7fbb94100a40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000b4a0 tx=0x7fbb9000b960 comp rx=0 tx=0).stop 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.213+0000 7fbb9c68b640 1 -- 172.21.15.62:0/9217861 shutdown_connections 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.213+0000 7fbb9c68b640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbb800a2c10 0x7fbb800a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.213+0000 7fbb9c68b640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb9410c180 0x7fbb941014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.213+0000 7fbb9c68b640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb94103700 0x7fbb94100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.213+0000 7fbb9c68b640 1 --2- 172.21.15.62:0/9217861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb94102de0 0x7fbb94100a40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.214+0000 7fbb9c68b640 1 -- 172.21.15.62:0/9217861 >> 172.21.15.62:0/9217861 conn(0x7fbb940fc7a0 msgr2=0x7fbb940fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.214+0000 7fbb9c68b640 1 -- 172.21.15.62:0/9217861 shutdown_connections 2024-09-11T23:51:20.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:20.214+0000 7fbb9c68b640 1 -- 172.21.15.62:0/9217861 wait complete. 2024-09-11T23:51:20.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:20 smithi120 ceph-mon[35182]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-09-11T23:51:20.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:20 smithi083 ceph-mon[34805]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-09-11T23:51:20.881 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-11T23:51:18.816832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655777,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":503,"num_read_kb":373,"num_write":346,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":296,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347840,"kb_used_data":11004,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593984,"statfs":{"total":1151940427776,"available":1151584239616,"internally_reserved":0,"allocated":11268096,"data_stored":7178622,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001929"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114410+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:51:14.114410+0000","last_peered":"2024-09-11T23:51:14.114410+0000","last_clean":"2024-09-11T23:51:14.114410+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:51:14.114410+0000","last_undegraded":"2024-09-11T23:51:14.114410+0000","last_fullsized":"2024-09-11T23:51:14.114410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105884+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:51:13.105884+0000","last_peered":"2024-09-11T23:51:13.105884+0000","last_clean":"2024-09-11T23:51:13.105884+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:51:13.105884+0000","last_undegraded":"2024-09-11T23:51:13.105884+0000","last_fullsized":"2024-09-11T23:51:13.105884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106685+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:51:13.106685+0000","last_peered":"2024-09-11T23:51:13.106685+0000","last_clean":"2024-09-11T23:51:13.106685+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:51:13.106685+0000","last_undegraded":"2024-09-11T23:51:13.106685+0000","last_fullsized":"2024-09-11T23:51:13.106685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.103950+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:51:13.103950+0000","last_peered":"2024-09-11T23:51:13.103950+0000","last_clean":"2024-09-11T23:51:13.103950+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:51:13.103950+0000","last_undegraded":"2024-09-11T23:51:13.103950+0000","last_fullsized":"2024-09-11T23:51:13.103950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116864+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:51:14.116864+0000","last_peered":"2024-09-11T23:51:14.116864+0000","last_clean":"2024-09-11T23:51:14.116864+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:51:14.116864+0000","last_undegraded":"2024-09-11T23:51:14.116864+0000","last_fullsized":"2024-09-11T23:51:14.116864+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.120226+0000","last_change":"2024-09-11T23:51:14.120226+0000","last_active":"2024-09-11T23:51:14.120226+0000","last_peered":"2024-09-11T23:51:14.120226+0000","last_clean":"2024-09-11T23:51:14.120226+0000","last_became_active":"2024-09-11T23:51:14.119811+0000","last_became_peered":"2024-09-11T23:51:14.119811+0000","last_unstale":"2024-09-11T23:51:14.120226+0000","last_undegraded":"2024-09-11T23:51:14.120226+0000","last_fullsized":"2024-09-11T23:51:14.120226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:58.851476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116252+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:51:14.116252+0000","last_peered":"2024-09-11T23:51:14.116252+0000","last_clean":"2024-09-11T23:51:14.116252+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:51:14.116252+0000","last_undegraded":"2024-09-11T23:51:14.116252+0000","last_fullsized":"2024-09-11T23:51:14.116252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116477+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:51:14.116477+0000","last_peered":"2024-09-11T23:51:14.116477+0000","last_clean":"2024-09-11T23:51:14.116477+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:51:14.116477+0000","last_undegraded":"2024-09-11T23:51:14.116477+0000","last_fullsized":"2024-09-11T23:51:14.116477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.118918+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:51:14.118918+0000","last_peered":"2024-09-11T23:51:14.118918+0000","last_clean":"2024-09-11T23:51:14.118918+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:51:14.118918+0000","last_undegraded":"2024-09-11T23:51:14.118918+0000","last_fullsized":"2024-09-11T23:51:14.118918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117507+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:51:14.117507+0000","last_peered":"2024-09-11T23:51:14.117507+0000","last_clean":"2024-09-11T23:51:14.117507+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:51:14.117507+0000","last_undegraded":"2024-09-11T23:51:14.117507+0000","last_fullsized":"2024-09-11T23:51:14.117507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116742+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:51:14.116742+0000","last_peered":"2024-09-11T23:51:14.116742+0000","last_clean":"2024-09-11T23:51:14.116742+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:51:14.116742+0000","last_undegraded":"2024-09-11T23:51:14.116742+0000","last_fullsized":"2024-09-11T23:51:14.116742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.104124+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:51:13.104124+0000","last_peered":"2024-09-11T23:51:13.104124+0000","last_clean":"2024-09-11T23:51:13.104124+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:51:13.104124+0000","last_undegraded":"2024-09-11T23:51:13.104124+0000","last_fullsized":"2024-09-11T23:51:13.104124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114515+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:51:14.114515+0000","last_peered":"2024-09-11T23:51:14.114515+0000","last_clean":"2024-09-11T23:51:14.114515+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:51:14.114515+0000","last_undegraded":"2024-09-11T23:51:14.114515+0000","last_fullsized":"2024-09-11T23:51:14.114515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105956+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:51:13.105956+0000","last_peered":"2024-09-11T23:51:13.105956+0000","last_clean":"2024-09-11T23:51:13.105956+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:51:13.105956+0000","last_undegraded":"2024-09-11T23:51:13.105956+0000","last_fullsized":"2024-09-11T23:51:13.105956+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105912+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:51:13.105912+0000","last_peered":"2024-09-11T23:51:13.105912+0000","last_clean":"2024-09-11T23:51:13.105912+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:51:13.105912+0000","last_undegraded":"2024-09-11T23:51:13.105912+0000","last_fullsized":"2024-09-11T23:51:13.105912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117381+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:51:14.117381+0000","last_peered":"2024-09-11T23:51:14.117381+0000","last_clean":"2024-09-11T23:51:14.117381+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:51:14.117381+0000","last_undegraded":"2024-09-11T23:51:14.117381+0000","last_fullsized":"2024-09-11T23:51:14.117381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114431+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:51:14.114431+0000","last_peered":"2024-09-11T23:51:14.114431+0000","last_clean":"2024-09-11T23:51:14.114431+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:51:14.114431+0000","last_undegraded":"2024-09-11T23:51:14.114431+0000","last_fullsized":"2024-09-11T23:51:14.114431+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.115067+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:51:14.115067+0000","last_peered":"2024-09-11T23:51:14.115067+0000","last_clean":"2024-09-11T23:51:14.115067+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:51:14.115067+0000","last_undegraded":"2024-09-11T23:51:14.115067+0000","last_fullsized":"2024-09-11T23:51:14.115067+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107132+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:51:13.107132+0000","last_peered":"2024-09-11T23:51:13.107132+0000","last_clean":"2024-09-11T23:51:13.107132+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:51:13.107132+0000","last_undegraded":"2024-09-11T23:51:13.107132+0000","last_fullsized":"2024-09-11T23:51:13.107132+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.113934+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:51:14.113934+0000","last_peered":"2024-09-11T23:51:14.113934+0000","last_clean":"2024-09-11T23:51:14.113934+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:51:14.113934+0000","last_undegraded":"2024-09-11T23:51:14.113934+0000","last_fullsized":"2024-09-11T23:51:14.113934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114619+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:51:14.114619+0000","last_peered":"2024-09-11T23:51:14.114619+0000","last_clean":"2024-09-11T23:51:14.114619+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:51:14.114619+0000","last_undegraded":"2024-09-11T23:51:14.114619+0000","last_fullsized":"2024-09-11T23:51:14.114619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117986+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:51:14.117986+0000","last_peered":"2024-09-11T23:51:14.117986+0000","last_clean":"2024-09-11T23:51:14.117986+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:51:14.117986+0000","last_undegraded":"2024-09-11T23:51:14.117986+0000","last_fullsized":"2024-09-11T23:51:14.117986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116716+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:51:14.116716+0000","last_peered":"2024-09-11T23:51:14.116716+0000","last_clean":"2024-09-11T23:51:14.116716+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:51:14.116716+0000","last_undegraded":"2024-09-11T23:51:14.116716+0000","last_fullsized":"2024-09-11T23:51:14.116716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116635+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:51:14.116635+0000","last_peered":"2024-09-11T23:51:14.116635+0000","last_clean":"2024-09-11T23:51:14.116635+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:51:14.116635+0000","last_undegraded":"2024-09-11T23:51:14.116635+0000","last_fullsized":"2024-09-11T23:51:14.116635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117214+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:51:14.117214+0000","last_peered":"2024-09-11T23:51:14.117214+0000","last_clean":"2024-09-11T23:51:14.117214+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:51:14.117214+0000","last_undegraded":"2024-09-11T23:51:14.117214+0000","last_fullsized":"2024-09-11T23:51:14.117214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107759+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:51:13.107759+0000","last_peered":"2024-09-11T23:51:13.107759+0000","last_clean":"2024-09-11T23:51:13.107759+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:51:13.107759+0000","last_undegraded":"2024-09-11T23:51:13.107759+0000","last_fullsized":"2024-09-11T23:51:13.107759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114121+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:51:14.114121+0000","last_peered":"2024-09-11T23:51:14.114121+0000","last_clean":"2024-09-11T23:51:14.114121+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:51:14.114121+0000","last_undegraded":"2024-09-11T23:51:14.114121+0000","last_fullsized":"2024-09-11T23:51:14.114121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.410108+0000","last_change":"2024-09-11T23:51:14.410108+0000","last_active":"2024-09-11T23:51:14.410108+0000","last_peered":"2024-09-11T23:51:14.410108+0000","last_clean":"2024-09-11T23:51:14.410108+0000","last_became_active":"2024-09-11T23:51:14.409588+0000","last_became_peered":"2024-09-11T23:51:14.409588+0000","last_unstale":"2024-09-11T23:51:14.410108+0000","last_undegraded":"2024-09-11T23:51:14.410108+0000","last_fullsized":"2024-09-11T23:51:14.410108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:29.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106107+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:51:13.106107+0000","last_peered":"2024-09-11T23:51:13.106107+0000","last_clean":"2024-09-11T23:51:13.106107+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:51:13.106107+0000","last_undegraded":"2024-09-11T23:51:13.106107+0000","last_fullsized":"2024-09-11T23:51:13.106107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.106485+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:51:13.106485+0000","last_peered":"2024-09-11T23:51:13.106485+0000","last_clean":"2024-09-11T23:51:13.106485+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:51:13.106485+0000","last_undegraded":"2024-09-11T23:51:13.106485+0000","last_fullsized":"2024-09-11T23:51:13.106485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T23:50:14.052022+0000","last_change":"2024-09-11T23:51:13.106565+0000","last_active":"2024-09-11T23:51:13.106565+0000","last_peered":"2024-09-11T23:50:14.052022+0000","last_clean":"2024-09-11T23:50:14.052022+0000","last_became_active":"2024-09-11T23:50:10.843259+0000","last_became_peered":"2024-09-11T23:50:10.843259+0000","last_unstale":"2024-09-11T23:50:14.052022+0000","last_undegraded":"2024-09-11T23:50:14.052022+0000","last_fullsized":"2024-09-11T23:50:14.052022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,9],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.113981+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:51:14.113981+0000","last_peered":"2024-09-11T23:51:14.113981+0000","last_clean":"2024-09-11T23:51:14.113981+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:51:14.113981+0000","last_undegraded":"2024-09-11T23:51:14.113981+0000","last_fullsized":"2024-09-11T23:51:14.113981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117356+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:51:14.117356+0000","last_peered":"2024-09-11T23:51:14.117356+0000","last_clean":"2024-09-11T23:51:14.117356+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:51:14.117356+0000","last_undegraded":"2024-09-11T23:51:14.117356+0000","last_fullsized":"2024-09-11T23:51:14.117356+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":84,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117335+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:51:14.117335+0000","last_peered":"2024-09-11T23:51:14.117335+0000","last_clean":"2024-09-11T23:51:14.117335+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:51:14.117335+0000","last_undegraded":"2024-09-11T23:51:14.117335+0000","last_fullsized":"2024-09-11T23:51:14.117335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116157+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:51:14.116157+0000","last_peered":"2024-09-11T23:51:14.116157+0000","last_clean":"2024-09-11T23:51:14.116157+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:51:14.116157+0000","last_undegraded":"2024-09-11T23:51:14.116157+0000","last_fullsized":"2024-09-11T23:51:14.116157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.118252+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:51:14.118252+0000","last_peered":"2024-09-11T23:51:14.118252+0000","last_clean":"2024-09-11T23:51:14.118252+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:51:14.118252+0000","last_undegraded":"2024-09-11T23:51:14.118252+0000","last_fullsized":"2024-09-11T23:51:14.118252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114143+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:51:14.114143+0000","last_peered":"2024-09-11T23:51:14.114143+0000","last_clean":"2024-09-11T23:51:14.114143+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:51:14.114143+0000","last_undegraded":"2024-09-11T23:51:14.114143+0000","last_fullsized":"2024-09-11T23:51:14.114143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116608+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:51:14.116608+0000","last_peered":"2024-09-11T23:51:14.116608+0000","last_clean":"2024-09-11T23:51:14.116608+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:51:14.116608+0000","last_undegraded":"2024-09-11T23:51:14.116608+0000","last_fullsized":"2024-09-11T23:51:14.116608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116877+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:51:14.116877+0000","last_peered":"2024-09-11T23:51:14.116877+0000","last_clean":"2024-09-11T23:51:14.116877+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:51:14.116877+0000","last_undegraded":"2024-09-11T23:51:14.116877+0000","last_fullsized":"2024-09-11T23:51:14.116877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114791+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:51:14.114791+0000","last_peered":"2024-09-11T23:51:14.114791+0000","last_clean":"2024-09-11T23:51:14.114791+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:51:14.114791+0000","last_undegraded":"2024-09-11T23:51:14.114791+0000","last_fullsized":"2024-09-11T23:51:14.114791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":83,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:16.949269+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:51:16.949269+0000","last_peered":"2024-09-11T23:51:16.949269+0000","last_clean":"2024-09-11T23:51:16.949269+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:51:16.949269+0000","last_undegraded":"2024-09-11T23:51:16.949269+0000","last_fullsized":"2024-09-11T23:51:16.949269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117746+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:51:14.117746+0000","last_peered":"2024-09-11T23:51:14.117746+0000","last_clean":"2024-09-11T23:51:14.117746+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:51:14.117746+0000","last_undegraded":"2024-09-11T23:51:14.117746+0000","last_fullsized":"2024-09-11T23:51:14.117746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116551+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:51:14.116551+0000","last_peered":"2024-09-11T23:51:14.116551+0000","last_clean":"2024-09-11T23:51:14.116551+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:51:14.116551+0000","last_undegraded":"2024-09-11T23:51:14.116551+0000","last_fullsized":"2024-09-11T23:51:14.116551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107122+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:51:13.107122+0000","last_peered":"2024-09-11T23:51:13.107122+0000","last_clean":"2024-09-11T23:51:13.107122+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:51:13.107122+0000","last_undegraded":"2024-09-11T23:51:13.107122+0000","last_fullsized":"2024-09-11T23:51:13.107122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116331+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:51:14.116331+0000","last_peered":"2024-09-11T23:51:14.116331+0000","last_clean":"2024-09-11T23:51:14.116331+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:51:14.116331+0000","last_undegraded":"2024-09-11T23:51:14.116331+0000","last_fullsized":"2024-09-11T23:51:14.116331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117779+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:51:14.117779+0000","last_peered":"2024-09-11T23:51:14.117779+0000","last_clean":"2024-09-11T23:51:14.117779+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:51:14.117779+0000","last_undegraded":"2024-09-11T23:51:14.117779+0000","last_fullsized":"2024-09-11T23:51:14.117779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":176,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:17.953115+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:51:17.953115+0000","last_peered":"2024-09-11T23:51:17.953115+0000","last_clean":"2024-09-11T23:51:17.953115+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:51:17.953115+0000","last_undegraded":"2024-09-11T23:51:17.953115+0000","last_fullsized":"2024-09-11T23:51:17.953115+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":125,"num_read_kb":121,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117646+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:51:14.117646+0000","last_peered":"2024-09-11T23:51:14.117646+0000","last_clean":"2024-09-11T23:51:14.117646+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:51:14.117646+0000","last_undegraded":"2024-09-11T23:51:14.117646+0000","last_fullsized":"2024-09-11T23:51:14.117646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107018+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:51:13.107018+0000","last_peered":"2024-09-11T23:51:13.107018+0000","last_clean":"2024-09-11T23:51:13.107018+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:51:13.107018+0000","last_undegraded":"2024-09-11T23:51:13.107018+0000","last_fullsized":"2024-09-11T23:51:13.107018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114193+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:51:14.114193+0000","last_peered":"2024-09-11T23:51:14.114193+0000","last_clean":"2024-09-11T23:51:14.114193+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:51:14.114193+0000","last_undegraded":"2024-09-11T23:51:14.114193+0000","last_fullsized":"2024-09-11T23:51:14.114193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116707+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:51:14.116707+0000","last_peered":"2024-09-11T23:51:14.116707+0000","last_clean":"2024-09-11T23:51:14.116707+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:51:14.116707+0000","last_undegraded":"2024-09-11T23:51:14.116707+0000","last_fullsized":"2024-09-11T23:51:14.116707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116498+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:51:14.116498+0000","last_peered":"2024-09-11T23:51:14.116498+0000","last_clean":"2024-09-11T23:51:14.116498+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:51:14.116498+0000","last_undegraded":"2024-09-11T23:51:14.116498+0000","last_fullsized":"2024-09-11T23:51:14.116498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":81,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.103914+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:51:13.103914+0000","last_peered":"2024-09-11T23:51:13.103914+0000","last_clean":"2024-09-11T23:51:13.103914+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:51:13.103914+0000","last_undegraded":"2024-09-11T23:51:13.103914+0000","last_fullsized":"2024-09-11T23:51:13.103914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116150+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:51:14.116150+0000","last_peered":"2024-09-11T23:51:14.116150+0000","last_clean":"2024-09-11T23:51:14.116150+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:51:14.116150+0000","last_undegraded":"2024-09-11T23:51:14.116150+0000","last_fullsized":"2024-09-11T23:51:14.116150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117088+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:51:14.117088+0000","last_peered":"2024-09-11T23:51:14.117088+0000","last_clean":"2024-09-11T23:51:14.117088+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:51:14.117088+0000","last_undegraded":"2024-09-11T23:51:14.117088+0000","last_fullsized":"2024-09-11T23:51:14.117088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116158+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:51:14.116158+0000","last_peered":"2024-09-11T23:51:14.116158+0000","last_clean":"2024-09-11T23:51:14.116158+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:51:14.116158+0000","last_undegraded":"2024-09-11T23:51:14.116158+0000","last_fullsized":"2024-09-11T23:51:14.116158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117866+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:51:14.117866+0000","last_peered":"2024-09-11T23:51:14.117866+0000","last_clean":"2024-09-11T23:51:14.117866+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:51:14.117866+0000","last_undegraded":"2024-09-11T23:51:14.117866+0000","last_fullsized":"2024-09-11T23:51:14.117866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114175+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:51:14.114175+0000","last_peered":"2024-09-11T23:51:14.114175+0000","last_clean":"2024-09-11T23:51:14.114175+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:51:14.114175+0000","last_undegraded":"2024-09-11T23:51:14.114175+0000","last_fullsized":"2024-09-11T23:51:14.114175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.413526+0000","last_change":"2024-09-11T23:51:14.413526+0000","last_active":"2024-09-11T23:51:14.413526+0000","last_peered":"2024-09-11T23:51:14.413526+0000","last_clean":"2024-09-11T23:51:14.413526+0000","last_became_active":"2024-09-11T23:51:14.411027+0000","last_became_peered":"2024-09-11T23:51:14.411027+0000","last_unstale":"2024-09-11T23:51:14.413526+0000","last_undegraded":"2024-09-11T23:51:14.413526+0000","last_fullsized":"2024-09-11T23:51:14.413526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:55.020567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116641+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:51:14.116641+0000","last_peered":"2024-09-11T23:51:14.116641+0000","last_clean":"2024-09-11T23:51:14.116641+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:51:14.116641+0000","last_undegraded":"2024-09-11T23:51:14.116641+0000","last_fullsized":"2024-09-11T23:51:14.116641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116380+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:51:14.116380+0000","last_peered":"2024-09-11T23:51:14.116380+0000","last_clean":"2024-09-11T23:51:14.116380+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:51:14.116380+0000","last_undegraded":"2024-09-11T23:51:14.116380+0000","last_fullsized":"2024-09-11T23:51:14.116380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116489+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:51:14.116489+0000","last_peered":"2024-09-11T23:51:14.116489+0000","last_clean":"2024-09-11T23:51:14.116489+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:51:14.116489+0000","last_undegraded":"2024-09-11T23:51:14.116489+0000","last_fullsized":"2024-09-11T23:51:14.116489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107580+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:51:13.107580+0000","last_peered":"2024-09-11T23:51:13.107580+0000","last_clean":"2024-09-11T23:51:13.107580+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:51:13.107580+0000","last_undegraded":"2024-09-11T23:51:13.107580+0000","last_fullsized":"2024-09-11T23:51:13.107580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116847+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:51:14.116847+0000","last_peered":"2024-09-11T23:51:14.116847+0000","last_clean":"2024-09-11T23:51:14.116847+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:51:14.116847+0000","last_undegraded":"2024-09-11T23:51:14.116847+0000","last_fullsized":"2024-09-11T23:51:14.116847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116351+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:51:14.116351+0000","last_peered":"2024-09-11T23:51:14.116351+0000","last_clean":"2024-09-11T23:51:14.116351+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:51:14.116351+0000","last_undegraded":"2024-09-11T23:51:14.116351+0000","last_fullsized":"2024-09-11T23:51:14.116351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.104045+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:51:13.104045+0000","last_peered":"2024-09-11T23:51:13.104045+0000","last_clean":"2024-09-11T23:51:13.104045+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:51:13.104045+0000","last_undegraded":"2024-09-11T23:51:13.104045+0000","last_fullsized":"2024-09-11T23:51:13.104045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116836+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:51:14.116836+0000","last_peered":"2024-09-11T23:51:14.116836+0000","last_clean":"2024-09-11T23:51:14.116836+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:51:14.116836+0000","last_undegraded":"2024-09-11T23:51:14.116836+0000","last_fullsized":"2024-09-11T23:51:14.116836+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.122022+0000","last_change":"2024-09-11T23:51:14.121480+0000","last_active":"2024-09-11T23:51:14.122022+0000","last_peered":"2024-09-11T23:51:14.122022+0000","last_clean":"2024-09-11T23:51:14.122022+0000","last_became_active":"2024-09-11T23:51:14.118435+0000","last_became_peered":"2024-09-11T23:51:14.118435+0000","last_unstale":"2024-09-11T23:51:14.122022+0000","last_undegraded":"2024-09-11T23:51:14.122022+0000","last_fullsized":"2024-09-11T23:51:14.122022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114117+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:51:14.114117+0000","last_peered":"2024-09-11T23:51:14.114117+0000","last_clean":"2024-09-11T23:51:14.114117+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:51:14.114117+0000","last_undegraded":"2024-09-11T23:51:14.114117+0000","last_fullsized":"2024-09-11T23:51:14.114117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.946369+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:51:14.946369+0000","last_peered":"2024-09-11T23:51:14.946369+0000","last_clean":"2024-09-11T23:51:14.946369+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:51:14.946369+0000","last_undegraded":"2024-09-11T23:51:14.946369+0000","last_fullsized":"2024-09-11T23:51:14.946369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116740+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:51:14.116740+0000","last_peered":"2024-09-11T23:51:14.116740+0000","last_clean":"2024-09-11T23:51:14.116740+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:51:14.116740+0000","last_undegraded":"2024-09-11T23:51:14.116740+0000","last_fullsized":"2024-09-11T23:51:14.116740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116523+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:51:14.116523+0000","last_peered":"2024-09-11T23:51:14.116523+0000","last_clean":"2024-09-11T23:51:14.116523+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:51:14.116523+0000","last_undegraded":"2024-09-11T23:51:14.116523+0000","last_fullsized":"2024-09-11T23:51:14.116523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.409908+0000","last_change":"2024-09-11T23:51:14.409908+0000","last_active":"2024-09-11T23:51:14.409908+0000","last_peered":"2024-09-11T23:51:14.409908+0000","last_clean":"2024-09-11T23:51:14.409908+0000","last_became_active":"2024-09-11T23:51:14.409432+0000","last_became_peered":"2024-09-11T23:51:14.409432+0000","last_unstale":"2024-09-11T23:51:14.409908+0000","last_undegraded":"2024-09-11T23:51:14.409908+0000","last_fullsized":"2024-09-11T23:51:14.409908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:06.371306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114780+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:51:14.114780+0000","last_peered":"2024-09-11T23:51:14.114780+0000","last_clean":"2024-09-11T23:51:14.114780+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:51:14.114780+0000","last_undegraded":"2024-09-11T23:51:14.114780+0000","last_fullsized":"2024-09-11T23:51:14.114780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117129+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:51:14.117129+0000","last_peered":"2024-09-11T23:51:14.117129+0000","last_clean":"2024-09-11T23:51:14.117129+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:51:14.117129+0000","last_undegraded":"2024-09-11T23:51:14.117129+0000","last_fullsized":"2024-09-11T23:51:14.117129+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117793+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:51:14.117793+0000","last_peered":"2024-09-11T23:51:14.117793+0000","last_clean":"2024-09-11T23:51:14.117793+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:51:14.117793+0000","last_undegraded":"2024-09-11T23:51:14.117793+0000","last_fullsized":"2024-09-11T23:51:14.117793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.115193+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:51:14.115193+0000","last_peered":"2024-09-11T23:51:14.115193+0000","last_clean":"2024-09-11T23:51:14.115193+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:51:14.115193+0000","last_undegraded":"2024-09-11T23:51:14.115193+0000","last_fullsized":"2024-09-11T23:51:14.115193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'11","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114976+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:51:14.114976+0000","last_peered":"2024-09-11T23:51:14.114976+0000","last_clean":"2024-09-11T23:51:14.114976+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:51:14.114976+0000","last_undegraded":"2024-09-11T23:51:14.114976+0000","last_fullsized":"2024-09-11T23:51:14.114976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117309+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:51:14.117309+0000","last_peered":"2024-09-11T23:51:14.117309+0000","last_clean":"2024-09-11T23:51:14.117309+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:51:14.117309+0000","last_undegraded":"2024-09-11T23:51:14.117309+0000","last_fullsized":"2024-09-11T23:51:14.117309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.104483+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:51:13.104483+0000","last_peered":"2024-09-11T23:51:13.104483+0000","last_clean":"2024-09-11T23:51:13.104483+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:51:13.104483+0000","last_undegraded":"2024-09-11T23:51:13.104483+0000","last_fullsized":"2024-09-11T23:51:13.104483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.105995+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:51:13.105995+0000","last_peered":"2024-09-11T23:51:13.105995+0000","last_clean":"2024-09-11T23:51:13.105995+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:51:13.105995+0000","last_undegraded":"2024-09-11T23:51:13.105995+0000","last_fullsized":"2024-09-11T23:51:13.105995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117576+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:51:14.117576+0000","last_peered":"2024-09-11T23:51:14.117576+0000","last_clean":"2024-09-11T23:51:14.117576+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:51:14.117576+0000","last_undegraded":"2024-09-11T23:51:14.117576+0000","last_fullsized":"2024-09-11T23:51:14.117576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117114+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:51:14.117114+0000","last_peered":"2024-09-11T23:51:14.117114+0000","last_clean":"2024-09-11T23:51:14.117114+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:51:14.117114+0000","last_undegraded":"2024-09-11T23:51:14.117114+0000","last_fullsized":"2024-09-11T23:51:14.117114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117228+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:51:14.117228+0000","last_peered":"2024-09-11T23:51:14.117228+0000","last_clean":"2024-09-11T23:51:14.117228+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:51:14.117228+0000","last_undegraded":"2024-09-11T23:51:14.117228+0000","last_fullsized":"2024-09-11T23:51:14.117228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114121+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:51:14.114121+0000","last_peered":"2024-09-11T23:51:14.114121+0000","last_clean":"2024-09-11T23:51:14.114121+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:51:14.114121+0000","last_undegraded":"2024-09-11T23:51:14.114121+0000","last_fullsized":"2024-09-11T23:51:14.114121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.124072+0000","last_change":"2024-09-11T23:51:14.123663+0000","last_active":"2024-09-11T23:51:14.124072+0000","last_peered":"2024-09-11T23:51:14.124072+0000","last_clean":"2024-09-11T23:51:14.124072+0000","last_became_active":"2024-09-11T23:51:14.118315+0000","last_became_peered":"2024-09-11T23:51:14.118315+0000","last_unstale":"2024-09-11T23:51:14.124072+0000","last_undegraded":"2024-09-11T23:51:14.124072+0000","last_fullsized":"2024-09-11T23:51:14.124072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:10.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116860+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:51:14.116860+0000","last_peered":"2024-09-11T23:51:14.116860+0000","last_clean":"2024-09-11T23:51:14.116860+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:51:14.116860+0000","last_undegraded":"2024-09-11T23:51:14.116860+0000","last_fullsized":"2024-09-11T23:51:14.116860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116980+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:51:14.116980+0000","last_peered":"2024-09-11T23:51:14.116980+0000","last_clean":"2024-09-11T23:51:14.116980+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:51:14.116980+0000","last_undegraded":"2024-09-11T23:51:14.116980+0000","last_fullsized":"2024-09-11T23:51:14.116980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116368+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:51:14.116368+0000","last_peered":"2024-09-11T23:51:14.116368+0000","last_clean":"2024-09-11T23:51:14.116368+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:51:14.116368+0000","last_undegraded":"2024-09-11T23:51:14.116368+0000","last_fullsized":"2024-09-11T23:51:14.116368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116161+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:51:14.116161+0000","last_peered":"2024-09-11T23:51:14.116161+0000","last_clean":"2024-09-11T23:51:14.116161+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:51:14.116161+0000","last_undegraded":"2024-09-11T23:51:14.116161+0000","last_fullsized":"2024-09-11T23:51:14.116161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117388+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:51:14.117388+0000","last_peered":"2024-09-11T23:51:14.117388+0000","last_clean":"2024-09-11T23:51:14.117388+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:51:14.117388+0000","last_undegraded":"2024-09-11T23:51:14.117388+0000","last_fullsized":"2024-09-11T23:51:14.117388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117643+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:51:14.117643+0000","last_peered":"2024-09-11T23:51:14.117643+0000","last_clean":"2024-09-11T23:51:14.117643+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:51:14.117643+0000","last_undegraded":"2024-09-11T23:51:14.117643+0000","last_fullsized":"2024-09-11T23:51:14.117643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117655+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:51:14.117655+0000","last_peered":"2024-09-11T23:51:14.117655+0000","last_clean":"2024-09-11T23:51:14.117655+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:51:14.117655+0000","last_undegraded":"2024-09-11T23:51:14.117655+0000","last_fullsized":"2024-09-11T23:51:14.117655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:51:13.107072+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:51:13.107072+0000","last_peered":"2024-09-11T23:51:13.107072+0000","last_clean":"2024-09-11T23:51:13.107072+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:51:13.107072+0000","last_undegraded":"2024-09-11T23:51:13.107072+0000","last_fullsized":"2024-09-11T23:51:13.107072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117360+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:51:14.117360+0000","last_peered":"2024-09-11T23:51:14.117360+0000","last_clean":"2024-09-11T23:51:14.117360+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:51:14.117360+0000","last_undegraded":"2024-09-11T23:51:14.117360+0000","last_fullsized":"2024-09-11T23:51:14.117360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117186+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:51:14.117186+0000","last_peered":"2024-09-11T23:51:14.117186+0000","last_clean":"2024-09-11T23:51:14.117186+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:51:14.117186+0000","last_undegraded":"2024-09-11T23:51:14.117186+0000","last_fullsized":"2024-09-11T23:51:14.117186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.125596+0000","last_change":"2024-09-11T23:51:14.125596+0000","last_active":"2024-09-11T23:51:14.125596+0000","last_peered":"2024-09-11T23:51:14.125596+0000","last_clean":"2024-09-11T23:51:14.125596+0000","last_became_active":"2024-09-11T23:51:14.119682+0000","last_became_peered":"2024-09-11T23:51:14.119682+0000","last_unstale":"2024-09-11T23:51:14.125596+0000","last_undegraded":"2024-09-11T23:51:14.125596+0000","last_fullsized":"2024-09-11T23:51:14.125596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:06.062602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65175,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":367,"num_read_kb":254,"num_write":163,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281926,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":436969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":1090015,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939686,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":437205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168618,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":436543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":962560,"data_stored":1029057,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":436594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":1028348,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":892928,"data_stored":428973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":436905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":490458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":966656,"data_stored":498141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":909312,"data_stored":429414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:51:21.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:21 smithi083 ceph-mon[34805]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:21.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:21 smithi083 ceph-mon[34805]: Health check update: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T23:51:21.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:21 smithi062 ceph-mon[30958]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:21.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:21 smithi062 ceph-mon[30958]: Health check update: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T23:51:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:21 smithi120 ceph-mon[35182]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:21 smithi120 ceph-mon[35182]: Health check update: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T23:51:22.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:22 smithi083 ceph-mon[34805]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-09-11T23:51:22.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:22 smithi083 ceph-mon[34805]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T23:51:22.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:22 smithi083 ceph-mon[34805]: Cluster is now healthy 2024-09-11T23:51:22.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:22 smithi062 ceph-mon[30958]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-09-11T23:51:22.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:22 smithi062 ceph-mon[30958]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T23:51:22.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:22 smithi062 ceph-mon[30958]: Cluster is now healthy 2024-09-11T23:51:22.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:22 smithi120 ceph-mon[35182]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-09-11T23:51:22.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:22 smithi120 ceph-mon[35182]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T23:51:22.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:22 smithi120 ceph-mon[35182]: Cluster is now healthy 2024-09-11T23:51:23.887 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:51:24.160 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:24.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:24 smithi062 ceph-mon[30958]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 59 B/s, 0 objects/s recovering 2024-09-11T23:51:24.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:24 smithi083 ceph-mon[34805]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 59 B/s, 0 objects/s recovering 2024-09-11T23:51:24.423 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:24 smithi120 ceph-mon[35182]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 59 B/s, 0 objects/s recovering 2024-09-11T23:51:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.656+0000 7f8a3665e640 1 -- 172.21.15.62:0/3760578197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a301027a0 msgr2=0x7f8a301067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.656+0000 7f8a3665e640 1 --2- 172.21.15.62:0/3760578197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a301027a0 0x7f8a301067f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009900 tx=0x7f8a1802a410 comp rx=0 tx=0).stop 2024-09-11T23:51:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.656+0000 7f8a3665e640 1 -- 172.21.15.62:0/3760578197 shutdown_connections 2024-09-11T23:51:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.656+0000 7f8a3665e640 1 --2- 172.21.15.62:0/3760578197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a30106f20 0x7f8a3010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.656+0000 7f8a3665e640 1 --2- 172.21.15.62:0/3760578197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a301027a0 0x7f8a301067f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.656+0000 7f8a3665e640 1 --2- 172.21.15.62:0/3760578197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a30101df0 0x7f8a301021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.656+0000 7f8a3665e640 1 -- 172.21.15.62:0/3760578197 >> 172.21.15.62:0/3760578197 conn(0x7f8a300fd8b0 msgr2=0x7f8a300ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.656+0000 7f8a3665e640 1 -- 172.21.15.62:0/3760578197 shutdown_connections 2024-09-11T23:51:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.656+0000 7f8a3665e640 1 -- 172.21.15.62:0/3760578197 wait complete. 2024-09-11T23:51:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a3665e640 1 Processor -- start 2024-09-11T23:51:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a3665e640 1 -- start start 2024-09-11T23:51:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a3665e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a30101df0 0x7f8a3019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a3665e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a301027a0 0x7f8a3019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a3665e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a30106f20 0x7f8a3019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a3665e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30111ff0 con 0x7f8a30106f20 2024-09-11T23:51:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a3665e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30111e70 con 0x7f8a30101df0 2024-09-11T23:51:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a3665e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30112170 con 0x7f8a301027a0 2024-09-11T23:51:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a34bfb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a30106f20 0x7f8a3019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a34bfb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a30106f20 0x7f8a3019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58304/0 (socket says 172.21.15.62:58304) 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a34bfb640 1 -- 172.21.15.62:0/1969212809 learned_addr learned my addr 172.21.15.62:0/1969212809 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.657+0000 7f8a2ffff640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a30101df0 0x7f8a3019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a34bfb640 1 -- 172.21.15.62:0/1969212809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a301027a0 msgr2=0x7f8a3019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a34bfb640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a301027a0 0x7f8a3019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a34bfb640 1 -- 172.21.15.62:0/1969212809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a30101df0 msgr2=0x7f8a3019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a34bfb640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a30101df0 0x7f8a3019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a34bfb640 1 -- 172.21.15.62:0/1969212809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3019f610 con 0x7f8a30106f20 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a34bfb640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a30106f20 0x7f8a3019ef30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800cce0 tx=0x7f8a2800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a2dffb640 1 -- 172.21.15.62:0/1969212809 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2800f450 con 0x7f8a30106f20 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a3665e640 1 -- 172.21.15.62:0/1969212809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3019f900 con 0x7f8a30106f20 2024-09-11T23:51:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a2dffb640 1 -- 172.21.15.62:0/1969212809 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a2800fad0 con 0x7f8a30106f20 2024-09-11T23:51:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a2dffb640 1 -- 172.21.15.62:0/1969212809 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28018b60 con 0x7f8a30106f20 2024-09-11T23:51:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.658+0000 7f8a3665e640 1 -- 172.21.15.62:0/1969212809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a301a71e0 con 0x7f8a30106f20 2024-09-11T23:51:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.659+0000 7f8a3665e640 1 -- 172.21.15.62:0/1969212809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a301021d0 con 0x7f8a30106f20 2024-09-11T23:51:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.660+0000 7f8a2dffb640 1 -- 172.21.15.62:0/1969212809 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8a2800fc80 con 0x7f8a30106f20 2024-09-11T23:51:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.661+0000 7f8a2dffb640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a1c0a2c10 0x7f8a1c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.661+0000 7f8a2ffff640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a1c0a2c10 0x7f8a1c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.661+0000 7f8a2dffb640 1 -- 172.21.15.62:0/1969212809 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8a280cf6f0 con 0x7f8a30106f20 2024-09-11T23:51:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.661+0000 7f8a2ffff640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a1c0a2c10 0x7f8a1c0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009990 tx=0x7f8a20009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.665+0000 7f8a2dffb640 1 -- 172.21.15.62:0/1969212809 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a28014030 con 0x7f8a30106f20 2024-09-11T23:51:24.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.836+0000 7f8a3665e640 1 -- 172.21.15.62:0/1969212809 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a3010de10 con 0x7f8a1c0a2c10 2024-09-11T23:51:24.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.843+0000 7f8a2dffb640 1 -- 172.21.15.62:0/1969212809 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258879 (secure 0 0 0) 0x7f8a3010de10 con 0x7f8a1c0a2c10 2024-09-11T23:51:24.844 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:24.845 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:51:24.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 -- 172.21.15.62:0/1969212809 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a1c0a2c10 msgr2=0x7f8a1c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:24.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a1c0a2c10 0x7f8a1c0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009990 tx=0x7f8a20009110 comp rx=0 tx=0).stop 2024-09-11T23:51:24.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 -- 172.21.15.62:0/1969212809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a30106f20 msgr2=0x7f8a3019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:24.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a30106f20 0x7f8a3019ef30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800cce0 tx=0x7f8a2800a620 comp rx=0 tx=0).stop 2024-09-11T23:51:24.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 -- 172.21.15.62:0/1969212809 shutdown_connections 2024-09-11T23:51:24.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f8a1c0a2c10 0x7f8a1c0a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:24.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a30106f20 0x7f8a3019ef30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:24.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a301027a0 0x7f8a3019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:24.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 --2- 172.21.15.62:0/1969212809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a30101df0 0x7f8a3019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:24.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 -- 172.21.15.62:0/1969212809 >> 172.21.15.62:0/1969212809 conn(0x7f8a300fd8b0 msgr2=0x7f8a300fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:24.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 -- 172.21.15.62:0/1969212809 shutdown_connections 2024-09-11T23:51:24.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:24.846+0000 7f8a3665e640 1 -- 172.21.15.62:0/1969212809 wait complete. 2024-09-11T23:51:25.492 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-11T23:51:24.818597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655777,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":538,"num_read_kb":388,"num_write":346,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347880,"kb_used_data":11040,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593944,"statfs":{"total":1151940427776,"available":1151584198656,"internally_reserved":0,"allocated":11304960,"data_stored":7203975,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.705419"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114410+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:51:14.114410+0000","last_peered":"2024-09-11T23:51:14.114410+0000","last_clean":"2024-09-11T23:51:14.114410+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:51:14.114410+0000","last_undegraded":"2024-09-11T23:51:14.114410+0000","last_fullsized":"2024-09-11T23:51:14.114410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.408869+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:51:14.408869+0000","last_peered":"2024-09-11T23:51:14.408869+0000","last_clean":"2024-09-11T23:51:14.408869+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:51:14.408869+0000","last_undegraded":"2024-09-11T23:51:14.408869+0000","last_fullsized":"2024-09-11T23:51:14.408869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.408668+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:51:14.408668+0000","last_peered":"2024-09-11T23:51:14.408668+0000","last_clean":"2024-09-11T23:51:14.408668+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:51:14.408668+0000","last_undegraded":"2024-09-11T23:51:14.408668+0000","last_fullsized":"2024-09-11T23:51:14.408668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.410077+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:51:14.410077+0000","last_peered":"2024-09-11T23:51:14.410077+0000","last_clean":"2024-09-11T23:51:14.410077+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:51:14.410077+0000","last_undegraded":"2024-09-11T23:51:14.410077+0000","last_fullsized":"2024-09-11T23:51:14.410077+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116864+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:51:14.116864+0000","last_peered":"2024-09-11T23:51:14.116864+0000","last_clean":"2024-09-11T23:51:14.116864+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:51:14.116864+0000","last_undegraded":"2024-09-11T23:51:14.116864+0000","last_fullsized":"2024-09-11T23:51:14.116864+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.120226+0000","last_change":"2024-09-11T23:51:14.120226+0000","last_active":"2024-09-11T23:51:14.120226+0000","last_peered":"2024-09-11T23:51:14.120226+0000","last_clean":"2024-09-11T23:51:14.120226+0000","last_became_active":"2024-09-11T23:51:14.119811+0000","last_became_peered":"2024-09-11T23:51:14.119811+0000","last_unstale":"2024-09-11T23:51:14.120226+0000","last_undegraded":"2024-09-11T23:51:14.120226+0000","last_fullsized":"2024-09-11T23:51:14.120226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:58.851476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116252+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:51:14.116252+0000","last_peered":"2024-09-11T23:51:14.116252+0000","last_clean":"2024-09-11T23:51:14.116252+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:51:14.116252+0000","last_undegraded":"2024-09-11T23:51:14.116252+0000","last_fullsized":"2024-09-11T23:51:14.116252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116477+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:51:14.116477+0000","last_peered":"2024-09-11T23:51:14.116477+0000","last_clean":"2024-09-11T23:51:14.116477+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:51:14.116477+0000","last_undegraded":"2024-09-11T23:51:14.116477+0000","last_fullsized":"2024-09-11T23:51:14.116477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.118918+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:51:14.118918+0000","last_peered":"2024-09-11T23:51:14.118918+0000","last_clean":"2024-09-11T23:51:14.118918+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:51:14.118918+0000","last_undegraded":"2024-09-11T23:51:14.118918+0000","last_fullsized":"2024-09-11T23:51:14.118918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117507+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:51:14.117507+0000","last_peered":"2024-09-11T23:51:14.117507+0000","last_clean":"2024-09-11T23:51:14.117507+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:51:14.117507+0000","last_undegraded":"2024-09-11T23:51:14.117507+0000","last_fullsized":"2024-09-11T23:51:14.117507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116742+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:51:14.116742+0000","last_peered":"2024-09-11T23:51:14.116742+0000","last_clean":"2024-09-11T23:51:14.116742+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:51:14.116742+0000","last_undegraded":"2024-09-11T23:51:14.116742+0000","last_fullsized":"2024-09-11T23:51:14.116742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.410210+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:51:14.410210+0000","last_peered":"2024-09-11T23:51:14.410210+0000","last_clean":"2024-09-11T23:51:14.410210+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:51:14.410210+0000","last_undegraded":"2024-09-11T23:51:14.410210+0000","last_fullsized":"2024-09-11T23:51:14.410210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114515+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:51:14.114515+0000","last_peered":"2024-09-11T23:51:14.114515+0000","last_clean":"2024-09-11T23:51:14.114515+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:51:14.114515+0000","last_undegraded":"2024-09-11T23:51:14.114515+0000","last_fullsized":"2024-09-11T23:51:14.114515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.408639+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:51:14.408639+0000","last_peered":"2024-09-11T23:51:14.408639+0000","last_clean":"2024-09-11T23:51:14.408639+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:51:14.408639+0000","last_undegraded":"2024-09-11T23:51:14.408639+0000","last_fullsized":"2024-09-11T23:51:14.408639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.408508+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:51:14.408508+0000","last_peered":"2024-09-11T23:51:14.408508+0000","last_clean":"2024-09-11T23:51:14.408508+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:51:14.408508+0000","last_undegraded":"2024-09-11T23:51:14.408508+0000","last_fullsized":"2024-09-11T23:51:14.408508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117381+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:51:14.117381+0000","last_peered":"2024-09-11T23:51:14.117381+0000","last_clean":"2024-09-11T23:51:14.117381+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:51:14.117381+0000","last_undegraded":"2024-09-11T23:51:14.117381+0000","last_fullsized":"2024-09-11T23:51:14.117381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114431+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:51:14.114431+0000","last_peered":"2024-09-11T23:51:14.114431+0000","last_clean":"2024-09-11T23:51:14.114431+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:51:14.114431+0000","last_undegraded":"2024-09-11T23:51:14.114431+0000","last_fullsized":"2024-09-11T23:51:14.114431+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.115067+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:51:14.115067+0000","last_peered":"2024-09-11T23:51:14.115067+0000","last_clean":"2024-09-11T23:51:14.115067+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:51:14.115067+0000","last_undegraded":"2024-09-11T23:51:14.115067+0000","last_fullsized":"2024-09-11T23:51:14.115067+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.112315+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:51:14.112315+0000","last_peered":"2024-09-11T23:51:14.112315+0000","last_clean":"2024-09-11T23:51:14.112315+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:51:14.112315+0000","last_undegraded":"2024-09-11T23:51:14.112315+0000","last_fullsized":"2024-09-11T23:51:14.112315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.113934+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:51:14.113934+0000","last_peered":"2024-09-11T23:51:14.113934+0000","last_clean":"2024-09-11T23:51:14.113934+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:51:14.113934+0000","last_undegraded":"2024-09-11T23:51:14.113934+0000","last_fullsized":"2024-09-11T23:51:14.113934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114619+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:51:14.114619+0000","last_peered":"2024-09-11T23:51:14.114619+0000","last_clean":"2024-09-11T23:51:14.114619+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:51:14.114619+0000","last_undegraded":"2024-09-11T23:51:14.114619+0000","last_fullsized":"2024-09-11T23:51:14.114619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117986+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:51:14.117986+0000","last_peered":"2024-09-11T23:51:14.117986+0000","last_clean":"2024-09-11T23:51:14.117986+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:51:14.117986+0000","last_undegraded":"2024-09-11T23:51:14.117986+0000","last_fullsized":"2024-09-11T23:51:14.117986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116716+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:51:14.116716+0000","last_peered":"2024-09-11T23:51:14.116716+0000","last_clean":"2024-09-11T23:51:14.116716+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:51:14.116716+0000","last_undegraded":"2024-09-11T23:51:14.116716+0000","last_fullsized":"2024-09-11T23:51:14.116716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116635+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:51:14.116635+0000","last_peered":"2024-09-11T23:51:14.116635+0000","last_clean":"2024-09-11T23:51:14.116635+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:51:14.116635+0000","last_undegraded":"2024-09-11T23:51:14.116635+0000","last_fullsized":"2024-09-11T23:51:14.116635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117214+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:51:14.117214+0000","last_peered":"2024-09-11T23:51:14.117214+0000","last_clean":"2024-09-11T23:51:14.117214+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:51:14.117214+0000","last_undegraded":"2024-09-11T23:51:14.117214+0000","last_fullsized":"2024-09-11T23:51:14.117214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.112378+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:51:14.112378+0000","last_peered":"2024-09-11T23:51:14.112378+0000","last_clean":"2024-09-11T23:51:14.112378+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:51:14.112378+0000","last_undegraded":"2024-09-11T23:51:14.112378+0000","last_fullsized":"2024-09-11T23:51:14.112378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114121+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:51:14.114121+0000","last_peered":"2024-09-11T23:51:14.114121+0000","last_clean":"2024-09-11T23:51:14.114121+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:51:14.114121+0000","last_undegraded":"2024-09-11T23:51:14.114121+0000","last_fullsized":"2024-09-11T23:51:14.114121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.410108+0000","last_change":"2024-09-11T23:51:14.410108+0000","last_active":"2024-09-11T23:51:14.410108+0000","last_peered":"2024-09-11T23:51:14.410108+0000","last_clean":"2024-09-11T23:51:14.410108+0000","last_became_active":"2024-09-11T23:51:14.409588+0000","last_became_peered":"2024-09-11T23:51:14.409588+0000","last_unstale":"2024-09-11T23:51:14.410108+0000","last_undegraded":"2024-09-11T23:51:14.410108+0000","last_fullsized":"2024-09-11T23:51:14.410108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:29.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.408824+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:51:14.408824+0000","last_peered":"2024-09-11T23:51:14.408824+0000","last_clean":"2024-09-11T23:51:14.408824+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:51:14.408824+0000","last_undegraded":"2024-09-11T23:51:14.408824+0000","last_fullsized":"2024-09-11T23:51:14.408824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.112067+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:51:14.112067+0000","last_peered":"2024-09-11T23:51:14.112067+0000","last_clean":"2024-09-11T23:51:14.112067+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:51:14.112067+0000","last_undegraded":"2024-09-11T23:51:14.112067+0000","last_fullsized":"2024-09-11T23:51:14.112067+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.410872+0000","last_change":"2024-09-11T23:51:14.410872+0000","last_active":"2024-09-11T23:51:14.410872+0000","last_peered":"2024-09-11T23:51:14.410872+0000","last_clean":"2024-09-11T23:51:14.410872+0000","last_became_active":"2024-09-11T23:51:14.410531+0000","last_became_peered":"2024-09-11T23:51:14.410531+0000","last_unstale":"2024-09-11T23:51:14.410872+0000","last_undegraded":"2024-09-11T23:51:14.410872+0000","last_fullsized":"2024-09-11T23:51:14.410872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:10.502914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.113981+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:51:14.113981+0000","last_peered":"2024-09-11T23:51:14.113981+0000","last_clean":"2024-09-11T23:51:14.113981+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:51:14.113981+0000","last_undegraded":"2024-09-11T23:51:14.113981+0000","last_fullsized":"2024-09-11T23:51:14.113981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117356+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:51:14.117356+0000","last_peered":"2024-09-11T23:51:14.117356+0000","last_clean":"2024-09-11T23:51:14.117356+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:51:14.117356+0000","last_undegraded":"2024-09-11T23:51:14.117356+0000","last_fullsized":"2024-09-11T23:51:14.117356+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":89,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:18.952628+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:51:18.952628+0000","last_peered":"2024-09-11T23:51:18.952628+0000","last_clean":"2024-09-11T23:51:18.952628+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:51:18.952628+0000","last_undegraded":"2024-09-11T23:51:18.952628+0000","last_fullsized":"2024-09-11T23:51:18.952628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116157+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:51:14.116157+0000","last_peered":"2024-09-11T23:51:14.116157+0000","last_clean":"2024-09-11T23:51:14.116157+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:51:14.116157+0000","last_undegraded":"2024-09-11T23:51:14.116157+0000","last_fullsized":"2024-09-11T23:51:14.116157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.118252+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:51:14.118252+0000","last_peered":"2024-09-11T23:51:14.118252+0000","last_clean":"2024-09-11T23:51:14.118252+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:51:14.118252+0000","last_undegraded":"2024-09-11T23:51:14.118252+0000","last_fullsized":"2024-09-11T23:51:14.118252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114143+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:51:14.114143+0000","last_peered":"2024-09-11T23:51:14.114143+0000","last_clean":"2024-09-11T23:51:14.114143+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:51:14.114143+0000","last_undegraded":"2024-09-11T23:51:14.114143+0000","last_fullsized":"2024-09-11T23:51:14.114143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116608+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:51:14.116608+0000","last_peered":"2024-09-11T23:51:14.116608+0000","last_clean":"2024-09-11T23:51:14.116608+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:51:14.116608+0000","last_undegraded":"2024-09-11T23:51:14.116608+0000","last_fullsized":"2024-09-11T23:51:14.116608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116877+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:51:14.116877+0000","last_peered":"2024-09-11T23:51:14.116877+0000","last_clean":"2024-09-11T23:51:14.116877+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:51:14.116877+0000","last_undegraded":"2024-09-11T23:51:14.116877+0000","last_fullsized":"2024-09-11T23:51:14.116877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114791+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:51:14.114791+0000","last_peered":"2024-09-11T23:51:14.114791+0000","last_clean":"2024-09-11T23:51:14.114791+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:51:14.114791+0000","last_undegraded":"2024-09-11T23:51:14.114791+0000","last_fullsized":"2024-09-11T23:51:14.114791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":88,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:21.955824+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:51:21.955824+0000","last_peered":"2024-09-11T23:51:21.955824+0000","last_clean":"2024-09-11T23:51:21.955824+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:51:21.955824+0000","last_undegraded":"2024-09-11T23:51:21.955824+0000","last_fullsized":"2024-09-11T23:51:21.955824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117746+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:51:14.117746+0000","last_peered":"2024-09-11T23:51:14.117746+0000","last_clean":"2024-09-11T23:51:14.117746+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:51:14.117746+0000","last_undegraded":"2024-09-11T23:51:14.117746+0000","last_fullsized":"2024-09-11T23:51:14.117746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116551+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:51:14.116551+0000","last_peered":"2024-09-11T23:51:14.116551+0000","last_clean":"2024-09-11T23:51:14.116551+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:51:14.116551+0000","last_undegraded":"2024-09-11T23:51:14.116551+0000","last_fullsized":"2024-09-11T23:51:14.116551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.409076+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:51:14.409076+0000","last_peered":"2024-09-11T23:51:14.409076+0000","last_clean":"2024-09-11T23:51:14.409076+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:51:14.409076+0000","last_undegraded":"2024-09-11T23:51:14.409076+0000","last_fullsized":"2024-09-11T23:51:14.409076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116331+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:51:14.116331+0000","last_peered":"2024-09-11T23:51:14.116331+0000","last_clean":"2024-09-11T23:51:14.116331+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:51:14.116331+0000","last_undegraded":"2024-09-11T23:51:14.116331+0000","last_fullsized":"2024-09-11T23:51:14.116331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117779+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:51:14.117779+0000","last_peered":"2024-09-11T23:51:14.117779+0000","last_clean":"2024-09-11T23:51:14.117779+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:51:14.117779+0000","last_undegraded":"2024-09-11T23:51:14.117779+0000","last_fullsized":"2024-09-11T23:51:14.117779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":186,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:22.959029+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:51:22.959029+0000","last_peered":"2024-09-11T23:51:22.959029+0000","last_clean":"2024-09-11T23:51:22.959029+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:51:22.959029+0000","last_undegraded":"2024-09-11T23:51:22.959029+0000","last_fullsized":"2024-09-11T23:51:22.959029+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":135,"num_read_kb":126,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117646+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:51:14.117646+0000","last_peered":"2024-09-11T23:51:14.117646+0000","last_clean":"2024-09-11T23:51:14.117646+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:51:14.117646+0000","last_undegraded":"2024-09-11T23:51:14.117646+0000","last_fullsized":"2024-09-11T23:51:14.117646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.112191+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:51:14.112191+0000","last_peered":"2024-09-11T23:51:14.112191+0000","last_clean":"2024-09-11T23:51:14.112191+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:51:14.112191+0000","last_undegraded":"2024-09-11T23:51:14.112191+0000","last_fullsized":"2024-09-11T23:51:14.112191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114193+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:51:14.114193+0000","last_peered":"2024-09-11T23:51:14.114193+0000","last_clean":"2024-09-11T23:51:14.114193+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:51:14.114193+0000","last_undegraded":"2024-09-11T23:51:14.114193+0000","last_fullsized":"2024-09-11T23:51:14.114193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116707+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:51:14.116707+0000","last_peered":"2024-09-11T23:51:14.116707+0000","last_clean":"2024-09-11T23:51:14.116707+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:51:14.116707+0000","last_undegraded":"2024-09-11T23:51:14.116707+0000","last_fullsized":"2024-09-11T23:51:14.116707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116498+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:51:14.116498+0000","last_peered":"2024-09-11T23:51:14.116498+0000","last_clean":"2024-09-11T23:51:14.116498+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:51:14.116498+0000","last_undegraded":"2024-09-11T23:51:14.116498+0000","last_fullsized":"2024-09-11T23:51:14.116498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:22.957504+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:51:22.957504+0000","last_peered":"2024-09-11T23:51:22.957504+0000","last_clean":"2024-09-11T23:51:22.957504+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:51:22.957504+0000","last_undegraded":"2024-09-11T23:51:22.957504+0000","last_fullsized":"2024-09-11T23:51:22.957504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116150+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:51:14.116150+0000","last_peered":"2024-09-11T23:51:14.116150+0000","last_clean":"2024-09-11T23:51:14.116150+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:51:14.116150+0000","last_undegraded":"2024-09-11T23:51:14.116150+0000","last_fullsized":"2024-09-11T23:51:14.116150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117088+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:51:14.117088+0000","last_peered":"2024-09-11T23:51:14.117088+0000","last_clean":"2024-09-11T23:51:14.117088+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:51:14.117088+0000","last_undegraded":"2024-09-11T23:51:14.117088+0000","last_fullsized":"2024-09-11T23:51:14.117088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116158+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:51:14.116158+0000","last_peered":"2024-09-11T23:51:14.116158+0000","last_clean":"2024-09-11T23:51:14.116158+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:51:14.116158+0000","last_undegraded":"2024-09-11T23:51:14.116158+0000","last_fullsized":"2024-09-11T23:51:14.116158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117866+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:51:14.117866+0000","last_peered":"2024-09-11T23:51:14.117866+0000","last_clean":"2024-09-11T23:51:14.117866+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:51:14.117866+0000","last_undegraded":"2024-09-11T23:51:14.117866+0000","last_fullsized":"2024-09-11T23:51:14.117866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114175+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:51:14.114175+0000","last_peered":"2024-09-11T23:51:14.114175+0000","last_clean":"2024-09-11T23:51:14.114175+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:51:14.114175+0000","last_undegraded":"2024-09-11T23:51:14.114175+0000","last_fullsized":"2024-09-11T23:51:14.114175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.413526+0000","last_change":"2024-09-11T23:51:14.413526+0000","last_active":"2024-09-11T23:51:14.413526+0000","last_peered":"2024-09-11T23:51:14.413526+0000","last_clean":"2024-09-11T23:51:14.413526+0000","last_became_active":"2024-09-11T23:51:14.411027+0000","last_became_peered":"2024-09-11T23:51:14.411027+0000","last_unstale":"2024-09-11T23:51:14.413526+0000","last_undegraded":"2024-09-11T23:51:14.413526+0000","last_fullsized":"2024-09-11T23:51:14.413526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:55.020567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116641+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:51:14.116641+0000","last_peered":"2024-09-11T23:51:14.116641+0000","last_clean":"2024-09-11T23:51:14.116641+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:51:14.116641+0000","last_undegraded":"2024-09-11T23:51:14.116641+0000","last_fullsized":"2024-09-11T23:51:14.116641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116380+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:51:14.116380+0000","last_peered":"2024-09-11T23:51:14.116380+0000","last_clean":"2024-09-11T23:51:14.116380+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:51:14.116380+0000","last_undegraded":"2024-09-11T23:51:14.116380+0000","last_fullsized":"2024-09-11T23:51:14.116380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116489+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:51:14.116489+0000","last_peered":"2024-09-11T23:51:14.116489+0000","last_clean":"2024-09-11T23:51:14.116489+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:51:14.116489+0000","last_undegraded":"2024-09-11T23:51:14.116489+0000","last_fullsized":"2024-09-11T23:51:14.116489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.112589+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:51:14.112589+0000","last_peered":"2024-09-11T23:51:14.112589+0000","last_clean":"2024-09-11T23:51:14.112589+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:51:14.112589+0000","last_undegraded":"2024-09-11T23:51:14.112589+0000","last_fullsized":"2024-09-11T23:51:14.112589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116847+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:51:14.116847+0000","last_peered":"2024-09-11T23:51:14.116847+0000","last_clean":"2024-09-11T23:51:14.116847+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:51:14.116847+0000","last_undegraded":"2024-09-11T23:51:14.116847+0000","last_fullsized":"2024-09-11T23:51:14.116847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116351+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:51:14.116351+0000","last_peered":"2024-09-11T23:51:14.116351+0000","last_clean":"2024-09-11T23:51:14.116351+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:51:14.116351+0000","last_undegraded":"2024-09-11T23:51:14.116351+0000","last_fullsized":"2024-09-11T23:51:14.116351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.410190+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:51:14.410190+0000","last_peered":"2024-09-11T23:51:14.410190+0000","last_clean":"2024-09-11T23:51:14.410190+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:51:14.410190+0000","last_undegraded":"2024-09-11T23:51:14.410190+0000","last_fullsized":"2024-09-11T23:51:14.410190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116836+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:51:14.116836+0000","last_peered":"2024-09-11T23:51:14.116836+0000","last_clean":"2024-09-11T23:51:14.116836+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:51:14.116836+0000","last_undegraded":"2024-09-11T23:51:14.116836+0000","last_fullsized":"2024-09-11T23:51:14.116836+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.122022+0000","last_change":"2024-09-11T23:51:14.121480+0000","last_active":"2024-09-11T23:51:14.122022+0000","last_peered":"2024-09-11T23:51:14.122022+0000","last_clean":"2024-09-11T23:51:14.122022+0000","last_became_active":"2024-09-11T23:51:14.118435+0000","last_became_peered":"2024-09-11T23:51:14.118435+0000","last_unstale":"2024-09-11T23:51:14.122022+0000","last_undegraded":"2024-09-11T23:51:14.122022+0000","last_fullsized":"2024-09-11T23:51:14.122022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114117+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:51:14.114117+0000","last_peered":"2024-09-11T23:51:14.114117+0000","last_clean":"2024-09-11T23:51:14.114117+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:51:14.114117+0000","last_undegraded":"2024-09-11T23:51:14.114117+0000","last_fullsized":"2024-09-11T23:51:14.114117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":87,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:19.952924+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:51:19.952924+0000","last_peered":"2024-09-11T23:51:19.952924+0000","last_clean":"2024-09-11T23:51:19.952924+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:51:19.952924+0000","last_undegraded":"2024-09-11T23:51:19.952924+0000","last_fullsized":"2024-09-11T23:51:19.952924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116740+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:51:14.116740+0000","last_peered":"2024-09-11T23:51:14.116740+0000","last_clean":"2024-09-11T23:51:14.116740+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:51:14.116740+0000","last_undegraded":"2024-09-11T23:51:14.116740+0000","last_fullsized":"2024-09-11T23:51:14.116740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116523+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:51:14.116523+0000","last_peered":"2024-09-11T23:51:14.116523+0000","last_clean":"2024-09-11T23:51:14.116523+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:51:14.116523+0000","last_undegraded":"2024-09-11T23:51:14.116523+0000","last_fullsized":"2024-09-11T23:51:14.116523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.409908+0000","last_change":"2024-09-11T23:51:14.409908+0000","last_active":"2024-09-11T23:51:14.409908+0000","last_peered":"2024-09-11T23:51:14.409908+0000","last_clean":"2024-09-11T23:51:14.409908+0000","last_became_active":"2024-09-11T23:51:14.409432+0000","last_became_peered":"2024-09-11T23:51:14.409432+0000","last_unstale":"2024-09-11T23:51:14.409908+0000","last_undegraded":"2024-09-11T23:51:14.409908+0000","last_fullsized":"2024-09-11T23:51:14.409908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:06.371306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114780+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:51:14.114780+0000","last_peered":"2024-09-11T23:51:14.114780+0000","last_clean":"2024-09-11T23:51:14.114780+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:51:14.114780+0000","last_undegraded":"2024-09-11T23:51:14.114780+0000","last_fullsized":"2024-09-11T23:51:14.114780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117129+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:51:14.117129+0000","last_peered":"2024-09-11T23:51:14.117129+0000","last_clean":"2024-09-11T23:51:14.117129+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:51:14.117129+0000","last_undegraded":"2024-09-11T23:51:14.117129+0000","last_fullsized":"2024-09-11T23:51:14.117129+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117793+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:51:14.117793+0000","last_peered":"2024-09-11T23:51:14.117793+0000","last_clean":"2024-09-11T23:51:14.117793+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:51:14.117793+0000","last_undegraded":"2024-09-11T23:51:14.117793+0000","last_fullsized":"2024-09-11T23:51:14.117793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.115193+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:51:14.115193+0000","last_peered":"2024-09-11T23:51:14.115193+0000","last_clean":"2024-09-11T23:51:14.115193+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:51:14.115193+0000","last_undegraded":"2024-09-11T23:51:14.115193+0000","last_fullsized":"2024-09-11T23:51:14.115193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'11","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114976+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:51:14.114976+0000","last_peered":"2024-09-11T23:51:14.114976+0000","last_clean":"2024-09-11T23:51:14.114976+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:51:14.114976+0000","last_undegraded":"2024-09-11T23:51:14.114976+0000","last_fullsized":"2024-09-11T23:51:14.114976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117309+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:51:14.117309+0000","last_peered":"2024-09-11T23:51:14.117309+0000","last_clean":"2024-09-11T23:51:14.117309+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:51:14.117309+0000","last_undegraded":"2024-09-11T23:51:14.117309+0000","last_fullsized":"2024-09-11T23:51:14.117309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.410001+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:51:14.410001+0000","last_peered":"2024-09-11T23:51:14.410001+0000","last_clean":"2024-09-11T23:51:14.410001+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:51:14.410001+0000","last_undegraded":"2024-09-11T23:51:14.410001+0000","last_fullsized":"2024-09-11T23:51:14.410001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.408922+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:51:14.408922+0000","last_peered":"2024-09-11T23:51:14.408922+0000","last_clean":"2024-09-11T23:51:14.408922+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:51:14.408922+0000","last_undegraded":"2024-09-11T23:51:14.408922+0000","last_fullsized":"2024-09-11T23:51:14.408922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117576+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:51:14.117576+0000","last_peered":"2024-09-11T23:51:14.117576+0000","last_clean":"2024-09-11T23:51:14.117576+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:51:14.117576+0000","last_undegraded":"2024-09-11T23:51:14.117576+0000","last_fullsized":"2024-09-11T23:51:14.117576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117114+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:51:14.117114+0000","last_peered":"2024-09-11T23:51:14.117114+0000","last_clean":"2024-09-11T23:51:14.117114+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:51:14.117114+0000","last_undegraded":"2024-09-11T23:51:14.117114+0000","last_fullsized":"2024-09-11T23:51:14.117114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117228+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:51:14.117228+0000","last_peered":"2024-09-11T23:51:14.117228+0000","last_clean":"2024-09-11T23:51:14.117228+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:51:14.117228+0000","last_undegraded":"2024-09-11T23:51:14.117228+0000","last_fullsized":"2024-09-11T23:51:14.117228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.114121+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:51:14.114121+0000","last_peered":"2024-09-11T23:51:14.114121+0000","last_clean":"2024-09-11T23:51:14.114121+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:51:14.114121+0000","last_undegraded":"2024-09-11T23:51:14.114121+0000","last_fullsized":"2024-09-11T23:51:14.114121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.124072+0000","last_change":"2024-09-11T23:51:14.123663+0000","last_active":"2024-09-11T23:51:14.124072+0000","last_peered":"2024-09-11T23:51:14.124072+0000","last_clean":"2024-09-11T23:51:14.124072+0000","last_became_active":"2024-09-11T23:51:14.118315+0000","last_became_peered":"2024-09-11T23:51:14.118315+0000","last_unstale":"2024-09-11T23:51:14.124072+0000","last_undegraded":"2024-09-11T23:51:14.124072+0000","last_fullsized":"2024-09-11T23:51:14.124072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:10.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116860+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:51:14.116860+0000","last_peered":"2024-09-11T23:51:14.116860+0000","last_clean":"2024-09-11T23:51:14.116860+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:51:14.116860+0000","last_undegraded":"2024-09-11T23:51:14.116860+0000","last_fullsized":"2024-09-11T23:51:14.116860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116980+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:51:14.116980+0000","last_peered":"2024-09-11T23:51:14.116980+0000","last_clean":"2024-09-11T23:51:14.116980+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:51:14.116980+0000","last_undegraded":"2024-09-11T23:51:14.116980+0000","last_fullsized":"2024-09-11T23:51:14.116980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116368+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:51:14.116368+0000","last_peered":"2024-09-11T23:51:14.116368+0000","last_clean":"2024-09-11T23:51:14.116368+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:51:14.116368+0000","last_undegraded":"2024-09-11T23:51:14.116368+0000","last_fullsized":"2024-09-11T23:51:14.116368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.116161+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:51:14.116161+0000","last_peered":"2024-09-11T23:51:14.116161+0000","last_clean":"2024-09-11T23:51:14.116161+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:51:14.116161+0000","last_undegraded":"2024-09-11T23:51:14.116161+0000","last_fullsized":"2024-09-11T23:51:14.116161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117388+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:51:14.117388+0000","last_peered":"2024-09-11T23:51:14.117388+0000","last_clean":"2024-09-11T23:51:14.117388+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:51:14.117388+0000","last_undegraded":"2024-09-11T23:51:14.117388+0000","last_fullsized":"2024-09-11T23:51:14.117388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117643+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:51:14.117643+0000","last_peered":"2024-09-11T23:51:14.117643+0000","last_clean":"2024-09-11T23:51:14.117643+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:51:14.117643+0000","last_undegraded":"2024-09-11T23:51:14.117643+0000","last_fullsized":"2024-09-11T23:51:14.117643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117655+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:51:14.117655+0000","last_peered":"2024-09-11T23:51:14.117655+0000","last_clean":"2024-09-11T23:51:14.117655+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:51:14.117655+0000","last_undegraded":"2024-09-11T23:51:14.117655+0000","last_fullsized":"2024-09-11T23:51:14.117655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.408983+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:51:14.408983+0000","last_peered":"2024-09-11T23:51:14.408983+0000","last_clean":"2024-09-11T23:51:14.408983+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:51:14.408983+0000","last_undegraded":"2024-09-11T23:51:14.408983+0000","last_fullsized":"2024-09-11T23:51:14.408983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117360+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:51:14.117360+0000","last_peered":"2024-09-11T23:51:14.117360+0000","last_clean":"2024-09-11T23:51:14.117360+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:51:14.117360+0000","last_undegraded":"2024-09-11T23:51:14.117360+0000","last_fullsized":"2024-09-11T23:51:14.117360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.117186+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:51:14.117186+0000","last_peered":"2024-09-11T23:51:14.117186+0000","last_clean":"2024-09-11T23:51:14.117186+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:51:14.117186+0000","last_undegraded":"2024-09-11T23:51:14.117186+0000","last_fullsized":"2024-09-11T23:51:14.117186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:51:14.125596+0000","last_change":"2024-09-11T23:51:14.125596+0000","last_active":"2024-09-11T23:51:14.125596+0000","last_peered":"2024-09-11T23:51:14.125596+0000","last_clean":"2024-09-11T23:51:14.125596+0000","last_became_active":"2024-09-11T23:51:14.119682+0000","last_became_peered":"2024-09-11T23:51:14.119682+0000","last_unstale":"2024-09-11T23:51:14.125596+0000","last_undegraded":"2024-09-11T23:51:14.125596+0000","last_fullsized":"2024-09-11T23:51:14.125596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:06.062602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65175,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":402,"num_read_kb":269,"num_write":163,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281926,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547226,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":436969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":1090015,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939687,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":437205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168619,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":436543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":962560,"data_stored":1029057,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":436594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":1028348,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":905216,"data_stored":437424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":436905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":498909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":966656,"data_stored":498141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":921600,"data_stored":437865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:51:25.498 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:51:25.498 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T23:51:25.498 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T23:51:25.498 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T23:51:25.756 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:26.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.249+0000 7fb5e813c640 1 -- 172.21.15.62:0/2729475936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e0101de0 msgr2=0x7fb5e01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:26.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.249+0000 7fb5e813c640 1 --2- 172.21.15.62:0/2729475936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e0101de0 0x7fb5e01021c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009900 tx=0x7fb5d402a410 comp rx=0 tx=0).stop 2024-09-11T23:51:26.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.249+0000 7fb5e813c640 1 -- 172.21.15.62:0/2729475936 shutdown_connections 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.249+0000 7fb5e813c640 1 --2- 172.21.15.62:0/2729475936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5e0106f10 0x7fb5e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.249+0000 7fb5e813c640 1 --2- 172.21.15.62:0/2729475936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5e0102790 0x7fb5e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.249+0000 7fb5e813c640 1 --2- 172.21.15.62:0/2729475936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e0101de0 0x7fb5e01021c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.249+0000 7fb5e813c640 1 -- 172.21.15.62:0/2729475936 >> 172.21.15.62:0/2729475936 conn(0x7fb5e00fd8c0 msgr2=0x7fb5e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.249+0000 7fb5e813c640 1 -- 172.21.15.62:0/2729475936 shutdown_connections 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.249+0000 7fb5e813c640 1 -- 172.21.15.62:0/2729475936 wait complete. 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e813c640 1 Processor -- start 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e813c640 1 -- start start 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e813c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e0101de0 0x7fb5e00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e813c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5e0102790 0x7fb5e0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e813c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5e0106f10 0x7fb5e0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e813c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e0111f70 con 0x7fb5e0101de0 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e813c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e0111df0 con 0x7fb5e0106f10 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e813c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e01120f0 con 0x7fb5e0102790 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e5ed8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e0101de0 0x7fb5e00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e5ed8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e0101de0 0x7fb5e00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58328/0 (socket says 172.21.15.62:58328) 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e5ed8640 1 -- 172.21.15.62:0/1456085027 learned_addr learned my addr 172.21.15.62:0/1456085027 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e56d7640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5e0102790 0x7fb5e0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.250+0000 7fb5e66d9640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5e0106f10 0x7fb5e0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5e5ed8640 1 -- 172.21.15.62:0/1456085027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5e0102790 msgr2=0x7fb5e0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5e5ed8640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5e0102790 0x7fb5e0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5e5ed8640 1 -- 172.21.15.62:0/1456085027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5e0106f10 msgr2=0x7fb5e0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5e5ed8640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5e0106f10 0x7fb5e0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5e5ed8640 1 -- 172.21.15.62:0/1456085027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e01a68f0 con 0x7fb5e0101de0 2024-09-11T23:51:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5e56d7640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5e0102790 0x7fb5e0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5e5ed8640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e0101de0 0x7fb5e00744f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4006010 tx=0x7fb5d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5d2ffd640 1 -- 172.21.15.62:0/1456085027 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d403f030 con 0x7fb5e0101de0 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5d2ffd640 1 -- 172.21.15.62:0/1456085027 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5d4003ce0 con 0x7fb5e0101de0 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5e813c640 1 -- 172.21.15.62:0/1456085027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e01a6b80 con 0x7fb5e0101de0 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5d2ffd640 1 -- 172.21.15.62:0/1456085027 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4035070 con 0x7fb5e0101de0 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.251+0000 7fb5e813c640 1 -- 172.21.15.62:0/1456085027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e01a7060 con 0x7fb5e0101de0 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.252+0000 7fb5e813c640 1 -- 172.21.15.62:0/1456085027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e01021c0 con 0x7fb5e0101de0 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.253+0000 7fb5d2ffd640 1 -- 172.21.15.62:0/1456085027 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb5d4042030 con 0x7fb5e0101de0 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.254+0000 7fb5d2ffd640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb5cc0a2c10 0x7fb5cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.254+0000 7fb5e56d7640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb5cc0a2c10 0x7fb5cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.254+0000 7fb5d2ffd640 1 -- 172.21.15.62:0/1456085027 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb5d40ef6f0 con 0x7fb5e0101de0 2024-09-11T23:51:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.254+0000 7fb5e56d7640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb5cc0a2c10 0x7fb5cc0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009fd0 tx=0x7fb5dc009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:26.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.258+0000 7fb5d2ffd640 1 -- 172.21.15.62:0/1456085027 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5d40bbf70 con 0x7fb5e0101de0 2024-09-11T23:51:26.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:26 smithi083 ceph-mon[34805]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 54 B/s, 0 objects/s recovering 2024-09-11T23:51:26.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:26 smithi083 ceph-mon[34805]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:26 smithi062 ceph-mon[30958]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 54 B/s, 0 objects/s recovering 2024-09-11T23:51:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:26 smithi062 ceph-mon[30958]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.472+0000 7fb5e813c640 1 -- 172.21.15.62:0/1456085027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb5e0109b60 con 0x7fb5e0101de0 2024-09-11T23:51:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.472+0000 7fb5d2ffd640 1 -- 172.21.15.62:0/1456085027 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb5d40c0270 con 0x7fb5e0101de0 2024-09-11T23:51:26.472 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:26.472 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T23:51:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 -- 172.21.15.62:0/1456085027 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb5cc0a2c10 msgr2=0x7fb5cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb5cc0a2c10 0x7fb5cc0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009fd0 tx=0x7fb5dc009680 comp rx=0 tx=0).stop 2024-09-11T23:51:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 -- 172.21.15.62:0/1456085027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e0101de0 msgr2=0x7fb5e00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e0101de0 0x7fb5e00744f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4006010 tx=0x7fb5d40042f0 comp rx=0 tx=0).stop 2024-09-11T23:51:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 -- 172.21.15.62:0/1456085027 shutdown_connections 2024-09-11T23:51:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fb5cc0a2c10 0x7fb5cc0a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5e0106f10 0x7fb5e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5e0102790 0x7fb5e0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 --2- 172.21.15.62:0/1456085027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e0101de0 0x7fb5e00744f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 -- 172.21.15.62:0/1456085027 >> 172.21.15.62:0/1456085027 conn(0x7fb5e00fd8c0 msgr2=0x7fb5e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 -- 172.21.15.62:0/1456085027 shutdown_connections 2024-09-11T23:51:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:26.474+0000 7fb5e813c640 1 -- 172.21.15.62:0/1456085027 wait complete. 2024-09-11T23:51:26.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:26 smithi120 ceph-mon[35182]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 54 B/s, 0 objects/s recovering 2024-09-11T23:51:26.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:26 smithi120 ceph-mon[35182]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:51:27.165 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T23:51:27.166 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:51:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:27 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1456085027' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:51:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:27.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:27 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1456085027' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:51:27.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:27 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1456085027' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:51:27.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.842+0000 7f36800da640 1 -- 172.21.15.62:0/1919194344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678102dd0 msgr2=0x7f36781031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:27.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.842+0000 7f36800da640 1 --2- 172.21.15.62:0/1919194344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678102dd0 0x7f36781031b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3668009cd0 tx=0x7f366802a8d0 comp rx=0 tx=0).stop 2024-09-11T23:51:27.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.842+0000 7f36800da640 1 -- 172.21.15.62:0/1919194344 shutdown_connections 2024-09-11T23:51:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.842+0000 7f36800da640 1 --2- 172.21.15.62:0/1919194344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3678108300 0x7f367810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.842+0000 7f36800da640 1 --2- 172.21.15.62:0/1919194344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3678103780 0x7f3678107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.842+0000 7f36800da640 1 --2- 172.21.15.62:0/1919194344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678102dd0 0x7f36781031b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.842+0000 7f36800da640 1 -- 172.21.15.62:0/1919194344 >> 172.21.15.62:0/1919194344 conn(0x7f36780fc7d0 msgr2=0x7f36780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.842+0000 7f36800da640 1 -- 172.21.15.62:0/1919194344 shutdown_connections 2024-09-11T23:51:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.842+0000 7f36800da640 1 -- 172.21.15.62:0/1919194344 wait complete. 2024-09-11T23:51:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f36800da640 1 Processor -- start 2024-09-11T23:51:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f36800da640 1 -- start start 2024-09-11T23:51:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f36800da640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678102dd0 0x7f36781985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f36800da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3678103780 0x7f3678198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f36800da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3678108300 0x7f367819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f36800da640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36781120b0 con 0x7f3678102dd0 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f36800da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3678111f30 con 0x7f3678103780 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f36800da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3678112230 con 0x7f3678108300 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f367de76640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678102dd0 0x7f36781985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f367de76640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678102dd0 0x7f36781985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58334/0 (socket says 172.21.15.62:58334) 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f367de76640 1 -- 172.21.15.62:0/208203301 learned_addr learned my addr 172.21.15.62:0/208203301 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.843+0000 7f367d675640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3678103780 0x7f3678198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f367e677640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3678108300 0x7f367819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f367de76640 1 -- 172.21.15.62:0/208203301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3678108300 msgr2=0x7f367819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f367de76640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3678108300 0x7f367819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f367de76640 1 -- 172.21.15.62:0/208203301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3678103780 msgr2=0x7f3678198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f367de76640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3678103780 0x7f3678198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f367de76640 1 -- 172.21.15.62:0/208203301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367819d5a0 con 0x7f3678102dd0 2024-09-11T23:51:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f367d675640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3678103780 0x7f3678198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f367de76640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678102dd0 0x7f36781985f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3668009df0 tx=0x7f3668031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f3676ffd640 1 -- 172.21.15.62:0/208203301 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668031840 con 0x7f3678102dd0 2024-09-11T23:51:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f36800da640 1 -- 172.21.15.62:0/208203301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367819d830 con 0x7f3678102dd0 2024-09-11T23:51:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f3676ffd640 1 -- 172.21.15.62:0/208203301 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3668003e50 con 0x7f3678102dd0 2024-09-11T23:51:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f3676ffd640 1 -- 172.21.15.62:0/208203301 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36680307f0 con 0x7f3678102dd0 2024-09-11T23:51:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.844+0000 7f36800da640 1 -- 172.21.15.62:0/208203301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36781a5170 con 0x7f3678102dd0 2024-09-11T23:51:27.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.845+0000 7f36800da640 1 -- 172.21.15.62:0/208203301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36781031b0 con 0x7f3678102dd0 2024-09-11T23:51:27.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.847+0000 7f3676ffd640 1 -- 172.21.15.62:0/208203301 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3668004340 con 0x7f3678102dd0 2024-09-11T23:51:27.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.847+0000 7f3676ffd640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f36600a2c90 0x7f36600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:27.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.847+0000 7f367d675640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f36600a2c90 0x7f36600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:27.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.847+0000 7f3676ffd640 1 -- 172.21.15.62:0/208203301 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f36680f0350 con 0x7f3678102dd0 2024-09-11T23:51:27.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.848+0000 7f367d675640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f36600a2c90 0x7f36600a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f366c0058c0 tx=0x7f366c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:27.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:27.850+0000 7f3676ffd640 1 -- 172.21.15.62:0/208203301 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f36680bd270 con 0x7f3678102dd0 2024-09-11T23:51:28.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.061+0000 7f36800da640 1 -- 172.21.15.62:0/208203301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f367810a290 con 0x7f3678102dd0 2024-09-11T23:51:28.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.061+0000 7f3676ffd640 1 -- 172.21.15.62:0/208203301 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11909 (secure 0 0 0) 0x7f36680bcb70 con 0x7f3678102dd0 2024-09-11T23:51:28.062 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:28.062 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"btime":"2024-09-11T23:50:22:864278+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:50:22.553275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:51:28.062 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2024-09-11T23:51:28.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.063+0000 7f36800da640 1 -- 172.21.15.62:0/208203301 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f36600a2c90 msgr2=0x7f36600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:28.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.063+0000 7f36800da640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f36600a2c90 0x7f36600a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f366c0058c0 tx=0x7f366c005850 comp rx=0 tx=0).stop 2024-09-11T23:51:28.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.063+0000 7f36800da640 1 -- 172.21.15.62:0/208203301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678102dd0 msgr2=0x7f36781985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:28.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.063+0000 7f36800da640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678102dd0 0x7f36781985f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3668009df0 tx=0x7f3668031420 comp rx=0 tx=0).stop 2024-09-11T23:51:28.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.063+0000 7f36800da640 1 -- 172.21.15.62:0/208203301 shutdown_connections 2024-09-11T23:51:28.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.064+0000 7f36800da640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f36600a2c90 0x7f36600a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:28.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.064+0000 7f36800da640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3678108300 0x7f367819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:28.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.064+0000 7f36800da640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3678103780 0x7f3678198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:28.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.064+0000 7f36800da640 1 --2- 172.21.15.62:0/208203301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678102dd0 0x7f36781985f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:28.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.064+0000 7f36800da640 1 -- 172.21.15.62:0/208203301 >> 172.21.15.62:0/208203301 conn(0x7f36780fc7d0 msgr2=0x7f3678105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:28.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.064+0000 7f36800da640 1 -- 172.21.15.62:0/208203301 shutdown_connections 2024-09-11T23:51:28.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:28.064+0000 7f36800da640 1 -- 172.21.15.62:0/208203301 wait complete. 2024-09-11T23:51:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:28 smithi083 ceph-mon[34805]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2024-09-11T23:51:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:28 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/208203301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:28.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:28 smithi062 ceph-mon[30958]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2024-09-11T23:51:28.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:28 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/208203301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:28.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:28 smithi120 ceph-mon[35182]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2024-09-11T23:51:28.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:28 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/208203301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:28.724 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:51:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:29.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.426+0000 7f4dceef5640 1 -- 172.21.15.62:0/3561417270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dc81036e0 msgr2=0x7f4dc8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:29.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.426+0000 7f4dceef5640 1 --2- 172.21.15.62:0/3561417270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dc81036e0 0x7f4dc8107b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009cd0 tx=0x7f4db802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.426+0000 7f4dceef5640 1 -- 172.21.15.62:0/3561417270 shutdown_connections 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.426+0000 7f4dceef5640 1 --2- 172.21.15.62:0/3561417270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc81082a0 0x7f4dc810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.426+0000 7f4dceef5640 1 --2- 172.21.15.62:0/3561417270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dc81036e0 0x7f4dc8107b90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.426+0000 7f4dceef5640 1 --2- 172.21.15.62:0/3561417270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dc8102d30 0x7f4dc8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.426+0000 7f4dceef5640 1 -- 172.21.15.62:0/3561417270 >> 172.21.15.62:0/3561417270 conn(0x7f4dc80fc7d0 msgr2=0x7f4dc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.427+0000 7f4dceef5640 1 -- 172.21.15.62:0/3561417270 shutdown_connections 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.427+0000 7f4dceef5640 1 -- 172.21.15.62:0/3561417270 wait complete. 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.427+0000 7f4dceef5640 1 Processor -- start 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.427+0000 7f4dceef5640 1 -- start start 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.427+0000 7f4dceef5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dc8102d30 0x7f4dc8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dceef5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc81036e0 0x7f4dc8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:29.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dceef5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dc81082a0 0x7f4dc819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dceef5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81120c0 con 0x7f4dc8102d30 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dceef5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8111f40 con 0x7f4dc81036e0 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dceef5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8112240 con 0x7f4dc81082a0 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dccc91640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dc8102d30 0x7f4dc8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dccc91640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dc8102d30 0x7f4dc8198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58346/0 (socket says 172.21.15.62:58346) 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dccc91640 1 -- 172.21.15.62:0/1555772590 learned_addr learned my addr 172.21.15.62:0/1555772590 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dcd492640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dc81082a0 0x7f4dc819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dc7fff640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc81036e0 0x7f4dc8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dccc91640 1 -- 172.21.15.62:0/1555772590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dc81082a0 msgr2=0x7f4dc819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dccc91640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dc81082a0 0x7f4dc819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dccc91640 1 -- 172.21.15.62:0/1555772590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc81036e0 msgr2=0x7f4dc8198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:29.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dccc91640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc81036e0 0x7f4dc8198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:29.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dccc91640 1 -- 172.21.15.62:0/1555772590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc819d5f0 con 0x7f4dc8102d30 2024-09-11T23:51:29.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dccc91640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dc8102d30 0x7f4dc8198640 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4db000b4a0 tx=0x7f4db000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:29.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dc5ffb640 1 -- 172.21.15.62:0/1555772590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0014070 con 0x7f4dc8102d30 2024-09-11T23:51:29.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dc5ffb640 1 -- 172.21.15.62:0/1555772590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db0010af0 con 0x7f4dc8102d30 2024-09-11T23:51:29.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dceef5640 1 -- 172.21.15.62:0/1555772590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc819d8e0 con 0x7f4dc8102d30 2024-09-11T23:51:29.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.428+0000 7f4dc5ffb640 1 -- 172.21.15.62:0/1555772590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0017d10 con 0x7f4dc8102d30 2024-09-11T23:51:29.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.429+0000 7f4dceef5640 1 -- 172.21.15.62:0/1555772590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc81a51c0 con 0x7f4dc8102d30 2024-09-11T23:51:29.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.430+0000 7f4dceef5640 1 -- 172.21.15.62:0/1555772590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc806a850 con 0x7f4dc8102d30 2024-09-11T23:51:29.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.431+0000 7f4dc5ffb640 1 -- 172.21.15.62:0/1555772590 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4db0017070 con 0x7f4dc8102d30 2024-09-11T23:51:29.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.432+0000 7f4dc5ffb640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4db40a2c10 0x7f4db40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:29.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.432+0000 7f4dc7fff640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4db40a2c10 0x7f4db40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:29.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.432+0000 7f4dc5ffb640 1 -- 172.21.15.62:0/1555772590 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4db00ce4f0 con 0x7f4dc8102d30 2024-09-11T23:51:29.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.433+0000 7f4dc7fff640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4db40a2c10 0x7f4db40a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4db8006010 tx=0x7f4db8009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:29.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.436+0000 7f4dc5ffb640 1 -- 172.21.15.62:0/1555772590 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4db009b410 con 0x7f4dc8102d30 2024-09-11T23:51:29.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.648+0000 7f4dceef5640 1 -- 172.21.15.62:0/1555772590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4dc81999d0 con 0x7f4dc8102d30 2024-09-11T23:51:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.648+0000 7f4dc5ffb640 1 -- 172.21.15.62:0/1555772590 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11909 (secure 0 0 0) 0x7f4dc81999d0 con 0x7f4dc8102d30 2024-09-11T23:51:29.648 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:29.648 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"btime":"2024-09-11T23:50:22:864278+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:50:22.553275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:51:29.649 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2024-09-11T23:51:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.650+0000 7f4dceef5640 1 -- 172.21.15.62:0/1555772590 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4db40a2c10 msgr2=0x7f4db40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.650+0000 7f4dceef5640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4db40a2c10 0x7f4db40a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4db8006010 tx=0x7f4db8009800 comp rx=0 tx=0).stop 2024-09-11T23:51:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.650+0000 7f4dceef5640 1 -- 172.21.15.62:0/1555772590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dc8102d30 msgr2=0x7f4dc8198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:29.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.650+0000 7f4dceef5640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dc8102d30 0x7f4dc8198640 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4db000b4a0 tx=0x7f4db000b960 comp rx=0 tx=0).stop 2024-09-11T23:51:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.650+0000 7f4dceef5640 1 -- 172.21.15.62:0/1555772590 shutdown_connections 2024-09-11T23:51:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.650+0000 7f4dceef5640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f4db40a2c10 0x7f4db40a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.650+0000 7f4dceef5640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4dc81082a0 0x7f4dc819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.650+0000 7f4dceef5640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc81036e0 0x7f4dc8198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.650+0000 7f4dceef5640 1 --2- 172.21.15.62:0/1555772590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dc8102d30 0x7f4dc8198640 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.651+0000 7f4dceef5640 1 -- 172.21.15.62:0/1555772590 >> 172.21.15.62:0/1555772590 conn(0x7f4dc80fc7d0 msgr2=0x7f4dc8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.651+0000 7f4dceef5640 1 -- 172.21.15.62:0/1555772590 shutdown_connections 2024-09-11T23:51:29.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:29.651+0000 7f4dceef5640 1 -- 172.21.15.62:0/1555772590 wait complete. 2024-09-11T23:51:30.323 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:51:30.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:30 smithi083 ceph-mon[34805]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-09-11T23:51:30.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:30 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1555772590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:30.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:30 smithi062 ceph-mon[30958]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-09-11T23:51:30.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:30 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1555772590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:30.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:30 smithi120 ceph-mon[35182]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-09-11T23:51:30.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:30 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1555772590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:31.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.021+0000 7f7a3ac93640 1 -- 172.21.15.62:0/2254905079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a34106a80 msgr2=0x7f7a34106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:31.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.021+0000 7f7a3ac93640 1 --2- 172.21.15.62:0/2254905079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a34106a80 0x7f7a34106e60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009930 tx=0x7f7a2802a420 comp rx=0 tx=0).stop 2024-09-11T23:51:31.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.021+0000 7f7a3ac93640 1 -- 172.21.15.62:0/2254905079 shutdown_connections 2024-09-11T23:51:31.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.021+0000 7f7a3ac93640 1 --2- 172.21.15.62:0/2254905079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a34101420 0x7f7a3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:31.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.021+0000 7f7a3ac93640 1 --2- 172.21.15.62:0/2254905079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a34100a80 0x7f7a34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:31.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.021+0000 7f7a3ac93640 1 --2- 172.21.15.62:0/2254905079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a34106a80 0x7f7a34106e60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:31.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.021+0000 7f7a3ac93640 1 -- 172.21.15.62:0/2254905079 >> 172.21.15.62:0/2254905079 conn(0x7f7a340fc7a0 msgr2=0x7f7a340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:31.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.021+0000 7f7a3ac93640 1 -- 172.21.15.62:0/2254905079 shutdown_connections 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.021+0000 7f7a3ac93640 1 -- 172.21.15.62:0/2254905079 wait complete. 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.022+0000 7f7a3ac93640 1 Processor -- start 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.022+0000 7f7a3ac93640 1 -- start start 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a3ac93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a34100a80 0x7f7a34199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a3ac93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a34101420 0x7f7a341997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a3ac93640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a34106a80 0x7f7a341a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a3ac93640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a34110e40 con 0x7f7a34106a80 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a3ac93640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a34110cc0 con 0x7f7a34101420 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a3ac93640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7a34110fc0 con 0x7f7a34100a80 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a39230640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a34106a80 0x7f7a341a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a39230640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a34106a80 0x7f7a341a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58368/0 (socket says 172.21.15.62:58368) 2024-09-11T23:51:31.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a39230640 1 -- 172.21.15.62:0/913439281 learned_addr learned my addr 172.21.15.62:0/913439281 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a38a2f640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a34100a80 0x7f7a34199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a33fff640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a34101420 0x7f7a341997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a39230640 1 -- 172.21.15.62:0/913439281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a34100a80 msgr2=0x7f7a34199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a39230640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a34100a80 0x7f7a34199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a39230640 1 -- 172.21.15.62:0/913439281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a34101420 msgr2=0x7f7a341997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a39230640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a34101420 0x7f7a341997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a39230640 1 -- 172.21.15.62:0/913439281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a341a4850 con 0x7f7a34106a80 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a39230640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a34106a80 0x7f7a341a4310 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00cad0 tx=0x7f7a2c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a31ffb640 1 -- 172.21.15.62:0/913439281 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00ae20 con 0x7f7a34106a80 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a31ffb640 1 -- 172.21.15.62:0/913439281 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a2c005020 con 0x7f7a34106a80 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a3ac93640 1 -- 172.21.15.62:0/913439281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a341a4a20 con 0x7f7a34106a80 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.023+0000 7f7a31ffb640 1 -- 172.21.15.62:0/913439281 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00f6f0 con 0x7f7a34106a80 2024-09-11T23:51:31.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.024+0000 7f7a3ac93640 1 -- 172.21.15.62:0/913439281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a341a4f30 con 0x7f7a34106a80 2024-09-11T23:51:31.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.025+0000 7f7a3ac93640 1 -- 172.21.15.62:0/913439281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a34100ee0 con 0x7f7a34106a80 2024-09-11T23:51:31.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.026+0000 7f7a31ffb640 1 -- 172.21.15.62:0/913439281 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7a2c0051d0 con 0x7f7a34106a80 2024-09-11T23:51:31.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.027+0000 7f7a31ffb640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7a240a2c90 0x7f7a240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:31.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.027+0000 7f7a38a2f640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7a240a2c90 0x7f7a240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:31.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.027+0000 7f7a31ffb640 1 -- 172.21.15.62:0/913439281 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7a2c0cf560 con 0x7f7a34106a80 2024-09-11T23:51:31.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.027+0000 7f7a38a2f640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7a240a2c90 0x7f7a240a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7a28000c00 tx=0x7f7a28031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:31.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.030+0000 7f7a31ffb640 1 -- 172.21.15.62:0/913439281 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a2c014030 con 0x7f7a34106a80 2024-09-11T23:51:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.207+0000 7f7a3ac93640 1 -- 172.21.15.62:0/913439281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a34108ff0 con 0x7f7a34106a80 2024-09-11T23:51:31.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.208+0000 7f7a31ffb640 1 -- 172.21.15.62:0/913439281 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22620 (secure 0 0 0) 0x7f7a2c09c3f0 con 0x7f7a34106a80 2024-09-11T23:51:31.209 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:31.209 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":83,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:51:14.101286+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:51:31.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.210+0000 7f7a3ac93640 1 -- 172.21.15.62:0/913439281 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7a240a2c90 msgr2=0x7f7a240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:31.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.210+0000 7f7a3ac93640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7a240a2c90 0x7f7a240a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7a28000c00 tx=0x7f7a28031490 comp rx=0 tx=0).stop 2024-09-11T23:51:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.210+0000 7f7a3ac93640 1 -- 172.21.15.62:0/913439281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a34106a80 msgr2=0x7f7a341a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.210+0000 7f7a3ac93640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a34106a80 0x7f7a341a4310 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00cad0 tx=0x7f7a2c00cf90 comp rx=0 tx=0).stop 2024-09-11T23:51:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.210+0000 7f7a3ac93640 1 -- 172.21.15.62:0/913439281 shutdown_connections 2024-09-11T23:51:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.210+0000 7f7a3ac93640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f7a240a2c90 0x7f7a240a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.210+0000 7f7a3ac93640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a34106a80 0x7f7a341a4310 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.210+0000 7f7a3ac93640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a34101420 0x7f7a341997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.210+0000 7f7a3ac93640 1 --2- 172.21.15.62:0/913439281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a34100a80 0x7f7a34199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.210+0000 7f7a3ac93640 1 -- 172.21.15.62:0/913439281 >> 172.21.15.62:0/913439281 conn(0x7f7a340fc7a0 msgr2=0x7f7a340fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.211+0000 7f7a3ac93640 1 -- 172.21.15.62:0/913439281 shutdown_connections 2024-09-11T23:51:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:31.211+0000 7f7a3ac93640 1 -- 172.21.15.62:0/913439281 wait complete. 2024-09-11T23:51:31.872 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:51:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:32.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:32 smithi062 ceph-mon[30958]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:32.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:32 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/913439281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:51:32.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:32 smithi083 ceph-mon[34805]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:32.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:32 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/913439281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:51:32.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 -- 172.21.15.62:0/3392405102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61a81027a0 msgr2=0x7f61a81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:32.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 --2- 172.21.15.62:0/3392405102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61a81027a0 0x7f61a81067f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f619c009900 tx=0x7f619c02a410 comp rx=0 tx=0).stop 2024-09-11T23:51:32.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 -- 172.21.15.62:0/3392405102 shutdown_connections 2024-09-11T23:51:32.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 --2- 172.21.15.62:0/3392405102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a8106f20 0x7f61a810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 --2- 172.21.15.62:0/3392405102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61a81027a0 0x7f61a81067f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 --2- 172.21.15.62:0/3392405102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61a8101df0 0x7f61a81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 -- 172.21.15.62:0/3392405102 >> 172.21.15.62:0/3392405102 conn(0x7f61a80fd8b0 msgr2=0x7f61a80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 -- 172.21.15.62:0/3392405102 shutdown_connections 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 -- 172.21.15.62:0/3392405102 wait complete. 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 Processor -- start 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.580+0000 7f61ae860640 1 -- start start 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61ae860640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61a8101df0 0x7f61a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61ae860640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61a81027a0 0x7f61a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61ae860640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a8106f20 0x7f61a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61ae860640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f61a8112070 con 0x7f61a81027a0 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61ae860640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61a8111ef0 con 0x7f61a8106f20 2024-09-11T23:51:32.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61ae860640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81121f0 con 0x7f61a8101df0 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61a81027a0 0x7f61a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61a81027a0 0x7f61a8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49642/0 (socket says 172.21.15.62:49642) 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a77fe640 1 -- 172.21.15.62:0/3867888666 learned_addr learned my addr 172.21.15.62:0/3867888666 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a7fff640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61a8101df0 0x7f61a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a77fe640 1 -- 172.21.15.62:0/3867888666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61a8101df0 msgr2=0x7f61a80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a77fe640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61a8101df0 0x7f61a80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a77fe640 1 -- 172.21.15.62:0/3867888666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a8106f20 msgr2=0x7f61a8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61acdfd640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a8106f20 0x7f61a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a77fe640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a8106f20 0x7f61a8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a77fe640 1 -- 172.21.15.62:0/3867888666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a81a6a70 con 0x7f61a81027a0 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a77fe640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61a81027a0 0x7f61a8074a30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f619c006010 tx=0x7f619c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a57fa640 1 -- 172.21.15.62:0/3867888666 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c03f070 con 0x7f61a81027a0 2024-09-11T23:51:32.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61ae860640 1 -- 172.21.15.62:0/3867888666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a81a6d00 con 0x7f61a81027a0 2024-09-11T23:51:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a57fa640 1 -- 172.21.15.62:0/3867888666 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f619c031c90 con 0x7f61a81027a0 2024-09-11T23:51:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.581+0000 7f61a57fa640 1 -- 172.21.15.62:0/3867888666 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c035030 con 0x7f61a81027a0 2024-09-11T23:51:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.582+0000 7f61ae860640 1 -- 172.21.15.62:0/3867888666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a81a7180 con 0x7f61a81027a0 2024-09-11T23:51:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.582+0000 7f61ae860640 1 -- 172.21.15.62:0/3867888666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a8102250 con 0x7f61a81027a0 2024-09-11T23:51:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.584+0000 7f61a57fa640 1 -- 172.21.15.62:0/3867888666 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f619c042070 con 0x7f61a81027a0 2024-09-11T23:51:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.584+0000 7f61a57fa640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61900a2c10 0x7f61900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.584+0000 7f61a7fff640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61900a2c10 0x7f61900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.584+0000 7f61a57fa640 1 -- 172.21.15.62:0/3867888666 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f619c0efb50 con 0x7f61a81027a0 2024-09-11T23:51:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.585+0000 7f61a7fff640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61900a2c10 0x7f61900a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6198009890 tx=0x7f6198008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.588+0000 7f61a57fa640 1 -- 172.21.15.62:0/3867888666 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f619c0bc960 con 0x7f61a81027a0 2024-09-11T23:51:32.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:32 smithi120 ceph-mon[35182]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:32.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:32 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/913439281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:51:32.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.801+0000 7f61ae860640 1 -- 172.21.15.62:0/3867888666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61a8109b70 con 0x7f61a81027a0 2024-09-11T23:51:32.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.802+0000 7f61a57fa640 1 -- 172.21.15.62:0/3867888666 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11909 (secure 0 0 0) 0x7f619c0bc260 con 0x7f61a81027a0 2024-09-11T23:51:32.802 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:32.802 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"btime":"2024-09-11T23:50:22:864278+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:50:22.553275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:51:32.802 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2024-09-11T23:51:32.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.804+0000 7f61ae860640 1 -- 172.21.15.62:0/3867888666 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61900a2c10 msgr2=0x7f61900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:32.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.804+0000 7f61ae860640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61900a2c10 0x7f61900a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6198009890 tx=0x7f6198008040 comp rx=0 tx=0).stop 2024-09-11T23:51:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.804+0000 7f61ae860640 1 -- 172.21.15.62:0/3867888666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61a81027a0 msgr2=0x7f61a8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.804+0000 7f61ae860640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61a81027a0 0x7f61a8074a30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f619c006010 tx=0x7f619c003ba0 comp rx=0 tx=0).stop 2024-09-11T23:51:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.805+0000 7f61ae860640 1 -- 172.21.15.62:0/3867888666 shutdown_connections 2024-09-11T23:51:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.805+0000 7f61ae860640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f61900a2c10 0x7f61900a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.805+0000 7f61ae860640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a8106f20 0x7f61a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.805+0000 7f61ae860640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61a81027a0 0x7f61a8074a30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.805+0000 7f61ae860640 1 --2- 172.21.15.62:0/3867888666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61a8101df0 0x7f61a80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.805+0000 7f61ae860640 1 -- 172.21.15.62:0/3867888666 >> 172.21.15.62:0/3867888666 conn(0x7f61a80fd8b0 msgr2=0x7f61a80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.805+0000 7f61ae860640 1 -- 172.21.15.62:0/3867888666 shutdown_connections 2024-09-11T23:51:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:32.805+0000 7f61ae860640 1 -- 172.21.15.62:0/3867888666 wait complete. 2024-09-11T23:51:33.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:33 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3867888666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:33.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:33 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3867888666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:33.432 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:50:11.406773+0000', 'modified': '2024-09-11T23:50:22.553275+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24484, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6835/3040146550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 3040146550}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3040146550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6837/337493680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 337493680}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 337493680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/2559115046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 2559115046}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 2559115046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.120:6837/1796168792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 1796168792}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 1796168792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.120:6839/2283411896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 2283411896}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 2283411896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6839/2530997346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 2530997346}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 2530997346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24469, 24484, 24488]} 2024-09-11T23:51:33.432 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-11T23:51:33.432 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T23:51:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:33 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3867888666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:33.690 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:34.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.155+0000 7faec6fdc640 1 -- 172.21.15.62:0/2955705448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0108290 msgr2=0x7faec010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.155+0000 7faec6fdc640 1 --2- 172.21.15.62:0/2955705448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0108290 0x7faec010ec30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009cd0 tx=0x7faeb802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.155+0000 7faec6fdc640 1 -- 172.21.15.62:0/2955705448 shutdown_connections 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.155+0000 7faec6fdc640 1 --2- 172.21.15.62:0/2955705448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0108290 0x7faec010ec30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.155+0000 7faec6fdc640 1 --2- 172.21.15.62:0/2955705448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faec0103740 0x7faec0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.155+0000 7faec6fdc640 1 --2- 172.21.15.62:0/2955705448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faec0102d90 0x7faec0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.155+0000 7faec6fdc640 1 -- 172.21.15.62:0/2955705448 >> 172.21.15.62:0/2955705448 conn(0x7faec00fc7d0 msgr2=0x7faec00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.155+0000 7faec6fdc640 1 -- 172.21.15.62:0/2955705448 shutdown_connections 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.155+0000 7faec6fdc640 1 -- 172.21.15.62:0/2955705448 wait complete. 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faec6fdc640 1 Processor -- start 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faec6fdc640 1 -- start start 2024-09-11T23:51:34.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faec6fdc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faec0102d90 0x7faec019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faec6fdc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0103740 0x7faec019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faec6fdc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faec0108290 0x7faec019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faec6fdc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faec0112090 con 0x7faec0103740 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faec6fdc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faec0111f10 con 0x7faec0108290 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faec6fdc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faec0112210 con 0x7faec0102d90 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faebffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0103740 0x7faec019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faebffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0103740 0x7faec019ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49668/0 (socket says 172.21.15.62:49668) 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faebffff640 1 -- 172.21.15.62:0/1752396257 learned_addr learned my addr 172.21.15.62:0/1752396257 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faec5579640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faec0108290 0x7faec019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.156+0000 7faec4d78640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faec0102d90 0x7faec019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faebffff640 1 -- 172.21.15.62:0/1752396257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faec0102d90 msgr2=0x7faec019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:34.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faebffff640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faec0102d90 0x7faec019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faebffff640 1 -- 172.21.15.62:0/1752396257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faec0108290 msgr2=0x7faec019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faebffff640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faec0108290 0x7faec019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faebffff640 1 -- 172.21.15.62:0/1752396257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec019f700 con 0x7faec0103740 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faebffff640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0103740 0x7faec019ac90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faeac00d740 tx=0x7faeac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faebdffb640 1 -- 172.21.15.62:0/1752396257 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac017070 con 0x7faec0103740 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faebdffb640 1 -- 172.21.15.62:0/1752396257 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeac00de50 con 0x7faec0103740 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faec6fdc640 1 -- 172.21.15.62:0/1752396257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec019f9f0 con 0x7faec0103740 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faebdffb640 1 -- 172.21.15.62:0/1752396257 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac004e00 con 0x7faec0103740 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.157+0000 7faec6fdc640 1 -- 172.21.15.62:0/1752396257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec01a72d0 con 0x7faec0103740 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.159+0000 7faebdffb640 1 -- 172.21.15.62:0/1752396257 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faeac004fa0 con 0x7faec0103740 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec93b5740 1 Processor -- start 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec93b5740 1 -- start start 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faebdffb640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faeb00a2c90 0x7faeb00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec93b5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b2929d0 0x55922b292db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec93b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b29d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec93b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b29dbb0 0x55922b29ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec93b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55922b1f9090 con 0x55922b29dbb0 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec93b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55922b1a0a70 con 0x55922b2932f0 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec93b5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55922b1f5050 con 0x55922b2929d0 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec5579640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faeb00a2c90 0x7faeb00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec5579640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b29dbb0 0x55922b29ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faebffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b29d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faebdffb640 1 -- 172.21.15.62:0/1752396257 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7faeac0cea10 con 0x7faec0103740 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec5579640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b29dbb0 0x55922b29ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49676/0 (socket says 172.21.15.62:49676) 2024-09-11T23:51:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec5579640 1 -- 172.21.15.62:0/2586875611 learned_addr learned my addr 172.21.15.62:0/2586875611 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.160+0000 7faec4d78640 1 --2- 172.21.15.62:0/2586875611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b2929d0 0x55922b292db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.161+0000 7faebffff640 1 -- 172.21.15.62:0/2586875611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b2929d0 msgr2=0x55922b292db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.161+0000 7faebffff640 1 --2- 172.21.15.62:0/2586875611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b2929d0 0x55922b292db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.161+0000 7faebffff640 1 -- 172.21.15.62:0/2586875611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b29dbb0 msgr2=0x55922b29ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.161+0000 7faebffff640 1 --2- 172.21.15.62:0/2586875611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b29dbb0 0x55922b29ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.161+0000 7faebffff640 1 -- 172.21.15.62:0/2586875611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55922b1ad130 con 0x55922b2932f0 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.161+0000 7faec5579640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faeb00a2c90 0x7faeb00a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009430 tx=0x7faeb8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.161+0000 7faebffff640 1 --2- 172.21.15.62:0/2586875611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b29d670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faeac0cc3d0 tx=0x7faeac0047f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c471cf0e05dd5e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.161+0000 7faeaaffd640 1 -- 172.21.15.62:0/2586875611 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac020040 con 0x55922b2932f0 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.161+0000 7faeaaffd640 1 -- 172.21.15.62:0/2586875611 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeac0994d0 con 0x55922b2932f0 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.161+0000 7faeaaffd640 1 -- 172.21.15.62:0/2586875611 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac0c83e0 con 0x55922b2932f0 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 -- 172.21.15.62:0/2586875611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 msgr2=0x55922b29d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 --2- 172.21.15.62:0/2586875611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b29d670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faeac0cc3d0 tx=0x7faeac0047f0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 -- 172.21.15.62:0/2586875611 shutdown_connections 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 --2- 172.21.15.62:0/2586875611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b29dbb0 0x55922b29ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 --2- 172.21.15.62:0/2586875611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b29d670 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 --2- 172.21.15.62:0/2586875611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b2929d0 0x55922b292db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 -- 172.21.15.62:0/2586875611 >> 172.21.15.62:0/2586875611 conn(0x55922b28ef60 msgr2=0x55922b28f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 -- 172.21.15.62:0/2586875611 shutdown_connections 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 -- 172.21.15.62:0/2586875611 wait complete. 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 -- start start 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.162+0000 7faec93b5740 1 Processor -- start 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec93b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b29d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec93b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b29a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec93b5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b29dbb0 0x55922b29acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec93b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55922b2a0780 con 0x55922b2929d0 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec93b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55922b1b3f20 con 0x55922b2932f0 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec93b5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55922b2a0900 con 0x55922b29dbb0 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec93b5740 10 client.0 _ll_register_callbacks cb 0x55922b17d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec4d78640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b29d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec4d78640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b29d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49680/0 (socket says 172.21.15.62:49680) 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec4d78640 1 -- 172.21.15.62:0/2183814349 learned_addr learned my addr 172.21.15.62:0/2183814349 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec5579640 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b29dbb0 0x55922b29acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faebffff640 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b29a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec4d78640 1 -- 172.21.15.62:0/2183814349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b29dbb0 msgr2=0x55922b29acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec4d78640 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b29dbb0 0x55922b29acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec4d78640 1 -- 172.21.15.62:0/2183814349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 msgr2=0x55922b29a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec4d78640 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b29a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec4d78640 1 -- 172.21.15.62:0/2183814349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55922af6a2b0 con 0x55922b2929d0 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faebffff640 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b29a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec4d78640 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b29d420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faeb400ba20 tx=0x7faeb400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a926219f683d114d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7fae9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7fae9ffff640 1 -- 172.21.15.62:0/2183814349 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb401b640 con 0x55922b2929d0 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7fae9ffff640 1 -- 172.21.15.62:0/2183814349 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7faeb401b7e0 con 0x55922b2929d0 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.163+0000 7faec93b5740 1 -- 172.21.15.62:0/2183814349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeac0994d0 con 0x55922b2929d0 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7fae9ffff640 1 -- 172.21.15.62:0/2183814349 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faeb4003ea0 con 0x55922b2929d0 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7faec93b5740 10 client.14961 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7faec93b5740 1 -- 172.21.15.62:0/2183814349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55922b2ac120 con 0x55922b2929d0 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7fae9ffff640 1 -- 172.21.15.62:0/2183814349 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeac0994d0 con 0x55922b2929d0 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7fae9ffff640 1 -- 172.21.15.62:0/2183814349 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7faeb4019020 con 0x55922b2929d0 2024-09-11T23:51:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7faec93b5740 10 client.14961 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7faec93b5740 10 client.14961 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:51:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7faec93b5740 20 client.14961 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7faec93b5740 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55922b2ae710 0x55922b2afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7faec93b5740 4 client.14961 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:51:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7faec93b5740 1 -- 172.21.15.62:0/2183814349 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55922b206600 con 0x55922b2ae710 2024-09-11T23:51:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.164+0000 7faebffff640 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55922b2ae710 0x55922b2afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.165+0000 7faebffff640 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55922b2ae710 0x55922b2afb80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b69d8a60e75c6b8c server_cookie=137e78315a106875 in_seq=0 out_seq=0 2024-09-11T23:51:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.165+0000 7fae9ffff640 10 client.14961 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:51:34.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.165+0000 7fae9ffff640 1 -- 172.21.15.62:0/2183814349 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55922b206600 con 0x55922b2ae710 2024-09-11T23:51:34.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.165+0000 7fae9ffff640 10 client.14961 handle_command_reply: tid=0 2024-09-11T23:51:34.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.165+0000 7faec93b5740 1 client.14961 shutdown 2024-09-11T23:51:34.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 1 -- 172.21.15.62:0/2183814349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 msgr2=0x55922b29d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:34.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b29d420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faeb400ba20 tx=0x7faeb400bee0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 1 -- 172.21.15.62:0/2183814349 shutdown_connections 2024-09-11T23:51:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55922b2ae710 0x55922b2afb80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b29dbb0 0x55922b29acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b29a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7fae9ffff640 0 client.14961 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:51:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 1 --2- 172.21.15.62:0/2183814349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b29d420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 1 -- 172.21.15.62:0/2183814349 >> 172.21.15.62:0/2183814349 conn(0x55922b28ef60 msgr2=0x55922b294c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 1 -- 172.21.15.62:0/2183814349 shutdown_connections 2024-09-11T23:51:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 1 -- 172.21.15.62:0/2183814349 wait complete. 2024-09-11T23:51:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.166+0000 7faec93b5740 20 client.14961 trim_cache size 0 max 16384 2024-09-11T23:51:34.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec93b5740 1 Processor -- start 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec93b5740 1 -- start start 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec93b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b2afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec93b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b289ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec93b5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b29dbb0 0x55922b29c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec93b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55922b2adec0 con 0x55922b2929d0 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec93b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55922b2ae590 con 0x55922b2932f0 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec93b5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55922b2aec60 con 0x55922b29dbb0 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec4d78640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b2afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec4d78640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b2afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49686/0 (socket says 172.21.15.62:49686) 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.184+0000 7faec4d78640 1 -- 172.21.15.62:0/3175319131 learned_addr learned my addr 172.21.15.62:0/3175319131 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:34.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.185+0000 7faec4d78640 1 -- 172.21.15.62:0/3175319131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b29dbb0 msgr2=0x55922b29c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:34.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.185+0000 7faec4d78640 1 --2- 172.21.15.62:0/3175319131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b29dbb0 0x55922b29c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.185+0000 7faebffff640 1 --2- 172.21.15.62:0/3175319131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b289ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.185+0000 7faec4d78640 1 -- 172.21.15.62:0/3175319131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 msgr2=0x55922b289ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:34.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.185+0000 7faec4d78640 1 --2- 172.21.15.62:0/3175319131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b289ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.185+0000 7faec4d78640 1 -- 172.21.15.62:0/3175319131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55922b299720 con 0x55922b2929d0 2024-09-11T23:51:34.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.185+0000 7faec4d78640 1 --2- 172.21.15.62:0/3175319131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b2afb80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faeb4001eb0 tx=0x7faeb40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4de8f98abfc5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:34.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.185+0000 7faeaa7fc640 1 -- 172.21.15.62:0/3175319131 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4005240 con 0x55922b2929d0 2024-09-11T23:51:34.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.185+0000 7faeaa7fc640 1 -- 172.21.15.62:0/3175319131 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeb4011920 con 0x55922b2929d0 2024-09-11T23:51:34.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.185+0000 7faeaa7fc640 1 -- 172.21.15.62:0/3175319131 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4004b70 con 0x55922b2929d0 2024-09-11T23:51:34.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 -- 172.21.15.62:0/3175319131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 msgr2=0x55922b2afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:34.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 --2- 172.21.15.62:0/3175319131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b2afb80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faeb4001eb0 tx=0x7faeb40044c0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 -- 172.21.15.62:0/3175319131 shutdown_connections 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 --2- 172.21.15.62:0/3175319131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b29dbb0 0x55922b29c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 --2- 172.21.15.62:0/3175319131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b2932f0 0x55922b289ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 --2- 172.21.15.62:0/3175319131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2929d0 0x55922b2afb80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 -- 172.21.15.62:0/3175319131 >> 172.21.15.62:0/3175319131 conn(0x55922b28ef60 msgr2=0x55922b29bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 -- 172.21.15.62:0/3175319131 shutdown_connections 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 -- 172.21.15.62:0/3175319131 wait complete. 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 -- start start 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.186+0000 7faec93b5740 1 Processor -- start 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faec93b5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b2929d0 0x55922b2b8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faec93b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2932f0 0x55922b2b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faec93b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b29dbb0 0x55922b2b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faec93b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55922b2afa00 con 0x55922b2932f0 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faec93b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55922b2af330 con 0x55922b29dbb0 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faec93b5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55922b2b00d0 con 0x55922b2929d0 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faec93b5740 10 client.0 _ll_register_callbacks cb 0x7faeb4004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faebffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2932f0 0x55922b2b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faebffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2932f0 0x55922b2b8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49698/0 (socket says 172.21.15.62:49698) 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faebffff640 1 -- 172.21.15.62:0/1159850075 learned_addr learned my addr 172.21.15.62:0/1159850075 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faec4d78640 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b2929d0 0x55922b2b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faec5579640 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b29dbb0 0x55922b2b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faebffff640 1 -- 172.21.15.62:0/1159850075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b2929d0 msgr2=0x55922b2b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faebffff640 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b2929d0 0x55922b2b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faebffff640 1 -- 172.21.15.62:0/1159850075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b29dbb0 msgr2=0x55922b2b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faebffff640 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b29dbb0 0x55922b2b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faebffff640 1 -- 172.21.15.62:0/1159850075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55922b293120 con 0x55922b2932f0 2024-09-11T23:51:34.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faebffff640 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2932f0 0x55922b2b8740 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faeac09b160 tx=0x7faeac09b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707e7a4abd2534ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7fae9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7fae9ffff640 1 -- 172.21.15.62:0/1159850075 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac020040 con 0x55922b2932f0 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7fae9ffff640 1 -- 172.21.15.62:0/1159850075 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7faeac0bb0b0 con 0x55922b2932f0 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.187+0000 7faec93b5740 1 -- 172.21.15.62:0/1159850075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeb4011920 con 0x55922b2932f0 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7fae9ffff640 1 -- 172.21.15.62:0/1159850075 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faeac097470 con 0x55922b2932f0 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7faec93b5740 10 client.14967 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7faec93b5740 1 -- 172.21.15.62:0/1159850075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55922b3188e0 con 0x55922b2932f0 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7fae9ffff640 1 -- 172.21.15.62:0/1159850075 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeb4011920 con 0x55922b2932f0 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7fae9ffff640 1 -- 172.21.15.62:0/1159850075 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7faeac09a620 con 0x55922b2932f0 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7faec93b5740 10 client.14967 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7faec93b5740 10 client.14967 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7faec93b5740 20 client.14967 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:34.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7faec93b5740 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55922b2ae710 0x55922b31bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7faec93b5740 4 client.14967 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7faec93b5740 1 -- 172.21.15.62:0/1159850075 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "status"}) -- 0x55922b200b10 con 0x55922b2ae710 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.188+0000 7faec4d78640 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55922b2ae710 0x55922b31bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7fae9ffff640 10 client.14967 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec4d78640 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55922b2ae710 0x55922b31bcc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2399534c3ff29fc8 server_cookie=b96e4da50ba0ac17 in_seq=0 out_seq=0 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7fae9ffff640 1 -- 172.21.15.62:0/1159850075 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55922b200b10 con 0x55922b2ae710 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7fae9ffff640 10 client.14967 handle_command_reply: tid=0 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 client.14967 shutdown 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 -- 172.21.15.62:0/1159850075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2932f0 msgr2=0x55922b2b8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2932f0 0x55922b2b8740 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faeac09b160 tx=0x7faeac09b350 comp rx=0 tx=0).stop 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 -- 172.21.15.62:0/1159850075 shutdown_connections 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55922b2ae710 0x55922b31bcc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55922b29dbb0 0x55922b2b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55922b2932f0 0x55922b2b8740 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7fae9ffff640 0 client.14967 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:51:34.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 --2- 172.21.15.62:0/1159850075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55922b2929d0 0x55922b2b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 -- 172.21.15.62:0/1159850075 >> 172.21.15.62:0/1159850075 conn(0x55922b28ef60 msgr2=0x55922b28e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:34.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 -- 172.21.15.62:0/1159850075 shutdown_connections 2024-09-11T23:51:34.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 1 -- 172.21.15.62:0/1159850075 wait complete. 2024-09-11T23:51:34.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.189+0000 7faec93b5740 20 client.14967 trim_cache size 0 max 16384 2024-09-11T23:51:34.194 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:51:34.194 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:51:34.194 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-09-11T23:51:34.194 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24469, 2024-09-11T23:51:34.195 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-09-11T23:51:34.195 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-09-11T23:51:34.195 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-09-11T23:51:34.195 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 74.331912897999999, 2024-09-11T23:51:34.195 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 24, 2024-09-11T23:51:34.195 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 83, 2024-09-11T23:51:34.195 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:51:34.195 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 107.398000688 2024-09-11T23:51:34.195 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 -- 172.21.15.62:0/1752396257 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faeb00a2c90 msgr2=0x7faeb00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faeb00a2c90 0x7faeb00a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009430 tx=0x7faeb8009360 comp rx=0 tx=0).stop 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 -- 172.21.15.62:0/1752396257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0103740 msgr2=0x7faec019ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0103740 0x7faec019ac90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faeac00d740 tx=0x7faeac00dc00 comp rx=0 tx=0).stop 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 -- 172.21.15.62:0/1752396257 shutdown_connections 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faeb00a2c90 0x7faeb00a5150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faec0108290 0x7faec019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0103740 0x7faec019ac90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 --2- 172.21.15.62:0/1752396257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faec0102d90 0x7faec019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 -- 172.21.15.62:0/1752396257 >> 172.21.15.62:0/1752396257 conn(0x7faec00fc7d0 msgr2=0x7faec0100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:34.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 -- 172.21.15.62:0/1752396257 shutdown_connections 2024-09-11T23:51:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:34.190+0000 7faec6fdc640 1 -- 172.21.15.62:0/1752396257 wait complete. 2024-09-11T23:51:34.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:34 smithi083 ceph-mon[34805]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:34.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:34 smithi062 ceph-mon[30958]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:34.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:34 smithi120 ceph-mon[35182]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:35.088 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:51:35.347 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:35.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.806+0000 7f2fb8881640 1 -- 172.21.15.62:0/637682837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb4102d30 msgr2=0x7f2fb4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.806+0000 7f2fb8881640 1 --2- 172.21.15.62:0/637682837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb4102d30 0x7f2fb4103110 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009cd0 tx=0x7f2fa402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.806+0000 7f2fb8881640 1 -- 172.21.15.62:0/637682837 shutdown_connections 2024-09-11T23:51:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.806+0000 7f2fb8881640 1 --2- 172.21.15.62:0/637682837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fb41082a0 0x7f2fb410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.806+0000 7f2fb8881640 1 --2- 172.21.15.62:0/637682837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb41036e0 0x7f2fb4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.806+0000 7f2fb8881640 1 --2- 172.21.15.62:0/637682837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb4102d30 0x7f2fb4103110 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.806+0000 7f2fb8881640 1 -- 172.21.15.62:0/637682837 >> 172.21.15.62:0/637682837 conn(0x7f2fb40fc7d0 msgr2=0x7f2fb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.806+0000 7f2fb8881640 1 -- 172.21.15.62:0/637682837 shutdown_connections 2024-09-11T23:51:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.806+0000 7f2fb8881640 1 -- 172.21.15.62:0/637682837 wait complete. 2024-09-11T23:51:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb8881640 1 Processor -- start 2024-09-11T23:51:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb8881640 1 -- start start 2024-09-11T23:51:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb8881640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fb4102d30 0x7f2fb4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb8881640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb41036e0 0x7f2fb4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb8881640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb41082a0 0x7f2fb419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb8881640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4112100 con 0x7f2fb41036e0 2024-09-11T23:51:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb8881640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4111f80 con 0x7f2fb41082a0 2024-09-11T23:51:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb8881640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4112280 con 0x7f2fb4102d30 2024-09-11T23:51:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb41036e0 0x7f2fb4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb41036e0 0x7f2fb4198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49706/0 (socket says 172.21.15.62:49706) 2024-09-11T23:51:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb1d9b640 1 -- 172.21.15.62:0/4147120244 learned_addr learned my addr 172.21.15.62:0/4147120244 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb2d9d640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb41082a0 0x7f2fb419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb259c640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fb4102d30 0x7f2fb4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb1d9b640 1 -- 172.21.15.62:0/4147120244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fb4102d30 msgr2=0x7f2fb4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fb4102d30 0x7f2fb4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb1d9b640 1 -- 172.21.15.62:0/4147120244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb41082a0 msgr2=0x7f2fb419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb41082a0 0x7f2fb419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.807+0000 7f2fb1d9b640 1 -- 172.21.15.62:0/4147120244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb419d5d0 con 0x7f2fb41036e0 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.808+0000 7f2fb259c640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fb4102d30 0x7f2fb4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.808+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb41036e0 0x7f2fb4198b60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00d740 tx=0x7f2f9c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.808+0000 7f2fab7fe640 1 -- 172.21.15.62:0/4147120244 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c017070 con 0x7f2fb41036e0 2024-09-11T23:51:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.808+0000 7f2fb8881640 1 -- 172.21.15.62:0/4147120244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb419d8c0 con 0x7f2fb41036e0 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.808+0000 7f2fab7fe640 1 -- 172.21.15.62:0/4147120244 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f9c00de50 con 0x7f2fb41036e0 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.808+0000 7f2fab7fe640 1 -- 172.21.15.62:0/4147120244 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c00f6b0 con 0x7f2fb41036e0 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.808+0000 7f2fb8881640 1 -- 172.21.15.62:0/4147120244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb41a51a0 con 0x7f2fb41036e0 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.811+0000 7f2fbac5a740 1 Processor -- start 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.811+0000 7f2fbac5a740 1 -- start start 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.811+0000 7f2fab7fe640 1 -- 172.21.15.62:0/4147120244 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2f9c01a070 con 0x7f2fb41036e0 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.811+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed6e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.811+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e12f0 0x5563ed6eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.811+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 0x5563ed6edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.811+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5563ed647090 con 0x5563ed6e09d0 2024-09-11T23:51:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.811+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563ed5eea70 con 0x5563ed6ebbb0 2024-09-11T23:51:35.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.811+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5563ed643050 con 0x5563ed6e12f0 2024-09-11T23:51:35.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed6e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed6e0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49718/0 (socket says 172.21.15.62:49718) 2024-09-11T23:51:35.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb2d9d640 1 -- 172.21.15.62:0/2114732953 learned_addr learned my addr 172.21.15.62:0/2114732953 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:35.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb259c640 1 --2- 172.21.15.62:0/2114732953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e12f0 0x5563ed6eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/2114732953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 0x5563ed6edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb2d9d640 1 -- 172.21.15.62:0/2114732953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e12f0 msgr2=0x5563ed6eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:35.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb2d9d640 1 --2- 172.21.15.62:0/2114732953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e12f0 0x5563ed6eb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb2d9d640 1 -- 172.21.15.62:0/2114732953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 msgr2=0x5563ed6edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:35.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb2d9d640 1 --2- 172.21.15.62:0/2114732953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 0x5563ed6edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb2d9d640 1 -- 172.21.15.62:0/2114732953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ed5fb130 con 0x5563ed6e09d0 2024-09-11T23:51:35.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/2114732953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 0x5563ed6edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:35.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fb2d9d640 1 --2- 172.21.15.62:0/2114732953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed6e0db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00a8a0 tx=0x7f2fac00ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4306cc43ddbaed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:35.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fa8ff9640 1 -- 172.21.15.62:0/2114732953 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac015070 con 0x5563ed6e09d0 2024-09-11T23:51:35.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fa8ff9640 1 -- 172.21.15.62:0/2114732953 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fac00cb40 con 0x5563ed6e09d0 2024-09-11T23:51:35.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fa8ff9640 1 -- 172.21.15.62:0/2114732953 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac018560 con 0x5563ed6e09d0 2024-09-11T23:51:35.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.812+0000 7f2fab7fe640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2fa00a2c10 0x7f2fa00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fb259c640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2fa00a2c10 0x7f2fa00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fab7fe640 1 -- 172.21.15.62:0/4147120244 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2f9c0cec80 con 0x7f2fb41036e0 2024-09-11T23:51:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2114732953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 msgr2=0x5563ed6e0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/2114732953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed6e0db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00a8a0 tx=0x7f2fac00ad60 comp rx=0 tx=0).stop 2024-09-11T23:51:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fb259c640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2fa00a2c10 0x7f2fa00a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4005e30 tx=0x7f2fa4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2114732953 shutdown_connections 2024-09-11T23:51:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/2114732953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 0x5563ed6edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/2114732953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e12f0 0x5563ed6eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/2114732953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed6e0db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2114732953 >> 172.21.15.62:0/2114732953 conn(0x5563ed6dcf60 msgr2=0x5563ed6dd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2114732953 shutdown_connections 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.813+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2114732953 wait complete. 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fbac5a740 1 -- start start 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fbac5a740 1 Processor -- start 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 0x5563ed6eb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e12f0 0x5563ed6e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 0x5563ed6e8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5563ed6ee780 con 0x5563ed6e12f0 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563ed601f20 con 0x5563ed6ebbb0 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5563ed6ee900 con 0x5563ed6e09d0 2024-09-11T23:51:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fbac5a740 10 client.0 _ll_register_callbacks cb 0x5563ed5cb030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e12f0 0x5563ed6e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e12f0 0x5563ed6e8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49730/0 (socket says 172.21.15.62:49730) 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb1d9b640 1 -- 172.21.15.62:0/1805388681 learned_addr learned my addr 172.21.15.62:0/1805388681 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb2d9d640 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 0x5563ed6e8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb259c640 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 0x5563ed6eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb1d9b640 1 -- 172.21.15.62:0/1805388681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 msgr2=0x5563ed6eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 0x5563ed6eb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb1d9b640 1 -- 172.21.15.62:0/1805388681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 msgr2=0x5563ed6e8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 0x5563ed6e8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb1d9b640 1 -- 172.21.15.62:0/1805388681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ed3b82b0 con 0x5563ed6e12f0 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.814+0000 7f2fb259c640 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 0x5563ed6eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e12f0 0x5563ed6e8760 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009610 tx=0x7f2f9c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18812fe6b0db6890 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2f99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2f99ffb640 1 -- 172.21.15.62:0/1805388681 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c021070 con 0x5563ed6e12f0 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2f99ffb640 1 -- 172.21.15.62:0/1805388681 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2f9c09a7e0 con 0x5563ed6e12f0 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2fbac5a740 1 -- 172.21.15.62:0/1805388681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fac00cb40 con 0x5563ed6e12f0 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2f99ffb640 1 -- 172.21.15.62:0/1805388681 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2f9c0a4000 con 0x5563ed6e12f0 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2fbac5a740 10 client.14979 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2fbac5a740 1 -- 172.21.15.62:0/1805388681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5563ed6f9fe0 con 0x5563ed6e12f0 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2f99ffb640 1 -- 172.21.15.62:0/1805388681 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fac00cb40 con 0x5563ed6e12f0 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2f99ffb640 1 -- 172.21.15.62:0/1805388681 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2f9c0a08d0 con 0x5563ed6e12f0 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2fbac5a740 10 client.14979 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2fbac5a740 10 client.14979 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2fbac5a740 20 client.14979 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5563ed6fc600 0x5563ed6fda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2fbac5a740 4 client.14979 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:51:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.815+0000 7f2fbac5a740 1 -- 172.21.15.62:0/1805388681 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563ed654600 con 0x5563ed6fc600 2024-09-11T23:51:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.816+0000 7f2fb259c640 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5563ed6fc600 0x5563ed6fda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.816+0000 7f2fb259c640 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5563ed6fc600 0x5563ed6fda70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5a31589552d1249 server_cookie=4ce0b20db411b5db in_seq=0 out_seq=0 2024-09-11T23:51:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.816+0000 7f2f99ffb640 10 client.14979 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:51:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2f99ffb640 1 -- 172.21.15.62:0/1805388681 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563ed654600 con 0x5563ed6fc600 2024-09-11T23:51:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2f99ffb640 10 client.14979 handle_command_reply: tid=0 2024-09-11T23:51:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 client.14979 shutdown 2024-09-11T23:51:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 -- 172.21.15.62:0/1805388681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e12f0 msgr2=0x5563ed6e8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e12f0 0x5563ed6e8760 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009610 tx=0x7f2f9c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 -- 172.21.15.62:0/1805388681 shutdown_connections 2024-09-11T23:51:35.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5563ed6fc600 0x5563ed6fda70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6ebbb0 0x5563ed6e8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e12f0 0x5563ed6e8760 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/1805388681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 0x5563ed6eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 -- 172.21.15.62:0/1805388681 >> 172.21.15.62:0/1805388681 conn(0x5563ed6dcf60 msgr2=0x5563ed6e2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:35.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2f99ffb640 0 client.14979 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:51:35.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 -- 172.21.15.62:0/1805388681 shutdown_connections 2024-09-11T23:51:35.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 1 -- 172.21.15.62:0/1805388681 wait complete. 2024-09-11T23:51:35.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.817+0000 7f2fbac5a740 20 client.14979 trim_cache size 0 max 16384 2024-09-11T23:51:35.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.835+0000 7f2fbac5a740 1 Processor -- start 2024-09-11T23:51:35.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.835+0000 7f2fbac5a740 1 -- start start 2024-09-11T23:51:35.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.835+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 0x5563ed6fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.835+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 0x5563ed6fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.835+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6ebbb0 0x5563ed6e8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.835+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5563ed6fbdb0 con 0x5563ed6ebbb0 2024-09-11T23:51:35.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.835+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563ed6fc480 con 0x5563ed6e12f0 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.835+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5563ed6fcb50 con 0x5563ed6e09d0 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6ebbb0 0x5563ed6e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6ebbb0 0x5563ed6e8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49742/0 (socket says 172.21.15.62:49742) 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb2d9d640 1 -- 172.21.15.62:0/941065657 learned_addr learned my addr 172.21.15.62:0/941065657 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb259c640 1 --2- 172.21.15.62:0/941065657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 0x5563ed6fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/941065657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 0x5563ed6fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb2d9d640 1 -- 172.21.15.62:0/941065657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 msgr2=0x5563ed6fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb2d9d640 1 --2- 172.21.15.62:0/941065657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 0x5563ed6fbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb2d9d640 1 -- 172.21.15.62:0/941065657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 msgr2=0x5563ed6fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb2d9d640 1 --2- 172.21.15.62:0/941065657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 0x5563ed6fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb2d9d640 1 -- 172.21.15.62:0/941065657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ed6ee310 con 0x5563ed6ebbb0 2024-09-11T23:51:35.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb259c640 1 --2- 172.21.15.62:0/941065657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 0x5563ed6fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2fb2d9d640 1 --2- 172.21.15.62:0/941065657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6ebbb0 0x5563ed6e8720 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2fac000c00 tx=0x7f2fac00a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20210a0e31ea9db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2f9bfff640 1 -- 172.21.15.62:0/941065657 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac029070 con 0x5563ed6ebbb0 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2f9bfff640 1 -- 172.21.15.62:0/941065657 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fac00ce20 con 0x5563ed6ebbb0 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.836+0000 7f2f9bfff640 1 -- 172.21.15.62:0/941065657 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac015040 con 0x5563ed6ebbb0 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 -- 172.21.15.62:0/941065657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6ebbb0 msgr2=0x5563ed6e8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/941065657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6ebbb0 0x5563ed6e8720 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2fac000c00 tx=0x7f2fac00a7a0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 -- 172.21.15.62:0/941065657 shutdown_connections 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/941065657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6ebbb0 0x5563ed6e8720 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/941065657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 0x5563ed6fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/941065657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6e09d0 0x5563ed6fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 -- 172.21.15.62:0/941065657 >> 172.21.15.62:0/941065657 conn(0x5563ed6dcf60 msgr2=0x5563ed6d7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 -- 172.21.15.62:0/941065657 shutdown_connections 2024-09-11T23:51:35.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 -- 172.21.15.62:0/941065657 wait complete. 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 -- start start 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 Processor -- start 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed700e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 0x5563ed7013a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6ebbb0 0x5563ed76d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5563ed6fd8f0 con 0x5563ed6e09d0 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563ed6fd220 con 0x5563ed6e12f0 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.837+0000 7f2fbac5a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5563ed6fdfc0 con 0x5563ed6ebbb0 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fbac5a740 10 client.0 _ll_register_callbacks cb 0x7f2f9c0976b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed700e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed700e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49758/0 (socket says 172.21.15.62:49758) 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb259c640 1 -- 172.21.15.62:0/2095268952 learned_addr learned my addr 172.21.15.62:0/2095268952 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 0x5563ed7013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb2d9d640 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6ebbb0 0x5563ed76d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb259c640 1 -- 172.21.15.62:0/2095268952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6ebbb0 msgr2=0x5563ed76d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb259c640 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6ebbb0 0x5563ed76d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb259c640 1 -- 172.21.15.62:0/2095268952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 msgr2=0x5563ed7013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb259c640 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 0x5563ed7013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb259c640 1 -- 172.21.15.62:0/2095268952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ed701c90 con 0x5563ed6e09d0 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 0x5563ed7013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fb259c640 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed700e60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009ca0 tx=0x7f2fa403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9163292c44be0d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2f99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2f99ffb640 1 -- 172.21.15.62:0/2095268952 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4054070 con 0x5563ed6e09d0 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2f99ffb640 1 -- 172.21.15.62:0/2095268952 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2fa40045d0 con 0x5563ed6e09d0 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2095268952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fac00ce20 con 0x5563ed6e09d0 2024-09-11T23:51:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2f99ffb640 1 -- 172.21.15.62:0/2095268952 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2fa4060050 con 0x5563ed6e09d0 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fbac5a740 10 client.14985 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.838+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2095268952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5563ed76eb50 con 0x5563ed6e09d0 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2f99ffb640 1 -- 172.21.15.62:0/2095268952 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fac00ce20 con 0x5563ed6e09d0 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2f99ffb640 1 -- 172.21.15.62:0/2095268952 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2fa4030070 con 0x5563ed6e09d0 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2fbac5a740 10 client.14985 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2fbac5a740 10 client.14985 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2fbac5a740 20 client.14985 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5563ed7736a0 0x5563ed7723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2fbac5a740 4 client.14985 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2095268952 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "status"}) -- 0x5563ed64eb10 con 0x5563ed7736a0 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5563ed7736a0 0x5563ed7723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2f99ffb640 10 client.14985 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.839+0000 7f2fb1d9b640 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5563ed7736a0 0x5563ed7723b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2b1b2ed95f005a3 server_cookie=b544afb90bc93d9a in_seq=0 out_seq=0 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2f99ffb640 1 -- 172.21.15.62:0/2095268952 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5563ed64eb10 con 0x5563ed7736a0 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2f99ffb640 10 client.14985 handle_command_reply: tid=0 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 client.14985 shutdown 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2095268952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 msgr2=0x5563ed700e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed700e60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009ca0 tx=0x7f2fa403e040 comp rx=0 tx=0).stop 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2095268952 shutdown_connections 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5563ed7736a0 0x5563ed7723b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5563ed6ebbb0 0x5563ed76d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ed6e12f0 0x5563ed7013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 --2- 172.21.15.62:0/2095268952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5563ed6e09d0 0x5563ed700e60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2f99ffb640 0 client.14985 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:51:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2095268952 >> 172.21.15.62:0/2095268952 conn(0x5563ed6dcf60 msgr2=0x5563ed6dd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2095268952 shutdown_connections 2024-09-11T23:51:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 1 -- 172.21.15.62:0/2095268952 wait complete. 2024-09-11T23:51:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.840+0000 7f2fbac5a740 20 client.14985 trim_cache size 0 max 16384 2024-09-11T23:51:35.862 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:51:35.862 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:51:35.862 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-09-11T23:51:35.862 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24484, 2024-09-11T23:51:35.862 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-09-11T23:51:35.862 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-09-11T23:51:35.862 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-09-11T23:51:35.863 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 76.987618466000001, 2024-09-11T23:51:35.863 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 24, 2024-09-11T23:51:35.863 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 83, 2024-09-11T23:51:35.863 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:51:35.863 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 103.77718369199999 2024-09-11T23:51:35.863 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:51:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.841+0000 7f2fb8881640 1 -- 172.21.15.62:0/4147120244 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2fa00a2c10 msgr2=0x7f2fa00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.841+0000 7f2fb8881640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2fa00a2c10 0x7f2fa00a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4005e30 tx=0x7f2fa4005d80 comp rx=0 tx=0).stop 2024-09-11T23:51:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.841+0000 7f2fb8881640 1 -- 172.21.15.62:0/4147120244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb41036e0 msgr2=0x7f2fb4198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.841+0000 7f2fb8881640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb41036e0 0x7f2fb4198b60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00d740 tx=0x7f2f9c00dc00 comp rx=0 tx=0).stop 2024-09-11T23:51:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.841+0000 7f2fb8881640 1 -- 172.21.15.62:0/4147120244 shutdown_connections 2024-09-11T23:51:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.841+0000 7f2fb8881640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f2fa00a2c10 0x7f2fa00a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.841+0000 7f2fb8881640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb41082a0 0x7f2fb419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.841+0000 7f2fb8881640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb41036e0 0x7f2fb4198b60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.841+0000 7f2fb8881640 1 --2- 172.21.15.62:0/4147120244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fb4102d30 0x7f2fb4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.841+0000 7f2fb8881640 1 -- 172.21.15.62:0/4147120244 >> 172.21.15.62:0/4147120244 conn(0x7f2fb40fc7d0 msgr2=0x7f2fb4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.842+0000 7f2fb8881640 1 -- 172.21.15.62:0/4147120244 shutdown_connections 2024-09-11T23:51:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:35.842+0000 7f2fb8881640 1 -- 172.21.15.62:0/4147120244 wait complete. 2024-09-11T23:51:36.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:36 smithi083 ceph-mon[34805]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:51:36.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:36 smithi062 ceph-mon[30958]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:51:36.573 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:51:36.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:36 smithi120 ceph-mon[35182]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:51:36.828 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:37.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.275+0000 7fd1f56a6640 1 -- 172.21.15.62:0/313560451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1f0103660 msgr2=0x7fd1f010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:37.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.275+0000 7fd1f56a6640 1 --2- 172.21.15.62:0/313560451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1f0103660 0x7fd1f010bb50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009900 tx=0x7fd1e402a480 comp rx=0 tx=0).stop 2024-09-11T23:51:37.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.275+0000 7fd1f56a6640 1 -- 172.21.15.62:0/313560451 shutdown_connections 2024-09-11T23:51:37.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.275+0000 7fd1f56a6640 1 --2- 172.21.15.62:0/313560451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1f010c170 0x7fd1f010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.275+0000 7fd1f56a6640 1 --2- 172.21.15.62:0/313560451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1f0103660 0x7fd1f010bb50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.275+0000 7fd1f56a6640 1 --2- 172.21.15.62:0/313560451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0102d40 0x7fd1f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.275+0000 7fd1f56a6640 1 -- 172.21.15.62:0/313560451 >> 172.21.15.62:0/313560451 conn(0x7fd1f00fc7a0 msgr2=0x7fd1f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:37.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.275+0000 7fd1f56a6640 1 -- 172.21.15.62:0/313560451 shutdown_connections 2024-09-11T23:51:37.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.275+0000 7fd1f56a6640 1 -- 172.21.15.62:0/313560451 wait complete. 2024-09-11T23:51:37.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1f56a6640 1 Processor -- start 2024-09-11T23:51:37.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1f56a6640 1 -- start start 2024-09-11T23:51:37.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1f56a6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1f0102d40 0x7fd1f01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1f56a6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1f0103660 0x7fd1f0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1f56a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f010c170 0x7fd1f0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1f56a6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0111fb0 con 0x7fd1f0102d40 2024-09-11T23:51:37.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1f56a6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0111e30 con 0x7fd1f010c170 2024-09-11T23:51:37.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1f56a6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0112130 con 0x7fd1f0103660 2024-09-11T23:51:37.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1eeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1f0102d40 0x7fd1f01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1eeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1f0102d40 0x7fd1f01009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49774/0 (socket says 172.21.15.62:49774) 2024-09-11T23:51:37.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1eeffd640 1 -- 172.21.15.62:0/1680633031 learned_addr learned my addr 172.21.15.62:0/1680633031 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:37.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.276+0000 7fd1ef7fe640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f010c170 0x7fd1f0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1eeffd640 1 -- 172.21.15.62:0/1680633031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1f0103660 msgr2=0x7fd1f0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:37.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1f0103660 0x7fd1f0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1eeffd640 1 -- 172.21.15.62:0/1680633031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f010c170 msgr2=0x7fd1f0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:37.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f010c170 0x7fd1f0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1eeffd640 1 -- 172.21.15.62:0/1680633031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f01a6930 con 0x7fd1f0102d40 2024-09-11T23:51:37.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1ef7fe640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f010c170 0x7fd1f0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:37.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1f0102d40 0x7fd1f01009a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c890 tx=0x7fd1e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:37.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1d7fff640 1 -- 172.21.15.62:0/1680633031 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0004220 con 0x7fd1f0102d40 2024-09-11T23:51:37.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1d7fff640 1 -- 172.21.15.62:0/1680633031 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e00043c0 con 0x7fd1f0102d40 2024-09-11T23:51:37.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1f56a6640 1 -- 172.21.15.62:0/1680633031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f01a6c20 con 0x7fd1f0102d40 2024-09-11T23:51:37.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1d7fff640 1 -- 172.21.15.62:0/1680633031 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00049e0 con 0x7fd1f0102d40 2024-09-11T23:51:37.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.277+0000 7fd1f56a6640 1 -- 172.21.15.62:0/1680633031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f01a71b0 con 0x7fd1f0102d40 2024-09-11T23:51:37.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.279+0000 7fd1d7fff640 1 -- 172.21.15.62:0/1680633031 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd1e00079b0 con 0x7fd1f0102d40 2024-09-11T23:51:37.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.279+0000 7fd1f7a7f740 1 Processor -- start 2024-09-11T23:51:37.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1f7a7f740 1 -- start start 2024-09-11T23:51:37.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f739ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 0x5615f73a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5615f7304090 con 0x5615f73a8bb0 2024-09-11T23:51:37.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5615f72aba70 con 0x5615f739e2f0 2024-09-11T23:51:37.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615f7300050 con 0x5615f739d9d0 2024-09-11T23:51:37.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49778/0 (socket says 172.21.15.62:49778) 2024-09-11T23:51:37.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/1912193042 learned_addr learned my addr 172.21.15.62:0/1912193042 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:37.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/1912193042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 0x5615f73a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1ef7fe640 1 --2- 172.21.15.62:0/1912193042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f739ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/1912193042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 msgr2=0x5615f739ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:37.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/1912193042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f739ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/1912193042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 msgr2=0x5615f73a8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:37.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/1912193042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 0x5615f73a8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.280+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/1912193042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f72b8130 con 0x5615f73a8bb0 2024-09-11T23:51:37.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.281+0000 7fd1d7fff640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1d80a2c90 0x7fd1d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.281+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/1912193042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73aafa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009a20 tx=0x7fd1e4031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c96fd8a077df25 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:37.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.281+0000 7fd1ef7fe640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1d80a2c90 0x7fd1d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.281+0000 7fd1d7fff640 1 -- 172.21.15.62:0/1680633031 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd1e00cf2b0 con 0x7fd1f0102d40 2024-09-11T23:51:37.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.281+0000 7fd1d57fa640 1 -- 172.21.15.62:0/1912193042 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4040070 con 0x5615f73a8bb0 2024-09-11T23:51:37.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.281+0000 7fd1d57fa640 1 -- 172.21.15.62:0/1912193042 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e4004040 con 0x5615f73a8bb0 2024-09-11T23:51:37.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.281+0000 7fd1d57fa640 1 -- 172.21.15.62:0/1912193042 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4035040 con 0x5615f73a8bb0 2024-09-11T23:51:37.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/1912193042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 msgr2=0x5615f73aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:37.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/1912193042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73aafa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009a20 tx=0x7fd1e4031510 comp rx=0 tx=0).stop 2024-09-11T23:51:37.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1ef7fe640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1d80a2c90 0x7fd1d80a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800a400 tx=0x7fd1e8009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:37.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/1912193042 shutdown_connections 2024-09-11T23:51:37.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/1912193042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73aafa0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/1912193042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 0x5615f73a8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/1912193042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f739ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/1912193042 >> 172.21.15.62:0/1912193042 conn(0x5615f7399f60 msgr2=0x5615f739a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:37.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/1912193042 shutdown_connections 2024-09-11T23:51:37.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/1912193042 wait complete. 2024-09-11T23:51:37.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 -- start start 2024-09-11T23:51:37.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.282+0000 7fd1f7a7f740 1 Processor -- start 2024-09-11T23:51:37.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f73a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f73a8bb0 0x5615f73a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5615f73ab780 con 0x5615f739e2f0 2024-09-11T23:51:37.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5615f72bef20 con 0x5615f73a8bb0 2024-09-11T23:51:37.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615f73ab900 con 0x5615f739d9d0 2024-09-11T23:51:37.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1f7a7f740 10 client.0 _ll_register_callbacks cb 0x5615f7288030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:37.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49792/0 (socket says 172.21.15.62:49792) 2024-09-11T23:51:37.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/189622935 learned_addr learned my addr 172.21.15.62:0/189622935 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:37.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f73a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/189622935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 msgr2=0x5615f73a8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:37.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f73a8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/189622935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f73a8bb0 msgr2=0x5615f73a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:37.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f73a8bb0 0x5615f73a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/189622935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f70752b0 con 0x5615f739e2f0 2024-09-11T23:51:37.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a56a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006010 tx=0x7fd1e4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8c404efb75d167 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:37.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:37.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/189622935 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4035040 con 0x5615f739e2f0 2024-09-11T23:51:37.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/189622935 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd1e40317b0 con 0x5615f739e2f0 2024-09-11T23:51:37.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.283+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/189622935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e4004040 con 0x5615f739e2f0 2024-09-11T23:51:37.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/189622935 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd1e4040040 con 0x5615f739e2f0 2024-09-11T23:51:37.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1f7a7f740 10 client.14997 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:37.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/189622935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5615f73b6fc0 con 0x5615f739e2f0 2024-09-11T23:51:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/189622935 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e4004040 con 0x5615f739e2f0 2024-09-11T23:51:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/189622935 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd1e40457c0 con 0x5615f739e2f0 2024-09-11T23:51:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1f7a7f740 10 client.14997 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1f7a7f740 10 client.14997 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:51:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1f7a7f740 20 client.14997 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615f73b9180 0x5615f73ba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1f7a7f740 4 client.14997 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:51:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/189622935 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f7311600 con 0x5615f73b9180 2024-09-11T23:51:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.284+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615f73b9180 0x5615f73ba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.285+0000 7fd1d2ff5640 10 client.14997 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:51:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.285+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615f73b9180 0x5615f73ba5f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a97f5937297d411 server_cookie=cfbba1b71000c978 in_seq=0 out_seq=0 2024-09-11T23:51:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/189622935 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615f7311600 con 0x5615f73b9180 2024-09-11T23:51:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1d2ff5640 10 client.14997 handle_command_reply: tid=0 2024-09-11T23:51:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 client.14997 shutdown 2024-09-11T23:51:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/189622935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 msgr2=0x5615f73a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a56a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006010 tx=0x7fd1e4009520 comp rx=0 tx=0).stop 2024-09-11T23:51:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/189622935 shutdown_connections 2024-09-11T23:51:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615f73b9180 0x5615f73ba5f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f73a8bb0 0x5615f73a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a56a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/189622935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f73a8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/189622935 >> 172.21.15.62:0/189622935 conn(0x5615f7399f60 msgr2=0x5615f739fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1d2ff5640 0 client.14997 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:51:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/189622935 shutdown_connections 2024-09-11T23:51:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/189622935 wait complete. 2024-09-11T23:51:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.286+0000 7fd1f7a7f740 20 client.14997 trim_cache size 0 max 16384 2024-09-11T23:51:37.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1f7a7f740 1 Processor -- start 2024-09-11T23:51:37.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1f7a7f740 1 -- start start 2024-09-11T23:51:37.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f73b9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 0x5615f7394ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5615f73b8930 con 0x5615f73a8bb0 2024-09-11T23:51:37.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5615f73b9000 con 0x5615f739e2f0 2024-09-11T23:51:37.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615f73b96d0 con 0x5615f739d9d0 2024-09-11T23:51:37.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1ef7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1ef7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73a2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49796/0 (socket says 172.21.15.62:49796) 2024-09-11T23:51:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1ef7fe640 1 -- 172.21.15.62:0/2449500433 learned_addr learned my addr 172.21.15.62:0/2449500433 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/2449500433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f73b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1ef7fe640 1 -- 172.21.15.62:0/2449500433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 msgr2=0x5615f73b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/2449500433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 0x5615f7394ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1ef7fe640 1 --2- 172.21.15.62:0/2449500433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f73b9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1ef7fe640 1 -- 172.21.15.62:0/2449500433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 msgr2=0x5615f7394ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:37.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1ef7fe640 1 --2- 172.21.15.62:0/2449500433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 0x5615f7394ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1ef7fe640 1 -- 172.21.15.62:0/2449500433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f739e120 con 0x5615f73a8bb0 2024-09-11T23:51:37.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.304+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/2449500433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 0x5615f7394ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:37.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1ef7fe640 1 --2- 172.21.15.62:0/2449500433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73a2940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1e801a750 tx=0x7fd1e800e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e35cd1763a6f60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:37.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1d4ff9640 1 -- 172.21.15.62:0/2449500433 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e801a0d0 con 0x5615f73a8bb0 2024-09-11T23:51:37.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1d4ff9640 1 -- 172.21.15.62:0/2449500433 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e801a270 con 0x5615f73a8bb0 2024-09-11T23:51:37.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1d4ff9640 1 -- 172.21.15.62:0/2449500433 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8023800 con 0x5615f73a8bb0 2024-09-11T23:51:37.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2449500433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 msgr2=0x5615f73a2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:37.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/2449500433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73a2940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1e801a750 tx=0x7fd1e800e050 comp rx=0 tx=0).stop 2024-09-11T23:51:37.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2449500433 shutdown_connections 2024-09-11T23:51:37.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/2449500433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f73a8bb0 0x5615f73a2940 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/2449500433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739e2f0 0x5615f7394ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/2449500433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f739d9d0 0x5615f73b9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2449500433 >> 172.21.15.62:0/2449500433 conn(0x5615f7399f60 msgr2=0x5615f73a0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.305+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2449500433 shutdown_connections 2024-09-11T23:51:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2449500433 wait complete. 2024-09-11T23:51:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1f7a7f740 1 -- start start 2024-09-11T23:51:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1f7a7f740 1 Processor -- start 2024-09-11T23:51:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739d9d0 0x5615f73a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1f7a7f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f73a8bb0 0x5615f73c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5615f73ba470 con 0x5615f739e2f0 2024-09-11T23:51:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5615f73b9da0 con 0x5615f739d9d0 2024-09-11T23:51:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1f7a7f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615f73bab40 con 0x5615f73a8bb0 2024-09-11T23:51:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1f7a7f740 10 client.0 _ll_register_callbacks cb 0x7fd1e00d2c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49802/0 (socket says 172.21.15.62:49802) 2024-09-11T23:51:37.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/2872871805 learned_addr learned my addr 172.21.15.62:0/2872871805 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:37.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1ef7fe640 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f73a8bb0 0x5615f73c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.306+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739d9d0 0x5615f73a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/2872871805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f73a8bb0 msgr2=0x5615f73c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:37.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f73a8bb0 0x5615f73c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/2872871805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739d9d0 msgr2=0x5615f73a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:37.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739d9d0 0x5615f73a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1ee7fc640 1 -- 172.21.15.62:0/2872871805 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f73a2b50 con 0x5615f739e2f0 2024-09-11T23:51:37.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1ee7fc640 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a2280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1e402a9a0 tx=0x7fd1e4032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efb90204f78fe63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:37.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:37.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/2872871805 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4035040 con 0x5615f739e2f0 2024-09-11T23:51:37.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2872871805 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615f730bb10 con 0x5615f739e2f0 2024-09-11T23:51:37.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/2872871805 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd1e4003660 con 0x5615f739e2f0 2024-09-11T23:51:37.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/2872871805 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd1e4005850 con 0x5615f739e2f0 2024-09-11T23:51:37.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1f7a7f740 10 client.15003 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:37.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2872871805 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5615f73c2db0 con 0x5615f739e2f0 2024-09-11T23:51:37.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.307+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/2872871805 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5615f730bb10 con 0x5615f739e2f0 2024-09-11T23:51:37.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.308+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/2872871805 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd1e4063420 con 0x5615f739e2f0 2024-09-11T23:51:37.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.308+0000 7fd1f7a7f740 10 client.15003 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:37.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.308+0000 7fd1f7a7f740 10 client.15003 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:51:37.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.308+0000 7fd1f7a7f740 20 client.15003 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:37.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.308+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615f7427970 0x5615f7426680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:37.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.308+0000 7fd1f7a7f740 4 client.15003 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:51:37.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.308+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2872871805 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "status"}) -- 0x5615f72880e0 con 0x5615f7427970 2024-09-11T23:51:37.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.308+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615f7427970 0x5615f7426680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:37.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.308+0000 7fd1eeffd640 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615f7427970 0x5615f7426680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25228deb6a9cf051 server_cookie=a111386adedb80b0 in_seq=0 out_seq=0 2024-09-11T23:51:37.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.308+0000 7fd1d2ff5640 10 client.15003 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:51:37.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1d2ff5640 1 -- 172.21.15.62:0/2872871805 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5615f72880e0 con 0x5615f7427970 2024-09-11T23:51:37.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1d2ff5640 10 client.15003 handle_command_reply: tid=0 2024-09-11T23:51:37.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 client.15003 shutdown 2024-09-11T23:51:37.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2872871805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 msgr2=0x5615f73a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:37.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a2280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1e402a9a0 tx=0x7fd1e4032bc0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2872871805 shutdown_connections 2024-09-11T23:51:37.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615f7427970 0x5615f7426680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f73a8bb0 0x5615f73c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1d2ff5640 0 client.15003 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:51:37.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615f739e2f0 0x5615f73a2280 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 --2- 172.21.15.62:0/2872871805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f739d9d0 0x5615f73a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2872871805 >> 172.21.15.62:0/2872871805 conn(0x5615f7399f60 msgr2=0x5615f73a47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:37.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2872871805 shutdown_connections 2024-09-11T23:51:37.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 1 -- 172.21.15.62:0/2872871805 wait complete. 2024-09-11T23:51:37.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f7a7f740 20 client.15003 trim_cache size 0 max 16384 2024-09-11T23:51:37.339 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:51:37.339 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:51:37.340 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-09-11T23:51:37.340 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24488, 2024-09-11T23:51:37.340 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-09-11T23:51:37.340 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-09-11T23:51:37.340 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-09-11T23:51:37.340 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 85.439848945999998, 2024-09-11T23:51:37.340 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 24, 2024-09-11T23:51:37.340 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 83, 2024-09-11T23:51:37.341 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:51:37.341 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 96.621506933999996 2024-09-11T23:51:37.341 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:51:37.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f56a6640 1 -- 172.21.15.62:0/1680633031 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1d80a2c90 msgr2=0x7fd1d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:37.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f56a6640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1d80a2c90 0x7fd1d80a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800a400 tx=0x7fd1e8009610 comp rx=0 tx=0).stop 2024-09-11T23:51:37.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f56a6640 1 -- 172.21.15.62:0/1680633031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1f0102d40 msgr2=0x7fd1f01009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:37.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.309+0000 7fd1f56a6640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1f0102d40 0x7fd1f01009a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c890 tx=0x7fd1e000cd50 comp rx=0 tx=0).stop 2024-09-11T23:51:37.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.310+0000 7fd1f56a6640 1 -- 172.21.15.62:0/1680633031 shutdown_connections 2024-09-11T23:51:37.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.310+0000 7fd1f56a6640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1d80a2c90 0x7fd1d80a5150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.310+0000 7fd1f56a6640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f010c170 0x7fd1f0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.310+0000 7fd1f56a6640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1f0103660 0x7fd1f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.310+0000 7fd1f56a6640 1 --2- 172.21.15.62:0/1680633031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1f0102d40 0x7fd1f01009a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:37.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.310+0000 7fd1f56a6640 1 -- 172.21.15.62:0/1680633031 >> 172.21.15.62:0/1680633031 conn(0x7fd1f00fc7a0 msgr2=0x7fd1f00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:37.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.310+0000 7fd1f56a6640 1 -- 172.21.15.62:0/1680633031 shutdown_connections 2024-09-11T23:51:37.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:37.310+0000 7fd1f56a6640 1 -- 172.21.15.62:0/1680633031 wait complete. 2024-09-11T23:51:38.079 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T23:51:38.079 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T23:51:38.089 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-09-11T23:51:38.089 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T23:51:38.349 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:38.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:38 smithi083 ceph-mon[34805]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:38.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:38 smithi062 ceph-mon[30958]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:38.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:38 smithi120 ceph-mon[35182]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:38.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.809+0000 7f99a2b73640 1 -- 172.21.15.62:0/3955063163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999c1026d0 msgr2=0x7f999c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:38.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.809+0000 7f99a2b73640 1 --2- 172.21.15.62:0/3955063163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999c1026d0 0x7f999c108f60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9994009ca0 tx=0x7f999402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:51:38.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.809+0000 7f99a2b73640 1 -- 172.21.15.62:0/3955063163 shutdown_connections 2024-09-11T23:51:38.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.809+0000 7f99a2b73640 1 --2- 172.21.15.62:0/3955063163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999c1026d0 0x7f999c108f60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:38.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.809+0000 7f99a2b73640 1 --2- 172.21.15.62:0/3955063163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f999c101d10 0x7f999c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:38.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.809+0000 7f99a2b73640 1 --2- 172.21.15.62:0/3955063163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c100b10 0x7f999c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:38.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.809+0000 7f99a2b73640 1 -- 172.21.15.62:0/3955063163 >> 172.21.15.62:0/3955063163 conn(0x7f999c0fc2a0 msgr2=0x7f999c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:38.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.809+0000 7f99a2b73640 1 -- 172.21.15.62:0/3955063163 shutdown_connections 2024-09-11T23:51:38.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.809+0000 7f99a2b73640 1 -- 172.21.15.62:0/3955063163 wait complete. 2024-09-11T23:51:38.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f99a2b73640 1 Processor -- start 2024-09-11T23:51:38.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f99a2b73640 1 -- start start 2024-09-11T23:51:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f99a2b73640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f999c100b10 0x7f999c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f99a2b73640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999c101d10 0x7f999c19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f99a2b73640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c1026d0 0x7f999c19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f99a2b73640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f999c10c770 con 0x7f999c101d10 2024-09-11T23:51:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f99a2b73640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f999c10c5f0 con 0x7f999c1026d0 2024-09-11T23:51:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f99a2b73640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f999c10c8f0 con 0x7f999c100b10 2024-09-11T23:51:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f999bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999c101d10 0x7f999c19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f999bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999c101d10 0x7f999c19cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49812/0 (socket says 172.21.15.62:49812) 2024-09-11T23:51:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f999bfff640 1 -- 172.21.15.62:0/2724434508 learned_addr learned my addr 172.21.15.62:0/2724434508 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f99a090f640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f999c100b10 0x7f999c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f99a1110640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c1026d0 0x7f999c19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f999bfff640 1 -- 172.21.15.62:0/2724434508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f999c100b10 msgr2=0x7f999c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.810+0000 7f999bfff640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f999c100b10 0x7f999c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.811+0000 7f999bfff640 1 -- 172.21.15.62:0/2724434508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c1026d0 msgr2=0x7f999c19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.811+0000 7f999bfff640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c1026d0 0x7f999c19f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.811+0000 7f999bfff640 1 -- 172.21.15.62:0/2724434508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c19fa70 con 0x7f999c101d10 2024-09-11T23:51:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.811+0000 7f999bfff640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999c101d10 0x7f999c19cb40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f998800c920 tx=0x7f998800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.811+0000 7f9999ffb640 1 -- 172.21.15.62:0/2724434508 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988007c70 con 0x7f999c101d10 2024-09-11T23:51:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.811+0000 7f99a2b73640 1 -- 172.21.15.62:0/2724434508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c19fd00 con 0x7f999c101d10 2024-09-11T23:51:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.811+0000 7f9999ffb640 1 -- 172.21.15.62:0/2724434508 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9988007e10 con 0x7f999c101d10 2024-09-11T23:51:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.811+0000 7f9999ffb640 1 -- 172.21.15.62:0/2724434508 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988018890 con 0x7f999c101d10 2024-09-11T23:51:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.811+0000 7f99a2b73640 1 -- 172.21.15.62:0/2724434508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c1a01e0 con 0x7f999c101d10 2024-09-11T23:51:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.812+0000 7f99a2b73640 1 -- 172.21.15.62:0/2724434508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c107e60 con 0x7f999c101d10 2024-09-11T23:51:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.813+0000 7f9999ffb640 1 -- 172.21.15.62:0/2724434508 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9988018b30 con 0x7f999c101d10 2024-09-11T23:51:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.814+0000 7f9999ffb640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f998c0a2c10 0x7f998c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.814+0000 7f99a090f640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f998c0a2c10 0x7f998c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.814+0000 7f9999ffb640 1 -- 172.21.15.62:0/2724434508 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f99880cf260 con 0x7f999c101d10 2024-09-11T23:51:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.814+0000 7f99a090f640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f998c0a2c10 0x7f998c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f999c101b70 tx=0x7f999000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:38.818+0000 7f9999ffb640 1 -- 172.21.15.62:0/2724434508 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9988014030 con 0x7f999c101d10 2024-09-11T23:51:39.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.016+0000 7f99a2b73640 1 -- 172.21.15.62:0/2724434508 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f999c062630 con 0x7f998c0a2c10 2024-09-11T23:51:39.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.018+0000 7f9999ffb640 1 -- 172.21.15.62:0/2724434508 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f999c062630 con 0x7f998c0a2c10 2024-09-11T23:51:39.016 INFO:teuthology.orchestra.run.smithi062.stdout:Backend: cephadm 2024-09-11T23:51:39.017 INFO:teuthology.orchestra.run.smithi062.stdout:Available: Yes 2024-09-11T23:51:39.017 INFO:teuthology.orchestra.run.smithi062.stdout:Paused: No 2024-09-11T23:51:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 -- 172.21.15.62:0/2724434508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f998c0a2c10 msgr2=0x7f998c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f998c0a2c10 0x7f998c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f999c101b70 tx=0x7f999000b390 comp rx=0 tx=0).stop 2024-09-11T23:51:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 -- 172.21.15.62:0/2724434508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999c101d10 msgr2=0x7f999c19cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999c101d10 0x7f999c19cb40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f998800c920 tx=0x7f998800cde0 comp rx=0 tx=0).stop 2024-09-11T23:51:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 -- 172.21.15.62:0/2724434508 shutdown_connections 2024-09-11T23:51:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f998c0a2c10 0x7f998c0a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c1026d0 0x7f999c19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999c101d10 0x7f999c19cb40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:39.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 --2- 172.21.15.62:0/2724434508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f999c100b10 0x7f999c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:39.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 -- 172.21.15.62:0/2724434508 >> 172.21.15.62:0/2724434508 conn(0x7f999c0fc2a0 msgr2=0x7f999c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:39.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 -- 172.21.15.62:0/2724434508 shutdown_connections 2024-09-11T23:51:39.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:39.020+0000 7f99a2b73640 1 -- 172.21.15.62:0/2724434508 wait complete. 2024-09-11T23:51:39.618 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T23:51:39.872 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:40.300 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:40 smithi062 ceph-mon[30958]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:51:40.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:40 smithi062 ceph-mon[30958]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:40.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.385+0000 7febc2184640 1 -- 172.21.15.62:0/3290463493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febbc104fd0 msgr2=0x7febbc1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:40.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.385+0000 7febc2184640 1 --2- 172.21.15.62:0/3290463493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febbc104fd0 0x7febbc1073c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7febb0009900 tx=0x7febb002a410 comp rx=0 tx=0).stop 2024-09-11T23:51:40.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.385+0000 7febc2184640 1 -- 172.21.15.62:0/3290463493 shutdown_connections 2024-09-11T23:51:40.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.385+0000 7febc2184640 1 --2- 172.21.15.62:0/3290463493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc107900 0x7febbc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:40.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.385+0000 7febc2184640 1 --2- 172.21.15.62:0/3290463493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febbc104fd0 0x7febbc1073c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:40.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.385+0000 7febc2184640 1 --2- 172.21.15.62:0/3290463493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc1026a0 0x7febbc104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:40.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.385+0000 7febc2184640 1 -- 172.21.15.62:0/3290463493 >> 172.21.15.62:0/3290463493 conn(0x7febbc0fc2a0 msgr2=0x7febbc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:40.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.385+0000 7febc2184640 1 -- 172.21.15.62:0/3290463493 shutdown_connections 2024-09-11T23:51:40.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.385+0000 7febc2184640 1 -- 172.21.15.62:0/3290463493 wait complete. 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.385+0000 7febc2184640 1 Processor -- start 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febc2184640 1 -- start start 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febc2184640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc1026a0 0x7febbc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febc2184640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febbc104fd0 0x7febbc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febc2184640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc107900 0x7febbc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febc2184640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7febbc10d970 con 0x7febbc104fd0 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febc2184640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7febbc10d7f0 con 0x7febbc107900 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febc2184640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7febbc10daf0 con 0x7febbc1026a0 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febbc104fd0 0x7febbc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febbc104fd0 0x7febbc1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49820/0 (socket says 172.21.15.62:49820) 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbaffd640 1 -- 172.21.15.62:0/1143413709 learned_addr learned my addr 172.21.15.62:0/1143413709 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:40.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbb7fe640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc1026a0 0x7febbc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbbfff640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc107900 0x7febbc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbaffd640 1 -- 172.21.15.62:0/1143413709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc1026a0 msgr2=0x7febbc198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbaffd640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc1026a0 0x7febbc198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbaffd640 1 -- 172.21.15.62:0/1143413709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc107900 msgr2=0x7febbc19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbaffd640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc107900 0x7febbc19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbaffd640 1 -- 172.21.15.62:0/1143413709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc19fcb0 con 0x7febbc104fd0 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbbfff640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc107900 0x7febbc19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.386+0000 7febbaffd640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febbc104fd0 0x7febbc1985c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7febb0031610 tx=0x7febb00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.387+0000 7febb8ff9640 1 -- 172.21.15.62:0/1143413709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb003f030 con 0x7febbc104fd0 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.387+0000 7febb8ff9640 1 -- 172.21.15.62:0/1143413709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb0003ce0 con 0x7febbc104fd0 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.387+0000 7febc2184640 1 -- 172.21.15.62:0/1143413709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc19ff40 con 0x7febbc104fd0 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.387+0000 7febb8ff9640 1 -- 172.21.15.62:0/1143413709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0035070 con 0x7febbc104fd0 2024-09-11T23:51:40.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.387+0000 7febc2184640 1 -- 172.21.15.62:0/1143413709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc1a0410 con 0x7febbc104fd0 2024-09-11T23:51:40.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.388+0000 7febc2184640 1 -- 172.21.15.62:0/1143413709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febbc0fdd90 con 0x7febbc104fd0 2024-09-11T23:51:40.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.389+0000 7febb8ff9640 1 -- 172.21.15.62:0/1143413709 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7febb0042030 con 0x7febbc104fd0 2024-09-11T23:51:40.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.389+0000 7febb8ff9640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feba40a2c40 0x7feba40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:40.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.390+0000 7febbb7fe640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feba40a2c40 0x7feba40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:40.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.390+0000 7febb8ff9640 1 -- 172.21.15.62:0/1143413709 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7febb00ef900 con 0x7febbc104fd0 2024-09-11T23:51:40.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.390+0000 7febbb7fe640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feba40a2c40 0x7feba40a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7febac009890 tx=0x7febac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:40.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.394+0000 7febb8ff9640 1 -- 172.21.15.62:0/1143413709 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febb00bc710 con 0x7febbc104fd0 2024-09-11T23:51:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:40 smithi083 ceph-mon[34805]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:51:40.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:40 smithi083 ceph-mon[34805]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:40.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.579+0000 7febc2184640 1 -- 172.21.15.62:0/1143413709 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7febbc062630 con 0x7feba40a2c40 2024-09-11T23:51:40.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.591+0000 7febb8ff9640 1 -- 172.21.15.62:0/1143413709 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7febbc062630 con 0x7feba40a2c40 2024-09-11T23:51:40.590 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T23:51:40.590 INFO:teuthology.orchestra.run.smithi062.stdout:mds.a smithi062 running (102s) 26s ago 102s 15.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 0ee0b356bad1 2024-09-11T23:51:40.590 INFO:teuthology.orchestra.run.smithi062.stdout:mds.b smithi083 running (100s) 33s ago 99s 28.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 0c5fa4907a3f 2024-09-11T23:51:40.590 INFO:teuthology.orchestra.run.smithi062.stdout:mds.c smithi120 running (114s) 33s ago 113s 18.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 518513c8e43a 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mds.d smithi062 running (2m) 26s ago 2m 15.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 2d408df0c1e5 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mds.e smithi083 running (119s) 33s ago 118s 16.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 f8cbd6eead05 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mds.f smithi120 running (116s) 33s ago 116s 19.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 33a9ed429cee 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mds.g smithi062 running (106s) 26s ago 105s 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 4589a60950bf 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mds.h smithi083 running (111s) 33s ago 110s 16.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 e72378e7512a 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mds.i smithi120 running (108s) 33s ago 108s 19.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 ab1501235c6d 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mds.j smithi062 running (2m) 26s ago 2m 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 200292cae119 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mds.k smithi083 running (2m) 33s ago 2m 16.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 b49664dd6ce3 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mds.l smithi120 running (2m) 33s ago 2m 16.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 86a7bf9d1c12 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi062 *:9283,8765 running (9m) 26s ago 9m 508M - 19.1.1-298-gb53b6d3e 54467abc55f8 bff0f724599e 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi083 *:8443,8765 running (7m) 33s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 15708569c0cb 2024-09-11T23:51:40.591 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.z smithi120 *:8443,8765 running (7m) 33s ago 7m 444M - 19.1.1-298-gb53b6d3e 54467abc55f8 251ba4218880 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (9m) 26s ago 9m 52.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 577ea3d597ec 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi083 running (7m) 33s ago 7m 39.9M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 123c1129f195 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi120 running (7m) 33s ago 7m 41.9M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 1e4cf4400adb 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (7m) 26s ago 7m 47.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 5637f0f671bf 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi083 running (6m) 33s ago 6m 45.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 b0c242843920 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi120 running (6m) 33s ago 6m 46.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 0acfb330dc23 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (5m) 26s ago 5m 42.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 103726feb9d6 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi083 running (5m) 33s ago 5m 54.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 b0dc95dde028 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi120 running (5m) 33s ago 5m 58.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 2b358fcec06d 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi062 running (4m) 26s ago 4m 53.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 4424946309ad 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi083 running (4m) 33s ago 4m 44.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 bee4081ba0bf 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 smithi120 running (3m) 33s ago 3m 43.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 dbc5bf1a6a5c 2024-09-11T23:51:40.592 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 smithi062 running (3m) 26s ago 3m 41.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 5c7c6b23b5ab 2024-09-11T23:51:40.593 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 smithi083 running (2m) 33s ago 2m 44.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 8e8d2d85217b 2024-09-11T23:51:40.593 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 smithi120 running (2m) 33s ago 2m 40.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 b699d2e4f0f2 2024-09-11T23:51:40.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 -- 172.21.15.62:0/1143413709 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feba40a2c40 msgr2=0x7feba40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:40.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feba40a2c40 0x7feba40a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7febac009890 tx=0x7febac008040 comp rx=0 tx=0).stop 2024-09-11T23:51:40.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 -- 172.21.15.62:0/1143413709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febbc104fd0 msgr2=0x7febbc1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:40.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febbc104fd0 0x7febbc1985c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7febb0031610 tx=0x7febb00043b0 comp rx=0 tx=0).stop 2024-09-11T23:51:40.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 -- 172.21.15.62:0/1143413709 shutdown_connections 2024-09-11T23:51:40.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7feba40a2c40 0x7feba40a5100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:40.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc107900 0x7febbc19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:40.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febbc104fd0 0x7febbc1985c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:40.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 --2- 172.21.15.62:0/1143413709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febbc1026a0 0x7febbc198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:40.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 -- 172.21.15.62:0/1143413709 >> 172.21.15.62:0/1143413709 conn(0x7febbc0fc2a0 msgr2=0x7febbc103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:40.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 -- 172.21.15.62:0/1143413709 shutdown_connections 2024-09-11T23:51:40.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:40.594+0000 7febc2184640 1 -- 172.21.15.62:0/1143413709 wait complete. 2024-09-11T23:51:40.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:40 smithi120 ceph-mon[35182]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:51:40.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:40 smithi120 ceph-mon[35182]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:41.315 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:41 smithi062 ceph-mon[30958]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:41.345 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T23:51:41.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:41 smithi083 ceph-mon[34805]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:41.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:41 smithi120 ceph-mon[35182]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:42.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.017+0000 7fafddadf640 1 -- 172.21.15.62:0/3848184486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8100ad0 msgr2=0x7fafd8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:42.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.017+0000 7fafddadf640 1 --2- 172.21.15.62:0/3848184486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8100ad0 0x7fafd8100ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009900 tx=0x7fafc802a3f0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.018+0000 7fafddadf640 1 -- 172.21.15.62:0/3848184486 shutdown_connections 2024-09-11T23:51:42.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.018+0000 7fafddadf640 1 --2- 172.21.15.62:0/3848184486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8102690 0x7fafd8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.018+0000 7fafddadf640 1 --2- 172.21.15.62:0/3848184486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafd8101cd0 0x7fafd8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.018+0000 7fafddadf640 1 --2- 172.21.15.62:0/3848184486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8100ad0 0x7fafd8100ed0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.018+0000 7fafddadf640 1 -- 172.21.15.62:0/3848184486 >> 172.21.15.62:0/3848184486 conn(0x7fafd80fc280 msgr2=0x7fafd80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:42.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.018+0000 7fafddadf640 1 -- 172.21.15.62:0/3848184486 shutdown_connections 2024-09-11T23:51:42.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.018+0000 7fafddadf640 1 -- 172.21.15.62:0/3848184486 wait complete. 2024-09-11T23:51:42.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.018+0000 7fafddadf640 1 Processor -- start 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.018+0000 7fafddadf640 1 -- start start 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafddadf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8100ad0 0x7fafd8197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafddadf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafd8101cd0 0x7fafd8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafddadf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8102690 0x7fafd819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafddadf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fafd810c780 con 0x7fafd8100ad0 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafddadf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafd810c600 con 0x7fafd8102690 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafddadf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafd810c900 con 0x7fafd8101cd0 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafd77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8100ad0 0x7fafd8197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafd77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8100ad0 0x7fafd8197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49954/0 (socket says 172.21.15.62:49954) 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafd77fe640 1 -- 172.21.15.62:0/3861495083 learned_addr learned my addr 172.21.15.62:0/3861495083 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafceffd640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafd8101cd0 0x7fafd8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafd7fff640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8102690 0x7fafd819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:42.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafd77fe640 1 -- 172.21.15.62:0/3861495083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafd8101cd0 msgr2=0x7fafd8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafd77fe640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafd8101cd0 0x7fafd8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafd77fe640 1 -- 172.21.15.62:0/3861495083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8102690 msgr2=0x7fafd819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafd77fe640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8102690 0x7fafd819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafd77fe640 1 -- 172.21.15.62:0/3861495083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd819fa60 con 0x7fafd8100ad0 2024-09-11T23:51:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.019+0000 7fafd77fe640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8100ad0 0x7fafd8197e30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fafc802a9e0 tx=0x7fafc8032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.020+0000 7fafd57fa640 1 -- 172.21.15.62:0/3861495083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc803f030 con 0x7fafd8100ad0 2024-09-11T23:51:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.020+0000 7fafddadf640 1 -- 172.21.15.62:0/3861495083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd819fcf0 con 0x7fafd8100ad0 2024-09-11T23:51:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.020+0000 7fafd57fa640 1 -- 172.21.15.62:0/3861495083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc8003ce0 con 0x7fafd8100ad0 2024-09-11T23:51:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.020+0000 7fafd57fa640 1 -- 172.21.15.62:0/3861495083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8035070 con 0x7fafd8100ad0 2024-09-11T23:51:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.020+0000 7fafddadf640 1 -- 172.21.15.62:0/3861495083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd81a0170 con 0x7fafd8100ad0 2024-09-11T23:51:42.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.021+0000 7fafddadf640 1 -- 172.21.15.62:0/3861495083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd81a03d0 con 0x7fafd8100ad0 2024-09-11T23:51:42.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.022+0000 7fafd57fa640 1 -- 172.21.15.62:0/3861495083 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fafc8042030 con 0x7fafd8100ad0 2024-09-11T23:51:42.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.023+0000 7fafd57fa640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fafc40a2cc0 0x7fafc40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:42.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.023+0000 7fafceffd640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fafc40a2cc0 0x7fafc40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:42.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.023+0000 7fafd57fa640 1 -- 172.21.15.62:0/3861495083 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fafc80ef8f0 con 0x7fafd8100ad0 2024-09-11T23:51:42.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.024+0000 7fafceffd640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fafc40a2cc0 0x7fafc40a5180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fafc0005e10 tx=0x7fafc0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:42.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.027+0000 7fafd57fa640 1 -- 172.21.15.62:0/3861495083 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafc80bc700 con 0x7fafd8100ad0 2024-09-11T23:51:42.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.209+0000 7fafddadf640 1 -- 172.21.15.62:0/3861495083 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fafd81a05a0 con 0x7fafc40a2cc0 2024-09-11T23:51:42.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.215+0000 7fafd57fa640 1 -- 172.21.15.62:0/3861495083 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fafd81a05a0 con 0x7fafc40a2cc0 2024-09-11T23:51:42.214 INFO:teuthology.orchestra.run.smithi062.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T23:51:42.214 INFO:teuthology.orchestra.run.smithi062.stdout:mds.all 12/12 35s ago 2m smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi120=c;smithi120=f;smithi120=i;smithi120=l;count:12 2024-09-11T23:51:42.214 INFO:teuthology.orchestra.run.smithi062.stdout:mgr 3/3 35s ago 7m smithi062=x;smithi083=y;smithi120=z;count:3 2024-09-11T23:51:42.214 INFO:teuthology.orchestra.run.smithi062.stdout:mon 3/3 35s ago 7m smithi062:172.21.15.62=a;smithi083:172.21.15.83=b;smithi120:172.21.15.120=c;count:3 2024-09-11T23:51:42.214 INFO:teuthology.orchestra.run.smithi062.stdout:osd 12 35s ago - 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 -- 172.21.15.62:0/3861495083 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fafc40a2cc0 msgr2=0x7fafc40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fafc40a2cc0 0x7fafc40a5180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fafc0005e10 tx=0x7fafc0005da0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 -- 172.21.15.62:0/3861495083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8100ad0 msgr2=0x7fafd8197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8100ad0 0x7fafd8197e30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fafc802a9e0 tx=0x7fafc8032da0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 -- 172.21.15.62:0/3861495083 shutdown_connections 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fafc40a2cc0 0x7fafc40a5180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8102690 0x7fafd819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafd8101cd0 0x7fafd8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 --2- 172.21.15.62:0/3861495083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8100ad0 0x7fafd8197e30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 -- 172.21.15.62:0/3861495083 >> 172.21.15.62:0/3861495083 conn(0x7fafd80fc280 msgr2=0x7fafd80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 -- 172.21.15.62:0/3861495083 shutdown_connections 2024-09-11T23:51:42.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:42.217+0000 7fafddadf640 1 -- 172.21.15.62:0/3861495083 wait complete. 2024-09-11T23:51:42.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:42 smithi083 ceph-mon[34805]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:42.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:42 smithi062 ceph-mon[30958]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:42.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:42 smithi120 ceph-mon[35182]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:51:42.919 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T23:51:43.188 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:43.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:43 smithi062 ceph-mon[30958]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:43 smithi120 ceph-mon[35182]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:43 smithi083 ceph-mon[34805]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.687+0000 7fbedf706640 1 -- 172.21.15.62:0/3029532892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8101d00 msgr2=0x7fbed8102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.687+0000 7fbedf706640 1 --2- 172.21.15.62:0/3029532892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8101d00 0x7fbed8102180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009900 tx=0x7fbed402a3f0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.687+0000 7fbedf706640 1 -- 172.21.15.62:0/3029532892 shutdown_connections 2024-09-11T23:51:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.687+0000 7fbedf706640 1 --2- 172.21.15.62:0/3029532892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbed81026c0 0x7fbed8108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.687+0000 7fbedf706640 1 --2- 172.21.15.62:0/3029532892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8101d00 0x7fbed8102180 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.687+0000 7fbedf706640 1 --2- 172.21.15.62:0/3029532892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbed8100b00 0x7fbed8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.687+0000 7fbedf706640 1 -- 172.21.15.62:0/3029532892 >> 172.21.15.62:0/3029532892 conn(0x7fbed80fc270 msgr2=0x7fbed80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.687+0000 7fbedf706640 1 -- 172.21.15.62:0/3029532892 shutdown_connections 2024-09-11T23:51:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.687+0000 7fbedf706640 1 -- 172.21.15.62:0/3029532892 wait complete. 2024-09-11T23:51:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.687+0000 7fbedf706640 1 Processor -- start 2024-09-11T23:51:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedf706640 1 -- start start 2024-09-11T23:51:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedf706640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8100b00 0x7fbed819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedf706640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbed8101d00 0x7fbed819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedf706640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbed81026c0 0x7fbed81a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedf706640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbed810d8f0 con 0x7fbed8100b00 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedf706640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbed810d770 con 0x7fbed8101d00 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedf706640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbed810da70 con 0x7fbed81026c0 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedd4a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8100b00 0x7fbed819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedd4a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8100b00 0x7fbed819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49980/0 (socket says 172.21.15.62:49980) 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedd4a2640 1 -- 172.21.15.62:0/3564178632 learned_addr learned my addr 172.21.15.62:0/3564178632 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedcca1640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbed8101d00 0x7fbed819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedd4a2640 1 -- 172.21.15.62:0/3564178632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbed81026c0 msgr2=0x7fbed81a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedd4a2640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbed81026c0 0x7fbed81a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedd4a2640 1 -- 172.21.15.62:0/3564178632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbed8101d00 msgr2=0x7fbed819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedd4a2640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbed8101d00 0x7fbed819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedd4a2640 1 -- 172.21.15.62:0/3564178632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed81a1e90 con 0x7fbed8100b00 2024-09-11T23:51:43.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.688+0000 7fbedd4a2640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8100b00 0x7fbed819a260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00d770 tx=0x7fbecc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:43.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.689+0000 7fbeca7fc640 1 -- 172.21.15.62:0/3564178632 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc017070 con 0x7fbed8100b00 2024-09-11T23:51:43.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.689+0000 7fbeca7fc640 1 -- 172.21.15.62:0/3564178632 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbecc009c30 con 0x7fbed8100b00 2024-09-11T23:51:43.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.689+0000 7fbedf706640 1 -- 172.21.15.62:0/3564178632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed81a2180 con 0x7fbed8100b00 2024-09-11T23:51:43.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.689+0000 7fbeca7fc640 1 -- 172.21.15.62:0/3564178632 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc003dd0 con 0x7fbed8100b00 2024-09-11T23:51:43.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.689+0000 7fbedf706640 1 -- 172.21.15.62:0/3564178632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed81a2740 con 0x7fbed8100b00 2024-09-11T23:51:43.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.690+0000 7fbedf706640 1 -- 172.21.15.62:0/3564178632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed81a29a0 con 0x7fbed8100b00 2024-09-11T23:51:43.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.691+0000 7fbeca7fc640 1 -- 172.21.15.62:0/3564178632 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbecc003f70 con 0x7fbed8100b00 2024-09-11T23:51:43.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.691+0000 7fbeca7fc640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbec40a2bb0 0x7fbec40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:43.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.692+0000 7fbeca7fc640 1 -- 172.21.15.62:0/3564178632 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fbecc0cea40 con 0x7fbed8100b00 2024-09-11T23:51:43.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.692+0000 7fbedcca1640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbec40a2bb0 0x7fbec40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:43.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.692+0000 7fbedcca1640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbec40a2bb0 0x7fbec40a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbed819b780 tx=0x7fbed40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:43.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.696+0000 7fbeca7fc640 1 -- 172.21.15.62:0/3564178632 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbecc09b9b0 con 0x7fbed8100b00 2024-09-11T23:51:43.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.876+0000 7fbedf706640 1 -- 172.21.15.62:0/3564178632 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fbed8108c40 con 0x7fbec40a2bb0 2024-09-11T23:51:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.877+0000 7fbeca7fc640 1 -- 172.21.15.62:0/3564178632 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fbed8108c40 con 0x7fbec40a2bb0 2024-09-11T23:51:43.876 INFO:teuthology.orchestra.run.smithi062.stdout:HOST ADDR LABELS STATUS 2024-09-11T23:51:43.876 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 172.21.15.62 2024-09-11T23:51:43.876 INFO:teuthology.orchestra.run.smithi062.stdout:smithi083 172.21.15.83 2024-09-11T23:51:43.876 INFO:teuthology.orchestra.run.smithi062.stdout:smithi120 172.21.15.120 2024-09-11T23:51:43.876 INFO:teuthology.orchestra.run.smithi062.stdout:3 hosts in cluster 2024-09-11T23:51:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 -- 172.21.15.62:0/3564178632 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbec40a2bb0 msgr2=0x7fbec40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbec40a2bb0 0x7fbec40a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbed819b780 tx=0x7fbed40315b0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 -- 172.21.15.62:0/3564178632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8100b00 msgr2=0x7fbed819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8100b00 0x7fbed819a260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00d770 tx=0x7fbecc00dc30 comp rx=0 tx=0).stop 2024-09-11T23:51:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 -- 172.21.15.62:0/3564178632 shutdown_connections 2024-09-11T23:51:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fbec40a2bb0 0x7fbec40a5070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbed81026c0 0x7fbed81a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbed8101d00 0x7fbed819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 --2- 172.21.15.62:0/3564178632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8100b00 0x7fbed819a260 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 -- 172.21.15.62:0/3564178632 >> 172.21.15.62:0/3564178632 conn(0x7fbed80fc270 msgr2=0x7fbed8107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 -- 172.21.15.62:0/3564178632 shutdown_connections 2024-09-11T23:51:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:43.879+0000 7fbedf706640 1 -- 172.21.15.62:0/3564178632 wait complete. 2024-09-11T23:51:44.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:44 smithi062 ceph-mon[30958]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:44.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:44 smithi062 ceph-mon[30958]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T23:51:44.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T23:51:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:44 smithi120 ceph-mon[35182]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:44 smithi120 ceph-mon[35182]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T23:51:44.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:44 smithi083 ceph-mon[34805]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:51:44.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:44 smithi083 ceph-mon[34805]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T23:51:44.832 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:45.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:45 smithi062 ceph-mon[30958]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.329+0000 7fe7d07aa640 1 -- 172.21.15.62:0/169340989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c8101c00 msgr2=0x7fe7c8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.329+0000 7fe7d07aa640 1 --2- 172.21.15.62:0/169340989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c8101c00 0x7fe7c8102000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009900 tx=0x7fe7c402a410 comp rx=0 tx=0).stop 2024-09-11T23:51:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.329+0000 7fe7d07aa640 1 -- 172.21.15.62:0/169340989 shutdown_connections 2024-09-11T23:51:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.329+0000 7fe7d07aa640 1 --2- 172.21.15.62:0/169340989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7c81037c0 0x7fe7c810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.329+0000 7fe7d07aa640 1 --2- 172.21.15.62:0/169340989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c8102e00 0x7fe7c8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.329+0000 7fe7d07aa640 1 --2- 172.21.15.62:0/169340989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c8101c00 0x7fe7c8102000 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.329+0000 7fe7d07aa640 1 -- 172.21.15.62:0/169340989 >> 172.21.15.62:0/169340989 conn(0x7fe7c80fd390 msgr2=0x7fe7c80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.329+0000 7fe7d07aa640 1 -- 172.21.15.62:0/169340989 shutdown_connections 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.329+0000 7fe7d07aa640 1 -- 172.21.15.62:0/169340989 wait complete. 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7d07aa640 1 Processor -- start 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7d07aa640 1 -- start start 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7d07aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c8101c00 0x7fe7c8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7d07aa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7c8102e00 0x7fe7c81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7d07aa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c81037c0 0x7fe7c819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7d07aa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c810d890 con 0x7fe7c81037c0 2024-09-11T23:51:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7d07aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c810d710 con 0x7fe7c8101c00 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7d07aa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c810da10 con 0x7fe7c8102e00 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7ced47640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c81037c0 0x7fe7c819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7ced47640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c81037c0 0x7fe7c819f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49992/0 (socket says 172.21.15.62:49992) 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7ced47640 1 -- 172.21.15.62:0/2143440710 learned_addr learned my addr 172.21.15.62:0/2143440710 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.330+0000 7fe7ce546640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c8101c00 0x7fe7c8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7cdd45640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7c8102e00 0x7fe7c81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7ced47640 1 -- 172.21.15.62:0/2143440710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7c8102e00 msgr2=0x7fe7c81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7ced47640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7c8102e00 0x7fe7c81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7ced47640 1 -- 172.21.15.62:0/2143440710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c8101c00 msgr2=0x7fe7c8197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7ced47640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c8101c00 0x7fe7c8197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7ced47640 1 -- 172.21.15.62:0/2143440710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c819fab0 con 0x7fe7c81037c0 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7cdd45640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7c8102e00 0x7fe7c81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7ced47640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c81037c0 0x7fe7c819f440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000eaa0 tx=0x7fe7c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:45.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7bf7fe640 1 -- 172.21.15.62:0/2143440710 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c000cdb0 con 0x7fe7c81037c0 2024-09-11T23:51:45.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7bf7fe640 1 -- 172.21.15.62:0/2143440710 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c0005020 con 0x7fe7c81037c0 2024-09-11T23:51:45.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7d07aa640 1 -- 172.21.15.62:0/2143440710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c819fda0 con 0x7fe7c81037c0 2024-09-11T23:51:45.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7bf7fe640 1 -- 172.21.15.62:0/2143440710 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00116b0 con 0x7fe7c81037c0 2024-09-11T23:51:45.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.331+0000 7fe7d07aa640 1 -- 172.21.15.62:0/2143440710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81a02b0 con 0x7fe7c81037c0 2024-09-11T23:51:45.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.332+0000 7fe7d07aa640 1 -- 172.21.15.62:0/2143440710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c8108f30 con 0x7fe7c81037c0 2024-09-11T23:51:45.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.333+0000 7fe7bf7fe640 1 -- 172.21.15.62:0/2143440710 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe7c0005560 con 0x7fe7c81037c0 2024-09-11T23:51:45.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.334+0000 7fe7bf7fe640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe7b40a2c30 0x7fe7b40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:45.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.334+0000 7fe7ce546640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe7b40a2c30 0x7fe7b40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:45.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.334+0000 7fe7bf7fe640 1 -- 172.21.15.62:0/2143440710 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe7c00cf190 con 0x7fe7c81037c0 2024-09-11T23:51:45.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.335+0000 7fe7ce546640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe7b40a2c30 0x7fe7b40a50f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4005ab0 tx=0x7fe7c4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:45.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.338+0000 7fe7bf7fe640 1 -- 172.21.15.62:0/2143440710 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7c0016030 con 0x7fe7c81037c0 2024-09-11T23:51:45.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.523+0000 7fe7d07aa640 1 -- 172.21.15.62:0/2143440710 --> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe7c8062630 con 0x7fe7b40a2c30 2024-09-11T23:51:45.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.525+0000 7fe7bf7fe640 1 -- 172.21.15.62:0/2143440710 <== mgr.14150 v2:172.21.15.62:6800/3106394779 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe7c8062630 con 0x7fe7b40a2c30 2024-09-11T23:51:45.524 INFO:teuthology.orchestra.run.smithi062.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T23:51:45.524 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-11T23:51:45.524 INFO:teuthology.orchestra.run.smithi062.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T23:51:45.524 INFO:teuthology.orchestra.run.smithi062.stdout:smithi120 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T23:51:45.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 -- 172.21.15.62:0/2143440710 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe7b40a2c30 msgr2=0x7fe7b40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:45.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe7b40a2c30 0x7fe7b40a50f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4005ab0 tx=0x7fe7c4000bc0 comp rx=0 tx=0).stop 2024-09-11T23:51:45.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 -- 172.21.15.62:0/2143440710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c81037c0 msgr2=0x7fe7c819f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:45.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c81037c0 0x7fe7c819f440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000eaa0 tx=0x7fe7c000ef60 comp rx=0 tx=0).stop 2024-09-11T23:51:45.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 -- 172.21.15.62:0/2143440710 shutdown_connections 2024-09-11T23:51:45.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fe7b40a2c30 0x7fe7b40a50f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:45.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c81037c0 0x7fe7c819f440 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:45.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7c8102e00 0x7fe7c81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:45.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 --2- 172.21.15.62:0/2143440710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c8101c00 0x7fe7c8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:45.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 -- 172.21.15.62:0/2143440710 >> 172.21.15.62:0/2143440710 conn(0x7fe7c80fd390 msgr2=0x7fe7c8106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:45.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 -- 172.21.15.62:0/2143440710 shutdown_connections 2024-09-11T23:51:45.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:45.527+0000 7fe7d07aa640 1 -- 172.21.15.62:0/2143440710 wait complete. 2024-09-11T23:51:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:45 smithi120 ceph-mon[35182]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:45.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:45 smithi083 ceph-mon[34805]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:46.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:46 smithi062 ceph-mon[30958]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-11T23:51:46.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:46 smithi062 ceph-mon[30958]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:46.295 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T23:51:46.304 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-09-11T23:51:46.304 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T23:51:46.561 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:46.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:46 smithi120 ceph-mon[35182]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-11T23:51:46.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:46 smithi120 ceph-mon[35182]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:46.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:46 smithi083 ceph-mon[34805]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-11T23:51:46.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:46 smithi083 ceph-mon[34805]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:51:47.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.028+0000 7faa6b2c1640 1 -- 172.21.15.62:0/172619000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa64101770 msgr2=0x7faa64101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:47.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.028+0000 7faa6b2c1640 1 --2- 172.21.15.62:0/172619000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa64101770 0x7faa64101c20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009ca0 tx=0x7faa5c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:51:47.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.028+0000 7faa6b2c1640 1 -- 172.21.15.62:0/172619000 shutdown_connections 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.028+0000 7faa6b2c1640 1 --2- 172.21.15.62:0/172619000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa64101770 0x7faa64101c20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.028+0000 7faa6b2c1640 1 --2- 172.21.15.62:0/172619000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa64100dd0 0x7faa64101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.028+0000 7faa6b2c1640 1 --2- 172.21.15.62:0/172619000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa64100490 0x7faa64100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.028+0000 7faa6b2c1640 1 -- 172.21.15.62:0/172619000 >> 172.21.15.62:0/172619000 conn(0x7faa640fc2a0 msgr2=0x7faa640fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.028+0000 7faa6b2c1640 1 -- 172.21.15.62:0/172619000 shutdown_connections 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.028+0000 7faa6b2c1640 1 -- 172.21.15.62:0/172619000 wait complete. 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6b2c1640 1 Processor -- start 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6b2c1640 1 -- start start 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6b2c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa64100490 0x7faa6419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6b2c1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa64100dd0 0x7faa6419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6b2c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa64101770 0x7faa641a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6b2c1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faa6410fae0 con 0x7faa64100dd0 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6b2c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa6410f960 con 0x7faa64100490 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6b2c1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faa6410fc60 con 0x7faa64101770 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6885c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa64100dd0 0x7faa6419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6885c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa64100dd0 0x7faa6419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50022/0 (socket says 172.21.15.62:50022) 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.029+0000 7faa6885c640 1 -- 172.21.15.62:0/3341336863 learned_addr learned my addr 172.21.15.62:0/3341336863 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6905d640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa64100490 0x7faa6419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6985e640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa64101770 0x7faa641a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6885c640 1 -- 172.21.15.62:0/3341336863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa64101770 msgr2=0x7faa641a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6885c640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa64101770 0x7faa641a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6885c640 1 -- 172.21.15.62:0/3341336863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa64100490 msgr2=0x7faa6419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6885c640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa64100490 0x7faa6419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:47.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6885c640 1 -- 172.21.15.62:0/3341336863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa641a1d40 con 0x7faa64100dd0 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6905d640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa64100490 0x7faa6419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6885c640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa64100dd0 0x7faa6419a650 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faa5400d950 tx=0x7faa5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa5a7fc640 1 -- 172.21.15.62:0/3341336863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54017030 con 0x7faa64100dd0 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6b2c1640 1 -- 172.21.15.62:0/3341336863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa641a1fd0 con 0x7faa64100dd0 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa5a7fc640 1 -- 172.21.15.62:0/3341336863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa54004020 con 0x7faa64100dd0 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa5a7fc640 1 -- 172.21.15.62:0/3341336863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa540047a0 con 0x7faa64100dd0 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.030+0000 7faa6b2c1640 1 -- 172.21.15.62:0/3341336863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa641a24e0 con 0x7faa64100dd0 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.031+0000 7faa6b2c1640 1 -- 172.21.15.62:0/3341336863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6410b1e0 con 0x7faa64100dd0 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.032+0000 7faa5a7fc640 1 -- 172.21.15.62:0/3341336863 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa540041d0 con 0x7faa64100dd0 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.033+0000 7faa5a7fc640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faa500a2bb0 0x7faa500a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.033+0000 7faa6905d640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faa500a2bb0 0x7faa500a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.033+0000 7faa5a7fc640 1 -- 172.21.15.62:0/3341336863 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7faa540ceb70 con 0x7faa64100dd0 2024-09-11T23:51:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.033+0000 7faa6905d640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faa500a2bb0 0x7faa500a5070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faa60009930 tx=0x7faa60008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.037+0000 7faa5a7fc640 1 -- 172.21.15.62:0/3341336863 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa5409ba80 con 0x7faa64100dd0 2024-09-11T23:51:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.252+0000 7faa6b2c1640 1 -- 172.21.15.62:0/3341336863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7faa64062e90 con 0x7faa64100dd0 2024-09-11T23:51:47.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.253+0000 7faa5a7fc640 1 -- 172.21.15.62:0/3341336863 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3085 (secure 0 0 0) 0x7faa5409b380 con 0x7faa64100dd0 2024-09-11T23:51:47.252 INFO:teuthology.orchestra.run.smithi062.stdout:e24 2024-09-11T23:51:47.252 INFO:teuthology.orchestra.run.smithi062.stdout:btime 2024-09-11T23:50:22:864278+0000 2024-09-11T23:51:47.252 INFO:teuthology.orchestra.run.smithi062.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T23:51:47.252 INFO:teuthology.orchestra.run.smithi062.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:51:47.252 INFO:teuthology.orchestra.run.smithi062.stdout:legacy client fscid: 1 2024-09-11T23:51:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:47.252 INFO:teuthology.orchestra.run.smithi062.stdout:Filesystem 'cephfs' (1) 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:fs_name cephfs 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 24 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-09-11T23:50:11.406773+0000 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2024-09-11T23:50:22.553275+0000 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:tableserver 0 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:root 0 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:session_timeout 300 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:session_autoclose 300 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:max_file_size 1099511627776 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:max_xattr_size 65536 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:required_client_features {} 2024-09-11T23:51:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure 0 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure_osd_epoch 0 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:max_mds 3 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:in 0,1,2 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:up {0=24488,1=24484,2=24469} 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:failed 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:damaged 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:stopped 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:data_pools [3] 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:metadata_pool 2 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:inline_data disabled 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:balancer 2024-09-11T23:51:47.254 INFO:teuthology.orchestra.run.smithi062.stdout:bal_rank_mask -1 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout:standby_count_wanted 0 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout:qdb_cluster leader: 24488 members: 24469,24484,24488 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b{0:24488} state up:active seq 4 addr [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.f{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.120:6834/2559115046,v1:172.21.15.120:6835/2559115046] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.i{1:24484} state up:active seq 8 addr [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.h{1:24461} state up:standby-replay seq 1 addr [v2:172.21.15.83:6836/337493680,v1:172.21.15.83:6837/337493680] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c{2:24469} state up:active seq 10 addr [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.e{2:24437} state up:standby-replay seq 1 addr [v2:172.21.15.83:6834/3040146550,v1:172.21.15.83:6835/3040146550] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout:Standby daemons: 2024-09-11T23:51:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:47.256 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.j{-1:14508} state up:standby seq 1 addr [v2:172.21.15.62:6834/1692157360,v1:172.21.15.62:6835/1692157360] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.256 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d{-1:14520} state up:standby seq 1 addr [v2:172.21.15.62:6836/1495913627,v1:172.21.15.62:6837/1495913627] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.256 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.g{-1:14565} state up:standby seq 1 addr [v2:172.21.15.62:6838/392450380,v1:172.21.15.62:6839/392450380] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.256 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a{-1:14577} state up:standby seq 1 addr [v2:172.21.15.62:6840/4034635892,v1:172.21.15.62:6841/4034635892] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.256 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.k{-1:24419} state up:standby seq 1 addr [v2:172.21.15.83:6832/3300661503,v1:172.21.15.83:6833/3300661503] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.256 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.l{-1:24427} state up:standby seq 1 addr [v2:172.21.15.120:6832/1221552790,v1:172.21.15.120:6833/1221552790] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:51:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2024-09-11T23:51:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 -- 172.21.15.62:0/3341336863 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faa500a2bb0 msgr2=0x7faa500a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faa500a2bb0 0x7faa500a5070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faa60009930 tx=0x7faa60008040 comp rx=0 tx=0).stop 2024-09-11T23:51:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 -- 172.21.15.62:0/3341336863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa64100dd0 msgr2=0x7faa6419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa64100dd0 0x7faa6419a650 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faa5400d950 tx=0x7faa5400de10 comp rx=0 tx=0).stop 2024-09-11T23:51:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 -- 172.21.15.62:0/3341336863 shutdown_connections 2024-09-11T23:51:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7faa500a2bb0 0x7faa500a5070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa64101770 0x7faa641a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa64100dd0 0x7faa6419a650 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:47.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 --2- 172.21.15.62:0/3341336863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa64100490 0x7faa6419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:47.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 -- 172.21.15.62:0/3341336863 >> 172.21.15.62:0/3341336863 conn(0x7faa640fc2a0 msgr2=0x7faa640fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:47.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.255+0000 7faa6b2c1640 1 -- 172.21.15.62:0/3341336863 shutdown_connections 2024-09-11T23:51:47.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:47.256+0000 7faa6b2c1640 1 -- 172.21.15.62:0/3341336863 wait complete. 2024-09-11T23:51:47.978 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 56675698-7097-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T23:51:48.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:48 smithi062 ceph-mon[30958]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:48.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:48 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3341336863' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:51:48.243 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:48.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:48 smithi120 ceph-mon[35182]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:48.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:48 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3341336863' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:51:48.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:48 smithi083 ceph-mon[34805]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:48.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:48 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3341336863' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:51:48.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.704+0000 7f6f40e70640 1 -- 172.21.15.62:0/3343990309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c101490 msgr2=0x7f6f3c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:48.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.704+0000 7f6f40e70640 1 --2- 172.21.15.62:0/3343990309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c101490 0x7f6f3c101910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009ca0 tx=0x7f6f3402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:51:48.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.704+0000 7f6f40e70640 1 -- 172.21.15.62:0/3343990309 shutdown_connections 2024-09-11T23:51:48.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.704+0000 7f6f40e70640 1 --2- 172.21.15.62:0/3343990309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c101490 0x7f6f3c101910 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:48.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.704+0000 7f6f40e70640 1 --2- 172.21.15.62:0/3343990309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c06a390 0x7f6f3c1007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.704+0000 7f6f40e70640 1 --2- 172.21.15.62:0/3343990309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f3c069f00 0x7f6f3c100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.704+0000 7f6f40e70640 1 -- 172.21.15.62:0/3343990309 >> 172.21.15.62:0/3343990309 conn(0x7f6f3c0fba40 msgr2=0x7f6f3c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.704+0000 7f6f40e70640 1 -- 172.21.15.62:0/3343990309 shutdown_connections 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.704+0000 7f6f40e70640 1 -- 172.21.15.62:0/3343990309 wait complete. 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f40e70640 1 Processor -- start 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f40e70640 1 -- start start 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f40e70640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c069f00 0x7f6f3c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f40e70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c06a390 0x7f6f3c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f40e70640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f3c101490 0x7f6f3c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f40e70640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c10c860 con 0x7f6f3c069f00 2024-09-11T23:51:48.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f40e70640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c10c6e0 con 0x7f6f3c06a390 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f40e70640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c10c9e0 con 0x7f6f3c101490 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f3a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c069f00 0x7f6f3c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f3a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c069f00 0x7f6f3c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50048/0 (socket says 172.21.15.62:50048) 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f3a59c640 1 -- 172.21.15.62:0/3253295762 learned_addr learned my addr 172.21.15.62:0/3253295762 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f3ad9d640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f3c101490 0x7f6f3c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f39d9b640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c06a390 0x7f6f3c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f3a59c640 1 -- 172.21.15.62:0/3253295762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f3c101490 msgr2=0x7f6f3c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f3a59c640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f3c101490 0x7f6f3c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f3a59c640 1 -- 172.21.15.62:0/3253295762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c06a390 msgr2=0x7f6f3c198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f3a59c640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c06a390 0x7f6f3c198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.705+0000 7f6f3a59c640 1 -- 172.21.15.62:0/3253295762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c19fcf0 con 0x7f6f3c069f00 2024-09-11T23:51:48.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.706+0000 7f6f3a59c640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c069f00 0x7f6f3c1980c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00b450 tx=0x7f6f2c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.706+0000 7f6f337fe640 1 -- 172.21.15.62:0/3253295762 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c015020 con 0x7f6f3c069f00 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.706+0000 7f6f40e70640 1 -- 172.21.15.62:0/3253295762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c19ffe0 con 0x7f6f3c069f00 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.706+0000 7f6f337fe640 1 -- 172.21.15.62:0/3253295762 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f2c011d70 con 0x7f6f3c069f00 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.706+0000 7f6f337fe640 1 -- 172.21.15.62:0/3253295762 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00f340 con 0x7f6f3c069f00 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.706+0000 7f6f40e70640 1 -- 172.21.15.62:0/3253295762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c1a04b0 con 0x7f6f3c069f00 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.707+0000 7f6f40e70640 1 -- 172.21.15.62:0/3253295762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f3c1090f0 con 0x7f6f3c069f00 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.708+0000 7f6f337fe640 1 -- 172.21.15.62:0/3253295762 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f2c018020 con 0x7f6f3c069f00 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.709+0000 7f6f337fe640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6f280a2bb0 0x7f6f280a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.709+0000 7f6f39d9b640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6f280a2bb0 0x7f6f280a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.709+0000 7f6f337fe640 1 -- 172.21.15.62:0/3253295762 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6f2c0ceb90 con 0x7f6f3c069f00 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.709+0000 7f6f39d9b640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6f280a2bb0 0x7f6f280a5070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c1995e0 tx=0x7f6f24008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:48.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.713+0000 7f6f337fe640 1 -- 172.21.15.62:0/3253295762 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f2c01f020 con 0x7f6f3c069f00 2024-09-11T23:51:48.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.891+0000 7f6f40e70640 1 -- 172.21.15.62:0/3253295762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6f3c062e90 con 0x7f6f3c069f00 2024-09-11T23:51:48.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.892+0000 7f6f337fe640 1 -- 172.21.15.62:0/3253295762 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5267 (secure 0 0 0) 0x7f6f2c00d040 con 0x7f6f3c069f00 2024-09-11T23:51:48.891 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 83 2024-09-11T23:51:48.891 INFO:teuthology.orchestra.run.smithi062.stdout:fsid 56675698-7097-11ef-bcea-c7b262605968 2024-09-11T23:51:48.892 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-09-11T23:42:20.574451+0000 2024-09-11T23:51:48.892 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2024-09-11T23:51:14.101286+0000 2024-09-11T23:51:48.892 INFO:teuthology.orchestra.run.smithi062.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T23:51:48.892 INFO:teuthology.orchestra.run.smithi062.stdout:crush_version 26 2024-09-11T23:51:48.892 INFO:teuthology.orchestra.run.smithi062.stdout:full_ratio 0.95 2024-09-11T23:51:48.893 INFO:teuthology.orchestra.run.smithi062.stdout:backfillfull_ratio 0.9 2024-09-11T23:51:48.893 INFO:teuthology.orchestra.run.smithi062.stdout:nearfull_ratio 0.85 2024-09-11T23:51:48.893 INFO:teuthology.orchestra.run.smithi062.stdout:require_min_compat_client luminous 2024-09-11T23:51:48.893 INFO:teuthology.orchestra.run.smithi062.stdout:min_compat_client luminous 2024-09-11T23:51:48.893 INFO:teuthology.orchestra.run.smithi062.stdout:require_osd_release squid 2024-09-11T23:51:48.893 INFO:teuthology.orchestra.run.smithi062.stdout:stretch_mode_enabled false 2024-09-11T23:51:48.894 INFO:teuthology.orchestra.run.smithi062.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T23:51:48.894 INFO:teuthology.orchestra.run.smithi062.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-11T23:51:48.894 INFO:teuthology.orchestra.run.smithi062.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-11T23:51:48.894 INFO:teuthology.orchestra.run.smithi062.stdout:max_osd 12 2024-09-11T23:51:48.894 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6802/3944758612,v1:172.21.15.62:6803/3944758612] [v2:172.21.15.62:6804/3944758612,v1:172.21.15.62:6805/3944758612] exists,up 74ed3595-d3b7-44d6-8aa3-1896f7602607 2024-09-11T23:51:48.895 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6800/2108490342,v1:172.21.15.83:6801/2108490342] [v2:172.21.15.83:6802/2108490342,v1:172.21.15.83:6803/2108490342] exists,up f352db6b-9b13-4d9b-9a26-7d6f16e9dd05 2024-09-11T23:51:48.895 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/3994166921,v1:172.21.15.120:6801/3994166921] [v2:172.21.15.120:6802/3994166921,v1:172.21.15.120:6803/3994166921] exists,up 5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f 2024-09-11T23:51:48.895 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 up in weight 1 up_from 25 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6810/1879962337,v1:172.21.15.62:6811/1879962337] [v2:172.21.15.62:6812/1879962337,v1:172.21.15.62:6813/1879962337] exists,up 37bbe64a-1332-4aee-83f4-67fb8f0092b7 2024-09-11T23:51:48.895 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 up in weight 1 up_from 32 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6808/3966827611,v1:172.21.15.83:6809/3966827611] [v2:172.21.15.83:6810/3966827611,v1:172.21.15.83:6811/3966827611] exists,up 8cb698b7-3471-4503-8039-6519c4d2943e 2024-09-11T23:51:48.895 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 up in weight 1 up_from 37 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/186624354,v1:172.21.15.120:6809/186624354] [v2:172.21.15.120:6810/186624354,v1:172.21.15.120:6811/186624354] exists,up ce904ade-27b5-4171-af4c-7c4894f40c16 2024-09-11T23:51:48.895 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 up in weight 1 up_from 44 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6818/461634356,v1:172.21.15.62:6819/461634356] [v2:172.21.15.62:6820/461634356,v1:172.21.15.62:6821/461634356] exists,up 7ea78a9f-4447-413d-a698-12bf6a2f1398 2024-09-11T23:51:48.896 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 up in weight 1 up_from 49 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6816/3138900766,v1:172.21.15.83:6817/3138900766] [v2:172.21.15.83:6818/3138900766,v1:172.21.15.83:6819/3138900766] exists,up 9eb34260-3535-4e62-8aa5-276282a679cb 2024-09-11T23:51:48.896 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 up in weight 1 up_from 56 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6816/1046771944,v1:172.21.15.120:6817/1046771944] [v2:172.21.15.120:6818/1046771944,v1:172.21.15.120:6819/1046771944] exists,up f2244a62-94a3-4ebf-9e65-aadfc025c3c7 2024-09-11T23:51:48.896 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 up in weight 1 up_from 63 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6826/343213579,v1:172.21.15.62:6827/343213579] [v2:172.21.15.62:6828/343213579,v1:172.21.15.62:6829/343213579] exists,up ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0 2024-09-11T23:51:48.896 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 up in weight 1 up_from 68 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6824/4134153557,v1:172.21.15.83:6825/4134153557] [v2:172.21.15.83:6826/4134153557,v1:172.21.15.83:6827/4134153557] exists,up 70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7 2024-09-11T23:51:48.896 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 up in weight 1 up_from 75 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/4266702819,v1:172.21.15.120:6825/4266702819] [v2:172.21.15.120:6826/4266702819,v1:172.21.15.120:6827/4266702819] exists,up 37bcf111-5dde-49c5-8f40-0a1a5644a081 2024-09-11T23:51:48.897 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.0 [7,11] 2024-09-11T23:51:48.897 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.a [2,11] 2024-09-11T23:51:48.897 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.e [2,4] 2024-09-11T23:51:48.897 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.1f [8,1] 2024-09-11T23:51:48.897 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.3 [4,9] 2024-09-11T23:51:48.897 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.21 [5,9] 2024-09-11T23:51:48.898 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.24 [4,9] 2024-09-11T23:51:48.898 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.3a [4,8] 2024-09-11T23:51:48.898 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3170436522 expires 2024-09-12T23:43:12.720281+0000 2024-09-11T23:51:48.898 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/3180930925 expires 2024-09-12T23:43:12.720281+0000 2024-09-11T23:51:48.898 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/1351339337 expires 2024-09-12T23:43:12.720281+0000 2024-09-11T23:51:48.899 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/2225508066 expires 2024-09-12T23:42:48.657359+0000 2024-09-11T23:51:48.899 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/2045175448 expires 2024-09-12T23:43:12.720281+0000 2024-09-11T23:51:48.899 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/4057479111 expires 2024-09-12T23:42:48.657359+0000 2024-09-11T23:51:48.899 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/3180930925 expires 2024-09-12T23:43:12.720281+0000 2024-09-11T23:51:48.899 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/1330539138 expires 2024-09-12T23:42:48.657359+0000 2024-09-11T23:51:48.899 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/1330539138 expires 2024-09-12T23:42:48.657359+0000 2024-09-11T23:51:48.900 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/352172300 expires 2024-09-12T23:42:48.657359+0000 2024-09-11T23:51:48.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 -- 172.21.15.62:0/3253295762 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6f280a2bb0 msgr2=0x7f6f280a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:48.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6f280a2bb0 0x7f6f280a5070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c1995e0 tx=0x7f6f24008040 comp rx=0 tx=0).stop 2024-09-11T23:51:48.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 -- 172.21.15.62:0/3253295762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c069f00 msgr2=0x7f6f3c1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:48.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c069f00 0x7f6f3c1980c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00b450 tx=0x7f6f2c00b910 comp rx=0 tx=0).stop 2024-09-11T23:51:48.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 -- 172.21.15.62:0/3253295762 shutdown_connections 2024-09-11T23:51:48.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6f280a2bb0 0x7f6f280a5070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:48.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f3c101490 0x7f6f3c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:48.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c06a390 0x7f6f3c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:48.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 --2- 172.21.15.62:0/3253295762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c069f00 0x7f6f3c1980c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:48.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 -- 172.21.15.62:0/3253295762 >> 172.21.15.62:0/3253295762 conn(0x7f6f3c0fba40 msgr2=0x7f6f3c102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:48.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 -- 172.21.15.62:0/3253295762 shutdown_connections 2024-09-11T23:51:48.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:48.894+0000 7f6f40e70640 1 -- 172.21.15.62:0/3253295762 wait complete. 2024-09-11T23:51:49.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:49 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3253295762' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:51:49.536 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T23:51:49.546 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:51:49.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:49 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3253295762' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:51:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:49 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3253295762' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:51:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.250+0000 7fd5fd517640 1 -- 172.21.15.62:0/375542571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f8101410 msgr2=0x7fd5f810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.250+0000 7fd5fd517640 1 --2- 172.21.15.62:0/375542571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f8101410 0x7fd5f810d940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009cd0 tx=0x7fd5f002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:51:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.250+0000 7fd5fd517640 1 -- 172.21.15.62:0/375542571 shutdown_connections 2024-09-11T23:51:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.250+0000 7fd5fd517640 1 --2- 172.21.15.62:0/375542571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f8101410 0x7fd5f810d940 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.250+0000 7fd5fd517640 1 --2- 172.21.15.62:0/375542571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5f8100a70 0x7fd5f8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.250+0000 7fd5fd517640 1 --2- 172.21.15.62:0/375542571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f8106a70 0x7fd5f8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.250+0000 7fd5fd517640 1 -- 172.21.15.62:0/375542571 >> 172.21.15.62:0/375542571 conn(0x7fd5f80fc790 msgr2=0x7fd5f80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.250+0000 7fd5fd517640 1 -- 172.21.15.62:0/375542571 shutdown_connections 2024-09-11T23:51:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.250+0000 7fd5fd517640 1 -- 172.21.15.62:0/375542571 wait complete. 2024-09-11T23:51:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.251+0000 7fd5fd517640 1 Processor -- start 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.251+0000 7fd5fd517640 1 -- start start 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.251+0000 7fd5fd517640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5f8100a70 0x7fd5f8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.251+0000 7fd5fd517640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f8101410 0x7fd5f81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.251+0000 7fd5fd517640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f8106a70 0x7fd5f819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.251+0000 7fd5fd517640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8110e70 con 0x7fd5f8106a70 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.251+0000 7fd5fd517640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8110cf0 con 0x7fd5f8101410 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.251+0000 7fd5fd517640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8110ff0 con 0x7fd5f8100a70 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f8106a70 0x7fd5f819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f8106a70 0x7fd5f819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50056/0 (socket says 172.21.15.62:50056) 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f77fe640 1 -- 172.21.15.62:0/2199724004 learned_addr learned my addr 172.21.15.62:0/2199724004 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f6ffd640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5f8100a70 0x7fd5f8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f67fc640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f8101410 0x7fd5f81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f77fe640 1 -- 172.21.15.62:0/2199724004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5f8100a70 msgr2=0x7fd5f8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f77fe640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5f8100a70 0x7fd5f8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f77fe640 1 -- 172.21.15.62:0/2199724004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f8101410 msgr2=0x7fd5f81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f77fe640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f8101410 0x7fd5f81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f77fe640 1 -- 172.21.15.62:0/2199724004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f819d340 con 0x7fd5f8106a70 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5f77fe640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f8106a70 0x7fd5f819cc60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0005cc0 tx=0x7fd5f0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5dffff640 1 -- 172.21.15.62:0/2199724004 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0004040 con 0x7fd5f8106a70 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5fd517640 1 -- 172.21.15.62:0/2199724004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f819d5d0 con 0x7fd5f8106a70 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.252+0000 7fd5dffff640 1 -- 172.21.15.62:0/2199724004 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f00046c0 con 0x7fd5f8106a70 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.253+0000 7fd5dffff640 1 -- 172.21.15.62:0/2199724004 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0030e70 con 0x7fd5f8106a70 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.253+0000 7fd5fd517640 1 -- 172.21.15.62:0/2199724004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f81a4f10 con 0x7fd5f8106a70 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.254+0000 7fd5fd517640 1 -- 172.21.15.62:0/2199724004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f8100ed0 con 0x7fd5f8106a70 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.255+0000 7fd5dffff640 1 -- 172.21.15.62:0/2199724004 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd5f00305a0 con 0x7fd5f8106a70 2024-09-11T23:51:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.255+0000 7fd5dffff640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd5e00a2b80 0x7fd5e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.256+0000 7fd5f6ffd640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd5e00a2b80 0x7fd5e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.256+0000 7fd5dffff640 1 -- 172.21.15.62:0/2199724004 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd5f002e030 con 0x7fd5f8106a70 2024-09-11T23:51:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.256+0000 7fd5f6ffd640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd5e00a2b80 0x7fd5e00a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800bb60 tx=0x7fd5e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.260+0000 7fd5dffff640 1 -- 172.21.15.62:0/2199724004 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5f00bd020 con 0x7fd5f8106a70 2024-09-11T23:51:50.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:50 smithi062 ceph-mon[30958]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.476+0000 7fd5fd517640 1 -- 172.21.15.62:0/2199724004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5f81a50e0 con 0x7fd5f8106a70 2024-09-11T23:51:50.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.476+0000 7fd5dffff640 1 -- 172.21.15.62:0/2199724004 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11909 (secure 0 0 0) 0x7fd5f00098b0 con 0x7fd5f8106a70 2024-09-11T23:51:50.476 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:50.476 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"btime":"2024-09-11T23:50:22:864278+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:50:22.553275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:51:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2024-09-11T23:51:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.478+0000 7fd5fd517640 1 -- 172.21.15.62:0/2199724004 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd5e00a2b80 msgr2=0x7fd5e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.478+0000 7fd5fd517640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd5e00a2b80 0x7fd5e00a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800bb60 tx=0x7fd5e800b390 comp rx=0 tx=0).stop 2024-09-11T23:51:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.478+0000 7fd5fd517640 1 -- 172.21.15.62:0/2199724004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f8106a70 msgr2=0x7fd5f819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.478+0000 7fd5fd517640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f8106a70 0x7fd5f819cc60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0005cc0 tx=0x7fd5f0003970 comp rx=0 tx=0).stop 2024-09-11T23:51:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.478+0000 7fd5fd517640 1 -- 172.21.15.62:0/2199724004 shutdown_connections 2024-09-11T23:51:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.478+0000 7fd5fd517640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd5e00a2b80 0x7fd5e00a5040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.478+0000 7fd5fd517640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f8106a70 0x7fd5f819cc60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.478+0000 7fd5fd517640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f8101410 0x7fd5f81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.478+0000 7fd5fd517640 1 --2- 172.21.15.62:0/2199724004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5f8100a70 0x7fd5f8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.478+0000 7fd5fd517640 1 -- 172.21.15.62:0/2199724004 >> 172.21.15.62:0/2199724004 conn(0x7fd5f80fc790 msgr2=0x7fd5f806a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.479+0000 7fd5fd517640 1 -- 172.21.15.62:0/2199724004 shutdown_connections 2024-09-11T23:51:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:50.479+0000 7fd5fd517640 1 -- 172.21.15.62:0/2199724004 wait complete. 2024-09-11T23:51:50.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:50 smithi120 ceph-mon[35182]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:50.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:50 smithi083 ceph-mon[34805]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:51.128 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:51:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:51.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:51 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2199724004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:51 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2199724004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:51.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:51 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2199724004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:51.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4069211760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e281019d0 msgr2=0x7f6e28101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:51.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 --2- 172.21.15.62:0/4069211760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e281019d0 0x7f6e28101e30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009930 tx=0x7f6e1c02a420 comp rx=0 tx=0).stop 2024-09-11T23:51:51.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4069211760 shutdown_connections 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 --2- 172.21.15.62:0/4069211760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e28102370 0x7f6e2810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 --2- 172.21.15.62:0/4069211760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e281019d0 0x7f6e28101e30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 --2- 172.21.15.62:0/4069211760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e281079d0 0x7f6e28107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4069211760 >> 172.21.15.62:0/4069211760 conn(0x7f6e280fd710 msgr2=0x7f6e280ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4069211760 shutdown_connections 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4069211760 wait complete. 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 Processor -- start 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.838+0000 7f6e2e5a2640 1 -- start start 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2e5a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e281019d0 0x7f6e280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2e5a2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e28102370 0x7f6e28074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2e5a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e281079d0 0x7f6e28074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2e5a2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6e28111dd0 con 0x7f6e281019d0 2024-09-11T23:51:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2e5a2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e28111c50 con 0x7f6e281079d0 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2e5a2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e28111f50 con 0x7f6e28102370 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2d5a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e281019d0 0x7f6e280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2d5a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e281019d0 0x7f6e280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47100/0 (socket says 172.21.15.62:47100) 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2d5a0640 1 -- 172.21.15.62:0/4235260674 learned_addr learned my addr 172.21.15.62:0/4235260674 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2cd9f640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e28102370 0x7f6e28074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2d5a0640 1 -- 172.21.15.62:0/4235260674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e28102370 msgr2=0x7f6e28074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2d5a0640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e28102370 0x7f6e28074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2d5a0640 1 -- 172.21.15.62:0/4235260674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e281079d0 msgr2=0x7f6e28074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2d5a0640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e281079d0 0x7f6e28074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2d5a0640 1 -- 172.21.15.62:0/4235260674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e281a6730 con 0x7f6e281019d0 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2d5a0640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e281019d0 0x7f6e280744f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400d950 tx=0x7f6e2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e1a7fc640 1 -- 172.21.15.62:0/4235260674 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24017070 con 0x7f6e281019d0 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4235260674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e281a6a20 con 0x7f6e281019d0 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e1a7fc640 1 -- 172.21.15.62:0/4235260674 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e24009c30 con 0x7f6e281019d0 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e1a7fc640 1 -- 172.21.15.62:0/4235260674 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2400f6b0 con 0x7f6e281019d0 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.839+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4235260674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e281a6f80 con 0x7f6e281019d0 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.841+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4235260674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e28101e30 con 0x7f6e281019d0 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.842+0000 7f6e1a7fc640 1 -- 172.21.15.62:0/4235260674 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6e240109d0 con 0x7f6e281019d0 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.842+0000 7f6e1a7fc640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6e140a2b80 0x7f6e140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.842+0000 7f6e2cd9f640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6e140a2b80 0x7f6e140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.842+0000 7f6e1a7fc640 1 -- 172.21.15.62:0/4235260674 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6e240cece0 con 0x7f6e281019d0 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.843+0000 7f6e2cd9f640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6e140a2b80 0x7f6e140a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000c00 tx=0x7f6e1c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:51.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:51.846+0000 7f6e1a7fc640 1 -- 172.21.15.62:0/4235260674 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e24009de0 con 0x7f6e281019d0 2024-09-11T23:51:52.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.056+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4235260674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e2802d7c0 con 0x7f6e281019d0 2024-09-11T23:51:52.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.057+0000 7f6e1a7fc640 1 -- 172.21.15.62:0/4235260674 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11909 (secure 0 0 0) 0x7f6e2409bc50 con 0x7f6e281019d0 2024-09-11T23:51:52.056 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:52.056 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"btime":"2024-09-11T23:50:22:864278+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:50:22.553275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:51:52.056 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2024-09-11T23:51:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4235260674 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6e140a2b80 msgr2=0x7f6e140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6e140a2b80 0x7f6e140a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000c00 tx=0x7f6e1c005a20 comp rx=0 tx=0).stop 2024-09-11T23:51:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4235260674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e281019d0 msgr2=0x7f6e280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e281019d0 0x7f6e280744f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400d950 tx=0x7f6e2400de10 comp rx=0 tx=0).stop 2024-09-11T23:51:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4235260674 shutdown_connections 2024-09-11T23:51:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6e140a2b80 0x7f6e140a5040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e281079d0 0x7f6e28074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e28102370 0x7f6e28074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:52.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 --2- 172.21.15.62:0/4235260674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e281019d0 0x7f6e280744f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:52.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4235260674 >> 172.21.15.62:0/4235260674 conn(0x7f6e280fd710 msgr2=0x7f6e280fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:52.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4235260674 shutdown_connections 2024-09-11T23:51:52.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:52.059+0000 7f6e2e5a2640 1 -- 172.21.15.62:0/4235260674 wait complete. 2024-09-11T23:51:52.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:52 smithi062 ceph-mon[30958]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:52.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:52 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/4235260674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:52.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:52 smithi120 ceph-mon[35182]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:52.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:52 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/4235260674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:52 smithi083 ceph-mon[34805]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:52 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/4235260674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:51:52.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:51:53.098 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:53.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.579+0000 7fcb3eabd640 1 -- 172.21.15.62:0/59215170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb38102d40 msgr2=0x7fcb38103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:53.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.579+0000 7fcb3eabd640 1 --2- 172.21.15.62:0/59215170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb38102d40 0x7fcb38103120 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009900 tx=0x7fcb2c02a480 comp rx=0 tx=0).stop 2024-09-11T23:51:53.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.579+0000 7fcb3eabd640 1 -- 172.21.15.62:0/59215170 shutdown_connections 2024-09-11T23:51:53.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.579+0000 7fcb3eabd640 1 --2- 172.21.15.62:0/59215170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb3810c170 0x7fcb3810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.579+0000 7fcb3eabd640 1 --2- 172.21.15.62:0/59215170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38103660 0x7fcb3810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.579+0000 7fcb3eabd640 1 --2- 172.21.15.62:0/59215170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb38102d40 0x7fcb38103120 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.579+0000 7fcb3eabd640 1 -- 172.21.15.62:0/59215170 >> 172.21.15.62:0/59215170 conn(0x7fcb380fc7a0 msgr2=0x7fcb380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.579+0000 7fcb3eabd640 1 -- 172.21.15.62:0/59215170 shutdown_connections 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.579+0000 7fcb3eabd640 1 -- 172.21.15.62:0/59215170 wait complete. 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3eabd640 1 Processor -- start 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3eabd640 1 -- start start 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3eabd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38102d40 0x7fcb381009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3eabd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb38103660 0x7fcb38100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3eabd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3810c170 0x7fcb38101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3eabd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb38111f70 con 0x7fcb3810c170 2024-09-11T23:51:53.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3eabd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb38111df0 con 0x7fcb38102d40 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3eabd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381120f0 con 0x7fcb38103660 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3d05a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3810c170 0x7fcb38101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3d05a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3810c170 0x7fcb38101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47126/0 (socket says 172.21.15.62:47126) 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.580+0000 7fcb3d05a640 1 -- 172.21.15.62:0/2256211704 learned_addr learned my addr 172.21.15.62:0/2256211704 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb3c859640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38102d40 0x7fcb381009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb3d05a640 1 -- 172.21.15.62:0/2256211704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb38103660 msgr2=0x7fcb38100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb3d05a640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb38103660 0x7fcb38100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb3d05a640 1 -- 172.21.15.62:0/2256211704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38102d40 msgr2=0x7fcb381009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb3d05a640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38102d40 0x7fcb381009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb3d05a640 1 -- 172.21.15.62:0/2256211704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb381a6990 con 0x7fcb3810c170 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb3c859640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38102d40 0x7fcb381009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb3d05a640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3810c170 0x7fcb38101420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000cad0 tx=0x7fcb3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb35ffb640 1 -- 172.21.15.62:0/2256211704 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000ae20 con 0x7fcb3810c170 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb3eabd640 1 -- 172.21.15.62:0/2256211704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb381a6c80 con 0x7fcb3810c170 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb35ffb640 1 -- 172.21.15.62:0/2256211704 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb30005020 con 0x7fcb3810c170 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb35ffb640 1 -- 172.21.15.62:0/2256211704 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000f6b0 con 0x7fcb3810c170 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.581+0000 7fcb3eabd640 1 -- 172.21.15.62:0/2256211704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb381a71c0 con 0x7fcb3810c170 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.582+0000 7fcb3eabd640 1 -- 172.21.15.62:0/2256211704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb38103120 con 0x7fcb3810c170 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.583+0000 7fcb35ffb640 1 -- 172.21.15.62:0/2256211704 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcb300051d0 con 0x7fcb3810c170 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.584+0000 7fcb35ffb640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb280a2b80 0x7fcb280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.584+0000 7fcb3c859640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb280a2b80 0x7fcb280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.584+0000 7fcb35ffb640 1 -- 172.21.15.62:0/2256211704 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcb300cf3c0 con 0x7fcb3810c170 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.585+0000 7fcb3c859640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb280a2b80 0x7fcb280a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c005ab0 tx=0x7fcb2c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.588+0000 7fcb35ffb640 1 -- 172.21.15.62:0/2256211704 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb30014030 con 0x7fcb3810c170 2024-09-11T23:51:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.766+0000 7fcb3eabd640 1 -- 172.21.15.62:0/2256211704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb38101ed0 con 0x7fcb3810c170 2024-09-11T23:51:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.768+0000 7fcb35ffb640 1 -- 172.21.15.62:0/2256211704 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22620 (secure 0 0 0) 0x7fcb3009c330 con 0x7fcb3810c170 2024-09-11T23:51:53.767 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:51:53.767 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":83,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:51:14.101286+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:51:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 -- 172.21.15.62:0/2256211704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb280a2b80 msgr2=0x7fcb280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb280a2b80 0x7fcb280a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c005ab0 tx=0x7fcb2c005a20 comp rx=0 tx=0).stop 2024-09-11T23:51:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 -- 172.21.15.62:0/2256211704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3810c170 msgr2=0x7fcb38101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3810c170 0x7fcb38101420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000cad0 tx=0x7fcb3000cf90 comp rx=0 tx=0).stop 2024-09-11T23:51:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 -- 172.21.15.62:0/2256211704 shutdown_connections 2024-09-11T23:51:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcb280a2b80 0x7fcb280a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3810c170 0x7fcb38101420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb38103660 0x7fcb38100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 --2- 172.21.15.62:0/2256211704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38102d40 0x7fcb381009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 -- 172.21.15.62:0/2256211704 >> 172.21.15.62:0/2256211704 conn(0x7fcb380fc7a0 msgr2=0x7fcb380fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 -- 172.21.15.62:0/2256211704 shutdown_connections 2024-09-11T23:51:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:53.770+0000 7fcb3eabd640 1 -- 172.21.15.62:0/2256211704 wait complete. 2024-09-11T23:51:54.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:54 smithi062 ceph-mon[30958]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:54.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:54 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2256211704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:51:54.550 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:50:11.406773+0000', 'modified': '2024-09-11T23:50:22.553275+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24484, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6835/3040146550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 3040146550}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3040146550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6837/337493680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 337493680}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 337493680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/2559115046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 2559115046}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 2559115046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.120:6837/1796168792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 1796168792}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 1796168792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.120:6839/2283411896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 2283411896}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 2283411896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6839/2530997346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 2530997346}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 2530997346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24469, 24484, 24488]} 2024-09-11T23:51:54.550 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-11T23:51:54.550 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T23:51:54.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:54 smithi120 ceph-mon[35182]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:54.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:54 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2256211704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:51:54.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:54 smithi083 ceph-mon[34805]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:51:54.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:54 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2256211704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:51:54.809 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:55.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.306+0000 7fcac6031640 1 -- 172.21.15.62:0/94059399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcac0102ef0 msgr2=0x7fcac010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:55.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.306+0000 7fcac6031640 1 --2- 172.21.15.62:0/94059399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcac0102ef0 0x7fcac010da40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009900 tx=0x7fcab802a480 comp rx=0 tx=0).stop 2024-09-11T23:51:55.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.306+0000 7fcac6031640 1 -- 172.21.15.62:0/94059399 shutdown_connections 2024-09-11T23:51:55.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.306+0000 7fcac6031640 1 --2- 172.21.15.62:0/94059399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcac0102ef0 0x7fcac010da40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.306+0000 7fcac6031640 1 --2- 172.21.15.62:0/94059399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0102570 0x7fcac01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.306+0000 7fcac6031640 1 --2- 172.21.15.62:0/94059399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac0100e20 0x7fcac0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.306+0000 7fcac6031640 1 -- 172.21.15.62:0/94059399 >> 172.21.15.62:0/94059399 conn(0x7fcac00fc7d0 msgr2=0x7fcac00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.306+0000 7fcac6031640 1 -- 172.21.15.62:0/94059399 shutdown_connections 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.306+0000 7fcac6031640 1 -- 172.21.15.62:0/94059399 wait complete. 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.307+0000 7fcac6031640 1 Processor -- start 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.307+0000 7fcac6031640 1 -- start start 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcac6031640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac0100e20 0x7fcac0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcac6031640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0102570 0x7fcac01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcac6031640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcac0102ef0 0x7fcac019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcac6031640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0110f80 con 0x7fcac0102ef0 2024-09-11T23:51:55.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcac6031640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0110e00 con 0x7fcac0100e20 2024-09-11T23:51:55.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcac6031640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0111100 con 0x7fcac0102570 2024-09-11T23:51:55.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcac0102ef0 0x7fcac019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcac0102ef0 0x7fcac019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47148/0 (socket says 172.21.15.62:47148) 2024-09-11T23:51:55.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabffff640 1 -- 172.21.15.62:0/1425922704 learned_addr learned my addr 172.21.15.62:0/1425922704 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:55.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabeffd640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0102570 0x7fcac01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac0100e20 0x7fcac0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabffff640 1 -- 172.21.15.62:0/1425922704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0102570 msgr2=0x7fcac01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabffff640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0102570 0x7fcac01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabffff640 1 -- 172.21.15.62:0/1425922704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac0100e20 msgr2=0x7fcac0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabffff640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac0100e20 0x7fcac0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabffff640 1 -- 172.21.15.62:0/1425922704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac019d430 con 0x7fcac0102ef0 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabffff640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcac0102ef0 0x7fcac019cd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcab8004770 tx=0x7fcab8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabcff9640 1 -- 172.21.15.62:0/1425922704 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab803f070 con 0x7fcac0102ef0 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcac6031640 1 -- 172.21.15.62:0/1425922704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac019d6c0 con 0x7fcac0102ef0 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabcff9640 1 -- 172.21.15.62:0/1425922704 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab8031c30 con 0x7fcac0102ef0 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.308+0000 7fcabcff9640 1 -- 172.21.15.62:0/1425922704 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8035030 con 0x7fcac0102ef0 2024-09-11T23:51:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.309+0000 7fcac6031640 1 -- 172.21.15.62:0/1425922704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac01a5000 con 0x7fcac0102ef0 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.311+0000 7fcabcff9640 1 -- 172.21.15.62:0/1425922704 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcab8042070 con 0x7fcac0102ef0 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.311+0000 7fcac840a740 1 Processor -- start 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.311+0000 7fcac840a740 1 -- start start 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.311+0000 7fcac840a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df66adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.311+0000 7fcac840a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df675670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcac840a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df677fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcac840a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d2df5d1090 con 0x55d2df675bb0 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcac840a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2df578a70 con 0x55d2df66a9d0 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcac840a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d2df5cd050 con 0x55d2df66b2f0 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df677fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df677fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47158/0 (socket says 172.21.15.62:47158) 2024-09-11T23:51:55.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabeffd640 1 -- 172.21.15.62:0/4052331626 learned_addr learned my addr 172.21.15.62:0/4052331626 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:55.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/4052331626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df675670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabeffd640 1 --2- 172.21.15.62:0/4052331626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df66adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabcff9640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaac0a2b80 0x7fcaac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabeffd640 1 -- 172.21.15.62:0/4052331626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 msgr2=0x55d2df675670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:55.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabeffd640 1 --2- 172.21.15.62:0/4052331626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df675670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabeffd640 1 -- 172.21.15.62:0/4052331626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 msgr2=0x55d2df66adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:55.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabeffd640 1 --2- 172.21.15.62:0/4052331626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df66adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabffff640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaac0a2b80 0x7fcaac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabeffd640 1 -- 172.21.15.62:0/4052331626 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2df585130 con 0x55d2df675bb0 2024-09-11T23:51:55.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabcff9640 1 -- 172.21.15.62:0/1425922704 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcab80ef7e0 con 0x7fcac0102ef0 2024-09-11T23:51:55.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabeffd640 1 --2- 172.21.15.62:0/4052331626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df677fa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80127a0 tx=0x7fcaa800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755ab57cd7192dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:55.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcab5ffb640 1 -- 172.21.15.62:0/4052331626 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8012d20 con 0x55d2df675bb0 2024-09-11T23:51:55.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.312+0000 7fcabffff640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaac0a2b80 0x7fcaac0a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcab80b8ad0 tx=0x7fcab80e6be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:55.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcab5ffb640 1 -- 172.21.15.62:0/4052331626 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa8005020 con 0x55d2df675bb0 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcab5ffb640 1 -- 172.21.15.62:0/4052331626 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8004ae0 con 0x55d2df675bb0 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcac840a740 1 -- 172.21.15.62:0/4052331626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 msgr2=0x55d2df677fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcac840a740 1 --2- 172.21.15.62:0/4052331626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df677fa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80127a0 tx=0x7fcaa800b7c0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcac840a740 1 -- 172.21.15.62:0/4052331626 shutdown_connections 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcac840a740 1 --2- 172.21.15.62:0/4052331626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df677fa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcac840a740 1 --2- 172.21.15.62:0/4052331626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df675670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcac840a740 1 --2- 172.21.15.62:0/4052331626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df66adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcac840a740 1 -- 172.21.15.62:0/4052331626 >> 172.21.15.62:0/4052331626 conn(0x55d2df666f60 msgr2=0x55d2df667340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcac840a740 1 -- 172.21.15.62:0/4052331626 shutdown_connections 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.313+0000 7fcac840a740 1 -- 172.21.15.62:0/4052331626 wait complete. 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcac840a740 1 -- start start 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcac840a740 1 Processor -- start 2024-09-11T23:51:55.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcac840a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df675460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcac840a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df6727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcac840a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df672cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcac840a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d2df678780 con 0x55d2df675bb0 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcac840a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2df58bf20 con 0x55d2df66a9d0 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcac840a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d2df678900 con 0x55d2df66b2f0 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcac840a740 10 client.0 _ll_register_callbacks cb 0x55d2df555030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcabeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df672cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcabeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df672cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47162/0 (socket says 172.21.15.62:47162) 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcabeffd640 1 -- 172.21.15.62:0/1291719479 learned_addr learned my addr 172.21.15.62:0/1291719479 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcabeffd640 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df675460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.314+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df6727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcabeffd640 1 -- 172.21.15.62:0/1291719479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 msgr2=0x55d2df6727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:55.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcabeffd640 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df6727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcabeffd640 1 -- 172.21.15.62:0/1291719479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 msgr2=0x55d2df675460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcabeffd640 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df675460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcabeffd640 1 -- 172.21.15.62:0/1291719479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2df3422b0 con 0x55d2df675bb0 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df6727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcabeffd640 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df672cf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8012f50 tx=0x7fcaa801b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1b7f24de036e99 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fca9effd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fca9effd640 1 -- 172.21.15.62:0/1291719479 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80054e0 con 0x55d2df675bb0 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcac840a740 1 -- 172.21.15.62:0/1291719479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa8005020 con 0x55d2df675bb0 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fca9effd640 1 -- 172.21.15.62:0/1291719479 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcaa8004880 con 0x55d2df675bb0 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fca9effd640 1 -- 172.21.15.62:0/1291719479 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcaa8004a80 con 0x55d2df675bb0 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcac840a740 10 client.15075 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcac840a740 1 -- 172.21.15.62:0/1291719479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2df6840a0 con 0x55d2df675bb0 2024-09-11T23:51:55.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fca9effd640 1 -- 172.21.15.62:0/1291719479 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa8005020 con 0x55d2df675bb0 2024-09-11T23:51:55.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fca9effd640 1 -- 172.21.15.62:0/1291719479 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fcaa802d450 con 0x55d2df675bb0 2024-09-11T23:51:55.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.315+0000 7fcac840a740 10 client.15075 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:55.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.316+0000 7fcac840a740 10 client.15075 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:51:55.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.316+0000 7fcac840a740 20 client.15075 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:55.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.316+0000 7fcac840a740 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55d2df686470 0x55d2df6878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.316+0000 7fcac840a740 4 client.15075 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:51:55.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.316+0000 7fcac840a740 1 -- 172.21.15.62:0/1291719479 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2df5de600 con 0x55d2df686470 2024-09-11T23:51:55.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.316+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55d2df686470 0x55d2df6878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.316+0000 7fca9effd640 10 client.15075 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:51:55.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.316+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55d2df686470 0x55d2df6878e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bc3e4cda4b7a6cf server_cookie=2f88e7084bccd604 in_seq=0 out_seq=0 2024-09-11T23:51:55.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fca9effd640 1 -- 172.21.15.62:0/1291719479 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2df5de600 con 0x55d2df686470 2024-09-11T23:51:55.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fca9effd640 10 client.15075 handle_command_reply: tid=0 2024-09-11T23:51:55.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 client.15075 shutdown 2024-09-11T23:51:55.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 -- 172.21.15.62:0/1291719479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 msgr2=0x55d2df672cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:55.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df672cf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8012f50 tx=0x7fcaa801b040 comp rx=0 tx=0).stop 2024-09-11T23:51:55.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 -- 172.21.15.62:0/1291719479 shutdown_connections 2024-09-11T23:51:55.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55d2df686470 0x55d2df6878e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df675bb0 0x55d2df672cf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df6727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fca9effd640 0 client.15075 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:51:55.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 --2- 172.21.15.62:0/1291719479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df675460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 -- 172.21.15.62:0/1291719479 >> 172.21.15.62:0/1291719479 conn(0x55d2df666f60 msgr2=0x55d2df66cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:55.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 -- 172.21.15.62:0/1291719479 shutdown_connections 2024-09-11T23:51:55.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 1 -- 172.21.15.62:0/1291719479 wait complete. 2024-09-11T23:51:55.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.317+0000 7fcac840a740 20 client.15075 trim_cache size 0 max 16384 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcac840a740 1 Processor -- start 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcac840a740 1 -- start start 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcac840a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df685da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcac840a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66b2f0 0x55d2df686b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcac840a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df675bb0 0x55d2df661ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcac840a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d2df685c20 con 0x55d2df66b2f0 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcac840a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2df6862f0 con 0x55d2df66a9d0 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcac840a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d2df6869c0 con 0x55d2df675bb0 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66b2f0 0x55d2df686b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66b2f0 0x55d2df686b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47176/0 (socket says 172.21.15.62:47176) 2024-09-11T23:51:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabf7fe640 1 -- 172.21.15.62:0/1153028956 learned_addr learned my addr 172.21.15.62:0/1153028956 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabeffd640 1 --2- 172.21.15.62:0/1153028956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df685da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabeffd640 1 --2- 172.21.15.62:0/1153028956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df675bb0 0x55d2df661ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabf7fe640 1 -- 172.21.15.62:0/1153028956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df675bb0 msgr2=0x55d2df661ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/1153028956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df675bb0 0x55d2df661ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabf7fe640 1 -- 172.21.15.62:0/1153028956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 msgr2=0x55d2df685da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/1153028956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df685da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabf7fe640 1 -- 172.21.15.62:0/1153028956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2df667bf0 con 0x55d2df66b2f0 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabeffd640 1 --2- 172.21.15.62:0/1153028956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df675bb0 0x55d2df661ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.336+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/1153028956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66b2f0 0x55d2df686b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55d2df672040 tx=0x7fcab0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9965136410262fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcab57fa640 1 -- 172.21.15.62:0/1153028956 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab001f070 con 0x55d2df66b2f0 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcab57fa640 1 -- 172.21.15.62:0/1153028956 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab0011df0 con 0x55d2df66b2f0 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcab57fa640 1 -- 172.21.15.62:0/1153028956 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab001bb90 con 0x55d2df66b2f0 2024-09-11T23:51:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcac840a740 1 -- 172.21.15.62:0/1153028956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66b2f0 msgr2=0x55d2df686b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:55.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcac840a740 1 --2- 172.21.15.62:0/1153028956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66b2f0 0x55d2df686b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55d2df672040 tx=0x7fcab0009680 comp rx=0 tx=0).stop 2024-09-11T23:51:55.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcac840a740 1 -- 172.21.15.62:0/1153028956 shutdown_connections 2024-09-11T23:51:55.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcac840a740 1 --2- 172.21.15.62:0/1153028956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df675bb0 0x55d2df661ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcac840a740 1 --2- 172.21.15.62:0/1153028956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66b2f0 0x55d2df686b40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcac840a740 1 --2- 172.21.15.62:0/1153028956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df66a9d0 0x55d2df685da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcac840a740 1 -- 172.21.15.62:0/1153028956 >> 172.21.15.62:0/1153028956 conn(0x55d2df666f60 msgr2=0x55d2df6854b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:55.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcac840a740 1 -- 172.21.15.62:0/1153028956 shutdown_connections 2024-09-11T23:51:55.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.337+0000 7fcac840a740 1 -- 172.21.15.62:0/1153028956 wait complete. 2024-09-11T23:51:55.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcac840a740 1 -- start start 2024-09-11T23:51:55.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcac840a740 1 Processor -- start 2024-09-11T23:51:55.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcac840a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66a9d0 0x55d2df688b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcac840a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df689090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcac840a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df675bb0 0x55d2df6895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcac840a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d2df687760 con 0x55d2df66a9d0 2024-09-11T23:51:55.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcac840a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2df687090 con 0x55d2df675bb0 2024-09-11T23:51:55.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcac840a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d2df687e30 con 0x55d2df66b2f0 2024-09-11T23:51:55.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcac840a740 10 client.0 _ll_register_callbacks cb 0x7fcaa800be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:55.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcabeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66a9d0 0x55d2df688b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcabeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66a9d0 0x55d2df688b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47178/0 (socket says 172.21.15.62:47178) 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcabeffd640 1 -- 172.21.15.62:0/2191627717 learned_addr learned my addr 172.21.15.62:0/2191627717 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcabeffd640 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df675bb0 0x55d2df6895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df689090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcabeffd640 1 -- 172.21.15.62:0/2191627717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 msgr2=0x55d2df689090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcabeffd640 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df689090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcabeffd640 1 -- 172.21.15.62:0/2191627717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df675bb0 msgr2=0x55d2df6895d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcabeffd640 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df675bb0 0x55d2df6895d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.338+0000 7fcabeffd640 1 -- 172.21.15.62:0/2191627717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2df6eeca0 con 0x55d2df66a9d0 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fcabeffd640 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66a9d0 0x55d2df688b50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8006010 tx=0x7fcaa800b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c730f0d61f667cab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:55.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fca9effd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:55.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fca9effd640 1 -- 172.21.15.62:0/2191627717 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa802dd10 con 0x55d2df66a9d0 2024-09-11T23:51:55.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fcac840a740 1 -- 172.21.15.62:0/2191627717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab0011df0 con 0x55d2df66a9d0 2024-09-11T23:51:55.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fca9effd640 1 -- 172.21.15.62:0/2191627717 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcaa80120d0 con 0x55d2df66a9d0 2024-09-11T23:51:55.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fca9effd640 1 -- 172.21.15.62:0/2191627717 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcaa80122d0 con 0x55d2df66a9d0 2024-09-11T23:51:55.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fcac840a740 10 client.15081 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:55.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fcac840a740 1 -- 172.21.15.62:0/2191627717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2df6efef0 con 0x55d2df66a9d0 2024-09-11T23:51:55.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fca9effd640 1 -- 172.21.15.62:0/2191627717 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab0011df0 con 0x55d2df66a9d0 2024-09-11T23:51:55.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fca9effd640 1 -- 172.21.15.62:0/2191627717 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fcaa8044080 con 0x55d2df66a9d0 2024-09-11T23:51:55.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fcac840a740 10 client.15081 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:55.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fcac840a740 10 client.15081 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:51:55.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fcac840a740 20 client.15081 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:55.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fcac840a740 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55d2df6f4a60 0x55d2df6f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:55.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fcac840a740 4 client.15081 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:51:55.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.339+0000 7fcac840a740 1 -- 172.21.15.62:0/2191627717 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "status"}) -- 0x55d2df5d8b10 con 0x55d2df6f4a60 2024-09-11T23:51:55.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.340+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55d2df6f4a60 0x55d2df6f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:55.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.340+0000 7fca9effd640 10 client.15081 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:51:55.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.340+0000 7fcabf7fe640 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55d2df6f4a60 0x55d2df6f3770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=529b68ece79ef6d7 server_cookie=21fb9c3e93afcbb9 in_seq=0 out_seq=0 2024-09-11T23:51:55.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.340+0000 7fca9effd640 1 -- 172.21.15.62:0/2191627717 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d2df5d8b10 con 0x55d2df6f4a60 2024-09-11T23:51:55.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.340+0000 7fca9effd640 10 client.15081 handle_command_reply: tid=0 2024-09-11T23:51:55.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 client.15081 shutdown 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 -- 172.21.15.62:0/2191627717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66a9d0 msgr2=0x55d2df688b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66a9d0 0x55d2df688b50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8006010 tx=0x7fcaa800b670 comp rx=0 tx=0).stop 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 -- 172.21.15.62:0/2191627717 shutdown_connections 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55d2df6f4a60 0x55d2df6f3770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2df675bb0 0x55d2df6895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2df66b2f0 0x55d2df689090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 --2- 172.21.15.62:0/2191627717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2df66a9d0 0x55d2df688b50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fca9effd640 0 client.15081 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 -- 172.21.15.62:0/2191627717 >> 172.21.15.62:0/2191627717 conn(0x55d2df666f60 msgr2=0x55d2df661ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 -- 172.21.15.62:0/2191627717 shutdown_connections 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 1 -- 172.21.15.62:0/2191627717 wait complete. 2024-09-11T23:51:55.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.341+0000 7fcac840a740 20 client.15081 trim_cache size 0 max 16384 2024-09-11T23:51:55.362 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:51:55.362 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:51:55.363 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-09-11T23:51:55.363 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24469, 2024-09-11T23:51:55.363 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-09-11T23:51:55.363 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-09-11T23:51:55.363 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-09-11T23:51:55.363 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 95.482403371000004, 2024-09-11T23:51:55.364 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 24, 2024-09-11T23:51:55.364 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 83, 2024-09-11T23:51:55.364 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:51:55.364 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 128.54849361999999 2024-09-11T23:51:55.364 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:51:55.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 -- 172.21.15.62:0/1425922704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaac0a2b80 msgr2=0x7fcaac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaac0a2b80 0x7fcaac0a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcab80b8ad0 tx=0x7fcab80e6be0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 -- 172.21.15.62:0/1425922704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcac0102ef0 msgr2=0x7fcac019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcac0102ef0 0x7fcac019cd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcab8004770 tx=0x7fcab8003c10 comp rx=0 tx=0).stop 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 -- 172.21.15.62:0/1425922704 shutdown_connections 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fcaac0a2b80 0x7fcaac0a5040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcac0102ef0 0x7fcac019cd50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0102570 0x7fcac01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 --2- 172.21.15.62:0/1425922704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac0100e20 0x7fcac0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 -- 172.21.15.62:0/1425922704 >> 172.21.15.62:0/1425922704 conn(0x7fcac00fc7d0 msgr2=0x7fcac006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 -- 172.21.15.62:0/1425922704 shutdown_connections 2024-09-11T23:51:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:55.342+0000 7fcac6031640 1 -- 172.21.15.62:0/1425922704 wait complete. 2024-09-11T23:51:56.151 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:51:56.411 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:56.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:56 smithi062 ceph-mon[30958]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T23:51:56.469 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:56 smithi083 ceph-mon[34805]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T23:51:56.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:56 smithi120 ceph-mon[35182]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T23:51:56.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.917+0000 7f148c823640 1 -- 172.21.15.62:0/1451455837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1488101420 msgr2=0x7f148810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:56.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.917+0000 7f148c823640 1 --2- 172.21.15.62:0/1451455837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1488101420 0x7f148810d950 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1480009cd0 tx=0x7f148002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.917+0000 7f148c823640 1 -- 172.21.15.62:0/1451455837 shutdown_connections 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.917+0000 7f148c823640 1 --2- 172.21.15.62:0/1451455837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1488101420 0x7f148810d950 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.917+0000 7f148c823640 1 --2- 172.21.15.62:0/1451455837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1488100a80 0x7f1488100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.917+0000 7f148c823640 1 --2- 172.21.15.62:0/1451455837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1488106a80 0x7f1488106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.917+0000 7f148c823640 1 -- 172.21.15.62:0/1451455837 >> 172.21.15.62:0/1451455837 conn(0x7f14880fc7a0 msgr2=0x7f14880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.917+0000 7f148c823640 1 -- 172.21.15.62:0/1451455837 shutdown_connections 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.917+0000 7f148c823640 1 -- 172.21.15.62:0/1451455837 wait complete. 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f148c823640 1 Processor -- start 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f148c823640 1 -- start start 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f148c823640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1488100a80 0x7f148819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f148c823640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1488101420 0x7f148819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f148c823640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1488106a80 0x7f148819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f148c823640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1488110ea0 con 0x7f1488101420 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f148c823640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1488110d20 con 0x7f1488106a80 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f148c823640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1488111020 con 0x7f1488100a80 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f1485d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1488101420 0x7f148819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f1485d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1488101420 0x7f148819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47196/0 (socket says 172.21.15.62:47196) 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f1485d9b640 1 -- 172.21.15.62:0/3318672795 learned_addr learned my addr 172.21.15.62:0/3318672795 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f1486d9d640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1488106a80 0x7f148819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.918+0000 7f148659c640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1488100a80 0x7f148819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f1485d9b640 1 -- 172.21.15.62:0/3318672795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1488100a80 msgr2=0x7f148819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f1485d9b640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1488100a80 0x7f148819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f1485d9b640 1 -- 172.21.15.62:0/3318672795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1488106a80 msgr2=0x7f148819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f1485d9b640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1488106a80 0x7f148819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f1485d9b640 1 -- 172.21.15.62:0/3318672795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148819f610 con 0x7f1488101420 2024-09-11T23:51:56.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f148659c640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1488100a80 0x7f148819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f1485d9b640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1488101420 0x7f148819aba0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f147c00d950 tx=0x7f147c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f14777fe640 1 -- 172.21.15.62:0/3318672795 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c017070 con 0x7f1488101420 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f14777fe640 1 -- 172.21.15.62:0/3318672795 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147c009c30 con 0x7f1488101420 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f148c823640 1 -- 172.21.15.62:0/3318672795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148819f900 con 0x7f1488101420 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f14777fe640 1 -- 172.21.15.62:0/3318672795 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00f6b0 con 0x7f1488101420 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.919+0000 7f148c823640 1 -- 172.21.15.62:0/3318672795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881a71e0 con 0x7f1488101420 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.921+0000 7f14777fe640 1 -- 172.21.15.62:0/3318672795 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f147c0109d0 con 0x7f1488101420 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f148ebfc740 1 Processor -- start 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f148ebfc740 1 -- start start 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 0x5576239f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a03670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557623a03bb0 0x557623a05fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f148ebfc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55762395f090 con 0x557623a03bb0 2024-09-11T23:51:56.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f148ebfc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557623906a70 con 0x5576239f92f0 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f148ebfc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55762395b050 con 0x5576239f89d0 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f1485d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557623a03bb0 0x557623a05fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f148659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a03670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f1485d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557623a03bb0 0x557623a05fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47208/0 (socket says 172.21.15.62:47208) 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f148659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a03670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:33056/0 (socket says 172.21.15.62:33056) 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.922+0000 7f1485d9b640 1 -- 172.21.15.62:0/4208333943 learned_addr learned my addr 172.21.15.62:0/4208333943 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f14777fe640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f14700a2c00 0x7f14700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f1486d9d640 1 --2- 172.21.15.62:0/4208333943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 0x5576239f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f1486d9d640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f14700a2c00 0x7f14700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f148659c640 1 -- 172.21.15.62:0/4208333943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 msgr2=0x5576239f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f148659c640 1 --2- 172.21.15.62:0/4208333943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 0x5576239f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f148659c640 1 -- 172.21.15.62:0/4208333943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557623a03bb0 msgr2=0x557623a05fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:56.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f14777fe640 1 -- 172.21.15.62:0/3318672795 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f147c0cec80 con 0x7f1488101420 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f148659c640 1 --2- 172.21.15.62:0/4208333943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557623a03bb0 0x557623a05fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f148659c640 1 -- 172.21.15.62:0/4208333943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557623913130 con 0x5576239f92f0 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f1486d9d640 1 --2- 172.21.15.62:0/4208333943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 0x5576239f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f1485d9b640 1 --2- 172.21.15.62:0/4208333943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557623a03bb0 0x557623a05fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f148659c640 1 --2- 172.21.15.62:0/4208333943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a03670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f147800a940 tx=0x7f147800ae00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d727baed0626bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f1486d9d640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f14700a2c00 0x7f14700a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1480004ab0 tx=0x7f1480004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f1474ff9640 1 -- 172.21.15.62:0/4208333943 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478015070 con 0x5576239f92f0 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f1474ff9640 1 -- 172.21.15.62:0/4208333943 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147800cc00 con 0x5576239f92f0 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.923+0000 7f1474ff9640 1 -- 172.21.15.62:0/4208333943 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780185c0 con 0x5576239f92f0 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.924+0000 7f148ebfc740 1 -- 172.21.15.62:0/4208333943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 msgr2=0x557623a03670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.924+0000 7f148ebfc740 1 --2- 172.21.15.62:0/4208333943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a03670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f147800a940 tx=0x7f147800ae00 comp rx=0 tx=0).stop 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.924+0000 7f148ebfc740 1 -- 172.21.15.62:0/4208333943 shutdown_connections 2024-09-11T23:51:56.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.924+0000 7f148ebfc740 1 --2- 172.21.15.62:0/4208333943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557623a03bb0 0x557623a05fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.924+0000 7f148ebfc740 1 --2- 172.21.15.62:0/4208333943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a03670 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.924+0000 7f148ebfc740 1 --2- 172.21.15.62:0/4208333943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 0x5576239f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.924+0000 7f148ebfc740 1 -- 172.21.15.62:0/4208333943 >> 172.21.15.62:0/4208333943 conn(0x5576239f4f60 msgr2=0x5576239f5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.924+0000 7f148ebfc740 1 -- 172.21.15.62:0/4208333943 shutdown_connections 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.924+0000 7f148ebfc740 1 -- 172.21.15.62:0/4208333943 wait complete. 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.924+0000 7f148ebfc740 1 -- start start 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f148ebfc740 1 Processor -- start 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 0x557623a033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f92f0 0x557623a006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557623a03bb0 0x557623a00c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f148ebfc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557623a06780 con 0x5576239f92f0 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f148ebfc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557623919f20 con 0x557623a03bb0 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f148ebfc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557623a06900 con 0x5576239f89d0 2024-09-11T23:51:56.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f148ebfc740 10 client.0 _ll_register_callbacks cb 0x5576238e3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1485d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f92f0 0x557623a006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1485d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f92f0 0x557623a006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47222/0 (socket says 172.21.15.62:47222) 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1485d9b640 1 -- 172.21.15.62:0/3469415184 learned_addr learned my addr 172.21.15.62:0/3469415184 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1486d9d640 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557623a03bb0 0x557623a00c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f148659c640 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 0x557623a033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1485d9b640 1 -- 172.21.15.62:0/3469415184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 msgr2=0x557623a033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1485d9b640 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 0x557623a033a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1485d9b640 1 -- 172.21.15.62:0/3469415184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557623a03bb0 msgr2=0x557623a00c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1485d9b640 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557623a03bb0 0x557623a00c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1485d9b640 1 -- 172.21.15.62:0/3469415184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576236d02b0 con 0x5576239f92f0 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1486d9d640 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557623a03bb0 0x557623a00c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f1485d9b640 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f92f0 0x557623a006f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f147c009610 tx=0x7f147c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd67fcaef65f670f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.925+0000 7f146dffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:56.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f146dffb640 1 -- 172.21.15.62:0/3469415184 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c020040 con 0x5576239f92f0 2024-09-11T23:51:56.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f146dffb640 1 -- 172.21.15.62:0/3469415184 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f147c09a8e0 con 0x5576239f92f0 2024-09-11T23:51:56.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f148ebfc740 1 -- 172.21.15.62:0/3469415184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147800cc00 con 0x5576239f92f0 2024-09-11T23:51:56.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f146dffb640 1 -- 172.21.15.62:0/3469415184 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f147c09b490 con 0x5576239f92f0 2024-09-11T23:51:56.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f148ebfc740 10 client.15093 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:56.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f148ebfc740 1 -- 172.21.15.62:0/3469415184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557623a120a0 con 0x5576239f92f0 2024-09-11T23:51:56.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f146dffb640 1 -- 172.21.15.62:0/3469415184 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147800cc00 con 0x5576239f92f0 2024-09-11T23:51:56.963 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24484, 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 98.097172354999998, 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 24, 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 83, 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 124.886736864 2024-09-11T23:51:56.964 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:51:56.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f146dffb640 1 -- 172.21.15.62:0/3469415184 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f147c0971e0 con 0x5576239f92f0 2024-09-11T23:51:56.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f148ebfc740 10 client.15093 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:56.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f148ebfc740 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:51:56.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f148ebfc740 20 client.15093 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:56.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f148ebfc740 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557623a146a0 0x557623a15b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f148ebfc740 4 client.15093 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:51:56.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f148ebfc740 1 -- 172.21.15.62:0/3469415184 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55762396c600 con 0x557623a146a0 2024-09-11T23:51:56.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.926+0000 7f148659c640 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557623a146a0 0x557623a15b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.927+0000 7f148659c640 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557623a146a0 0x557623a15b10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57d898ad3abe4b5c server_cookie=165e93fddd121a05 in_seq=0 out_seq=0 2024-09-11T23:51:56.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.927+0000 7f146dffb640 10 client.15093 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f146dffb640 1 -- 172.21.15.62:0/3469415184 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55762396c600 con 0x557623a146a0 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f146dffb640 10 client.15093 handle_command_reply: tid=0 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 client.15093 shutdown 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 -- 172.21.15.62:0/3469415184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f92f0 msgr2=0x557623a006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f92f0 0x557623a006f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f147c009610 tx=0x7f147c010d70 comp rx=0 tx=0).stop 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 -- 172.21.15.62:0/3469415184 shutdown_connections 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557623a146a0 0x557623a15b10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557623a03bb0 0x557623a00c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f92f0 0x557623a006f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 --2- 172.21.15.62:0/3469415184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576239f89d0 0x557623a033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f146dffb640 0 client.15093 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 -- 172.21.15.62:0/3469415184 >> 172.21.15.62:0/3469415184 conn(0x5576239f4f60 msgr2=0x5576239fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:56.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 -- 172.21.15.62:0/3469415184 shutdown_connections 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 1 -- 172.21.15.62:0/3469415184 wait complete. 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.928+0000 7f148ebfc740 20 client.15093 trim_cache size 0 max 16384 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148ebfc740 1 Processor -- start 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148ebfc740 1 -- start start 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a13fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a14d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557623a03bb0 0x5576239fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148ebfc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557623a13e50 con 0x5576239f89d0 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148ebfc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557623a14520 con 0x5576239f92f0 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148ebfc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557623a14bf0 con 0x557623a03bb0 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a13fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a13fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47232/0 (socket says 172.21.15.62:47232) 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148659c640 1 -- 172.21.15.62:0/3056835268 learned_addr learned my addr 172.21.15.62:0/3056835268 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:56.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148659c640 1 -- 172.21.15.62:0/3056835268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557623a03bb0 msgr2=0x5576239fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148659c640 1 --2- 172.21.15.62:0/3056835268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557623a03bb0 0x5576239fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148659c640 1 -- 172.21.15.62:0/3056835268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 msgr2=0x557623a14d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148659c640 1 --2- 172.21.15.62:0/3056835268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a14d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148659c640 1 -- 172.21.15.62:0/3056835268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576239f4d90 con 0x5576239f89d0 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.946+0000 7f148659c640 1 --2- 172.21.15.62:0/3056835268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a13fd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1478004700 tx=0x7f1478011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be5e08f925eb306 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f146ffff640 1 -- 172.21.15.62:0/3056835268 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147802e070 con 0x5576239f89d0 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f146ffff640 1 -- 172.21.15.62:0/3056835268 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14780118d0 con 0x5576239f89d0 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f146ffff640 1 -- 172.21.15.62:0/3056835268 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478015040 con 0x5576239f89d0 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f148ebfc740 1 -- 172.21.15.62:0/3056835268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 msgr2=0x557623a13fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f148ebfc740 1 --2- 172.21.15.62:0/3056835268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a13fd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1478004700 tx=0x7f1478011d30 comp rx=0 tx=0).stop 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f148ebfc740 1 -- 172.21.15.62:0/3056835268 shutdown_connections 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f148ebfc740 1 --2- 172.21.15.62:0/3056835268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557623a03bb0 0x5576239fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f148ebfc740 1 --2- 172.21.15.62:0/3056835268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a14d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f148ebfc740 1 --2- 172.21.15.62:0/3056835268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a13fd0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f148ebfc740 1 -- 172.21.15.62:0/3056835268 >> 172.21.15.62:0/3056835268 conn(0x5576239f4f60 msgr2=0x5576239f5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f148ebfc740 1 -- 172.21.15.62:0/3056835268 shutdown_connections 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.947+0000 7f148ebfc740 1 -- 172.21.15.62:0/3056835268 wait complete. 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148ebfc740 1 -- start start 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148ebfc740 1 Processor -- start 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a1e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a19570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148ebfc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557623a03bb0 0x557623a19ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148ebfc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557623a15990 con 0x5576239f89d0 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148ebfc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557623a152c0 con 0x5576239f92f0 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148ebfc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557623a16060 con 0x557623a03bb0 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148ebfc740 10 client.0 _ll_register_callbacks cb 0x7f147c09b350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:56.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a1e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a1e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47240/0 (socket says 172.21.15.62:47240) 2024-09-11T23:51:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148659c640 1 -- 172.21.15.62:0/612126242 learned_addr learned my addr 172.21.15.62:0/612126242 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f1486d9d640 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557623a03bb0 0x557623a19ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f1485d9b640 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a19570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148659c640 1 -- 172.21.15.62:0/612126242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557623a03bb0 msgr2=0x557623a19ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148659c640 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557623a03bb0 0x557623a19ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148659c640 1 -- 172.21.15.62:0/612126242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 msgr2=0x557623a19570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148659c640 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a19570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.948+0000 7f148659c640 1 -- 172.21.15.62:0/612126242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557623a19ff0 con 0x5576239f89d0 2024-09-11T23:51:56.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f148659c640 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a1e1f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1478000c00 tx=0x7f1478011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e867d6c4541feb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f146dffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f146dffb640 1 -- 172.21.15.62:0/612126242 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147802e070 con 0x5576239f89d0 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f146dffb640 1 -- 172.21.15.62:0/612126242 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f14780113f0 con 0x5576239f89d0 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f148ebfc740 1 -- 172.21.15.62:0/612126242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14780118d0 con 0x5576239f89d0 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f146dffb640 1 -- 172.21.15.62:0/612126242 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1478015050 con 0x5576239f89d0 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f148ebfc740 10 client.15099 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f148ebfc740 1 -- 172.21.15.62:0/612126242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557623a7ea80 con 0x5576239f89d0 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f146dffb640 1 -- 172.21.15.62:0/612126242 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14780118d0 con 0x5576239f89d0 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f146dffb640 1 -- 172.21.15.62:0/612126242 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f147801a450 con 0x5576239f89d0 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f148ebfc740 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f148ebfc740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f148ebfc740 20 client.15099 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:56.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f148ebfc740 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557623a82d30 0x557623a81a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f148ebfc740 4 client.15099 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.949+0000 7f148ebfc740 1 -- 172.21.15.62:0/612126242 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "status"}) -- 0x557623966b10 con 0x557623a82d30 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f1485d9b640 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557623a82d30 0x557623a81a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f146dffb640 10 client.15099 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f1485d9b640 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557623a82d30 0x557623a81a40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be003fac8caabf18 server_cookie=8d6ef921ab9284cd in_seq=0 out_seq=0 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f146dffb640 1 -- 172.21.15.62:0/612126242 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557623966b10 con 0x557623a82d30 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f146dffb640 10 client.15099 handle_command_reply: tid=0 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 client.15099 shutdown 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 -- 172.21.15.62:0/612126242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 msgr2=0x557623a1e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a1e1f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1478000c00 tx=0x7f1478011ca0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 -- 172.21.15.62:0/612126242 shutdown_connections 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557623a82d30 0x557623a81a40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557623a03bb0 0x557623a19ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f146dffb640 0 client.15099 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576239f92f0 0x557623a19570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 --2- 172.21.15.62:0/612126242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576239f89d0 0x557623a1e1f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 -- 172.21.15.62:0/612126242 >> 172.21.15.62:0/612126242 conn(0x5576239f4f60 msgr2=0x5576239feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 -- 172.21.15.62:0/612126242 shutdown_connections 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 1 -- 172.21.15.62:0/612126242 wait complete. 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.950+0000 7f148ebfc740 20 client.15099 trim_cache size 0 max 16384 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 -- 172.21.15.62:0/3318672795 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f14700a2c00 msgr2=0x7f14700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f14700a2c00 0x7f14700a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1480004ab0 tx=0x7f1480004a40 comp rx=0 tx=0).stop 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 -- 172.21.15.62:0/3318672795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1488101420 msgr2=0x7f148819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1488101420 0x7f148819aba0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f147c00d950 tx=0x7f147c00de10 comp rx=0 tx=0).stop 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 -- 172.21.15.62:0/3318672795 shutdown_connections 2024-09-11T23:51:56.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f14700a2c00 0x7f14700a50c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1488106a80 0x7f148819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1488101420 0x7f148819aba0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 --2- 172.21.15.62:0/3318672795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1488100a80 0x7f148819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 -- 172.21.15.62:0/3318672795 >> 172.21.15.62:0/3318672795 conn(0x7f14880fc7a0 msgr2=0x7f148806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 -- 172.21.15.62:0/3318672795 shutdown_connections 2024-09-11T23:51:56.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:56.951+0000 7f148c823640 1 -- 172.21.15.62:0/3318672795 wait complete. 2024-09-11T23:51:57.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:51:58.016 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:51:58.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:51:58 smithi062 ceph-mon[30958]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T23:51:58.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.483+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/368280144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0102da0 msgr2=0x7fa9c0103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:58.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.483+0000 7fa9c4dbe640 1 --2- 172.21.15.62:0/368280144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0102da0 0x7fa9c0103180 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009ca0 tx=0x7fa9b802a950 comp rx=0 tx=0).stop 2024-09-11T23:51:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.483+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/368280144 shutdown_connections 2024-09-11T23:51:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.483+0000 7fa9c4dbe640 1 --2- 172.21.15.62:0/368280144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c010c180 0x7fa9c010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.483+0000 7fa9c4dbe640 1 --2- 172.21.15.62:0/368280144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c01036c0 0x7fa9c010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.483+0000 7fa9c4dbe640 1 --2- 172.21.15.62:0/368280144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0102da0 0x7fa9c0103180 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.483+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/368280144 >> 172.21.15.62:0/368280144 conn(0x7fa9c00fc7a0 msgr2=0x7fa9c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.483+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/368280144 shutdown_connections 2024-09-11T23:51:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.483+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/368280144 wait complete. 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.484+0000 7fa9c4dbe640 1 Processor -- start 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.484+0000 7fa9c4dbe640 1 -- start start 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.484+0000 7fa9c4dbe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0102da0 0x7fa9c0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.484+0000 7fa9c4dbe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c01036c0 0x7fa9c01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.484+0000 7fa9c4dbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c010c180 0x7fa9c019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.484+0000 7fa9c4dbe640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0111f60 con 0x7fa9c0102da0 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.484+0000 7fa9c4dbe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0111de0 con 0x7fa9c01036c0 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.484+0000 7fa9c4dbe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01120e0 con 0x7fa9c010c180 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.484+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0102da0 0x7fa9c0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0102da0 0x7fa9c0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47262/0 (socket says 172.21.15.62:47262) 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9be59c640 1 -- 172.21.15.62:0/663787225 learned_addr learned my addr 172.21.15.62:0/663787225 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9bdd9b640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c01036c0 0x7fa9c01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9bed9d640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c010c180 0x7fa9c019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9be59c640 1 -- 172.21.15.62:0/663787225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c010c180 msgr2=0x7fa9c019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9be59c640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c010c180 0x7fa9c019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9be59c640 1 -- 172.21.15.62:0/663787225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c01036c0 msgr2=0x7fa9c01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9be59c640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c01036c0 0x7fa9c01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9be59c640 1 -- 172.21.15.62:0/663787225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c019d410 con 0x7fa9c0102da0 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9be59c640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0102da0 0x7fa9c0198460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005cc0 tx=0x7fa9b802f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9b77fe640 1 -- 172.21.15.62:0/663787225 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8034040 con 0x7fa9c0102da0 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9b77fe640 1 -- 172.21.15.62:0/663787225 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b8030920 con 0x7fa9c0102da0 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/663787225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c019d6a0 con 0x7fa9c0102da0 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9b77fe640 1 -- 172.21.15.62:0/663787225 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8004830 con 0x7fa9c0102da0 2024-09-11T23:51:58.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.485+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/663787225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c01a4fe0 con 0x7fa9c0102da0 2024-09-11T23:51:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.488+0000 7fa9b77fe640 1 -- 172.21.15.62:0/663787225 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa9b8037020 con 0x7fa9c0102da0 2024-09-11T23:51:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.488+0000 7fa9c7197740 1 Processor -- start 2024-09-11T23:51:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.488+0000 7fa9c7197740 1 -- start start 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cfea9d0 0x55900cfeadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cff5bb0 0x55900cff7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9b77fe640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa9ac0a2c00 0x7fa9ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9c7197740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55900cf51090 con 0x55900cff5bb0 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9c7197740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55900cef8a70 con 0x55900cfea9d0 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9c7197740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55900cf4d050 con 0x55900cfeb2f0 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9bed9d640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa9ac0a2c00 0x7fa9ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9bdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cff5bb0 0x55900cff7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33844/0 (socket says 172.21.15.62:33844) 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9be59c640 1 -- 172.21.15.62:0/3630257699 learned_addr learned my addr 172.21.15.62:0/3630257699 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9bed9d640 1 --2- 172.21.15.62:0/3630257699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cfea9d0 0x55900cfeadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9b77fe640 1 -- 172.21.15.62:0/663787225 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa9b80eff60 con 0x7fa9c0102da0 2024-09-11T23:51:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9be59c640 1 -- 172.21.15.62:0/3630257699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cfea9d0 msgr2=0x55900cfeadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9be59c640 1 --2- 172.21.15.62:0/3630257699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cfea9d0 0x55900cfeadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9be59c640 1 -- 172.21.15.62:0/3630257699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cff5bb0 msgr2=0x55900cff7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9be59c640 1 --2- 172.21.15.62:0/3630257699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cff5bb0 0x55900cff7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9be59c640 1 -- 172.21.15.62:0/3630257699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900cf05130 con 0x55900cfeb2f0 2024-09-11T23:51:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9bed9d640 1 --2- 172.21.15.62:0/3630257699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cfea9d0 0x55900cfeadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9bdd9b640 1 --2- 172.21.15.62:0/3630257699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cff5bb0 0x55900cff7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:51:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.489+0000 7fa9bed9d640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa9ac0a2c00 0x7fa9ac0a50c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000bf20 tx=0x7fa9b00127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.490+0000 7fa9be59c640 1 --2- 172.21.15.62:0/3630257699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80b91a0 tx=0x7fa9b8004300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=882f48d38ff19edf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:58.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.490+0000 7fa9b4ff9640 1 -- 172.21.15.62:0/3630257699 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b802c040 con 0x55900cfeb2f0 2024-09-11T23:51:58.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.490+0000 7fa9b4ff9640 1 -- 172.21.15.62:0/3630257699 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b80bc7d0 con 0x55900cfeb2f0 2024-09-11T23:51:58.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.490+0000 7fa9b4ff9640 1 -- 172.21.15.62:0/3630257699 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80a70d0 con 0x55900cfeb2f0 2024-09-11T23:51:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.490+0000 7fa9c7197740 1 -- 172.21.15.62:0/3630257699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 msgr2=0x55900cff5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.490+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3630257699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80b91a0 tx=0x7fa9b8004300 comp rx=0 tx=0).stop 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 -- 172.21.15.62:0/3630257699 shutdown_connections 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3630257699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cff5bb0 0x55900cff7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3630257699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff5670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3630257699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cfea9d0 0x55900cfeadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 -- 172.21.15.62:0/3630257699 >> 172.21.15.62:0/3630257699 conn(0x55900cfe6f60 msgr2=0x55900cfe7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 -- 172.21.15.62:0/3630257699 shutdown_connections 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 -- 172.21.15.62:0/3630257699 wait complete. 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 -- start start 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 Processor -- start 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900cff54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900cff2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55900cff8540 con 0x55900cfea9d0 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55900cf0bf20 con 0x55900cff5bb0 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55900cff86c0 con 0x55900cfeb2f0 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9c7197740 10 client.0 _ll_register_callbacks cb 0x55900ced5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900cff54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900cff54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47284/0 (socket says 172.21.15.62:47284) 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.491+0000 7fa9be59c640 1 -- 172.21.15.62:0/2130811476 learned_addr learned my addr 172.21.15.62:0/2130811476 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9bdd9b640 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9bed9d640 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900cff2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9be59c640 1 -- 172.21.15.62:0/2130811476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 msgr2=0x55900cff2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9be59c640 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9be59c640 1 -- 172.21.15.62:0/2130811476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 msgr2=0x55900cff2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:58.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9be59c640 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900cff2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9be59c640 1 -- 172.21.15.62:0/2130811476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900ccc22b0 con 0x55900cfea9d0 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9bdd9b640 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9bed9d640 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900cff2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9be59c640 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900cff54b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80edde0 tx=0x7fa9b8003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd3a0e23e812dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/2130811476 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80c0b80 con 0x55900cfea9d0 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/2130811476 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa9b80a70d0 con 0x55900cfea9d0 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9c7197740 1 -- 172.21.15.62:0/2130811476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b80bc7d0 con 0x55900cfea9d0 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/2130811476 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa9b80db1e0 con 0x55900cfea9d0 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9c7197740 10 client.15111 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9c7197740 1 -- 172.21.15.62:0/2130811476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55900cff8540 con 0x55900cfea9d0 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.492+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/2130811476 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b80bc7d0 con 0x55900cfea9d0 2024-09-11T23:51:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.493+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/2130811476 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa9b804d030 con 0x55900cfea9d0 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.493+0000 7fa9c7197740 10 client.15111 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.493+0000 7fa9c7197740 10 client.15111 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.493+0000 7fa9c7197740 20 client.15111 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.493+0000 7fa9c7197740 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55900d006280 0x55900d0076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.493+0000 7fa9c7197740 4 client.15111 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.493+0000 7fa9c7197740 1 -- 172.21.15.62:0/2130811476 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55900cf5e600 con 0x55900d006280 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.493+0000 7fa9bdd9b640 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55900d006280 0x55900d0076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.493+0000 7fa9a5ffb640 10 client.15111 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.493+0000 7fa9bdd9b640 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55900d006280 0x55900d0076f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267ad23df9172ab1 server_cookie=deecccfc1d5beba5 in_seq=0 out_seq=0 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/2130811476 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55900cf5e600 con 0x55900d006280 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9a5ffb640 10 client.15111 handle_command_reply: tid=0 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9c7197740 1 client.15111 shutdown 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9c7197740 1 -- 172.21.15.62:0/2130811476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 msgr2=0x55900cff54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9c7197740 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900cff54b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80edde0 tx=0x7fa9b8003e80 comp rx=0 tx=0).stop 2024-09-11T23:51:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9c7197740 1 -- 172.21.15.62:0/2130811476 shutdown_connections 2024-09-11T23:51:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9c7197740 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55900d006280 0x55900d0076f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9c7197740 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900cff2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9c7197740 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900cff2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9c7197740 1 --2- 172.21.15.62:0/2130811476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900cff54b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9c7197740 1 -- 172.21.15.62:0/2130811476 >> 172.21.15.62:0/2130811476 conn(0x55900cfe6f60 msgr2=0x55900cfecc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.494+0000 7fa9a5ffb640 0 client.15111 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:51:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.495+0000 7fa9c7197740 1 -- 172.21.15.62:0/2130811476 shutdown_connections 2024-09-11T23:51:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.495+0000 7fa9c7197740 1 -- 172.21.15.62:0/2130811476 wait complete. 2024-09-11T23:51:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.495+0000 7fa9c7197740 20 client.15111 trim_cache size 0 max 16384 2024-09-11T23:51:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.512+0000 7fa9c7197740 1 Processor -- start 2024-09-11T23:51:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.512+0000 7fa9c7197740 1 -- start start 2024-09-11T23:51:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900d007020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900d00f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9c7197740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55900d005a30 con 0x55900cfea9d0 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9c7197740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55900d006100 con 0x55900cff5bb0 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9c7197740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55900d0067d0 con 0x55900cfeb2f0 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d005bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47288/0 (socket says 172.21.15.62:47288) 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9be59c640 1 -- 172.21.15.62:0/3270957244 learned_addr learned my addr 172.21.15.62:0/3270957244 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9bdd9b640 1 --2- 172.21.15.62:0/3270957244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900d007020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9bed9d640 1 --2- 172.21.15.62:0/3270957244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900d00f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9be59c640 1 -- 172.21.15.62:0/3270957244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 msgr2=0x55900d007020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9be59c640 1 --2- 172.21.15.62:0/3270957244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900d007020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9be59c640 1 -- 172.21.15.62:0/3270957244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 msgr2=0x55900d00f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9be59c640 1 --2- 172.21.15.62:0/3270957244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900d00f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9be59c640 1 -- 172.21.15.62:0/3270957244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900cfe6a00 con 0x55900cfea9d0 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9be59c640 1 --2- 172.21.15.62:0/3270957244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d005bb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80f0310 tx=0x7fa9b80bace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a622ef4de7b83a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.513+0000 7fa9a7fff640 1 -- 172.21.15.62:0/3270957244 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80db1e0 con 0x55900cfea9d0 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9a7fff640 1 -- 172.21.15.62:0/3270957244 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b80bc7d0 con 0x55900cfea9d0 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9a7fff640 1 -- 172.21.15.62:0/3270957244 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80bbc20 con 0x55900cfea9d0 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9c7197740 1 -- 172.21.15.62:0/3270957244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 msgr2=0x55900d005bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3270957244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d005bb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80f0310 tx=0x7fa9b80bace0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9c7197740 1 -- 172.21.15.62:0/3270957244 shutdown_connections 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3270957244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900d00f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3270957244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900d007020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3270957244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d005bb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9c7197740 1 -- 172.21.15.62:0/3270957244 >> 172.21.15.62:0/3270957244 conn(0x55900cfe6f60 msgr2=0x55900cff1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9c7197740 1 -- 172.21.15.62:0/3270957244 shutdown_connections 2024-09-11T23:51:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.514+0000 7fa9c7197740 1 -- 172.21.15.62:0/3270957244 wait complete. 2024-09-11T23:51:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9c7197740 1 -- start start 2024-09-11T23:51:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9c7197740 1 Processor -- start 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d00a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900d00ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9c7197740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900d00b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9c7197740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55900d007570 con 0x55900cfea9d0 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9c7197740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55900d006ea0 con 0x55900cff5bb0 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9c7197740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55900d007c40 con 0x55900cfeb2f0 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9c7197740 10 client.0 _ll_register_callbacks cb 0x7fa9a8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d00a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d00a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47292/0 (socket says 172.21.15.62:47292) 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9be59c640 1 -- 172.21.15.62:0/3507781098 learned_addr learned my addr 172.21.15.62:0/3507781098 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9bdd9b640 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900d00ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9bed9d640 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900d00b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9be59c640 1 -- 172.21.15.62:0/3507781098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 msgr2=0x55900d00ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9be59c640 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900d00ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9be59c640 1 -- 172.21.15.62:0/3507781098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 msgr2=0x55900d00b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9be59c640 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900d00b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.515+0000 7fa9be59c640 1 -- 172.21.15.62:0/3507781098 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900d06e9d0 con 0x55900cfea9d0 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9be59c640 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d00a700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80b91a0 tx=0x7fa9b8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152cf98422b0cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/3507781098 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80db1e0 con 0x55900cfea9d0 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/3507781098 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa9b80bdb50 con 0x55900cfea9d0 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9c7197740 1 -- 172.21.15.62:0/3507781098 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b80bc7d0 con 0x55900cfea9d0 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/3507781098 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa9b80bdd50 con 0x55900cfea9d0 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9c7197740 10 client.15117 fetch_fsmap learned FSMap version 24 2024-09-11T23:51:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9c7197740 1 -- 172.21.15.62:0/3507781098 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55900d06e9d0 con 0x55900cfea9d0 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/3507781098 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b80bc7d0 con 0x55900cfea9d0 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/3507781098 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa9b80b0020 con 0x55900cfea9d0 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9c7197740 10 client.15117 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9c7197740 10 client.15117 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9c7197740 20 client.15117 populate_metadata read hostname 'smithi062' 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55900d074170 0x55900d072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9c7197740 4 client.15117 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.516+0000 7fa9c7197740 1 -- 172.21.15.62:0/3507781098 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "status"}) -- 0x55900cf58b10 con 0x55900d074170 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9bdd9b640 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55900d074170 0x55900d072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9bdd9b640 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55900d074170 0x55900d072e80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74af381c645e8dac server_cookie=54fac59a639c8bed in_seq=0 out_seq=0 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9a5ffb640 10 client.15117 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9a5ffb640 1 -- 172.21.15.62:0/3507781098 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55900cf58b10 con 0x55900d074170 2024-09-11T23:51:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9a5ffb640 10 client.15117 handle_command_reply: tid=0 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 client.15117 shutdown 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 -- 172.21.15.62:0/3507781098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 msgr2=0x55900d00a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d00a700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80b91a0 tx=0x7fa9b8003f10 comp rx=0 tx=0).stop 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 -- 172.21.15.62:0/3507781098 shutdown_connections 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55900d074170 0x55900d072e80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55900cff5bb0 0x55900d00b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55900cfeb2f0 0x55900d00ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 --2- 172.21.15.62:0/3507781098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55900cfea9d0 0x55900d00a700 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 -- 172.21.15.62:0/3507781098 >> 172.21.15.62:0/3507781098 conn(0x55900cfe6f60 msgr2=0x55900cff2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9a5ffb640 0 client.15117 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 -- 172.21.15.62:0/3507781098 shutdown_connections 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.517+0000 7fa9c7197740 1 -- 172.21.15.62:0/3507781098 wait complete. 2024-09-11T23:51:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c7197740 20 client.15117 trim_cache size 0 max 16384 2024-09-11T23:51:58.524 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:51:58.524 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:51:58.524 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-09-11T23:51:58.524 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24488, 2024-09-11T23:51:58.525 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-09-11T23:51:58.525 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-09-11T23:51:58.525 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-09-11T23:51:58.525 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 106.648304596, 2024-09-11T23:51:58.525 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 24, 2024-09-11T23:51:58.525 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 83, 2024-09-11T23:51:58.525 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:51:58.525 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 117.829962198 2024-09-11T23:51:58.525 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:51:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/663787225 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa9ac0a2c00 msgr2=0x7fa9ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa9ac0a2c00 0x7fa9ac0a50c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000bf20 tx=0x7fa9b00127c0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/663787225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0102da0 msgr2=0x7fa9c0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0102da0 0x7fa9c0198460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005cc0 tx=0x7fa9b802f630 comp rx=0 tx=0).stop 2024-09-11T23:51:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/663787225 shutdown_connections 2024-09-11T23:51:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fa9ac0a2c00 0x7fa9ac0a50c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c010c180 0x7fa9c019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c01036c0 0x7fa9c01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 --2- 172.21.15.62:0/663787225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0102da0 0x7fa9c0198460 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:58.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/663787225 >> 172.21.15.62:0/663787225 conn(0x7fa9c00fc7a0 msgr2=0x7fa9c00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:58.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/663787225 shutdown_connections 2024-09-11T23:51:58.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:58.518+0000 7fa9c4dbe640 1 -- 172.21.15.62:0/663787225 wait complete. 2024-09-11T23:51:58.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:51:58 smithi120 ceph-mon[35182]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T23:51:58.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:51:58 smithi083 ceph-mon[34805]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T23:51:59.175 INFO:teuthology.run_tasks:Running task sequential... 2024-09-11T23:51:59.184 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-11T23:51:59.184 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-11T23:51:59.184 INFO:teuthology.task.print:Enabling mgr modules 2024-09-11T23:51:59.185 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-11T23:51:59.185 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:51:59.186 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-09-11T23:51:59.186 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-11T23:51:59.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.292+0000 7f5cc9add640 1 -- 172.21.15.62:0/661401650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cc41025d0 msgr2=0x7f5cc41049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.292+0000 7f5cc9add640 1 --2- 172.21.15.62:0/661401650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cc41025d0 0x7f5cc41049c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009900 tx=0x7f5cb402a410 comp rx=0 tx=0).stop 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.292+0000 7f5cc9add640 1 -- 172.21.15.62:0/661401650 shutdown_connections 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.292+0000 7f5cc9add640 1 --2- 172.21.15.62:0/661401650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc4107830 0x7f5cc4109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.292+0000 7f5cc9add640 1 --2- 172.21.15.62:0/661401650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc4104f00 0x7f5cc41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 --2- 172.21.15.62:0/661401650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cc41025d0 0x7f5cc41049c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 -- 172.21.15.62:0/661401650 >> 172.21.15.62:0/661401650 conn(0x7f5cc40fc1b0 msgr2=0x7f5cc40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 -- 172.21.15.62:0/661401650 shutdown_connections 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 -- 172.21.15.62:0/661401650 wait complete. 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 Processor -- start 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 -- start start 2024-09-11T23:51:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cc41025d0 0x7f5cc4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc4104f00 0x7f5cc4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc4107830 0x7f5cc419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410d810 con 0x7f5cc41025d0 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.293+0000 7f5cc9add640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410d690 con 0x7f5cc4104f00 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc9add640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410d990 con 0x7f5cc4107830 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cc41025d0 0x7f5cc4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cc41025d0 0x7f5cc4197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47310/0 (socket says 172.21.15.62:47310) 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc37fe640 1 -- 172.21.15.62:0/2202689899 learned_addr learned my addr 172.21.15.62:0/2202689899 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc3fff640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc4107830 0x7f5cc419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc2ffd640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc4104f00 0x7f5cc4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc37fe640 1 -- 172.21.15.62:0/2202689899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc4107830 msgr2=0x7f5cc419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc37fe640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc4107830 0x7f5cc419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc37fe640 1 -- 172.21.15.62:0/2202689899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc4104f00 msgr2=0x7f5cc4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc37fe640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc4104f00 0x7f5cc4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc37fe640 1 -- 172.21.15.62:0/2202689899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc419fa50 con 0x7f5cc41025d0 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc3fff640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc4107830 0x7f5cc419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc2ffd640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc4104f00 0x7f5cc4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc37fe640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cc41025d0 0x7f5cc4197e30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009230 tx=0x7f5cb4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc0ff9640 1 -- 172.21.15.62:0/2202689899 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb403f030 con 0x7f5cc41025d0 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc0ff9640 1 -- 172.21.15.62:0/2202689899 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb4003ce0 con 0x7f5cc41025d0 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc9add640 1 -- 172.21.15.62:0/2202689899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc419fce0 con 0x7f5cc41025d0 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc0ff9640 1 -- 172.21.15.62:0/2202689899 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4035070 con 0x7f5cc41025d0 2024-09-11T23:51:59.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.294+0000 7f5cc9add640 1 -- 172.21.15.62:0/2202689899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc41a01d0 con 0x7f5cc41025d0 2024-09-11T23:51:59.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.295+0000 7f5cc9add640 1 -- 172.21.15.62:0/2202689899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc40fdca0 con 0x7f5cc41025d0 2024-09-11T23:51:59.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.296+0000 7f5cc0ff9640 1 -- 172.21.15.62:0/2202689899 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5cb4042030 con 0x7f5cc41025d0 2024-09-11T23:51:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.297+0000 7f5cc0ff9640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5cac0a2c30 0x7f5cac0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:51:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.297+0000 7f5cc2ffd640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5cac0a2c30 0x7f5cac0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:51:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.297+0000 7f5cc0ff9640 1 -- 172.21.15.62:0/2202689899 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f5cb40efad0 con 0x7f5cc41025d0 2024-09-11T23:51:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.298+0000 7f5cc2ffd640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5cac0a2c30 0x7f5cac0a50f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4199340 tx=0x7f5cb8005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:51:59.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.301+0000 7f5cc0ff9640 1 -- 172.21.15.62:0/2202689899 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cb40bca40 con 0x7f5cc41025d0 2024-09-11T23:51:59.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:51:59.525+0000 7f5cc9add640 1 -- 172.21.15.62:0/2202689899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f5cc4062df0 con 0x7f5cc41025d0 2024-09-11T23:52:00.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.218+0000 7f5cc0ff9640 1 -- 172.21.15.62:0/2202689899 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f5cb40bc340 con 0x7f5cc41025d0 2024-09-11T23:52:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.221+0000 7f5cc9add640 1 -- 172.21.15.62:0/2202689899 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5cac0a2c30 msgr2=0x7f5cac0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.221+0000 7f5cc9add640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5cac0a2c30 0x7f5cac0a50f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4199340 tx=0x7f5cb8005ec0 comp rx=0 tx=0).stop 2024-09-11T23:52:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.221+0000 7f5cc9add640 1 -- 172.21.15.62:0/2202689899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cc41025d0 msgr2=0x7f5cc4197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.221+0000 7f5cc9add640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cc41025d0 0x7f5cc4197e30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009230 tx=0x7f5cb4004360 comp rx=0 tx=0).stop 2024-09-11T23:52:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.221+0000 7f5cc9add640 1 -- 172.21.15.62:0/2202689899 shutdown_connections 2024-09-11T23:52:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.221+0000 7f5cc9add640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f5cac0a2c30 0x7f5cac0a50f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.221+0000 7f5cc9add640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc4107830 0x7f5cc419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.221+0000 7f5cc9add640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc4104f00 0x7f5cc4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:00.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.221+0000 7f5cc9add640 1 --2- 172.21.15.62:0/2202689899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cc41025d0 0x7f5cc4197e30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:00.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.221+0000 7f5cc9add640 1 -- 172.21.15.62:0/2202689899 >> 172.21.15.62:0/2202689899 conn(0x7f5cc40fc1b0 msgr2=0x7f5cc4102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:00.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.222+0000 7f5cc9add640 1 -- 172.21.15.62:0/2202689899 shutdown_connections 2024-09-11T23:52:00.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:00.222+0000 7f5cc9add640 1 -- 172.21.15.62:0/2202689899 wait complete. 2024-09-11T23:52:00.239 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T23:52:00.269 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T23:52:00.269 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-11T23:52:00.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:52:00.269 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-09-11T23:52:00.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:52:00.289 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-09-11T23:52:00.358 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-09-11T23:52:00.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:52:00.360 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-09-11T23:52:00.360 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T23:52:00.360 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-11T23:52:00.360 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-11T23:52:00.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T23:52:00.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:00 smithi062 ceph-mon[30958]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:52:00.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:00 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2202689899' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:52:00.465 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: ignoring --setuser ceph since I am not root 2024-09-11T23:52:00.465 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: ignoring --setgroup ceph since I am not root 2024-09-11T23:52:00.466 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:00.380+0000 7f7815a67040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:52:00.466 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:00.441+0000 7f7815a67040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:52:00.571 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-mon[34805]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:52:00.571 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2202689899' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:52:00.571 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: ignoring --setuser ceph since I am not root 2024-09-11T23:52:00.571 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: ignoring --setgroup ceph since I am not root 2024-09-11T23:52:00.572 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-mgr[36175]: -- 172.21.15.83:0/204209044 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f49f5570e0 con 0x55f49f535400 2024-09-11T23:52:00.572 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:00.370+0000 7f55f9850040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:52:00.572 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:00.426+0000 7f55f9850040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:52:00.572 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:00.571+0000 7f55f9850040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:52:00.574 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-mon[35182]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:52:00.574 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2202689899' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:52:00.574 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: ignoring --setuser ceph since I am not root 2024-09-11T23:52:00.574 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: ignoring --setgroup ceph since I am not root 2024-09-11T23:52:00.574 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-mgr[36988]: -- 172.21.15.120:0/1401691413 <== mon.1 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583c36fb0e0 con 0x5583c36d8800 2024-09-11T23:52:00.574 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:00.372+0000 7f73f5c15040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:52:00.574 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:00.429+0000 7f73f5c15040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:52:00.672 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:00.870 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:00.573+0000 7f73f5c15040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:52:00.871 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:00.639+0000 7f73f5c15040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:52:00.871 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:00.787+0000 7f73f5c15040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:52:00.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:00.636+0000 7f55f9850040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:52:00.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:00.786+0000 7f55f9850040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:52:00.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:00.599+0000 7f7815a67040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:52:00.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:00.677+0000 7f7815a67040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:52:00.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:00.835+0000 7f7815a67040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:52:01.214 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:00.986+0000 7f7815a67040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:52:01.221 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:00.929+0000 7f73f5c15040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:52:01.221 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:00.928+0000 7f55f9850040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:52:01.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.210+0000 7f5042ef3640 1 -- 172.21.15.62:0/1085754122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503c1027c0 msgr2=0x7f503c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:01.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.210+0000 7f5042ef3640 1 --2- 172.21.15.62:0/1085754122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503c1027c0 0x7f503c106810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5028009860 tx=0x7f502802a340 comp rx=0 tx=0).stop 2024-09-11T23:52:01.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.210+0000 7f5042ef3640 1 -- 172.21.15.62:0/1085754122 shutdown_connections 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.210+0000 7f5042ef3640 1 --2- 172.21.15.62:0/1085754122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f503c106f40 0x7f503c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.210+0000 7f5042ef3640 1 --2- 172.21.15.62:0/1085754122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503c1027c0 0x7f503c106810 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.210+0000 7f5042ef3640 1 --2- 172.21.15.62:0/1085754122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c101e10 0x7f503c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.210+0000 7f5042ef3640 1 -- 172.21.15.62:0/1085754122 >> 172.21.15.62:0/1085754122 conn(0x7f503c0fd8d0 msgr2=0x7f503c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.211+0000 7f5042ef3640 1 -- 172.21.15.62:0/1085754122 shutdown_connections 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.211+0000 7f5042ef3640 1 -- 172.21.15.62:0/1085754122 wait complete. 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.211+0000 7f5042ef3640 1 Processor -- start 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.211+0000 7f5042ef3640 1 -- start start 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f5042ef3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f503c101e10 0x7f503c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f5042ef3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c1027c0 0x7f503c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f5042ef3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503c106f40 0x7f503c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:01.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f5042ef3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f503c111fa0 con 0x7f503c106f40 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f5042ef3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f503c111e20 con 0x7f503c1027c0 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f5042ef3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f503c112120 con 0x7f503c101e10 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f5040c8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f503c101e10 0x7f503c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f5040c8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f503c101e10 0x7f503c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33908/0 (socket says 172.21.15.62:33908) 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f503bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c1027c0 0x7f503c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f5040c8f640 1 -- 172.21.15.62:0/2633621846 learned_addr learned my addr 172.21.15.62:0/2633621846 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f5041490640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503c106f40 0x7f503c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f503bfff640 1 -- 172.21.15.62:0/2633621846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f503c101e10 msgr2=0x7f503c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f503bfff640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f503c101e10 0x7f503c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f503bfff640 1 -- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503c106f40 msgr2=0x7f503c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f503bfff640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503c106f40 0x7f503c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.212+0000 7f503bfff640 1 -- 172.21.15.62:0/2633621846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c19f5e0 con 0x7f503c1027c0 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.213+0000 7f503bfff640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c1027c0 0x7f503c19ab70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5028035040 tx=0x7f5028033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.213+0000 7f5039ffb640 1 -- 172.21.15.62:0/2633621846 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502803f020 con 0x7f503c1027c0 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.213+0000 7f5039ffb640 1 -- 172.21.15.62:0/2633621846 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5028030550 con 0x7f503c1027c0 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.213+0000 7f5042ef3640 1 -- 172.21.15.62:0/2633621846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503c19f870 con 0x7f503c1027c0 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.213+0000 7f5039ffb640 1 -- 172.21.15.62:0/2633621846 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028037040 con 0x7f503c1027c0 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.213+0000 7f5042ef3640 1 -- 172.21.15.62:0/2633621846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f503c1a71b0 con 0x7f503c1027c0 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.214+0000 7f5042ef3640 1 -- 172.21.15.62:0/2633621846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f503c1021f0 con 0x7f503c1027c0 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.215+0000 7f5039ffb640 1 -- 172.21.15.62:0/2633621846 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f5028042020 con 0x7f503c1027c0 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.216+0000 7f5039ffb640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f502c0a2c50 0x7f502c0a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.216+0000 7f5040c8f640 1 -- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f502c0a2c50 msgr2=0x7f502c0a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3106394779 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.216+0000 7f5039ffb640 1 -- 172.21.15.62:0/2633621846 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f50280ef160 con 0x7f503c1027c0 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.216+0000 7f5040c8f640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f502c0a2c50 0x7f502c0a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:52:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.220+0000 7f5039ffb640 1 -- 172.21.15.62:0/2633621846 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5028049020 con 0x7f503c1027c0 2024-09-11T23:52:01.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.417+0000 7f5040c8f640 1 -- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f502c0a2c50 msgr2=0x7f502c0a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3106394779 2024-09-11T23:52:01.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.417+0000 7f5040c8f640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f502c0a2c50 0x7f502c0a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:52:01.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.437+0000 7f5042ef3640 1 -- 172.21.15.62:0/2633621846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f503c1a7380 con 0x7f503c1027c0 2024-09-11T23:52:01.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.438+0000 7f5039ffb640 1 -- 172.21.15.62:0/2633621846 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f50280bc070 con 0x7f503c1027c0 2024-09-11T23:52:01.438 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T23:52:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 -- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f502c0a2c50 msgr2=0x7f502c0a5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:52:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f502c0a2c50 0x7f502c0a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 -- 172.21.15.62:0/2633621846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c1027c0 msgr2=0x7f503c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c1027c0 0x7f503c19ab70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5028035040 tx=0x7f5028033b60 comp rx=0 tx=0).stop 2024-09-11T23:52:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 -- 172.21.15.62:0/2633621846 shutdown_connections 2024-09-11T23:52:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f502c0a2c50 0x7f502c0a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:01.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503c106f40 0x7f503c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:01.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c1027c0 0x7f503c19ab70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:01.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 --2- 172.21.15.62:0/2633621846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f503c101e10 0x7f503c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:01.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 -- 172.21.15.62:0/2633621846 >> 172.21.15.62:0/2633621846 conn(0x7f503c0fd8d0 msgr2=0x7f503c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:01.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.440+0000 7f5042ef3640 1 -- 172.21.15.62:0/2633621846 shutdown_connections 2024-09-11T23:52:01.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:01.441+0000 7f5042ef3640 1 -- 172.21.15.62:0/2633621846 wait complete. 2024-09-11T23:52:01.482 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:01 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2202689899' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:52:01.482 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:01 smithi083 ceph-mon[34805]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T23:52:01.482 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:01 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:01.238+0000 7f55f9850040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:52:01.482 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:01 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:01.299+0000 7f55f9850040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:52:01.484 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:01 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2202689899' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:52:01.484 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:01 smithi120 ceph-mon[35182]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T23:52:01.484 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:01.240+0000 7f73f5c15040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:52:01.484 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:01.300+0000 7f73f5c15040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:52:01.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:01 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2202689899' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:52:01.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:01 smithi062 ceph-mon[30958]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T23:52:01.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:01 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:01.321+0000 7f7815a67040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:52:01.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:01 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:01.385+0000 7f7815a67040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:52:01.591 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:01 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:01.591+0000 7f7815a67040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:52:01.870 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:01.484+0000 7f73f5c15040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:52:01.916 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:01 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:01.482+0000 7f55f9850040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:52:02.195 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T23:52:02.195 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T23:52:02.195 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-09-11T23:52:02.195 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T23:52:02.195 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-11T23:52:02.196 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T23:52:02.196 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T23:52:02.196 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T23:52:02.196 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-11T23:52:02.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:52:02.196 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-09-11T23:52:02.213 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T23:52:02.213 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T23:52:02.213 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T23:52:02.213 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-11T23:52:02.214 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42193sec preferred_lft 42193sec 2024-09-11T23:52:02.215 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link noprefixroute 2024-09-11T23:52:02.215 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:52:02.215 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T23:52:02.215 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:52:02.215 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-11T23:52:02.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-09-11T23:52:02.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-09-11T23:52:02.216 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-09-11T23:52:02.216 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T23:52:02.216 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-11T23:52:02.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:52:02.416 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:02 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2633621846' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:52:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:52:02.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:52:02.436 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T23:52:02.463 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-09-11T23:52:02.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:52:02.466 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-09-11T23:52:02.534 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-09-11T23:52:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T23:52:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T23:52:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T23:52:02.535 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T23:52:02.536 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:52:02.536 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-11T23:52:02.536 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T23:52:02.536 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T23:52:02.537 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T23:52:02.537 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-11T23:52:02.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:02 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2633621846' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:52:02.621 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:02.370+0000 7f73f5c15040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:52:02.621 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:02.437+0000 7f73f5c15040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:52:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:52:02.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:02 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2633621846' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:52:02.667 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:02 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:02.357+0000 7f55f9850040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:52:02.667 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:02 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:02.424+0000 7f55f9850040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:52:02.715 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:02 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:02.539+0000 7f7815a67040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:52:02.715 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:02 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:02.612+0000 7f7815a67040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:52:02.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:52:02.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:52:02.852 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-09-11T23:52:02.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:52:02.869 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-09-11T23:52:02.925 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:52:02.925 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-11T23:52:02.925 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-11T23:52:02.925 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-11T23:52:02.925 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-11T23:52:03.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:52:03.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:52:03.048 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-11T23:52:03.048 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:52:03.048 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-11T23:52:03.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-11T23:52:03.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T23:52:03.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-11T23:52:03.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-11T23:52:03.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-11T23:52:03.048 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-11T23:52:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:52:03.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:52:03.178 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:52:03.178 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-11T23:52:03.178 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-09-11T23:52:03.178 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T23:52:03.178 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-11T23:52:03.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:52:03.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:52:03.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T23:52:03.260 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /tmp/cephfs 2024-09-11T23:52:03.277 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-11T23:52:03.278 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T23:52:03.278 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /tmp/cephfs 2024-09-11T23:52:03.334 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-11T23:52:03.334 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-11T23:52:03.334 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/324e6219-dcd5-475c-bde4-eff522131966 2024-09-11T23:52:03.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:52:03.335 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/324e6219-dcd5-475c-bde4-eff522131966 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-09-11T23:52:03.386 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:03.130+0000 7f55f9850040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:52:03.386 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:03.245+0000 7f55f9850040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:52:03.404 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:03.148+0000 7f73f5c15040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:52:03.404 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:03.264+0000 7f73f5c15040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:52:03.433 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-11T23:52:03.433 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-11T23:52:03.433 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-09-11T23:52:03.576 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-11T23:52:03.576 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-11T23:52:03.576 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-09-11T23:52:03.577 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.83:3300,172.21.15.120:3300" 2024-09-11T23:52:03.577 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:3300,172.21.15.83:3300,172.21.15.120:3300:/volumes/qa/sv_1/324e6219-dcd5-475c-bde4-eff522131966 2024-09-11T23:52:03.577 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=56675698-7097-11ef-bcea-c7b262605968" will pass to kernel 2024-09-11T23:52:03.579 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T23:52:03.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:52:03.580 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /tmp/cephfs 2024-09-11T23:52:03.610 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:52:03.661 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:03.386+0000 7f55f9850040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:52:03.661 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:03.446+0000 7f55f9850040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:52:03.661 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:03.594+0000 7f55f9850040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:52:03.661 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:03.661+0000 7f55f9850040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:52:03.671 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:03.390+0000 7f7815a67040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:52:03.671 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:03.516+0000 7f7815a67040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:52:03.680 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:03.404+0000 7f73f5c15040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:52:03.680 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:03.464+0000 7f73f5c15040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:52:03.680 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:03.613+0000 7f73f5c15040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:52:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:52:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:52:03.709 INFO:teuthology.orchestra.run.smithi062.stdout:client24838 2024-09-11T23:52:03.710 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-11T23:52:03.710 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:52:03.769 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:52:03.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:52:03.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:52:03.869 INFO:teuthology.orchestra.run.smithi062.stdout:client24838 2024-09-11T23:52:03.869 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:52:03.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:03.722+0000 7f55f9850040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:52:03.917 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:03.879+0000 7f55f9850040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:52:03.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:52:03.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:52:03.928 INFO:teuthology.orchestra.run.smithi062.stdout:client24838 2024-09-11T23:52:03.929 DEBUG:teuthology.orchestra.run.smithi062:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/56675698-7097-11ef-bcea-c7b262605968.client24838/status') 2024-09-11T23:52:03.964 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:03.680+0000 7f73f5c15040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:52:03.964 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:03.741+0000 7f73f5c15040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:52:03.964 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:03.898+0000 7f73f5c15040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:52:03.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:03.671+0000 7f7815a67040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:52:03.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:03.738+0000 7f7815a67040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:52:03.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:03.900+0000 7f7815a67040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:52:03.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:52:03.990 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.24838 (3)172.21.15.62:0/2275220097 2024-09-11T23:52:03.990 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-09-11T23:52:03.990 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-09-11T23:52:03.990 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-09-11T23:52:03.990 INFO:teuthology.orchestra.run.smithi062.stderr:71 bytes copied, 2.4918e-05 s, 2.8 MB/s 2024-09-11T23:52:03.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:52:03.998 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T23:52:04.006 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:52:04.007 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-09-11T23:52:04.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-11T23:52:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.145+0000 7fd140e8b640 1 -- 172.21.15.62:0/1965692419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c1078c0 msgr2=0x7fd13c109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.145+0000 7fd140e8b640 1 --2- 172.21.15.62:0/1965692419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c1078c0 0x7fd13c109cb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd134009ca0 tx=0x7fd13402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.145+0000 7fd140e8b640 1 -- 172.21.15.62:0/1965692419 shutdown_connections 2024-09-11T23:52:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.145+0000 7fd140e8b640 1 --2- 172.21.15.62:0/1965692419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c1078c0 0x7fd13c109cb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.145+0000 7fd140e8b640 1 --2- 172.21.15.62:0/1965692419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd13c104f90 0x7fd13c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.145+0000 7fd140e8b640 1 --2- 172.21.15.62:0/1965692419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c102660 0x7fd13c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.145+0000 7fd140e8b640 1 -- 172.21.15.62:0/1965692419 >> 172.21.15.62:0/1965692419 conn(0x7fd13c0fc1e0 msgr2=0x7fd13c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.146+0000 7fd140e8b640 1 -- 172.21.15.62:0/1965692419 shutdown_connections 2024-09-11T23:52:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.146+0000 7fd140e8b640 1 -- 172.21.15.62:0/1965692419 wait complete. 2024-09-11T23:52:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.146+0000 7fd140e8b640 1 Processor -- start 2024-09-11T23:52:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.146+0000 7fd140e8b640 1 -- start start 2024-09-11T23:52:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd140e8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c102660 0x7fd13c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd140e8b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c104f90 0x7fd13c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd140e8b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd13c1078c0 0x7fd13c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd140e8b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c10d800 con 0x7fd13c104f90 2024-09-11T23:52:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd140e8b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c10d680 con 0x7fd13c102660 2024-09-11T23:52:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd140e8b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c10d980 con 0x7fd13c1078c0 2024-09-11T23:52:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd139d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c104f90 0x7fd13c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd13a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c102660 0x7fd13c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd13ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd13c1078c0 0x7fd13c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd13ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd13c1078c0 0x7fd13c19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33920/0 (socket says 172.21.15.62:33920) 2024-09-11T23:52:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd13a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c102660 0x7fd13c197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:45828/0 (socket says 172.21.15.62:45828) 2024-09-11T23:52:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd139d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c104f90 0x7fd13c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45696/0 (socket says 172.21.15.62:45696) 2024-09-11T23:52:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd13ad9d640 1 -- 172.21.15.62:0/656253829 learned_addr learned my addr 172.21.15.62:0/656253829 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd139d9b640 1 -- 172.21.15.62:0/656253829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd13c1078c0 msgr2=0x7fd13c19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd139d9b640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd13c1078c0 0x7fd13c19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd139d9b640 1 -- 172.21.15.62:0/656253829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c102660 msgr2=0x7fd13c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd139d9b640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c102660 0x7fd13c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd139d9b640 1 -- 172.21.15.62:0/656253829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c19fa20 con 0x7fd13c104f90 2024-09-11T23:52:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd139d9b640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c104f90 0x7fd13c198340 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd12400e9c0 tx=0x7fd12400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd1337fe640 1 -- 172.21.15.62:0/656253829 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12400ccc0 con 0x7fd13c104f90 2024-09-11T23:52:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd1337fe640 1 -- 172.21.15.62:0/656253829 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd124005020 con 0x7fd13c104f90 2024-09-11T23:52:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd140e8b640 1 -- 172.21.15.62:0/656253829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c19fd10 con 0x7fd13c104f90 2024-09-11T23:52:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd1337fe640 1 -- 172.21.15.62:0/656253829 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124004a00 con 0x7fd13c104f90 2024-09-11T23:52:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.147+0000 7fd140e8b640 1 -- 172.21.15.62:0/656253829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c1a01b0 con 0x7fd13c104f90 2024-09-11T23:52:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.149+0000 7fd140e8b640 1 -- 172.21.15.62:0/656253829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13c0fdcd0 con 0x7fd13c104f90 2024-09-11T23:52:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.150+0000 7fd1337fe640 1 -- 172.21.15.62:0/656253829 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fd1240051d0 con 0x7fd13c104f90 2024-09-11T23:52:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.151+0000 7fd1337fe640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1280a2c80 0x7fd1280a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.151+0000 7fd13a59c640 1 -- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1280a2c80 msgr2=0x7fd1280a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3106394779 2024-09-11T23:52:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.151+0000 7fd1337fe640 1 -- 172.21.15.62:0/656253829 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd1240cefc0 con 0x7fd13c104f90 2024-09-11T23:52:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.151+0000 7fd13a59c640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1280a2c80 0x7fd1280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:52:04.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.155+0000 7fd1337fe640 1 -- 172.21.15.62:0/656253829 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd12409bf30 con 0x7fd13c104f90 2024-09-11T23:52:04.211 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:03.944+0000 7f55f9850040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:52:04.211 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:04 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:04.010+0000 7f55f9850040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:52:04.211 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:04 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:04.093+0000 7f55f9850040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:52:04.211 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:04 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:04.152+0000 7f55f9850040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:52:04.211 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:04 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:04.211+0000 7f55f9850040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:52:04.232 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:03.964+0000 7f73f5c15040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:52:04.232 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:04 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:04.031+0000 7f73f5c15040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:52:04.232 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:04 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:04.114+0000 7f73f5c15040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:52:04.232 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:04 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:04.172+0000 7f73f5c15040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:52:04.232 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:04 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:04.231+0000 7f73f5c15040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:52:04.280 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:03.974+0000 7f7815a67040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:52:04.280 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:04.038+0000 7f7815a67040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:52:04.280 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:04.210+0000 7f7815a67040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:52:04.280 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:04.279+0000 7f7815a67040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:52:04.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.352+0000 7fd13a59c640 1 -- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1280a2c80 msgr2=0x7fd1280a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3106394779 2024-09-11T23:52:04.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.352+0000 7fd13a59c640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1280a2c80 0x7fd1280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:52:04.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.370+0000 7fd140e8b640 1 -- 172.21.15.62:0/656253829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fd13c062e90 con 0x7fd13c104f90 2024-09-11T23:52:04.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.422+0000 7fd1337fe640 1 -- 172.21.15.62:0/656253829 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7fd12409b830 con 0x7fd13c104f90 2024-09-11T23:52:04.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 -- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1280a2c80 msgr2=0x7fd1280a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:52:04.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1280a2c80 0x7fd1280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 -- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c104f90 msgr2=0x7fd13c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c104f90 0x7fd13c198340 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd12400e9c0 tx=0x7fd12400ee80 comp rx=0 tx=0).stop 2024-09-11T23:52:04.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 -- 172.21.15.62:0/656253829 shutdown_connections 2024-09-11T23:52:04.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fd1280a2c80 0x7fd1280a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd13c1078c0 0x7fd13c19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c104f90 0x7fd13c198340 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 --2- 172.21.15.62:0/656253829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c102660 0x7fd13c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 -- 172.21.15.62:0/656253829 >> 172.21.15.62:0/656253829 conn(0x7fd13c0fc1e0 msgr2=0x7fd13c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:04.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 -- 172.21.15.62:0/656253829 shutdown_connections 2024-09-11T23:52:04.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.424+0000 7fd140e8b640 1 -- 172.21.15.62:0/656253829 wait complete. 2024-09-11T23:52:04.436 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T23:52:04.451 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-11T23:52:04.494 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T23:52:04.567 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:04 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/656253829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T23:52:04.567 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:04.350+0000 7f7815a67040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:52:04.567 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:04.440+0000 7f7815a67040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:52:04.567 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:04.502+0000 7f7815a67040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:52:04.567 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:04.567+0000 7f7815a67040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:52:04.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 -- 172.21.15.62:0/3118973177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8107810 msgr2=0x7f27c8109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 --2- 172.21.15.62:0/3118973177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8107810 0x7f27c8109c80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009ca0 tx=0x7f27c002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 -- 172.21.15.62:0/3118973177 shutdown_connections 2024-09-11T23:52:04.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 --2- 172.21.15.62:0/3118973177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8107810 0x7f27c8109c80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 --2- 172.21.15.62:0/3118973177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c8104ee0 0x7f27c81072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 --2- 172.21.15.62:0/3118973177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c81025b0 0x7f27c81049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 -- 172.21.15.62:0/3118973177 >> 172.21.15.62:0/3118973177 conn(0x7f27c80fc1b0 msgr2=0x7f27c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:04.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 -- 172.21.15.62:0/3118973177 shutdown_connections 2024-09-11T23:52:04.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 -- 172.21.15.62:0/3118973177 wait complete. 2024-09-11T23:52:04.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 Processor -- start 2024-09-11T23:52:04.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 -- start start 2024-09-11T23:52:04.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c81025b0 0x7f27c8197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.601+0000 7f27cd4f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8104ee0 0x7f27c8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27cd4f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8107810 0x7f27c819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27cd4f8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27c810d7f0 con 0x7f27c8104ee0 2024-09-11T23:52:04.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27cd4f8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27c810d670 con 0x7f27c8107810 2024-09-11T23:52:04.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27cd4f8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27c810d970 con 0x7f27c81025b0 2024-09-11T23:52:04.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27bffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8104ee0 0x7f27c8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:04.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27bffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8104ee0 0x7f27c8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45704/0 (socket says 172.21.15.62:45704) 2024-09-11T23:52:04.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27bffff640 1 -- 172.21.15.62:0/76266054 learned_addr learned my addr 172.21.15.62:0/76266054 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:04.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27c6ffd640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c81025b0 0x7f27c8197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27c77fe640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8107810 0x7f27c819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27bffff640 1 -- 172.21.15.62:0/76266054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c81025b0 msgr2=0x7f27c8197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27bffff640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c81025b0 0x7f27c8197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27bffff640 1 -- 172.21.15.62:0/76266054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8107810 msgr2=0x7f27c819f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27bffff640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8107810 0x7f27c819f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27bffff640 1 -- 172.21.15.62:0/76266054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c819fa70 con 0x7f27c8104ee0 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27c6ffd640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c81025b0 0x7f27c8197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27bffff640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8104ee0 0x7f27c8198390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f27b000e9f0 tx=0x7f27b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27c4ff9640 1 -- 172.21.15.62:0/76266054 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b000ccc0 con 0x7f27c8104ee0 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27c4ff9640 1 -- 172.21.15.62:0/76266054 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b0005500 con 0x7f27c8104ee0 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27cd4f8640 1 -- 172.21.15.62:0/76266054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c819fd60 con 0x7f27c8104ee0 2024-09-11T23:52:04.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.602+0000 7f27c4ff9640 1 -- 172.21.15.62:0/76266054 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0010690 con 0x7f27c8104ee0 2024-09-11T23:52:04.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.603+0000 7f27cd4f8640 1 -- 172.21.15.62:0/76266054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c81a0200 con 0x7f27c8104ee0 2024-09-11T23:52:04.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.603+0000 7f27cd4f8640 1 -- 172.21.15.62:0/76266054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c80fdca0 con 0x7f27c8104ee0 2024-09-11T23:52:04.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.606+0000 7f27c4ff9640 1 -- 172.21.15.62:0/76266054 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f27b00046c0 con 0x7f27c8104ee0 2024-09-11T23:52:04.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.607+0000 7f27c4ff9640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f27b40a2be0 0x7f27b40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.607+0000 7f27c6ffd640 1 -- 172.21.15.62:0/76266054 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f27b40a2be0 msgr2=0x7f27b40a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3106394779 2024-09-11T23:52:04.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.607+0000 7f27c4ff9640 1 -- 172.21.15.62:0/76266054 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f27b00cf400 con 0x7f27c8104ee0 2024-09-11T23:52:04.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.607+0000 7f27c6ffd640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f27b40a2be0 0x7f27b40a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:52:04.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.611+0000 7f27c4ff9640 1 -- 172.21.15.62:0/76266054 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27b009c370 con 0x7f27c8104ee0 2024-09-11T23:52:04.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:04 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/656253829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T23:52:04.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:04 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/656253829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T23:52:04.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.789+0000 7f27cd4f8640 1 -- 172.21.15.62:0/76266054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27c8062e90 con 0x7f27c8104ee0 2024-09-11T23:52:04.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.791+0000 7f27c4ff9640 1 -- 172.21.15.62:0/76266054 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22620 (secure 0 0 0) 0x7f27b0086070 con 0x7f27c8104ee0 2024-09-11T23:52:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:04.791 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":83,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:51:14.101286+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:52:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 -- 172.21.15.62:0/76266054 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f27b40a2be0 msgr2=0x7f27b40a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:52:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f27b40a2be0 0x7f27b40a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 -- 172.21.15.62:0/76266054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8104ee0 msgr2=0x7f27c8198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8104ee0 0x7f27c8198390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f27b000e9f0 tx=0x7f27b000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 -- 172.21.15.62:0/76266054 shutdown_connections 2024-09-11T23:52:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f27b40a2be0 0x7f27b40a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8107810 0x7f27c819f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8104ee0 0x7f27c8198390 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 --2- 172.21.15.62:0/76266054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c81025b0 0x7f27c8197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 -- 172.21.15.62:0/76266054 >> 172.21.15.62:0/76266054 conn(0x7f27c80fc1b0 msgr2=0x7f27c8102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 -- 172.21.15.62:0/76266054 shutdown_connections 2024-09-11T23:52:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.793+0000 7f27cd4f8640 1 -- 172.21.15.62:0/76266054 wait complete. 2024-09-11T23:52:04.805 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:45:29.220992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T23:50:07.698272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T23:50:09.813785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:52:04.805 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T23:52:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.913+0000 7fc774a7b640 1 -- 172.21.15.62:0/3839786350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc770107860 msgr2=0x7fc770109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.913+0000 7fc774a7b640 1 --2- 172.21.15.62:0/3839786350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc770107860 0x7fc770109c50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc768009ca0 tx=0x7fc76802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.913+0000 7fc774a7b640 1 -- 172.21.15.62:0/3839786350 shutdown_connections 2024-09-11T23:52:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.913+0000 7fc774a7b640 1 --2- 172.21.15.62:0/3839786350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc770107860 0x7fc770109c50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.913+0000 7fc774a7b640 1 --2- 172.21.15.62:0/3839786350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc770104f30 0x7fc770107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.913+0000 7fc774a7b640 1 --2- 172.21.15.62:0/3839786350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc770102600 0x7fc7701049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.913+0000 7fc774a7b640 1 -- 172.21.15.62:0/3839786350 >> 172.21.15.62:0/3839786350 conn(0x7fc7700fc140 msgr2=0x7fc7700fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.913+0000 7fc774a7b640 1 -- 172.21.15.62:0/3839786350 shutdown_connections 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.913+0000 7fc774a7b640 1 -- 172.21.15.62:0/3839786350 wait complete. 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc774a7b640 1 Processor -- start 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc774a7b640 1 -- start start 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc774a7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc770102600 0x7fc770101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc774a7b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc770104f30 0x7fc7701005b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc774a7b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc770107860 0x7fc770100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc774a7b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc77010d810 con 0x7fc770107860 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc774a7b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc77010d690 con 0x7fc770104f30 2024-09-11T23:52:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc774a7b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc77010d990 con 0x7fc770102600 2024-09-11T23:52:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc76ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc770107860 0x7fc770100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc76ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc770107860 0x7fc770100af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45728/0 (socket says 172.21.15.62:45728) 2024-09-11T23:52:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc767fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc770104f30 0x7fc7701005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc76ed9d640 1 -- 172.21.15.62:0/229776556 learned_addr learned my addr 172.21.15.62:0/229776556 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc76e59c640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc770102600 0x7fc770101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc76ed9d640 1 -- 172.21.15.62:0/229776556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc770102600 msgr2=0x7fc770101f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc76ed9d640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc770102600 0x7fc770101f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc76ed9d640 1 -- 172.21.15.62:0/229776556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc770104f30 msgr2=0x7fc7701005b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc76ed9d640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc770104f30 0x7fc7701005b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc76ed9d640 1 -- 172.21.15.62:0/229776556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc770101380 con 0x7fc770107860 2024-09-11T23:52:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.914+0000 7fc767fff640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc770104f30 0x7fc7701005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.915+0000 7fc76ed9d640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc770107860 0x7fc770100af0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc768031ae0 tx=0x7fc768031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.915+0000 7fc7677fe640 1 -- 172.21.15.62:0/229776556 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768003b70 con 0x7fc770107860 2024-09-11T23:52:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.915+0000 7fc7677fe640 1 -- 172.21.15.62:0/229776556 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc768003d10 con 0x7fc770107860 2024-09-11T23:52:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.915+0000 7fc7677fe640 1 -- 172.21.15.62:0/229776556 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76802fb70 con 0x7fc770107860 2024-09-11T23:52:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.915+0000 7fc774a7b640 1 -- 172.21.15.62:0/229776556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7701a1fc0 con 0x7fc770107860 2024-09-11T23:52:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.915+0000 7fc774a7b640 1 -- 172.21.15.62:0/229776556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7701a2420 con 0x7fc770107860 2024-09-11T23:52:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.916+0000 7fc774a7b640 1 -- 172.21.15.62:0/229776556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7700fdc30 con 0x7fc770107860 2024-09-11T23:52:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.917+0000 7fc7677fe640 1 -- 172.21.15.62:0/229776556 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fc76802fd10 con 0x7fc770107860 2024-09-11T23:52:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.918+0000 7fc7677fe640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fc75c0a2be0 0x7fc75c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.918+0000 7fc76e59c640 1 -- 172.21.15.62:0/229776556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fc75c0a2be0 msgr2=0x7fc75c0a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3106394779 2024-09-11T23:52:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.918+0000 7fc7677fe640 1 -- 172.21.15.62:0/229776556 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc7680f0400 con 0x7fc770107860 2024-09-11T23:52:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.918+0000 7fc76e59c640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fc75c0a2be0 0x7fc75c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:52:04.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:04.922+0000 7fc7677fe640 1 -- 172.21.15.62:0/229776556 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7680bd370 con 0x7fc770107860 2024-09-11T23:52:05.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.100+0000 7fc774a7b640 1 -- 172.21.15.62:0/229776556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc770062e90 con 0x7fc770107860 2024-09-11T23:52:05.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.100+0000 7fc7677fe640 1 -- 172.21.15.62:0/229776556 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fc7680a7070 con 0x7fc770107860 2024-09-11T23:52:05.100 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-09-11T23:52:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 -- 172.21.15.62:0/229776556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fc75c0a2be0 msgr2=0x7fc75c0a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:52:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fc75c0a2be0 0x7fc75c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 -- 172.21.15.62:0/229776556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc770107860 msgr2=0x7fc770100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc770107860 0x7fc770100af0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc768031ae0 tx=0x7fc768031750 comp rx=0 tx=0).stop 2024-09-11T23:52:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 -- 172.21.15.62:0/229776556 shutdown_connections 2024-09-11T23:52:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7fc75c0a2be0 0x7fc75c0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc770107860 0x7fc770100af0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc770104f30 0x7fc7701005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 --2- 172.21.15.62:0/229776556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc770102600 0x7fc770101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 -- 172.21.15.62:0/229776556 >> 172.21.15.62:0/229776556 conn(0x7fc7700fc140 msgr2=0x7fc770102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 -- 172.21.15.62:0/229776556 shutdown_connections 2024-09-11T23:52:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.102+0000 7fc774a7b640 1 -- 172.21.15.62:0/229776556 wait complete. 2024-09-11T23:52:05.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T23:52:05.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.222+0000 7f6d91742640 1 -- 172.21.15.62:0/1198584756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8c102e60 msgr2=0x7f6d8c1032e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.222+0000 7f6d91742640 1 --2- 172.21.15.62:0/1198584756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8c102e60 0x7f6d8c1032e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009900 tx=0x7f6d7402a410 comp rx=0 tx=0).stop 2024-09-11T23:52:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.222+0000 7f6d91742640 1 -- 172.21.15.62:0/1198584756 shutdown_connections 2024-09-11T23:52:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.222+0000 7f6d91742640 1 --2- 172.21.15.62:0/1198584756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c103820 0x7f6d8c10a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.222+0000 7f6d91742640 1 --2- 172.21.15.62:0/1198584756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8c102e60 0x7f6d8c1032e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.222+0000 7f6d91742640 1 --2- 172.21.15.62:0/1198584756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d8c0ffdf0 0x7f6d8c1001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.222+0000 7f6d91742640 1 -- 172.21.15.62:0/1198584756 >> 172.21.15.62:0/1198584756 conn(0x7f6d8c0fb910 msgr2=0x7f6d8c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.222+0000 7f6d91742640 1 -- 172.21.15.62:0/1198584756 shutdown_connections 2024-09-11T23:52:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.222+0000 7f6d91742640 1 -- 172.21.15.62:0/1198584756 wait complete. 2024-09-11T23:52:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d91742640 1 Processor -- start 2024-09-11T23:52:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d91742640 1 -- start start 2024-09-11T23:52:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d91742640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8c0ffdf0 0x7f6d8c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d91742640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d8c102e60 0x7f6d8c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d91742640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c103820 0x7f6d8c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d91742640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10d7f0 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d91742640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10d670 con 0x7f6d8c102e60 2024-09-11T23:52:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d91742640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10d970 con 0x7f6d8c103820 2024-09-11T23:52:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d8affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8c0ffdf0 0x7f6d8c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d8affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8c0ffdf0 0x7f6d8c197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45752/0 (socket says 172.21.15.62:45752) 2024-09-11T23:52:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d8affd640 1 -- 172.21.15.62:0/1558411016 learned_addr learned my addr 172.21.15.62:0/1558411016 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d8b7fe640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c103820 0x7f6d8c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d83fff640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d8c102e60 0x7f6d8c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d8affd640 1 -- 172.21.15.62:0/1558411016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c103820 msgr2=0x7f6d8c19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d8affd640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c103820 0x7f6d8c19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d8affd640 1 -- 172.21.15.62:0/1558411016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d8c102e60 msgr2=0x7f6d8c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d8affd640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d8c102e60 0x7f6d8c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d8affd640 1 -- 172.21.15.62:0/1558411016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c19fa70 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.223+0000 7f6d83fff640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d8c102e60 0x7f6d8c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.224+0000 7f6d8affd640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8c0ffdf0 0x7f6d8c197e50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00e9c0 tx=0x7f6d7c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.224+0000 7f6d88ff9640 1 -- 172.21.15.62:0/1558411016 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00cd80 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.224+0000 7f6d88ff9640 1 -- 172.21.15.62:0/1558411016 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d7c0035f0 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.224+0000 7f6d91742640 1 -- 172.21.15.62:0/1558411016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c19fd60 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.224+0000 7f6d88ff9640 1 -- 172.21.15.62:0/1558411016 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c010690 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.224+0000 7f6d91742640 1 -- 172.21.15.62:0/1558411016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c1a01d0 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.225+0000 7f6d91742640 1 -- 172.21.15.62:0/1558411016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c1090b0 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.226+0000 7f6d88ff9640 1 -- 172.21.15.62:0/1558411016 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f6d7c0037a0 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.227+0000 7f6d88ff9640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6d780a2be0 0x7f6d780a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:05.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.227+0000 7f6d83fff640 1 -- 172.21.15.62:0/1558411016 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6d780a2be0 msgr2=0x7f6d780a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3106394779 2024-09-11T23:52:05.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.227+0000 7f6d88ff9640 1 -- 172.21.15.62:0/1558411016 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6d7c0cf5d0 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.227+0000 7f6d83fff640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6d780a2be0 0x7f6d780a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:52:05.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.231+0000 7f6d88ff9640 1 -- 172.21.15.62:0/1558411016 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d7c09c540 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.371 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z[36984]: 2024-09-11T23:52:05.020+0000 7f73f5c15040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:52:05.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.409+0000 7f6d91742640 1 -- 172.21.15.62:0/1558411016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6d8c062e90 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.409+0000 7f6d88ff9640 1 -- 172.21.15.62:0/1558411016 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f6d7c09be40 con 0x7f6d8c0ffdf0 2024-09-11T23:52:05.409 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-09-11T23:52:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 -- 172.21.15.62:0/1558411016 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6d780a2be0 msgr2=0x7f6d780a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:52:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6d780a2be0 0x7f6d780a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 -- 172.21.15.62:0/1558411016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8c0ffdf0 msgr2=0x7f6d8c197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8c0ffdf0 0x7f6d8c197e50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00e9c0 tx=0x7f6d7c00ee80 comp rx=0 tx=0).stop 2024-09-11T23:52:05.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 -- 172.21.15.62:0/1558411016 shutdown_connections 2024-09-11T23:52:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.62:6800/3106394779,v1:172.21.15.62:6801/3106394779] conn(0x7f6d780a2be0 0x7f6d780a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d8c103820 0x7f6d8c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d8c102e60 0x7f6d8c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 --2- 172.21.15.62:0/1558411016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d8c0ffdf0 0x7f6d8c197e50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 -- 172.21.15.62:0/1558411016 >> 172.21.15.62:0/1558411016 conn(0x7f6d8c0fb910 msgr2=0x7f6d8c106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 -- 172.21.15.62:0/1558411016 shutdown_connections 2024-09-11T23:52:05.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.411+0000 7f6d91742640 1 -- 172.21.15.62:0/1558411016 wait complete. 2024-09-11T23:52:05.417 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y[36171]: 2024-09-11T23:52:04.999+0000 7f55f9850040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:52:05.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T23:52:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.555+0000 7fb4d27fc640 1 -- 172.21.15.62:0/2725793282 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc030e30 con 0x7fb4d41025b0 2024-09-11T23:52:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.555+0000 7fb4da1bd640 1 -- 172.21.15.62:0/2725793282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d41025b0 msgr2=0x7fb4d4108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.555+0000 7fb4da1bd640 1 --2- 172.21.15.62:0/2725793282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d41025b0 0x7fb4d4108e40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009be0 tx=0x7fb4cc02a6c0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.557+0000 7fb4da1bd640 1 -- 172.21.15.62:0/2725793282 shutdown_connections 2024-09-11T23:52:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.557+0000 7fb4da1bd640 1 --2- 172.21.15.62:0/2725793282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d41025b0 0x7fb4d4108e40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.557+0000 7fb4da1bd640 1 --2- 172.21.15.62:0/2725793282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4101bf0 0x7fb4d4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.557+0000 7fb4da1bd640 1 --2- 172.21.15.62:0/2725793282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d4100a00 0x7fb4d4100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.557+0000 7fb4da1bd640 1 -- 172.21.15.62:0/2725793282 >> 172.21.15.62:0/2725793282 conn(0x7fb4d40fc140 msgr2=0x7fb4d40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.557+0000 7fb4da1bd640 1 -- 172.21.15.62:0/2725793282 shutdown_connections 2024-09-11T23:52:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.557+0000 7fb4da1bd640 1 -- 172.21.15.62:0/2725793282 wait complete. 2024-09-11T23:52:05.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.558+0000 7fb4da1bd640 1 Processor -- start 2024-09-11T23:52:05.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.558+0000 7fb4da1bd640 1 -- start start 2024-09-11T23:52:05.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4da1bd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4100a00 0x7fb4d406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:05.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4da1bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4101bf0 0x7fb4d406e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:05.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4da1bd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d41025b0 0x7fb4d406e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:05.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4da1bd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d410c6b0 con 0x7fb4d41025b0 2024-09-11T23:52:05.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4da1bd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d410c530 con 0x7fb4d4100a00 2024-09-11T23:52:05.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4da1bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d410c830 con 0x7fb4d4101bf0 2024-09-11T23:52:05.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d41025b0 0x7fb4d406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:05.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d41025b0 0x7fb4d406e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45832/0 (socket says 172.21.15.62:45832) 2024-09-11T23:52:05.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4d3fff640 1 -- 172.21.15.62:0/1703877550 learned_addr learned my addr 172.21.15.62:0/1703877550 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:05.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4d2ffd640 1 --2- 172.21.15.62:0/1703877550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4101bf0 0x7fb4d406e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:05.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4d2ffd640 1 -- 172.21.15.62:0/1703877550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4100a00 msgr2=0x7fb4d406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:05.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4d2ffd640 1 --2- 172.21.15.62:0/1703877550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4100a00 0x7fb4d406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4d2ffd640 1 -- 172.21.15.62:0/1703877550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d41025b0 msgr2=0x7fb4d406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:05.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4d2ffd640 1 --2- 172.21.15.62:0/1703877550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d41025b0 0x7fb4d406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.559+0000 7fb4d2ffd640 1 -- 172.21.15.62:0/1703877550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d406f230 con 0x7fb4d4101bf0 2024-09-11T23:52:05.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.560+0000 7fb4d2ffd640 1 --2- 172.21.15.62:0/1703877550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4101bf0 0x7fb4d406e310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8005fd0 tx=0x7fb4c800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:05.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.560+0000 7fb4d0ff9640 1 -- 172.21.15.62:0/1703877550 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8009c30 con 0x7fb4d4101bf0 2024-09-11T23:52:05.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.560+0000 7fb4da1bd640 1 -- 172.21.15.62:0/1703877550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d406f520 con 0x7fb4d4101bf0 2024-09-11T23:52:05.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.560+0000 7fb4da1bd640 1 -- 172.21.15.62:0/1703877550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d406fac0 con 0x7fb4d4101bf0 2024-09-11T23:52:05.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.560+0000 7fb4d0ff9640 1 -- 172.21.15.62:0/1703877550 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c8009dd0 con 0x7fb4d4101bf0 2024-09-11T23:52:05.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.560+0000 7fb4d0ff9640 1 -- 172.21.15.62:0/1703877550 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8010460 con 0x7fb4d4101bf0 2024-09-11T23:52:05.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.561+0000 7fb4da1bd640 1 -- 172.21.15.62:0/1703877550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b8005290 con 0x7fb4d4101bf0 2024-09-11T23:52:05.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.563+0000 7fb4d0ff9640 1 -- 172.21.15.62:0/1703877550 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7fb4c8004030 con 0x7fb4d4101bf0 2024-09-11T23:52:05.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.564+0000 7fb4d0ff9640 1 -- 172.21.15.62:0/1703877550 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb4c80ce670 con 0x7fb4d4101bf0 2024-09-11T23:52:05.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.569+0000 7fb4d0ff9640 1 -- 172.21.15.62:0/1703877550 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4c8017020 con 0x7fb4d4101bf0 2024-09-11T23:52:05.705 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.423+0000 7f7815a67040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:52:05.707 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/656253829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T23:52:05.707 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:52:05.707 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/76266054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:52:05.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: Standby manager daemon y restarted 2024-09-11T23:52:05.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: Standby manager daemon y started 2024-09-11T23:52:05.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:52:05.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:52:05.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:52:05.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:52:05.709 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: Standby manager daemon z restarted 2024-09-11T23:52:05.709 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: Standby manager daemon z started 2024-09-11T23:52:05.709 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:52:05.709 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:52:05.709 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:52:05.709 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:52:05.709 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/229776556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:52:05.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1558411016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:52:05.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: Active manager daemon x restarted 2024-09-11T23:52:05.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-mon[30958]: Activating manager daemon x 2024-09-11T23:52:05.790 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/656253829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T23:52:05.791 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:52:05.791 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/76266054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:52:05.791 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: Standby manager daemon y restarted 2024-09-11T23:52:05.791 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: Standby manager daemon y started 2024-09-11T23:52:05.791 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:52:05.791 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:52:05.791 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:52:05.791 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:52:05.791 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: Standby manager daemon z restarted 2024-09-11T23:52:05.791 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: Standby manager daemon z started 2024-09-11T23:52:05.792 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:52:05.792 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:52:05.792 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:52:05.792 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:52:05.792 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/229776556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:52:05.792 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1558411016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:52:05.792 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: Active manager daemon x restarted 2024-09-11T23:52:05.792 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:05 smithi120 ceph-mon[35182]: Activating manager daemon x 2024-09-11T23:52:05.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.814+0000 7fb4da1bd640 1 -- 172.21.15.62:0/1703877550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb4b8005d90 con 0x7fb4d4101bf0 2024-09-11T23:52:05.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.814+0000 7fb4d0ff9640 1 -- 172.21.15.62:0/1703877550 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7fb4c809b880 con 0x7fb4d4101bf0 2024-09-11T23:52:05.814 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-09-11T23:52:05.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.816+0000 7fb4da1bd640 1 -- 172.21.15.62:0/1703877550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4101bf0 msgr2=0x7fb4d406e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:05.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.816+0000 7fb4da1bd640 1 --2- 172.21.15.62:0/1703877550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4101bf0 0x7fb4d406e310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8005fd0 tx=0x7fb4c800dde0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.816+0000 7fb4da1bd640 1 -- 172.21.15.62:0/1703877550 shutdown_connections 2024-09-11T23:52:05.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.816+0000 7fb4da1bd640 1 --2- 172.21.15.62:0/1703877550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d41025b0 0x7fb4d406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.816+0000 7fb4da1bd640 1 --2- 172.21.15.62:0/1703877550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4101bf0 0x7fb4d406e310 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.816+0000 7fb4da1bd640 1 --2- 172.21.15.62:0/1703877550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4100a00 0x7fb4d406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:05.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.816+0000 7fb4da1bd640 1 -- 172.21.15.62:0/1703877550 >> 172.21.15.62:0/1703877550 conn(0x7fb4d40fc140 msgr2=0x7fb4d40fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:05.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.816+0000 7fb4da1bd640 1 -- 172.21.15.62:0/1703877550 shutdown_connections 2024-09-11T23:52:05.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:05.816+0000 7fb4da1bd640 1 -- 172.21.15.62:0/1703877550 wait complete. 2024-09-11T23:52:05.827 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-11T23:52:05.827 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:52:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/656253829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T23:52:05.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:52:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/76266054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:52:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: Standby manager daemon y restarted 2024-09-11T23:52:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: Standby manager daemon y started 2024-09-11T23:52:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:52:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:52:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:52:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.83:0/424176262' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:52:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: Standby manager daemon z restarted 2024-09-11T23:52:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: Standby manager daemon z started 2024-09-11T23:52:05.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:52:05.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:52:05.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:52:05.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='mgr.? 172.21.15.120:0/3062779499' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:52:05.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/229776556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:52:05.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1558411016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:52:05.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: Active manager daemon x restarted 2024-09-11T23:52:05.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:05 smithi083 ceph-mon[34805]: Activating manager daemon x 2024-09-11T23:52:05.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.704+0000 7f777d2f9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:52:05.965 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.704+0000 7f777d2f9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:52:05.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.704+0000 7f777d2f9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:52:05.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.704+0000 7f777d2f9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:52:05.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.704+0000 7f777d2f9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:52:05.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.738+0000 7f777fb7e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:52:05.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.738+0000 7f777fb7e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:52:05.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.738+0000 7f777fb7e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:52:05.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.738+0000 7f777fb7e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:52:05.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:05.738+0000 7f777fb7e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:52:06.110 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.622+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1231308685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e8102ef0 msgr2=0x7fe4e810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.622+0000 7fe4eec7d640 1 --2- 172.21.15.62:0/1231308685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e8102ef0 0x7fe4e810da40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009ca0 tx=0x7fe4e002a950 comp rx=0 tx=0).stop 2024-09-11T23:52:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.622+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1231308685 shutdown_connections 2024-09-11T23:52:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.622+0000 7fe4eec7d640 1 --2- 172.21.15.62:0/1231308685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e8102ef0 0x7fe4e810da40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.622+0000 7fe4eec7d640 1 --2- 172.21.15.62:0/1231308685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4e8102570 0x7fe4e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.622+0000 7fe4eec7d640 1 --2- 172.21.15.62:0/1231308685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e8100e20 0x7fe4e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.622+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1231308685 >> 172.21.15.62:0/1231308685 conn(0x7fe4e80fc7d0 msgr2=0x7fe4e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.622+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1231308685 shutdown_connections 2024-09-11T23:52:06.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.622+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1231308685 wait complete. 2024-09-11T23:52:06.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4eec7d640 1 Processor -- start 2024-09-11T23:52:06.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4eec7d640 1 -- start start 2024-09-11T23:52:06.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4eec7d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4e8100e20 0x7fe4e81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:06.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4eec7d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e8102570 0x7fe4e81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4eec7d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e8102ef0 0x7fe4e81a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4eec7d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8110f80 con 0x7fe4e8102ef0 2024-09-11T23:52:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4eec7d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8110e00 con 0x7fe4e8100e20 2024-09-11T23:52:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4eec7d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8111100 con 0x7fe4e8102570 2024-09-11T23:52:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4ed21a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e8102ef0 0x7fe4e81a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4ed21a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e8102ef0 0x7fe4e81a4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45908/0 (socket says 172.21.15.62:45908) 2024-09-11T23:52:06.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4ed21a640 1 -- 172.21.15.62:0/1922776025 learned_addr learned my addr 172.21.15.62:0/1922776025 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:06.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4eca19640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4e8100e20 0x7fe4e81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:06.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.623+0000 7fe4e7fff640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e8102570 0x7fe4e81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:06.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4eca19640 1 -- 172.21.15.62:0/1922776025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e8102570 msgr2=0x7fe4e81996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:06.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4eca19640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e8102570 0x7fe4e81996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:06.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4eca19640 1 -- 172.21.15.62:0/1922776025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e8102ef0 msgr2=0x7fe4e81a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:06.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4eca19640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e8102ef0 0x7fe4e81a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:06.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4eca19640 1 -- 172.21.15.62:0/1922776025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e81a4770 con 0x7fe4e8100e20 2024-09-11T23:52:06.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4ed21a640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e8102ef0 0x7fe4e81a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:06.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4eca19640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4e8100e20 0x7fe4e81991b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00c970 tx=0x7fe4dc00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:06.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4e5ffb640 1 -- 172.21.15.62:0/1922776025 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc007bb0 con 0x7fe4e8100e20 2024-09-11T23:52:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1922776025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e81a4940 con 0x7fe4e8100e20 2024-09-11T23:52:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4e5ffb640 1 -- 172.21.15.62:0/1922776025 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc007d50 con 0x7fe4e8100e20 2024-09-11T23:52:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4e5ffb640 1 -- 172.21.15.62:0/1922776025 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc018730 con 0x7fe4e8100e20 2024-09-11T23:52:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.624+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1922776025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e81a4e30 con 0x7fe4e8100e20 2024-09-11T23:52:06.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:52:06.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: mgrmap e20: x(active, starting, since 0.0135328s), standbys: z, y 2024-09-11T23:52:06.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:52:06.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:52:06.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:52:06.634 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:52:06.635 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:52:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:52:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:52:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:52:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:52:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:52:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:52:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: Manager daemon x is now available 2024-09-11T23:52:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:52:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:52:06.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:06 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1703877550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:52:06.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.626+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1922776025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e8101200 con 0x7fe4e8100e20 2024-09-11T23:52:06.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.627+0000 7fe4e5ffb640 1 -- 172.21.15.62:0/1922776025 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fe4dc00f450 con 0x7fe4e8100e20 2024-09-11T23:52:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.627+0000 7fe4e5ffb640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4d80a2970 0x7fe4d80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.627+0000 7fe4e7fff640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4d80a2970 0x7fe4d80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.627+0000 7fe4e5ffb640 1 -- 172.21.15.62:0/1922776025 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe4dc0ced30 con 0x7fe4e8100e20 2024-09-11T23:52:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.628+0000 7fe4e7fff640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4d80a2970 0x7fe4d80a4e30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4e806aac0 tx=0x7fe4d400a2d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.632+0000 7fe4e5ffb640 1 -- 172.21.15.62:0/1922776025 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4dc014030 con 0x7fe4e8100e20 2024-09-11T23:52:06.788 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:52:06.788 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: mgrmap e20: x(active, starting, since 0.0135328s), standbys: z, y 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:52:06.789 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:52:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: Manager daemon x is now available 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:52:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:52:06.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:06 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1703877550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:52:06.836 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:52:06.836 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: mgrmap e20: x(active, starting, since 0.0135328s), standbys: z, y 2024-09-11T23:52:06.836 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:52:06.836 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:52:06.836 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:52:06.836 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:52:06.836 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:52:06.836 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:52:06.837 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:52:06.837 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:52:06.837 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:52:06.837 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:52:06.837 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:52:06.837 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:52:06.837 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:52:06.837 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:52:06.837 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:52:06.838 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: Manager daemon x is now available 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:52:06.839 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:06 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1703877550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:52:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.858+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1922776025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4e81a50c0 con 0x7fe4e8100e20 2024-09-11T23:52:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.859+0000 7fe4e5ffb640 1 -- 172.21.15.62:0/1922776025 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7fe4dc09be50 con 0x7fe4e8100e20 2024-09-11T23:52:06.859 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:06.859 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"btime":"2024-09-11T23:52:04:422374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:04.372188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:52:06.859 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1922776025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4d80a2970 msgr2=0x7fe4d80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4d80a2970 0x7fe4d80a4e30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4e806aac0 tx=0x7fe4d400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1922776025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4e8100e20 msgr2=0x7fe4e81991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4e8100e20 0x7fe4e81991b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00c970 tx=0x7fe4dc00cd70 comp rx=0 tx=0).stop 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1922776025 shutdown_connections 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4d80a2970 0x7fe4d80a4e30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e8102ef0 0x7fe4e81a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e8102570 0x7fe4e81996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 --2- 172.21.15.62:0/1922776025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4e8100e20 0x7fe4e81991b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1922776025 >> 172.21.15.62:0/1922776025 conn(0x7fe4e80fc7d0 msgr2=0x7fe4e8105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1922776025 shutdown_connections 2024-09-11T23:52:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:06.861+0000 7fe4eec7d640 1 -- 172.21.15.62:0/1922776025 wait complete. 2024-09-11T23:52:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-mon[30958]: mgrmap e21: x(active, since 1.03305s), standbys: z, y 2024-09-11T23:52:07.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1922776025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:07.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-mon[30958]: [11/Sep/2024:23:52:07] ENGINE Bus STARTING 2024-09-11T23:52:07.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-mon[30958]: [11/Sep/2024:23:52:07] ENGINE Serving on http://172.21.15.62:8765 2024-09-11T23:52:07.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:07.335+0000 7f7763a87640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:52:07] ENGINE Error in HTTPServer.serve 2024-09-11T23:52:07.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: Traceback (most recent call last): 2024-09-11T23:52:07.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:52:07.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self._connections.run(self.expiration_interval) 2024-09-11T23:52:07.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:52:07.716 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self._run(expiration_interval) 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: s = self.context.wrap_socket( 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: return self.sslsocket_class._create( 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self.do_handshake() 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:52:07.717 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: self._sslobj.do_handshake() 2024-09-11T23:52:07.718 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:52:07.718 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:52:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:52:07.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:07 smithi120 ceph-mon[35182]: mgrmap e21: x(active, since 1.03305s), standbys: z, y 2024-09-11T23:52:07.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:07 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1922776025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:07.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:07 smithi120 ceph-mon[35182]: [11/Sep/2024:23:52:07] ENGINE Bus STARTING 2024-09-11T23:52:07.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:07 smithi120 ceph-mon[35182]: [11/Sep/2024:23:52:07] ENGINE Serving on http://172.21.15.62:8765 2024-09-11T23:52:07.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:07 smithi083 ceph-mon[34805]: mgrmap e21: x(active, since 1.03305s), standbys: z, y 2024-09-11T23:52:07.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:07 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1922776025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:07.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:07 smithi083 ceph-mon[34805]: [11/Sep/2024:23:52:07] ENGINE Bus STARTING 2024-09-11T23:52:07.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:07 smithi083 ceph-mon[34805]: [11/Sep/2024:23:52:07] ENGINE Serving on http://172.21.15.62:8765 2024-09-11T23:52:08.350 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-11T23:52:08.351 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-11T23:52:08.351 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:52:08.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.474+0000 7f61315cd640 1 -- 172.21.15.62:0/2271962116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c104f20 msgr2=0x7f612c107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:08.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.474+0000 7f61315cd640 1 --2- 172.21.15.62:0/2271962116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c104f20 0x7f612c107310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6120009900 tx=0x7f612002a3f0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.475+0000 7f61315cd640 1 -- 172.21.15.62:0/2271962116 shutdown_connections 2024-09-11T23:52:08.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.475+0000 7f61315cd640 1 --2- 172.21.15.62:0/2271962116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f612c107850 0x7f612c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.475+0000 7f61315cd640 1 --2- 172.21.15.62:0/2271962116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c104f20 0x7f612c107310 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.475+0000 7f61315cd640 1 --2- 172.21.15.62:0/2271962116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f612c1025f0 0x7f612c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.475+0000 7f61315cd640 1 -- 172.21.15.62:0/2271962116 >> 172.21.15.62:0/2271962116 conn(0x7f612c0fc170 msgr2=0x7f612c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:08.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.475+0000 7f61315cd640 1 -- 172.21.15.62:0/2271962116 shutdown_connections 2024-09-11T23:52:08.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.475+0000 7f61315cd640 1 -- 172.21.15.62:0/2271962116 wait complete. 2024-09-11T23:52:08.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.475+0000 7f61315cd640 1 Processor -- start 2024-09-11T23:52:08.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f61315cd640 1 -- start start 2024-09-11T23:52:08.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f61315cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f612c1025f0 0x7f612c101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f61315cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c104f20 0x7f612c1005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f61315cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f612c107850 0x7f612c100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f61315cd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f612c10d7c0 con 0x7f612c104f20 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f61315cd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f612c10d640 con 0x7f612c1025f0 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f61315cd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f612c10d940 con 0x7f612c107850 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f612a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c104f20 0x7f612c1005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f612a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c104f20 0x7f612c1005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45940/0 (socket says 172.21.15.62:45940) 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f612a7fc640 1 -- 172.21.15.62:0/2927945596 learned_addr learned my addr 172.21.15.62:0/2927945596 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f612affd640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f612c1025f0 0x7f612c101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f612b7fe640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f612c107850 0x7f612c100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.476+0000 7f612a7fc640 1 -- 172.21.15.62:0/2927945596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f612c107850 msgr2=0x7f612c100ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f612a7fc640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f612c107850 0x7f612c100ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f612a7fc640 1 -- 172.21.15.62:0/2927945596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f612c1025f0 msgr2=0x7f612c101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f612a7fc640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f612c1025f0 0x7f612c101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f612a7fc640 1 -- 172.21.15.62:0/2927945596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c101370 con 0x7f612c104f20 2024-09-11T23:52:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f612affd640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f612c1025f0 0x7f612c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f612a7fc640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c104f20 0x7f612c1005a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6120031610 tx=0x7f6120004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f6113fff640 1 -- 172.21.15.62:0/2927945596 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612003f030 con 0x7f612c104f20 2024-09-11T23:52:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f6113fff640 1 -- 172.21.15.62:0/2927945596 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6120003ad0 con 0x7f612c104f20 2024-09-11T23:52:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f61315cd640 1 -- 172.21.15.62:0/2927945596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c1a1f70 con 0x7f612c104f20 2024-09-11T23:52:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f6113fff640 1 -- 172.21.15.62:0/2927945596 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120035070 con 0x7f612c104f20 2024-09-11T23:52:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.477+0000 7f61315cd640 1 -- 172.21.15.62:0/2927945596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c1a23d0 con 0x7f612c104f20 2024-09-11T23:52:08.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.478+0000 7f61315cd640 1 -- 172.21.15.62:0/2927945596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612c0fdc60 con 0x7f612c104f20 2024-09-11T23:52:08.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.480+0000 7f6113fff640 1 -- 172.21.15.62:0/2927945596 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f6120042030 con 0x7f612c104f20 2024-09-11T23:52:08.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.481+0000 7f6113fff640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f61140a2d00 0x7f61140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:08.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.481+0000 7f612affd640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f61140a2d00 0x7f61140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:08.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.481+0000 7f6113fff640 1 -- 172.21.15.62:0/2927945596 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f61200efc30 con 0x7f612c104f20 2024-09-11T23:52:08.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.482+0000 7f612affd640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f61140a2d00 0x7f61140a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f611c005d00 tx=0x7f611c00b410 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:08.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.484+0000 7f6113fff640 1 -- 172.21.15.62:0/2927945596 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61200bc230 con 0x7f612c104f20 2024-09-11T23:52:08.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.697+0000 7f61315cd640 1 -- 172.21.15.62:0/2927945596 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f612c062630 con 0x7f61140a2d00 2024-09-11T23:52:08.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.704+0000 7f6113fff640 1 -- 172.21.15.62:0/2927945596 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257965 (secure 0 0 0) 0x7f612c062630 con 0x7f61140a2d00 2024-09-11T23:52:08.707 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:08.708 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:52:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 -- 172.21.15.62:0/2927945596 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f61140a2d00 msgr2=0x7f61140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f61140a2d00 0x7f61140a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f611c005d00 tx=0x7f611c00b410 comp rx=0 tx=0).stop 2024-09-11T23:52:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 -- 172.21.15.62:0/2927945596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c104f20 msgr2=0x7f612c1005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c104f20 0x7f612c1005a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6120031610 tx=0x7f6120004270 comp rx=0 tx=0).stop 2024-09-11T23:52:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 -- 172.21.15.62:0/2927945596 shutdown_connections 2024-09-11T23:52:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f61140a2d00 0x7f61140a51c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f612c107850 0x7f612c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c104f20 0x7f612c1005a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 --2- 172.21.15.62:0/2927945596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f612c1025f0 0x7f612c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 -- 172.21.15.62:0/2927945596 >> 172.21.15.62:0/2927945596 conn(0x7f612c0fc170 msgr2=0x7f612c102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:08.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 -- 172.21.15.62:0/2927945596 shutdown_connections 2024-09-11T23:52:08.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.708+0000 7f61315cd640 1 -- 172.21.15.62:0/2927945596 wait complete. 2024-09-11T23:52:08.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: [11/Sep/2024:23:52:07] ENGINE Serving on https://172.21.15.62:7150 2024-09-11T23:52:08.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: [11/Sep/2024:23:52:07] ENGINE Bus STARTED 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: [11/Sep/2024:23:52:07] ENGINE Error in HTTPServer.serve 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._run(expiration_interval) 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:52:08.716 INFO:journalctl@ceph.mon.a.smithi062.stdout: s = self.context.wrap_socket( 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.sslsocket_class._create( 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout: self.do_handshake() 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._sslobj.do_handshake() 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: Unable to set osd_memory_target on smithi120 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:08 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.721 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T23:52:07.445934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187092,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":899,"num_read_kb":609,"num_write":478,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348104,"kb_used_data":11296,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593720,"statfs":{"total":1151940427776,"available":1151583969280,"internally_reserved":0,"allocated":11567104,"data_stored":8873556,"data_compressed":209571,"data_compressed_allocated":270336,"data_compressed_original":3391488,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.977168"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872692+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:52:05.872692+0000","last_peered":"2024-09-11T23:52:05.872692+0000","last_clean":"2024-09-11T23:52:05.872692+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:52:05.872692+0000","last_undegraded":"2024-09-11T23:52:05.872692+0000","last_fullsized":"2024-09-11T23:52:05.872692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158483+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:52:06.158483+0000","last_peered":"2024-09-11T23:52:06.158483+0000","last_clean":"2024-09-11T23:52:06.158483+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:52:06.158483+0000","last_undegraded":"2024-09-11T23:52:06.158483+0000","last_fullsized":"2024-09-11T23:52:06.158483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158255+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:52:06.158255+0000","last_peered":"2024-09-11T23:52:06.158255+0000","last_clean":"2024-09-11T23:52:06.158255+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:52:06.158255+0000","last_undegraded":"2024-09-11T23:52:06.158255+0000","last_fullsized":"2024-09-11T23:52:06.158255+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744925+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:52:05.744925+0000","last_peered":"2024-09-11T23:52:05.744925+0000","last_clean":"2024-09-11T23:52:05.744925+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:52:05.744925+0000","last_undegraded":"2024-09-11T23:52:05.744925+0000","last_fullsized":"2024-09-11T23:52:05.744925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516648+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:52:05.516648+0000","last_peered":"2024-09-11T23:52:05.516648+0000","last_clean":"2024-09-11T23:52:05.516648+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:52:05.516648+0000","last_undegraded":"2024-09-11T23:52:05.516648+0000","last_fullsized":"2024-09-11T23:52:05.516648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434378+0000","last_change":"2024-09-11T23:51:14.120226+0000","last_active":"2024-09-11T23:52:05.434378+0000","last_peered":"2024-09-11T23:52:05.434378+0000","last_clean":"2024-09-11T23:52:05.434378+0000","last_became_active":"2024-09-11T23:51:14.119811+0000","last_became_peered":"2024-09-11T23:51:14.119811+0000","last_unstale":"2024-09-11T23:52:05.434378+0000","last_undegraded":"2024-09-11T23:52:05.434378+0000","last_fullsized":"2024-09-11T23:52:05.434378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:58.851476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746317+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:52:05.746317+0000","last_peered":"2024-09-11T23:52:05.746317+0000","last_clean":"2024-09-11T23:52:05.746317+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:52:05.746317+0000","last_undegraded":"2024-09-11T23:52:05.746317+0000","last_fullsized":"2024-09-11T23:52:05.746317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523319+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:52:05.523319+0000","last_peered":"2024-09-11T23:52:05.523319+0000","last_clean":"2024-09-11T23:52:05.523319+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:52:05.523319+0000","last_undegraded":"2024-09-11T23:52:05.523319+0000","last_fullsized":"2024-09-11T23:52:05.523319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694954+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:52:05.694954+0000","last_peered":"2024-09-11T23:52:05.694954+0000","last_clean":"2024-09-11T23:52:05.694954+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:52:05.694954+0000","last_undegraded":"2024-09-11T23:52:05.694954+0000","last_fullsized":"2024-09-11T23:52:05.694954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435402+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:52:05.435402+0000","last_peered":"2024-09-11T23:52:05.435402+0000","last_clean":"2024-09-11T23:52:05.435402+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:52:05.435402+0000","last_undegraded":"2024-09-11T23:52:05.435402+0000","last_fullsized":"2024-09-11T23:52:05.435402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746473+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:52:05.746473+0000","last_peered":"2024-09-11T23:52:05.746473+0000","last_clean":"2024-09-11T23:52:05.746473+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:52:05.746473+0000","last_undegraded":"2024-09-11T23:52:05.746473+0000","last_fullsized":"2024-09-11T23:52:05.746473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744935+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:52:05.744935+0000","last_peered":"2024-09-11T23:52:05.744935+0000","last_clean":"2024-09-11T23:52:05.744935+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:52:05.744935+0000","last_undegraded":"2024-09-11T23:52:05.744935+0000","last_fullsized":"2024-09-11T23:52:05.744935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431713+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:52:05.431713+0000","last_peered":"2024-09-11T23:52:05.431713+0000","last_clean":"2024-09-11T23:52:05.431713+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:52:05.431713+0000","last_undegraded":"2024-09-11T23:52:05.431713+0000","last_fullsized":"2024-09-11T23:52:05.431713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158046+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:52:06.158046+0000","last_peered":"2024-09-11T23:52:06.158046+0000","last_clean":"2024-09-11T23:52:06.158046+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:52:06.158046+0000","last_undegraded":"2024-09-11T23:52:06.158046+0000","last_fullsized":"2024-09-11T23:52:06.158046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.157901+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:52:06.157901+0000","last_peered":"2024-09-11T23:52:06.157901+0000","last_clean":"2024-09-11T23:52:06.157901+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:52:06.157901+0000","last_undegraded":"2024-09-11T23:52:06.157901+0000","last_fullsized":"2024-09-11T23:52:06.157901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694419+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:52:05.694419+0000","last_peered":"2024-09-11T23:52:05.694419+0000","last_clean":"2024-09-11T23:52:05.694419+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:52:05.694419+0000","last_undegraded":"2024-09-11T23:52:05.694419+0000","last_fullsized":"2024-09-11T23:52:05.694419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872787+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:52:05.872787+0000","last_peered":"2024-09-11T23:52:05.872787+0000","last_clean":"2024-09-11T23:52:05.872787+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:52:05.872787+0000","last_undegraded":"2024-09-11T23:52:05.872787+0000","last_fullsized":"2024-09-11T23:52:05.872787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520950+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:52:05.520950+0000","last_peered":"2024-09-11T23:52:05.520950+0000","last_clean":"2024-09-11T23:52:05.520950+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:52:05.520950+0000","last_undegraded":"2024-09-11T23:52:05.520950+0000","last_fullsized":"2024-09-11T23:52:05.520950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694752+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:52:05.694752+0000","last_peered":"2024-09-11T23:52:05.694752+0000","last_clean":"2024-09-11T23:52:05.694752+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:52:05.694752+0000","last_undegraded":"2024-09-11T23:52:05.694752+0000","last_fullsized":"2024-09-11T23:52:05.694752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872460+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:52:05.872460+0000","last_peered":"2024-09-11T23:52:05.872460+0000","last_clean":"2024-09-11T23:52:05.872460+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:52:05.872460+0000","last_undegraded":"2024-09-11T23:52:05.872460+0000","last_fullsized":"2024-09-11T23:52:05.872460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431851+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:52:05.431851+0000","last_peered":"2024-09-11T23:52:05.431851+0000","last_clean":"2024-09-11T23:52:05.431851+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:52:05.431851+0000","last_undegraded":"2024-09-11T23:52:05.431851+0000","last_fullsized":"2024-09-11T23:52:05.431851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694603+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:52:05.694603+0000","last_peered":"2024-09-11T23:52:05.694603+0000","last_clean":"2024-09-11T23:52:05.694603+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:52:05.694603+0000","last_undegraded":"2024-09-11T23:52:05.694603+0000","last_fullsized":"2024-09-11T23:52:05.694603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523509+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:52:05.523509+0000","last_peered":"2024-09-11T23:52:05.523509+0000","last_clean":"2024-09-11T23:52:05.523509+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:52:05.523509+0000","last_undegraded":"2024-09-11T23:52:05.523509+0000","last_fullsized":"2024-09-11T23:52:05.523509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432812+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:52:05.432812+0000","last_peered":"2024-09-11T23:52:05.432812+0000","last_clean":"2024-09-11T23:52:05.432812+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:52:05.432812+0000","last_undegraded":"2024-09-11T23:52:05.432812+0000","last_fullsized":"2024-09-11T23:52:05.432812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525015+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:52:05.525015+0000","last_peered":"2024-09-11T23:52:05.525015+0000","last_clean":"2024-09-11T23:52:05.525015+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:52:05.525015+0000","last_undegraded":"2024-09-11T23:52:05.525015+0000","last_fullsized":"2024-09-11T23:52:05.525015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694943+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:52:05.694943+0000","last_peered":"2024-09-11T23:52:05.694943+0000","last_clean":"2024-09-11T23:52:05.694943+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:52:05.694943+0000","last_undegraded":"2024-09-11T23:52:05.694943+0000","last_fullsized":"2024-09-11T23:52:05.694943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872635+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:52:05.872635+0000","last_peered":"2024-09-11T23:52:05.872635+0000","last_clean":"2024-09-11T23:52:05.872635+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:52:05.872635+0000","last_undegraded":"2024-09-11T23:52:05.872635+0000","last_fullsized":"2024-09-11T23:52:05.872635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431698+0000","last_change":"2024-09-11T23:51:14.410108+0000","last_active":"2024-09-11T23:52:05.431698+0000","last_peered":"2024-09-11T23:52:05.431698+0000","last_clean":"2024-09-11T23:52:05.431698+0000","last_became_active":"2024-09-11T23:51:14.409588+0000","last_became_peered":"2024-09-11T23:51:14.409588+0000","last_unstale":"2024-09-11T23:52:05.431698+0000","last_undegraded":"2024-09-11T23:52:05.431698+0000","last_fullsized":"2024-09-11T23:52:05.431698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:29.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158121+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:52:06.158121+0000","last_peered":"2024-09-11T23:52:06.158121+0000","last_clean":"2024-09-11T23:52:06.158121+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:52:06.158121+0000","last_undegraded":"2024-09-11T23:52:06.158121+0000","last_fullsized":"2024-09-11T23:52:06.158121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694617+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:52:05.694617+0000","last_peered":"2024-09-11T23:52:05.694617+0000","last_clean":"2024-09-11T23:52:05.694617+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:52:05.694617+0000","last_undegraded":"2024-09-11T23:52:05.694617+0000","last_fullsized":"2024-09-11T23:52:05.694617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694660+0000","last_change":"2024-09-11T23:51:14.410872+0000","last_active":"2024-09-11T23:52:05.694660+0000","last_peered":"2024-09-11T23:52:05.694660+0000","last_clean":"2024-09-11T23:52:05.694660+0000","last_became_active":"2024-09-11T23:51:14.410531+0000","last_became_peered":"2024-09-11T23:51:14.410531+0000","last_unstale":"2024-09-11T23:52:05.694660+0000","last_undegraded":"2024-09-11T23:52:05.694660+0000","last_fullsized":"2024-09-11T23:52:05.694660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:10.502914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872498+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:52:05.872498+0000","last_peered":"2024-09-11T23:52:05.872498+0000","last_clean":"2024-09-11T23:52:05.872498+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:52:05.872498+0000","last_undegraded":"2024-09-11T23:52:05.872498+0000","last_fullsized":"2024-09-11T23:52:05.872498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516667+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:52:05.516667+0000","last_peered":"2024-09-11T23:52:05.516667+0000","last_clean":"2024-09-11T23:52:05.516667+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:52:05.516667+0000","last_undegraded":"2024-09-11T23:52:05.516667+0000","last_fullsized":"2024-09-11T23:52:05.516667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.014133+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:52:06.014133+0000","last_peered":"2024-09-11T23:52:06.014133+0000","last_clean":"2024-09-11T23:52:06.014133+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:52:06.014133+0000","last_undegraded":"2024-09-11T23:52:06.014133+0000","last_fullsized":"2024-09-11T23:52:06.014133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432914+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:52:05.432914+0000","last_peered":"2024-09-11T23:52:05.432914+0000","last_clean":"2024-09-11T23:52:05.432914+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:52:05.432914+0000","last_undegraded":"2024-09-11T23:52:05.432914+0000","last_fullsized":"2024-09-11T23:52:05.432914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434413+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:52:05.434413+0000","last_peered":"2024-09-11T23:52:05.434413+0000","last_clean":"2024-09-11T23:52:05.434413+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:52:05.434413+0000","last_undegraded":"2024-09-11T23:52:05.434413+0000","last_fullsized":"2024-09-11T23:52:05.434413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431356+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:52:05.431356+0000","last_peered":"2024-09-11T23:52:05.431356+0000","last_clean":"2024-09-11T23:52:05.431356+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:52:05.431356+0000","last_undegraded":"2024-09-11T23:52:05.431356+0000","last_fullsized":"2024-09-11T23:52:05.431356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523343+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:52:05.523343+0000","last_peered":"2024-09-11T23:52:05.523343+0000","last_clean":"2024-09-11T23:52:05.523343+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:52:05.523343+0000","last_undegraded":"2024-09-11T23:52:05.523343+0000","last_fullsized":"2024-09-11T23:52:05.523343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433385+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:52:05.433385+0000","last_peered":"2024-09-11T23:52:05.433385+0000","last_clean":"2024-09-11T23:52:05.433385+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:52:05.433385+0000","last_undegraded":"2024-09-11T23:52:05.433385+0000","last_fullsized":"2024-09-11T23:52:05.433385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520843+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:52:05.520843+0000","last_peered":"2024-09-11T23:52:05.520843+0000","last_clean":"2024-09-11T23:52:05.520843+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:52:05.520843+0000","last_undegraded":"2024-09-11T23:52:05.520843+0000","last_fullsized":"2024-09-11T23:52:05.520843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":134,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.748619+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:52:05.748619+0000","last_peered":"2024-09-11T23:52:05.748619+0000","last_clean":"2024-09-11T23:52:05.748619+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:52:05.748619+0000","last_undegraded":"2024-09-11T23:52:05.748619+0000","last_fullsized":"2024-09-11T23:52:05.748619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694869+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:52:05.694869+0000","last_peered":"2024-09-11T23:52:05.694869+0000","last_clean":"2024-09-11T23:52:05.694869+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:52:05.694869+0000","last_undegraded":"2024-09-11T23:52:05.694869+0000","last_fullsized":"2024-09-11T23:52:05.694869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746642+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:52:05.746642+0000","last_peered":"2024-09-11T23:52:05.746642+0000","last_clean":"2024-09-11T23:52:05.746642+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:52:05.746642+0000","last_undegraded":"2024-09-11T23:52:05.746642+0000","last_fullsized":"2024-09-11T23:52:05.746642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158655+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:52:06.158655+0000","last_peered":"2024-09-11T23:52:06.158655+0000","last_clean":"2024-09-11T23:52:06.158655+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:52:06.158655+0000","last_undegraded":"2024-09-11T23:52:06.158655+0000","last_fullsized":"2024-09-11T23:52:06.158655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433170+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:52:05.433170+0000","last_peered":"2024-09-11T23:52:05.433170+0000","last_clean":"2024-09-11T23:52:05.433170+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:52:05.433170+0000","last_undegraded":"2024-09-11T23:52:05.433170+0000","last_fullsized":"2024-09-11T23:52:05.433170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694598+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:52:05.694598+0000","last_peered":"2024-09-11T23:52:05.694598+0000","last_clean":"2024-09-11T23:52:05.694598+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:52:05.694598+0000","last_undegraded":"2024-09-11T23:52:05.694598+0000","last_fullsized":"2024-09-11T23:52:05.694598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'36","reported_seq":284,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.014229+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:52:06.014229+0000","last_peered":"2024-09-11T23:52:06.014229+0000","last_clean":"2024-09-11T23:52:06.014229+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:52:06.014229+0000","last_undegraded":"2024-09-11T23:52:06.014229+0000","last_fullsized":"2024-09-11T23:52:06.014229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66436,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":224,"num_read_kb":176,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516931+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:52:05.516931+0000","last_peered":"2024-09-11T23:52:05.516931+0000","last_clean":"2024-09-11T23:52:05.516931+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:52:05.516931+0000","last_undegraded":"2024-09-11T23:52:05.516931+0000","last_fullsized":"2024-09-11T23:52:05.516931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694704+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:52:05.694704+0000","last_peered":"2024-09-11T23:52:05.694704+0000","last_clean":"2024-09-11T23:52:05.694704+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:52:05.694704+0000","last_undegraded":"2024-09-11T23:52:05.694704+0000","last_fullsized":"2024-09-11T23:52:05.694704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431407+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:52:05.431407+0000","last_peered":"2024-09-11T23:52:05.431407+0000","last_clean":"2024-09-11T23:52:05.431407+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:52:05.431407+0000","last_undegraded":"2024-09-11T23:52:05.431407+0000","last_fullsized":"2024-09-11T23:52:05.431407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433521+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:52:05.433521+0000","last_peered":"2024-09-11T23:52:05.433521+0000","last_clean":"2024-09-11T23:52:05.433521+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:52:05.433521+0000","last_undegraded":"2024-09-11T23:52:05.433521+0000","last_fullsized":"2024-09-11T23:52:05.433521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746620+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:52:05.746620+0000","last_peered":"2024-09-11T23:52:05.746620+0000","last_clean":"2024-09-11T23:52:05.746620+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:52:05.746620+0000","last_undegraded":"2024-09-11T23:52:05.746620+0000","last_fullsized":"2024-09-11T23:52:05.746620+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.014353+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:52:06.014353+0000","last_peered":"2024-09-11T23:52:06.014353+0000","last_clean":"2024-09-11T23:52:06.014353+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:52:06.014353+0000","last_undegraded":"2024-09-11T23:52:06.014353+0000","last_fullsized":"2024-09-11T23:52:06.014353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746281+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:52:05.746281+0000","last_peered":"2024-09-11T23:52:05.746281+0000","last_clean":"2024-09-11T23:52:05.746281+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:52:05.746281+0000","last_undegraded":"2024-09-11T23:52:05.746281+0000","last_fullsized":"2024-09-11T23:52:05.746281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516591+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:52:05.516591+0000","last_peered":"2024-09-11T23:52:05.516591+0000","last_clean":"2024-09-11T23:52:05.516591+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:52:05.516591+0000","last_undegraded":"2024-09-11T23:52:05.516591+0000","last_fullsized":"2024-09-11T23:52:05.516591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432801+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:52:05.432801+0000","last_peered":"2024-09-11T23:52:05.432801+0000","last_clean":"2024-09-11T23:52:05.432801+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:52:05.432801+0000","last_undegraded":"2024-09-11T23:52:05.432801+0000","last_fullsized":"2024-09-11T23:52:05.432801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516946+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:52:05.516946+0000","last_peered":"2024-09-11T23:52:05.516946+0000","last_clean":"2024-09-11T23:52:05.516946+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:52:05.516946+0000","last_undegraded":"2024-09-11T23:52:05.516946+0000","last_fullsized":"2024-09-11T23:52:05.516946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872534+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:52:05.872534+0000","last_peered":"2024-09-11T23:52:05.872534+0000","last_clean":"2024-09-11T23:52:05.872534+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:52:05.872534+0000","last_undegraded":"2024-09-11T23:52:05.872534+0000","last_fullsized":"2024-09-11T23:52:05.872534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431687+0000","last_change":"2024-09-11T23:51:14.413526+0000","last_active":"2024-09-11T23:52:05.431687+0000","last_peered":"2024-09-11T23:52:05.431687+0000","last_clean":"2024-09-11T23:52:05.431687+0000","last_became_active":"2024-09-11T23:51:14.411027+0000","last_became_peered":"2024-09-11T23:51:14.411027+0000","last_unstale":"2024-09-11T23:52:05.431687+0000","last_undegraded":"2024-09-11T23:52:05.431687+0000","last_fullsized":"2024-09-11T23:52:05.431687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:55.020567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746675+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:52:05.746675+0000","last_peered":"2024-09-11T23:52:05.746675+0000","last_clean":"2024-09-11T23:52:05.746675+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:52:05.746675+0000","last_undegraded":"2024-09-11T23:52:05.746675+0000","last_fullsized":"2024-09-11T23:52:05.746675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.642614+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:52:05.642614+0000","last_peered":"2024-09-11T23:52:05.642614+0000","last_clean":"2024-09-11T23:52:05.642614+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:52:05.642614+0000","last_undegraded":"2024-09-11T23:52:05.642614+0000","last_fullsized":"2024-09-11T23:52:05.642614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746458+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:52:05.746458+0000","last_peered":"2024-09-11T23:52:05.746458+0000","last_clean":"2024-09-11T23:52:05.746458+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:52:05.746458+0000","last_undegraded":"2024-09-11T23:52:05.746458+0000","last_fullsized":"2024-09-11T23:52:05.746458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694804+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:52:05.694804+0000","last_peered":"2024-09-11T23:52:05.694804+0000","last_clean":"2024-09-11T23:52:05.694804+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:52:05.694804+0000","last_undegraded":"2024-09-11T23:52:05.694804+0000","last_fullsized":"2024-09-11T23:52:05.694804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746570+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:52:05.746570+0000","last_peered":"2024-09-11T23:52:05.746570+0000","last_clean":"2024-09-11T23:52:05.746570+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:52:05.746570+0000","last_undegraded":"2024-09-11T23:52:05.746570+0000","last_fullsized":"2024-09-11T23:52:05.746570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746409+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:52:05.746409+0000","last_peered":"2024-09-11T23:52:05.746409+0000","last_clean":"2024-09-11T23:52:05.746409+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:52:05.746409+0000","last_undegraded":"2024-09-11T23:52:05.746409+0000","last_fullsized":"2024-09-11T23:52:05.746409+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744872+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:52:05.744872+0000","last_peered":"2024-09-11T23:52:05.744872+0000","last_clean":"2024-09-11T23:52:05.744872+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:52:05.744872+0000","last_undegraded":"2024-09-11T23:52:05.744872+0000","last_fullsized":"2024-09-11T23:52:05.744872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523710+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:52:05.523710+0000","last_peered":"2024-09-11T23:52:05.523710+0000","last_clean":"2024-09-11T23:52:05.523710+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:52:05.523710+0000","last_undegraded":"2024-09-11T23:52:05.523710+0000","last_fullsized":"2024-09-11T23:52:05.523710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872415+0000","last_change":"2024-09-11T23:51:14.121480+0000","last_active":"2024-09-11T23:52:05.872415+0000","last_peered":"2024-09-11T23:52:05.872415+0000","last_clean":"2024-09-11T23:52:05.872415+0000","last_became_active":"2024-09-11T23:51:14.118435+0000","last_became_peered":"2024-09-11T23:51:14.118435+0000","last_unstale":"2024-09-11T23:52:05.872415+0000","last_undegraded":"2024-09-11T23:52:05.872415+0000","last_fullsized":"2024-09-11T23:52:05.872415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431342+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:52:05.431342+0000","last_peered":"2024-09-11T23:52:05.431342+0000","last_clean":"2024-09-11T23:52:05.431342+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:52:05.431342+0000","last_undegraded":"2024-09-11T23:52:05.431342+0000","last_fullsized":"2024-09-11T23:52:05.431342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":136,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431550+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:52:05.431550+0000","last_peered":"2024-09-11T23:52:05.431550+0000","last_clean":"2024-09-11T23:52:05.431550+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:52:05.431550+0000","last_undegraded":"2024-09-11T23:52:05.431550+0000","last_fullsized":"2024-09-11T23:52:05.431550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432859+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:52:05.432859+0000","last_peered":"2024-09-11T23:52:05.432859+0000","last_clean":"2024-09-11T23:52:05.432859+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:52:05.432859+0000","last_undegraded":"2024-09-11T23:52:05.432859+0000","last_fullsized":"2024-09-11T23:52:05.432859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746522+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:52:05.746522+0000","last_peered":"2024-09-11T23:52:05.746522+0000","last_clean":"2024-09-11T23:52:05.746522+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:52:05.746522+0000","last_undegraded":"2024-09-11T23:52:05.746522+0000","last_fullsized":"2024-09-11T23:52:05.746522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435012+0000","last_change":"2024-09-11T23:51:14.409908+0000","last_active":"2024-09-11T23:52:05.435012+0000","last_peered":"2024-09-11T23:52:05.435012+0000","last_clean":"2024-09-11T23:52:05.435012+0000","last_became_active":"2024-09-11T23:51:14.409432+0000","last_became_peered":"2024-09-11T23:51:14.409432+0000","last_unstale":"2024-09-11T23:52:05.435012+0000","last_undegraded":"2024-09-11T23:52:05.435012+0000","last_fullsized":"2024-09-11T23:52:05.435012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:06.371306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.521172+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:52:05.521172+0000","last_peered":"2024-09-11T23:52:05.521172+0000","last_clean":"2024-09-11T23:52:05.521172+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:52:05.521172+0000","last_undegraded":"2024-09-11T23:52:05.521172+0000","last_fullsized":"2024-09-11T23:52:05.521172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435167+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:52:05.435167+0000","last_peered":"2024-09-11T23:52:05.435167+0000","last_clean":"2024-09-11T23:52:05.435167+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:52:05.435167+0000","last_undegraded":"2024-09-11T23:52:05.435167+0000","last_fullsized":"2024-09-11T23:52:05.435167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.517017+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:52:05.517017+0000","last_peered":"2024-09-11T23:52:05.517017+0000","last_clean":"2024-09-11T23:52:05.517017+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:52:05.517017+0000","last_undegraded":"2024-09-11T23:52:05.517017+0000","last_fullsized":"2024-09-11T23:52:05.517017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431772+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:52:05.431772+0000","last_peered":"2024-09-11T23:52:05.431772+0000","last_clean":"2024-09-11T23:52:05.431772+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:52:05.431772+0000","last_undegraded":"2024-09-11T23:52:05.431772+0000","last_fullsized":"2024-09-11T23:52:05.431772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'48","reported_seq":95,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.649913+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:52:05.649913+0000","last_peered":"2024-09-11T23:52:05.649913+0000","last_clean":"2024-09-11T23:52:05.649913+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:52:05.649913+0000","last_undegraded":"2024-09-11T23:52:05.649913+0000","last_fullsized":"2024-09-11T23:52:05.649913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525022+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:52:05.525022+0000","last_peered":"2024-09-11T23:52:05.525022+0000","last_clean":"2024-09-11T23:52:05.525022+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:52:05.525022+0000","last_undegraded":"2024-09-11T23:52:05.525022+0000","last_fullsized":"2024-09-11T23:52:05.525022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.745108+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:52:05.745108+0000","last_peered":"2024-09-11T23:52:05.745108+0000","last_clean":"2024-09-11T23:52:05.745108+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:52:05.745108+0000","last_undegraded":"2024-09-11T23:52:05.745108+0000","last_fullsized":"2024-09-11T23:52:05.745108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158521+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:52:06.158521+0000","last_peered":"2024-09-11T23:52:06.158521+0000","last_clean":"2024-09-11T23:52:06.158521+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:52:06.158521+0000","last_undegraded":"2024-09-11T23:52:06.158521+0000","last_fullsized":"2024-09-11T23:52:06.158521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694501+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:52:05.694501+0000","last_peered":"2024-09-11T23:52:05.694501+0000","last_clean":"2024-09-11T23:52:05.694501+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:52:05.694501+0000","last_undegraded":"2024-09-11T23:52:05.694501+0000","last_fullsized":"2024-09-11T23:52:05.694501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516588+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:52:05.516588+0000","last_peered":"2024-09-11T23:52:05.516588+0000","last_clean":"2024-09-11T23:52:05.516588+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:52:05.516588+0000","last_undegraded":"2024-09-11T23:52:05.516588+0000","last_fullsized":"2024-09-11T23:52:05.516588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523605+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:52:05.523605+0000","last_peered":"2024-09-11T23:52:05.523605+0000","last_clean":"2024-09-11T23:52:05.523605+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:52:05.523605+0000","last_undegraded":"2024-09-11T23:52:05.523605+0000","last_fullsized":"2024-09-11T23:52:05.523605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872645+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:52:05.872645+0000","last_peered":"2024-09-11T23:52:05.872645+0000","last_clean":"2024-09-11T23:52:05.872645+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:52:05.872645+0000","last_undegraded":"2024-09-11T23:52:05.872645+0000","last_fullsized":"2024-09-11T23:52:05.872645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872496+0000","last_change":"2024-09-11T23:51:14.123663+0000","last_active":"2024-09-11T23:52:05.872496+0000","last_peered":"2024-09-11T23:52:05.872496+0000","last_clean":"2024-09-11T23:52:05.872496+0000","last_became_active":"2024-09-11T23:51:14.118315+0000","last_became_peered":"2024-09-11T23:51:14.118315+0000","last_unstale":"2024-09-11T23:52:05.872496+0000","last_undegraded":"2024-09-11T23:52:05.872496+0000","last_fullsized":"2024-09-11T23:52:05.872496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:10.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.524009+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:52:05.524009+0000","last_peered":"2024-09-11T23:52:05.524009+0000","last_clean":"2024-09-11T23:52:05.524009+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:52:05.524009+0000","last_undegraded":"2024-09-11T23:52:05.524009+0000","last_fullsized":"2024-09-11T23:52:05.524009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516698+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:52:05.516698+0000","last_peered":"2024-09-11T23:52:05.516698+0000","last_clean":"2024-09-11T23:52:05.516698+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:52:05.516698+0000","last_undegraded":"2024-09-11T23:52:05.516698+0000","last_fullsized":"2024-09-11T23:52:05.516698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746447+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:52:05.746447+0000","last_peered":"2024-09-11T23:52:05.746447+0000","last_clean":"2024-09-11T23:52:05.746447+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:52:05.746447+0000","last_undegraded":"2024-09-11T23:52:05.746447+0000","last_fullsized":"2024-09-11T23:52:05.746447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433119+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:52:05.433119+0000","last_peered":"2024-09-11T23:52:05.433119+0000","last_clean":"2024-09-11T23:52:05.433119+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:52:05.433119+0000","last_undegraded":"2024-09-11T23:52:05.433119+0000","last_fullsized":"2024-09-11T23:52:05.433119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516897+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:52:05.516897+0000","last_peered":"2024-09-11T23:52:05.516897+0000","last_clean":"2024-09-11T23:52:05.516897+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:52:05.516897+0000","last_undegraded":"2024-09-11T23:52:05.516897+0000","last_fullsized":"2024-09-11T23:52:05.516897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694561+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:52:05.694561+0000","last_peered":"2024-09-11T23:52:05.694561+0000","last_clean":"2024-09-11T23:52:05.694561+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:52:05.694561+0000","last_undegraded":"2024-09-11T23:52:05.694561+0000","last_fullsized":"2024-09-11T23:52:05.694561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516945+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:52:05.516945+0000","last_peered":"2024-09-11T23:52:05.516945+0000","last_clean":"2024-09-11T23:52:05.516945+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:52:05.516945+0000","last_undegraded":"2024-09-11T23:52:05.516945+0000","last_fullsized":"2024-09-11T23:52:05.516945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158617+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:52:06.158617+0000","last_peered":"2024-09-11T23:52:06.158617+0000","last_clean":"2024-09-11T23:52:06.158617+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:52:06.158617+0000","last_undegraded":"2024-09-11T23:52:06.158617+0000","last_fullsized":"2024-09-11T23:52:06.158617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525435+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:52:05.525435+0000","last_peered":"2024-09-11T23:52:05.525435+0000","last_clean":"2024-09-11T23:52:05.525435+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:52:05.525435+0000","last_undegraded":"2024-09-11T23:52:05.525435+0000","last_fullsized":"2024-09-11T23:52:05.525435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525158+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:52:05.525158+0000","last_peered":"2024-09-11T23:52:05.525158+0000","last_clean":"2024-09-11T23:52:05.525158+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:52:05.525158+0000","last_undegraded":"2024-09-11T23:52:05.525158+0000","last_fullsized":"2024-09-11T23:52:05.525158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.521532+0000","last_change":"2024-09-11T23:51:14.125596+0000","last_active":"2024-09-11T23:52:05.521532+0000","last_peered":"2024-09-11T23:52:05.521532+0000","last_clean":"2024-09-11T23:52:05.521532+0000","last_became_active":"2024-09-11T23:51:14.119682+0000","last_became_peered":"2024-09-11T23:51:14.119682+0000","last_unstale":"2024-09-11T23:52:05.521532+0000","last_undegraded":"2024-09-11T23:52:05.521532+0000","last_fullsized":"2024-09-11T23:52:05.521532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:06.062602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596490,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":763,"num_read_kb":490,"num_write":295,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1789470,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":436969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1089536,"data_stored":1236648,"data_compressed":44911,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":839348,"data_compressed":22046,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":838686,"data_compressed":22046,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1039219,"data_compressed":38755,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445521,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1169268,"data_compressed":44911,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":447193,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":974848,"data_stored":970648,"data_compressed":28202,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":513493,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":498698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":437865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133469,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":200849,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:52:08.725 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:52:08.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: [11/Sep/2024:23:52:07] ENGINE Serving on https://172.21.15.62:7150 2024-09-11T23:52:08.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: [11/Sep/2024:23:52:07] ENGINE Bus STARTED 2024-09-11T23:52:08.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: [11/Sep/2024:23:52:07] ENGINE Error in HTTPServer.serve 2024-09-11T23:52:08.871 INFO:journalctl@ceph.mon.c.smithi120.stdout: Traceback (most recent call last): 2024-09-11T23:52:08.872 INFO:journalctl@ceph.mon.c.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:52:08.872 INFO:journalctl@ceph.mon.c.smithi120.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:52:08.872 INFO:journalctl@ceph.mon.c.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:52:08.872 INFO:journalctl@ceph.mon.c.smithi120.stdout: self._run(expiration_interval) 2024-09-11T23:52:08.872 INFO:journalctl@ceph.mon.c.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:52:08.872 INFO:journalctl@ceph.mon.c.smithi120.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:52:08.872 INFO:journalctl@ceph.mon.c.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:52:08.872 INFO:journalctl@ceph.mon.c.smithi120.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:52:08.873 INFO:journalctl@ceph.mon.c.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:52:08.873 INFO:journalctl@ceph.mon.c.smithi120.stdout: s = self.context.wrap_socket( 2024-09-11T23:52:08.873 INFO:journalctl@ceph.mon.c.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:52:08.873 INFO:journalctl@ceph.mon.c.smithi120.stdout: return self.sslsocket_class._create( 2024-09-11T23:52:08.873 INFO:journalctl@ceph.mon.c.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:52:08.873 INFO:journalctl@ceph.mon.c.smithi120.stdout: self.do_handshake() 2024-09-11T23:52:08.873 INFO:journalctl@ceph.mon.c.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:52:08.873 INFO:journalctl@ceph.mon.c.smithi120.stdout: self._sslobj.do_handshake() 2024-09-11T23:52:08.873 INFO:journalctl@ceph.mon.c.smithi120.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:52:08.874 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:52:08.874 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-11T23:52:08.874 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.874 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.874 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.874 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.874 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.874 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: Unable to set osd_memory_target on smithi120 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.876 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:08 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.872+0000 7f68e4ff6640 1 -- 172.21.15.62:0/1497928441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e0101c00 msgr2=0x7f68e0102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:08.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.872+0000 7f68e4ff6640 1 --2- 172.21.15.62:0/1497928441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e0101c00 0x7f68e0102080 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009900 tx=0x7f68d402a410 comp rx=0 tx=0).stop 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 -- 172.21.15.62:0/1497928441 shutdown_connections 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 --2- 172.21.15.62:0/1497928441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68e01025c0 0x7f68e0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 --2- 172.21.15.62:0/1497928441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e0101c00 0x7f68e0102080 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 --2- 172.21.15.62:0/1497928441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68e0100a10 0x7f68e0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 -- 172.21.15.62:0/1497928441 >> 172.21.15.62:0/1497928441 conn(0x7f68e00fc170 msgr2=0x7f68e00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 -- 172.21.15.62:0/1497928441 shutdown_connections 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 -- 172.21.15.62:0/1497928441 wait complete. 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 Processor -- start 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 -- start start 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e0100a10 0x7f68e0069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.873+0000 7f68e4ff6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68e0101c00 0x7f68e019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68e4ff6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68e01025c0 0x7f68e019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:08.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68e4ff6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010c690 con 0x7f68e0100a10 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68e4ff6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010c510 con 0x7f68e0101c00 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68e4ff6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010c810 con 0x7f68e01025c0 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68de59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e0100a10 0x7f68e0069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68de59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e0100a10 0x7f68e0069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45952/0 (socket says 172.21.15.62:45952) 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68de59c640 1 -- 172.21.15.62:0/3668834744 learned_addr learned my addr 172.21.15.62:0/3668834744 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68ded9d640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68e01025c0 0x7f68e019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68ddd9b640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68e0101c00 0x7f68e019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68de59c640 1 -- 172.21.15.62:0/3668834744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68e01025c0 msgr2=0x7f68e019f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68de59c640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68e01025c0 0x7f68e019f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68de59c640 1 -- 172.21.15.62:0/3668834744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68e0101c00 msgr2=0x7f68e019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68de59c640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68e0101c00 0x7f68e019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68de59c640 1 -- 172.21.15.62:0/3668834744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e019faf0 con 0x7f68e0100a10 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68de59c640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e0100a10 0x7f68e0069ef0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f68d000e9c0 tx=0x7f68d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:08.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68cf7fe640 1 -- 172.21.15.62:0/3668834744 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d000cd80 con 0x7f68e0100a10 2024-09-11T23:52:08.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68cf7fe640 1 -- 172.21.15.62:0/3668834744 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d00035f0 con 0x7f68e0100a10 2024-09-11T23:52:08.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68e4ff6640 1 -- 172.21.15.62:0/3668834744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e019fd80 con 0x7f68e0100a10 2024-09-11T23:52:08.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68cf7fe640 1 -- 172.21.15.62:0/3668834744 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0010690 con 0x7f68e0100a10 2024-09-11T23:52:08.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.874+0000 7f68e4ff6640 1 -- 172.21.15.62:0/3668834744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e01a0190 con 0x7f68e0100a10 2024-09-11T23:52:08.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.876+0000 7f68e4ff6640 1 -- 172.21.15.62:0/3668834744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e0107e70 con 0x7f68e0100a10 2024-09-11T23:52:08.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.877+0000 7f68cf7fe640 1 -- 172.21.15.62:0/3668834744 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f68d00037a0 con 0x7f68e0100a10 2024-09-11T23:52:08.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.877+0000 7f68cf7fe640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f68c80a2d00 0x7f68c80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:08.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.877+0000 7f68cf7fe640 1 -- 172.21.15.62:0/3668834744 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f68d00cf770 con 0x7f68e0100a10 2024-09-11T23:52:08.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.877+0000 7f68ddd9b640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f68c80a2d00 0x7f68c80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:08.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.878+0000 7f68ddd9b640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f68c80a2d00 0x7f68c80a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f68d40314e0 tx=0x7f68d4000bc0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:08.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:08.882+0000 7f68cf7fe640 1 -- 172.21.15.62:0/3668834744 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68d009c3c0 con 0x7f68e0100a10 2024-09-11T23:52:08.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: [11/Sep/2024:23:52:07] ENGINE Serving on https://172.21.15.62:7150 2024-09-11T23:52:08.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: [11/Sep/2024:23:52:07] ENGINE Bus STARTED 2024-09-11T23:52:08.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: [11/Sep/2024:23:52:07] ENGINE Error in HTTPServer.serve 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: Traceback (most recent call last): 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: self._run(expiration_interval) 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: s = self.context.wrap_socket( 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:52:08.918 INFO:journalctl@ceph.mon.b.smithi083.stdout: return self.sslsocket_class._create( 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout: self.do_handshake() 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout: self._sslobj.do_handshake() 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: Unable to set osd_memory_target on smithi120 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:08.920 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:08 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:09.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.060+0000 7f68e4ff6640 1 -- 172.21.15.62:0/3668834744 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68e0062630 con 0x7f68c80a2d00 2024-09-11T23:52:09.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.067+0000 7f68cf7fe640 1 -- 172.21.15.62:0/3668834744 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257965 (secure 0 0 0) 0x7f68e0062630 con 0x7f68c80a2d00 2024-09-11T23:52:09.069 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:09.073 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 -- 172.21.15.62:0/3668834744 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f68c80a2d00 msgr2=0x7f68c80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f68c80a2d00 0x7f68c80a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f68d40314e0 tx=0x7f68d4000bc0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 -- 172.21.15.62:0/3668834744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e0100a10 msgr2=0x7f68e0069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e0100a10 0x7f68e0069ef0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f68d000e9c0 tx=0x7f68d000ee80 comp rx=0 tx=0).stop 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 -- 172.21.15.62:0/3668834744 shutdown_connections 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f68c80a2d00 0x7f68c80a51c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68e01025c0 0x7f68e019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68e0101c00 0x7f68e019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 --2- 172.21.15.62:0/3668834744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e0100a10 0x7f68e0069ef0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 -- 172.21.15.62:0/3668834744 >> 172.21.15.62:0/3668834744 conn(0x7f68e00fc170 msgr2=0x7f68e0105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:09.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 -- 172.21.15.62:0/3668834744 shutdown_connections 2024-09-11T23:52:09.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.070+0000 7f68e4ff6640 1 -- 172.21.15.62:0/3668834744 wait complete. 2024-09-11T23:52:09.081 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T23:52:07.445934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187092,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":899,"num_read_kb":609,"num_write":478,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348104,"kb_used_data":11296,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593720,"statfs":{"total":1151940427776,"available":1151583969280,"internally_reserved":0,"allocated":11567104,"data_stored":8873556,"data_compressed":209571,"data_compressed_allocated":270336,"data_compressed_original":3391488,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.977168"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872692+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:52:05.872692+0000","last_peered":"2024-09-11T23:52:05.872692+0000","last_clean":"2024-09-11T23:52:05.872692+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:52:05.872692+0000","last_undegraded":"2024-09-11T23:52:05.872692+0000","last_fullsized":"2024-09-11T23:52:05.872692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158483+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:52:06.158483+0000","last_peered":"2024-09-11T23:52:06.158483+0000","last_clean":"2024-09-11T23:52:06.158483+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:52:06.158483+0000","last_undegraded":"2024-09-11T23:52:06.158483+0000","last_fullsized":"2024-09-11T23:52:06.158483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158255+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:52:06.158255+0000","last_peered":"2024-09-11T23:52:06.158255+0000","last_clean":"2024-09-11T23:52:06.158255+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:52:06.158255+0000","last_undegraded":"2024-09-11T23:52:06.158255+0000","last_fullsized":"2024-09-11T23:52:06.158255+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744925+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:52:05.744925+0000","last_peered":"2024-09-11T23:52:05.744925+0000","last_clean":"2024-09-11T23:52:05.744925+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:52:05.744925+0000","last_undegraded":"2024-09-11T23:52:05.744925+0000","last_fullsized":"2024-09-11T23:52:05.744925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516648+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:52:05.516648+0000","last_peered":"2024-09-11T23:52:05.516648+0000","last_clean":"2024-09-11T23:52:05.516648+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:52:05.516648+0000","last_undegraded":"2024-09-11T23:52:05.516648+0000","last_fullsized":"2024-09-11T23:52:05.516648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434378+0000","last_change":"2024-09-11T23:51:14.120226+0000","last_active":"2024-09-11T23:52:05.434378+0000","last_peered":"2024-09-11T23:52:05.434378+0000","last_clean":"2024-09-11T23:52:05.434378+0000","last_became_active":"2024-09-11T23:51:14.119811+0000","last_became_peered":"2024-09-11T23:51:14.119811+0000","last_unstale":"2024-09-11T23:52:05.434378+0000","last_undegraded":"2024-09-11T23:52:05.434378+0000","last_fullsized":"2024-09-11T23:52:05.434378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:58.851476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746317+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:52:05.746317+0000","last_peered":"2024-09-11T23:52:05.746317+0000","last_clean":"2024-09-11T23:52:05.746317+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:52:05.746317+0000","last_undegraded":"2024-09-11T23:52:05.746317+0000","last_fullsized":"2024-09-11T23:52:05.746317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523319+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:52:05.523319+0000","last_peered":"2024-09-11T23:52:05.523319+0000","last_clean":"2024-09-11T23:52:05.523319+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:52:05.523319+0000","last_undegraded":"2024-09-11T23:52:05.523319+0000","last_fullsized":"2024-09-11T23:52:05.523319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694954+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:52:05.694954+0000","last_peered":"2024-09-11T23:52:05.694954+0000","last_clean":"2024-09-11T23:52:05.694954+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:52:05.694954+0000","last_undegraded":"2024-09-11T23:52:05.694954+0000","last_fullsized":"2024-09-11T23:52:05.694954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435402+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:52:05.435402+0000","last_peered":"2024-09-11T23:52:05.435402+0000","last_clean":"2024-09-11T23:52:05.435402+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:52:05.435402+0000","last_undegraded":"2024-09-11T23:52:05.435402+0000","last_fullsized":"2024-09-11T23:52:05.435402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746473+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:52:05.746473+0000","last_peered":"2024-09-11T23:52:05.746473+0000","last_clean":"2024-09-11T23:52:05.746473+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:52:05.746473+0000","last_undegraded":"2024-09-11T23:52:05.746473+0000","last_fullsized":"2024-09-11T23:52:05.746473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744935+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:52:05.744935+0000","last_peered":"2024-09-11T23:52:05.744935+0000","last_clean":"2024-09-11T23:52:05.744935+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:52:05.744935+0000","last_undegraded":"2024-09-11T23:52:05.744935+0000","last_fullsized":"2024-09-11T23:52:05.744935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431713+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:52:05.431713+0000","last_peered":"2024-09-11T23:52:05.431713+0000","last_clean":"2024-09-11T23:52:05.431713+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:52:05.431713+0000","last_undegraded":"2024-09-11T23:52:05.431713+0000","last_fullsized":"2024-09-11T23:52:05.431713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158046+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:52:06.158046+0000","last_peered":"2024-09-11T23:52:06.158046+0000","last_clean":"2024-09-11T23:52:06.158046+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:52:06.158046+0000","last_undegraded":"2024-09-11T23:52:06.158046+0000","last_fullsized":"2024-09-11T23:52:06.158046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.157901+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:52:06.157901+0000","last_peered":"2024-09-11T23:52:06.157901+0000","last_clean":"2024-09-11T23:52:06.157901+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:52:06.157901+0000","last_undegraded":"2024-09-11T23:52:06.157901+0000","last_fullsized":"2024-09-11T23:52:06.157901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694419+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:52:05.694419+0000","last_peered":"2024-09-11T23:52:05.694419+0000","last_clean":"2024-09-11T23:52:05.694419+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:52:05.694419+0000","last_undegraded":"2024-09-11T23:52:05.694419+0000","last_fullsized":"2024-09-11T23:52:05.694419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872787+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:52:05.872787+0000","last_peered":"2024-09-11T23:52:05.872787+0000","last_clean":"2024-09-11T23:52:05.872787+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:52:05.872787+0000","last_undegraded":"2024-09-11T23:52:05.872787+0000","last_fullsized":"2024-09-11T23:52:05.872787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520950+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:52:05.520950+0000","last_peered":"2024-09-11T23:52:05.520950+0000","last_clean":"2024-09-11T23:52:05.520950+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:52:05.520950+0000","last_undegraded":"2024-09-11T23:52:05.520950+0000","last_fullsized":"2024-09-11T23:52:05.520950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694752+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:52:05.694752+0000","last_peered":"2024-09-11T23:52:05.694752+0000","last_clean":"2024-09-11T23:52:05.694752+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:52:05.694752+0000","last_undegraded":"2024-09-11T23:52:05.694752+0000","last_fullsized":"2024-09-11T23:52:05.694752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872460+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:52:05.872460+0000","last_peered":"2024-09-11T23:52:05.872460+0000","last_clean":"2024-09-11T23:52:05.872460+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:52:05.872460+0000","last_undegraded":"2024-09-11T23:52:05.872460+0000","last_fullsized":"2024-09-11T23:52:05.872460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431851+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:52:05.431851+0000","last_peered":"2024-09-11T23:52:05.431851+0000","last_clean":"2024-09-11T23:52:05.431851+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:52:05.431851+0000","last_undegraded":"2024-09-11T23:52:05.431851+0000","last_fullsized":"2024-09-11T23:52:05.431851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694603+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:52:05.694603+0000","last_peered":"2024-09-11T23:52:05.694603+0000","last_clean":"2024-09-11T23:52:05.694603+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:52:05.694603+0000","last_undegraded":"2024-09-11T23:52:05.694603+0000","last_fullsized":"2024-09-11T23:52:05.694603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523509+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:52:05.523509+0000","last_peered":"2024-09-11T23:52:05.523509+0000","last_clean":"2024-09-11T23:52:05.523509+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:52:05.523509+0000","last_undegraded":"2024-09-11T23:52:05.523509+0000","last_fullsized":"2024-09-11T23:52:05.523509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432812+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:52:05.432812+0000","last_peered":"2024-09-11T23:52:05.432812+0000","last_clean":"2024-09-11T23:52:05.432812+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:52:05.432812+0000","last_undegraded":"2024-09-11T23:52:05.432812+0000","last_fullsized":"2024-09-11T23:52:05.432812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525015+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:52:05.525015+0000","last_peered":"2024-09-11T23:52:05.525015+0000","last_clean":"2024-09-11T23:52:05.525015+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:52:05.525015+0000","last_undegraded":"2024-09-11T23:52:05.525015+0000","last_fullsized":"2024-09-11T23:52:05.525015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694943+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:52:05.694943+0000","last_peered":"2024-09-11T23:52:05.694943+0000","last_clean":"2024-09-11T23:52:05.694943+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:52:05.694943+0000","last_undegraded":"2024-09-11T23:52:05.694943+0000","last_fullsized":"2024-09-11T23:52:05.694943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872635+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:52:05.872635+0000","last_peered":"2024-09-11T23:52:05.872635+0000","last_clean":"2024-09-11T23:52:05.872635+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:52:05.872635+0000","last_undegraded":"2024-09-11T23:52:05.872635+0000","last_fullsized":"2024-09-11T23:52:05.872635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431698+0000","last_change":"2024-09-11T23:51:14.410108+0000","last_active":"2024-09-11T23:52:05.431698+0000","last_peered":"2024-09-11T23:52:05.431698+0000","last_clean":"2024-09-11T23:52:05.431698+0000","last_became_active":"2024-09-11T23:51:14.409588+0000","last_became_peered":"2024-09-11T23:51:14.409588+0000","last_unstale":"2024-09-11T23:52:05.431698+0000","last_undegraded":"2024-09-11T23:52:05.431698+0000","last_fullsized":"2024-09-11T23:52:05.431698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:29.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158121+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:52:06.158121+0000","last_peered":"2024-09-11T23:52:06.158121+0000","last_clean":"2024-09-11T23:52:06.158121+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:52:06.158121+0000","last_undegraded":"2024-09-11T23:52:06.158121+0000","last_fullsized":"2024-09-11T23:52:06.158121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694617+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:52:05.694617+0000","last_peered":"2024-09-11T23:52:05.694617+0000","last_clean":"2024-09-11T23:52:05.694617+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:52:05.694617+0000","last_undegraded":"2024-09-11T23:52:05.694617+0000","last_fullsized":"2024-09-11T23:52:05.694617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694660+0000","last_change":"2024-09-11T23:51:14.410872+0000","last_active":"2024-09-11T23:52:05.694660+0000","last_peered":"2024-09-11T23:52:05.694660+0000","last_clean":"2024-09-11T23:52:05.694660+0000","last_became_active":"2024-09-11T23:51:14.410531+0000","last_became_peered":"2024-09-11T23:51:14.410531+0000","last_unstale":"2024-09-11T23:52:05.694660+0000","last_undegraded":"2024-09-11T23:52:05.694660+0000","last_fullsized":"2024-09-11T23:52:05.694660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:10.502914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872498+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:52:05.872498+0000","last_peered":"2024-09-11T23:52:05.872498+0000","last_clean":"2024-09-11T23:52:05.872498+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:52:05.872498+0000","last_undegraded":"2024-09-11T23:52:05.872498+0000","last_fullsized":"2024-09-11T23:52:05.872498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516667+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:52:05.516667+0000","last_peered":"2024-09-11T23:52:05.516667+0000","last_clean":"2024-09-11T23:52:05.516667+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:52:05.516667+0000","last_undegraded":"2024-09-11T23:52:05.516667+0000","last_fullsized":"2024-09-11T23:52:05.516667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.014133+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:52:06.014133+0000","last_peered":"2024-09-11T23:52:06.014133+0000","last_clean":"2024-09-11T23:52:06.014133+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:52:06.014133+0000","last_undegraded":"2024-09-11T23:52:06.014133+0000","last_fullsized":"2024-09-11T23:52:06.014133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432914+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:52:05.432914+0000","last_peered":"2024-09-11T23:52:05.432914+0000","last_clean":"2024-09-11T23:52:05.432914+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:52:05.432914+0000","last_undegraded":"2024-09-11T23:52:05.432914+0000","last_fullsized":"2024-09-11T23:52:05.432914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434413+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:52:05.434413+0000","last_peered":"2024-09-11T23:52:05.434413+0000","last_clean":"2024-09-11T23:52:05.434413+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:52:05.434413+0000","last_undegraded":"2024-09-11T23:52:05.434413+0000","last_fullsized":"2024-09-11T23:52:05.434413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431356+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:52:05.431356+0000","last_peered":"2024-09-11T23:52:05.431356+0000","last_clean":"2024-09-11T23:52:05.431356+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:52:05.431356+0000","last_undegraded":"2024-09-11T23:52:05.431356+0000","last_fullsized":"2024-09-11T23:52:05.431356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523343+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:52:05.523343+0000","last_peered":"2024-09-11T23:52:05.523343+0000","last_clean":"2024-09-11T23:52:05.523343+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:52:05.523343+0000","last_undegraded":"2024-09-11T23:52:05.523343+0000","last_fullsized":"2024-09-11T23:52:05.523343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433385+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:52:05.433385+0000","last_peered":"2024-09-11T23:52:05.433385+0000","last_clean":"2024-09-11T23:52:05.433385+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:52:05.433385+0000","last_undegraded":"2024-09-11T23:52:05.433385+0000","last_fullsized":"2024-09-11T23:52:05.433385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520843+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:52:05.520843+0000","last_peered":"2024-09-11T23:52:05.520843+0000","last_clean":"2024-09-11T23:52:05.520843+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:52:05.520843+0000","last_undegraded":"2024-09-11T23:52:05.520843+0000","last_fullsized":"2024-09-11T23:52:05.520843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":134,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.748619+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:52:05.748619+0000","last_peered":"2024-09-11T23:52:05.748619+0000","last_clean":"2024-09-11T23:52:05.748619+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:52:05.748619+0000","last_undegraded":"2024-09-11T23:52:05.748619+0000","last_fullsized":"2024-09-11T23:52:05.748619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694869+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:52:05.694869+0000","last_peered":"2024-09-11T23:52:05.694869+0000","last_clean":"2024-09-11T23:52:05.694869+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:52:05.694869+0000","last_undegraded":"2024-09-11T23:52:05.694869+0000","last_fullsized":"2024-09-11T23:52:05.694869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746642+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:52:05.746642+0000","last_peered":"2024-09-11T23:52:05.746642+0000","last_clean":"2024-09-11T23:52:05.746642+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:52:05.746642+0000","last_undegraded":"2024-09-11T23:52:05.746642+0000","last_fullsized":"2024-09-11T23:52:05.746642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158655+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:52:06.158655+0000","last_peered":"2024-09-11T23:52:06.158655+0000","last_clean":"2024-09-11T23:52:06.158655+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:52:06.158655+0000","last_undegraded":"2024-09-11T23:52:06.158655+0000","last_fullsized":"2024-09-11T23:52:06.158655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433170+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:52:05.433170+0000","last_peered":"2024-09-11T23:52:05.433170+0000","last_clean":"2024-09-11T23:52:05.433170+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:52:05.433170+0000","last_undegraded":"2024-09-11T23:52:05.433170+0000","last_fullsized":"2024-09-11T23:52:05.433170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694598+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:52:05.694598+0000","last_peered":"2024-09-11T23:52:05.694598+0000","last_clean":"2024-09-11T23:52:05.694598+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:52:05.694598+0000","last_undegraded":"2024-09-11T23:52:05.694598+0000","last_fullsized":"2024-09-11T23:52:05.694598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'36","reported_seq":284,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.014229+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:52:06.014229+0000","last_peered":"2024-09-11T23:52:06.014229+0000","last_clean":"2024-09-11T23:52:06.014229+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:52:06.014229+0000","last_undegraded":"2024-09-11T23:52:06.014229+0000","last_fullsized":"2024-09-11T23:52:06.014229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66436,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":224,"num_read_kb":176,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516931+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:52:05.516931+0000","last_peered":"2024-09-11T23:52:05.516931+0000","last_clean":"2024-09-11T23:52:05.516931+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:52:05.516931+0000","last_undegraded":"2024-09-11T23:52:05.516931+0000","last_fullsized":"2024-09-11T23:52:05.516931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694704+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:52:05.694704+0000","last_peered":"2024-09-11T23:52:05.694704+0000","last_clean":"2024-09-11T23:52:05.694704+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:52:05.694704+0000","last_undegraded":"2024-09-11T23:52:05.694704+0000","last_fullsized":"2024-09-11T23:52:05.694704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431407+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:52:05.431407+0000","last_peered":"2024-09-11T23:52:05.431407+0000","last_clean":"2024-09-11T23:52:05.431407+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:52:05.431407+0000","last_undegraded":"2024-09-11T23:52:05.431407+0000","last_fullsized":"2024-09-11T23:52:05.431407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433521+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:52:05.433521+0000","last_peered":"2024-09-11T23:52:05.433521+0000","last_clean":"2024-09-11T23:52:05.433521+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:52:05.433521+0000","last_undegraded":"2024-09-11T23:52:05.433521+0000","last_fullsized":"2024-09-11T23:52:05.433521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746620+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:52:05.746620+0000","last_peered":"2024-09-11T23:52:05.746620+0000","last_clean":"2024-09-11T23:52:05.746620+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:52:05.746620+0000","last_undegraded":"2024-09-11T23:52:05.746620+0000","last_fullsized":"2024-09-11T23:52:05.746620+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.014353+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:52:06.014353+0000","last_peered":"2024-09-11T23:52:06.014353+0000","last_clean":"2024-09-11T23:52:06.014353+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:52:06.014353+0000","last_undegraded":"2024-09-11T23:52:06.014353+0000","last_fullsized":"2024-09-11T23:52:06.014353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746281+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:52:05.746281+0000","last_peered":"2024-09-11T23:52:05.746281+0000","last_clean":"2024-09-11T23:52:05.746281+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:52:05.746281+0000","last_undegraded":"2024-09-11T23:52:05.746281+0000","last_fullsized":"2024-09-11T23:52:05.746281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516591+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:52:05.516591+0000","last_peered":"2024-09-11T23:52:05.516591+0000","last_clean":"2024-09-11T23:52:05.516591+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:52:05.516591+0000","last_undegraded":"2024-09-11T23:52:05.516591+0000","last_fullsized":"2024-09-11T23:52:05.516591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432801+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:52:05.432801+0000","last_peered":"2024-09-11T23:52:05.432801+0000","last_clean":"2024-09-11T23:52:05.432801+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:52:05.432801+0000","last_undegraded":"2024-09-11T23:52:05.432801+0000","last_fullsized":"2024-09-11T23:52:05.432801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516946+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:52:05.516946+0000","last_peered":"2024-09-11T23:52:05.516946+0000","last_clean":"2024-09-11T23:52:05.516946+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:52:05.516946+0000","last_undegraded":"2024-09-11T23:52:05.516946+0000","last_fullsized":"2024-09-11T23:52:05.516946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872534+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:52:05.872534+0000","last_peered":"2024-09-11T23:52:05.872534+0000","last_clean":"2024-09-11T23:52:05.872534+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:52:05.872534+0000","last_undegraded":"2024-09-11T23:52:05.872534+0000","last_fullsized":"2024-09-11T23:52:05.872534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431687+0000","last_change":"2024-09-11T23:51:14.413526+0000","last_active":"2024-09-11T23:52:05.431687+0000","last_peered":"2024-09-11T23:52:05.431687+0000","last_clean":"2024-09-11T23:52:05.431687+0000","last_became_active":"2024-09-11T23:51:14.411027+0000","last_became_peered":"2024-09-11T23:51:14.411027+0000","last_unstale":"2024-09-11T23:52:05.431687+0000","last_undegraded":"2024-09-11T23:52:05.431687+0000","last_fullsized":"2024-09-11T23:52:05.431687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:55.020567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746675+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:52:05.746675+0000","last_peered":"2024-09-11T23:52:05.746675+0000","last_clean":"2024-09-11T23:52:05.746675+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:52:05.746675+0000","last_undegraded":"2024-09-11T23:52:05.746675+0000","last_fullsized":"2024-09-11T23:52:05.746675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.642614+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:52:05.642614+0000","last_peered":"2024-09-11T23:52:05.642614+0000","last_clean":"2024-09-11T23:52:05.642614+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:52:05.642614+0000","last_undegraded":"2024-09-11T23:52:05.642614+0000","last_fullsized":"2024-09-11T23:52:05.642614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746458+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:52:05.746458+0000","last_peered":"2024-09-11T23:52:05.746458+0000","last_clean":"2024-09-11T23:52:05.746458+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:52:05.746458+0000","last_undegraded":"2024-09-11T23:52:05.746458+0000","last_fullsized":"2024-09-11T23:52:05.746458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694804+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:52:05.694804+0000","last_peered":"2024-09-11T23:52:05.694804+0000","last_clean":"2024-09-11T23:52:05.694804+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:52:05.694804+0000","last_undegraded":"2024-09-11T23:52:05.694804+0000","last_fullsized":"2024-09-11T23:52:05.694804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746570+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:52:05.746570+0000","last_peered":"2024-09-11T23:52:05.746570+0000","last_clean":"2024-09-11T23:52:05.746570+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:52:05.746570+0000","last_undegraded":"2024-09-11T23:52:05.746570+0000","last_fullsized":"2024-09-11T23:52:05.746570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746409+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:52:05.746409+0000","last_peered":"2024-09-11T23:52:05.746409+0000","last_clean":"2024-09-11T23:52:05.746409+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:52:05.746409+0000","last_undegraded":"2024-09-11T23:52:05.746409+0000","last_fullsized":"2024-09-11T23:52:05.746409+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744872+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:52:05.744872+0000","last_peered":"2024-09-11T23:52:05.744872+0000","last_clean":"2024-09-11T23:52:05.744872+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:52:05.744872+0000","last_undegraded":"2024-09-11T23:52:05.744872+0000","last_fullsized":"2024-09-11T23:52:05.744872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523710+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:52:05.523710+0000","last_peered":"2024-09-11T23:52:05.523710+0000","last_clean":"2024-09-11T23:52:05.523710+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:52:05.523710+0000","last_undegraded":"2024-09-11T23:52:05.523710+0000","last_fullsized":"2024-09-11T23:52:05.523710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872415+0000","last_change":"2024-09-11T23:51:14.121480+0000","last_active":"2024-09-11T23:52:05.872415+0000","last_peered":"2024-09-11T23:52:05.872415+0000","last_clean":"2024-09-11T23:52:05.872415+0000","last_became_active":"2024-09-11T23:51:14.118435+0000","last_became_peered":"2024-09-11T23:51:14.118435+0000","last_unstale":"2024-09-11T23:52:05.872415+0000","last_undegraded":"2024-09-11T23:52:05.872415+0000","last_fullsized":"2024-09-11T23:52:05.872415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431342+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:52:05.431342+0000","last_peered":"2024-09-11T23:52:05.431342+0000","last_clean":"2024-09-11T23:52:05.431342+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:52:05.431342+0000","last_undegraded":"2024-09-11T23:52:05.431342+0000","last_fullsized":"2024-09-11T23:52:05.431342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":136,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431550+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:52:05.431550+0000","last_peered":"2024-09-11T23:52:05.431550+0000","last_clean":"2024-09-11T23:52:05.431550+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:52:05.431550+0000","last_undegraded":"2024-09-11T23:52:05.431550+0000","last_fullsized":"2024-09-11T23:52:05.431550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432859+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:52:05.432859+0000","last_peered":"2024-09-11T23:52:05.432859+0000","last_clean":"2024-09-11T23:52:05.432859+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:52:05.432859+0000","last_undegraded":"2024-09-11T23:52:05.432859+0000","last_fullsized":"2024-09-11T23:52:05.432859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746522+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:52:05.746522+0000","last_peered":"2024-09-11T23:52:05.746522+0000","last_clean":"2024-09-11T23:52:05.746522+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:52:05.746522+0000","last_undegraded":"2024-09-11T23:52:05.746522+0000","last_fullsized":"2024-09-11T23:52:05.746522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435012+0000","last_change":"2024-09-11T23:51:14.409908+0000","last_active":"2024-09-11T23:52:05.435012+0000","last_peered":"2024-09-11T23:52:05.435012+0000","last_clean":"2024-09-11T23:52:05.435012+0000","last_became_active":"2024-09-11T23:51:14.409432+0000","last_became_peered":"2024-09-11T23:51:14.409432+0000","last_unstale":"2024-09-11T23:52:05.435012+0000","last_undegraded":"2024-09-11T23:52:05.435012+0000","last_fullsized":"2024-09-11T23:52:05.435012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:06.371306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.521172+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:52:05.521172+0000","last_peered":"2024-09-11T23:52:05.521172+0000","last_clean":"2024-09-11T23:52:05.521172+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:52:05.521172+0000","last_undegraded":"2024-09-11T23:52:05.521172+0000","last_fullsized":"2024-09-11T23:52:05.521172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435167+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:52:05.435167+0000","last_peered":"2024-09-11T23:52:05.435167+0000","last_clean":"2024-09-11T23:52:05.435167+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:52:05.435167+0000","last_undegraded":"2024-09-11T23:52:05.435167+0000","last_fullsized":"2024-09-11T23:52:05.435167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.517017+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:52:05.517017+0000","last_peered":"2024-09-11T23:52:05.517017+0000","last_clean":"2024-09-11T23:52:05.517017+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:52:05.517017+0000","last_undegraded":"2024-09-11T23:52:05.517017+0000","last_fullsized":"2024-09-11T23:52:05.517017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431772+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:52:05.431772+0000","last_peered":"2024-09-11T23:52:05.431772+0000","last_clean":"2024-09-11T23:52:05.431772+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:52:05.431772+0000","last_undegraded":"2024-09-11T23:52:05.431772+0000","last_fullsized":"2024-09-11T23:52:05.431772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'48","reported_seq":95,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.649913+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:52:05.649913+0000","last_peered":"2024-09-11T23:52:05.649913+0000","last_clean":"2024-09-11T23:52:05.649913+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:52:05.649913+0000","last_undegraded":"2024-09-11T23:52:05.649913+0000","last_fullsized":"2024-09-11T23:52:05.649913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525022+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:52:05.525022+0000","last_peered":"2024-09-11T23:52:05.525022+0000","last_clean":"2024-09-11T23:52:05.525022+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:52:05.525022+0000","last_undegraded":"2024-09-11T23:52:05.525022+0000","last_fullsized":"2024-09-11T23:52:05.525022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.745108+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:52:05.745108+0000","last_peered":"2024-09-11T23:52:05.745108+0000","last_clean":"2024-09-11T23:52:05.745108+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:52:05.745108+0000","last_undegraded":"2024-09-11T23:52:05.745108+0000","last_fullsized":"2024-09-11T23:52:05.745108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158521+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:52:06.158521+0000","last_peered":"2024-09-11T23:52:06.158521+0000","last_clean":"2024-09-11T23:52:06.158521+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:52:06.158521+0000","last_undegraded":"2024-09-11T23:52:06.158521+0000","last_fullsized":"2024-09-11T23:52:06.158521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694501+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:52:05.694501+0000","last_peered":"2024-09-11T23:52:05.694501+0000","last_clean":"2024-09-11T23:52:05.694501+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:52:05.694501+0000","last_undegraded":"2024-09-11T23:52:05.694501+0000","last_fullsized":"2024-09-11T23:52:05.694501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516588+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:52:05.516588+0000","last_peered":"2024-09-11T23:52:05.516588+0000","last_clean":"2024-09-11T23:52:05.516588+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:52:05.516588+0000","last_undegraded":"2024-09-11T23:52:05.516588+0000","last_fullsized":"2024-09-11T23:52:05.516588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523605+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:52:05.523605+0000","last_peered":"2024-09-11T23:52:05.523605+0000","last_clean":"2024-09-11T23:52:05.523605+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:52:05.523605+0000","last_undegraded":"2024-09-11T23:52:05.523605+0000","last_fullsized":"2024-09-11T23:52:05.523605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872645+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:52:05.872645+0000","last_peered":"2024-09-11T23:52:05.872645+0000","last_clean":"2024-09-11T23:52:05.872645+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:52:05.872645+0000","last_undegraded":"2024-09-11T23:52:05.872645+0000","last_fullsized":"2024-09-11T23:52:05.872645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872496+0000","last_change":"2024-09-11T23:51:14.123663+0000","last_active":"2024-09-11T23:52:05.872496+0000","last_peered":"2024-09-11T23:52:05.872496+0000","last_clean":"2024-09-11T23:52:05.872496+0000","last_became_active":"2024-09-11T23:51:14.118315+0000","last_became_peered":"2024-09-11T23:51:14.118315+0000","last_unstale":"2024-09-11T23:52:05.872496+0000","last_undegraded":"2024-09-11T23:52:05.872496+0000","last_fullsized":"2024-09-11T23:52:05.872496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:10.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.524009+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:52:05.524009+0000","last_peered":"2024-09-11T23:52:05.524009+0000","last_clean":"2024-09-11T23:52:05.524009+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:52:05.524009+0000","last_undegraded":"2024-09-11T23:52:05.524009+0000","last_fullsized":"2024-09-11T23:52:05.524009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516698+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:52:05.516698+0000","last_peered":"2024-09-11T23:52:05.516698+0000","last_clean":"2024-09-11T23:52:05.516698+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:52:05.516698+0000","last_undegraded":"2024-09-11T23:52:05.516698+0000","last_fullsized":"2024-09-11T23:52:05.516698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746447+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:52:05.746447+0000","last_peered":"2024-09-11T23:52:05.746447+0000","last_clean":"2024-09-11T23:52:05.746447+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:52:05.746447+0000","last_undegraded":"2024-09-11T23:52:05.746447+0000","last_fullsized":"2024-09-11T23:52:05.746447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433119+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:52:05.433119+0000","last_peered":"2024-09-11T23:52:05.433119+0000","last_clean":"2024-09-11T23:52:05.433119+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:52:05.433119+0000","last_undegraded":"2024-09-11T23:52:05.433119+0000","last_fullsized":"2024-09-11T23:52:05.433119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516897+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:52:05.516897+0000","last_peered":"2024-09-11T23:52:05.516897+0000","last_clean":"2024-09-11T23:52:05.516897+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:52:05.516897+0000","last_undegraded":"2024-09-11T23:52:05.516897+0000","last_fullsized":"2024-09-11T23:52:05.516897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694561+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:52:05.694561+0000","last_peered":"2024-09-11T23:52:05.694561+0000","last_clean":"2024-09-11T23:52:05.694561+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:52:05.694561+0000","last_undegraded":"2024-09-11T23:52:05.694561+0000","last_fullsized":"2024-09-11T23:52:05.694561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516945+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:52:05.516945+0000","last_peered":"2024-09-11T23:52:05.516945+0000","last_clean":"2024-09-11T23:52:05.516945+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:52:05.516945+0000","last_undegraded":"2024-09-11T23:52:05.516945+0000","last_fullsized":"2024-09-11T23:52:05.516945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158617+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:52:06.158617+0000","last_peered":"2024-09-11T23:52:06.158617+0000","last_clean":"2024-09-11T23:52:06.158617+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:52:06.158617+0000","last_undegraded":"2024-09-11T23:52:06.158617+0000","last_fullsized":"2024-09-11T23:52:06.158617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525435+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:52:05.525435+0000","last_peered":"2024-09-11T23:52:05.525435+0000","last_clean":"2024-09-11T23:52:05.525435+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:52:05.525435+0000","last_undegraded":"2024-09-11T23:52:05.525435+0000","last_fullsized":"2024-09-11T23:52:05.525435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525158+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:52:05.525158+0000","last_peered":"2024-09-11T23:52:05.525158+0000","last_clean":"2024-09-11T23:52:05.525158+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:52:05.525158+0000","last_undegraded":"2024-09-11T23:52:05.525158+0000","last_fullsized":"2024-09-11T23:52:05.525158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.521532+0000","last_change":"2024-09-11T23:51:14.125596+0000","last_active":"2024-09-11T23:52:05.521532+0000","last_peered":"2024-09-11T23:52:05.521532+0000","last_clean":"2024-09-11T23:52:05.521532+0000","last_became_active":"2024-09-11T23:51:14.119682+0000","last_became_peered":"2024-09-11T23:51:14.119682+0000","last_unstale":"2024-09-11T23:52:05.521532+0000","last_undegraded":"2024-09-11T23:52:05.521532+0000","last_fullsized":"2024-09-11T23:52:05.521532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:06.062602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596490,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":763,"num_read_kb":490,"num_write":295,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1789470,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":436969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1089536,"data_stored":1236648,"data_compressed":44911,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":839348,"data_compressed":22046,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":838686,"data_compressed":22046,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1039219,"data_compressed":38755,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445521,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1169268,"data_compressed":44911,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":447193,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":974848,"data_stored":970648,"data_compressed":28202,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":513493,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":498698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":437865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133469,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":200849,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:52:09.085 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-11T23:52:09.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:52:09.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.229+0000 7f9d94511640 1 -- 172.21.15.62:0/2284395132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d8c101c00 msgr2=0x7f9d8c102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:09.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.229+0000 7f9d94511640 1 --2- 172.21.15.62:0/2284395132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d8c101c00 0x7f9d8c102080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009900 tx=0x7f9d8002a410 comp rx=0 tx=0).stop 2024-09-11T23:52:09.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.229+0000 7f9d94511640 1 -- 172.21.15.62:0/2284395132 shutdown_connections 2024-09-11T23:52:09.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.229+0000 7f9d94511640 1 --2- 172.21.15.62:0/2284395132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c1025c0 0x7f9d8c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.229+0000 7f9d94511640 1 --2- 172.21.15.62:0/2284395132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d8c101c00 0x7f9d8c102080 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.229+0000 7f9d94511640 1 --2- 172.21.15.62:0/2284395132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d8c100a10 0x7f9d8c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.229+0000 7f9d94511640 1 -- 172.21.15.62:0/2284395132 >> 172.21.15.62:0/2284395132 conn(0x7f9d8c0fc170 msgr2=0x7f9d8c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:09.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.229+0000 7f9d94511640 1 -- 172.21.15.62:0/2284395132 shutdown_connections 2024-09-11T23:52:09.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.229+0000 7f9d94511640 1 -- 172.21.15.62:0/2284395132 wait complete. 2024-09-11T23:52:09.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d94511640 1 Processor -- start 2024-09-11T23:52:09.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d94511640 1 -- start start 2024-09-11T23:52:09.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d94511640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c100a10 0x7f9d8c069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:09.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d94511640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d8c101c00 0x7f9d8c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:09.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d94511640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d8c1025c0 0x7f9d8c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:09.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d94511640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c10c6e0 con 0x7f9d8c1025c0 2024-09-11T23:52:09.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d94511640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c10c560 con 0x7f9d8c100a10 2024-09-11T23:52:09.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d94511640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c10c860 con 0x7f9d8c101c00 2024-09-11T23:52:09.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d92aae640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d8c1025c0 0x7f9d8c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:09.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d92aae640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d8c1025c0 0x7f9d8c19f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45966/0 (socket says 172.21.15.62:45966) 2024-09-11T23:52:09.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d92aae640 1 -- 172.21.15.62:0/910462021 learned_addr learned my addr 172.21.15.62:0/910462021 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:09.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d922ad640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c100a10 0x7f9d8c069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:09.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d91aac640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d8c101c00 0x7f9d8c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:09.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d92aae640 1 -- 172.21.15.62:0/910462021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d8c101c00 msgr2=0x7f9d8c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:09.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d92aae640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d8c101c00 0x7f9d8c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d92aae640 1 -- 172.21.15.62:0/910462021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c100a10 msgr2=0x7f9d8c069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d92aae640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c100a10 0x7f9d8c069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d92aae640 1 -- 172.21.15.62:0/910462021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c19fae0 con 0x7f9d8c1025c0 2024-09-11T23:52:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d91aac640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d8c101c00 0x7f9d8c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.230+0000 7f9d922ad640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c100a10 0x7f9d8c069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.231+0000 7f9d92aae640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d8c1025c0 0x7f9d8c19f1d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00eb10 tx=0x7f9d7c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.231+0000 7f9d8b7fe640 1 -- 172.21.15.62:0/910462021 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c004b20 con 0x7f9d8c1025c0 2024-09-11T23:52:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.231+0000 7f9d8b7fe640 1 -- 172.21.15.62:0/910462021 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c004cc0 con 0x7f9d8c1025c0 2024-09-11T23:52:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.231+0000 7f9d94511640 1 -- 172.21.15.62:0/910462021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c19fd70 con 0x7f9d8c1025c0 2024-09-11T23:52:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.231+0000 7f9d8b7fe640 1 -- 172.21.15.62:0/910462021 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c010760 con 0x7f9d8c1025c0 2024-09-11T23:52:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.231+0000 7f9d94511640 1 -- 172.21.15.62:0/910462021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c1a0230 con 0x7f9d8c1025c0 2024-09-11T23:52:09.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.232+0000 7f9d94511640 1 -- 172.21.15.62:0/910462021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8c107e50 con 0x7f9d8c1025c0 2024-09-11T23:52:09.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.233+0000 7f9d8b7fe640 1 -- 172.21.15.62:0/910462021 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f9d7c005020 con 0x7f9d8c1025c0 2024-09-11T23:52:09.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.234+0000 7f9d8b7fe640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9d740a2d80 0x7f9d740a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:09.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.234+0000 7f9d922ad640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9d740a2d80 0x7f9d740a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:09.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.234+0000 7f9d8b7fe640 1 -- 172.21.15.62:0/910462021 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f9d7c0cfa00 con 0x7f9d8c1025c0 2024-09-11T23:52:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.235+0000 7f9d922ad640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9d740a2d80 0x7f9d740a5240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c101a60 tx=0x7f9d84005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.238+0000 7f9d8b7fe640 1 -- 172.21.15.62:0/910462021 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d7c09c650 con 0x7f9d8c1025c0 2024-09-11T23:52:09.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.412+0000 7f9d94511640 1 -- 172.21.15.62:0/910462021 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d8c062630 con 0x7f9d740a2d80 2024-09-11T23:52:09.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.425+0000 7f9d8b7fe640 1 -- 172.21.15.62:0/910462021 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257965 (secure 0 0 0) 0x7f9d8c062630 con 0x7f9d740a2d80 2024-09-11T23:52:09.428 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:09.429 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:52:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 -- 172.21.15.62:0/910462021 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9d740a2d80 msgr2=0x7f9d740a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:09.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9d740a2d80 0x7f9d740a5240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c101a60 tx=0x7f9d84005f90 comp rx=0 tx=0).stop 2024-09-11T23:52:09.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 -- 172.21.15.62:0/910462021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d8c1025c0 msgr2=0x7f9d8c19f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:09.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d8c1025c0 0x7f9d8c19f1d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00eb10 tx=0x7f9d7c00c2d0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 -- 172.21.15.62:0/910462021 shutdown_connections 2024-09-11T23:52:09.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9d740a2d80 0x7f9d740a5240 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d8c1025c0 0x7f9d8c19f1d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d8c101c00 0x7f9d8c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 --2- 172.21.15.62:0/910462021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c100a10 0x7f9d8c069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 -- 172.21.15.62:0/910462021 >> 172.21.15.62:0/910462021 conn(0x7f9d8c0fc170 msgr2=0x7f9d8c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 -- 172.21.15.62:0/910462021 shutdown_connections 2024-09-11T23:52:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:09.428+0000 7f9d94511640 1 -- 172.21.15.62:0/910462021 wait complete. 2024-09-11T23:52:09.440 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T23:52:07.445934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187092,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":899,"num_read_kb":609,"num_write":478,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348104,"kb_used_data":11296,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593720,"statfs":{"total":1151940427776,"available":1151583969280,"internally_reserved":0,"allocated":11567104,"data_stored":8873556,"data_compressed":209571,"data_compressed_allocated":270336,"data_compressed_original":3391488,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.977168"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872692+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:52:05.872692+0000","last_peered":"2024-09-11T23:52:05.872692+0000","last_clean":"2024-09-11T23:52:05.872692+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:52:05.872692+0000","last_undegraded":"2024-09-11T23:52:05.872692+0000","last_fullsized":"2024-09-11T23:52:05.872692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158483+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:52:06.158483+0000","last_peered":"2024-09-11T23:52:06.158483+0000","last_clean":"2024-09-11T23:52:06.158483+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:52:06.158483+0000","last_undegraded":"2024-09-11T23:52:06.158483+0000","last_fullsized":"2024-09-11T23:52:06.158483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158255+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:52:06.158255+0000","last_peered":"2024-09-11T23:52:06.158255+0000","last_clean":"2024-09-11T23:52:06.158255+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:52:06.158255+0000","last_undegraded":"2024-09-11T23:52:06.158255+0000","last_fullsized":"2024-09-11T23:52:06.158255+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744925+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:52:05.744925+0000","last_peered":"2024-09-11T23:52:05.744925+0000","last_clean":"2024-09-11T23:52:05.744925+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:52:05.744925+0000","last_undegraded":"2024-09-11T23:52:05.744925+0000","last_fullsized":"2024-09-11T23:52:05.744925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516648+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:52:05.516648+0000","last_peered":"2024-09-11T23:52:05.516648+0000","last_clean":"2024-09-11T23:52:05.516648+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:52:05.516648+0000","last_undegraded":"2024-09-11T23:52:05.516648+0000","last_fullsized":"2024-09-11T23:52:05.516648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434378+0000","last_change":"2024-09-11T23:51:14.120226+0000","last_active":"2024-09-11T23:52:05.434378+0000","last_peered":"2024-09-11T23:52:05.434378+0000","last_clean":"2024-09-11T23:52:05.434378+0000","last_became_active":"2024-09-11T23:51:14.119811+0000","last_became_peered":"2024-09-11T23:51:14.119811+0000","last_unstale":"2024-09-11T23:52:05.434378+0000","last_undegraded":"2024-09-11T23:52:05.434378+0000","last_fullsized":"2024-09-11T23:52:05.434378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:58.851476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746317+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:52:05.746317+0000","last_peered":"2024-09-11T23:52:05.746317+0000","last_clean":"2024-09-11T23:52:05.746317+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:52:05.746317+0000","last_undegraded":"2024-09-11T23:52:05.746317+0000","last_fullsized":"2024-09-11T23:52:05.746317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523319+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:52:05.523319+0000","last_peered":"2024-09-11T23:52:05.523319+0000","last_clean":"2024-09-11T23:52:05.523319+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:52:05.523319+0000","last_undegraded":"2024-09-11T23:52:05.523319+0000","last_fullsized":"2024-09-11T23:52:05.523319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694954+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:52:05.694954+0000","last_peered":"2024-09-11T23:52:05.694954+0000","last_clean":"2024-09-11T23:52:05.694954+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:52:05.694954+0000","last_undegraded":"2024-09-11T23:52:05.694954+0000","last_fullsized":"2024-09-11T23:52:05.694954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435402+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:52:05.435402+0000","last_peered":"2024-09-11T23:52:05.435402+0000","last_clean":"2024-09-11T23:52:05.435402+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:52:05.435402+0000","last_undegraded":"2024-09-11T23:52:05.435402+0000","last_fullsized":"2024-09-11T23:52:05.435402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746473+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:52:05.746473+0000","last_peered":"2024-09-11T23:52:05.746473+0000","last_clean":"2024-09-11T23:52:05.746473+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:52:05.746473+0000","last_undegraded":"2024-09-11T23:52:05.746473+0000","last_fullsized":"2024-09-11T23:52:05.746473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744935+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:52:05.744935+0000","last_peered":"2024-09-11T23:52:05.744935+0000","last_clean":"2024-09-11T23:52:05.744935+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:52:05.744935+0000","last_undegraded":"2024-09-11T23:52:05.744935+0000","last_fullsized":"2024-09-11T23:52:05.744935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431713+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:52:05.431713+0000","last_peered":"2024-09-11T23:52:05.431713+0000","last_clean":"2024-09-11T23:52:05.431713+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:52:05.431713+0000","last_undegraded":"2024-09-11T23:52:05.431713+0000","last_fullsized":"2024-09-11T23:52:05.431713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158046+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:52:06.158046+0000","last_peered":"2024-09-11T23:52:06.158046+0000","last_clean":"2024-09-11T23:52:06.158046+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:52:06.158046+0000","last_undegraded":"2024-09-11T23:52:06.158046+0000","last_fullsized":"2024-09-11T23:52:06.158046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.157901+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:52:06.157901+0000","last_peered":"2024-09-11T23:52:06.157901+0000","last_clean":"2024-09-11T23:52:06.157901+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:52:06.157901+0000","last_undegraded":"2024-09-11T23:52:06.157901+0000","last_fullsized":"2024-09-11T23:52:06.157901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694419+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:52:05.694419+0000","last_peered":"2024-09-11T23:52:05.694419+0000","last_clean":"2024-09-11T23:52:05.694419+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:52:05.694419+0000","last_undegraded":"2024-09-11T23:52:05.694419+0000","last_fullsized":"2024-09-11T23:52:05.694419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872787+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:52:05.872787+0000","last_peered":"2024-09-11T23:52:05.872787+0000","last_clean":"2024-09-11T23:52:05.872787+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:52:05.872787+0000","last_undegraded":"2024-09-11T23:52:05.872787+0000","last_fullsized":"2024-09-11T23:52:05.872787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520950+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:52:05.520950+0000","last_peered":"2024-09-11T23:52:05.520950+0000","last_clean":"2024-09-11T23:52:05.520950+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:52:05.520950+0000","last_undegraded":"2024-09-11T23:52:05.520950+0000","last_fullsized":"2024-09-11T23:52:05.520950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694752+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:52:05.694752+0000","last_peered":"2024-09-11T23:52:05.694752+0000","last_clean":"2024-09-11T23:52:05.694752+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:52:05.694752+0000","last_undegraded":"2024-09-11T23:52:05.694752+0000","last_fullsized":"2024-09-11T23:52:05.694752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872460+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:52:05.872460+0000","last_peered":"2024-09-11T23:52:05.872460+0000","last_clean":"2024-09-11T23:52:05.872460+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:52:05.872460+0000","last_undegraded":"2024-09-11T23:52:05.872460+0000","last_fullsized":"2024-09-11T23:52:05.872460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431851+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:52:05.431851+0000","last_peered":"2024-09-11T23:52:05.431851+0000","last_clean":"2024-09-11T23:52:05.431851+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:52:05.431851+0000","last_undegraded":"2024-09-11T23:52:05.431851+0000","last_fullsized":"2024-09-11T23:52:05.431851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694603+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:52:05.694603+0000","last_peered":"2024-09-11T23:52:05.694603+0000","last_clean":"2024-09-11T23:52:05.694603+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:52:05.694603+0000","last_undegraded":"2024-09-11T23:52:05.694603+0000","last_fullsized":"2024-09-11T23:52:05.694603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523509+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:52:05.523509+0000","last_peered":"2024-09-11T23:52:05.523509+0000","last_clean":"2024-09-11T23:52:05.523509+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:52:05.523509+0000","last_undegraded":"2024-09-11T23:52:05.523509+0000","last_fullsized":"2024-09-11T23:52:05.523509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432812+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:52:05.432812+0000","last_peered":"2024-09-11T23:52:05.432812+0000","last_clean":"2024-09-11T23:52:05.432812+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:52:05.432812+0000","last_undegraded":"2024-09-11T23:52:05.432812+0000","last_fullsized":"2024-09-11T23:52:05.432812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525015+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:52:05.525015+0000","last_peered":"2024-09-11T23:52:05.525015+0000","last_clean":"2024-09-11T23:52:05.525015+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:52:05.525015+0000","last_undegraded":"2024-09-11T23:52:05.525015+0000","last_fullsized":"2024-09-11T23:52:05.525015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694943+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:52:05.694943+0000","last_peered":"2024-09-11T23:52:05.694943+0000","last_clean":"2024-09-11T23:52:05.694943+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:52:05.694943+0000","last_undegraded":"2024-09-11T23:52:05.694943+0000","last_fullsized":"2024-09-11T23:52:05.694943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872635+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:52:05.872635+0000","last_peered":"2024-09-11T23:52:05.872635+0000","last_clean":"2024-09-11T23:52:05.872635+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:52:05.872635+0000","last_undegraded":"2024-09-11T23:52:05.872635+0000","last_fullsized":"2024-09-11T23:52:05.872635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431698+0000","last_change":"2024-09-11T23:51:14.410108+0000","last_active":"2024-09-11T23:52:05.431698+0000","last_peered":"2024-09-11T23:52:05.431698+0000","last_clean":"2024-09-11T23:52:05.431698+0000","last_became_active":"2024-09-11T23:51:14.409588+0000","last_became_peered":"2024-09-11T23:51:14.409588+0000","last_unstale":"2024-09-11T23:52:05.431698+0000","last_undegraded":"2024-09-11T23:52:05.431698+0000","last_fullsized":"2024-09-11T23:52:05.431698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:29.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158121+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:52:06.158121+0000","last_peered":"2024-09-11T23:52:06.158121+0000","last_clean":"2024-09-11T23:52:06.158121+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:52:06.158121+0000","last_undegraded":"2024-09-11T23:52:06.158121+0000","last_fullsized":"2024-09-11T23:52:06.158121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694617+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:52:05.694617+0000","last_peered":"2024-09-11T23:52:05.694617+0000","last_clean":"2024-09-11T23:52:05.694617+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:52:05.694617+0000","last_undegraded":"2024-09-11T23:52:05.694617+0000","last_fullsized":"2024-09-11T23:52:05.694617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694660+0000","last_change":"2024-09-11T23:51:14.410872+0000","last_active":"2024-09-11T23:52:05.694660+0000","last_peered":"2024-09-11T23:52:05.694660+0000","last_clean":"2024-09-11T23:52:05.694660+0000","last_became_active":"2024-09-11T23:51:14.410531+0000","last_became_peered":"2024-09-11T23:51:14.410531+0000","last_unstale":"2024-09-11T23:52:05.694660+0000","last_undegraded":"2024-09-11T23:52:05.694660+0000","last_fullsized":"2024-09-11T23:52:05.694660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:10.502914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872498+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:52:05.872498+0000","last_peered":"2024-09-11T23:52:05.872498+0000","last_clean":"2024-09-11T23:52:05.872498+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:52:05.872498+0000","last_undegraded":"2024-09-11T23:52:05.872498+0000","last_fullsized":"2024-09-11T23:52:05.872498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516667+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:52:05.516667+0000","last_peered":"2024-09-11T23:52:05.516667+0000","last_clean":"2024-09-11T23:52:05.516667+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:52:05.516667+0000","last_undegraded":"2024-09-11T23:52:05.516667+0000","last_fullsized":"2024-09-11T23:52:05.516667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.014133+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:52:06.014133+0000","last_peered":"2024-09-11T23:52:06.014133+0000","last_clean":"2024-09-11T23:52:06.014133+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:52:06.014133+0000","last_undegraded":"2024-09-11T23:52:06.014133+0000","last_fullsized":"2024-09-11T23:52:06.014133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432914+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:52:05.432914+0000","last_peered":"2024-09-11T23:52:05.432914+0000","last_clean":"2024-09-11T23:52:05.432914+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:52:05.432914+0000","last_undegraded":"2024-09-11T23:52:05.432914+0000","last_fullsized":"2024-09-11T23:52:05.432914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434413+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:52:05.434413+0000","last_peered":"2024-09-11T23:52:05.434413+0000","last_clean":"2024-09-11T23:52:05.434413+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:52:05.434413+0000","last_undegraded":"2024-09-11T23:52:05.434413+0000","last_fullsized":"2024-09-11T23:52:05.434413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431356+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:52:05.431356+0000","last_peered":"2024-09-11T23:52:05.431356+0000","last_clean":"2024-09-11T23:52:05.431356+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:52:05.431356+0000","last_undegraded":"2024-09-11T23:52:05.431356+0000","last_fullsized":"2024-09-11T23:52:05.431356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523343+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:52:05.523343+0000","last_peered":"2024-09-11T23:52:05.523343+0000","last_clean":"2024-09-11T23:52:05.523343+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:52:05.523343+0000","last_undegraded":"2024-09-11T23:52:05.523343+0000","last_fullsized":"2024-09-11T23:52:05.523343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433385+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:52:05.433385+0000","last_peered":"2024-09-11T23:52:05.433385+0000","last_clean":"2024-09-11T23:52:05.433385+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:52:05.433385+0000","last_undegraded":"2024-09-11T23:52:05.433385+0000","last_fullsized":"2024-09-11T23:52:05.433385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520843+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:52:05.520843+0000","last_peered":"2024-09-11T23:52:05.520843+0000","last_clean":"2024-09-11T23:52:05.520843+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:52:05.520843+0000","last_undegraded":"2024-09-11T23:52:05.520843+0000","last_fullsized":"2024-09-11T23:52:05.520843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":134,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.748619+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:52:05.748619+0000","last_peered":"2024-09-11T23:52:05.748619+0000","last_clean":"2024-09-11T23:52:05.748619+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:52:05.748619+0000","last_undegraded":"2024-09-11T23:52:05.748619+0000","last_fullsized":"2024-09-11T23:52:05.748619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694869+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:52:05.694869+0000","last_peered":"2024-09-11T23:52:05.694869+0000","last_clean":"2024-09-11T23:52:05.694869+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:52:05.694869+0000","last_undegraded":"2024-09-11T23:52:05.694869+0000","last_fullsized":"2024-09-11T23:52:05.694869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746642+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:52:05.746642+0000","last_peered":"2024-09-11T23:52:05.746642+0000","last_clean":"2024-09-11T23:52:05.746642+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:52:05.746642+0000","last_undegraded":"2024-09-11T23:52:05.746642+0000","last_fullsized":"2024-09-11T23:52:05.746642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158655+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:52:06.158655+0000","last_peered":"2024-09-11T23:52:06.158655+0000","last_clean":"2024-09-11T23:52:06.158655+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:52:06.158655+0000","last_undegraded":"2024-09-11T23:52:06.158655+0000","last_fullsized":"2024-09-11T23:52:06.158655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433170+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:52:05.433170+0000","last_peered":"2024-09-11T23:52:05.433170+0000","last_clean":"2024-09-11T23:52:05.433170+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:52:05.433170+0000","last_undegraded":"2024-09-11T23:52:05.433170+0000","last_fullsized":"2024-09-11T23:52:05.433170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694598+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:52:05.694598+0000","last_peered":"2024-09-11T23:52:05.694598+0000","last_clean":"2024-09-11T23:52:05.694598+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:52:05.694598+0000","last_undegraded":"2024-09-11T23:52:05.694598+0000","last_fullsized":"2024-09-11T23:52:05.694598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'36","reported_seq":284,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.014229+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:52:06.014229+0000","last_peered":"2024-09-11T23:52:06.014229+0000","last_clean":"2024-09-11T23:52:06.014229+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:52:06.014229+0000","last_undegraded":"2024-09-11T23:52:06.014229+0000","last_fullsized":"2024-09-11T23:52:06.014229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66436,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":224,"num_read_kb":176,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516931+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:52:05.516931+0000","last_peered":"2024-09-11T23:52:05.516931+0000","last_clean":"2024-09-11T23:52:05.516931+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:52:05.516931+0000","last_undegraded":"2024-09-11T23:52:05.516931+0000","last_fullsized":"2024-09-11T23:52:05.516931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694704+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:52:05.694704+0000","last_peered":"2024-09-11T23:52:05.694704+0000","last_clean":"2024-09-11T23:52:05.694704+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:52:05.694704+0000","last_undegraded":"2024-09-11T23:52:05.694704+0000","last_fullsized":"2024-09-11T23:52:05.694704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431407+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:52:05.431407+0000","last_peered":"2024-09-11T23:52:05.431407+0000","last_clean":"2024-09-11T23:52:05.431407+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:52:05.431407+0000","last_undegraded":"2024-09-11T23:52:05.431407+0000","last_fullsized":"2024-09-11T23:52:05.431407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433521+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:52:05.433521+0000","last_peered":"2024-09-11T23:52:05.433521+0000","last_clean":"2024-09-11T23:52:05.433521+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:52:05.433521+0000","last_undegraded":"2024-09-11T23:52:05.433521+0000","last_fullsized":"2024-09-11T23:52:05.433521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746620+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:52:05.746620+0000","last_peered":"2024-09-11T23:52:05.746620+0000","last_clean":"2024-09-11T23:52:05.746620+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:52:05.746620+0000","last_undegraded":"2024-09-11T23:52:05.746620+0000","last_fullsized":"2024-09-11T23:52:05.746620+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.014353+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:52:06.014353+0000","last_peered":"2024-09-11T23:52:06.014353+0000","last_clean":"2024-09-11T23:52:06.014353+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:52:06.014353+0000","last_undegraded":"2024-09-11T23:52:06.014353+0000","last_fullsized":"2024-09-11T23:52:06.014353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746281+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:52:05.746281+0000","last_peered":"2024-09-11T23:52:05.746281+0000","last_clean":"2024-09-11T23:52:05.746281+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:52:05.746281+0000","last_undegraded":"2024-09-11T23:52:05.746281+0000","last_fullsized":"2024-09-11T23:52:05.746281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516591+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:52:05.516591+0000","last_peered":"2024-09-11T23:52:05.516591+0000","last_clean":"2024-09-11T23:52:05.516591+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:52:05.516591+0000","last_undegraded":"2024-09-11T23:52:05.516591+0000","last_fullsized":"2024-09-11T23:52:05.516591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432801+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:52:05.432801+0000","last_peered":"2024-09-11T23:52:05.432801+0000","last_clean":"2024-09-11T23:52:05.432801+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:52:05.432801+0000","last_undegraded":"2024-09-11T23:52:05.432801+0000","last_fullsized":"2024-09-11T23:52:05.432801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516946+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:52:05.516946+0000","last_peered":"2024-09-11T23:52:05.516946+0000","last_clean":"2024-09-11T23:52:05.516946+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:52:05.516946+0000","last_undegraded":"2024-09-11T23:52:05.516946+0000","last_fullsized":"2024-09-11T23:52:05.516946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872534+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:52:05.872534+0000","last_peered":"2024-09-11T23:52:05.872534+0000","last_clean":"2024-09-11T23:52:05.872534+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:52:05.872534+0000","last_undegraded":"2024-09-11T23:52:05.872534+0000","last_fullsized":"2024-09-11T23:52:05.872534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431687+0000","last_change":"2024-09-11T23:51:14.413526+0000","last_active":"2024-09-11T23:52:05.431687+0000","last_peered":"2024-09-11T23:52:05.431687+0000","last_clean":"2024-09-11T23:52:05.431687+0000","last_became_active":"2024-09-11T23:51:14.411027+0000","last_became_peered":"2024-09-11T23:51:14.411027+0000","last_unstale":"2024-09-11T23:52:05.431687+0000","last_undegraded":"2024-09-11T23:52:05.431687+0000","last_fullsized":"2024-09-11T23:52:05.431687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:55.020567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746675+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:52:05.746675+0000","last_peered":"2024-09-11T23:52:05.746675+0000","last_clean":"2024-09-11T23:52:05.746675+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:52:05.746675+0000","last_undegraded":"2024-09-11T23:52:05.746675+0000","last_fullsized":"2024-09-11T23:52:05.746675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.642614+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:52:05.642614+0000","last_peered":"2024-09-11T23:52:05.642614+0000","last_clean":"2024-09-11T23:52:05.642614+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:52:05.642614+0000","last_undegraded":"2024-09-11T23:52:05.642614+0000","last_fullsized":"2024-09-11T23:52:05.642614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746458+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:52:05.746458+0000","last_peered":"2024-09-11T23:52:05.746458+0000","last_clean":"2024-09-11T23:52:05.746458+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:52:05.746458+0000","last_undegraded":"2024-09-11T23:52:05.746458+0000","last_fullsized":"2024-09-11T23:52:05.746458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694804+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:52:05.694804+0000","last_peered":"2024-09-11T23:52:05.694804+0000","last_clean":"2024-09-11T23:52:05.694804+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:52:05.694804+0000","last_undegraded":"2024-09-11T23:52:05.694804+0000","last_fullsized":"2024-09-11T23:52:05.694804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746570+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:52:05.746570+0000","last_peered":"2024-09-11T23:52:05.746570+0000","last_clean":"2024-09-11T23:52:05.746570+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:52:05.746570+0000","last_undegraded":"2024-09-11T23:52:05.746570+0000","last_fullsized":"2024-09-11T23:52:05.746570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746409+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:52:05.746409+0000","last_peered":"2024-09-11T23:52:05.746409+0000","last_clean":"2024-09-11T23:52:05.746409+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:52:05.746409+0000","last_undegraded":"2024-09-11T23:52:05.746409+0000","last_fullsized":"2024-09-11T23:52:05.746409+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744872+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:52:05.744872+0000","last_peered":"2024-09-11T23:52:05.744872+0000","last_clean":"2024-09-11T23:52:05.744872+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:52:05.744872+0000","last_undegraded":"2024-09-11T23:52:05.744872+0000","last_fullsized":"2024-09-11T23:52:05.744872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523710+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:52:05.523710+0000","last_peered":"2024-09-11T23:52:05.523710+0000","last_clean":"2024-09-11T23:52:05.523710+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:52:05.523710+0000","last_undegraded":"2024-09-11T23:52:05.523710+0000","last_fullsized":"2024-09-11T23:52:05.523710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872415+0000","last_change":"2024-09-11T23:51:14.121480+0000","last_active":"2024-09-11T23:52:05.872415+0000","last_peered":"2024-09-11T23:52:05.872415+0000","last_clean":"2024-09-11T23:52:05.872415+0000","last_became_active":"2024-09-11T23:51:14.118435+0000","last_became_peered":"2024-09-11T23:51:14.118435+0000","last_unstale":"2024-09-11T23:52:05.872415+0000","last_undegraded":"2024-09-11T23:52:05.872415+0000","last_fullsized":"2024-09-11T23:52:05.872415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431342+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:52:05.431342+0000","last_peered":"2024-09-11T23:52:05.431342+0000","last_clean":"2024-09-11T23:52:05.431342+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:52:05.431342+0000","last_undegraded":"2024-09-11T23:52:05.431342+0000","last_fullsized":"2024-09-11T23:52:05.431342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":136,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431550+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:52:05.431550+0000","last_peered":"2024-09-11T23:52:05.431550+0000","last_clean":"2024-09-11T23:52:05.431550+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:52:05.431550+0000","last_undegraded":"2024-09-11T23:52:05.431550+0000","last_fullsized":"2024-09-11T23:52:05.431550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432859+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:52:05.432859+0000","last_peered":"2024-09-11T23:52:05.432859+0000","last_clean":"2024-09-11T23:52:05.432859+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:52:05.432859+0000","last_undegraded":"2024-09-11T23:52:05.432859+0000","last_fullsized":"2024-09-11T23:52:05.432859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746522+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:52:05.746522+0000","last_peered":"2024-09-11T23:52:05.746522+0000","last_clean":"2024-09-11T23:52:05.746522+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:52:05.746522+0000","last_undegraded":"2024-09-11T23:52:05.746522+0000","last_fullsized":"2024-09-11T23:52:05.746522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435012+0000","last_change":"2024-09-11T23:51:14.409908+0000","last_active":"2024-09-11T23:52:05.435012+0000","last_peered":"2024-09-11T23:52:05.435012+0000","last_clean":"2024-09-11T23:52:05.435012+0000","last_became_active":"2024-09-11T23:51:14.409432+0000","last_became_peered":"2024-09-11T23:51:14.409432+0000","last_unstale":"2024-09-11T23:52:05.435012+0000","last_undegraded":"2024-09-11T23:52:05.435012+0000","last_fullsized":"2024-09-11T23:52:05.435012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:06.371306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.521172+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:52:05.521172+0000","last_peered":"2024-09-11T23:52:05.521172+0000","last_clean":"2024-09-11T23:52:05.521172+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:52:05.521172+0000","last_undegraded":"2024-09-11T23:52:05.521172+0000","last_fullsized":"2024-09-11T23:52:05.521172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435167+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:52:05.435167+0000","last_peered":"2024-09-11T23:52:05.435167+0000","last_clean":"2024-09-11T23:52:05.435167+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:52:05.435167+0000","last_undegraded":"2024-09-11T23:52:05.435167+0000","last_fullsized":"2024-09-11T23:52:05.435167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.517017+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:52:05.517017+0000","last_peered":"2024-09-11T23:52:05.517017+0000","last_clean":"2024-09-11T23:52:05.517017+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:52:05.517017+0000","last_undegraded":"2024-09-11T23:52:05.517017+0000","last_fullsized":"2024-09-11T23:52:05.517017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431772+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:52:05.431772+0000","last_peered":"2024-09-11T23:52:05.431772+0000","last_clean":"2024-09-11T23:52:05.431772+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:52:05.431772+0000","last_undegraded":"2024-09-11T23:52:05.431772+0000","last_fullsized":"2024-09-11T23:52:05.431772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'48","reported_seq":95,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.649913+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:52:05.649913+0000","last_peered":"2024-09-11T23:52:05.649913+0000","last_clean":"2024-09-11T23:52:05.649913+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:52:05.649913+0000","last_undegraded":"2024-09-11T23:52:05.649913+0000","last_fullsized":"2024-09-11T23:52:05.649913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525022+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:52:05.525022+0000","last_peered":"2024-09-11T23:52:05.525022+0000","last_clean":"2024-09-11T23:52:05.525022+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:52:05.525022+0000","last_undegraded":"2024-09-11T23:52:05.525022+0000","last_fullsized":"2024-09-11T23:52:05.525022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.745108+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:52:05.745108+0000","last_peered":"2024-09-11T23:52:05.745108+0000","last_clean":"2024-09-11T23:52:05.745108+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:52:05.745108+0000","last_undegraded":"2024-09-11T23:52:05.745108+0000","last_fullsized":"2024-09-11T23:52:05.745108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158521+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:52:06.158521+0000","last_peered":"2024-09-11T23:52:06.158521+0000","last_clean":"2024-09-11T23:52:06.158521+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:52:06.158521+0000","last_undegraded":"2024-09-11T23:52:06.158521+0000","last_fullsized":"2024-09-11T23:52:06.158521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694501+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:52:05.694501+0000","last_peered":"2024-09-11T23:52:05.694501+0000","last_clean":"2024-09-11T23:52:05.694501+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:52:05.694501+0000","last_undegraded":"2024-09-11T23:52:05.694501+0000","last_fullsized":"2024-09-11T23:52:05.694501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516588+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:52:05.516588+0000","last_peered":"2024-09-11T23:52:05.516588+0000","last_clean":"2024-09-11T23:52:05.516588+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:52:05.516588+0000","last_undegraded":"2024-09-11T23:52:05.516588+0000","last_fullsized":"2024-09-11T23:52:05.516588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523605+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:52:05.523605+0000","last_peered":"2024-09-11T23:52:05.523605+0000","last_clean":"2024-09-11T23:52:05.523605+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:52:05.523605+0000","last_undegraded":"2024-09-11T23:52:05.523605+0000","last_fullsized":"2024-09-11T23:52:05.523605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872645+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:52:05.872645+0000","last_peered":"2024-09-11T23:52:05.872645+0000","last_clean":"2024-09-11T23:52:05.872645+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:52:05.872645+0000","last_undegraded":"2024-09-11T23:52:05.872645+0000","last_fullsized":"2024-09-11T23:52:05.872645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872496+0000","last_change":"2024-09-11T23:51:14.123663+0000","last_active":"2024-09-11T23:52:05.872496+0000","last_peered":"2024-09-11T23:52:05.872496+0000","last_clean":"2024-09-11T23:52:05.872496+0000","last_became_active":"2024-09-11T23:51:14.118315+0000","last_became_peered":"2024-09-11T23:51:14.118315+0000","last_unstale":"2024-09-11T23:52:05.872496+0000","last_undegraded":"2024-09-11T23:52:05.872496+0000","last_fullsized":"2024-09-11T23:52:05.872496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:10.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.524009+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:52:05.524009+0000","last_peered":"2024-09-11T23:52:05.524009+0000","last_clean":"2024-09-11T23:52:05.524009+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:52:05.524009+0000","last_undegraded":"2024-09-11T23:52:05.524009+0000","last_fullsized":"2024-09-11T23:52:05.524009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516698+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:52:05.516698+0000","last_peered":"2024-09-11T23:52:05.516698+0000","last_clean":"2024-09-11T23:52:05.516698+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:52:05.516698+0000","last_undegraded":"2024-09-11T23:52:05.516698+0000","last_fullsized":"2024-09-11T23:52:05.516698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746447+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:52:05.746447+0000","last_peered":"2024-09-11T23:52:05.746447+0000","last_clean":"2024-09-11T23:52:05.746447+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:52:05.746447+0000","last_undegraded":"2024-09-11T23:52:05.746447+0000","last_fullsized":"2024-09-11T23:52:05.746447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433119+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:52:05.433119+0000","last_peered":"2024-09-11T23:52:05.433119+0000","last_clean":"2024-09-11T23:52:05.433119+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:52:05.433119+0000","last_undegraded":"2024-09-11T23:52:05.433119+0000","last_fullsized":"2024-09-11T23:52:05.433119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516897+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:52:05.516897+0000","last_peered":"2024-09-11T23:52:05.516897+0000","last_clean":"2024-09-11T23:52:05.516897+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:52:05.516897+0000","last_undegraded":"2024-09-11T23:52:05.516897+0000","last_fullsized":"2024-09-11T23:52:05.516897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694561+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:52:05.694561+0000","last_peered":"2024-09-11T23:52:05.694561+0000","last_clean":"2024-09-11T23:52:05.694561+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:52:05.694561+0000","last_undegraded":"2024-09-11T23:52:05.694561+0000","last_fullsized":"2024-09-11T23:52:05.694561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516945+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:52:05.516945+0000","last_peered":"2024-09-11T23:52:05.516945+0000","last_clean":"2024-09-11T23:52:05.516945+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:52:05.516945+0000","last_undegraded":"2024-09-11T23:52:05.516945+0000","last_fullsized":"2024-09-11T23:52:05.516945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158617+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:52:06.158617+0000","last_peered":"2024-09-11T23:52:06.158617+0000","last_clean":"2024-09-11T23:52:06.158617+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:52:06.158617+0000","last_undegraded":"2024-09-11T23:52:06.158617+0000","last_fullsized":"2024-09-11T23:52:06.158617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525435+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:52:05.525435+0000","last_peered":"2024-09-11T23:52:05.525435+0000","last_clean":"2024-09-11T23:52:05.525435+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:52:05.525435+0000","last_undegraded":"2024-09-11T23:52:05.525435+0000","last_fullsized":"2024-09-11T23:52:05.525435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525158+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:52:05.525158+0000","last_peered":"2024-09-11T23:52:05.525158+0000","last_clean":"2024-09-11T23:52:05.525158+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:52:05.525158+0000","last_undegraded":"2024-09-11T23:52:05.525158+0000","last_fullsized":"2024-09-11T23:52:05.525158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.521532+0000","last_change":"2024-09-11T23:51:14.125596+0000","last_active":"2024-09-11T23:52:05.521532+0000","last_peered":"2024-09-11T23:52:05.521532+0000","last_clean":"2024-09-11T23:52:05.521532+0000","last_became_active":"2024-09-11T23:51:14.119682+0000","last_became_peered":"2024-09-11T23:51:14.119682+0000","last_unstale":"2024-09-11T23:52:05.521532+0000","last_undegraded":"2024-09-11T23:52:05.521532+0000","last_fullsized":"2024-09-11T23:52:05.521532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:06.062602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596490,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":763,"num_read_kb":490,"num_write":295,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1789470,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":436969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1089536,"data_stored":1236648,"data_compressed":44911,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":839348,"data_compressed":22046,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":838686,"data_compressed":22046,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1039219,"data_compressed":38755,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445521,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1169268,"data_compressed":44911,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":447193,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":974848,"data_stored":970648,"data_compressed":28202,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":513493,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":498698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":437865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133469,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":200849,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:52:09.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:52:09.748 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:10.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-11T23:52:10.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: Unable to set osd_memory_target on smithi083 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-11T23:52:10.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:10.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.025 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:10.025 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.025 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.025 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.025 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.025 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.025 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.025 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:10.025 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:10.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: Unable to set osd_memory_target on smithi083 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:10.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-11T23:52:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: Unable to set osd_memory_target on smithi083 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-11T23:52:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:52:10.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:10.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:10.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.258+0000 7f2bb13b0640 1 -- 172.21.15.62:0/3885163928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac102d30 msgr2=0x7f2bac103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:10.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.258+0000 7f2bb13b0640 1 --2- 172.21.15.62:0/3885163928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac102d30 0x7f2bac103110 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009930 tx=0x7f2b9c02a420 comp rx=0 tx=0).stop 2024-09-11T23:52:10.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.258+0000 7f2bb13b0640 1 -- 172.21.15.62:0/3885163928 shutdown_connections 2024-09-11T23:52:10.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.258+0000 7f2bb13b0640 1 --2- 172.21.15.62:0/3885163928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bac1082a0 0x7f2bac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.258+0000 7f2bb13b0640 1 --2- 172.21.15.62:0/3885163928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2bac1036e0 0x7f2bac107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.258+0000 7f2bb13b0640 1 --2- 172.21.15.62:0/3885163928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac102d30 0x7f2bac103110 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.258+0000 7f2bb13b0640 1 -- 172.21.15.62:0/3885163928 >> 172.21.15.62:0/3885163928 conn(0x7f2bac0fc7d0 msgr2=0x7f2bac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:10.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.258+0000 7f2bb13b0640 1 -- 172.21.15.62:0/3885163928 shutdown_connections 2024-09-11T23:52:10.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.259+0000 7f2bb13b0640 1 -- 172.21.15.62:0/3885163928 wait complete. 2024-09-11T23:52:10.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.259+0000 7f2bb13b0640 1 Processor -- start 2024-09-11T23:52:10.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2bb13b0640 1 -- start start 2024-09-11T23:52:10.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2bb13b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bac102d30 0x7f2bac1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:10.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2bb13b0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac1036e0 0x7f2bac198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:10.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2bb13b0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2bac1082a0 0x7f2bac19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:10.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2bb13b0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac112100 con 0x7f2bac1036e0 2024-09-11T23:52:10.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2bb13b0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac111f80 con 0x7f2bac102d30 2024-09-11T23:52:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2bb13b0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac112280 con 0x7f2bac1082a0 2024-09-11T23:52:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2baa7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac1036e0 0x7f2bac198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2bab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2bac1082a0 0x7f2bac19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2baa7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac1036e0 0x7f2bac198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45980/0 (socket says 172.21.15.62:45980) 2024-09-11T23:52:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2baaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bac102d30 0x7f2bac1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2bab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2bac1082a0 0x7f2bac19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:34230/0 (socket says 172.21.15.62:34230) 2024-09-11T23:52:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.260+0000 7f2baa7fc640 1 -- 172.21.15.62:0/2329163488 learned_addr learned my addr 172.21.15.62:0/2329163488 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2bab7fe640 1 -- 172.21.15.62:0/2329163488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bac102d30 msgr2=0x7f2bac1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2bab7fe640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bac102d30 0x7f2bac1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2bab7fe640 1 -- 172.21.15.62:0/2329163488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac1036e0 msgr2=0x7f2bac198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2bab7fe640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac1036e0 0x7f2bac198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2bab7fe640 1 -- 172.21.15.62:0/2329163488 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac19d570 con 0x7f2bac1082a0 2024-09-11T23:52:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2baa7fc640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac1036e0 0x7f2bac198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2bab7fe640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2bac1082a0 0x7f2bac19ce90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400ec50 tx=0x7f2ba400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2b93fff640 1 -- 172.21.15.62:0/2329163488 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4003b70 con 0x7f2bac1082a0 2024-09-11T23:52:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2bb13b0640 1 -- 172.21.15.62:0/2329163488 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bac19d860 con 0x7f2bac1082a0 2024-09-11T23:52:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2b93fff640 1 -- 172.21.15.62:0/2329163488 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba4003d10 con 0x7f2bac1082a0 2024-09-11T23:52:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2b93fff640 1 -- 172.21.15.62:0/2329163488 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4010890 con 0x7f2bac1082a0 2024-09-11T23:52:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.261+0000 7f2bb13b0640 1 -- 172.21.15.62:0/2329163488 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bac1a5140 con 0x7f2bac1082a0 2024-09-11T23:52:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.262+0000 7f2bb13b0640 1 -- 172.21.15.62:0/2329163488 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bac06a850 con 0x7f2bac1082a0 2024-09-11T23:52:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.263+0000 7f2b93fff640 1 -- 172.21.15.62:0/2329163488 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f2ba4043b90 con 0x7f2bac1082a0 2024-09-11T23:52:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.264+0000 7f2b93fff640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b940a2d80 0x7f2b940a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.264+0000 7f2baaffd640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b940a2d80 0x7f2b940a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.264+0000 7f2b93fff640 1 -- 172.21.15.62:0/2329163488 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f2ba40cfae0 con 0x7f2bac1082a0 2024-09-11T23:52:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.265+0000 7f2baaffd640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b940a2d80 0x7f2b940a5240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006010 tx=0x7f2b9c000bc0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.267+0000 7f2b93fff640 1 -- 172.21.15.62:0/2329163488 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ba409c5c0 con 0x7f2bac1082a0 2024-09-11T23:52:10.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.488+0000 7f2bb13b0640 1 -- 172.21.15.62:0/2329163488 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bac103110 con 0x7f2bac1082a0 2024-09-11T23:52:10.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.489+0000 7f2b93fff640 1 -- 172.21.15.62:0/2329163488 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f2bac103110 con 0x7f2bac1082a0 2024-09-11T23:52:10.489 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:10.489 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"btime":"2024-09-11T23:52:04:422374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:04.372188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:52:10.490 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2024-09-11T23:52:10.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 -- 172.21.15.62:0/2329163488 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b940a2d80 msgr2=0x7f2b940a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:10.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b940a2d80 0x7f2b940a5240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006010 tx=0x7f2b9c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 -- 172.21.15.62:0/2329163488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2bac1082a0 msgr2=0x7f2bac19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:10.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2bac1082a0 0x7f2bac19ce90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400ec50 tx=0x7f2ba400c6a0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 -- 172.21.15.62:0/2329163488 shutdown_connections 2024-09-11T23:52:10.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b940a2d80 0x7f2b940a5240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2bac1082a0 0x7f2bac19ce90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac1036e0 0x7f2bac198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 --2- 172.21.15.62:0/2329163488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bac102d30 0x7f2bac1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:10.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 -- 172.21.15.62:0/2329163488 >> 172.21.15.62:0/2329163488 conn(0x7f2bac0fc7d0 msgr2=0x7f2bac106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:10.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 -- 172.21.15.62:0/2329163488 shutdown_connections 2024-09-11T23:52:10.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:10.491+0000 7f2bb13b0640 1 -- 172.21.15.62:0/2329163488 wait complete. 2024-09-11T23:52:11.125 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:11.125 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:52:11.125 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-09-11T23:52:11.125 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Unable to set osd_memory_target on smithi062 to 184586035: error parsing value: Value '184586035' is below minimum 939524096 2024-09-11T23:52:11.125 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi062:/etc/ceph/ceph.conf 2024-09-11T23:52:11.125 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-11T23:52:11.126 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-11T23:52:11.126 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:52:11.126 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:52:11.126 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:52:11.126 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-11T23:52:11.126 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:11.126 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:11.126 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:11.126 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:52:11.126 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2329163488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:11.127 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:11.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:52:11.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Unable to set osd_memory_target on smithi062 to 184586035: error parsing value: Value '184586035' is below minimum 939524096 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi062:/etc/ceph/ceph.conf 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:52:11.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2329163488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:11.169 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:11.170 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:11.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:52:11.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-09-11T23:52:11.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Unable to set osd_memory_target on smithi062 to 184586035: error parsing value: Value '184586035' is below minimum 939524096 2024-09-11T23:52:11.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi062:/etc/ceph/ceph.conf 2024-09-11T23:52:11.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-11T23:52:11.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-11T23:52:11.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: Updating smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2329163488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:11.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:11.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:11.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:11.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:11.286 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:52:11.546 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.988+0000 7f73a91e4640 1 -- 172.21.15.62:0/1554727515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73a4103660 msgr2=0x7f73a410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.988+0000 7f73a91e4640 1 --2- 172.21.15.62:0/1554727515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73a4103660 0x7f73a410bb50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f738c009900 tx=0x7f738c02a480 comp rx=0 tx=0).stop 2024-09-11T23:52:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.988+0000 7f73a91e4640 1 -- 172.21.15.62:0/1554727515 shutdown_connections 2024-09-11T23:52:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.988+0000 7f73a91e4640 1 --2- 172.21.15.62:0/1554727515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73a410c170 0x7f73a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.988+0000 7f73a91e4640 1 --2- 172.21.15.62:0/1554727515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73a4103660 0x7f73a410bb50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.988+0000 7f73a91e4640 1 --2- 172.21.15.62:0/1554727515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73a4102d40 0x7f73a4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.988+0000 7f73a91e4640 1 -- 172.21.15.62:0/1554727515 >> 172.21.15.62:0/1554727515 conn(0x7f73a40fc7a0 msgr2=0x7f73a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.988+0000 7f73a91e4640 1 -- 172.21.15.62:0/1554727515 shutdown_connections 2024-09-11T23:52:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.988+0000 7f73a91e4640 1 -- 172.21.15.62:0/1554727515 wait complete. 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.989+0000 7f73a91e4640 1 Processor -- start 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.989+0000 7f73a91e4640 1 -- start start 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a91e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73a4102d40 0x7f73a41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a91e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73a4103660 0x7f73a4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a91e4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73a410c170 0x7f73a4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a91e4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4111f70 con 0x7f73a410c170 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a91e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4111df0 con 0x7f73a4102d40 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a91e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41120f0 con 0x7f73a4103660 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73a410c170 0x7f73a4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73a410c170 0x7f73a4101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53430/0 (socket says 172.21.15.62:53430) 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a359e640 1 -- 172.21.15.62:0/255618732 learned_addr learned my addr 172.21.15.62:0/255618732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:12.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a259c640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73a4103660 0x7f73a4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a2d9d640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73a4102d40 0x7f73a41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a359e640 1 -- 172.21.15.62:0/255618732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73a4103660 msgr2=0x7f73a4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a359e640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73a4103660 0x7f73a4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a359e640 1 -- 172.21.15.62:0/255618732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73a4102d40 msgr2=0x7f73a41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a359e640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73a4102d40 0x7f73a41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a359e640 1 -- 172.21.15.62:0/255618732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a41a6910 con 0x7f73a410c170 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a2d9d640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73a4102d40 0x7f73a41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f73a359e640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73a410c170 0x7f73a4101420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f739c009fd0 tx=0x7f739c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.990+0000 7f739bfff640 1 -- 172.21.15.62:0/255618732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c017070 con 0x7f73a410c170 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.991+0000 7f739bfff640 1 -- 172.21.15.62:0/255618732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739c003e50 con 0x7f73a410c170 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.991+0000 7f73a91e4640 1 -- 172.21.15.62:0/255618732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a41a6c00 con 0x7f73a410c170 2024-09-11T23:52:12.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.991+0000 7f739bfff640 1 -- 172.21.15.62:0/255618732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c0106c0 con 0x7f73a410c170 2024-09-11T23:52:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.991+0000 7f73a91e4640 1 -- 172.21.15.62:0/255618732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a41a7110 con 0x7f73a410c170 2024-09-11T23:52:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.992+0000 7f73a91e4640 1 -- 172.21.15.62:0/255618732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a4103120 con 0x7f73a410c170 2024-09-11T23:52:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.993+0000 7f739bfff640 1 -- 172.21.15.62:0/255618732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f739c003970 con 0x7f73a410c170 2024-09-11T23:52:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.994+0000 7f739bfff640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f73900a2e00 0x7f73900a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.994+0000 7f73a2d9d640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f73900a2e00 0x7f73900a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.994+0000 7f739bfff640 1 -- 172.21.15.62:0/255618732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f739c0cf250 con 0x7f73a410c170 2024-09-11T23:52:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.994+0000 7f73a2d9d640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f73900a2e00 0x7f73900a52c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f73940058c0 tx=0x7f7394005ec0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:11.997+0000 7f739bfff640 1 -- 172.21.15.62:0/255618732 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f739c09bdb0 con 0x7f73a410c170 2024-09-11T23:52:12.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.175+0000 7f73a91e4640 1 -- 172.21.15.62:0/255618732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73a41071e0 con 0x7f73a410c170 2024-09-11T23:52:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.177+0000 7f739bfff640 1 -- 172.21.15.62:0/255618732 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22995 (secure 0 0 0) 0x7f739c09b6b0 con 0x7f73a410c170 2024-09-11T23:52:12.177 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:12.177 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":84,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:52:05.426157+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/518295197":"2024-09-12T23:52:05.426127+0000","172.21.15.62:6801/3106394779":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/908998551":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/133064408":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6800/3106394779":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2894445674":"2024-09-12T23:52:05.426127+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 -- 172.21.15.62:0/255618732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f73900a2e00 msgr2=0x7f73900a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f73900a2e00 0x7f73900a52c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f73940058c0 tx=0x7f7394005ec0 comp rx=0 tx=0).stop 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 -- 172.21.15.62:0/255618732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73a410c170 msgr2=0x7f73a4101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73a410c170 0x7f73a4101420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f739c009fd0 tx=0x7f739c00b260 comp rx=0 tx=0).stop 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 -- 172.21.15.62:0/255618732 shutdown_connections 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f73900a2e00 0x7f73900a52c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73a410c170 0x7f73a4101420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73a4103660 0x7f73a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 --2- 172.21.15.62:0/255618732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73a4102d40 0x7f73a41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 -- 172.21.15.62:0/255618732 >> 172.21.15.62:0/255618732 conn(0x7f73a40fc7a0 msgr2=0x7f73a40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:12.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 -- 172.21.15.62:0/255618732 shutdown_connections 2024-09-11T23:52:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:12.179+0000 7f73a91e4640 1 -- 172.21.15.62:0/255618732 wait complete. 2024-09-11T23:52:12.929 DEBUG:tasks.fwd_scrub:Yielding 2024-09-11T23:52:12.930 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T23:52:12.939 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:52:12.939 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-09-11T23:52:12.939 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-11T23:52:12.940 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-11T23:52:12.940 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:52:12.940 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:52:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.051+0000 7fb20f7b7640 1 -- 172.21.15.62:0/2429586217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2081009b0 msgr2=0x7fb208100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.051+0000 7fb20f7b7640 1 --2- 172.21.15.62:0/2429586217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2081009b0 0x7fb208100e10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009930 tx=0x7fb1f802a420 comp rx=0 tx=0).stop 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.051+0000 7fb20f7b7640 1 -- 172.21.15.62:0/2429586217 shutdown_connections 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.051+0000 7fb20f7b7640 1 --2- 172.21.15.62:0/2429586217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb208101350 0x7fb20810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.051+0000 7fb20f7b7640 1 --2- 172.21.15.62:0/2429586217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2081009b0 0x7fb208100e10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.051+0000 7fb20f7b7640 1 --2- 172.21.15.62:0/2429586217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2081069c0 0x7fb208106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.051+0000 7fb20f7b7640 1 -- 172.21.15.62:0/2429586217 >> 172.21.15.62:0/2429586217 conn(0x7fb208077990 msgr2=0x7fb2080fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.051+0000 7fb20f7b7640 1 -- 172.21.15.62:0/2429586217 shutdown_connections 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.051+0000 7fb20f7b7640 1 -- 172.21.15.62:0/2429586217 wait complete. 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20f7b7640 1 Processor -- start 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20f7b7640 1 -- start start 2024-09-11T23:52:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20f7b7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2081009b0 0x7fb208198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20f7b7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb208101350 0x7fb2081988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20f7b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2081069c0 0x7fb20819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20f7b7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb208111ef0 con 0x7fb208101350 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20f7b7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb208111d70 con 0x7fb2081069c0 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20f7b7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb208112070 con 0x7fb2081009b0 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20cd52640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb208101350 0x7fb2081988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20cd52640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb208101350 0x7fb2081988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53456/0 (socket says 172.21.15.62:53456) 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20cd52640 1 -- 172.21.15.62:0/16448999 learned_addr learned my addr 172.21.15.62:0/16448999 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20dd54640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2081069c0 0x7fb20819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.052+0000 7fb20d553640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2081009b0 0x7fb208198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb20cd52640 1 -- 172.21.15.62:0/16448999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2081009b0 msgr2=0x7fb208198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb20cd52640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2081009b0 0x7fb208198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb20cd52640 1 -- 172.21.15.62:0/16448999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2081069c0 msgr2=0x7fb20819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb20cd52640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2081069c0 0x7fb20819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb20cd52640 1 -- 172.21.15.62:0/16448999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20819d360 con 0x7fb208101350 2024-09-11T23:52:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb20dd54640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2081069c0 0x7fb20819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb20cd52640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb208101350 0x7fb2081988b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009230 tx=0x7fb1f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb1fe7fc640 1 -- 172.21.15.62:0/16448999 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f803f030 con 0x7fb208101350 2024-09-11T23:52:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb1fe7fc640 1 -- 172.21.15.62:0/16448999 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1f8003ad0 con 0x7fb208101350 2024-09-11T23:52:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb1fe7fc640 1 -- 172.21.15.62:0/16448999 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8035070 con 0x7fb208101350 2024-09-11T23:52:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb20f7b7640 1 -- 172.21.15.62:0/16448999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20819d5f0 con 0x7fb208101350 2024-09-11T23:52:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.053+0000 7fb20f7b7640 1 -- 172.21.15.62:0/16448999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2081a4ee0 con 0x7fb208101350 2024-09-11T23:52:13.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.054+0000 7fb20f7b7640 1 -- 172.21.15.62:0/16448999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb208100e10 con 0x7fb208101350 2024-09-11T23:52:13.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.055+0000 7fb1fe7fc640 1 -- 172.21.15.62:0/16448999 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fb1f8042030 con 0x7fb208101350 2024-09-11T23:52:13.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.056+0000 7fb1fe7fc640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb1f40a2ec0 0x7fb1f40a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.056+0000 7fb20d553640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb1f40a2ec0 0x7fb1f40a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.056+0000 7fb1fe7fc640 1 -- 172.21.15.62:0/16448999 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb1f80f0030 con 0x7fb208101350 2024-09-11T23:52:13.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.056+0000 7fb20d553640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb1f40a2ec0 0x7fb1f40a5380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb204009ae0 tx=0x7fb204005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.063+0000 7fb1fe7fc640 1 -- 172.21.15.62:0/16448999 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1f80bcb10 con 0x7fb208101350 2024-09-11T23:52:13.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:12 smithi120 ceph-mon[35182]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:52:13.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:12 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/255618732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:52:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:12 smithi083 ceph-mon[34805]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:52:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:12 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/255618732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:52:13.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:12 smithi062 ceph-mon[30958]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:52:13.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:12 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/255618732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:52:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:13.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.257+0000 7fb20f7b7640 1 -- 172.21.15.62:0/16448999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fb2081996c0 con 0x7fb208101350 2024-09-11T23:52:13.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.261+0000 7fb1fe7fc640 1 -- 172.21.15.62:0/16448999 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v26)=0 v26) ==== 129+0+0 (secure 0 0 0) 0x7fb2081996c0 con 0x7fb208101350 2024-09-11T23:52:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 -- 172.21.15.62:0/16448999 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb1f40a2ec0 msgr2=0x7fb1f40a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb1f40a2ec0 0x7fb1f40a5380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb204009ae0 tx=0x7fb204005f90 comp rx=0 tx=0).stop 2024-09-11T23:52:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 -- 172.21.15.62:0/16448999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb208101350 msgr2=0x7fb2081988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb208101350 0x7fb2081988b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009230 tx=0x7fb1f8004270 comp rx=0 tx=0).stop 2024-09-11T23:52:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 -- 172.21.15.62:0/16448999 shutdown_connections 2024-09-11T23:52:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb1f40a2ec0 0x7fb1f40a5380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2081069c0 0x7fb20819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb208101350 0x7fb2081988b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 --2- 172.21.15.62:0/16448999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2081009b0 0x7fb208198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 -- 172.21.15.62:0/16448999 >> 172.21.15.62:0/16448999 conn(0x7fb208077990 msgr2=0x7fb208103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 -- 172.21.15.62:0/16448999 shutdown_connections 2024-09-11T23:52:13.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.264+0000 7fb20f7b7640 1 -- 172.21.15.62:0/16448999 wait complete. 2024-09-11T23:52:13.275 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-11T23:52:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.383+0000 7ff0890c9640 1 -- 172.21.15.62:0/322259407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff084107b70 msgr2=0x7ff084107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.383+0000 7ff0890c9640 1 --2- 172.21.15.62:0/322259407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff084107b70 0x7ff084107f50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff074009cd0 tx=0x7ff07402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.383+0000 7ff0890c9640 1 -- 172.21.15.62:0/322259407 shutdown_connections 2024-09-11T23:52:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.383+0000 7ff0890c9640 1 --2- 172.21.15.62:0/322259407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084102470 0x7ff08410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.383+0000 7ff0890c9640 1 --2- 172.21.15.62:0/322259407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff084101ad0 0x7ff084101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.383+0000 7ff0890c9640 1 --2- 172.21.15.62:0/322259407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff084107b70 0x7ff084107f50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.383+0000 7ff0890c9640 1 -- 172.21.15.62:0/322259407 >> 172.21.15.62:0/322259407 conn(0x7ff0840fd7d0 msgr2=0x7ff0840ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.383+0000 7ff0890c9640 1 -- 172.21.15.62:0/322259407 shutdown_connections 2024-09-11T23:52:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.383+0000 7ff0890c9640 1 -- 172.21.15.62:0/322259407 wait complete. 2024-09-11T23:52:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff0890c9640 1 Processor -- start 2024-09-11T23:52:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff0890c9640 1 -- start start 2024-09-11T23:52:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff0890c9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff084101ad0 0x7ff084198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff0890c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff084102470 0x7ff0841988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff0890c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084107b70 0x7ff08419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff0890c9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff084111f20 con 0x7ff084101ad0 2024-09-11T23:52:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff0890c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff084111da0 con 0x7ff084107b70 2024-09-11T23:52:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff0890c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841120a0 con 0x7ff084102470 2024-09-11T23:52:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff082d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff084101ad0 0x7ff084198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff082d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff084101ad0 0x7ff084198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53460/0 (socket says 172.21.15.62:53460) 2024-09-11T23:52:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff082d9d640 1 -- 172.21.15.62:0/1171247683 learned_addr learned my addr 172.21.15.62:0/1171247683 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff08259c640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff084102470 0x7ff0841988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.384+0000 7ff08359e640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084107b70 0x7ff08419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff082d9d640 1 -- 172.21.15.62:0/1171247683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff084102470 msgr2=0x7ff0841988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff082d9d640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff084102470 0x7ff0841988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff082d9d640 1 -- 172.21.15.62:0/1171247683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084107b70 msgr2=0x7ff08419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff082d9d640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084107b70 0x7ff08419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff082d9d640 1 -- 172.21.15.62:0/1171247683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08419d380 con 0x7ff084101ad0 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff082d9d640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff084101ad0 0x7ff084198390 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff074005e30 tx=0x7ff074032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff07bfff640 1 -- 172.21.15.62:0/1171247683 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074004040 con 0x7ff084101ad0 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff07bfff640 1 -- 172.21.15.62:0/1171247683 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0740046c0 con 0x7ff084101ad0 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff0890c9640 1 -- 172.21.15.62:0/1171247683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08419d610 con 0x7ff084101ad0 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff07bfff640 1 -- 172.21.15.62:0/1171247683 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074046480 con 0x7ff084101ad0 2024-09-11T23:52:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.385+0000 7ff0890c9640 1 -- 172.21.15.62:0/1171247683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0841a4f00 con 0x7ff084101ad0 2024-09-11T23:52:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.386+0000 7ff0890c9640 1 -- 172.21.15.62:0/1171247683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff084101f30 con 0x7ff084101ad0 2024-09-11T23:52:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.387+0000 7ff07bfff640 1 -- 172.21.15.62:0/1171247683 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7ff0740305a0 con 0x7ff084101ad0 2024-09-11T23:52:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.388+0000 7ff07bfff640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff0700a2e40 0x7ff0700a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.388+0000 7ff08259c640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff0700a2e40 0x7ff0700a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.388+0000 7ff07bfff640 1 -- 172.21.15.62:0/1171247683 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7ff07402e030 con 0x7ff084101ad0 2024-09-11T23:52:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.389+0000 7ff08259c640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff0700a2e40 0x7ff0700a5300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff0840ffd10 tx=0x7ff06c005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.392+0000 7ff07bfff640 1 -- 172.21.15.62:0/1171247683 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff07404b060 con 0x7ff084101ad0 2024-09-11T23:52:13.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.596+0000 7ff0890c9640 1 -- 172.21.15.62:0/1171247683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7ff0841a5130 con 0x7ff084101ad0 2024-09-11T23:52:13.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.599+0000 7ff07bfff640 1 -- 172.21.15.62:0/1171247683 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) ==== 138+0+0 (secure 0 0 0) 0x7ff0740bd560 con 0x7ff084101ad0 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 -- 172.21.15.62:0/1171247683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff0700a2e40 msgr2=0x7ff0700a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff0700a2e40 0x7ff0700a5300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff0840ffd10 tx=0x7ff06c005f90 comp rx=0 tx=0).stop 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 -- 172.21.15.62:0/1171247683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff084101ad0 msgr2=0x7ff084198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff084101ad0 0x7ff084198390 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff074005e30 tx=0x7ff074032fb0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 -- 172.21.15.62:0/1171247683 shutdown_connections 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff0700a2e40 0x7ff0700a5300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084107b70 0x7ff08419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff084102470 0x7ff0841988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 --2- 172.21.15.62:0/1171247683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff084101ad0 0x7ff084198390 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 -- 172.21.15.62:0/1171247683 >> 172.21.15.62:0/1171247683 conn(0x7ff0840fd7d0 msgr2=0x7ff0840fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 -- 172.21.15.62:0/1171247683 shutdown_connections 2024-09-11T23:52:13.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.602+0000 7ff0890c9640 1 -- 172.21.15.62:0/1171247683 wait complete. 2024-09-11T23:52:13.615 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-11T23:52:13.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.713+0000 7f383b19d640 1 -- 172.21.15.62:0/4045770805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38341068d0 msgr2=0x7f3834106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.713+0000 7f383b19d640 1 --2- 172.21.15.62:0/4045770805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38341068d0 0x7f3834106cb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3824009cd0 tx=0x7f382402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.714+0000 7f383b19d640 1 -- 172.21.15.62:0/4045770805 shutdown_connections 2024-09-11T23:52:13.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.714+0000 7f383b19d640 1 --2- 172.21.15.62:0/4045770805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3834101270 0x7f383410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.714+0000 7f383b19d640 1 --2- 172.21.15.62:0/4045770805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38341008d0 0x7f3834100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.714+0000 7f383b19d640 1 --2- 172.21.15.62:0/4045770805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38341068d0 0x7f3834106cb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.714+0000 7f383b19d640 1 -- 172.21.15.62:0/4045770805 >> 172.21.15.62:0/4045770805 conn(0x7f38340fc610 msgr2=0x7f38340fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.714+0000 7f383b19d640 1 -- 172.21.15.62:0/4045770805 shutdown_connections 2024-09-11T23:52:13.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.714+0000 7f383b19d640 1 -- 172.21.15.62:0/4045770805 wait complete. 2024-09-11T23:52:13.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.714+0000 7f383b19d640 1 Processor -- start 2024-09-11T23:52:13.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.714+0000 7f383b19d640 1 -- start start 2024-09-11T23:52:13.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.714+0000 7f383b19d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38341008d0 0x7f38340fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383b19d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3834101270 0x7f38340fd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383b19d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38341068d0 0x7f38340fd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383b19d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3834110cd0 con 0x7f38341008d0 2024-09-11T23:52:13.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383b19d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3834110b50 con 0x7f3834101270 2024-09-11T23:52:13.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383b19d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3834110e50 con 0x7f38341068d0 2024-09-11T23:52:13.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383a19b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38341008d0 0x7f38340fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383a19b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38341008d0 0x7f38340fcbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53476/0 (socket says 172.21.15.62:53476) 2024-09-11T23:52:13.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383a19b640 1 -- 172.21.15.62:0/543930389 learned_addr learned my addr 172.21.15.62:0/543930389 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:13.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383999a640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3834101270 0x7f38340fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383a99c640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38341068d0 0x7f38340fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383a19b640 1 -- 172.21.15.62:0/543930389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38341068d0 msgr2=0x7f38340fd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383a19b640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38341068d0 0x7f38340fd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383a19b640 1 -- 172.21.15.62:0/543930389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3834101270 msgr2=0x7f38340fd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383a19b640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3834101270 0x7f38340fd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383a19b640 1 -- 172.21.15.62:0/543930389 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38341a6770 con 0x7f38341008d0 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.715+0000 7f383a19b640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38341008d0 0x7f38340fcbc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f382402ae00 tx=0x7f3824031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.716+0000 7f38337fe640 1 -- 172.21.15.62:0/543930389 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824003e00 con 0x7f38341008d0 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.716+0000 7f38337fe640 1 -- 172.21.15.62:0/543930389 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3824004520 con 0x7f38341008d0 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.716+0000 7f383b19d640 1 -- 172.21.15.62:0/543930389 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38341a6a00 con 0x7f38341008d0 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.716+0000 7f38337fe640 1 -- 172.21.15.62:0/543930389 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824030de0 con 0x7f38341008d0 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.716+0000 7f383b19d640 1 -- 172.21.15.62:0/543930389 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38341a6f40 con 0x7f38341008d0 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.718+0000 7f38337fe640 1 -- 172.21.15.62:0/543930389 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f38240046d0 con 0x7f38341008d0 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.718+0000 7f383e7d8740 1 Processor -- start 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.718+0000 7f383e7d8740 1 -- start start 2024-09-11T23:52:13.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ecd70 0x555f859ed150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 0x555f859f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f859fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383e7d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555f85952280 con 0x555f859ecd70 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383e7d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555f858f9b10 con 0x555f859ed690 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383e7d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555f8594e240 con 0x555f859f7f50 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f38337fe640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f381c0a2d80 0x7f381c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383a99c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ecd70 0x555f859ed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383a99c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ecd70 0x555f859ed150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53482/0 (socket says 172.21.15.62:53482) 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383a99c640 1 -- 172.21.15.62:0/2645408423 learned_addr learned my addr 172.21.15.62:0/2645408423 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383999a640 1 --2- 172.21.15.62:0/2645408423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f859fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f38337fe640 1 -- 172.21.15.62:0/543930389 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f382402e030 con 0x7f38341008d0 2024-09-11T23:52:13.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383a99c640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f381c0a2d80 0x7f381c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383a19b640 1 --2- 172.21.15.62:0/2645408423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 0x555f859f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383999a640 1 -- 172.21.15.62:0/2645408423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 msgr2=0x555f859f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:13.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383999a640 1 --2- 172.21.15.62:0/2645408423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 0x555f859f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383999a640 1 -- 172.21.15.62:0/2645408423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ecd70 msgr2=0x555f859ed150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:13.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383999a640 1 --2- 172.21.15.62:0/2645408423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ecd70 0x555f859ed150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383999a640 1 -- 172.21.15.62:0/2645408423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f859061f0 con 0x555f859f7f50 2024-09-11T23:52:13.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.719+0000 7f383a19b640 1 --2- 172.21.15.62:0/2645408423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 0x555f859f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:52:13.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.720+0000 7f383999a640 1 --2- 172.21.15.62:0/2645408423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f859fa340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3828009bd0 tx=0x7f3828011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc04038666faad05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.720+0000 7f383a99c640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f381c0a2d80 0x7f381c0a5240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f382c013920 tx=0x7f382c004730 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.720+0000 7f3830dea640 1 -- 172.21.15.62:0/2645408423 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828004230 con 0x555f859f7f50 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.720+0000 7f3830dea640 1 -- 172.21.15.62:0/2645408423 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3828009e40 con 0x555f859f7f50 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.720+0000 7f3830dea640 1 -- 172.21.15.62:0/2645408423 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828004d70 con 0x555f859f7f50 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 -- 172.21.15.62:0/2645408423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 msgr2=0x555f859fa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 --2- 172.21.15.62:0/2645408423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f859fa340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3828009bd0 tx=0x7f3828011810 comp rx=0 tx=0).stop 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 -- 172.21.15.62:0/2645408423 shutdown_connections 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 --2- 172.21.15.62:0/2645408423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f859fa340 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 --2- 172.21.15.62:0/2645408423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 0x555f859f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 --2- 172.21.15.62:0/2645408423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ecd70 0x555f859ed150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 -- 172.21.15.62:0/2645408423 >> 172.21.15.62:0/2645408423 conn(0x555f859e9300 msgr2=0x555f859e96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 -- 172.21.15.62:0/2645408423 shutdown_connections 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 -- 172.21.15.62:0/2645408423 wait complete. 2024-09-11T23:52:13.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 -- start start 2024-09-11T23:52:13.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 Processor -- start 2024-09-11T23:52:13.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 0x555f859f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f859f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f859f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555f859fab20 con 0x555f859ed690 2024-09-11T23:52:13.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555f8590cf10 con 0x555f859ecd70 2024-09-11T23:52:13.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.721+0000 7f383e7d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555f859faca0 con 0x555f859f7f50 2024-09-11T23:52:13.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383e7d8740 10 client.0 _ll_register_callbacks cb 0x555f858d5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:13.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383999a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f859f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383a99c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f859f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383999a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f859f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53492/0 (socket says 172.21.15.62:53492) 2024-09-11T23:52:13.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383999a640 1 -- 172.21.15.62:0/3240837361 learned_addr learned my addr 172.21.15.62:0/3240837361 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:13.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383a19b640 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 0x555f859f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383999a640 1 -- 172.21.15.62:0/3240837361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 msgr2=0x555f859f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:13.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383999a640 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f859f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383999a640 1 -- 172.21.15.62:0/3240837361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 msgr2=0x555f859f79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:13.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383999a640 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 0x555f859f79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383999a640 1 -- 172.21.15.62:0/3240837361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f856c36a0 con 0x555f859ed690 2024-09-11T23:52:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383a19b640 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 0x555f859f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383a99c640 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f859f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f383999a640 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f859f4d40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3828005890 tx=0x7f38280038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d162547dde8d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f3819ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.722+0000 7f3819ffb640 1 -- 172.21.15.62:0/3240837361 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828027020 con 0x555f859ed690 2024-09-11T23:52:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f3819ffb640 1 -- 172.21.15.62:0/3240837361 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f3828003cc0 con 0x555f859ed690 2024-09-11T23:52:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f383e7d8740 1 -- 172.21.15.62:0/3240837361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f8595f7f0 con 0x555f859ed690 2024-09-11T23:52:13.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f3819ffb640 1 -- 172.21.15.62:0/3240837361 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3828032050 con 0x555f859ed690 2024-09-11T23:52:13.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f383e7d8740 10 client.15270 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:13.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f383e7d8740 1 -- 172.21.15.62:0/3240837361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f85a06600 con 0x555f859ed690 2024-09-11T23:52:13.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f3819ffb640 1 -- 172.21.15.62:0/3240837361 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555f8595f7f0 con 0x555f859ed690 2024-09-11T23:52:13.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f3819ffb640 1 -- 172.21.15.62:0/3240837361 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3828018070 con 0x555f859ed690 2024-09-11T23:52:13.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f383e7d8740 10 client.15270 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:13.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f383e7d8740 10 client.15270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:13.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f383e7d8740 20 client.15270 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:13.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f383e7d8740 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555f85a07ac0 0x555f85a09eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f383e7d8740 4 client.15270 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:13.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.723+0000 7f383e7d8740 1 -- 172.21.15.62:0/3240837361 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f85959d00 con 0x555f85a07ac0 2024-09-11T23:52:13.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.724+0000 7f383a19b640 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555f85a07ac0 0x555f85a09eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.724+0000 7f383a19b640 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555f85a07ac0 0x555f85a09eb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a8bda91bd464116 server_cookie=32fed58124eb71e5 in_seq=0 out_seq=0 2024-09-11T23:52:13.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.724+0000 7f3819ffb640 10 client.15270 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:13.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f3819ffb640 1 -- 172.21.15.62:0/3240837361 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f85959d00 con 0x555f85a07ac0 2024-09-11T23:52:13.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f3819ffb640 10 client.15270 handle_command_reply: tid=0 2024-09-11T23:52:13.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 client.15270 shutdown 2024-09-11T23:52:13.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 -- 172.21.15.62:0/3240837361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 msgr2=0x555f859f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f859f4d40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3828005890 tx=0x7f38280038a0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 -- 172.21.15.62:0/3240837361 shutdown_connections 2024-09-11T23:52:13.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555f85a07ac0 0x555f85a09eb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f3819ffb640 0 client.15270 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:13.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f859f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f859f4d40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 --2- 172.21.15.62:0/3240837361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 0x555f859f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 -- 172.21.15.62:0/3240837361 >> 172.21.15.62:0/3240837361 conn(0x555f859e9300 msgr2=0x555f859ef030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 -- 172.21.15.62:0/3240837361 shutdown_connections 2024-09-11T23:52:13.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 1 -- 172.21.15.62:0/3240837361 wait complete. 2024-09-11T23:52:13.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.725+0000 7f383e7d8740 20 client.15270 trim_cache size 0 max 16384 2024-09-11T23:52:13.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.728+0000 7f9f5d96d640 1 -- 172.21.15.62:0/2369377658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58102ca0 msgr2=0x7f9f58103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.728+0000 7f9f5d96d640 1 --2- 172.21.15.62:0/2369377658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58102ca0 0x7f9f58103080 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009900 tx=0x7f9f4802a410 comp rx=0 tx=0).stop 2024-09-11T23:52:13.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.728+0000 7f9f5d96d640 1 -- 172.21.15.62:0/2369377658 shutdown_connections 2024-09-11T23:52:13.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.728+0000 7f9f5d96d640 1 --2- 172.21.15.62:0/2369377658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f5810c0d0 0x7f9f5810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.728+0000 7f9f5d96d640 1 --2- 172.21.15.62:0/2369377658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f581035c0 0x7f9f5810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.728+0000 7f9f5d96d640 1 --2- 172.21.15.62:0/2369377658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58102ca0 0x7f9f58103080 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.728+0000 7f9f5d96d640 1 -- 172.21.15.62:0/2369377658 >> 172.21.15.62:0/2369377658 conn(0x7f9f580fc6b0 msgr2=0x7f9f580fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.728+0000 7f9f5d96d640 1 -- 172.21.15.62:0/2369377658 shutdown_connections 2024-09-11T23:52:13.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.728+0000 7f9f5d96d640 1 -- 172.21.15.62:0/2369377658 wait complete. 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f5d96d640 1 Processor -- start 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f5d96d640 1 -- start start 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f5d96d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f58102ca0 0x7f9f581983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f5d96d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f581035c0 0x7f9f581988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f5d96d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f5810c0d0 0x7f9f5819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f5d96d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58111ed0 con 0x7f9f58102ca0 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f5d96d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58111d50 con 0x7f9f581035c0 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f5d96d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58112050 con 0x7f9f5810c0d0 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f56ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f58102ca0 0x7f9f581983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f56ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f58102ca0 0x7f9f581983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53516/0 (socket says 172.21.15.62:53516) 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f56ffd640 1 -- 172.21.15.62:0/1304680899 learned_addr learned my addr 172.21.15.62:0/1304680899 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f567fc640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f581035c0 0x7f9f581988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.729+0000 7f9f577fe640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f5810c0d0 0x7f9f5819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f56ffd640 1 -- 172.21.15.62:0/1304680899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f5810c0d0 msgr2=0x7f9f5819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f56ffd640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f5810c0d0 0x7f9f5819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f56ffd640 1 -- 172.21.15.62:0/1304680899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f581035c0 msgr2=0x7f9f581988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f56ffd640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f581035c0 0x7f9f581988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f56ffd640 1 -- 172.21.15.62:0/1304680899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5819d390 con 0x7f9f58102ca0 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f56ffd640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f58102ca0 0x7f9f581983a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9f48006010 tx=0x7f9f480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f5c96b640 1 -- 172.21.15.62:0/1304680899 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4803f030 con 0x7f9f58102ca0 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f5c96b640 1 -- 172.21.15.62:0/1304680899 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f48031990 con 0x7f9f58102ca0 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f5d96d640 1 -- 172.21.15.62:0/1304680899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5819d620 con 0x7f9f58102ca0 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f5c96b640 1 -- 172.21.15.62:0/1304680899 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48035070 con 0x7f9f58102ca0 2024-09-11T23:52:13.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.730+0000 7f9f5d96d640 1 -- 172.21.15.62:0/1304680899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f581a4f10 con 0x7f9f58102ca0 2024-09-11T23:52:13.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.731+0000 7f9f5d96d640 1 -- 172.21.15.62:0/1304680899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5806a9a0 con 0x7f9f58102ca0 2024-09-11T23:52:13.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.732+0000 7f9f5c96b640 1 -- 172.21.15.62:0/1304680899 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f9f48042030 con 0x7f9f58102ca0 2024-09-11T23:52:13.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.733+0000 7f9f5c96b640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9f400a2e40 0x7f9f400a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.733+0000 7f9f5c96b640 1 -- 172.21.15.62:0/1304680899 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f9f480ef9a0 con 0x7f9f58102ca0 2024-09-11T23:52:13.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.733+0000 7f9f567fc640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9f400a2e40 0x7f9f400a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.734+0000 7f9f567fc640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9f400a2e40 0x7f9f400a5300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f5810d530 tx=0x7f9f4c005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.737+0000 7f9f5c96b640 1 -- 172.21.15.62:0/1304680899 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f480bbef0 con 0x7f9f58102ca0 2024-09-11T23:52:13.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383e7d8740 1 Processor -- start 2024-09-11T23:52:13.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383e7d8740 1 -- start start 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 0x555f859f4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f85a0f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f85a0fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383e7d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f38341b1ff0 con 0x555f859ed690 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383e7d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38341af8c0 con 0x555f859ecd70 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383e7d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38341ad710 con 0x555f859f7f50 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383999a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f85a0f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383999a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f85a0f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53524/0 (socket says 172.21.15.62:53524) 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383999a640 1 -- 172.21.15.62:0/585924477 learned_addr learned my addr 172.21.15.62:0/585924477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383a19b640 1 --2- 172.21.15.62:0/585924477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 0x555f859f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.745+0000 7f383a99c640 1 --2- 172.21.15.62:0/585924477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f85a0fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.746+0000 7f383999a640 1 -- 172.21.15.62:0/585924477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 msgr2=0x555f85a0fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.746+0000 7f383999a640 1 --2- 172.21.15.62:0/585924477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f85a0fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.746+0000 7f383999a640 1 -- 172.21.15.62:0/585924477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 msgr2=0x555f859f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.746+0000 7f383999a640 1 --2- 172.21.15.62:0/585924477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 0x555f859f4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.746+0000 7f383999a640 1 -- 172.21.15.62:0/585924477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f859f5840 con 0x555f859ed690 2024-09-11T23:52:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.746+0000 7f383999a640 1 --2- 172.21.15.62:0/585924477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f85a0f710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3828009bd0 tx=0x7f3828004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25a59fc2a2db129 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.746+0000 7f381bfff640 1 -- 172.21.15.62:0/585924477 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828027040 con 0x555f859ed690 2024-09-11T23:52:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.746+0000 7f381bfff640 1 -- 172.21.15.62:0/585924477 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3828009780 con 0x555f859ed690 2024-09-11T23:52:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.746+0000 7f381bfff640 1 -- 172.21.15.62:0/585924477 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382801e040 con 0x555f859ed690 2024-09-11T23:52:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 -- 172.21.15.62:0/585924477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 msgr2=0x555f85a0f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 --2- 172.21.15.62:0/585924477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f85a0f710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3828009bd0 tx=0x7f3828004370 comp rx=0 tx=0).stop 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 -- 172.21.15.62:0/585924477 shutdown_connections 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 --2- 172.21.15.62:0/585924477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859f7f50 0x555f85a0fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 --2- 172.21.15.62:0/585924477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859ed690 0x555f85a0f710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 --2- 172.21.15.62:0/585924477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ecd70 0x555f859f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 -- 172.21.15.62:0/585924477 >> 172.21.15.62:0/585924477 conn(0x555f859e9300 msgr2=0x555f859f6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 -- 172.21.15.62:0/585924477 shutdown_connections 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 -- 172.21.15.62:0/585924477 wait complete. 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 -- start start 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.747+0000 7f383e7d8740 1 Processor -- start 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859ecd70 0x555f85a006d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 0x555f859f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383e7d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859f7f50 0x555f859f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383e7d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f38341add80 con 0x555f859f7f50 2024-09-11T23:52:13.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383e7d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38341af1f0 con 0x555f859ed690 2024-09-11T23:52:13.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383e7d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38341ae450 con 0x555f859ecd70 2024-09-11T23:52:13.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383e7d8740 10 client.0 _ll_register_callbacks cb 0x7f382c009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:13.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383a99c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859f7f50 0x555f859f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383a99c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859f7f50 0x555f859f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53534/0 (socket says 172.21.15.62:53534) 2024-09-11T23:52:13.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383a99c640 1 -- 172.21.15.62:0/2888705616 learned_addr learned my addr 172.21.15.62:0/2888705616 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:13.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383a19b640 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859ecd70 0x555f85a006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383999a640 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 0x555f859f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383a19b640 1 -- 172.21.15.62:0/2888705616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 msgr2=0x555f859f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383a19b640 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 0x555f859f1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383a19b640 1 -- 172.21.15.62:0/2888705616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859f7f50 msgr2=0x555f859f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383a19b640 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859f7f50 0x555f859f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383a19b640 1 -- 172.21.15.62:0/2888705616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f859ed4c0 con 0x555f859ecd70 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.748+0000 7f383a99c640 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859f7f50 0x555f859f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f383a19b640 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859ecd70 0x555f85a006d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38240ee2f0 tx=0x7f38240dd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d2fffd3ffe22b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f3819ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f3819ffb640 1 -- 172.21.15.62:0/2888705616 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824004040 con 0x555f859ecd70 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f3819ffb640 1 -- 172.21.15.62:0/2888705616 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f38240b9680 con 0x555f859ecd70 2024-09-11T23:52:13.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f383e7d8740 1 -- 172.21.15.62:0/2888705616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3828009780 con 0x555f859ecd70 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f3819ffb640 1 -- 172.21.15.62:0/2888705616 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f38240305a0 con 0x555f859ecd70 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f383e7d8740 10 client.24967 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f383e7d8740 1 -- 172.21.15.62:0/2888705616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f85a725d0 con 0x555f859ecd70 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f3819ffb640 1 -- 172.21.15.62:0/2888705616 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3824036040 con 0x555f859ecd70 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f3819ffb640 1 -- 172.21.15.62:0/2888705616 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f382404e070 con 0x555f859ecd70 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f383e7d8740 10 client.24967 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f383e7d8740 10 client.24967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f383e7d8740 20 client.24967 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f383e7d8740 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555f85a07ac0 0x555f85a74c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f383e7d8740 4 client.24967 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.749+0000 7f383e7d8740 1 -- 172.21.15.62:0/2888705616 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f3828009780 con 0x555f85a07ac0 2024-09-11T23:52:13.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.750+0000 7f383999a640 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555f85a07ac0 0x555f85a74c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:13.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.750+0000 7f3819ffb640 10 client.24967 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:13.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.750+0000 7f383999a640 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555f85a07ac0 0x555f85a74c60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed47af6e45ac155 server_cookie=b7eca080408af07f in_seq=0 out_seq=0 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f3819ffb640 1 -- 172.21.15.62:0/2888705616 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f3828009780 con 0x555f85a07ac0 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f3819ffb640 10 client.24967 handle_command_reply: tid=0 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f383e7d8740 1 client.24967 shutdown 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f383e7d8740 1 -- 172.21.15.62:0/2888705616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859ecd70 msgr2=0x555f85a006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f383e7d8740 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859ecd70 0x555f85a006d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38240ee2f0 tx=0x7f38240dd040 comp rx=0 tx=0).stop 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f383e7d8740 1 -- 172.21.15.62:0/2888705616 shutdown_connections 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f383e7d8740 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555f85a07ac0 0x555f85a74c60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f383e7d8740 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f859f7f50 0x555f859f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f383e7d8740 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f859ed690 0x555f859f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f383e7d8740 1 --2- 172.21.15.62:0/2888705616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f859ecd70 0x555f85a006d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f383e7d8740 1 -- 172.21.15.62:0/2888705616 >> 172.21.15.62:0/2888705616 conn(0x555f859e9300 msgr2=0x555f859e7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.751+0000 7f3819ffb640 0 client.24967 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:13.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.752+0000 7f383e7d8740 1 -- 172.21.15.62:0/2888705616 shutdown_connections 2024-09-11T23:52:13.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.752+0000 7f383e7d8740 1 -- 172.21.15.62:0/2888705616 wait complete. 2024-09-11T23:52:13.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.752+0000 7f383e7d8740 20 client.24967 trim_cache size 0 max 16384 2024-09-11T23:52:13.758 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:52:13.758 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:52:13.758 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "2e29c1c6-de2f-4296-b73f-c9ba759c820d", 2024-09-11T23:52:13.758 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:52:13.758 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:52:13.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.752+0000 7f383b19d640 1 -- 172.21.15.62:0/543930389 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f381c0a2d80 msgr2=0x7f381c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.752+0000 7f383b19d640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f381c0a2d80 0x7f381c0a5240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f382c013920 tx=0x7f382c004730 comp rx=0 tx=0).stop 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.752+0000 7f383b19d640 1 -- 172.21.15.62:0/543930389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38341008d0 msgr2=0x7f38340fcbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.752+0000 7f383b19d640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38341008d0 0x7f38340fcbc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f382402ae00 tx=0x7f3824031e60 comp rx=0 tx=0).stop 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.753+0000 7f383b19d640 1 -- 172.21.15.62:0/543930389 shutdown_connections 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.753+0000 7f383b19d640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f381c0a2d80 0x7f381c0a5240 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.753+0000 7f383b19d640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38341068d0 0x7f38340fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.753+0000 7f383b19d640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3834101270 0x7f38340fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.753+0000 7f383b19d640 1 --2- 172.21.15.62:0/543930389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38341008d0 0x7f38340fcbc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.753+0000 7f383b19d640 1 -- 172.21.15.62:0/543930389 >> 172.21.15.62:0/543930389 conn(0x7f38340fc610 msgr2=0x7f383406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.753+0000 7f383b19d640 1 -- 172.21.15.62:0/543930389 shutdown_connections 2024-09-11T23:52:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.753+0000 7f383b19d640 1 -- 172.21.15.62:0/543930389 wait complete. 2024-09-11T23:52:13.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.919+0000 7f9f5d96d640 1 -- 172.21.15.62:0/1304680899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f9f58107120 con 0x7f9f58102ca0 2024-09-11T23:52:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.923+0000 7f9f5c96b640 1 -- 172.21.15.62:0/1304680899 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7f9f480c01f0 con 0x7f9f58102ca0 2024-09-11T23:52:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 -- 172.21.15.62:0/1304680899 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9f400a2e40 msgr2=0x7f9f400a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9f400a2e40 0x7f9f400a5300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f5810d530 tx=0x7f9f4c005f90 comp rx=0 tx=0).stop 2024-09-11T23:52:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 -- 172.21.15.62:0/1304680899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f58102ca0 msgr2=0x7f9f581983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f58102ca0 0x7f9f581983a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9f48006010 tx=0x7f9f480042f0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 -- 172.21.15.62:0/1304680899 shutdown_connections 2024-09-11T23:52:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9f400a2e40 0x7f9f400a5300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f5810c0d0 0x7f9f5819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f581035c0 0x7f9f581988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 --2- 172.21.15.62:0/1304680899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f58102ca0 0x7f9f581983a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 -- 172.21.15.62:0/1304680899 >> 172.21.15.62:0/1304680899 conn(0x7f9f580fc6b0 msgr2=0x7f9f580fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 -- 172.21.15.62:0/1304680899 shutdown_connections 2024-09-11T23:52:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:13.925+0000 7f9f5d96d640 1 -- 172.21.15.62:0/1304680899 wait complete. 2024-09-11T23:52:13.938 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-11T23:52:14.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.054+0000 7fabd4bd6640 1 -- 172.21.15.62:0/770813188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd0106a00 msgr2=0x7fabd0106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.054+0000 7fabd4bd6640 1 --2- 172.21.15.62:0/770813188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd0106a00 0x7fabd0106de0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009930 tx=0x7fabc002a420 comp rx=0 tx=0).stop 2024-09-11T23:52:14.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.054+0000 7fabd4bd6640 1 -- 172.21.15.62:0/770813188 shutdown_connections 2024-09-11T23:52:14.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.054+0000 7fabd4bd6640 1 --2- 172.21.15.62:0/770813188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd0101390 0x7fabd010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.054+0000 7fabd4bd6640 1 --2- 172.21.15.62:0/770813188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd01009f0 0x7fabd0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.054+0000 7fabd4bd6640 1 --2- 172.21.15.62:0/770813188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd0106a00 0x7fabd0106de0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.054+0000 7fabd4bd6640 1 -- 172.21.15.62:0/770813188 >> 172.21.15.62:0/770813188 conn(0x7fabd00fc6b0 msgr2=0x7fabd00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.054+0000 7fabd4bd6640 1 -- 172.21.15.62:0/770813188 shutdown_connections 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.054+0000 7fabd4bd6640 1 -- 172.21.15.62:0/770813188 wait complete. 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.054+0000 7fabd4bd6640 1 Processor -- start 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabd4bd6640 1 -- start start 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabd4bd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd01009f0 0x7fabd01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabd4bd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd0101390 0x7fabd01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabd4bd6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd0106a00 0x7fabd019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabd4bd6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fabd0110e00 con 0x7fabd0106a00 2024-09-11T23:52:14.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabd4bd6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fabd0110c80 con 0x7fabd0101390 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabd4bd6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabd0110f80 con 0x7fabd01009f0 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabced9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd0106a00 0x7fabd019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabced9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd0106a00 0x7fabd019bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53558/0 (socket says 172.21.15.62:53558) 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabce59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd01009f0 0x7fabd01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabced9d640 1 -- 172.21.15.62:0/3157816340 learned_addr learned my addr 172.21.15.62:0/3157816340 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabc5d9b640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd0101390 0x7fabd01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabced9d640 1 -- 172.21.15.62:0/3157816340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd01009f0 msgr2=0x7fabd01972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabced9d640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd01009f0 0x7fabd01972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabced9d640 1 -- 172.21.15.62:0/3157816340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd0101390 msgr2=0x7fabd01977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabced9d640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd0101390 0x7fabd01977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabced9d640 1 -- 172.21.15.62:0/3157816340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd019c2a0 con 0x7fabd0106a00 2024-09-11T23:52:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.055+0000 7fabc5d9b640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd0101390 0x7fabd01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.056+0000 7fabced9d640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd0106a00 0x7fabd019bb70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009fd0 tx=0x7fabc800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.056+0000 7fabc7fff640 1 -- 172.21.15.62:0/3157816340 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8017070 con 0x7fabd0106a00 2024-09-11T23:52:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.056+0000 7fabd4bd6640 1 -- 172.21.15.62:0/3157816340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd019c590 con 0x7fabd0106a00 2024-09-11T23:52:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.056+0000 7fabc7fff640 1 -- 172.21.15.62:0/3157816340 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc8003e50 con 0x7fabd0106a00 2024-09-11T23:52:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.056+0000 7fabc7fff640 1 -- 172.21.15.62:0/3157816340 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc80106c0 con 0x7fabd0106a00 2024-09-11T23:52:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.056+0000 7fabd4bd6640 1 -- 172.21.15.62:0/3157816340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd01a3e20 con 0x7fabd0106a00 2024-09-11T23:52:14.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.057+0000 7fabd4bd6640 1 -- 172.21.15.62:0/3157816340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd0100e50 con 0x7fabd0106a00 2024-09-11T23:52:14.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.059+0000 7fabc7fff640 1 -- 172.21.15.62:0/3157816340 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fabc8003970 con 0x7fabd0106a00 2024-09-11T23:52:14.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.059+0000 7fabc7fff640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fabbc0a2e40 0x7fabbc0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.060+0000 7fabce59c640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fabbc0a2e40 0x7fabbc0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.060+0000 7fabc7fff640 1 -- 172.21.15.62:0/3157816340 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fabc80cf2f0 con 0x7fabd0106a00 2024-09-11T23:52:14.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.060+0000 7fabce59c640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fabbc0a2e40 0x7fabbc0a5300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009900 tx=0x7fabc00058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.064+0000 7fabc7fff640 1 -- 172.21.15.62:0/3157816340 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabc809bdd0 con 0x7fabd0106a00 2024-09-11T23:52:14.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.265+0000 7fabd4bd6640 1 -- 172.21.15.62:0/3157816340 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fabd0062630 con 0x7fabbc0a2e40 2024-09-11T23:52:14.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.334+0000 7fabc7fff640 1 -- 172.21.15.62:0/3157816340 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fabd0062630 con 0x7fabbc0a2e40 2024-09-11T23:52:14.334 INFO:teuthology.orchestra.run.smithi062.stdout:Schedule set for path / 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 -- 172.21.15.62:0/3157816340 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fabbc0a2e40 msgr2=0x7fabbc0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fabbc0a2e40 0x7fabbc0a5300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009900 tx=0x7fabc00058d0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 -- 172.21.15.62:0/3157816340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd0106a00 msgr2=0x7fabd019bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd0106a00 0x7fabd019bb70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009fd0 tx=0x7fabc800b260 comp rx=0 tx=0).stop 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 -- 172.21.15.62:0/3157816340 shutdown_connections 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fabbc0a2e40 0x7fabbc0a5300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd0106a00 0x7fabd019bb70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd0101390 0x7fabd01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 --2- 172.21.15.62:0/3157816340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabd01009f0 0x7fabd01972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 -- 172.21.15.62:0/3157816340 >> 172.21.15.62:0/3157816340 conn(0x7fabd00fc6b0 msgr2=0x7fabd00fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:14.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 -- 172.21.15.62:0/3157816340 shutdown_connections 2024-09-11T23:52:14.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.336+0000 7fabd4bd6640 1 -- 172.21.15.62:0/3157816340 wait complete. 2024-09-11T23:52:14.346 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-11T23:52:14.440 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:52:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.455+0000 7f1b53a01640 1 -- 172.21.15.62:0/80875141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4c101350 msgr2=0x7f1b4c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.455+0000 7f1b53a01640 1 --2- 172.21.15.62:0/80875141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4c101350 0x7f1b4c10d880 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009cd0 tx=0x7f1b4402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.455+0000 7f1b53a01640 1 -- 172.21.15.62:0/80875141 shutdown_connections 2024-09-11T23:52:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.455+0000 7f1b53a01640 1 --2- 172.21.15.62:0/80875141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4c101350 0x7f1b4c10d880 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.455+0000 7f1b53a01640 1 --2- 172.21.15.62:0/80875141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c1009b0 0x7f1b4c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.455+0000 7f1b53a01640 1 --2- 172.21.15.62:0/80875141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b4c1069c0 0x7f1b4c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.455+0000 7f1b53a01640 1 -- 172.21.15.62:0/80875141 >> 172.21.15.62:0/80875141 conn(0x7f1b4c077990 msgr2=0x7f1b4c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.456+0000 7f1b53a01640 1 -- 172.21.15.62:0/80875141 shutdown_connections 2024-09-11T23:52:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.456+0000 7f1b53a01640 1 -- 172.21.15.62:0/80875141 wait complete. 2024-09-11T23:52:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.456+0000 7f1b53a01640 1 Processor -- start 2024-09-11T23:52:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.456+0000 7f1b53a01640 1 -- start start 2024-09-11T23:52:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.456+0000 7f1b53a01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b4c1009b0 0x7f1b4c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.456+0000 7f1b53a01640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c101350 0x7f1b4c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b53a01640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4c1069c0 0x7f1b4c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b53a01640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c110e00 con 0x7f1b4c1069c0 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b53a01640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c110c80 con 0x7f1b4c1009b0 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b53a01640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c110f80 con 0x7f1b4c101350 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b51f9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4c1069c0 0x7f1b4c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b51f9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4c1069c0 0x7f1b4c19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53574/0 (socket says 172.21.15.62:53574) 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b51f9e640 1 -- 172.21.15.62:0/3545462279 learned_addr learned my addr 172.21.15.62:0/3545462279 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b50f9c640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c101350 0x7f1b4c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b5179d640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b4c1009b0 0x7f1b4c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b51f9e640 1 -- 172.21.15.62:0/3545462279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c101350 msgr2=0x7f1b4c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b51f9e640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c101350 0x7f1b4c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b51f9e640 1 -- 172.21.15.62:0/3545462279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b4c1009b0 msgr2=0x7f1b4c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b51f9e640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b4c1009b0 0x7f1b4c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b51f9e640 1 -- 172.21.15.62:0/3545462279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c19d380 con 0x7f1b4c1069c0 2024-09-11T23:52:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b50f9c640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c101350 0x7f1b4c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b51f9e640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4c1069c0 0x7f1b4c19cc50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1b44005cc0 tx=0x7f1b44032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b427fc640 1 -- 172.21.15.62:0/3545462279 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44004040 con 0x7f1b4c1069c0 2024-09-11T23:52:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b53a01640 1 -- 172.21.15.62:0/3545462279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c19d610 con 0x7f1b4c1069c0 2024-09-11T23:52:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b427fc640 1 -- 172.21.15.62:0/3545462279 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b440046c0 con 0x7f1b4c1069c0 2024-09-11T23:52:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.457+0000 7f1b427fc640 1 -- 172.21.15.62:0/3545462279 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44030e00 con 0x7f1b4c1069c0 2024-09-11T23:52:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.458+0000 7f1b53a01640 1 -- 172.21.15.62:0/3545462279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c1a4f00 con 0x7f1b4c1069c0 2024-09-11T23:52:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.459+0000 7f1b53a01640 1 -- 172.21.15.62:0/3545462279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4c100e10 con 0x7f1b4c1069c0 2024-09-11T23:52:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.460+0000 7f1b427fc640 1 -- 172.21.15.62:0/3545462279 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f1b440305a0 con 0x7f1b4c1069c0 2024-09-11T23:52:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.461+0000 7f1b427fc640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1b380a2e40 0x7f1b380a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.461+0000 7f1b5179d640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1b380a2e40 0x7f1b380a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.461+0000 7f1b427fc640 1 -- 172.21.15.62:0/3545462279 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f1b4402e030 con 0x7f1b4c1069c0 2024-09-11T23:52:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.461+0000 7f1b5179d640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1b380a2e40 0x7f1b380a5300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009ae0 tx=0x7f1b48005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.465+0000 7f1b427fc640 1 -- 172.21.15.62:0/3545462279 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b4404b060 con 0x7f1b4c1069c0 2024-09-11T23:52:14.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/16448999' entity='client.admin' 2024-09-11T23:52:14.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:14.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:14.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:14.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:14.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1171247683' entity='client.admin' 2024-09-11T23:52:14.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:14.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:14.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:14.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:14.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1304680899' entity='client.admin' 2024-09-11T23:52:14.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:14.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:14.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:14.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:14.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.665+0000 7f1b53a01640 1 -- 172.21.15.62:0/3545462279 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f1b4c109fc0 con 0x7f1b380a2e40 2024-09-11T23:52:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/16448999' entity='client.admin' 2024-09-11T23:52:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1171247683' entity='client.admin' 2024-09-11T23:52:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:14.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1304680899' entity='client.admin' 2024-09-11T23:52:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:14.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:14.706 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:14.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/16448999' entity='client.admin' 2024-09-11T23:52:14.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:14.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1171247683' entity='client.admin' 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1304680899' entity='client.admin' 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:52:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:52:14.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.728+0000 7f1b427fc640 1 -- 172.21.15.62:0/3545462279 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f1b48011b20 con 0x7f1b380a2e40 2024-09-11T23:52:14.728 INFO:teuthology.orchestra.run.smithi062.stdout:Retention added to path / 2024-09-11T23:52:14.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 -- 172.21.15.62:0/3545462279 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1b380a2e40 msgr2=0x7f1b380a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1b380a2e40 0x7f1b380a5300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009ae0 tx=0x7f1b48005f90 comp rx=0 tx=0).stop 2024-09-11T23:52:14.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 -- 172.21.15.62:0/3545462279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4c1069c0 msgr2=0x7f1b4c19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4c1069c0 0x7f1b4c19cc50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1b44005cc0 tx=0x7f1b44032f40 comp rx=0 tx=0).stop 2024-09-11T23:52:14.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 -- 172.21.15.62:0/3545462279 shutdown_connections 2024-09-11T23:52:14.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1b380a2e40 0x7f1b380a5300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4c1069c0 0x7f1b4c19cc50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c101350 0x7f1b4c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 --2- 172.21.15.62:0/3545462279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b4c1009b0 0x7f1b4c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 -- 172.21.15.62:0/3545462279 >> 172.21.15.62:0/3545462279 conn(0x7f1b4c077990 msgr2=0x7f1b4c069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:14.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 -- 172.21.15.62:0/3545462279 shutdown_connections 2024-09-11T23:52:14.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.731+0000 7f1b53a01640 1 -- 172.21.15.62:0/3545462279 wait complete. 2024-09-11T23:52:14.742 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T23:52:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.888+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/1990153297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5d4103640 msgr2=0x7fb5d410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.888+0000 7fb5d8a4c640 1 --2- 172.21.15.62:0/1990153297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5d4103640 0x7fb5d410bb20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009900 tx=0x7fb5bc02a480 comp rx=0 tx=0).stop 2024-09-11T23:52:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.889+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/1990153297 shutdown_connections 2024-09-11T23:52:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.889+0000 7fb5d8a4c640 1 --2- 172.21.15.62:0/1990153297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d410c0f0 0x7fb5d410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.889+0000 7fb5d8a4c640 1 --2- 172.21.15.62:0/1990153297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5d4103640 0x7fb5d410bb20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.889+0000 7fb5d8a4c640 1 --2- 172.21.15.62:0/1990153297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d4102d20 0x7fb5d4103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.889+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/1990153297 >> 172.21.15.62:0/1990153297 conn(0x7fb5d4077990 msgr2=0x7fb5d40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.889+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/1990153297 shutdown_connections 2024-09-11T23:52:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.889+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/1990153297 wait complete. 2024-09-11T23:52:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.889+0000 7fb5d8a4c640 1 Processor -- start 2024-09-11T23:52:14.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.889+0000 7fb5d8a4c640 1 -- start start 2024-09-11T23:52:14.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d8a4c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5d4102d20 0x7fb5d41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d8a4c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d4103640 0x7fb5d4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d8a4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d410c0f0 0x7fb5d419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d8a4c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4111f10 con 0x7fb5d4102d20 2024-09-11T23:52:14.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d8a4c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4111d90 con 0x7fb5d410c0f0 2024-09-11T23:52:14.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d8a4c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4112090 con 0x7fb5d4103640 2024-09-11T23:52:14.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5d4102d20 0x7fb5d41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5d4102d20 0x7fb5d41982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53588/0 (socket says 172.21.15.62:53588) 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d4103640 0x7fb5d4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d259c640 1 -- 172.21.15.62:0/4097303033 learned_addr learned my addr 172.21.15.62:0/4097303033 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d2d9d640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d410c0f0 0x7fb5d419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d259c640 1 -- 172.21.15.62:0/4097303033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d4103640 msgr2=0x7fb5d4198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d259c640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d4103640 0x7fb5d4198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d259c640 1 -- 172.21.15.62:0/4097303033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d410c0f0 msgr2=0x7fb5d419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d259c640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d410c0f0 0x7fb5d419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d259c640 1 -- 172.21.15.62:0/4097303033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d419d2e0 con 0x7fb5d4102d20 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d259c640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5d4102d20 0x7fb5d41982f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c920 tx=0x7fb5cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5cb7fe640 1 -- 172.21.15.62:0/4097303033 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc007c70 con 0x7fb5d4102d20 2024-09-11T23:52:14.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5cb7fe640 1 -- 172.21.15.62:0/4097303033 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5cc007e10 con 0x7fb5d4102d20 2024-09-11T23:52:14.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/4097303033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d419d5d0 con 0x7fb5d4102d20 2024-09-11T23:52:14.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5cb7fe640 1 -- 172.21.15.62:0/4097303033 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc018890 con 0x7fb5d4102d20 2024-09-11T23:52:14.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.890+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/4097303033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d41a4e60 con 0x7fb5d4102d20 2024-09-11T23:52:14.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.892+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/4097303033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d4103100 con 0x7fb5d4102d20 2024-09-11T23:52:14.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.893+0000 7fb5cb7fe640 1 -- 172.21.15.62:0/4097303033 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fb5cc018a70 con 0x7fb5d4102d20 2024-09-11T23:52:14.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.893+0000 7fb5cb7fe640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5c00a2e40 0x7fb5c00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:14.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.894+0000 7fb5d1d9b640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5c00a2e40 0x7fb5c00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:14.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.894+0000 7fb5cb7fe640 1 -- 172.21.15.62:0/4097303033 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb5cc0cf660 con 0x7fb5d4102d20 2024-09-11T23:52:14.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.894+0000 7fb5d1d9b640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5c00a2e40 0x7fb5c00a5300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc000c00 tx=0x7fb5bc005b50 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:14.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:14.897+0000 7fb5cb7fe640 1 -- 172.21.15.62:0/4097303033 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5cc014030 con 0x7fb5d4102d20 2024-09-11T23:52:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.098+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/4097303033 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb5d40feed0 con 0x7fb5c00a2e40 2024-09-11T23:52:15.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.105+0000 7fb5cb7fe640 1 -- 172.21.15.62:0/4097303033 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fb5d40feed0 con 0x7fb5c00a2e40 2024-09-11T23:52:15.105 INFO:teuthology.orchestra.run.smithi062.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T23:52:14", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-11T23:52:15.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/4097303033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5c00a2e40 msgr2=0x7fb5c00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5c00a2e40 0x7fb5c00a5300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc000c00 tx=0x7fb5bc005b50 comp rx=0 tx=0).stop 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/4097303033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5d4102d20 msgr2=0x7fb5d41982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5d4102d20 0x7fb5d41982f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c920 tx=0x7fb5cc00cde0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/4097303033 shutdown_connections 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5c00a2e40 0x7fb5c00a5300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d410c0f0 0x7fb5d419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d4103640 0x7fb5d4198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 --2- 172.21.15.62:0/4097303033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5d4102d20 0x7fb5d41982f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/4097303033 >> 172.21.15.62:0/4097303033 conn(0x7fb5d4077990 msgr2=0x7fb5d40fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/4097303033 shutdown_connections 2024-09-11T23:52:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.108+0000 7fb5d8a4c640 1 -- 172.21.15.62:0/4097303033 wait complete. 2024-09-11T23:52:15.120 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T23:52:15.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.201+0000 7f3c9576f640 1 -- 172.21.15.62:0/1026335432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c90101df0 msgr2=0x7f3c901021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.201+0000 7f3c9576f640 1 --2- 172.21.15.62:0/1026335432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c90101df0 0x7f3c901021d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009900 tx=0x7f3c8002a410 comp rx=0 tx=0).stop 2024-09-11T23:52:15.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.201+0000 7f3c9576f640 1 -- 172.21.15.62:0/1026335432 shutdown_connections 2024-09-11T23:52:15.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.201+0000 7f3c9576f640 1 --2- 172.21.15.62:0/1026335432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c90106f20 0x7f3c9010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.201+0000 7f3c9576f640 1 --2- 172.21.15.62:0/1026335432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c901027a0 0x7f3c901067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.201+0000 7f3c9576f640 1 --2- 172.21.15.62:0/1026335432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c90101df0 0x7f3c901021d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.201+0000 7f3c9576f640 1 -- 172.21.15.62:0/1026335432 >> 172.21.15.62:0/1026335432 conn(0x7f3c900fd8f0 msgr2=0x7f3c900ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:15.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.201+0000 7f3c9576f640 1 -- 172.21.15.62:0/1026335432 shutdown_connections 2024-09-11T23:52:15.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.201+0000 7f3c9576f640 1 -- 172.21.15.62:0/1026335432 wait complete. 2024-09-11T23:52:15.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.202+0000 7f3c9576f640 1 Processor -- start 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.202+0000 7f3c9576f640 1 -- start start 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c9576f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c90101df0 0x7f3c900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c9576f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c901027a0 0x7f3c90074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c9576f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c90106f20 0x7f3c90074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c9576f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c90111f80 con 0x7f3c901027a0 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c9576f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c90111e00 con 0x7f3c90106f20 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c9576f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c90112100 con 0x7f3c90101df0 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c901027a0 0x7f3c90074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c901027a0 0x7f3c90074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53608/0 (socket says 172.21.15.62:53608) 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/2144288212 learned_addr learned my addr 172.21.15.62:0/2144288212 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8effd640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c90101df0 0x7f3c900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8f7fe640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c90106f20 0x7f3c90074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/2144288212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c90101df0 msgr2=0x7f3c900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c90101df0 0x7f3c900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/2144288212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c90106f20 msgr2=0x7f3c90074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c90106f20 0x7f3c90074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/2144288212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c901a6920 con 0x7f3c901027a0 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8f7fe640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c90106f20 0x7f3c90074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c901027a0 0x7f3c90074a30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400e9f0 tx=0x7f3c8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c77fff640 1 -- 172.21.15.62:0/2144288212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8400cd80 con 0x7f3c901027a0 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c9576f640 1 -- 172.21.15.62:0/2144288212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c901a6c10 con 0x7f3c901027a0 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c77fff640 1 -- 172.21.15.62:0/2144288212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c84010910 con 0x7f3c901027a0 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.203+0000 7f3c77fff640 1 -- 172.21.15.62:0/2144288212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840188f0 con 0x7f3c901027a0 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.204+0000 7f3c9576f640 1 -- 172.21.15.62:0/2144288212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c901a71a0 con 0x7f3c901027a0 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.206+0000 7f3c77fff640 1 -- 172.21.15.62:0/2144288212 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f3c84010ac0 con 0x7f3c901027a0 2024-09-11T23:52:15.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.206+0000 7f3c97b48740 1 Processor -- start 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.206+0000 7f3c97b48740 1 -- start start 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.206+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 0x55cee954e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.206+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954edc0 0x55cee9559140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee9559680 0x55cee955ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c97b48740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cee94b42e0 con 0x55cee9559680 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c97b48740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cee945bce0 con 0x55cee954edc0 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c97b48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cee94b02a0 con 0x55cee954e4a0 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c77fff640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3c780a2d80 0x7f3c780a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee9559680 0x55cee955ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee9559680 0x55cee955ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53618/0 (socket says 172.21.15.62:53618) 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/2254672810 learned_addr learned my addr 172.21.15.62:0/2254672810 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954edc0 0x55cee9559140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8f7fe640 1 --2- 172.21.15.62:0/2254672810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 0x55cee954e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c77fff640 1 -- 172.21.15.62:0/2144288212 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f3c840cfa30 con 0x7f3c901027a0 2024-09-11T23:52:15.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8f7fe640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3c780a2d80 0x7f3c780a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/2254672810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 msgr2=0x55cee954e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:15.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/2254672810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 0x55cee954e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/2254672810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954edc0 msgr2=0x55cee9559140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:15.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/2254672810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954edc0 0x55cee9559140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.207+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/2254672810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cee9440440 con 0x55cee9559680 2024-09-11T23:52:15.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.208+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/2254672810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee9559680 0x55cee955ba70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3c840cd7b0 tx=0x7f3c8406ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2579e86439a478 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:15.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.208+0000 7f3c8f7fe640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3c780a2d80 0x7f3c780a5240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3c88005c40 tx=0x7f3c88009200 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:15.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.208+0000 7f3c757fa640 1 -- 172.21.15.62:0/2254672810 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840a01c0 con 0x55cee9559680 2024-09-11T23:52:15.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.208+0000 7f3c757fa640 1 -- 172.21.15.62:0/2254672810 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c840a4a80 con 0x55cee9559680 2024-09-11T23:52:15.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.208+0000 7f3c757fa640 1 -- 172.21.15.62:0/2254672810 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84018b30 con 0x55cee9559680 2024-09-11T23:52:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 -- 172.21.15.62:0/2254672810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee9559680 msgr2=0x55cee955ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 --2- 172.21.15.62:0/2254672810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee9559680 0x55cee955ba70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3c840cd7b0 tx=0x7f3c8406ef40 comp rx=0 tx=0).stop 2024-09-11T23:52:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 -- 172.21.15.62:0/2254672810 shutdown_connections 2024-09-11T23:52:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 --2- 172.21.15.62:0/2254672810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee9559680 0x55cee955ba70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 --2- 172.21.15.62:0/2254672810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954edc0 0x55cee9559140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 --2- 172.21.15.62:0/2254672810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 0x55cee954e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 -- 172.21.15.62:0/2254672810 >> 172.21.15.62:0/2254672810 conn(0x55cee954aa30 msgr2=0x55cee954ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 -- 172.21.15.62:0/2254672810 shutdown_connections 2024-09-11T23:52:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 -- 172.21.15.62:0/2254672810 wait complete. 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 -- start start 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 Processor -- start 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 0x55cee9559030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee95562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee9559680 0x55cee9556880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cee955c1f0 con 0x55cee954edc0 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cee946f050 con 0x55cee9559680 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.209+0000 7f3c97b48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cee955c370 con 0x55cee954e4a0 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c97b48740 10 client.0 _ll_register_callbacks cb 0x55cee94af160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee95562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee95562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53622/0 (socket says 172.21.15.62:53622) 2024-09-11T23:52:15.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 0x55cee9559030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/1145081862 learned_addr learned my addr 172.21.15.62:0/1145081862 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:15.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8f7fe640 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee9559680 0x55cee9556880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/1145081862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 msgr2=0x55cee9559030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:15.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 0x55cee9559030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/1145081862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee9559680 msgr2=0x55cee9556880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:15.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee9559680 0x55cee9556880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/1145081862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cee92253c0 con 0x55cee954edc0 2024-09-11T23:52:15.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8effd640 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 0x55cee9559030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee95562e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3c8409a0c0 tx=0x7f3c840a4980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf06d706f840aa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1145081862 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840cd5b0 con 0x55cee954edc0 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1145081862 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f3c8409b260 con 0x55cee954edc0 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c97b48740 1 -- 172.21.15.62:0/1145081862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c840a4a80 con 0x55cee954edc0 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.210+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1145081862 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3c840a01c0 con 0x55cee954edc0 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c97b48740 10 client.15312 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c97b48740 1 -- 172.21.15.62:0/1145081862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cee9567c00 con 0x55cee954edc0 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1145081862 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c840a4a80 con 0x55cee954edc0 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1145081862 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3c8409ad60 con 0x55cee954edc0 2024-09-11T23:52:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c97b48740 10 client.15312 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c97b48740 10 client.15312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c97b48740 20 client.15312 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cee9568f50 0x55cee956b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c97b48740 4 client.15312 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c97b48740 1 -- 172.21.15.62:0/1145081862 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cee94c1850 con 0x55cee9568f50 2024-09-11T23:52:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.211+0000 7f3c8effd640 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cee9568f50 0x55cee956b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.212+0000 7f3c72ff5640 10 client.15312 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.212+0000 7f3c8effd640 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cee9568f50 0x55cee956b340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c024f2b072e18c74 server_cookie=5d1dfb5d424ce7a4 in_seq=0 out_seq=0 2024-09-11T23:52:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1145081862 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cee94c1850 con 0x55cee9568f50 2024-09-11T23:52:15.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c72ff5640 10 client.15312 handle_command_reply: tid=0 2024-09-11T23:52:15.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 client.15312 shutdown 2024-09-11T23:52:15.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 -- 172.21.15.62:0/1145081862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 msgr2=0x55cee95562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee95562e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3c8409a0c0 tx=0x7f3c840a4980 comp rx=0 tx=0).stop 2024-09-11T23:52:15.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 -- 172.21.15.62:0/1145081862 shutdown_connections 2024-09-11T23:52:15.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cee9568f50 0x55cee956b340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee9559680 0x55cee9556880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee95562e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1145081862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954e4a0 0x55cee9559030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 -- 172.21.15.62:0/1145081862 >> 172.21.15.62:0/1145081862 conn(0x55cee954aa30 msgr2=0x55cee9550710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c72ff5640 0 client.15312 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 -- 172.21.15.62:0/1145081862 shutdown_connections 2024-09-11T23:52:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 1 -- 172.21.15.62:0/1145081862 wait complete. 2024-09-11T23:52:15.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.213+0000 7f3c97b48740 20 client.15312 trim_cache size 0 max 16384 2024-09-11T23:52:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f0eb42f1640 1 -- 172.21.15.62:0/2016848839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac101230 msgr2=0x7f0eac10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f0eb42f1640 1 --2- 172.21.15.62:0/2016848839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac101230 0x7f0eac10def0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009900 tx=0x7f0e9c02a410 comp rx=0 tx=0).stop 2024-09-11T23:52:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f0eb42f1640 1 -- 172.21.15.62:0/2016848839 shutdown_connections 2024-09-11T23:52:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f0eb42f1640 1 --2- 172.21.15.62:0/2016848839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eac10e430 0x7f0eac110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f0eb42f1640 1 --2- 172.21.15.62:0/2016848839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac101230 0x7f0eac10def0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f0eb42f1640 1 --2- 172.21.15.62:0/2016848839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0eac100880 0x7f0eac100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f0eb42f1640 1 -- 172.21.15.62:0/2016848839 >> 172.21.15.62:0/2016848839 conn(0x7f0eac077990 msgr2=0x7f0eac0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f0eb42f1640 1 -- 172.21.15.62:0/2016848839 shutdown_connections 2024-09-11T23:52:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f0eb42f1640 1 -- 172.21.15.62:0/2016848839 wait complete. 2024-09-11T23:52:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f3c97b48740 1 Processor -- start 2024-09-11T23:52:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f3c97b48740 1 -- start start 2024-09-11T23:52:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954e4a0 0x55cee9553540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.231+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee9553a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee9559680 0x55cee9551090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c97b48740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c901adee0 con 0x55cee954edc0 2024-09-11T23:52:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c97b48740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c901ab7b0 con 0x55cee954e4a0 2024-09-11T23:52:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c97b48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c901a9600 con 0x55cee9559680 2024-09-11T23:52:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee9553a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee9553a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53634/0 (socket says 172.21.15.62:53634) 2024-09-11T23:52:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/3169150458 learned_addr learned my addr 172.21.15.62:0/3169150458 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8effd640 1 --2- 172.21.15.62:0/3169150458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954e4a0 0x55cee9553540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8f7fe640 1 --2- 172.21.15.62:0/3169150458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee9559680 0x55cee9551090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/3169150458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee9559680 msgr2=0x55cee9551090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/3169150458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee9559680 0x55cee9551090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/3169150458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954e4a0 msgr2=0x55cee9553540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f0eb42f1640 1 Processor -- start 2024-09-11T23:52:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/3169150458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954e4a0 0x55cee9553540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/3169150458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cee9556bb0 con 0x55cee954edc0 2024-09-11T23:52:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8effd640 1 --2- 172.21.15.62:0/3169150458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954e4a0 0x55cee9553540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f0eb42f1640 1 -- start start 2024-09-11T23:52:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.232+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/3169150458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee9553a80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3c8409b130 tx=0x7f3c840a4660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916a316cdfff7136 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:15.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb42f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac100880 0x7f0eac19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb42f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eac101230 0x7f0eac19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb42f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0eac10e430 0x7f0eac19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb42f1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1140f0 con 0x7f0eac100880 2024-09-11T23:52:15.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb42f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac113f70 con 0x7f0eac101230 2024-09-11T23:52:15.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb42f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac114270 con 0x7f0eac10e430 2024-09-11T23:52:15.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f3c74ff9640 1 -- 172.21.15.62:0/3169150458 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840a4a80 con 0x55cee954edc0 2024-09-11T23:52:15.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f3c74ff9640 1 -- 172.21.15.62:0/3169150458 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c84065070 con 0x55cee954edc0 2024-09-11T23:52:15.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f3c74ff9640 1 -- 172.21.15.62:0/3169150458 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8409b260 con 0x55cee954edc0 2024-09-11T23:52:15.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb208d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac100880 0x7f0eac19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb208d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac100880 0x7f0eac19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53640/0 (socket says 172.21.15.62:53640) 2024-09-11T23:52:15.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb208d640 1 -- 172.21.15.62:0/3268548381 learned_addr learned my addr 172.21.15.62:0/3268548381 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:15.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb288e640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0eac10e430 0x7f0eac19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb188c640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eac101230 0x7f0eac19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb208d640 1 -- 172.21.15.62:0/3268548381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0eac10e430 msgr2=0x7f0eac19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb208d640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0eac10e430 0x7f0eac19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb208d640 1 -- 172.21.15.62:0/3268548381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eac101230 msgr2=0x7f0eac19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb208d640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eac101230 0x7f0eac19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb208d640 1 -- 172.21.15.62:0/3268548381 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac19f590 con 0x7f0eac100880 2024-09-11T23:52:15.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb288e640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0eac10e430 0x7f0eac19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:15.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb188c640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eac101230 0x7f0eac19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:15.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f3c97b48740 1 -- 172.21.15.62:0/3169150458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 msgr2=0x55cee9553a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f3c97b48740 1 --2- 172.21.15.62:0/3169150458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee9553a80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3c8409b130 tx=0x7f3c840a4660 comp rx=0 tx=0).stop 2024-09-11T23:52:15.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f3c97b48740 1 -- 172.21.15.62:0/3169150458 shutdown_connections 2024-09-11T23:52:15.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f3c97b48740 1 --2- 172.21.15.62:0/3169150458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee9559680 0x55cee9551090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f3c97b48740 1 --2- 172.21.15.62:0/3169150458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954edc0 0x55cee9553a80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f3c97b48740 1 --2- 172.21.15.62:0/3169150458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee954e4a0 0x55cee9553540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f0eb208d640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac100880 0x7f0eac19a5a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800e9f0 tx=0x7f0ea800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.233+0000 7f3c97b48740 1 -- 172.21.15.62:0/3169150458 >> 172.21.15.62:0/3169150458 conn(0x55cee954aa30 msgr2=0x55cee95553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.234+0000 7f3c97b48740 1 -- 172.21.15.62:0/3169150458 shutdown_connections 2024-09-11T23:52:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.234+0000 7f3c97b48740 1 -- 172.21.15.62:0/3169150458 wait complete. 2024-09-11T23:52:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.234+0000 7f0ea37fe640 1 -- 172.21.15.62:0/3268548381 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800cd80 con 0x7f0eac100880 2024-09-11T23:52:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.234+0000 7f0ea37fe640 1 -- 172.21.15.62:0/3268548381 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea8005500 con 0x7f0eac100880 2024-09-11T23:52:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.234+0000 7f0eb42f1640 1 -- 172.21.15.62:0/3268548381 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac19f880 con 0x7f0eac100880 2024-09-11T23:52:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.234+0000 7f0ea37fe640 1 -- 172.21.15.62:0/3268548381 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8010690 con 0x7f0eac100880 2024-09-11T23:52:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.234+0000 7f0eb42f1640 1 -- 172.21.15.62:0/3268548381 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac1a7110 con 0x7f0eac100880 2024-09-11T23:52:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.234+0000 7f3c97b48740 1 -- start start 2024-09-11T23:52:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.234+0000 7f3c97b48740 1 Processor -- start 2024-09-11T23:52:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954e4a0 0x55cee956e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954edc0 0x55cee956e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c97b48740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee9559680 0x55cee956b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c97b48740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c901a9c70 con 0x55cee954e4a0 2024-09-11T23:52:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c97b48740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c901ab0e0 con 0x55cee9559680 2024-09-11T23:52:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c97b48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c901aa340 con 0x55cee954edc0 2024-09-11T23:52:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c97b48740 10 client.0 _ll_register_callbacks cb 0x7f3c80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:15.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f0eb42f1640 1 -- 172.21.15.62:0/3268548381 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eac100c60 con 0x7f0eac100880 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954edc0 0x55cee956e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954edc0 0x55cee956e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:58780/0 (socket says 172.21.15.62:58780) 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/1992639400 learned_addr learned my addr 172.21.15.62:0/1992639400 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c8f7fe640 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee9559680 0x55cee956b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/1992639400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee9559680 msgr2=0x55cee956b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee9559680 0x55cee956b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/1992639400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954e4a0 msgr2=0x55cee956e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954e4a0 0x55cee956e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.235+0000 7f3c8e7fc640 1 -- 172.21.15.62:0/1992639400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cee956bd10 con 0x55cee954edc0 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.236+0000 7f3c8e7fc640 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954edc0 0x55cee956e640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c840cd7b0 tx=0x7f3c8409c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8a713475f60c76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:15.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.236+0000 7f3c72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:52:15.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.236+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1992639400 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840cd5b0 con 0x55cee954edc0 2024-09-11T23:52:15.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.236+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1992639400 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f3c8409b260 con 0x55cee954edc0 2024-09-11T23:52:15.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.236+0000 7f3c97b48740 1 -- 172.21.15.62:0/1992639400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c84065070 con 0x55cee954edc0 2024-09-11T23:52:15.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.236+0000 7f0ea37fe640 1 -- 172.21.15.62:0/3268548381 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f0ea80046c0 con 0x7f0eac100880 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.236+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1992639400 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3c840a4a80 con 0x55cee954edc0 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.236+0000 7f3c97b48740 10 client.25009 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.236+0000 7f3c97b48740 1 -- 172.21.15.62:0/1992639400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cee956bd10 con 0x55cee954edc0 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1992639400 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c84065070 con 0x55cee954edc0 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1992639400 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3c840baa50 con 0x55cee954edc0 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f3c97b48740 10 client.25009 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f3c97b48740 10 client.25009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f3c97b48740 20 client.25009 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cee95d3660 0x55cee95d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f3c97b48740 4 client.25009 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:52:15.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f3c97b48740 1 -- 172.21.15.62:0/1992639400 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cee94bbd60 con 0x55cee95d3660 2024-09-11T23:52:15.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f0ea37fe640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0e980a2e40 0x7f0e980a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:15.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f0ea37fe640 1 -- 172.21.15.62:0/3268548381 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f0ea80cfc20 con 0x7f0eac100880 2024-09-11T23:52:15.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f3c8effd640 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cee95d3660 0x55cee95d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.237+0000 7f0eb188c640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0e980a2e40 0x7f0e980a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:15.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c72ff5640 10 client.25009 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:15.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c8effd640 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cee95d3660 0x55cee95d5a50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9dc1e9edd500964 server_cookie=9aeed9128221a150 in_seq=0 out_seq=0 2024-09-11T23:52:15.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c72ff5640 1 -- 172.21.15.62:0/1992639400 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cee94bbd60 con 0x55cee95d3660 2024-09-11T23:52:15.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c72ff5640 10 client.25009 handle_command_reply: tid=0 2024-09-11T23:52:15.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c97b48740 1 client.25009 shutdown 2024-09-11T23:52:15.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f0eb188c640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0e980a2e40 0x7f0e980a5300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0eac1026f0 tx=0x7f0e9c005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:15.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c97b48740 1 -- 172.21.15.62:0/1992639400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954edc0 msgr2=0x55cee956e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954edc0 0x55cee956e640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c840cd7b0 tx=0x7f3c8409c510 comp rx=0 tx=0).stop 2024-09-11T23:52:15.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c97b48740 1 -- 172.21.15.62:0/1992639400 shutdown_connections 2024-09-11T23:52:15.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cee95d3660 0x55cee95d5a50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cee9559680 0x55cee956b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c72ff5640 0 client.25009 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:15.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cee954edc0 0x55cee956e640 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c97b48740 1 --2- 172.21.15.62:0/1992639400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cee954e4a0 0x55cee956e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c97b48740 1 -- 172.21.15.62:0/1992639400 >> 172.21.15.62:0/1992639400 conn(0x55cee954aa30 msgr2=0x55cee9549350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:15.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.238+0000 7f3c97b48740 1 -- 172.21.15.62:0/1992639400 shutdown_connections 2024-09-11T23:52:15.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.239+0000 7f3c97b48740 1 -- 172.21.15.62:0/1992639400 wait complete. 2024-09-11T23:52:15.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.239+0000 7f3c97b48740 20 client.25009 trim_cache size 0 max 16384 2024-09-11T23:52:15.251 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:52:15.252 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:52:15.252 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:52:15.252 INFO:teuthology.orchestra.run.smithi062.stdout: "2e29c1c6-de2f-4296-b73f-c9ba759c820d": { 2024-09-11T23:52:15.252 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:52:15.252 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "2e29c1c6-de2f-4296-b73f-c9ba759c820d", 2024-09-11T23:52:15.252 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:52:15.252 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:52:15.252 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:52:15.252 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:52:15.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 -- 172.21.15.62:0/2144288212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3c780a2d80 msgr2=0x7f3c780a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3c780a2d80 0x7f3c780a5240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3c88005c40 tx=0x7f3c88009200 comp rx=0 tx=0).stop 2024-09-11T23:52:15.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 -- 172.21.15.62:0/2144288212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c901027a0 msgr2=0x7f3c90074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c901027a0 0x7f3c90074a30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400e9f0 tx=0x7f3c8400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 -- 172.21.15.62:0/2144288212 shutdown_connections 2024-09-11T23:52:15.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3c780a2d80 0x7f3c780a5240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c90106f20 0x7f3c90074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c901027a0 0x7f3c90074a30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 --2- 172.21.15.62:0/2144288212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c90101df0 0x7f3c900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 -- 172.21.15.62:0/2144288212 >> 172.21.15.62:0/2144288212 conn(0x7f3c900fd8f0 msgr2=0x7f3c900fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:15.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 -- 172.21.15.62:0/2144288212 shutdown_connections 2024-09-11T23:52:15.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.240+0000 7f3c9576f640 1 -- 172.21.15.62:0/2144288212 wait complete. 2024-09-11T23:52:15.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.241+0000 7f0ea37fe640 1 -- 172.21.15.62:0/3268548381 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ea80541b0 con 0x7f0eac100880 2024-09-11T23:52:15.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.453+0000 7f0eb42f1640 1 -- 172.21.15.62:0/3268548381 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0eac062630 con 0x7f0e980a2e40 2024-09-11T23:52:15.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.460+0000 7f0ea37fe640 1 -- 172.21.15.62:0/3268548381 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0eac062630 con 0x7f0e980a2e40 2024-09-11T23:52:15.460 INFO:teuthology.orchestra.run.smithi062.stdout:/ 1m 6m3h 2024-09-11T23:52:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 -- 172.21.15.62:0/3268548381 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0e980a2e40 msgr2=0x7f0e980a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0e980a2e40 0x7f0e980a5300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0eac1026f0 tx=0x7f0e9c005a20 comp rx=0 tx=0).stop 2024-09-11T23:52:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 -- 172.21.15.62:0/3268548381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac100880 msgr2=0x7f0eac19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac100880 0x7f0eac19a5a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800e9f0 tx=0x7f0ea800edf0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 -- 172.21.15.62:0/3268548381 shutdown_connections 2024-09-11T23:52:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0e980a2e40 0x7f0e980a5300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0eac10e430 0x7f0eac19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0eac101230 0x7f0eac19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 --2- 172.21.15.62:0/3268548381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac100880 0x7f0eac19a5a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 -- 172.21.15.62:0/3268548381 >> 172.21.15.62:0/3268548381 conn(0x7f0eac077990 msgr2=0x7f0eac0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 -- 172.21.15.62:0/3268548381 shutdown_connections 2024-09-11T23:52:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:15.463+0000 7f0eb42f1640 1 -- 172.21.15.62:0/3268548381 wait complete. 2024-09-11T23:52:15.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:15 smithi062 ceph-mon[30958]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 B/s wr, 21 op/s 2024-09-11T23:52:15.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:15 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:52:15.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:15 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:15.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:15 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:52:15.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:15 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:15.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:15 smithi062 ceph-mon[30958]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:15.474 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-11T23:52:15.502 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-11T23:52:15.513 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-11T23:52:15.557 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:52:15.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:52:15.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:15 smithi120 ceph-mon[35182]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 B/s wr, 21 op/s 2024-09-11T23:52:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:15 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:52:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:15 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:15 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:52:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:15 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:15 smithi120 ceph-mon[35182]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:15 smithi083 ceph-mon[34805]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 B/s wr, 21 op/s 2024-09-11T23:52:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:15 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:52:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:15 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:15 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:52:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:15 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:15 smithi083 ceph-mon[34805]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:15.959 INFO:tasks.cephfs.filesystem:scrub status for tag:2e29c1c6-de2f-4296-b73f-c9ba759c820d - {'path': '/', 'tag': '2e29c1c6-de2f-4296-b73f-c9ba759c820d', 'options': 'recursive,force'} 2024-09-11T23:52:16.047 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:16.543 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:16 smithi062 ceph-mon[30958]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:16.543 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:16 smithi062 ceph-mon[30958]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:16.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 -- 172.21.15.62:0/1749969962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3960101400 msgr2=0x7f396010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:16.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 --2- 172.21.15.62:0/1749969962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3960101400 0x7f396010d930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3958009930 tx=0x7f395802a420 comp rx=0 tx=0).stop 2024-09-11T23:52:16.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 -- 172.21.15.62:0/1749969962 shutdown_connections 2024-09-11T23:52:16.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 --2- 172.21.15.62:0/1749969962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3960101400 0x7f396010d930 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:16.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 --2- 172.21.15.62:0/1749969962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3960100a60 0x7f3960100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:16.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 --2- 172.21.15.62:0/1749969962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3960106a60 0x7f3960106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 -- 172.21.15.62:0/1749969962 >> 172.21.15.62:0/1749969962 conn(0x7f39600fc780 msgr2=0x7f39600feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 -- 172.21.15.62:0/1749969962 shutdown_connections 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 -- 172.21.15.62:0/1749969962 wait complete. 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 Processor -- start 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.543+0000 7f3966bd4640 1 -- start start 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f3966bd4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3960100a60 0x7f3960198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f3966bd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3960101400 0x7f3960198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f3966bd4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3960106a60 0x7f396019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f3966bd4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3960110ed0 con 0x7f3960101400 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f3966bd4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3960110d50 con 0x7f3960100a60 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f3966bd4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3960111050 con 0x7f3960106a60 2024-09-11T23:52:16.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f395ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3960101400 0x7f3960198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f395ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3960101400 0x7f3960198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53668/0 (socket says 172.21.15.62:53668) 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f395ffff640 1 -- 172.21.15.62:0/2174145361 learned_addr learned my addr 172.21.15.62:0/2174145361 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f3965171640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3960106a60 0x7f396019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f3964970640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3960100a60 0x7f3960198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f395ffff640 1 -- 172.21.15.62:0/2174145361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3960106a60 msgr2=0x7f396019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f395ffff640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3960106a60 0x7f396019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f395ffff640 1 -- 172.21.15.62:0/2174145361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3960100a60 msgr2=0x7f3960198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f395ffff640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3960100a60 0x7f3960198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f395ffff640 1 -- 172.21.15.62:0/2174145361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396019d400 con 0x7f3960101400 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.544+0000 7f3964970640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3960100a60 0x7f3960198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.545+0000 7f395ffff640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3960101400 0x7f3960198990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f395000eab0 tx=0x7f395000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.545+0000 7f395dffb640 1 -- 172.21.15.62:0/2174145361 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395000ce40 con 0x7f3960101400 2024-09-11T23:52:16.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.545+0000 7f3966bd4640 1 -- 172.21.15.62:0/2174145361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396019d6f0 con 0x7f3960101400 2024-09-11T23:52:16.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.545+0000 7f395dffb640 1 -- 172.21.15.62:0/2174145361 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3950011950 con 0x7f3960101400 2024-09-11T23:52:16.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.545+0000 7f395dffb640 1 -- 172.21.15.62:0/2174145361 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950010ae0 con 0x7f3960101400 2024-09-11T23:52:16.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.545+0000 7f3966bd4640 1 -- 172.21.15.62:0/2174145361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39601a4fd0 con 0x7f3960101400 2024-09-11T23:52:16.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.546+0000 7f3966bd4640 1 -- 172.21.15.62:0/2174145361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3960100ec0 con 0x7f3960101400 2024-09-11T23:52:16.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.547+0000 7f395dffb640 1 -- 172.21.15.62:0/2174145361 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f3950011b00 con 0x7f3960101400 2024-09-11T23:52:16.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.548+0000 7f395dffb640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f394c0a2d80 0x7f394c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:16.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.548+0000 7f3964970640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f394c0a2d80 0x7f394c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:16.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.548+0000 7f395dffb640 1 -- 172.21.15.62:0/2174145361 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f39500cfcb0 con 0x7f3960101400 2024-09-11T23:52:16.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.548+0000 7f3964970640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f394c0a2d80 0x7f394c0a5240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3948009930 tx=0x7f3948008040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:16.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.552+0000 7f395dffb640 1 -- 172.21.15.62:0/2174145361 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3950016030 con 0x7f3960101400 2024-09-11T23:52:16.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:16 smithi120 ceph-mon[35182]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:16.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:16 smithi120 ceph-mon[35182]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:16.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:16 smithi083 ceph-mon[34805]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:16.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:16 smithi083 ceph-mon[34805]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:16.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.724+0000 7f3966bd4640 1 -- 172.21.15.62:0/2174145361 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3960062630 con 0x7f394c0a2d80 2024-09-11T23:52:16.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.731+0000 7f395dffb640 1 -- 172.21.15.62:0/2174145361 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257998 (secure 0 0 0) 0x7f3960062630 con 0x7f394c0a2d80 2024-09-11T23:52:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:16.735 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:52:16.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.733+0000 7f3966bd4640 1 -- 172.21.15.62:0/2174145361 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f394c0a2d80 msgr2=0x7f394c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:16.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.733+0000 7f3966bd4640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f394c0a2d80 0x7f394c0a5240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3948009930 tx=0x7f3948008040 comp rx=0 tx=0).stop 2024-09-11T23:52:16.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.733+0000 7f3966bd4640 1 -- 172.21.15.62:0/2174145361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3960101400 msgr2=0x7f3960198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.733+0000 7f3966bd4640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3960101400 0x7f3960198990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f395000eab0 tx=0x7f395000ef70 comp rx=0 tx=0).stop 2024-09-11T23:52:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.733+0000 7f3966bd4640 1 -- 172.21.15.62:0/2174145361 shutdown_connections 2024-09-11T23:52:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.733+0000 7f3966bd4640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f394c0a2d80 0x7f394c0a5240 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.733+0000 7f3966bd4640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3960106a60 0x7f396019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.733+0000 7f3966bd4640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3960101400 0x7f3960198990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.733+0000 7f3966bd4640 1 --2- 172.21.15.62:0/2174145361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3960100a60 0x7f3960198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:16.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.733+0000 7f3966bd4640 1 -- 172.21.15.62:0/2174145361 >> 172.21.15.62:0/2174145361 conn(0x7f39600fc780 msgr2=0x7f396006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:16.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.734+0000 7f3966bd4640 1 -- 172.21.15.62:0/2174145361 shutdown_connections 2024-09-11T23:52:16.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:16.734+0000 7f3966bd4640 1 -- 172.21.15.62:0/2174145361 wait complete. 2024-09-11T23:52:17.345 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-11T23:52:15.447327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187092,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1030,"num_read_kb":835,"num_write":498,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":880,"ondisk_log_size":880,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348156,"kb_used_data":11340,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593668,"statfs":{"total":1151940427776,"available":1151583916032,"internally_reserved":0,"allocated":11612160,"data_stored":8905044,"data_compressed":218271,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":226,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.978561"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872692+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:52:05.872692+0000","last_peered":"2024-09-11T23:52:05.872692+0000","last_clean":"2024-09-11T23:52:05.872692+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:52:05.872692+0000","last_undegraded":"2024-09-11T23:52:05.872692+0000","last_fullsized":"2024-09-11T23:52:05.872692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158483+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:52:06.158483+0000","last_peered":"2024-09-11T23:52:06.158483+0000","last_clean":"2024-09-11T23:52:06.158483+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:52:06.158483+0000","last_undegraded":"2024-09-11T23:52:06.158483+0000","last_fullsized":"2024-09-11T23:52:06.158483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158255+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:52:06.158255+0000","last_peered":"2024-09-11T23:52:06.158255+0000","last_clean":"2024-09-11T23:52:06.158255+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:52:06.158255+0000","last_undegraded":"2024-09-11T23:52:06.158255+0000","last_fullsized":"2024-09-11T23:52:06.158255+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744925+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:52:05.744925+0000","last_peered":"2024-09-11T23:52:05.744925+0000","last_clean":"2024-09-11T23:52:05.744925+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:52:05.744925+0000","last_undegraded":"2024-09-11T23:52:05.744925+0000","last_fullsized":"2024-09-11T23:52:05.744925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516648+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:52:05.516648+0000","last_peered":"2024-09-11T23:52:05.516648+0000","last_clean":"2024-09-11T23:52:05.516648+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:52:05.516648+0000","last_undegraded":"2024-09-11T23:52:05.516648+0000","last_fullsized":"2024-09-11T23:52:05.516648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434378+0000","last_change":"2024-09-11T23:51:14.120226+0000","last_active":"2024-09-11T23:52:05.434378+0000","last_peered":"2024-09-11T23:52:05.434378+0000","last_clean":"2024-09-11T23:52:05.434378+0000","last_became_active":"2024-09-11T23:51:14.119811+0000","last_became_peered":"2024-09-11T23:51:14.119811+0000","last_unstale":"2024-09-11T23:52:05.434378+0000","last_undegraded":"2024-09-11T23:52:05.434378+0000","last_fullsized":"2024-09-11T23:52:05.434378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:58.851476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746317+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:52:05.746317+0000","last_peered":"2024-09-11T23:52:05.746317+0000","last_clean":"2024-09-11T23:52:05.746317+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:52:05.746317+0000","last_undegraded":"2024-09-11T23:52:05.746317+0000","last_fullsized":"2024-09-11T23:52:05.746317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523319+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:52:05.523319+0000","last_peered":"2024-09-11T23:52:05.523319+0000","last_clean":"2024-09-11T23:52:05.523319+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:52:05.523319+0000","last_undegraded":"2024-09-11T23:52:05.523319+0000","last_fullsized":"2024-09-11T23:52:05.523319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694954+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:52:05.694954+0000","last_peered":"2024-09-11T23:52:05.694954+0000","last_clean":"2024-09-11T23:52:05.694954+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:52:05.694954+0000","last_undegraded":"2024-09-11T23:52:05.694954+0000","last_fullsized":"2024-09-11T23:52:05.694954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435402+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:52:05.435402+0000","last_peered":"2024-09-11T23:52:05.435402+0000","last_clean":"2024-09-11T23:52:05.435402+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:52:05.435402+0000","last_undegraded":"2024-09-11T23:52:05.435402+0000","last_fullsized":"2024-09-11T23:52:05.435402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746473+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:52:05.746473+0000","last_peered":"2024-09-11T23:52:05.746473+0000","last_clean":"2024-09-11T23:52:05.746473+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:52:05.746473+0000","last_undegraded":"2024-09-11T23:52:05.746473+0000","last_fullsized":"2024-09-11T23:52:05.746473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744935+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:52:05.744935+0000","last_peered":"2024-09-11T23:52:05.744935+0000","last_clean":"2024-09-11T23:52:05.744935+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:52:05.744935+0000","last_undegraded":"2024-09-11T23:52:05.744935+0000","last_fullsized":"2024-09-11T23:52:05.744935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431713+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:52:05.431713+0000","last_peered":"2024-09-11T23:52:05.431713+0000","last_clean":"2024-09-11T23:52:05.431713+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:52:05.431713+0000","last_undegraded":"2024-09-11T23:52:05.431713+0000","last_fullsized":"2024-09-11T23:52:05.431713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158046+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:52:06.158046+0000","last_peered":"2024-09-11T23:52:06.158046+0000","last_clean":"2024-09-11T23:52:06.158046+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:52:06.158046+0000","last_undegraded":"2024-09-11T23:52:06.158046+0000","last_fullsized":"2024-09-11T23:52:06.158046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.157901+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:52:06.157901+0000","last_peered":"2024-09-11T23:52:06.157901+0000","last_clean":"2024-09-11T23:52:06.157901+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:52:06.157901+0000","last_undegraded":"2024-09-11T23:52:06.157901+0000","last_fullsized":"2024-09-11T23:52:06.157901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694419+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:52:05.694419+0000","last_peered":"2024-09-11T23:52:05.694419+0000","last_clean":"2024-09-11T23:52:05.694419+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:52:05.694419+0000","last_undegraded":"2024-09-11T23:52:05.694419+0000","last_fullsized":"2024-09-11T23:52:05.694419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872787+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:52:05.872787+0000","last_peered":"2024-09-11T23:52:05.872787+0000","last_clean":"2024-09-11T23:52:05.872787+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:52:05.872787+0000","last_undegraded":"2024-09-11T23:52:05.872787+0000","last_fullsized":"2024-09-11T23:52:05.872787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520950+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:52:05.520950+0000","last_peered":"2024-09-11T23:52:05.520950+0000","last_clean":"2024-09-11T23:52:05.520950+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:52:05.520950+0000","last_undegraded":"2024-09-11T23:52:05.520950+0000","last_fullsized":"2024-09-11T23:52:05.520950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694752+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:52:05.694752+0000","last_peered":"2024-09-11T23:52:05.694752+0000","last_clean":"2024-09-11T23:52:05.694752+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:52:05.694752+0000","last_undegraded":"2024-09-11T23:52:05.694752+0000","last_fullsized":"2024-09-11T23:52:05.694752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872460+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:52:05.872460+0000","last_peered":"2024-09-11T23:52:05.872460+0000","last_clean":"2024-09-11T23:52:05.872460+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:52:05.872460+0000","last_undegraded":"2024-09-11T23:52:05.872460+0000","last_fullsized":"2024-09-11T23:52:05.872460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431851+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:52:05.431851+0000","last_peered":"2024-09-11T23:52:05.431851+0000","last_clean":"2024-09-11T23:52:05.431851+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:52:05.431851+0000","last_undegraded":"2024-09-11T23:52:05.431851+0000","last_fullsized":"2024-09-11T23:52:05.431851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694603+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:52:05.694603+0000","last_peered":"2024-09-11T23:52:05.694603+0000","last_clean":"2024-09-11T23:52:05.694603+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:52:05.694603+0000","last_undegraded":"2024-09-11T23:52:05.694603+0000","last_fullsized":"2024-09-11T23:52:05.694603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523509+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:52:05.523509+0000","last_peered":"2024-09-11T23:52:05.523509+0000","last_clean":"2024-09-11T23:52:05.523509+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:52:05.523509+0000","last_undegraded":"2024-09-11T23:52:05.523509+0000","last_fullsized":"2024-09-11T23:52:05.523509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432812+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:52:05.432812+0000","last_peered":"2024-09-11T23:52:05.432812+0000","last_clean":"2024-09-11T23:52:05.432812+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:52:05.432812+0000","last_undegraded":"2024-09-11T23:52:05.432812+0000","last_fullsized":"2024-09-11T23:52:05.432812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525015+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:52:05.525015+0000","last_peered":"2024-09-11T23:52:05.525015+0000","last_clean":"2024-09-11T23:52:05.525015+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:52:05.525015+0000","last_undegraded":"2024-09-11T23:52:05.525015+0000","last_fullsized":"2024-09-11T23:52:05.525015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694943+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:52:05.694943+0000","last_peered":"2024-09-11T23:52:05.694943+0000","last_clean":"2024-09-11T23:52:05.694943+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:52:05.694943+0000","last_undegraded":"2024-09-11T23:52:05.694943+0000","last_fullsized":"2024-09-11T23:52:05.694943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872635+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:52:05.872635+0000","last_peered":"2024-09-11T23:52:05.872635+0000","last_clean":"2024-09-11T23:52:05.872635+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:52:05.872635+0000","last_undegraded":"2024-09-11T23:52:05.872635+0000","last_fullsized":"2024-09-11T23:52:05.872635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431698+0000","last_change":"2024-09-11T23:51:14.410108+0000","last_active":"2024-09-11T23:52:05.431698+0000","last_peered":"2024-09-11T23:52:05.431698+0000","last_clean":"2024-09-11T23:52:05.431698+0000","last_became_active":"2024-09-11T23:51:14.409588+0000","last_became_peered":"2024-09-11T23:51:14.409588+0000","last_unstale":"2024-09-11T23:52:05.431698+0000","last_undegraded":"2024-09-11T23:52:05.431698+0000","last_fullsized":"2024-09-11T23:52:05.431698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:29.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158121+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:52:06.158121+0000","last_peered":"2024-09-11T23:52:06.158121+0000","last_clean":"2024-09-11T23:52:06.158121+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:52:06.158121+0000","last_undegraded":"2024-09-11T23:52:06.158121+0000","last_fullsized":"2024-09-11T23:52:06.158121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694617+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:52:05.694617+0000","last_peered":"2024-09-11T23:52:05.694617+0000","last_clean":"2024-09-11T23:52:05.694617+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:52:05.694617+0000","last_undegraded":"2024-09-11T23:52:05.694617+0000","last_fullsized":"2024-09-11T23:52:05.694617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694660+0000","last_change":"2024-09-11T23:51:14.410872+0000","last_active":"2024-09-11T23:52:05.694660+0000","last_peered":"2024-09-11T23:52:05.694660+0000","last_clean":"2024-09-11T23:52:05.694660+0000","last_became_active":"2024-09-11T23:51:14.410531+0000","last_became_peered":"2024-09-11T23:51:14.410531+0000","last_unstale":"2024-09-11T23:52:05.694660+0000","last_undegraded":"2024-09-11T23:52:05.694660+0000","last_fullsized":"2024-09-11T23:52:05.694660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:10.502914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872498+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:52:05.872498+0000","last_peered":"2024-09-11T23:52:05.872498+0000","last_clean":"2024-09-11T23:52:05.872498+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:52:05.872498+0000","last_undegraded":"2024-09-11T23:52:05.872498+0000","last_fullsized":"2024-09-11T23:52:05.872498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516667+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:52:05.516667+0000","last_peered":"2024-09-11T23:52:05.516667+0000","last_clean":"2024-09-11T23:52:05.516667+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:52:05.516667+0000","last_undegraded":"2024-09-11T23:52:05.516667+0000","last_fullsized":"2024-09-11T23:52:05.516667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:10.018988+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:52:10.018988+0000","last_peered":"2024-09-11T23:52:10.018988+0000","last_clean":"2024-09-11T23:52:10.018988+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:52:10.018988+0000","last_undegraded":"2024-09-11T23:52:10.018988+0000","last_fullsized":"2024-09-11T23:52:10.018988+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432914+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:52:05.432914+0000","last_peered":"2024-09-11T23:52:05.432914+0000","last_clean":"2024-09-11T23:52:05.432914+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:52:05.432914+0000","last_undegraded":"2024-09-11T23:52:05.432914+0000","last_fullsized":"2024-09-11T23:52:05.432914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434413+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:52:05.434413+0000","last_peered":"2024-09-11T23:52:05.434413+0000","last_clean":"2024-09-11T23:52:05.434413+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:52:05.434413+0000","last_undegraded":"2024-09-11T23:52:05.434413+0000","last_fullsized":"2024-09-11T23:52:05.434413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431356+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:52:05.431356+0000","last_peered":"2024-09-11T23:52:05.431356+0000","last_clean":"2024-09-11T23:52:05.431356+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:52:05.431356+0000","last_undegraded":"2024-09-11T23:52:05.431356+0000","last_fullsized":"2024-09-11T23:52:05.431356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523343+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:52:05.523343+0000","last_peered":"2024-09-11T23:52:05.523343+0000","last_clean":"2024-09-11T23:52:05.523343+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:52:05.523343+0000","last_undegraded":"2024-09-11T23:52:05.523343+0000","last_fullsized":"2024-09-11T23:52:05.523343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433385+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:52:05.433385+0000","last_peered":"2024-09-11T23:52:05.433385+0000","last_clean":"2024-09-11T23:52:05.433385+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:52:05.433385+0000","last_undegraded":"2024-09-11T23:52:05.433385+0000","last_fullsized":"2024-09-11T23:52:05.433385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520843+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:52:05.520843+0000","last_peered":"2024-09-11T23:52:05.520843+0000","last_clean":"2024-09-11T23:52:05.520843+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:52:05.520843+0000","last_undegraded":"2024-09-11T23:52:05.520843+0000","last_fullsized":"2024-09-11T23:52:05.520843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:10.018497+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:52:10.018497+0000","last_peered":"2024-09-11T23:52:10.018497+0000","last_clean":"2024-09-11T23:52:10.018497+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:52:10.018497+0000","last_undegraded":"2024-09-11T23:52:10.018497+0000","last_fullsized":"2024-09-11T23:52:10.018497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694869+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:52:05.694869+0000","last_peered":"2024-09-11T23:52:05.694869+0000","last_clean":"2024-09-11T23:52:05.694869+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:52:05.694869+0000","last_undegraded":"2024-09-11T23:52:05.694869+0000","last_fullsized":"2024-09-11T23:52:05.694869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746642+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:52:05.746642+0000","last_peered":"2024-09-11T23:52:05.746642+0000","last_clean":"2024-09-11T23:52:05.746642+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:52:05.746642+0000","last_undegraded":"2024-09-11T23:52:05.746642+0000","last_fullsized":"2024-09-11T23:52:05.746642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158655+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:52:06.158655+0000","last_peered":"2024-09-11T23:52:06.158655+0000","last_clean":"2024-09-11T23:52:06.158655+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:52:06.158655+0000","last_undegraded":"2024-09-11T23:52:06.158655+0000","last_fullsized":"2024-09-11T23:52:06.158655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433170+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:52:05.433170+0000","last_peered":"2024-09-11T23:52:05.433170+0000","last_clean":"2024-09-11T23:52:05.433170+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:52:05.433170+0000","last_undegraded":"2024-09-11T23:52:05.433170+0000","last_fullsized":"2024-09-11T23:52:05.433170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694598+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:52:05.694598+0000","last_peered":"2024-09-11T23:52:05.694598+0000","last_clean":"2024-09-11T23:52:05.694598+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:52:05.694598+0000","last_undegraded":"2024-09-11T23:52:05.694598+0000","last_fullsized":"2024-09-11T23:52:05.694598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'36","reported_seq":294,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:11.019777+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:52:11.019777+0000","last_peered":"2024-09-11T23:52:11.019777+0000","last_clean":"2024-09-11T23:52:11.019777+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:52:11.019777+0000","last_undegraded":"2024-09-11T23:52:11.019777+0000","last_fullsized":"2024-09-11T23:52:11.019777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66436,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":181,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516931+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:52:05.516931+0000","last_peered":"2024-09-11T23:52:05.516931+0000","last_clean":"2024-09-11T23:52:05.516931+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:52:05.516931+0000","last_undegraded":"2024-09-11T23:52:05.516931+0000","last_fullsized":"2024-09-11T23:52:05.516931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694704+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:52:05.694704+0000","last_peered":"2024-09-11T23:52:05.694704+0000","last_clean":"2024-09-11T23:52:05.694704+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:52:05.694704+0000","last_undegraded":"2024-09-11T23:52:05.694704+0000","last_fullsized":"2024-09-11T23:52:05.694704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431407+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:52:05.431407+0000","last_peered":"2024-09-11T23:52:05.431407+0000","last_clean":"2024-09-11T23:52:05.431407+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:52:05.431407+0000","last_undegraded":"2024-09-11T23:52:05.431407+0000","last_fullsized":"2024-09-11T23:52:05.431407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433521+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:52:05.433521+0000","last_peered":"2024-09-11T23:52:05.433521+0000","last_clean":"2024-09-11T23:52:05.433521+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:52:05.433521+0000","last_undegraded":"2024-09-11T23:52:05.433521+0000","last_fullsized":"2024-09-11T23:52:05.433521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746620+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:52:05.746620+0000","last_peered":"2024-09-11T23:52:05.746620+0000","last_clean":"2024-09-11T23:52:05.746620+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:52:05.746620+0000","last_undegraded":"2024-09-11T23:52:05.746620+0000","last_fullsized":"2024-09-11T23:52:05.746620+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":148,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:10.018967+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:52:10.018967+0000","last_peered":"2024-09-11T23:52:10.018967+0000","last_clean":"2024-09-11T23:52:10.018967+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:52:10.018967+0000","last_undegraded":"2024-09-11T23:52:10.018967+0000","last_fullsized":"2024-09-11T23:52:10.018967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746281+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:52:05.746281+0000","last_peered":"2024-09-11T23:52:05.746281+0000","last_clean":"2024-09-11T23:52:05.746281+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:52:05.746281+0000","last_undegraded":"2024-09-11T23:52:05.746281+0000","last_fullsized":"2024-09-11T23:52:05.746281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516591+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:52:05.516591+0000","last_peered":"2024-09-11T23:52:05.516591+0000","last_clean":"2024-09-11T23:52:05.516591+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:52:05.516591+0000","last_undegraded":"2024-09-11T23:52:05.516591+0000","last_fullsized":"2024-09-11T23:52:05.516591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432801+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:52:05.432801+0000","last_peered":"2024-09-11T23:52:05.432801+0000","last_clean":"2024-09-11T23:52:05.432801+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:52:05.432801+0000","last_undegraded":"2024-09-11T23:52:05.432801+0000","last_fullsized":"2024-09-11T23:52:05.432801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516946+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:52:05.516946+0000","last_peered":"2024-09-11T23:52:05.516946+0000","last_clean":"2024-09-11T23:52:05.516946+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:52:05.516946+0000","last_undegraded":"2024-09-11T23:52:05.516946+0000","last_fullsized":"2024-09-11T23:52:05.516946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872534+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:52:05.872534+0000","last_peered":"2024-09-11T23:52:05.872534+0000","last_clean":"2024-09-11T23:52:05.872534+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:52:05.872534+0000","last_undegraded":"2024-09-11T23:52:05.872534+0000","last_fullsized":"2024-09-11T23:52:05.872534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431687+0000","last_change":"2024-09-11T23:51:14.413526+0000","last_active":"2024-09-11T23:52:05.431687+0000","last_peered":"2024-09-11T23:52:05.431687+0000","last_clean":"2024-09-11T23:52:05.431687+0000","last_became_active":"2024-09-11T23:51:14.411027+0000","last_became_peered":"2024-09-11T23:51:14.411027+0000","last_unstale":"2024-09-11T23:52:05.431687+0000","last_undegraded":"2024-09-11T23:52:05.431687+0000","last_fullsized":"2024-09-11T23:52:05.431687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:55.020567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746675+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:52:05.746675+0000","last_peered":"2024-09-11T23:52:05.746675+0000","last_clean":"2024-09-11T23:52:05.746675+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:52:05.746675+0000","last_undegraded":"2024-09-11T23:52:05.746675+0000","last_fullsized":"2024-09-11T23:52:05.746675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.642614+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:52:05.642614+0000","last_peered":"2024-09-11T23:52:05.642614+0000","last_clean":"2024-09-11T23:52:05.642614+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:52:05.642614+0000","last_undegraded":"2024-09-11T23:52:05.642614+0000","last_fullsized":"2024-09-11T23:52:05.642614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746458+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:52:05.746458+0000","last_peered":"2024-09-11T23:52:05.746458+0000","last_clean":"2024-09-11T23:52:05.746458+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:52:05.746458+0000","last_undegraded":"2024-09-11T23:52:05.746458+0000","last_fullsized":"2024-09-11T23:52:05.746458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694804+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:52:05.694804+0000","last_peered":"2024-09-11T23:52:05.694804+0000","last_clean":"2024-09-11T23:52:05.694804+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:52:05.694804+0000","last_undegraded":"2024-09-11T23:52:05.694804+0000","last_fullsized":"2024-09-11T23:52:05.694804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746570+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:52:05.746570+0000","last_peered":"2024-09-11T23:52:05.746570+0000","last_clean":"2024-09-11T23:52:05.746570+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:52:05.746570+0000","last_undegraded":"2024-09-11T23:52:05.746570+0000","last_fullsized":"2024-09-11T23:52:05.746570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"84'148","reported_seq":394,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:07.529793+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:52:07.529793+0000","last_peered":"2024-09-11T23:52:07.529793+0000","last_clean":"2024-09-11T23:52:07.529793+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:52:07.529793+0000","last_undegraded":"2024-09-11T23:52:07.529793+0000","last_fullsized":"2024-09-11T23:52:07.529793+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744872+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:52:05.744872+0000","last_peered":"2024-09-11T23:52:05.744872+0000","last_clean":"2024-09-11T23:52:05.744872+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:52:05.744872+0000","last_undegraded":"2024-09-11T23:52:05.744872+0000","last_fullsized":"2024-09-11T23:52:05.744872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523710+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:52:05.523710+0000","last_peered":"2024-09-11T23:52:05.523710+0000","last_clean":"2024-09-11T23:52:05.523710+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:52:05.523710+0000","last_undegraded":"2024-09-11T23:52:05.523710+0000","last_fullsized":"2024-09-11T23:52:05.523710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872415+0000","last_change":"2024-09-11T23:51:14.121480+0000","last_active":"2024-09-11T23:52:05.872415+0000","last_peered":"2024-09-11T23:52:05.872415+0000","last_clean":"2024-09-11T23:52:05.872415+0000","last_became_active":"2024-09-11T23:51:14.118435+0000","last_became_peered":"2024-09-11T23:51:14.118435+0000","last_unstale":"2024-09-11T23:52:05.872415+0000","last_undegraded":"2024-09-11T23:52:05.872415+0000","last_fullsized":"2024-09-11T23:52:05.872415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431342+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:52:05.431342+0000","last_peered":"2024-09-11T23:52:05.431342+0000","last_clean":"2024-09-11T23:52:05.431342+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:52:05.431342+0000","last_undegraded":"2024-09-11T23:52:05.431342+0000","last_fullsized":"2024-09-11T23:52:05.431342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":141,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:10.018639+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:52:10.018639+0000","last_peered":"2024-09-11T23:52:10.018639+0000","last_clean":"2024-09-11T23:52:10.018639+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:52:10.018639+0000","last_undegraded":"2024-09-11T23:52:10.018639+0000","last_fullsized":"2024-09-11T23:52:10.018639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432911+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:52:05.432911+0000","last_peered":"2024-09-11T23:52:05.432911+0000","last_clean":"2024-09-11T23:52:05.432911+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:52:05.432911+0000","last_undegraded":"2024-09-11T23:52:05.432911+0000","last_fullsized":"2024-09-11T23:52:05.432911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746522+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:52:05.746522+0000","last_peered":"2024-09-11T23:52:05.746522+0000","last_clean":"2024-09-11T23:52:05.746522+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:52:05.746522+0000","last_undegraded":"2024-09-11T23:52:05.746522+0000","last_fullsized":"2024-09-11T23:52:05.746522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435012+0000","last_change":"2024-09-11T23:51:14.409908+0000","last_active":"2024-09-11T23:52:05.435012+0000","last_peered":"2024-09-11T23:52:05.435012+0000","last_clean":"2024-09-11T23:52:05.435012+0000","last_became_active":"2024-09-11T23:51:14.409432+0000","last_became_peered":"2024-09-11T23:51:14.409432+0000","last_unstale":"2024-09-11T23:52:05.435012+0000","last_undegraded":"2024-09-11T23:52:05.435012+0000","last_fullsized":"2024-09-11T23:52:05.435012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:06.371306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.521172+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:52:05.521172+0000","last_peered":"2024-09-11T23:52:05.521172+0000","last_clean":"2024-09-11T23:52:05.521172+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:52:05.521172+0000","last_undegraded":"2024-09-11T23:52:05.521172+0000","last_fullsized":"2024-09-11T23:52:05.521172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435167+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:52:05.435167+0000","last_peered":"2024-09-11T23:52:05.435167+0000","last_clean":"2024-09-11T23:52:05.435167+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:52:05.435167+0000","last_undegraded":"2024-09-11T23:52:05.435167+0000","last_fullsized":"2024-09-11T23:52:05.435167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.517017+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:52:05.517017+0000","last_peered":"2024-09-11T23:52:05.517017+0000","last_clean":"2024-09-11T23:52:05.517017+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:52:05.517017+0000","last_undegraded":"2024-09-11T23:52:05.517017+0000","last_fullsized":"2024-09-11T23:52:05.517017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431772+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:52:05.431772+0000","last_peered":"2024-09-11T23:52:05.431772+0000","last_clean":"2024-09-11T23:52:05.431772+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:52:05.431772+0000","last_undegraded":"2024-09-11T23:52:05.431772+0000","last_fullsized":"2024-09-11T23:52:05.431772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'48","reported_seq":95,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.649913+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:52:05.649913+0000","last_peered":"2024-09-11T23:52:05.649913+0000","last_clean":"2024-09-11T23:52:05.649913+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:52:05.649913+0000","last_undegraded":"2024-09-11T23:52:05.649913+0000","last_fullsized":"2024-09-11T23:52:05.649913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525022+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:52:05.525022+0000","last_peered":"2024-09-11T23:52:05.525022+0000","last_clean":"2024-09-11T23:52:05.525022+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:52:05.525022+0000","last_undegraded":"2024-09-11T23:52:05.525022+0000","last_fullsized":"2024-09-11T23:52:05.525022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.745108+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:52:05.745108+0000","last_peered":"2024-09-11T23:52:05.745108+0000","last_clean":"2024-09-11T23:52:05.745108+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:52:05.745108+0000","last_undegraded":"2024-09-11T23:52:05.745108+0000","last_fullsized":"2024-09-11T23:52:05.745108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158521+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:52:06.158521+0000","last_peered":"2024-09-11T23:52:06.158521+0000","last_clean":"2024-09-11T23:52:06.158521+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:52:06.158521+0000","last_undegraded":"2024-09-11T23:52:06.158521+0000","last_fullsized":"2024-09-11T23:52:06.158521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694501+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:52:05.694501+0000","last_peered":"2024-09-11T23:52:05.694501+0000","last_clean":"2024-09-11T23:52:05.694501+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:52:05.694501+0000","last_undegraded":"2024-09-11T23:52:05.694501+0000","last_fullsized":"2024-09-11T23:52:05.694501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516588+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:52:05.516588+0000","last_peered":"2024-09-11T23:52:05.516588+0000","last_clean":"2024-09-11T23:52:05.516588+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:52:05.516588+0000","last_undegraded":"2024-09-11T23:52:05.516588+0000","last_fullsized":"2024-09-11T23:52:05.516588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523605+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:52:05.523605+0000","last_peered":"2024-09-11T23:52:05.523605+0000","last_clean":"2024-09-11T23:52:05.523605+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:52:05.523605+0000","last_undegraded":"2024-09-11T23:52:05.523605+0000","last_fullsized":"2024-09-11T23:52:05.523605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872645+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:52:05.872645+0000","last_peered":"2024-09-11T23:52:05.872645+0000","last_clean":"2024-09-11T23:52:05.872645+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:52:05.872645+0000","last_undegraded":"2024-09-11T23:52:05.872645+0000","last_fullsized":"2024-09-11T23:52:05.872645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872496+0000","last_change":"2024-09-11T23:51:14.123663+0000","last_active":"2024-09-11T23:52:05.872496+0000","last_peered":"2024-09-11T23:52:05.872496+0000","last_clean":"2024-09-11T23:52:05.872496+0000","last_became_active":"2024-09-11T23:51:14.118315+0000","last_became_peered":"2024-09-11T23:51:14.118315+0000","last_unstale":"2024-09-11T23:52:05.872496+0000","last_undegraded":"2024-09-11T23:52:05.872496+0000","last_fullsized":"2024-09-11T23:52:05.872496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:10.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.524009+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:52:05.524009+0000","last_peered":"2024-09-11T23:52:05.524009+0000","last_clean":"2024-09-11T23:52:05.524009+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:52:05.524009+0000","last_undegraded":"2024-09-11T23:52:05.524009+0000","last_fullsized":"2024-09-11T23:52:05.524009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516698+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:52:05.516698+0000","last_peered":"2024-09-11T23:52:05.516698+0000","last_clean":"2024-09-11T23:52:05.516698+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:52:05.516698+0000","last_undegraded":"2024-09-11T23:52:05.516698+0000","last_fullsized":"2024-09-11T23:52:05.516698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746447+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:52:05.746447+0000","last_peered":"2024-09-11T23:52:05.746447+0000","last_clean":"2024-09-11T23:52:05.746447+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:52:05.746447+0000","last_undegraded":"2024-09-11T23:52:05.746447+0000","last_fullsized":"2024-09-11T23:52:05.746447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433119+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:52:05.433119+0000","last_peered":"2024-09-11T23:52:05.433119+0000","last_clean":"2024-09-11T23:52:05.433119+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:52:05.433119+0000","last_undegraded":"2024-09-11T23:52:05.433119+0000","last_fullsized":"2024-09-11T23:52:05.433119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516897+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:52:05.516897+0000","last_peered":"2024-09-11T23:52:05.516897+0000","last_clean":"2024-09-11T23:52:05.516897+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:52:05.516897+0000","last_undegraded":"2024-09-11T23:52:05.516897+0000","last_fullsized":"2024-09-11T23:52:05.516897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694561+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:52:05.694561+0000","last_peered":"2024-09-11T23:52:05.694561+0000","last_clean":"2024-09-11T23:52:05.694561+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:52:05.694561+0000","last_undegraded":"2024-09-11T23:52:05.694561+0000","last_fullsized":"2024-09-11T23:52:05.694561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516945+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:52:05.516945+0000","last_peered":"2024-09-11T23:52:05.516945+0000","last_clean":"2024-09-11T23:52:05.516945+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:52:05.516945+0000","last_undegraded":"2024-09-11T23:52:05.516945+0000","last_fullsized":"2024-09-11T23:52:05.516945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158617+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:52:06.158617+0000","last_peered":"2024-09-11T23:52:06.158617+0000","last_clean":"2024-09-11T23:52:06.158617+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:52:06.158617+0000","last_undegraded":"2024-09-11T23:52:06.158617+0000","last_fullsized":"2024-09-11T23:52:06.158617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525435+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:52:05.525435+0000","last_peered":"2024-09-11T23:52:05.525435+0000","last_clean":"2024-09-11T23:52:05.525435+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:52:05.525435+0000","last_undegraded":"2024-09-11T23:52:05.525435+0000","last_fullsized":"2024-09-11T23:52:05.525435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525158+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:52:05.525158+0000","last_peered":"2024-09-11T23:52:05.525158+0000","last_clean":"2024-09-11T23:52:05.525158+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:52:05.525158+0000","last_undegraded":"2024-09-11T23:52:05.525158+0000","last_fullsized":"2024-09-11T23:52:05.525158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":51,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.521585+0000","last_change":"2024-09-11T23:51:14.125596+0000","last_active":"2024-09-11T23:52:05.521585+0000","last_peered":"2024-09-11T23:52:05.521585+0000","last_clean":"2024-09-11T23:52:05.521585+0000","last_became_active":"2024-09-11T23:51:14.119682+0000","last_became_peered":"2024-09-11T23:51:14.119682+0000","last_unstale":"2024-09-11T23:52:05.521585+0000","last_undegraded":"2024-09-11T23:52:05.521585+0000","last_fullsized":"2024-09-11T23:52:05.521585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:06.062602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596490,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":793,"num_read_kb":505,"num_write":295,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1789470,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":909312,"data_stored":445896,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1089536,"data_stored":1236648,"data_compressed":44911,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":839348,"data_compressed":22046,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":838686,"data_compressed":22046,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1039219,"data_compressed":38755,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445521,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1169268,"data_compressed":44911,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":447193,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":974848,"data_stored":970648,"data_compressed":28202,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":513493,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":512332,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":446792,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133469,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":200849,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:52:17.350 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:52:17.606 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:17 smithi120 ceph-mon[35182]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-11T23:52:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:17 smithi120 ceph-mon[35182]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:17 smithi120 ceph-mon[35182]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:17 smithi083 ceph-mon[34805]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-11T23:52:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:17 smithi083 ceph-mon[34805]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:17 smithi083 ceph-mon[34805]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:17 smithi062 ceph-mon[30958]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-11T23:52:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:17 smithi062 ceph-mon[30958]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:52:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:17 smithi062 ceph-mon[30958]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.062+0000 7ff85cb63640 1 -- 172.21.15.62:0/1909509830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8581036b0 msgr2=0x7ff85810bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.062+0000 7ff85cb63640 1 --2- 172.21.15.62:0/1909509830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8581036b0 0x7ff85810bba0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009900 tx=0x7ff84c02a480 comp rx=0 tx=0).stop 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.062+0000 7ff85cb63640 1 -- 172.21.15.62:0/1909509830 shutdown_connections 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.062+0000 7ff85cb63640 1 --2- 172.21.15.62:0/1909509830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff85810c170 0x7ff85810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.062+0000 7ff85cb63640 1 --2- 172.21.15.62:0/1909509830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8581036b0 0x7ff85810bba0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.062+0000 7ff85cb63640 1 --2- 172.21.15.62:0/1909509830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff858102d90 0x7ff858103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.062+0000 7ff85cb63640 1 -- 172.21.15.62:0/1909509830 >> 172.21.15.62:0/1909509830 conn(0x7ff8580fc790 msgr2=0x7ff8580febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.062+0000 7ff85cb63640 1 -- 172.21.15.62:0/1909509830 shutdown_connections 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.062+0000 7ff85cb63640 1 -- 172.21.15.62:0/1909509830 wait complete. 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff85cb63640 1 Processor -- start 2024-09-11T23:52:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff85cb63640 1 -- start start 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff85cb63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff858102d90 0x7ff858198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff85cb63640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8581036b0 0x7ff8581989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff85cb63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff85810c170 0x7ff85819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff85cb63640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff858111f50 con 0x7ff8581036b0 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff85cb63640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff858111dd0 con 0x7ff858102d90 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff85cb63640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581120d0 con 0x7ff85810c170 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff855d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8581036b0 0x7ff8581989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff855d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8581036b0 0x7ff8581989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53684/0 (socket says 172.21.15.62:53684) 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff855d9b640 1 -- 172.21.15.62:0/2190593494 learned_addr learned my addr 172.21.15.62:0/2190593494 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.063+0000 7ff856d9d640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff85810c170 0x7ff85819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff85659c640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff858102d90 0x7ff858198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff855d9b640 1 -- 172.21.15.62:0/2190593494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff85810c170 msgr2=0x7ff85819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff855d9b640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff85810c170 0x7ff85819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff855d9b640 1 -- 172.21.15.62:0/2190593494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff858102d90 msgr2=0x7ff858198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff855d9b640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff858102d90 0x7ff858198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff855d9b640 1 -- 172.21.15.62:0/2190593494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85819d440 con 0x7ff8581036b0 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff855d9b640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8581036b0 0x7ff8581989d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff84c006010 tx=0x7ff84c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff8477fe640 1 -- 172.21.15.62:0/2190593494 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c03f030 con 0x7ff8581036b0 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff85cb63640 1 -- 172.21.15.62:0/2190593494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85819d6d0 con 0x7ff8581036b0 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff8477fe640 1 -- 172.21.15.62:0/2190593494 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff84c003ce0 con 0x7ff8581036b0 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff8477fe640 1 -- 172.21.15.62:0/2190593494 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c035070 con 0x7ff8581036b0 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.064+0000 7ff85cb63640 1 -- 172.21.15.62:0/2190593494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8581a5010 con 0x7ff8581036b0 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.065+0000 7ff85cb63640 1 -- 172.21.15.62:0/2190593494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff858103170 con 0x7ff8581036b0 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.066+0000 7ff8477fe640 1 -- 172.21.15.62:0/2190593494 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7ff84c042030 con 0x7ff8581036b0 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.067+0000 7ff8477fe640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff8400a2d80 0x7ff8400a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.067+0000 7ff8477fe640 1 -- 172.21.15.62:0/2190593494 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7ff84c0efb40 con 0x7ff8581036b0 2024-09-11T23:52:18.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.067+0000 7ff85659c640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff8400a2d80 0x7ff8400a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:18.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.068+0000 7ff85659c640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff8400a2d80 0x7ff8400a5240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff848005fd0 tx=0x7ff84800b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:18.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.071+0000 7ff8477fe640 1 -- 172.21.15.62:0/2190593494 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff84c0bc090 con 0x7ff8581036b0 2024-09-11T23:52:18.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.246+0000 7ff85cb63640 1 -- 172.21.15.62:0/2190593494 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff858062630 con 0x7ff8400a2d80 2024-09-11T23:52:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.253+0000 7ff8477fe640 1 -- 172.21.15.62:0/2190593494 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258026 (secure 0 0 0) 0x7ff858062630 con 0x7ff8400a2d80 2024-09-11T23:52:18.255 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:18.256 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:52:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 -- 172.21.15.62:0/2190593494 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff8400a2d80 msgr2=0x7ff8400a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff8400a2d80 0x7ff8400a5240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff848005fd0 tx=0x7ff84800b390 comp rx=0 tx=0).stop 2024-09-11T23:52:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 -- 172.21.15.62:0/2190593494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8581036b0 msgr2=0x7ff8581989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8581036b0 0x7ff8581989d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff84c006010 tx=0x7ff84c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:52:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 -- 172.21.15.62:0/2190593494 shutdown_connections 2024-09-11T23:52:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff8400a2d80 0x7ff8400a5240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:18.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff85810c170 0x7ff85819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:18.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8581036b0 0x7ff8581989d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:18.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 --2- 172.21.15.62:0/2190593494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff858102d90 0x7ff858198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:18.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 -- 172.21.15.62:0/2190593494 >> 172.21.15.62:0/2190593494 conn(0x7ff8580fc790 msgr2=0x7ff8580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:18.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 -- 172.21.15.62:0/2190593494 shutdown_connections 2024-09-11T23:52:18.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:18.256+0000 7ff85cb63640 1 -- 172.21.15.62:0/2190593494 wait complete. 2024-09-11T23:52:18.574 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:18 smithi062 ceph-mon[30958]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-09-11T23:52:18.574 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:18 smithi062 ceph-mon[30958]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:18.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:18 smithi120 ceph-mon[35182]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-09-11T23:52:18.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:18 smithi120 ceph-mon[35182]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:18 smithi083 ceph-mon[34805]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-09-11T23:52:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:18 smithi083 ceph-mon[34805]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:18.895 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-11T23:52:17.447618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384862,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1316,"num_read_kb":1084,"num_write":625,"num_write_kb":5294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":960,"ondisk_log_size":960,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348228,"kb_used_data":11412,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593596,"statfs":{"total":1151940427776,"available":1151583842304,"internally_reserved":0,"allocated":11685888,"data_stored":9498354,"data_compressed":245838,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197770,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":417,"num_read_kb":475,"num_write":147,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.978852"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872755+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:52:05.872755+0000","last_peered":"2024-09-11T23:52:05.872755+0000","last_clean":"2024-09-11T23:52:05.872755+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:52:05.872755+0000","last_undegraded":"2024-09-11T23:52:05.872755+0000","last_fullsized":"2024-09-11T23:52:05.872755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158536+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:52:06.158536+0000","last_peered":"2024-09-11T23:52:06.158536+0000","last_clean":"2024-09-11T23:52:06.158536+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:52:06.158536+0000","last_undegraded":"2024-09-11T23:52:06.158536+0000","last_fullsized":"2024-09-11T23:52:06.158536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158307+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:52:06.158307+0000","last_peered":"2024-09-11T23:52:06.158307+0000","last_clean":"2024-09-11T23:52:06.158307+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:52:06.158307+0000","last_undegraded":"2024-09-11T23:52:06.158307+0000","last_fullsized":"2024-09-11T23:52:06.158307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744974+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:52:05.744974+0000","last_peered":"2024-09-11T23:52:05.744974+0000","last_clean":"2024-09-11T23:52:05.744974+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:52:05.744974+0000","last_undegraded":"2024-09-11T23:52:05.744974+0000","last_fullsized":"2024-09-11T23:52:05.744974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516648+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:52:05.516648+0000","last_peered":"2024-09-11T23:52:05.516648+0000","last_clean":"2024-09-11T23:52:05.516648+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:52:05.516648+0000","last_undegraded":"2024-09-11T23:52:05.516648+0000","last_fullsized":"2024-09-11T23:52:05.516648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434378+0000","last_change":"2024-09-11T23:51:14.120226+0000","last_active":"2024-09-11T23:52:05.434378+0000","last_peered":"2024-09-11T23:52:05.434378+0000","last_clean":"2024-09-11T23:52:05.434378+0000","last_became_active":"2024-09-11T23:51:14.119811+0000","last_became_peered":"2024-09-11T23:51:14.119811+0000","last_unstale":"2024-09-11T23:52:05.434378+0000","last_undegraded":"2024-09-11T23:52:05.434378+0000","last_fullsized":"2024-09-11T23:52:05.434378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:58.851476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746317+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:52:05.746317+0000","last_peered":"2024-09-11T23:52:05.746317+0000","last_clean":"2024-09-11T23:52:05.746317+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:52:05.746317+0000","last_undegraded":"2024-09-11T23:52:05.746317+0000","last_fullsized":"2024-09-11T23:52:05.746317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523319+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:52:05.523319+0000","last_peered":"2024-09-11T23:52:05.523319+0000","last_clean":"2024-09-11T23:52:05.523319+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:52:05.523319+0000","last_undegraded":"2024-09-11T23:52:05.523319+0000","last_fullsized":"2024-09-11T23:52:05.523319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694954+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:52:05.694954+0000","last_peered":"2024-09-11T23:52:05.694954+0000","last_clean":"2024-09-11T23:52:05.694954+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:52:05.694954+0000","last_undegraded":"2024-09-11T23:52:05.694954+0000","last_fullsized":"2024-09-11T23:52:05.694954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435402+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:52:05.435402+0000","last_peered":"2024-09-11T23:52:05.435402+0000","last_clean":"2024-09-11T23:52:05.435402+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:52:05.435402+0000","last_undegraded":"2024-09-11T23:52:05.435402+0000","last_fullsized":"2024-09-11T23:52:05.435402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746473+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:52:05.746473+0000","last_peered":"2024-09-11T23:52:05.746473+0000","last_clean":"2024-09-11T23:52:05.746473+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:52:05.746473+0000","last_undegraded":"2024-09-11T23:52:05.746473+0000","last_fullsized":"2024-09-11T23:52:05.746473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744935+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:52:05.744935+0000","last_peered":"2024-09-11T23:52:05.744935+0000","last_clean":"2024-09-11T23:52:05.744935+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:52:05.744935+0000","last_undegraded":"2024-09-11T23:52:05.744935+0000","last_fullsized":"2024-09-11T23:52:05.744935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431713+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:52:05.431713+0000","last_peered":"2024-09-11T23:52:05.431713+0000","last_clean":"2024-09-11T23:52:05.431713+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:52:05.431713+0000","last_undegraded":"2024-09-11T23:52:05.431713+0000","last_fullsized":"2024-09-11T23:52:05.431713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158046+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:52:06.158046+0000","last_peered":"2024-09-11T23:52:06.158046+0000","last_clean":"2024-09-11T23:52:06.158046+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:52:06.158046+0000","last_undegraded":"2024-09-11T23:52:06.158046+0000","last_fullsized":"2024-09-11T23:52:06.158046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.157901+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:52:06.157901+0000","last_peered":"2024-09-11T23:52:06.157901+0000","last_clean":"2024-09-11T23:52:06.157901+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:52:06.157901+0000","last_undegraded":"2024-09-11T23:52:06.157901+0000","last_fullsized":"2024-09-11T23:52:06.157901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694419+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:52:05.694419+0000","last_peered":"2024-09-11T23:52:05.694419+0000","last_clean":"2024-09-11T23:52:05.694419+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:52:05.694419+0000","last_undegraded":"2024-09-11T23:52:05.694419+0000","last_fullsized":"2024-09-11T23:52:05.694419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872787+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:52:05.872787+0000","last_peered":"2024-09-11T23:52:05.872787+0000","last_clean":"2024-09-11T23:52:05.872787+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:52:05.872787+0000","last_undegraded":"2024-09-11T23:52:05.872787+0000","last_fullsized":"2024-09-11T23:52:05.872787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520950+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:52:05.520950+0000","last_peered":"2024-09-11T23:52:05.520950+0000","last_clean":"2024-09-11T23:52:05.520950+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:52:05.520950+0000","last_undegraded":"2024-09-11T23:52:05.520950+0000","last_fullsized":"2024-09-11T23:52:05.520950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694752+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:52:05.694752+0000","last_peered":"2024-09-11T23:52:05.694752+0000","last_clean":"2024-09-11T23:52:05.694752+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:52:05.694752+0000","last_undegraded":"2024-09-11T23:52:05.694752+0000","last_fullsized":"2024-09-11T23:52:05.694752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872460+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:52:05.872460+0000","last_peered":"2024-09-11T23:52:05.872460+0000","last_clean":"2024-09-11T23:52:05.872460+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:52:05.872460+0000","last_undegraded":"2024-09-11T23:52:05.872460+0000","last_fullsized":"2024-09-11T23:52:05.872460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431851+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:52:05.431851+0000","last_peered":"2024-09-11T23:52:05.431851+0000","last_clean":"2024-09-11T23:52:05.431851+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:52:05.431851+0000","last_undegraded":"2024-09-11T23:52:05.431851+0000","last_fullsized":"2024-09-11T23:52:05.431851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694603+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:52:05.694603+0000","last_peered":"2024-09-11T23:52:05.694603+0000","last_clean":"2024-09-11T23:52:05.694603+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:52:05.694603+0000","last_undegraded":"2024-09-11T23:52:05.694603+0000","last_fullsized":"2024-09-11T23:52:05.694603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523509+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:52:05.523509+0000","last_peered":"2024-09-11T23:52:05.523509+0000","last_clean":"2024-09-11T23:52:05.523509+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:52:05.523509+0000","last_undegraded":"2024-09-11T23:52:05.523509+0000","last_fullsized":"2024-09-11T23:52:05.523509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432812+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:52:05.432812+0000","last_peered":"2024-09-11T23:52:05.432812+0000","last_clean":"2024-09-11T23:52:05.432812+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:52:05.432812+0000","last_undegraded":"2024-09-11T23:52:05.432812+0000","last_fullsized":"2024-09-11T23:52:05.432812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525015+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:52:05.525015+0000","last_peered":"2024-09-11T23:52:05.525015+0000","last_clean":"2024-09-11T23:52:05.525015+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:52:05.525015+0000","last_undegraded":"2024-09-11T23:52:05.525015+0000","last_fullsized":"2024-09-11T23:52:05.525015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694943+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:52:05.694943+0000","last_peered":"2024-09-11T23:52:05.694943+0000","last_clean":"2024-09-11T23:52:05.694943+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:52:05.694943+0000","last_undegraded":"2024-09-11T23:52:05.694943+0000","last_fullsized":"2024-09-11T23:52:05.694943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872635+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:52:05.872635+0000","last_peered":"2024-09-11T23:52:05.872635+0000","last_clean":"2024-09-11T23:52:05.872635+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:52:05.872635+0000","last_undegraded":"2024-09-11T23:52:05.872635+0000","last_fullsized":"2024-09-11T23:52:05.872635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431698+0000","last_change":"2024-09-11T23:51:14.410108+0000","last_active":"2024-09-11T23:52:05.431698+0000","last_peered":"2024-09-11T23:52:05.431698+0000","last_clean":"2024-09-11T23:52:05.431698+0000","last_became_active":"2024-09-11T23:51:14.409588+0000","last_became_peered":"2024-09-11T23:51:14.409588+0000","last_unstale":"2024-09-11T23:52:05.431698+0000","last_undegraded":"2024-09-11T23:52:05.431698+0000","last_fullsized":"2024-09-11T23:52:05.431698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:29.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158121+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:52:06.158121+0000","last_peered":"2024-09-11T23:52:06.158121+0000","last_clean":"2024-09-11T23:52:06.158121+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:52:06.158121+0000","last_undegraded":"2024-09-11T23:52:06.158121+0000","last_fullsized":"2024-09-11T23:52:06.158121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694617+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:52:05.694617+0000","last_peered":"2024-09-11T23:52:05.694617+0000","last_clean":"2024-09-11T23:52:05.694617+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:52:05.694617+0000","last_undegraded":"2024-09-11T23:52:05.694617+0000","last_fullsized":"2024-09-11T23:52:05.694617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694660+0000","last_change":"2024-09-11T23:51:14.410872+0000","last_active":"2024-09-11T23:52:05.694660+0000","last_peered":"2024-09-11T23:52:05.694660+0000","last_clean":"2024-09-11T23:52:05.694660+0000","last_became_active":"2024-09-11T23:51:14.410531+0000","last_became_peered":"2024-09-11T23:51:14.410531+0000","last_unstale":"2024-09-11T23:52:05.694660+0000","last_undegraded":"2024-09-11T23:52:05.694660+0000","last_fullsized":"2024-09-11T23:52:05.694660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:10.502914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872542+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:52:05.872542+0000","last_peered":"2024-09-11T23:52:05.872542+0000","last_clean":"2024-09-11T23:52:05.872542+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:52:05.872542+0000","last_undegraded":"2024-09-11T23:52:05.872542+0000","last_fullsized":"2024-09-11T23:52:05.872542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516667+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:52:05.516667+0000","last_peered":"2024-09-11T23:52:05.516667+0000","last_clean":"2024-09-11T23:52:05.516667+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:52:05.516667+0000","last_undegraded":"2024-09-11T23:52:05.516667+0000","last_fullsized":"2024-09-11T23:52:05.516667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:15.025850+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:52:15.025850+0000","last_peered":"2024-09-11T23:52:15.025850+0000","last_clean":"2024-09-11T23:52:15.025850+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:52:15.025850+0000","last_undegraded":"2024-09-11T23:52:15.025850+0000","last_fullsized":"2024-09-11T23:52:15.025850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432914+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:52:05.432914+0000","last_peered":"2024-09-11T23:52:05.432914+0000","last_clean":"2024-09-11T23:52:05.432914+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:52:05.432914+0000","last_undegraded":"2024-09-11T23:52:05.432914+0000","last_fullsized":"2024-09-11T23:52:05.432914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434413+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:52:05.434413+0000","last_peered":"2024-09-11T23:52:05.434413+0000","last_clean":"2024-09-11T23:52:05.434413+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:52:05.434413+0000","last_undegraded":"2024-09-11T23:52:05.434413+0000","last_fullsized":"2024-09-11T23:52:05.434413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431356+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:52:05.431356+0000","last_peered":"2024-09-11T23:52:05.431356+0000","last_clean":"2024-09-11T23:52:05.431356+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:52:05.431356+0000","last_undegraded":"2024-09-11T23:52:05.431356+0000","last_fullsized":"2024-09-11T23:52:05.431356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523343+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:52:05.523343+0000","last_peered":"2024-09-11T23:52:05.523343+0000","last_clean":"2024-09-11T23:52:05.523343+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:52:05.523343+0000","last_undegraded":"2024-09-11T23:52:05.523343+0000","last_fullsized":"2024-09-11T23:52:05.523343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433385+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:52:05.433385+0000","last_peered":"2024-09-11T23:52:05.433385+0000","last_clean":"2024-09-11T23:52:05.433385+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:52:05.433385+0000","last_undegraded":"2024-09-11T23:52:05.433385+0000","last_fullsized":"2024-09-11T23:52:05.433385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520843+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:52:05.520843+0000","last_peered":"2024-09-11T23:52:05.520843+0000","last_clean":"2024-09-11T23:52:05.520843+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:52:05.520843+0000","last_undegraded":"2024-09-11T23:52:05.520843+0000","last_fullsized":"2024-09-11T23:52:05.520843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:15.025211+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:52:15.025211+0000","last_peered":"2024-09-11T23:52:15.025211+0000","last_clean":"2024-09-11T23:52:15.025211+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:52:15.025211+0000","last_undegraded":"2024-09-11T23:52:15.025211+0000","last_fullsized":"2024-09-11T23:52:15.025211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694869+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:52:05.694869+0000","last_peered":"2024-09-11T23:52:05.694869+0000","last_clean":"2024-09-11T23:52:05.694869+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:52:05.694869+0000","last_undegraded":"2024-09-11T23:52:05.694869+0000","last_fullsized":"2024-09-11T23:52:05.694869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746642+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:52:05.746642+0000","last_peered":"2024-09-11T23:52:05.746642+0000","last_clean":"2024-09-11T23:52:05.746642+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:52:05.746642+0000","last_undegraded":"2024-09-11T23:52:05.746642+0000","last_fullsized":"2024-09-11T23:52:05.746642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158655+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:52:06.158655+0000","last_peered":"2024-09-11T23:52:06.158655+0000","last_clean":"2024-09-11T23:52:06.158655+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:52:06.158655+0000","last_undegraded":"2024-09-11T23:52:06.158655+0000","last_fullsized":"2024-09-11T23:52:06.158655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433170+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:52:05.433170+0000","last_peered":"2024-09-11T23:52:05.433170+0000","last_clean":"2024-09-11T23:52:05.433170+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:52:05.433170+0000","last_undegraded":"2024-09-11T23:52:05.433170+0000","last_fullsized":"2024-09-11T23:52:05.433170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694598+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:52:05.694598+0000","last_peered":"2024-09-11T23:52:05.694598+0000","last_clean":"2024-09-11T23:52:05.694598+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:52:05.694598+0000","last_undegraded":"2024-09-11T23:52:05.694598+0000","last_fullsized":"2024-09-11T23:52:05.694598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'37","reported_seq":307,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:16.026810+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:52:16.026810+0000","last_peered":"2024-09-11T23:52:16.026810+0000","last_clean":"2024-09-11T23:52:16.026810+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:52:16.026810+0000","last_undegraded":"2024-09-11T23:52:16.026810+0000","last_fullsized":"2024-09-11T23:52:16.026810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67574,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":189,"num_write":38,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516931+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:52:05.516931+0000","last_peered":"2024-09-11T23:52:05.516931+0000","last_clean":"2024-09-11T23:52:05.516931+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:52:05.516931+0000","last_undegraded":"2024-09-11T23:52:05.516931+0000","last_fullsized":"2024-09-11T23:52:05.516931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694704+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:52:05.694704+0000","last_peered":"2024-09-11T23:52:05.694704+0000","last_clean":"2024-09-11T23:52:05.694704+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:52:05.694704+0000","last_undegraded":"2024-09-11T23:52:05.694704+0000","last_fullsized":"2024-09-11T23:52:05.694704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431407+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:52:05.431407+0000","last_peered":"2024-09-11T23:52:05.431407+0000","last_clean":"2024-09-11T23:52:05.431407+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:52:05.431407+0000","last_undegraded":"2024-09-11T23:52:05.431407+0000","last_fullsized":"2024-09-11T23:52:05.431407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433521+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:52:05.433521+0000","last_peered":"2024-09-11T23:52:05.433521+0000","last_clean":"2024-09-11T23:52:05.433521+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:52:05.433521+0000","last_undegraded":"2024-09-11T23:52:05.433521+0000","last_fullsized":"2024-09-11T23:52:05.433521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746620+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:52:05.746620+0000","last_peered":"2024-09-11T23:52:05.746620+0000","last_clean":"2024-09-11T23:52:05.746620+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:52:05.746620+0000","last_undegraded":"2024-09-11T23:52:05.746620+0000","last_fullsized":"2024-09-11T23:52:05.746620+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":153,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:15.040273+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:52:15.040273+0000","last_peered":"2024-09-11T23:52:15.040273+0000","last_clean":"2024-09-11T23:52:15.040273+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:52:15.040273+0000","last_undegraded":"2024-09-11T23:52:15.040273+0000","last_fullsized":"2024-09-11T23:52:15.040273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746281+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:52:05.746281+0000","last_peered":"2024-09-11T23:52:05.746281+0000","last_clean":"2024-09-11T23:52:05.746281+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:52:05.746281+0000","last_undegraded":"2024-09-11T23:52:05.746281+0000","last_fullsized":"2024-09-11T23:52:05.746281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516591+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:52:05.516591+0000","last_peered":"2024-09-11T23:52:05.516591+0000","last_clean":"2024-09-11T23:52:05.516591+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:52:05.516591+0000","last_undegraded":"2024-09-11T23:52:05.516591+0000","last_fullsized":"2024-09-11T23:52:05.516591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432801+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:52:05.432801+0000","last_peered":"2024-09-11T23:52:05.432801+0000","last_clean":"2024-09-11T23:52:05.432801+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:52:05.432801+0000","last_undegraded":"2024-09-11T23:52:05.432801+0000","last_fullsized":"2024-09-11T23:52:05.432801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516946+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:52:05.516946+0000","last_peered":"2024-09-11T23:52:05.516946+0000","last_clean":"2024-09-11T23:52:05.516946+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:52:05.516946+0000","last_undegraded":"2024-09-11T23:52:05.516946+0000","last_fullsized":"2024-09-11T23:52:05.516946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872534+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:52:05.872534+0000","last_peered":"2024-09-11T23:52:05.872534+0000","last_clean":"2024-09-11T23:52:05.872534+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:52:05.872534+0000","last_undegraded":"2024-09-11T23:52:05.872534+0000","last_fullsized":"2024-09-11T23:52:05.872534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431687+0000","last_change":"2024-09-11T23:51:14.413526+0000","last_active":"2024-09-11T23:52:05.431687+0000","last_peered":"2024-09-11T23:52:05.431687+0000","last_clean":"2024-09-11T23:52:05.431687+0000","last_became_active":"2024-09-11T23:51:14.411027+0000","last_became_peered":"2024-09-11T23:51:14.411027+0000","last_unstale":"2024-09-11T23:52:05.431687+0000","last_undegraded":"2024-09-11T23:52:05.431687+0000","last_fullsized":"2024-09-11T23:52:05.431687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:55.020567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746675+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:52:05.746675+0000","last_peered":"2024-09-11T23:52:05.746675+0000","last_clean":"2024-09-11T23:52:05.746675+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:52:05.746675+0000","last_undegraded":"2024-09-11T23:52:05.746675+0000","last_fullsized":"2024-09-11T23:52:05.746675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'65","reported_seq":120,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:15.104271+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:52:15.104271+0000","last_peered":"2024-09-11T23:52:15.104271+0000","last_clean":"2024-09-11T23:52:15.104271+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:52:15.104271+0000","last_undegraded":"2024-09-11T23:52:15.104271+0000","last_fullsized":"2024-09-11T23:52:15.104271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746458+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:52:05.746458+0000","last_peered":"2024-09-11T23:52:05.746458+0000","last_clean":"2024-09-11T23:52:05.746458+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:52:05.746458+0000","last_undegraded":"2024-09-11T23:52:05.746458+0000","last_fullsized":"2024-09-11T23:52:05.746458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694804+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:52:05.694804+0000","last_peered":"2024-09-11T23:52:05.694804+0000","last_clean":"2024-09-11T23:52:05.694804+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:52:05.694804+0000","last_undegraded":"2024-09-11T23:52:05.694804+0000","last_fullsized":"2024-09-11T23:52:05.694804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746570+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:52:05.746570+0000","last_peered":"2024-09-11T23:52:05.746570+0000","last_clean":"2024-09-11T23:52:05.746570+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:52:05.746570+0000","last_undegraded":"2024-09-11T23:52:05.746570+0000","last_fullsized":"2024-09-11T23:52:05.746570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"84'148","reported_seq":394,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:07.529793+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:52:07.529793+0000","last_peered":"2024-09-11T23:52:07.529793+0000","last_clean":"2024-09-11T23:52:07.529793+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:52:07.529793+0000","last_undegraded":"2024-09-11T23:52:07.529793+0000","last_fullsized":"2024-09-11T23:52:07.529793+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744951+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:52:05.744951+0000","last_peered":"2024-09-11T23:52:05.744951+0000","last_clean":"2024-09-11T23:52:05.744951+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:52:05.744951+0000","last_undegraded":"2024-09-11T23:52:05.744951+0000","last_fullsized":"2024-09-11T23:52:05.744951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523757+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:52:05.523757+0000","last_peered":"2024-09-11T23:52:05.523757+0000","last_clean":"2024-09-11T23:52:05.523757+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:52:05.523757+0000","last_undegraded":"2024-09-11T23:52:05.523757+0000","last_fullsized":"2024-09-11T23:52:05.523757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872415+0000","last_change":"2024-09-11T23:51:14.121480+0000","last_active":"2024-09-11T23:52:05.872415+0000","last_peered":"2024-09-11T23:52:05.872415+0000","last_clean":"2024-09-11T23:52:05.872415+0000","last_became_active":"2024-09-11T23:51:14.118435+0000","last_became_peered":"2024-09-11T23:51:14.118435+0000","last_unstale":"2024-09-11T23:52:05.872415+0000","last_undegraded":"2024-09-11T23:52:05.872415+0000","last_fullsized":"2024-09-11T23:52:05.872415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431409+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:52:05.431409+0000","last_peered":"2024-09-11T23:52:05.431409+0000","last_clean":"2024-09-11T23:52:05.431409+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:52:05.431409+0000","last_undegraded":"2024-09-11T23:52:05.431409+0000","last_fullsized":"2024-09-11T23:52:05.431409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":147,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:16.041284+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:52:16.041284+0000","last_peered":"2024-09-11T23:52:16.041284+0000","last_clean":"2024-09-11T23:52:16.041284+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:52:16.041284+0000","last_undegraded":"2024-09-11T23:52:16.041284+0000","last_fullsized":"2024-09-11T23:52:16.041284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432911+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:52:05.432911+0000","last_peered":"2024-09-11T23:52:05.432911+0000","last_clean":"2024-09-11T23:52:05.432911+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:52:05.432911+0000","last_undegraded":"2024-09-11T23:52:05.432911+0000","last_fullsized":"2024-09-11T23:52:05.432911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746571+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:52:05.746571+0000","last_peered":"2024-09-11T23:52:05.746571+0000","last_clean":"2024-09-11T23:52:05.746571+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:52:05.746571+0000","last_undegraded":"2024-09-11T23:52:05.746571+0000","last_fullsized":"2024-09-11T23:52:05.746571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435062+0000","last_change":"2024-09-11T23:51:14.409908+0000","last_active":"2024-09-11T23:52:05.435062+0000","last_peered":"2024-09-11T23:52:05.435062+0000","last_clean":"2024-09-11T23:52:05.435062+0000","last_became_active":"2024-09-11T23:51:14.409432+0000","last_became_peered":"2024-09-11T23:51:14.409432+0000","last_unstale":"2024-09-11T23:52:05.435062+0000","last_undegraded":"2024-09-11T23:52:05.435062+0000","last_fullsized":"2024-09-11T23:52:05.435062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:06.371306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.521215+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:52:05.521215+0000","last_peered":"2024-09-11T23:52:05.521215+0000","last_clean":"2024-09-11T23:52:05.521215+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:52:05.521215+0000","last_undegraded":"2024-09-11T23:52:05.521215+0000","last_fullsized":"2024-09-11T23:52:05.521215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435167+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:52:05.435167+0000","last_peered":"2024-09-11T23:52:05.435167+0000","last_clean":"2024-09-11T23:52:05.435167+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:52:05.435167+0000","last_undegraded":"2024-09-11T23:52:05.435167+0000","last_fullsized":"2024-09-11T23:52:05.435167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.517017+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:52:05.517017+0000","last_peered":"2024-09-11T23:52:05.517017+0000","last_clean":"2024-09-11T23:52:05.517017+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:52:05.517017+0000","last_undegraded":"2024-09-11T23:52:05.517017+0000","last_fullsized":"2024-09-11T23:52:05.517017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431819+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:52:05.431819+0000","last_peered":"2024-09-11T23:52:05.431819+0000","last_clean":"2024-09-11T23:52:05.431819+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:52:05.431819+0000","last_undegraded":"2024-09-11T23:52:05.431819+0000","last_fullsized":"2024-09-11T23:52:05.431819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'101","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:15.461394+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:52:15.461394+0000","last_peered":"2024-09-11T23:52:15.461394+0000","last_clean":"2024-09-11T23:52:15.461394+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:52:15.461394+0000","last_undegraded":"2024-09-11T23:52:15.461394+0000","last_fullsized":"2024-09-11T23:52:15.461394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525022+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:52:05.525022+0000","last_peered":"2024-09-11T23:52:05.525022+0000","last_clean":"2024-09-11T23:52:05.525022+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:52:05.525022+0000","last_undegraded":"2024-09-11T23:52:05.525022+0000","last_fullsized":"2024-09-11T23:52:05.525022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.745158+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:52:05.745158+0000","last_peered":"2024-09-11T23:52:05.745158+0000","last_clean":"2024-09-11T23:52:05.745158+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:52:05.745158+0000","last_undegraded":"2024-09-11T23:52:05.745158+0000","last_fullsized":"2024-09-11T23:52:05.745158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158521+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:52:06.158521+0000","last_peered":"2024-09-11T23:52:06.158521+0000","last_clean":"2024-09-11T23:52:06.158521+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:52:06.158521+0000","last_undegraded":"2024-09-11T23:52:06.158521+0000","last_fullsized":"2024-09-11T23:52:06.158521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694550+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:52:05.694550+0000","last_peered":"2024-09-11T23:52:05.694550+0000","last_clean":"2024-09-11T23:52:05.694550+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:52:05.694550+0000","last_undegraded":"2024-09-11T23:52:05.694550+0000","last_fullsized":"2024-09-11T23:52:05.694550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516588+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:52:05.516588+0000","last_peered":"2024-09-11T23:52:05.516588+0000","last_clean":"2024-09-11T23:52:05.516588+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:52:05.516588+0000","last_undegraded":"2024-09-11T23:52:05.516588+0000","last_fullsized":"2024-09-11T23:52:05.516588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523605+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:52:05.523605+0000","last_peered":"2024-09-11T23:52:05.523605+0000","last_clean":"2024-09-11T23:52:05.523605+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:52:05.523605+0000","last_undegraded":"2024-09-11T23:52:05.523605+0000","last_fullsized":"2024-09-11T23:52:05.523605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872645+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:52:05.872645+0000","last_peered":"2024-09-11T23:52:05.872645+0000","last_clean":"2024-09-11T23:52:05.872645+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:52:05.872645+0000","last_undegraded":"2024-09-11T23:52:05.872645+0000","last_fullsized":"2024-09-11T23:52:05.872645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872496+0000","last_change":"2024-09-11T23:51:14.123663+0000","last_active":"2024-09-11T23:52:05.872496+0000","last_peered":"2024-09-11T23:52:05.872496+0000","last_clean":"2024-09-11T23:52:05.872496+0000","last_became_active":"2024-09-11T23:51:14.118315+0000","last_became_peered":"2024-09-11T23:51:14.118315+0000","last_unstale":"2024-09-11T23:52:05.872496+0000","last_undegraded":"2024-09-11T23:52:05.872496+0000","last_fullsized":"2024-09-11T23:52:05.872496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:10.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.524009+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:52:05.524009+0000","last_peered":"2024-09-11T23:52:05.524009+0000","last_clean":"2024-09-11T23:52:05.524009+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:52:05.524009+0000","last_undegraded":"2024-09-11T23:52:05.524009+0000","last_fullsized":"2024-09-11T23:52:05.524009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516698+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:52:05.516698+0000","last_peered":"2024-09-11T23:52:05.516698+0000","last_clean":"2024-09-11T23:52:05.516698+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:52:05.516698+0000","last_undegraded":"2024-09-11T23:52:05.516698+0000","last_fullsized":"2024-09-11T23:52:05.516698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746447+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:52:05.746447+0000","last_peered":"2024-09-11T23:52:05.746447+0000","last_clean":"2024-09-11T23:52:05.746447+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:52:05.746447+0000","last_undegraded":"2024-09-11T23:52:05.746447+0000","last_fullsized":"2024-09-11T23:52:05.746447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433119+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:52:05.433119+0000","last_peered":"2024-09-11T23:52:05.433119+0000","last_clean":"2024-09-11T23:52:05.433119+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:52:05.433119+0000","last_undegraded":"2024-09-11T23:52:05.433119+0000","last_fullsized":"2024-09-11T23:52:05.433119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516897+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:52:05.516897+0000","last_peered":"2024-09-11T23:52:05.516897+0000","last_clean":"2024-09-11T23:52:05.516897+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:52:05.516897+0000","last_undegraded":"2024-09-11T23:52:05.516897+0000","last_fullsized":"2024-09-11T23:52:05.516897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694561+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:52:05.694561+0000","last_peered":"2024-09-11T23:52:05.694561+0000","last_clean":"2024-09-11T23:52:05.694561+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:52:05.694561+0000","last_undegraded":"2024-09-11T23:52:05.694561+0000","last_fullsized":"2024-09-11T23:52:05.694561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516945+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:52:05.516945+0000","last_peered":"2024-09-11T23:52:05.516945+0000","last_clean":"2024-09-11T23:52:05.516945+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:52:05.516945+0000","last_undegraded":"2024-09-11T23:52:05.516945+0000","last_fullsized":"2024-09-11T23:52:05.516945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158665+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:52:06.158665+0000","last_peered":"2024-09-11T23:52:06.158665+0000","last_clean":"2024-09-11T23:52:06.158665+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:52:06.158665+0000","last_undegraded":"2024-09-11T23:52:06.158665+0000","last_fullsized":"2024-09-11T23:52:06.158665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525482+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:52:05.525482+0000","last_peered":"2024-09-11T23:52:05.525482+0000","last_clean":"2024-09-11T23:52:05.525482+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:52:05.525482+0000","last_undegraded":"2024-09-11T23:52:05.525482+0000","last_fullsized":"2024-09-11T23:52:05.525482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525206+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:52:05.525206+0000","last_peered":"2024-09-11T23:52:05.525206+0000","last_clean":"2024-09-11T23:52:05.525206+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:52:05.525206+0000","last_undegraded":"2024-09-11T23:52:05.525206+0000","last_fullsized":"2024-09-11T23:52:05.525206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:11.458220+0000","last_change":"2024-09-11T23:51:14.125596+0000","last_active":"2024-09-11T23:52:11.458220+0000","last_peered":"2024-09-11T23:52:11.458220+0000","last_clean":"2024-09-11T23:52:11.458220+0000","last_became_active":"2024-09-11T23:51:14.119682+0000","last_became_peered":"2024-09-11T23:51:14.119682+0000","last_unstale":"2024-09-11T23:52:11.458220+0000","last_undegraded":"2024-09-11T23:52:11.458220+0000","last_fullsized":"2024-09-11T23:52:11.458220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:06.062602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":794260,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1079,"num_read_kb":754,"num_write":422,"num_write_kb":2354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2382780,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":909312,"data_stored":445896,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1114112,"data_stored":1434418,"data_compressed":53957,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939697,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":839348,"data_compressed":22189,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":838686,"data_compressed":22189,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1039219,"data_compressed":38755,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445521,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1032192,"data_stored":1365900,"data_compressed":53957,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":447193,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":999424,"data_stored":1167280,"data_compressed":37391,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":514631,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":513470,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":446792,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330101,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":398619,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:52:18.899 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:52:18.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:52:19.154 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:19.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.611+0000 7f74abc2f640 1 -- 172.21.15.62:0/1679639531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a4112b90 msgr2=0x7f74a4114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:19.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.611+0000 7f74abc2f640 1 --2- 172.21.15.62:0/1679639531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a4112b90 0x7f74a4114f80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f749c009ca0 tx=0x7f749c02a950 comp rx=0 tx=0).stop 2024-09-11T23:52:19.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.611+0000 7f74abc2f640 1 -- 172.21.15.62:0/1679639531 shutdown_connections 2024-09-11T23:52:19.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.611+0000 7f74abc2f640 1 --2- 172.21.15.62:0/1679639531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a4112b90 0x7f74a4114f80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:19.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.611+0000 7f74abc2f640 1 --2- 172.21.15.62:0/1679639531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74a4074eb0 0x7f74a4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:19.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.611+0000 7f74abc2f640 1 --2- 172.21.15.62:0/1679639531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a4074590 0x7f74a4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:19.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.611+0000 7f74abc2f640 1 -- 172.21.15.62:0/1679639531 >> 172.21.15.62:0/1679639531 conn(0x7f74a40fd8c0 msgr2=0x7f74a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:19.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.612+0000 7f74abc2f640 1 -- 172.21.15.62:0/1679639531 shutdown_connections 2024-09-11T23:52:19.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.612+0000 7f74abc2f640 1 -- 172.21.15.62:0/1679639531 wait complete. 2024-09-11T23:52:19.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.612+0000 7f74abc2f640 1 Processor -- start 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.612+0000 7f74abc2f640 1 -- start start 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.612+0000 7f74abc2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a4074590 0x7f74a41a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.612+0000 7f74abc2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74a4074eb0 0x7f74a41a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.612+0000 7f74abc2f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a4112b90 0x7f74a419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.612+0000 7f74abc2f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74a4118550 con 0x7f74a4112b90 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.612+0000 7f74abc2f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74a41183d0 con 0x7f74a4074590 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.612+0000 7f74abc2f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f74a41186d0 con 0x7f74a4074eb0 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74aa1cc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a4112b90 0x7f74a419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74aa1cc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a4112b90 0x7f74a419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53706/0 (socket says 172.21.15.62:53706) 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74aa1cc640 1 -- 172.21.15.62:0/937841068 learned_addr learned my addr 172.21.15.62:0/937841068 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74a99cb640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a4074590 0x7f74a41a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:19.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74a91ca640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74a4074eb0 0x7f74a41a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74aa1cc640 1 -- 172.21.15.62:0/937841068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74a4074eb0 msgr2=0x7f74a41a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74aa1cc640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74a4074eb0 0x7f74a41a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74aa1cc640 1 -- 172.21.15.62:0/937841068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a4074590 msgr2=0x7f74a41a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74aa1cc640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a4074590 0x7f74a41a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74aa1cc640 1 -- 172.21.15.62:0/937841068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a419efa0 con 0x7f74a4112b90 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74aa1cc640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a4112b90 0x7f74a419e870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f749c004af0 tx=0x7f749c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f749affd640 1 -- 172.21.15.62:0/937841068 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c034040 con 0x7f74a4112b90 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f749affd640 1 -- 172.21.15.62:0/937841068 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f749c030920 con 0x7f74a4112b90 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74abc2f640 1 -- 172.21.15.62:0/937841068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a419f230 con 0x7f74a4112b90 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f749affd640 1 -- 172.21.15.62:0/937841068 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c004780 con 0x7f74a4112b90 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.613+0000 7f74abc2f640 1 -- 172.21.15.62:0/937841068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a41ab580 con 0x7f74a4112b90 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.614+0000 7f74abc2f640 1 -- 172.21.15.62:0/937841068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a4074970 con 0x7f74a4112b90 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.616+0000 7f749affd640 1 -- 172.21.15.62:0/937841068 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f749c037020 con 0x7f74a4112b90 2024-09-11T23:52:19.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.616+0000 7f749affd640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74900a2e00 0x7f74900a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:19.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.616+0000 7f74a99cb640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74900a2e00 0x7f74900a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:19.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.616+0000 7f749affd640 1 -- 172.21.15.62:0/937841068 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f749c0f02d0 con 0x7f74a4112b90 2024-09-11T23:52:19.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.617+0000 7f74a99cb640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74900a2e00 0x7f74900a52c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f74a419f9b0 tx=0x7f74a0005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:19.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.619+0000 7f749affd640 1 -- 172.21.15.62:0/937841068 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f749c0bcec0 con 0x7f74a4112b90 2024-09-11T23:52:19.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.796+0000 7f74abc2f640 1 -- 172.21.15.62:0/937841068 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74a40ff7d0 con 0x7f74900a2e00 2024-09-11T23:52:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.802+0000 7f749affd640 1 -- 172.21.15.62:0/937841068 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258027 (secure 0 0 0) 0x7f74a40ff7d0 con 0x7f74900a2e00 2024-09-11T23:52:19.804 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:19.805 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-09-11T23:52:19.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 -- 172.21.15.62:0/937841068 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74900a2e00 msgr2=0x7f74900a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:19.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74900a2e00 0x7f74900a52c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f74a419f9b0 tx=0x7f74a0005f90 comp rx=0 tx=0).stop 2024-09-11T23:52:19.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 -- 172.21.15.62:0/937841068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a4112b90 msgr2=0x7f74a419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a4112b90 0x7f74a419e870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f749c004af0 tx=0x7f749c02f5b0 comp rx=0 tx=0).stop 2024-09-11T23:52:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 -- 172.21.15.62:0/937841068 shutdown_connections 2024-09-11T23:52:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74900a2e00 0x7f74900a52c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a4112b90 0x7f74a419e870 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74a4074eb0 0x7f74a41a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 --2- 172.21.15.62:0/937841068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a4074590 0x7f74a41a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 -- 172.21.15.62:0/937841068 >> 172.21.15.62:0/937841068 conn(0x7f74a40fd8c0 msgr2=0x7f74a40ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 -- 172.21.15.62:0/937841068 shutdown_connections 2024-09-11T23:52:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:19.806+0000 7f74abc2f640 1 -- 172.21.15.62:0/937841068 wait complete. 2024-09-11T23:52:20.517 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-11T23:52:19.448187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384862,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1316,"num_read_kb":1084,"num_write":625,"num_write_kb":5294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":960,"ondisk_log_size":960,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348228,"kb_used_data":11412,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593596,"statfs":{"total":1151940427776,"available":1151583842304,"internally_reserved":0,"allocated":11685888,"data_stored":9498354,"data_compressed":245838,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197770,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":417,"num_read_kb":475,"num_write":147,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002252"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872755+0000","last_change":"2024-09-11T23:50:10.847826+0000","last_active":"2024-09-11T23:52:05.872755+0000","last_peered":"2024-09-11T23:52:05.872755+0000","last_clean":"2024-09-11T23:52:05.872755+0000","last_became_active":"2024-09-11T23:50:10.847499+0000","last_became_peered":"2024-09-11T23:50:10.847499+0000","last_unstale":"2024-09-11T23:52:05.872755+0000","last_undegraded":"2024-09-11T23:52:05.872755+0000","last_fullsized":"2024-09-11T23:52:05.872755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:10.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158536+0000","last_change":"2024-09-11T23:50:10.842891+0000","last_active":"2024-09-11T23:52:06.158536+0000","last_peered":"2024-09-11T23:52:06.158536+0000","last_clean":"2024-09-11T23:52:06.158536+0000","last_became_active":"2024-09-11T23:50:10.841028+0000","last_became_peered":"2024-09-11T23:50:10.841028+0000","last_unstale":"2024-09-11T23:52:06.158536+0000","last_undegraded":"2024-09-11T23:52:06.158536+0000","last_fullsized":"2024-09-11T23:52:06.158536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:06.319832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158307+0000","last_change":"2024-09-11T23:50:10.847799+0000","last_active":"2024-09-11T23:52:06.158307+0000","last_peered":"2024-09-11T23:52:06.158307+0000","last_clean":"2024-09-11T23:52:06.158307+0000","last_became_active":"2024-09-11T23:50:10.847506+0000","last_became_peered":"2024-09-11T23:50:10.847506+0000","last_unstale":"2024-09-11T23:52:06.158307+0000","last_undegraded":"2024-09-11T23:52:06.158307+0000","last_fullsized":"2024-09-11T23:52:06.158307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:33.133758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744974+0000","last_change":"2024-09-11T23:50:10.851988+0000","last_active":"2024-09-11T23:52:05.744974+0000","last_peered":"2024-09-11T23:52:05.744974+0000","last_clean":"2024-09-11T23:52:05.744974+0000","last_became_active":"2024-09-11T23:50:10.851536+0000","last_became_peered":"2024-09-11T23:50:10.851536+0000","last_unstale":"2024-09-11T23:52:05.744974+0000","last_undegraded":"2024-09-11T23:52:05.744974+0000","last_fullsized":"2024-09-11T23:52:05.744974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:08.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516648+0000","last_change":"2024-09-11T23:50:10.846513+0000","last_active":"2024-09-11T23:52:05.516648+0000","last_peered":"2024-09-11T23:52:05.516648+0000","last_clean":"2024-09-11T23:52:05.516648+0000","last_became_active":"2024-09-11T23:50:10.846221+0000","last_became_peered":"2024-09-11T23:50:10.846221+0000","last_unstale":"2024-09-11T23:52:05.516648+0000","last_undegraded":"2024-09-11T23:52:05.516648+0000","last_fullsized":"2024-09-11T23:52:05.516648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.117347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434378+0000","last_change":"2024-09-11T23:51:14.120226+0000","last_active":"2024-09-11T23:52:05.434378+0000","last_peered":"2024-09-11T23:52:05.434378+0000","last_clean":"2024-09-11T23:52:05.434378+0000","last_became_active":"2024-09-11T23:51:14.119811+0000","last_became_peered":"2024-09-11T23:51:14.119811+0000","last_unstale":"2024-09-11T23:52:05.434378+0000","last_undegraded":"2024-09-11T23:52:05.434378+0000","last_fullsized":"2024-09-11T23:52:05.434378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:58.851476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746317+0000","last_change":"2024-09-11T23:50:10.846980+0000","last_active":"2024-09-11T23:52:05.746317+0000","last_peered":"2024-09-11T23:52:05.746317+0000","last_clean":"2024-09-11T23:52:05.746317+0000","last_became_active":"2024-09-11T23:50:10.846587+0000","last_became_peered":"2024-09-11T23:50:10.846587+0000","last_unstale":"2024-09-11T23:52:05.746317+0000","last_undegraded":"2024-09-11T23:52:05.746317+0000","last_fullsized":"2024-09-11T23:52:05.746317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:29.617254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523319+0000","last_change":"2024-09-11T23:50:10.831098+0000","last_active":"2024-09-11T23:52:05.523319+0000","last_peered":"2024-09-11T23:52:05.523319+0000","last_clean":"2024-09-11T23:52:05.523319+0000","last_became_active":"2024-09-11T23:50:10.830716+0000","last_became_peered":"2024-09-11T23:50:10.830716+0000","last_unstale":"2024-09-11T23:52:05.523319+0000","last_undegraded":"2024-09-11T23:52:05.523319+0000","last_fullsized":"2024-09-11T23:52:05.523319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:26.097209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694954+0000","last_change":"2024-09-11T23:50:10.846568+0000","last_active":"2024-09-11T23:52:05.694954+0000","last_peered":"2024-09-11T23:52:05.694954+0000","last_clean":"2024-09-11T23:52:05.694954+0000","last_became_active":"2024-09-11T23:50:10.846186+0000","last_became_peered":"2024-09-11T23:50:10.846186+0000","last_unstale":"2024-09-11T23:52:05.694954+0000","last_undegraded":"2024-09-11T23:52:05.694954+0000","last_fullsized":"2024-09-11T23:52:05.694954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:09:02.706670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435402+0000","last_change":"2024-09-11T23:50:10.843610+0000","last_active":"2024-09-11T23:52:05.435402+0000","last_peered":"2024-09-11T23:52:05.435402+0000","last_clean":"2024-09-11T23:52:05.435402+0000","last_became_active":"2024-09-11T23:50:10.843311+0000","last_became_peered":"2024-09-11T23:50:10.843311+0000","last_unstale":"2024-09-11T23:52:05.435402+0000","last_undegraded":"2024-09-11T23:52:05.435402+0000","last_fullsized":"2024-09-11T23:52:05.435402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:17.994095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746473+0000","last_change":"2024-09-11T23:50:10.847204+0000","last_active":"2024-09-11T23:52:05.746473+0000","last_peered":"2024-09-11T23:52:05.746473+0000","last_clean":"2024-09-11T23:52:05.746473+0000","last_became_active":"2024-09-11T23:50:10.846836+0000","last_became_peered":"2024-09-11T23:50:10.846836+0000","last_unstale":"2024-09-11T23:52:05.746473+0000","last_undegraded":"2024-09-11T23:52:05.746473+0000","last_fullsized":"2024-09-11T23:52:05.746473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:57:55.094922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744935+0000","last_change":"2024-09-11T23:50:10.849375+0000","last_active":"2024-09-11T23:52:05.744935+0000","last_peered":"2024-09-11T23:52:05.744935+0000","last_clean":"2024-09-11T23:52:05.744935+0000","last_became_active":"2024-09-11T23:50:10.848671+0000","last_became_peered":"2024-09-11T23:50:10.848671+0000","last_unstale":"2024-09-11T23:52:05.744935+0000","last_undegraded":"2024-09-11T23:52:05.744935+0000","last_fullsized":"2024-09-11T23:52:05.744935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:57.241873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431713+0000","last_change":"2024-09-11T23:50:10.848424+0000","last_active":"2024-09-11T23:52:05.431713+0000","last_peered":"2024-09-11T23:52:05.431713+0000","last_clean":"2024-09-11T23:52:05.431713+0000","last_became_active":"2024-09-11T23:50:10.847886+0000","last_became_peered":"2024-09-11T23:50:10.847886+0000","last_unstale":"2024-09-11T23:52:05.431713+0000","last_undegraded":"2024-09-11T23:52:05.431713+0000","last_fullsized":"2024-09-11T23:52:05.431713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:22.087207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158046+0000","last_change":"2024-09-11T23:50:10.842253+0000","last_active":"2024-09-11T23:52:06.158046+0000","last_peered":"2024-09-11T23:52:06.158046+0000","last_clean":"2024-09-11T23:52:06.158046+0000","last_became_active":"2024-09-11T23:50:10.841659+0000","last_became_peered":"2024-09-11T23:50:10.841659+0000","last_unstale":"2024-09-11T23:52:06.158046+0000","last_undegraded":"2024-09-11T23:52:06.158046+0000","last_fullsized":"2024-09-11T23:52:06.158046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:54.727461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.157901+0000","last_change":"2024-09-11T23:50:10.849615+0000","last_active":"2024-09-11T23:52:06.157901+0000","last_peered":"2024-09-11T23:52:06.157901+0000","last_clean":"2024-09-11T23:52:06.157901+0000","last_became_active":"2024-09-11T23:50:10.849215+0000","last_became_peered":"2024-09-11T23:50:10.849215+0000","last_unstale":"2024-09-11T23:52:06.157901+0000","last_undegraded":"2024-09-11T23:52:06.157901+0000","last_fullsized":"2024-09-11T23:52:06.157901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:48.125047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694419+0000","last_change":"2024-09-11T23:50:10.843063+0000","last_active":"2024-09-11T23:52:05.694419+0000","last_peered":"2024-09-11T23:52:05.694419+0000","last_clean":"2024-09-11T23:52:05.694419+0000","last_became_active":"2024-09-11T23:50:10.842716+0000","last_became_peered":"2024-09-11T23:50:10.842716+0000","last_unstale":"2024-09-11T23:52:05.694419+0000","last_undegraded":"2024-09-11T23:52:05.694419+0000","last_fullsized":"2024-09-11T23:52:05.694419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:06.515503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872787+0000","last_change":"2024-09-11T23:50:10.848593+0000","last_active":"2024-09-11T23:52:05.872787+0000","last_peered":"2024-09-11T23:52:05.872787+0000","last_clean":"2024-09-11T23:52:05.872787+0000","last_became_active":"2024-09-11T23:50:10.848269+0000","last_became_peered":"2024-09-11T23:50:10.848269+0000","last_unstale":"2024-09-11T23:52:05.872787+0000","last_undegraded":"2024-09-11T23:52:05.872787+0000","last_fullsized":"2024-09-11T23:52:05.872787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:27.254003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520950+0000","last_change":"2024-09-11T23:50:10.840901+0000","last_active":"2024-09-11T23:52:05.520950+0000","last_peered":"2024-09-11T23:52:05.520950+0000","last_clean":"2024-09-11T23:52:05.520950+0000","last_became_active":"2024-09-11T23:50:10.840615+0000","last_became_peered":"2024-09-11T23:50:10.840615+0000","last_unstale":"2024-09-11T23:52:05.520950+0000","last_undegraded":"2024-09-11T23:52:05.520950+0000","last_fullsized":"2024-09-11T23:52:05.520950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:14.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694752+0000","last_change":"2024-09-11T23:50:10.842498+0000","last_active":"2024-09-11T23:52:05.694752+0000","last_peered":"2024-09-11T23:52:05.694752+0000","last_clean":"2024-09-11T23:52:05.694752+0000","last_became_active":"2024-09-11T23:50:10.841093+0000","last_became_peered":"2024-09-11T23:50:10.841093+0000","last_unstale":"2024-09-11T23:52:05.694752+0000","last_undegraded":"2024-09-11T23:52:05.694752+0000","last_fullsized":"2024-09-11T23:52:05.694752+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:50.263240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872460+0000","last_change":"2024-09-11T23:50:10.843593+0000","last_active":"2024-09-11T23:52:05.872460+0000","last_peered":"2024-09-11T23:52:05.872460+0000","last_clean":"2024-09-11T23:52:05.872460+0000","last_became_active":"2024-09-11T23:50:10.843271+0000","last_became_peered":"2024-09-11T23:50:10.843271+0000","last_unstale":"2024-09-11T23:52:05.872460+0000","last_undegraded":"2024-09-11T23:52:05.872460+0000","last_fullsized":"2024-09-11T23:52:05.872460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:14:05.633030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431851+0000","last_change":"2024-09-11T23:50:10.850259+0000","last_active":"2024-09-11T23:52:05.431851+0000","last_peered":"2024-09-11T23:52:05.431851+0000","last_clean":"2024-09-11T23:52:05.431851+0000","last_became_active":"2024-09-11T23:50:10.849971+0000","last_became_peered":"2024-09-11T23:50:10.849971+0000","last_unstale":"2024-09-11T23:52:05.431851+0000","last_undegraded":"2024-09-11T23:52:05.431851+0000","last_fullsized":"2024-09-11T23:52:05.431851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:03.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694603+0000","last_change":"2024-09-11T23:50:10.842281+0000","last_active":"2024-09-11T23:52:05.694603+0000","last_peered":"2024-09-11T23:52:05.694603+0000","last_clean":"2024-09-11T23:52:05.694603+0000","last_became_active":"2024-09-11T23:50:10.841936+0000","last_became_peered":"2024-09-11T23:50:10.841936+0000","last_unstale":"2024-09-11T23:52:05.694603+0000","last_undegraded":"2024-09-11T23:52:05.694603+0000","last_fullsized":"2024-09-11T23:52:05.694603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:51.970495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523509+0000","last_change":"2024-09-11T23:50:10.842062+0000","last_active":"2024-09-11T23:52:05.523509+0000","last_peered":"2024-09-11T23:52:05.523509+0000","last_clean":"2024-09-11T23:52:05.523509+0000","last_became_active":"2024-09-11T23:50:10.841552+0000","last_became_peered":"2024-09-11T23:50:10.841552+0000","last_unstale":"2024-09-11T23:52:05.523509+0000","last_undegraded":"2024-09-11T23:52:05.523509+0000","last_fullsized":"2024-09-11T23:52:05.523509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:06.464518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432812+0000","last_change":"2024-09-11T23:50:10.837006+0000","last_active":"2024-09-11T23:52:05.432812+0000","last_peered":"2024-09-11T23:52:05.432812+0000","last_clean":"2024-09-11T23:52:05.432812+0000","last_became_active":"2024-09-11T23:50:10.836190+0000","last_became_peered":"2024-09-11T23:50:10.836190+0000","last_unstale":"2024-09-11T23:52:05.432812+0000","last_undegraded":"2024-09-11T23:52:05.432812+0000","last_fullsized":"2024-09-11T23:52:05.432812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:04.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525015+0000","last_change":"2024-09-11T23:50:10.841805+0000","last_active":"2024-09-11T23:52:05.525015+0000","last_peered":"2024-09-11T23:52:05.525015+0000","last_clean":"2024-09-11T23:52:05.525015+0000","last_became_active":"2024-09-11T23:50:10.841318+0000","last_became_peered":"2024-09-11T23:50:10.841318+0000","last_unstale":"2024-09-11T23:52:05.525015+0000","last_undegraded":"2024-09-11T23:52:05.525015+0000","last_fullsized":"2024-09-11T23:52:05.525015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:39.229090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694943+0000","last_change":"2024-09-11T23:50:10.838368+0000","last_active":"2024-09-11T23:52:05.694943+0000","last_peered":"2024-09-11T23:52:05.694943+0000","last_clean":"2024-09-11T23:52:05.694943+0000","last_became_active":"2024-09-11T23:50:10.837997+0000","last_became_peered":"2024-09-11T23:50:10.837997+0000","last_unstale":"2024-09-11T23:52:05.694943+0000","last_undegraded":"2024-09-11T23:52:05.694943+0000","last_fullsized":"2024-09-11T23:52:05.694943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:34.655827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872635+0000","last_change":"2024-09-11T23:50:10.845267+0000","last_active":"2024-09-11T23:52:05.872635+0000","last_peered":"2024-09-11T23:52:05.872635+0000","last_clean":"2024-09-11T23:52:05.872635+0000","last_became_active":"2024-09-11T23:50:10.844964+0000","last_became_peered":"2024-09-11T23:50:10.844964+0000","last_unstale":"2024-09-11T23:52:05.872635+0000","last_undegraded":"2024-09-11T23:52:05.872635+0000","last_fullsized":"2024-09-11T23:52:05.872635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:43.968613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431698+0000","last_change":"2024-09-11T23:51:14.410108+0000","last_active":"2024-09-11T23:52:05.431698+0000","last_peered":"2024-09-11T23:52:05.431698+0000","last_clean":"2024-09-11T23:52:05.431698+0000","last_became_active":"2024-09-11T23:51:14.409588+0000","last_became_peered":"2024-09-11T23:51:14.409588+0000","last_unstale":"2024-09-11T23:52:05.431698+0000","last_undegraded":"2024-09-11T23:52:05.431698+0000","last_fullsized":"2024-09-11T23:52:05.431698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:29.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158121+0000","last_change":"2024-09-11T23:50:10.847770+0000","last_active":"2024-09-11T23:52:06.158121+0000","last_peered":"2024-09-11T23:52:06.158121+0000","last_clean":"2024-09-11T23:52:06.158121+0000","last_became_active":"2024-09-11T23:50:10.847455+0000","last_became_peered":"2024-09-11T23:50:10.847455+0000","last_unstale":"2024-09-11T23:52:06.158121+0000","last_undegraded":"2024-09-11T23:52:06.158121+0000","last_fullsized":"2024-09-11T23:52:06.158121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:45:25.281970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694617+0000","last_change":"2024-09-11T23:50:10.840552+0000","last_active":"2024-09-11T23:52:05.694617+0000","last_peered":"2024-09-11T23:52:05.694617+0000","last_clean":"2024-09-11T23:52:05.694617+0000","last_became_active":"2024-09-11T23:50:10.840112+0000","last_became_peered":"2024-09-11T23:50:10.840112+0000","last_unstale":"2024-09-11T23:52:05.694617+0000","last_undegraded":"2024-09-11T23:52:05.694617+0000","last_fullsized":"2024-09-11T23:52:05.694617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:46:50.436697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694660+0000","last_change":"2024-09-11T23:51:14.410872+0000","last_active":"2024-09-11T23:52:05.694660+0000","last_peered":"2024-09-11T23:52:05.694660+0000","last_clean":"2024-09-11T23:52:05.694660+0000","last_became_active":"2024-09-11T23:51:14.410531+0000","last_became_peered":"2024-09-11T23:51:14.410531+0000","last_unstale":"2024-09-11T23:52:05.694660+0000","last_undegraded":"2024-09-11T23:52:05.694660+0000","last_fullsized":"2024-09-11T23:52:05.694660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:10.502914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872542+0000","last_change":"2024-09-11T23:50:10.850195+0000","last_active":"2024-09-11T23:52:05.872542+0000","last_peered":"2024-09-11T23:52:05.872542+0000","last_clean":"2024-09-11T23:52:05.872542+0000","last_became_active":"2024-09-11T23:50:10.849375+0000","last_became_peered":"2024-09-11T23:50:10.849375+0000","last_unstale":"2024-09-11T23:52:05.872542+0000","last_undegraded":"2024-09-11T23:52:05.872542+0000","last_fullsized":"2024-09-11T23:52:05.872542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:31.833764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516667+0000","last_change":"2024-09-11T23:50:10.836997+0000","last_active":"2024-09-11T23:52:05.516667+0000","last_peered":"2024-09-11T23:52:05.516667+0000","last_clean":"2024-09-11T23:52:05.516667+0000","last_became_active":"2024-09-11T23:50:10.836669+0000","last_became_peered":"2024-09-11T23:50:10.836669+0000","last_unstale":"2024-09-11T23:52:05.516667+0000","last_undegraded":"2024-09-11T23:52:05.516667+0000","last_fullsized":"2024-09-11T23:52:05.516667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:37.182090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:15.025850+0000","last_change":"2024-09-11T23:50:08.767179+0000","last_active":"2024-09-11T23:52:15.025850+0000","last_peered":"2024-09-11T23:52:15.025850+0000","last_clean":"2024-09-11T23:52:15.025850+0000","last_became_active":"2024-09-11T23:50:08.766818+0000","last_became_peered":"2024-09-11T23:50:08.766818+0000","last_unstale":"2024-09-11T23:52:15.025850+0000","last_undegraded":"2024-09-11T23:52:15.025850+0000","last_fullsized":"2024-09-11T23:52:15.025850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:42.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432914+0000","last_change":"2024-09-11T23:50:10.843261+0000","last_active":"2024-09-11T23:52:05.432914+0000","last_peered":"2024-09-11T23:52:05.432914+0000","last_clean":"2024-09-11T23:52:05.432914+0000","last_became_active":"2024-09-11T23:50:10.841010+0000","last_became_peered":"2024-09-11T23:50:10.841010+0000","last_unstale":"2024-09-11T23:52:05.432914+0000","last_undegraded":"2024-09-11T23:52:05.432914+0000","last_fullsized":"2024-09-11T23:52:05.432914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:05.184062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.434413+0000","last_change":"2024-09-11T23:50:08.762680+0000","last_active":"2024-09-11T23:52:05.434413+0000","last_peered":"2024-09-11T23:52:05.434413+0000","last_clean":"2024-09-11T23:52:05.434413+0000","last_became_active":"2024-09-11T23:50:08.762333+0000","last_became_peered":"2024-09-11T23:50:08.762333+0000","last_unstale":"2024-09-11T23:52:05.434413+0000","last_undegraded":"2024-09-11T23:52:05.434413+0000","last_fullsized":"2024-09-11T23:52:05.434413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:15.241696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431356+0000","last_change":"2024-09-11T23:50:08.767080+0000","last_active":"2024-09-11T23:52:05.431356+0000","last_peered":"2024-09-11T23:52:05.431356+0000","last_clean":"2024-09-11T23:52:05.431356+0000","last_became_active":"2024-09-11T23:50:08.766532+0000","last_became_peered":"2024-09-11T23:50:08.766532+0000","last_unstale":"2024-09-11T23:52:05.431356+0000","last_undegraded":"2024-09-11T23:52:05.431356+0000","last_fullsized":"2024-09-11T23:52:05.431356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:02.413677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523343+0000","last_change":"2024-09-11T23:50:10.841147+0000","last_active":"2024-09-11T23:52:05.523343+0000","last_peered":"2024-09-11T23:52:05.523343+0000","last_clean":"2024-09-11T23:52:05.523343+0000","last_became_active":"2024-09-11T23:50:10.840631+0000","last_became_peered":"2024-09-11T23:50:10.840631+0000","last_unstale":"2024-09-11T23:52:05.523343+0000","last_undegraded":"2024-09-11T23:52:05.523343+0000","last_fullsized":"2024-09-11T23:52:05.523343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:10.485946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433385+0000","last_change":"2024-09-11T23:50:08.763254+0000","last_active":"2024-09-11T23:52:05.433385+0000","last_peered":"2024-09-11T23:52:05.433385+0000","last_clean":"2024-09-11T23:52:05.433385+0000","last_became_active":"2024-09-11T23:50:08.762673+0000","last_became_peered":"2024-09-11T23:50:08.762673+0000","last_unstale":"2024-09-11T23:52:05.433385+0000","last_undegraded":"2024-09-11T23:52:05.433385+0000","last_fullsized":"2024-09-11T23:52:05.433385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:12.517711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.520843+0000","last_change":"2024-09-11T23:50:10.840152+0000","last_active":"2024-09-11T23:52:05.520843+0000","last_peered":"2024-09-11T23:52:05.520843+0000","last_clean":"2024-09-11T23:52:05.520843+0000","last_became_active":"2024-09-11T23:50:10.839851+0000","last_became_peered":"2024-09-11T23:50:10.839851+0000","last_unstale":"2024-09-11T23:52:05.520843+0000","last_undegraded":"2024-09-11T23:52:05.520843+0000","last_fullsized":"2024-09-11T23:52:05.520843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:21.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:15.025211+0000","last_change":"2024-09-11T23:50:08.769461+0000","last_active":"2024-09-11T23:52:15.025211+0000","last_peered":"2024-09-11T23:52:15.025211+0000","last_clean":"2024-09-11T23:52:15.025211+0000","last_became_active":"2024-09-11T23:50:08.769079+0000","last_became_peered":"2024-09-11T23:50:08.769079+0000","last_unstale":"2024-09-11T23:52:15.025211+0000","last_undegraded":"2024-09-11T23:52:15.025211+0000","last_fullsized":"2024-09-11T23:52:15.025211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:53.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694869+0000","last_change":"2024-09-11T23:50:10.841841+0000","last_active":"2024-09-11T23:52:05.694869+0000","last_peered":"2024-09-11T23:52:05.694869+0000","last_clean":"2024-09-11T23:52:05.694869+0000","last_became_active":"2024-09-11T23:50:10.841329+0000","last_became_peered":"2024-09-11T23:50:10.841329+0000","last_unstale":"2024-09-11T23:52:05.694869+0000","last_undegraded":"2024-09-11T23:52:05.694869+0000","last_fullsized":"2024-09-11T23:52:05.694869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:06.943923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746642+0000","last_change":"2024-09-11T23:50:08.764356+0000","last_active":"2024-09-11T23:52:05.746642+0000","last_peered":"2024-09-11T23:52:05.746642+0000","last_clean":"2024-09-11T23:52:05.746642+0000","last_became_active":"2024-09-11T23:50:08.764006+0000","last_became_peered":"2024-09-11T23:50:08.764006+0000","last_unstale":"2024-09-11T23:52:05.746642+0000","last_undegraded":"2024-09-11T23:52:05.746642+0000","last_fullsized":"2024-09-11T23:52:05.746642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:41.859319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158655+0000","last_change":"2024-09-11T23:50:10.845513+0000","last_active":"2024-09-11T23:52:06.158655+0000","last_peered":"2024-09-11T23:52:06.158655+0000","last_clean":"2024-09-11T23:52:06.158655+0000","last_became_active":"2024-09-11T23:50:10.845201+0000","last_became_peered":"2024-09-11T23:50:10.845201+0000","last_unstale":"2024-09-11T23:52:06.158655+0000","last_undegraded":"2024-09-11T23:52:06.158655+0000","last_fullsized":"2024-09-11T23:52:06.158655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:36.571951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433170+0000","last_change":"2024-09-11T23:50:08.953223+0000","last_active":"2024-09-11T23:52:05.433170+0000","last_peered":"2024-09-11T23:52:05.433170+0000","last_clean":"2024-09-11T23:52:05.433170+0000","last_became_active":"2024-09-11T23:50:08.952764+0000","last_became_peered":"2024-09-11T23:50:08.952764+0000","last_unstale":"2024-09-11T23:52:05.433170+0000","last_undegraded":"2024-09-11T23:52:05.433170+0000","last_fullsized":"2024-09-11T23:52:05.433170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:47.232894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694598+0000","last_change":"2024-09-11T23:50:10.842739+0000","last_active":"2024-09-11T23:52:05.694598+0000","last_peered":"2024-09-11T23:52:05.694598+0000","last_clean":"2024-09-11T23:52:05.694598+0000","last_became_active":"2024-09-11T23:50:10.842381+0000","last_became_peered":"2024-09-11T23:50:10.842381+0000","last_unstale":"2024-09-11T23:52:05.694598+0000","last_undegraded":"2024-09-11T23:52:05.694598+0000","last_fullsized":"2024-09-11T23:52:05.694598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:32.825206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'37","reported_seq":307,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:16.026810+0000","last_change":"2024-09-11T23:50:08.765477+0000","last_active":"2024-09-11T23:52:16.026810+0000","last_peered":"2024-09-11T23:52:16.026810+0000","last_clean":"2024-09-11T23:52:16.026810+0000","last_became_active":"2024-09-11T23:50:08.763746+0000","last_became_peered":"2024-09-11T23:50:08.763746+0000","last_unstale":"2024-09-11T23:52:16.026810+0000","last_undegraded":"2024-09-11T23:52:16.026810+0000","last_fullsized":"2024-09-11T23:52:16.026810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:26.019290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67574,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":189,"num_write":38,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516931+0000","last_change":"2024-09-11T23:50:10.844694+0000","last_active":"2024-09-11T23:52:05.516931+0000","last_peered":"2024-09-11T23:52:05.516931+0000","last_clean":"2024-09-11T23:52:05.516931+0000","last_became_active":"2024-09-11T23:50:10.844398+0000","last_became_peered":"2024-09-11T23:50:10.844398+0000","last_unstale":"2024-09-11T23:52:05.516931+0000","last_undegraded":"2024-09-11T23:52:05.516931+0000","last_fullsized":"2024-09-11T23:52:05.516931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:18.945403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694704+0000","last_change":"2024-09-11T23:50:08.763712+0000","last_active":"2024-09-11T23:52:05.694704+0000","last_peered":"2024-09-11T23:52:05.694704+0000","last_clean":"2024-09-11T23:52:05.694704+0000","last_became_active":"2024-09-11T23:50:08.763331+0000","last_became_peered":"2024-09-11T23:50:08.763331+0000","last_unstale":"2024-09-11T23:52:05.694704+0000","last_undegraded":"2024-09-11T23:52:05.694704+0000","last_fullsized":"2024-09-11T23:52:05.694704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:42.618199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431407+0000","last_change":"2024-09-11T23:50:10.851285+0000","last_active":"2024-09-11T23:52:05.431407+0000","last_peered":"2024-09-11T23:52:05.431407+0000","last_clean":"2024-09-11T23:52:05.431407+0000","last_became_active":"2024-09-11T23:50:10.851000+0000","last_became_peered":"2024-09-11T23:50:10.851000+0000","last_unstale":"2024-09-11T23:52:05.431407+0000","last_undegraded":"2024-09-11T23:52:05.431407+0000","last_fullsized":"2024-09-11T23:52:05.431407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:46.987207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433521+0000","last_change":"2024-09-11T23:50:08.763822+0000","last_active":"2024-09-11T23:52:05.433521+0000","last_peered":"2024-09-11T23:52:05.433521+0000","last_clean":"2024-09-11T23:52:05.433521+0000","last_became_active":"2024-09-11T23:50:08.763445+0000","last_became_peered":"2024-09-11T23:50:08.763445+0000","last_unstale":"2024-09-11T23:52:05.433521+0000","last_undegraded":"2024-09-11T23:52:05.433521+0000","last_fullsized":"2024-09-11T23:52:05.433521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:13.583486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746620+0000","last_change":"2024-09-11T23:50:10.851978+0000","last_active":"2024-09-11T23:52:05.746620+0000","last_peered":"2024-09-11T23:52:05.746620+0000","last_clean":"2024-09-11T23:52:05.746620+0000","last_became_active":"2024-09-11T23:50:10.851606+0000","last_became_peered":"2024-09-11T23:50:10.851606+0000","last_unstale":"2024-09-11T23:52:05.746620+0000","last_undegraded":"2024-09-11T23:52:05.746620+0000","last_fullsized":"2024-09-11T23:52:05.746620+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:46.396722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":153,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:15.040273+0000","last_change":"2024-09-11T23:50:08.766621+0000","last_active":"2024-09-11T23:52:15.040273+0000","last_peered":"2024-09-11T23:52:15.040273+0000","last_clean":"2024-09-11T23:52:15.040273+0000","last_became_active":"2024-09-11T23:50:08.766301+0000","last_became_peered":"2024-09-11T23:50:08.766301+0000","last_unstale":"2024-09-11T23:52:15.040273+0000","last_undegraded":"2024-09-11T23:52:15.040273+0000","last_fullsized":"2024-09-11T23:52:15.040273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:34.339668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746281+0000","last_change":"2024-09-11T23:50:10.847781+0000","last_active":"2024-09-11T23:52:05.746281+0000","last_peered":"2024-09-11T23:52:05.746281+0000","last_clean":"2024-09-11T23:52:05.746281+0000","last_became_active":"2024-09-11T23:50:10.847458+0000","last_became_peered":"2024-09-11T23:50:10.847458+0000","last_unstale":"2024-09-11T23:52:05.746281+0000","last_undegraded":"2024-09-11T23:52:05.746281+0000","last_fullsized":"2024-09-11T23:52:05.746281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:52.816749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516591+0000","last_change":"2024-09-11T23:50:08.951590+0000","last_active":"2024-09-11T23:52:05.516591+0000","last_peered":"2024-09-11T23:52:05.516591+0000","last_clean":"2024-09-11T23:52:05.516591+0000","last_became_active":"2024-09-11T23:50:08.951211+0000","last_became_peered":"2024-09-11T23:50:08.951211+0000","last_unstale":"2024-09-11T23:52:05.516591+0000","last_undegraded":"2024-09-11T23:52:05.516591+0000","last_fullsized":"2024-09-11T23:52:05.516591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:35.299744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432801+0000","last_change":"2024-09-11T23:50:10.841794+0000","last_active":"2024-09-11T23:52:05.432801+0000","last_peered":"2024-09-11T23:52:05.432801+0000","last_clean":"2024-09-11T23:52:05.432801+0000","last_became_active":"2024-09-11T23:50:10.841457+0000","last_became_peered":"2024-09-11T23:50:10.841457+0000","last_unstale":"2024-09-11T23:52:05.432801+0000","last_undegraded":"2024-09-11T23:52:05.432801+0000","last_fullsized":"2024-09-11T23:52:05.432801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:35.374414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516946+0000","last_change":"2024-09-11T23:50:08.766948+0000","last_active":"2024-09-11T23:52:05.516946+0000","last_peered":"2024-09-11T23:52:05.516946+0000","last_clean":"2024-09-11T23:52:05.516946+0000","last_became_active":"2024-09-11T23:50:08.766629+0000","last_became_peered":"2024-09-11T23:50:08.766629+0000","last_unstale":"2024-09-11T23:52:05.516946+0000","last_undegraded":"2024-09-11T23:52:05.516946+0000","last_fullsized":"2024-09-11T23:52:05.516946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:35.959441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872534+0000","last_change":"2024-09-11T23:50:10.840138+0000","last_active":"2024-09-11T23:52:05.872534+0000","last_peered":"2024-09-11T23:52:05.872534+0000","last_clean":"2024-09-11T23:52:05.872534+0000","last_became_active":"2024-09-11T23:50:10.839351+0000","last_became_peered":"2024-09-11T23:50:10.839351+0000","last_unstale":"2024-09-11T23:52:05.872534+0000","last_undegraded":"2024-09-11T23:52:05.872534+0000","last_fullsized":"2024-09-11T23:52:05.872534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:13.841541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431687+0000","last_change":"2024-09-11T23:51:14.413526+0000","last_active":"2024-09-11T23:52:05.431687+0000","last_peered":"2024-09-11T23:52:05.431687+0000","last_clean":"2024-09-11T23:52:05.431687+0000","last_became_active":"2024-09-11T23:51:14.411027+0000","last_became_peered":"2024-09-11T23:51:14.411027+0000","last_unstale":"2024-09-11T23:52:05.431687+0000","last_undegraded":"2024-09-11T23:52:05.431687+0000","last_fullsized":"2024-09-11T23:52:05.431687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:55.020567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746675+0000","last_change":"2024-09-11T23:50:10.846200+0000","last_active":"2024-09-11T23:52:05.746675+0000","last_peered":"2024-09-11T23:52:05.746675+0000","last_clean":"2024-09-11T23:52:05.746675+0000","last_became_active":"2024-09-11T23:50:10.845471+0000","last_became_peered":"2024-09-11T23:50:10.845471+0000","last_unstale":"2024-09-11T23:52:05.746675+0000","last_undegraded":"2024-09-11T23:52:05.746675+0000","last_fullsized":"2024-09-11T23:52:05.746675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:58.566313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'65","reported_seq":120,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:15.104271+0000","last_change":"2024-09-11T23:50:08.764339+0000","last_active":"2024-09-11T23:52:15.104271+0000","last_peered":"2024-09-11T23:52:15.104271+0000","last_clean":"2024-09-11T23:52:15.104271+0000","last_became_active":"2024-09-11T23:50:08.762911+0000","last_became_peered":"2024-09-11T23:50:08.762911+0000","last_unstale":"2024-09-11T23:52:15.104271+0000","last_undegraded":"2024-09-11T23:52:15.104271+0000","last_fullsized":"2024-09-11T23:52:15.104271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:33.861017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746458+0000","last_change":"2024-09-11T23:50:10.851165+0000","last_active":"2024-09-11T23:52:05.746458+0000","last_peered":"2024-09-11T23:52:05.746458+0000","last_clean":"2024-09-11T23:52:05.746458+0000","last_became_active":"2024-09-11T23:50:10.850794+0000","last_became_peered":"2024-09-11T23:50:10.850794+0000","last_unstale":"2024-09-11T23:52:05.746458+0000","last_undegraded":"2024-09-11T23:52:05.746458+0000","last_fullsized":"2024-09-11T23:52:05.746458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:57.925514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694804+0000","last_change":"2024-09-11T23:50:08.952467+0000","last_active":"2024-09-11T23:52:05.694804+0000","last_peered":"2024-09-11T23:52:05.694804+0000","last_clean":"2024-09-11T23:52:05.694804+0000","last_became_active":"2024-09-11T23:50:08.952027+0000","last_became_peered":"2024-09-11T23:50:08.952027+0000","last_unstale":"2024-09-11T23:52:05.694804+0000","last_undegraded":"2024-09-11T23:52:05.694804+0000","last_fullsized":"2024-09-11T23:52:05.694804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:07:18.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746570+0000","last_change":"2024-09-11T23:50:10.846694+0000","last_active":"2024-09-11T23:52:05.746570+0000","last_peered":"2024-09-11T23:52:05.746570+0000","last_clean":"2024-09-11T23:52:05.746570+0000","last_became_active":"2024-09-11T23:50:10.846085+0000","last_became_peered":"2024-09-11T23:50:10.846085+0000","last_unstale":"2024-09-11T23:52:05.746570+0000","last_undegraded":"2024-09-11T23:52:05.746570+0000","last_fullsized":"2024-09-11T23:52:05.746570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:22.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"84'148","reported_seq":394,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:07.529793+0000","last_change":"2024-09-11T23:48:56.656100+0000","last_active":"2024-09-11T23:52:07.529793+0000","last_peered":"2024-09-11T23:52:07.529793+0000","last_clean":"2024-09-11T23:52:07.529793+0000","last_became_active":"2024-09-11T23:48:56.655706+0000","last_became_peered":"2024-09-11T23:48:56.655706+0000","last_unstale":"2024-09-11T23:52:07.529793+0000","last_undegraded":"2024-09-11T23:52:07.529793+0000","last_fullsized":"2024-09-11T23:52:07.529793+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:45:29.271141+0000","last_clean_scrub_stamp":"2024-09-11T23:45:29.271141+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:59.816726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.744951+0000","last_change":"2024-09-11T23:50:08.764814+0000","last_active":"2024-09-11T23:52:05.744951+0000","last_peered":"2024-09-11T23:52:05.744951+0000","last_clean":"2024-09-11T23:52:05.744951+0000","last_became_active":"2024-09-11T23:50:08.764490+0000","last_became_peered":"2024-09-11T23:50:08.764490+0000","last_unstale":"2024-09-11T23:52:05.744951+0000","last_undegraded":"2024-09-11T23:52:05.744951+0000","last_fullsized":"2024-09-11T23:52:05.744951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:16:01.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523757+0000","last_change":"2024-09-11T23:50:10.838146+0000","last_active":"2024-09-11T23:52:05.523757+0000","last_peered":"2024-09-11T23:52:05.523757+0000","last_clean":"2024-09-11T23:52:05.523757+0000","last_became_active":"2024-09-11T23:50:10.837823+0000","last_became_peered":"2024-09-11T23:50:10.837823+0000","last_unstale":"2024-09-11T23:52:05.523757+0000","last_undegraded":"2024-09-11T23:52:05.523757+0000","last_fullsized":"2024-09-11T23:52:05.523757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:07.622362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872415+0000","last_change":"2024-09-11T23:51:14.121480+0000","last_active":"2024-09-11T23:52:05.872415+0000","last_peered":"2024-09-11T23:52:05.872415+0000","last_clean":"2024-09-11T23:52:05.872415+0000","last_became_active":"2024-09-11T23:51:14.118435+0000","last_became_peered":"2024-09-11T23:51:14.118435+0000","last_unstale":"2024-09-11T23:52:05.872415+0000","last_undegraded":"2024-09-11T23:52:05.872415+0000","last_fullsized":"2024-09-11T23:52:05.872415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431409+0000","last_change":"2024-09-11T23:50:10.849567+0000","last_active":"2024-09-11T23:52:05.431409+0000","last_peered":"2024-09-11T23:52:05.431409+0000","last_clean":"2024-09-11T23:52:05.431409+0000","last_became_active":"2024-09-11T23:50:10.849237+0000","last_became_peered":"2024-09-11T23:50:10.849237+0000","last_unstale":"2024-09-11T23:52:05.431409+0000","last_undegraded":"2024-09-11T23:52:05.431409+0000","last_fullsized":"2024-09-11T23:52:05.431409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":147,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:16.041284+0000","last_change":"2024-09-11T23:50:08.950924+0000","last_active":"2024-09-11T23:52:16.041284+0000","last_peered":"2024-09-11T23:52:16.041284+0000","last_clean":"2024-09-11T23:52:16.041284+0000","last_became_active":"2024-09-11T23:50:08.950638+0000","last_became_peered":"2024-09-11T23:50:08.950638+0000","last_unstale":"2024-09-11T23:52:16.041284+0000","last_undegraded":"2024-09-11T23:52:16.041284+0000","last_fullsized":"2024-09-11T23:52:16.041284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:22.425397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.432911+0000","last_change":"2024-09-11T23:50:10.835518+0000","last_active":"2024-09-11T23:52:05.432911+0000","last_peered":"2024-09-11T23:52:05.432911+0000","last_clean":"2024-09-11T23:52:05.432911+0000","last_became_active":"2024-09-11T23:50:10.835175+0000","last_became_peered":"2024-09-11T23:50:10.835175+0000","last_unstale":"2024-09-11T23:52:05.432911+0000","last_undegraded":"2024-09-11T23:52:05.432911+0000","last_fullsized":"2024-09-11T23:52:05.432911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:08:58.888960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746571+0000","last_change":"2024-09-11T23:50:08.765263+0000","last_active":"2024-09-11T23:52:05.746571+0000","last_peered":"2024-09-11T23:52:05.746571+0000","last_clean":"2024-09-11T23:52:05.746571+0000","last_became_active":"2024-09-11T23:50:08.764952+0000","last_became_peered":"2024-09-11T23:50:08.764952+0000","last_unstale":"2024-09-11T23:52:05.746571+0000","last_undegraded":"2024-09-11T23:52:05.746571+0000","last_fullsized":"2024-09-11T23:52:05.746571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:27.803513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435062+0000","last_change":"2024-09-11T23:51:14.409908+0000","last_active":"2024-09-11T23:52:05.435062+0000","last_peered":"2024-09-11T23:52:05.435062+0000","last_clean":"2024-09-11T23:52:05.435062+0000","last_became_active":"2024-09-11T23:51:14.409432+0000","last_became_peered":"2024-09-11T23:51:14.409432+0000","last_unstale":"2024-09-11T23:52:05.435062+0000","last_undegraded":"2024-09-11T23:52:05.435062+0000","last_fullsized":"2024-09-11T23:52:05.435062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:06.371306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.521215+0000","last_change":"2024-09-11T23:50:08.952147+0000","last_active":"2024-09-11T23:52:05.521215+0000","last_peered":"2024-09-11T23:52:05.521215+0000","last_clean":"2024-09-11T23:52:05.521215+0000","last_became_active":"2024-09-11T23:50:08.951743+0000","last_became_peered":"2024-09-11T23:50:08.951743+0000","last_unstale":"2024-09-11T23:52:05.521215+0000","last_undegraded":"2024-09-11T23:52:05.521215+0000","last_fullsized":"2024-09-11T23:52:05.521215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:16:29.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.435167+0000","last_change":"2024-09-11T23:50:10.844712+0000","last_active":"2024-09-11T23:52:05.435167+0000","last_peered":"2024-09-11T23:52:05.435167+0000","last_clean":"2024-09-11T23:52:05.435167+0000","last_became_active":"2024-09-11T23:50:10.844219+0000","last_became_peered":"2024-09-11T23:50:10.844219+0000","last_unstale":"2024-09-11T23:52:05.435167+0000","last_undegraded":"2024-09-11T23:52:05.435167+0000","last_fullsized":"2024-09-11T23:52:05.435167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:36.860301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.517017+0000","last_change":"2024-09-11T23:50:08.953113+0000","last_active":"2024-09-11T23:52:05.517017+0000","last_peered":"2024-09-11T23:52:05.517017+0000","last_clean":"2024-09-11T23:52:05.517017+0000","last_became_active":"2024-09-11T23:50:08.952771+0000","last_became_peered":"2024-09-11T23:50:08.952771+0000","last_unstale":"2024-09-11T23:52:05.517017+0000","last_undegraded":"2024-09-11T23:52:05.517017+0000","last_fullsized":"2024-09-11T23:52:05.517017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:07.103708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.431819+0000","last_change":"2024-09-11T23:50:10.849818+0000","last_active":"2024-09-11T23:52:05.431819+0000","last_peered":"2024-09-11T23:52:05.431819+0000","last_clean":"2024-09-11T23:52:05.431819+0000","last_became_active":"2024-09-11T23:50:10.849463+0000","last_became_peered":"2024-09-11T23:50:10.849463+0000","last_unstale":"2024-09-11T23:52:05.431819+0000","last_undegraded":"2024-09-11T23:52:05.431819+0000","last_fullsized":"2024-09-11T23:52:05.431819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:56.770397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'101","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:15.461394+0000","last_change":"2024-09-11T23:50:08.763552+0000","last_active":"2024-09-11T23:52:15.461394+0000","last_peered":"2024-09-11T23:52:15.461394+0000","last_clean":"2024-09-11T23:52:15.461394+0000","last_became_active":"2024-09-11T23:50:08.763205+0000","last_became_peered":"2024-09-11T23:50:08.763205+0000","last_unstale":"2024-09-11T23:52:15.461394+0000","last_undegraded":"2024-09-11T23:52:15.461394+0000","last_fullsized":"2024-09-11T23:52:15.461394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:47:32.731450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525022+0000","last_change":"2024-09-11T23:50:10.843148+0000","last_active":"2024-09-11T23:52:05.525022+0000","last_peered":"2024-09-11T23:52:05.525022+0000","last_clean":"2024-09-11T23:52:05.525022+0000","last_became_active":"2024-09-11T23:50:10.842843+0000","last_became_peered":"2024-09-11T23:50:10.842843+0000","last_unstale":"2024-09-11T23:52:05.525022+0000","last_undegraded":"2024-09-11T23:52:05.525022+0000","last_fullsized":"2024-09-11T23:52:05.525022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:53.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.745158+0000","last_change":"2024-09-11T23:50:08.819314+0000","last_active":"2024-09-11T23:52:05.745158+0000","last_peered":"2024-09-11T23:52:05.745158+0000","last_clean":"2024-09-11T23:52:05.745158+0000","last_became_active":"2024-09-11T23:50:08.818944+0000","last_became_peered":"2024-09-11T23:50:08.818944+0000","last_unstale":"2024-09-11T23:52:05.745158+0000","last_undegraded":"2024-09-11T23:52:05.745158+0000","last_fullsized":"2024-09-11T23:52:05.745158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:32.191758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158521+0000","last_change":"2024-09-11T23:50:10.843815+0000","last_active":"2024-09-11T23:52:06.158521+0000","last_peered":"2024-09-11T23:52:06.158521+0000","last_clean":"2024-09-11T23:52:06.158521+0000","last_became_active":"2024-09-11T23:50:10.843181+0000","last_became_peered":"2024-09-11T23:50:10.843181+0000","last_unstale":"2024-09-11T23:52:06.158521+0000","last_undegraded":"2024-09-11T23:52:06.158521+0000","last_fullsized":"2024-09-11T23:52:06.158521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:04.487838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694550+0000","last_change":"2024-09-11T23:50:08.951457+0000","last_active":"2024-09-11T23:52:05.694550+0000","last_peered":"2024-09-11T23:52:05.694550+0000","last_clean":"2024-09-11T23:52:05.694550+0000","last_became_active":"2024-09-11T23:50:08.951122+0000","last_became_peered":"2024-09-11T23:50:08.951122+0000","last_unstale":"2024-09-11T23:52:05.694550+0000","last_undegraded":"2024-09-11T23:52:05.694550+0000","last_fullsized":"2024-09-11T23:52:05.694550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.979897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516588+0000","last_change":"2024-09-11T23:50:10.845866+0000","last_active":"2024-09-11T23:52:05.516588+0000","last_peered":"2024-09-11T23:52:05.516588+0000","last_clean":"2024-09-11T23:52:05.516588+0000","last_became_active":"2024-09-11T23:50:10.845553+0000","last_became_peered":"2024-09-11T23:50:10.845553+0000","last_unstale":"2024-09-11T23:52:05.516588+0000","last_undegraded":"2024-09-11T23:52:05.516588+0000","last_fullsized":"2024-09-11T23:52:05.516588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:13.603819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.523605+0000","last_change":"2024-09-11T23:50:08.760202+0000","last_active":"2024-09-11T23:52:05.523605+0000","last_peered":"2024-09-11T23:52:05.523605+0000","last_clean":"2024-09-11T23:52:05.523605+0000","last_became_active":"2024-09-11T23:50:08.759892+0000","last_became_peered":"2024-09-11T23:50:08.759892+0000","last_unstale":"2024-09-11T23:52:05.523605+0000","last_undegraded":"2024-09-11T23:52:05.523605+0000","last_fullsized":"2024-09-11T23:52:05.523605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:51.158749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872645+0000","last_change":"2024-09-11T23:50:10.849524+0000","last_active":"2024-09-11T23:52:05.872645+0000","last_peered":"2024-09-11T23:52:05.872645+0000","last_clean":"2024-09-11T23:52:05.872645+0000","last_became_active":"2024-09-11T23:50:10.849199+0000","last_became_peered":"2024-09-11T23:50:10.849199+0000","last_unstale":"2024-09-11T23:52:05.872645+0000","last_undegraded":"2024-09-11T23:52:05.872645+0000","last_fullsized":"2024-09-11T23:52:05.872645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:40.159458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.872496+0000","last_change":"2024-09-11T23:51:14.123663+0000","last_active":"2024-09-11T23:52:05.872496+0000","last_peered":"2024-09-11T23:52:05.872496+0000","last_clean":"2024-09-11T23:52:05.872496+0000","last_became_active":"2024-09-11T23:51:14.118315+0000","last_became_peered":"2024-09-11T23:51:14.118315+0000","last_unstale":"2024-09-11T23:52:05.872496+0000","last_undegraded":"2024-09-11T23:52:05.872496+0000","last_fullsized":"2024-09-11T23:52:05.872496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:59:10.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.524009+0000","last_change":"2024-09-11T23:50:10.837084+0000","last_active":"2024-09-11T23:52:05.524009+0000","last_peered":"2024-09-11T23:52:05.524009+0000","last_clean":"2024-09-11T23:52:05.524009+0000","last_became_active":"2024-09-11T23:50:10.836770+0000","last_became_peered":"2024-09-11T23:50:10.836770+0000","last_unstale":"2024-09-11T23:52:05.524009+0000","last_undegraded":"2024-09-11T23:52:05.524009+0000","last_fullsized":"2024-09-11T23:52:05.524009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:25.663931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516698+0000","last_change":"2024-09-11T23:50:08.768627+0000","last_active":"2024-09-11T23:52:05.516698+0000","last_peered":"2024-09-11T23:52:05.516698+0000","last_clean":"2024-09-11T23:52:05.516698+0000","last_became_active":"2024-09-11T23:50:08.768195+0000","last_became_peered":"2024-09-11T23:50:08.768195+0000","last_unstale":"2024-09-11T23:52:05.516698+0000","last_undegraded":"2024-09-11T23:52:05.516698+0000","last_fullsized":"2024-09-11T23:52:05.516698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:17.452089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.746447+0000","last_change":"2024-09-11T23:50:10.844036+0000","last_active":"2024-09-11T23:52:05.746447+0000","last_peered":"2024-09-11T23:52:05.746447+0000","last_clean":"2024-09-11T23:52:05.746447+0000","last_became_active":"2024-09-11T23:50:10.843470+0000","last_became_peered":"2024-09-11T23:50:10.843470+0000","last_unstale":"2024-09-11T23:52:05.746447+0000","last_undegraded":"2024-09-11T23:52:05.746447+0000","last_fullsized":"2024-09-11T23:52:05.746447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:36.260637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.433119+0000","last_change":"2024-09-11T23:50:10.845447+0000","last_active":"2024-09-11T23:52:05.433119+0000","last_peered":"2024-09-11T23:52:05.433119+0000","last_clean":"2024-09-11T23:52:05.433119+0000","last_became_active":"2024-09-11T23:50:10.845107+0000","last_became_peered":"2024-09-11T23:50:10.845107+0000","last_unstale":"2024-09-11T23:52:05.433119+0000","last_undegraded":"2024-09-11T23:52:05.433119+0000","last_fullsized":"2024-09-11T23:52:05.433119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:16.435529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516897+0000","last_change":"2024-09-11T23:50:08.770077+0000","last_active":"2024-09-11T23:52:05.516897+0000","last_peered":"2024-09-11T23:52:05.516897+0000","last_clean":"2024-09-11T23:52:05.516897+0000","last_became_active":"2024-09-11T23:50:08.769659+0000","last_became_peered":"2024-09-11T23:50:08.769659+0000","last_unstale":"2024-09-11T23:52:05.516897+0000","last_undegraded":"2024-09-11T23:52:05.516897+0000","last_fullsized":"2024-09-11T23:52:05.516897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:47.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.694561+0000","last_change":"2024-09-11T23:50:10.843596+0000","last_active":"2024-09-11T23:52:05.694561+0000","last_peered":"2024-09-11T23:52:05.694561+0000","last_clean":"2024-09-11T23:52:05.694561+0000","last_became_active":"2024-09-11T23:50:10.843258+0000","last_became_peered":"2024-09-11T23:50:10.843258+0000","last_unstale":"2024-09-11T23:52:05.694561+0000","last_undegraded":"2024-09-11T23:52:05.694561+0000","last_fullsized":"2024-09-11T23:52:05.694561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:02:40.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.516945+0000","last_change":"2024-09-11T23:50:08.769936+0000","last_active":"2024-09-11T23:52:05.516945+0000","last_peered":"2024-09-11T23:52:05.516945+0000","last_clean":"2024-09-11T23:52:05.516945+0000","last_became_active":"2024-09-11T23:50:08.769550+0000","last_became_peered":"2024-09-11T23:50:08.769550+0000","last_unstale":"2024-09-11T23:52:05.516945+0000","last_undegraded":"2024-09-11T23:52:05.516945+0000","last_fullsized":"2024-09-11T23:52:05.516945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:02.721754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:06.158665+0000","last_change":"2024-09-11T23:50:10.845429+0000","last_active":"2024-09-11T23:52:06.158665+0000","last_peered":"2024-09-11T23:52:06.158665+0000","last_clean":"2024-09-11T23:52:06.158665+0000","last_became_active":"2024-09-11T23:50:10.845113+0000","last_became_peered":"2024-09-11T23:50:10.845113+0000","last_unstale":"2024-09-11T23:52:06.158665+0000","last_undegraded":"2024-09-11T23:52:06.158665+0000","last_fullsized":"2024-09-11T23:52:06.158665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:54.456453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525482+0000","last_change":"2024-09-11T23:50:08.951078+0000","last_active":"2024-09-11T23:52:05.525482+0000","last_peered":"2024-09-11T23:52:05.525482+0000","last_clean":"2024-09-11T23:52:05.525482+0000","last_became_active":"2024-09-11T23:50:08.950707+0000","last_became_peered":"2024-09-11T23:50:08.950707+0000","last_unstale":"2024-09-11T23:52:05.525482+0000","last_undegraded":"2024-09-11T23:52:05.525482+0000","last_fullsized":"2024-09-11T23:52:05.525482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:15.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:05.525206+0000","last_change":"2024-09-11T23:50:10.836286+0000","last_active":"2024-09-11T23:52:05.525206+0000","last_peered":"2024-09-11T23:52:05.525206+0000","last_clean":"2024-09-11T23:52:05.525206+0000","last_became_active":"2024-09-11T23:50:10.835968+0000","last_became_peered":"2024-09-11T23:50:10.835968+0000","last_unstale":"2024-09-11T23:52:05.525206+0000","last_undegraded":"2024-09-11T23:52:05.525206+0000","last_fullsized":"2024-09-11T23:52:05.525206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:09.814327+0000","last_clean_scrub_stamp":"2024-09-11T23:50:09.814327+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:59.881133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:52:11.458220+0000","last_change":"2024-09-11T23:51:14.125596+0000","last_active":"2024-09-11T23:52:11.458220+0000","last_peered":"2024-09-11T23:52:11.458220+0000","last_clean":"2024-09-11T23:52:11.458220+0000","last_became_active":"2024-09-11T23:51:14.119682+0000","last_became_peered":"2024-09-11T23:51:14.119682+0000","last_unstale":"2024-09-11T23:52:11.458220+0000","last_undegraded":"2024-09-11T23:52:11.458220+0000","last_fullsized":"2024-09-11T23:52:11.458220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:50:07.748464+0000","last_clean_scrub_stamp":"2024-09-11T23:50:07.748464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:06.062602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":794260,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1079,"num_read_kb":754,"num_write":422,"num_write_kb":2354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2382780,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107565,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":909312,"data_stored":445896,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1114112,"data_stored":1434418,"data_compressed":53957,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939697,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":839348,"data_compressed":22189,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":838686,"data_compressed":22189,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1039219,"data_compressed":38755,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445521,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1032192,"data_stored":1365900,"data_compressed":53957,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":447193,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":999424,"data_stored":1167280,"data_compressed":37391,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":514631,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":513470,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":446792,"data_compressed":2900,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35855,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330101,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":398619,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:52:20.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:52:20.794 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:20 smithi062 ceph-mon[30958]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-09-11T23:52:20.794 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:20 smithi062 ceph-mon[30958]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:20.839 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:20.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:20 smithi120 ceph-mon[35182]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-09-11T23:52:20.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:20 smithi120 ceph-mon[35182]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:20 smithi083 ceph-mon[34805]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-09-11T23:52:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:20 smithi083 ceph-mon[34805]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:52:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.355+0000 7f4f073ae640 1 -- 172.21.15.62:0/1327019954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00102ef0 msgr2=0x7f4f0010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.355+0000 7f4f073ae640 1 --2- 172.21.15.62:0/1327019954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00102ef0 0x7f4f0010da40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009ca0 tx=0x7f4ef802a950 comp rx=0 tx=0).stop 2024-09-11T23:52:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.355+0000 7f4f073ae640 1 -- 172.21.15.62:0/1327019954 shutdown_connections 2024-09-11T23:52:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.355+0000 7f4f073ae640 1 --2- 172.21.15.62:0/1327019954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00102ef0 0x7f4f0010da40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.355+0000 7f4f073ae640 1 --2- 172.21.15.62:0/1327019954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00102570 0x7f4f001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.355+0000 7f4f073ae640 1 --2- 172.21.15.62:0/1327019954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00100e20 0x7f4f00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.355+0000 7f4f073ae640 1 -- 172.21.15.62:0/1327019954 >> 172.21.15.62:0/1327019954 conn(0x7f4f000fc7d0 msgr2=0x7f4f000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:21.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.355+0000 7f4f073ae640 1 -- 172.21.15.62:0/1327019954 shutdown_connections 2024-09-11T23:52:21.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.355+0000 7f4f073ae640 1 -- 172.21.15.62:0/1327019954 wait complete. 2024-09-11T23:52:21.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f073ae640 1 Processor -- start 2024-09-11T23:52:21.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f073ae640 1 -- start start 2024-09-11T23:52:21.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f073ae640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:21.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f073ae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00102570 0x7f4f000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:21.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f073ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00102ef0 0x7f4f000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f073ae640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00110fa0 con 0x7f4f00100e20 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f073ae640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00110e20 con 0x7f4f00102ef0 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f073ae640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00111120 con 0x7f4f00102570 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f0514a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f0514a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f000fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47286/0 (socket says 172.21.15.62:47286) 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f0514a640 1 -- 172.21.15.62:0/330783819 learned_addr learned my addr 172.21.15.62:0/330783819 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f0594b640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00102ef0 0x7f4f000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.356+0000 7f4f04949640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00102570 0x7f4f000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4f0514a640 1 -- 172.21.15.62:0/330783819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00102570 msgr2=0x7f4f000fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4f0514a640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00102570 0x7f4f000fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4f0514a640 1 -- 172.21.15.62:0/330783819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00102ef0 msgr2=0x7f4f000fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4f0514a640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00102ef0 0x7f4f000fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4f0514a640 1 -- 172.21.15.62:0/330783819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f001a6a50 con 0x7f4f00100e20 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4f0514a640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f000fcc80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00d950 tx=0x7f4efc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4ef67fc640 1 -- 172.21.15.62:0/330783819 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc017070 con 0x7f4f00100e20 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4ef67fc640 1 -- 172.21.15.62:0/330783819 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4efc009c30 con 0x7f4f00100e20 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4f073ae640 1 -- 172.21.15.62:0/330783819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f001a6d40 con 0x7f4f00100e20 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4ef67fc640 1 -- 172.21.15.62:0/330783819 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00f6b0 con 0x7f4f00100e20 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.357+0000 7f4f073ae640 1 -- 172.21.15.62:0/330783819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f001a7220 con 0x7f4f00100e20 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.358+0000 7f4f073ae640 1 -- 172.21.15.62:0/330783819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f00101200 con 0x7f4f00100e20 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.359+0000 7f4ef67fc640 1 -- 172.21.15.62:0/330783819 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f4efc0109d0 con 0x7f4f00100e20 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.360+0000 7f4ef67fc640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4eec0a2d80 0x7f4eec0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.361+0000 7f4f04949640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4eec0a2d80 0x7f4eec0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.361+0000 7f4ef67fc640 1 -- 172.21.15.62:0/330783819 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f4efc0cf270 con 0x7f4f00100e20 2024-09-11T23:52:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.361+0000 7f4f04949640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4eec0a2d80 0x7f4eec0a5240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f000fe710 tx=0x7f4ef0008040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.364+0000 7f4ef67fc640 1 -- 172.21.15.62:0/330783819 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4efc009de0 con 0x7f4f00100e20 2024-09-11T23:52:21.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.579+0000 7f4f073ae640 1 -- 172.21.15.62:0/330783819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f0010a0a0 con 0x7f4f00100e20 2024-09-11T23:52:21.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.580+0000 7f4ef67fc640 1 -- 172.21.15.62:0/330783819 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f4efc09bcf0 con 0x7f4f00100e20 2024-09-11T23:52:21.580 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:21.580 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"btime":"2024-09-11T23:52:04:422374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:04.372188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:52:21.580 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2024-09-11T23:52:21.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 -- 172.21.15.62:0/330783819 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4eec0a2d80 msgr2=0x7f4eec0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:21.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4eec0a2d80 0x7f4eec0a5240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f000fe710 tx=0x7f4ef0008040 comp rx=0 tx=0).stop 2024-09-11T23:52:21.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 -- 172.21.15.62:0/330783819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 msgr2=0x7f4f000fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:21.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f000fcc80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00d950 tx=0x7f4efc00de10 comp rx=0 tx=0).stop 2024-09-11T23:52:21.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 -- 172.21.15.62:0/330783819 shutdown_connections 2024-09-11T23:52:21.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4eec0a2d80 0x7f4eec0a5240 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:21.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00102ef0 0x7f4f000fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:21.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f00102570 0x7f4f000fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:21.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 --2- 172.21.15.62:0/330783819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f00100e20 0x7f4f000fcc80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:21.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 -- 172.21.15.62:0/330783819 >> 172.21.15.62:0/330783819 conn(0x7f4f000fc7d0 msgr2=0x7f4f0006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:21.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 -- 172.21.15.62:0/330783819 shutdown_connections 2024-09-11T23:52:21.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:21.582+0000 7f4f073ae640 1 -- 172.21.15.62:0/330783819 wait complete. 2024-09-11T23:52:22.246 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:52:22.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:22 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/330783819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:22.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:22 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/330783819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:22.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:22 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/330783819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:22.503 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.013+0000 7f3929f9f640 1 -- 172.21.15.62:0/2762273427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924102d40 msgr2=0x7f3924103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.013+0000 7f3929f9f640 1 --2- 172.21.15.62:0/2762273427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924102d40 0x7f3924103120 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3914009900 tx=0x7f391402a480 comp rx=0 tx=0).stop 2024-09-11T23:52:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.013+0000 7f3929f9f640 1 -- 172.21.15.62:0/2762273427 shutdown_connections 2024-09-11T23:52:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.013+0000 7f3929f9f640 1 --2- 172.21.15.62:0/2762273427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f392410c170 0x7f392410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.013+0000 7f3929f9f640 1 --2- 172.21.15.62:0/2762273427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3924103660 0x7f392410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.013+0000 7f3929f9f640 1 --2- 172.21.15.62:0/2762273427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924102d40 0x7f3924103120 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.013+0000 7f3929f9f640 1 -- 172.21.15.62:0/2762273427 >> 172.21.15.62:0/2762273427 conn(0x7f39240fc7a0 msgr2=0x7f39240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.013+0000 7f3929f9f640 1 -- 172.21.15.62:0/2762273427 shutdown_connections 2024-09-11T23:52:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.013+0000 7f3929f9f640 1 -- 172.21.15.62:0/2762273427 wait complete. 2024-09-11T23:52:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.013+0000 7f3929f9f640 1 Processor -- start 2024-09-11T23:52:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3929f9f640 1 -- start start 2024-09-11T23:52:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3929f9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3924102d40 0x7f3924198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3929f9f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924103660 0x7f39241989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3929f9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392410c170 0x7f392419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3929f9f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3924111fb0 con 0x7f3924103660 2024-09-11T23:52:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3929f9f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3924111e30 con 0x7f3924102d40 2024-09-11T23:52:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3929f9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3924112130 con 0x7f392410c170 2024-09-11T23:52:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f39237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3924102d40 0x7f3924198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3922ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924103660 0x7f39241989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3923fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392410c170 0x7f392419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3922ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924103660 0x7f39241989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47312/0 (socket says 172.21.15.62:47312) 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f39237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3924102d40 0x7f3924198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35118/0 (socket says 172.21.15.62:35118) 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.014+0000 7f3922ffd640 1 -- 172.21.15.62:0/1143100454 learned_addr learned my addr 172.21.15.62:0/1143100454 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3922ffd640 1 -- 172.21.15.62:0/1143100454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392410c170 msgr2=0x7f392419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3922ffd640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392410c170 0x7f392419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3922ffd640 1 -- 172.21.15.62:0/1143100454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3924102d40 msgr2=0x7f3924198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3922ffd640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3924102d40 0x7f3924198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3922ffd640 1 -- 172.21.15.62:0/1143100454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392419d420 con 0x7f3924103660 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3922ffd640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924103660 0x7f39241989b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f391800b3f0 tx=0x7f391800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3920ff9640 1 -- 172.21.15.62:0/1143100454 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918015020 con 0x7f3924103660 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3920ff9640 1 -- 172.21.15.62:0/1143100454 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3918011d10 con 0x7f3924103660 2024-09-11T23:52:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3929f9f640 1 -- 172.21.15.62:0/1143100454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392419d710 con 0x7f3924103660 2024-09-11T23:52:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3920ff9640 1 -- 172.21.15.62:0/1143100454 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391800f2e0 con 0x7f3924103660 2024-09-11T23:52:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.015+0000 7f3929f9f640 1 -- 172.21.15.62:0/1143100454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39241a4ff0 con 0x7f3924103660 2024-09-11T23:52:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.016+0000 7f3929f9f640 1 -- 172.21.15.62:0/1143100454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3924103120 con 0x7f3924103660 2024-09-11T23:52:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.017+0000 7f3920ff9640 1 -- 172.21.15.62:0/1143100454 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f3918018020 con 0x7f3924103660 2024-09-11T23:52:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.018+0000 7f3920ff9640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f390c0a2d80 0x7f390c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.018+0000 7f39237fe640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f390c0a2d80 0x7f390c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.018+0000 7f3920ff9640 1 -- 172.21.15.62:0/1143100454 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f39180cf190 con 0x7f3924103660 2024-09-11T23:52:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.019+0000 7f39237fe640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f390c0a2d80 0x7f390c0a5240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3914009230 tx=0x7f39140058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.021+0000 7f3920ff9640 1 -- 172.21.15.62:0/1143100454 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f391801f020 con 0x7f3924103660 2024-09-11T23:52:23.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:23 smithi062 ceph-mon[30958]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-09-11T23:52:23.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.232+0000 7f3929f9f640 1 -- 172.21.15.62:0/1143100454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39241997d0 con 0x7f3924103660 2024-09-11T23:52:23.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.233+0000 7f3920ff9640 1 -- 172.21.15.62:0/1143100454 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f39241997d0 con 0x7f3924103660 2024-09-11T23:52:23.233 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:23.233 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"btime":"2024-09-11T23:52:04:422374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:04.372188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:52:23.233 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2024-09-11T23:52:23.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.234+0000 7f3929f9f640 1 -- 172.21.15.62:0/1143100454 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f390c0a2d80 msgr2=0x7f390c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:23.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.234+0000 7f3929f9f640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f390c0a2d80 0x7f390c0a5240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3914009230 tx=0x7f39140058d0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.235+0000 7f3929f9f640 1 -- 172.21.15.62:0/1143100454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924103660 msgr2=0x7f39241989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:23.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.235+0000 7f3929f9f640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924103660 0x7f39241989b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f391800b3f0 tx=0x7f391800b8b0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.235+0000 7f3929f9f640 1 -- 172.21.15.62:0/1143100454 shutdown_connections 2024-09-11T23:52:23.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.235+0000 7f3929f9f640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f390c0a2d80 0x7f390c0a5240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.235+0000 7f3929f9f640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392410c170 0x7f392419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.235+0000 7f3929f9f640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924103660 0x7f39241989b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.235+0000 7f3929f9f640 1 --2- 172.21.15.62:0/1143100454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3924102d40 0x7f3924198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:23.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.235+0000 7f3929f9f640 1 -- 172.21.15.62:0/1143100454 >> 172.21.15.62:0/1143100454 conn(0x7f39240fc7a0 msgr2=0x7f39240fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:23.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.235+0000 7f3929f9f640 1 -- 172.21.15.62:0/1143100454 shutdown_connections 2024-09-11T23:52:23.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:23.235+0000 7f3929f9f640 1 -- 172.21.15.62:0/1143100454 wait complete. 2024-09-11T23:52:23.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:23 smithi120 ceph-mon[35182]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-09-11T23:52:23.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:23 smithi083 ceph-mon[34805]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-09-11T23:52:23.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:52:24.159 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:24.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:24 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1143100454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:24.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:24 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1143100454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:24.543 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:24 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1143100454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/4254745407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee4101dd0 msgr2=0x7f5ee41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 --2- 172.21.15.62:0/4254745407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee4101dd0 0x7f5ee41021b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009ca0 tx=0x7f5ed402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/4254745407 shutdown_connections 2024-09-11T23:52:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 --2- 172.21.15.62:0/4254745407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee4106f00 0x7f5ee410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 --2- 172.21.15.62:0/4254745407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ee4102780 0x7f5ee41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 --2- 172.21.15.62:0/4254745407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee4101dd0 0x7f5ee41021b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/4254745407 >> 172.21.15.62:0/4254745407 conn(0x7f5ee40fd8b0 msgr2=0x7f5ee40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/4254745407 shutdown_connections 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/4254745407 wait complete. 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 Processor -- start 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 -- start start 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.664+0000 7f5eeb1f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee4101dd0 0x7f5ee419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5eeb1f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee4102780 0x7f5ee419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5eeb1f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ee4106f00 0x7f5ee419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5eeb1f7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4111f60 con 0x7f5ee4101dd0 2024-09-11T23:52:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5eeb1f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4111de0 con 0x7f5ee4102780 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5eeb1f7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee41120e0 con 0x7f5ee4106f00 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee8f93640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee4101dd0 0x7f5ee419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee8f93640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee4101dd0 0x7f5ee419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47318/0 (socket says 172.21.15.62:47318) 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee8f93640 1 -- 172.21.15.62:0/3870655108 learned_addr learned my addr 172.21.15.62:0/3870655108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee3fff640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee4102780 0x7f5ee419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee9794640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ee4106f00 0x7f5ee419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee8f93640 1 -- 172.21.15.62:0/3870655108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ee4106f00 msgr2=0x7f5ee419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee8f93640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ee4106f00 0x7f5ee419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee8f93640 1 -- 172.21.15.62:0/3870655108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee4102780 msgr2=0x7f5ee419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee8f93640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee4102780 0x7f5ee419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee8f93640 1 -- 172.21.15.62:0/3870655108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee419f630 con 0x7f5ee4101dd0 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee8f93640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee4101dd0 0x7f5ee419a680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5ed402ade0 tx=0x7f5ed40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee1ffb640 1 -- 172.21.15.62:0/3870655108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed402fa20 con 0x7f5ee4101dd0 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/3870655108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee419f8c0 con 0x7f5ee4101dd0 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee1ffb640 1 -- 172.21.15.62:0/3870655108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ed402fbc0 con 0x7f5ee4101dd0 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.665+0000 7f5ee1ffb640 1 -- 172.21.15.62:0/3870655108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40466e0 con 0x7f5ee4101dd0 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.666+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/3870655108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee41a7200 con 0x7f5ee4101dd0 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.667+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/3870655108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee41021b0 con 0x7f5ee4101dd0 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.668+0000 7f5ee1ffb640 1 -- 172.21.15.62:0/3870655108 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f5ed4046940 con 0x7f5ee4101dd0 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.669+0000 7f5ee1ffb640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5ed00a2d80 0x7f5ed00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.669+0000 7f5ee1ffb640 1 -- 172.21.15.62:0/3870655108 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f5ed40f0b20 con 0x7f5ee4101dd0 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.669+0000 7f5ee3fff640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5ed00a2d80 0x7f5ed00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.669+0000 7f5ee3fff640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5ed00a2d80 0x7f5ed00a5240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40ff780 tx=0x7f5ecc005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.672+0000 7f5ee1ffb640 1 -- 172.21.15.62:0/3870655108 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ed40bd710 con 0x7f5ee4101dd0 2024-09-11T23:52:24.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.852+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/3870655108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ee41a73d0 con 0x7f5ee4101dd0 2024-09-11T23:52:24.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.853+0000 7f5ee1ffb640 1 -- 172.21.15.62:0/3870655108 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22995 (secure 0 0 0) 0x7f5ed40bd010 con 0x7f5ee4101dd0 2024-09-11T23:52:24.853 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:24.853 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":84,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-11T23:52:05.426157+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/518295197":"2024-09-12T23:52:05.426127+0000","172.21.15.62:6801/3106394779":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/908998551":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/133064408":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6800/3106394779":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2894445674":"2024-09-12T23:52:05.426127+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:52:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.855+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/3870655108 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5ed00a2d80 msgr2=0x7f5ed00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.855+0000 7f5eeb1f7640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5ed00a2d80 0x7f5ed00a5240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40ff780 tx=0x7f5ecc005f90 comp rx=0 tx=0).stop 2024-09-11T23:52:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.856+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/3870655108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee4101dd0 msgr2=0x7f5ee419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.856+0000 7f5eeb1f7640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee4101dd0 0x7f5ee419a680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5ed402ade0 tx=0x7f5ed40319f0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.856+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/3870655108 shutdown_connections 2024-09-11T23:52:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.856+0000 7f5eeb1f7640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5ed00a2d80 0x7f5ed00a5240 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.856+0000 7f5eeb1f7640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ee4106f00 0x7f5ee419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.856+0000 7f5eeb1f7640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee4102780 0x7f5ee419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.856+0000 7f5eeb1f7640 1 --2- 172.21.15.62:0/3870655108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee4101dd0 0x7f5ee419a680 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.856+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/3870655108 >> 172.21.15.62:0/3870655108 conn(0x7f5ee40fd8b0 msgr2=0x7f5ee40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.856+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/3870655108 shutdown_connections 2024-09-11T23:52:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:24.856+0000 7f5eeb1f7640 1 -- 172.21.15.62:0/3870655108 wait complete. 2024-09-11T23:52:25.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:25 smithi083 ceph-mon[34805]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-09-11T23:52:25.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:25 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3870655108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:52:25.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:25 smithi062 ceph-mon[30958]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-09-11T23:52:25.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:25 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3870655108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:52:25.494 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:25 smithi120 ceph-mon[35182]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-09-11T23:52:25.494 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:25 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3870655108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:52:25.506 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-11T23:52:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:26.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:26 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:52:26.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:26 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:52:26.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:26 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:52:26.960 INFO:teuthology.orchestra.run.smithi062.stdout:{"run_dir":"/var/run/ceph"} 2024-09-11T23:52:26.961 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-11T23:52:26.961 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-11T23:52:26.961 DEBUG:tasks.quiescer:Yielding 2024-09-11T23:52:26.961 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T23:52:26.972 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:52:26.972 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi062.front.sepia.ceph.com 2024-09-11T23:52:26.972 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-11T23:52:26.973 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:52:27.013 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:27.028 INFO:teuthology.orchestra.run.smithi062.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-11T23:52:27.028 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:3300,172.21.15.83:3300,172.21.15.120:3300:/volumes/qa/sv_1/324e6219-dcd5-475c-bde4-eff522131966 340G 0 340G 0% /tmp/cephfs 2024-09-11T23:52:27.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-11T23:52:27.124 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-11T23:52:27.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-11T23:52:27.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:27 smithi062 ceph-mon[30958]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-11T23:52:27.244 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:27 smithi083 ceph-mon[34805]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-11T23:52:27.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:27 smithi120 ceph-mon[35182]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-11T23:52:27.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.733+0000 7f509d61a640 1 -- 172.21.15.62:0/939166586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5098106a90 msgr2=0x7f5098106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:27.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.733+0000 7f509d61a640 1 --2- 172.21.15.62:0/939166586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5098106a90 0x7f5098106e70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5080009930 tx=0x7f508002a420 comp rx=0 tx=0).stop 2024-09-11T23:52:27.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.733+0000 7f509d61a640 1 -- 172.21.15.62:0/939166586 shutdown_connections 2024-09-11T23:52:27.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.733+0000 7f509d61a640 1 --2- 172.21.15.62:0/939166586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5098101430 0x7f509810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:27.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.733+0000 7f509d61a640 1 --2- 172.21.15.62:0/939166586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5098100a90 0x7f5098100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:27.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.733+0000 7f509d61a640 1 --2- 172.21.15.62:0/939166586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5098106a90 0x7f5098106e70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:27.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.733+0000 7f509d61a640 1 -- 172.21.15.62:0/939166586 >> 172.21.15.62:0/939166586 conn(0x7f50980fc7d0 msgr2=0x7f50980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:27.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.733+0000 7f509d61a640 1 -- 172.21.15.62:0/939166586 shutdown_connections 2024-09-11T23:52:27.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.733+0000 7f509d61a640 1 -- 172.21.15.62:0/939166586 wait complete. 2024-09-11T23:52:27.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.733+0000 7f509d61a640 1 Processor -- start 2024-09-11T23:52:27.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f509d61a640 1 -- start start 2024-09-11T23:52:27.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f509d61a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5098100a90 0x7f5098198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:27.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f509d61a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5098101430 0x7f50981989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:27.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f509d61a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5098106a90 0x7f509819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:27.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f509d61a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5098110e90 con 0x7f5098106a90 2024-09-11T23:52:27.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f509d61a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5098110d10 con 0x7f5098100a90 2024-09-11T23:52:27.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f509d61a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5098111010 con 0x7f5098101430 2024-09-11T23:52:27.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f50977fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5098106a90 0x7f509819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:27.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f50967fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5098101430 0x7f50981989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:27.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f50977fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5098106a90 0x7f509819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47338/0 (socket says 172.21.15.62:47338) 2024-09-11T23:52:27.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f50977fe640 1 -- 172.21.15.62:0/1921432502 learned_addr learned my addr 172.21.15.62:0/1921432502 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:27.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.734+0000 7f5096ffd640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5098100a90 0x7f5098198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:27.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f50977fe640 1 -- 172.21.15.62:0/1921432502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5098101430 msgr2=0x7f50981989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:27.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f50977fe640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5098101430 0x7f50981989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:27.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f50977fe640 1 -- 172.21.15.62:0/1921432502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5098100a90 msgr2=0x7f5098198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:27.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f50977fe640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5098100a90 0x7f5098198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:27.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f50977fe640 1 -- 172.21.15.62:0/1921432502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509819d420 con 0x7f5098106a90 2024-09-11T23:52:27.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f5096ffd640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5098100a90 0x7f5098198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:27.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f50967fc640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5098101430 0x7f50981989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:27.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f50977fe640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5098106a90 0x7f509819cd40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f509000eaa0 tx=0x7f509000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:27.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f508ffff640 1 -- 172.21.15.62:0/1921432502 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509000cdb0 con 0x7f5098106a90 2024-09-11T23:52:27.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f509d61a640 1 -- 172.21.15.62:0/1921432502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509819d710 con 0x7f5098106a90 2024-09-11T23:52:27.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f508ffff640 1 -- 172.21.15.62:0/1921432502 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5090005020 con 0x7f5098106a90 2024-09-11T23:52:27.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f508ffff640 1 -- 172.21.15.62:0/1921432502 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50900116d0 con 0x7f5098106a90 2024-09-11T23:52:27.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.735+0000 7f509d61a640 1 -- 172.21.15.62:0/1921432502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5098102590 con 0x7f5098106a90 2024-09-11T23:52:27.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.736+0000 7f509d61a640 1 -- 172.21.15.62:0/1921432502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5098100ef0 con 0x7f5098106a90 2024-09-11T23:52:27.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.738+0000 7f508ffff640 1 -- 172.21.15.62:0/1921432502 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f50900054d0 con 0x7f5098106a90 2024-09-11T23:52:27.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.738+0000 7f508ffff640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f50840a2d80 0x7f50840a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:27.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.738+0000 7f508ffff640 1 -- 172.21.15.62:0/1921432502 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f50900cf6c0 con 0x7f5098106a90 2024-09-11T23:52:27.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.738+0000 7f5096ffd640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f50840a2d80 0x7f50840a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:27.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.739+0000 7f5096ffd640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f50840a2d80 0x7f50840a5240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5080009900 tx=0x7f50800058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:27.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.743+0000 7f508ffff640 1 -- 172.21.15.62:0/1921432502 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5090016030 con 0x7f5098106a90 2024-09-11T23:52:27.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.956+0000 7f509d61a640 1 -- 172.21.15.62:0/1921432502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5098102910 con 0x7f5098106a90 2024-09-11T23:52:27.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.957+0000 7f508ffff640 1 -- 172.21.15.62:0/1921432502 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f509009bb50 con 0x7f5098106a90 2024-09-11T23:52:27.957 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:27.957 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"btime":"2024-09-11T23:52:04:422374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:04.372188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:52:27.958 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2024-09-11T23:52:27.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.959+0000 7f509d61a640 1 -- 172.21.15.62:0/1921432502 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f50840a2d80 msgr2=0x7f50840a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:27.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.959+0000 7f509d61a640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f50840a2d80 0x7f50840a5240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5080009900 tx=0x7f50800058d0 comp rx=0 tx=0).stop 2024-09-11T23:52:27.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.959+0000 7f509d61a640 1 -- 172.21.15.62:0/1921432502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5098106a90 msgr2=0x7f509819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:27.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.959+0000 7f509d61a640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5098106a90 0x7f509819cd40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f509000eaa0 tx=0x7f509000ef60 comp rx=0 tx=0).stop 2024-09-11T23:52:27.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.959+0000 7f509d61a640 1 -- 172.21.15.62:0/1921432502 shutdown_connections 2024-09-11T23:52:27.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.959+0000 7f509d61a640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f50840a2d80 0x7f50840a5240 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:27.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.959+0000 7f509d61a640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5098106a90 0x7f509819cd40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:27.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.959+0000 7f509d61a640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5098101430 0x7f50981989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:27.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.959+0000 7f509d61a640 1 --2- 172.21.15.62:0/1921432502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5098100a90 0x7f5098198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:27.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.959+0000 7f509d61a640 1 -- 172.21.15.62:0/1921432502 >> 172.21.15.62:0/1921432502 conn(0x7f50980fc7d0 msgr2=0x7f509806a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:27.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.960+0000 7f509d61a640 1 -- 172.21.15.62:0/1921432502 shutdown_connections 2024-09-11T23:52:27.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:27.960+0000 7f509d61a640 1 -- 172.21.15.62:0/1921432502 wait complete. 2024-09-11T23:52:28.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:28 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1921432502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:28.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:28 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1921432502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:28.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:28 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1921432502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:29.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:29 smithi062 ceph-mon[30958]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-11T23:52:29.348 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-11T23:52:29.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:29 smithi083 ceph-mon[34805]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-11T23:52:29.419 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:50:11.406773+0000', 'modified': '2024-09-11T23:52:04.372188+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24484, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6835/3040146550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 3040146550}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3040146550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6837/337493680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 337493680}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 337493680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/2559115046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 2559115046}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 2559115046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.120:6837/1796168792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 1796168792}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 1796168792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.120:6839/2283411896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 2283411896}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 2283411896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6839/2530997346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 2530997346}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 2530997346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24488, 24484, 24469]} 2024-09-11T23:52:29.419 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-11T23:52:29.419 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T23:52:29.430 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-11T23:52:29.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-11T23:52:29.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-11T23:52:29.592 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/: 2024-09-11T23:52:29.592 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:29.593 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 . 2024-09-11T23:52:29.593 INFO:teuthology.orchestra.run.smithi062.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 .. 2024-09-11T23:52:29.593 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 data 2024-09-11T23:52:29.593 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:29.593 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data: 2024-09-11T23:52:29.593 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:29.593 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:29.594 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 .. 2024-09-11T23:52:29.595 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-11T23:52:29.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:29 smithi120 ceph-mon[35182]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-11T23:52:29.686 INFO:teuthology.orchestra.run.smithi062.stdout:[Service] 2024-09-11T23:52:29.686 INFO:teuthology.orchestra.run.smithi062.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-11T23:52:29.686 INFO:teuthology.orchestra.run.smithi062.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-11T23:52:29.688 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-11T23:52:29.694 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:29.783 INFO:teuthology.orchestra.run.smithi062.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-11T23:52:30.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.279+0000 7f407d4ca640 1 -- 172.21.15.62:0/147524700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40781082a0 msgr2=0x7f407810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:30.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.279+0000 7f407d4ca640 1 --2- 172.21.15.62:0/147524700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40781082a0 0x7f407810ec40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4070009930 tx=0x7f407002a420 comp rx=0 tx=0).stop 2024-09-11T23:52:30.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.279+0000 7f407d4ca640 1 -- 172.21.15.62:0/147524700 shutdown_connections 2024-09-11T23:52:30.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.279+0000 7f407d4ca640 1 --2- 172.21.15.62:0/147524700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40781082a0 0x7f407810ec40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.279+0000 7f407d4ca640 1 --2- 172.21.15.62:0/147524700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40781036e0 0x7f4078107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.279+0000 7f407d4ca640 1 --2- 172.21.15.62:0/147524700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4078102d30 0x7f4078103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.279+0000 7f407d4ca640 1 -- 172.21.15.62:0/147524700 >> 172.21.15.62:0/147524700 conn(0x7f40780fc7d0 msgr2=0x7f40780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.279+0000 7f407d4ca640 1 -- 172.21.15.62:0/147524700 shutdown_connections 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.279+0000 7f407d4ca640 1 -- 172.21.15.62:0/147524700 wait complete. 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.280+0000 7f407d4ca640 1 Processor -- start 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.280+0000 7f407d4ca640 1 -- start start 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.280+0000 7f407d4ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4078102d30 0x7f40781a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.280+0000 7f407d4ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40781036e0 0x7f40781a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.280+0000 7f407d4ca640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40781082a0 0x7f407819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.280+0000 7f407d4ca640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f40781120a0 con 0x7f40781082a0 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.280+0000 7f407d4ca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4078111f20 con 0x7f4078102d30 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.280+0000 7f407d4ca640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4078112220 con 0x7f40781036e0 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.280+0000 7f4076ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4078102d30 0x7f40781a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f4076ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4078102d30 0x7f40781a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35166/0 (socket says 172.21.15.62:35166) 2024-09-11T23:52:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f4076ffd640 1 -- 172.21.15.62:0/2070206406 learned_addr learned my addr 172.21.15.62:0/2070206406 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f40777fe640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40781082a0 0x7f407819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f40767fc640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40781036e0 0x7f40781a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f4076ffd640 1 -- 172.21.15.62:0/2070206406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40781036e0 msgr2=0x7f40781a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f4076ffd640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40781036e0 0x7f40781a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f4076ffd640 1 -- 172.21.15.62:0/2070206406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40781082a0 msgr2=0x7f407819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f4076ffd640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40781082a0 0x7f407819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f4076ffd640 1 -- 172.21.15.62:0/2070206406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407819ace0 con 0x7f4078102d30 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f40767fc640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40781036e0 0x7f40781a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f4076ffd640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4078102d30 0x7f40781a03f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4068009d50 tx=0x7f406800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f406ffff640 1 -- 172.21.15.62:0/2070206406 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068003b70 con 0x7f4078102d30 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f406ffff640 1 -- 172.21.15.62:0/2070206406 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4068003d10 con 0x7f4078102d30 2024-09-11T23:52:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f406ffff640 1 -- 172.21.15.62:0/2070206406 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068010a20 con 0x7f4078102d30 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f407d4ca640 1 -- 172.21.15.62:0/2070206406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407819afd0 con 0x7f4078102d30 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.281+0000 7f407d4ca640 1 -- 172.21.15.62:0/2070206406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40781a72c0 con 0x7f4078102d30 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f406ffff640 1 -- 172.21.15.62:0/2070206406 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f4068003ec0 con 0x7f4078102d30 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f407f8a3740 1 Processor -- start 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f407f8a3740 1 -- start start 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef9779d0 0x55ccef977db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef982670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef982bb0 0x55ccef984fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f407f8a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ccef8de090 con 0x55ccef9782f0 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f407f8a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ccef885a70 con 0x55ccef982bb0 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f407f8a3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ccef8da050 con 0x55ccef9779d0 2024-09-11T23:52:30.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f40767fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef982bb0 0x55ccef984fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f4076ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef982670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.284+0000 7f40767fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef982bb0 0x55ccef984fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35176/0 (socket says 172.21.15.62:35176) 2024-09-11T23:52:30.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f40767fc640 1 -- 172.21.15.62:0/2791988813 learned_addr learned my addr 172.21.15.62:0/2791988813 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:30.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f40777fe640 1 --2- 172.21.15.62:0/2791988813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef9779d0 0x55ccef977db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f406ffff640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f40640a2e00 0x7f40640a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f40767fc640 1 -- 172.21.15.62:0/2791988813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef9779d0 msgr2=0x55ccef977db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:30.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f40767fc640 1 --2- 172.21.15.62:0/2791988813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef9779d0 0x55ccef977db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f40767fc640 1 -- 172.21.15.62:0/2791988813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 msgr2=0x55ccef982670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:30.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f40767fc640 1 --2- 172.21.15.62:0/2791988813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef982670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f40767fc640 1 -- 172.21.15.62:0/2791988813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccef892130 con 0x55ccef982bb0 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f40777fe640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f40640a2e00 0x7f40640a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f406ffff640 1 -- 172.21.15.62:0/2070206406 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f40680cfc80 con 0x7f4078102d30 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f40767fc640 1 --2- 172.21.15.62:0/2791988813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef982bb0 0x55ccef984fa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f406000bc30 tx=0x7f406000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=505d1e20e156f192 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f406d7fa640 1 -- 172.21.15.62:0/2791988813 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060015070 con 0x55ccef982bb0 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f406d7fa640 1 -- 172.21.15.62:0/2791988813 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4060005500 con 0x55ccef982bb0 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.285+0000 7f406d7fa640 1 -- 172.21.15.62:0/2791988813 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40600116a0 con 0x55ccef982bb0 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.286+0000 7f407f8a3740 1 -- 172.21.15.62:0/2791988813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef982bb0 msgr2=0x55ccef984fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.286+0000 7f407f8a3740 1 --2- 172.21.15.62:0/2791988813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef982bb0 0x55ccef984fa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f406000bc30 tx=0x7f406000f730 comp rx=0 tx=0).stop 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.286+0000 7f40777fe640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f40640a2e00 0x7f40640a52c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4070006010 tx=0x7f407002a420 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.286+0000 7f407f8a3740 1 -- 172.21.15.62:0/2791988813 shutdown_connections 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.286+0000 7f407f8a3740 1 --2- 172.21.15.62:0/2791988813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef982bb0 0x55ccef984fa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.286+0000 7f407f8a3740 1 --2- 172.21.15.62:0/2791988813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef982670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.286+0000 7f407f8a3740 1 --2- 172.21.15.62:0/2791988813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef9779d0 0x55ccef977db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.286+0000 7f407f8a3740 1 -- 172.21.15.62:0/2791988813 >> 172.21.15.62:0/2791988813 conn(0x55ccef973f60 msgr2=0x55ccef974340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.286+0000 7f407f8a3740 1 -- 172.21.15.62:0/2791988813 shutdown_connections 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.286+0000 7f407f8a3740 1 -- 172.21.15.62:0/2791988813 wait complete. 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f407f8a3740 1 -- start start 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f407f8a3740 1 Processor -- start 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef982480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef97f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef97fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f407f8a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ccef985780 con 0x55ccef9782f0 2024-09-11T23:52:30.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f407f8a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ccef898f20 con 0x55ccef9779d0 2024-09-11T23:52:30.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f407f8a3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ccef985900 con 0x55ccef982bb0 2024-09-11T23:52:30.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f407f8a3740 10 client.0 _ll_register_callbacks cb 0x55ccef862030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:30.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f40767fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef97f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f4076ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef982480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f40767fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef97f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47366/0 (socket says 172.21.15.62:47366) 2024-09-11T23:52:30.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f40767fc640 1 -- 172.21.15.62:0/1265666926 learned_addr learned my addr 172.21.15.62:0/1265666926 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:30.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f40777fe640 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef97fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f40767fc640 1 -- 172.21.15.62:0/1265666926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 msgr2=0x55ccef97fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:30.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f40767fc640 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef97fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f40767fc640 1 -- 172.21.15.62:0/1265666926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 msgr2=0x55ccef982480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:30.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f40767fc640 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef982480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f40767fc640 1 -- 172.21.15.62:0/1265666926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccef64f2b0 con 0x55ccef9782f0 2024-09-11T23:52:30.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.287+0000 7f4076ffd640 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef982480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:30.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.288+0000 7f40767fc640 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef97f7d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4060004c10 tx=0x7f4060004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed405a4082f4ded server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:30.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.288+0000 7f405e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:30.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.288+0000 7f405e7fc640 1 -- 172.21.15.62:0/1265666926 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060015070 con 0x55ccef9782f0 2024-09-11T23:52:30.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.288+0000 7f405e7fc640 1 -- 172.21.15.62:0/1265666926 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f406000bd20 con 0x55ccef9782f0 2024-09-11T23:52:30.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.288+0000 7f407f8a3740 1 -- 172.21.15.62:0/1265666926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4060005500 con 0x55ccef9782f0 2024-09-11T23:52:30.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.288+0000 7f405e7fc640 1 -- 172.21.15.62:0/1265666926 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4060004e10 con 0x55ccef9782f0 2024-09-11T23:52:30.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.288+0000 7f407f8a3740 10 client.15375 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:30.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.288+0000 7f407f8a3740 1 -- 172.21.15.62:0/1265666926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ccef9910d0 con 0x55ccef9782f0 2024-09-11T23:52:30.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.288+0000 7f405e7fc640 1 -- 172.21.15.62:0/1265666926 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4060005500 con 0x55ccef9782f0 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.288+0000 7f405e7fc640 1 -- 172.21.15.62:0/1265666926 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4060011c30 con 0x55ccef9782f0 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.289+0000 7f407f8a3740 10 client.15375 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.289+0000 7f407f8a3740 10 client.15375 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.289+0000 7f407f8a3740 20 client.15375 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.289+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ccef993570 0x55ccef9949e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.289+0000 7f407f8a3740 4 client.15375 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.289+0000 7f407f8a3740 1 -- 172.21.15.62:0/1265666926 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccef8eb600 con 0x55ccef993570 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.289+0000 7f4076ffd640 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ccef993570 0x55ccef9949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.289+0000 7f4076ffd640 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ccef993570 0x55ccef9949e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e462d8a91974d841 server_cookie=1029e4821c0db80 in_seq=0 out_seq=0 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.289+0000 7f405e7fc640 10 client.15375 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:52:30.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f405e7fc640 1 -- 172.21.15.62:0/1265666926 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccef8eb600 con 0x55ccef993570 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f405e7fc640 10 client.15375 handle_command_reply: tid=0 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f407f8a3740 1 client.15375 shutdown 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f407f8a3740 1 -- 172.21.15.62:0/1265666926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 msgr2=0x55ccef97f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef97f7d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4060004c10 tx=0x7f4060004a30 comp rx=0 tx=0).stop 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f407f8a3740 1 -- 172.21.15.62:0/1265666926 shutdown_connections 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ccef993570 0x55ccef9949e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef97fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef97f7d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1265666926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef982480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f405e7fc640 0 client.15375 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.290+0000 7f407f8a3740 1 -- 172.21.15.62:0/1265666926 >> 172.21.15.62:0/1265666926 conn(0x55ccef973f60 msgr2=0x55ccef979c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:30.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.291+0000 7f407f8a3740 1 -- 172.21.15.62:0/1265666926 shutdown_connections 2024-09-11T23:52:30.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.291+0000 7f407f8a3740 1 -- 172.21.15.62:0/1265666926 wait complete. 2024-09-11T23:52:30.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.291+0000 7f407f8a3740 20 client.15375 trim_cache size 0 max 16384 2024-09-11T23:52:30.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.309+0000 7f407f8a3740 1 Processor -- start 2024-09-11T23:52:30.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.309+0000 7f407f8a3740 1 -- start start 2024-09-11T23:52:30.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9779d0 0x55ccef993c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9782f0 0x55ccef992590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef973000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f407f8a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ccef992d20 con 0x55ccef9779d0 2024-09-11T23:52:30.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f407f8a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ccef9933f0 con 0x55ccef9782f0 2024-09-11T23:52:30.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f407f8a3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ccef993ac0 con 0x55ccef982bb0 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f4076ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9779d0 0x55ccef993c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f4076ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9779d0 0x55ccef993c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47374/0 (socket says 172.21.15.62:47374) 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f4076ffd640 1 -- 172.21.15.62:0/1837760790 learned_addr learned my addr 172.21.15.62:0/1837760790 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f40767fc640 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9782f0 0x55ccef992590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f40777fe640 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef973000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f4076ffd640 1 -- 172.21.15.62:0/1837760790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 msgr2=0x55ccef973000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f4076ffd640 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef973000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f4076ffd640 1 -- 172.21.15.62:0/1837760790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9782f0 msgr2=0x55ccef992590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f4076ffd640 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9782f0 0x55ccef992590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f4076ffd640 1 -- 172.21.15.62:0/1837760790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccef978120 con 0x55ccef9779d0 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f40777fe640 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef973000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:52:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.310+0000 7f40767fc640 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9782f0 0x55ccef992590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:30.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.311+0000 7f4076ffd640 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9779d0 0x55ccef993c40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f40680984c0 tx=0x7f40680d3700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d6bbb272226767 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:30.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.311+0000 7f406cff9640 1 -- 172.21.15.62:0/1837760790 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406809bba0 con 0x55ccef9779d0 2024-09-11T23:52:30.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.311+0000 7f406cff9640 1 -- 172.21.15.62:0/1837760790 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4068009820 con 0x55ccef9779d0 2024-09-11T23:52:30.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.311+0000 7f406cff9640 1 -- 172.21.15.62:0/1837760790 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406809b9c0 con 0x55ccef9779d0 2024-09-11T23:52:30.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 -- 172.21.15.62:0/1837760790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9779d0 msgr2=0x55ccef993c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:30.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9779d0 0x55ccef993c40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f40680984c0 tx=0x7f40680d3700 comp rx=0 tx=0).stop 2024-09-11T23:52:30.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 -- 172.21.15.62:0/1837760790 shutdown_connections 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef973000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9782f0 0x55ccef992590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1837760790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9779d0 0x55ccef993c40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 -- 172.21.15.62:0/1837760790 >> 172.21.15.62:0/1837760790 conn(0x55ccef973f60 msgr2=0x55ccef982480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 -- 172.21.15.62:0/1837760790 shutdown_connections 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 -- 172.21.15.62:0/1837760790 wait complete. 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 -- start start 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.312+0000 7f407f8a3740 1 Processor -- start 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef99ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef99b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f407f8a3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef9961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f407f8a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ccef994860 con 0x55ccef9782f0 2024-09-11T23:52:30.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f407f8a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ccef994190 con 0x55ccef9779d0 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f407f8a3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ccef994f30 con 0x55ccef982bb0 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f407f8a3740 10 client.0 _ll_register_callbacks cb 0x7f40680cd1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f4076ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef99ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f4076ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef99ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35220/0 (socket says 172.21.15.62:35220) 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f4076ffd640 1 -- 172.21.15.62:0/1245408293 learned_addr learned my addr 172.21.15.62:0/1245408293 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f40767fc640 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef99b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f40777fe640 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef9961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f4076ffd640 1 -- 172.21.15.62:0/1245408293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 msgr2=0x55ccef9961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f4076ffd640 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef9961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f4076ffd640 1 -- 172.21.15.62:0/1245408293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 msgr2=0x55ccef99b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f4076ffd640 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef99b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f4076ffd640 1 -- 172.21.15.62:0/1245408293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccef996930 con 0x55ccef9779d0 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f40767fc640 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef99b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f4076ffd640 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef99ab60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40680a47a0 tx=0x7f40680cd2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7c17b0bffe87554 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f405e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.313+0000 7f405e7fc640 1 -- 172.21.15.62:0/1245408293 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406809b9c0 con 0x55ccef9779d0 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f405e7fc640 1 -- 172.21.15.62:0/1245408293 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f406809a360 con 0x55ccef9779d0 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f407f8a3740 1 -- 172.21.15.62:0/1245408293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4068009820 con 0x55ccef9779d0 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f405e7fc640 1 -- 172.21.15.62:0/1245408293 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f406809bba0 con 0x55ccef9779d0 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f407f8a3740 10 client.25067 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f407f8a3740 1 -- 172.21.15.62:0/1245408293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ccef99a230 con 0x55ccef9779d0 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f405e7fc640 1 -- 172.21.15.62:0/1245408293 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4068009820 con 0x55ccef9779d0 2024-09-11T23:52:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f405e7fc640 1 -- 172.21.15.62:0/1245408293 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40680b7070 con 0x55ccef9779d0 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f407f8a3740 10 client.25067 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f407f8a3740 10 client.25067 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f407f8a3740 20 client.25067 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ccefa01cd0 0x55ccefa009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f407f8a3740 4 client.25067 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f407f8a3740 1 -- 172.21.15.62:0/1245408293 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "status"}) -- 0x55ccef8e5b10 con 0x55ccefa01cd0 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.314+0000 7f40767fc640 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ccefa01cd0 0x55ccefa009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f40767fc640 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ccefa01cd0 0x55ccefa009e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3c1edd032c33559 server_cookie=b427cb40f085c9c6 in_seq=0 out_seq=0 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f405e7fc640 10 client.25067 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f405e7fc640 1 -- 172.21.15.62:0/1245408293 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ccef8e5b10 con 0x55ccefa01cd0 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f405e7fc640 10 client.25067 handle_command_reply: tid=0 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 client.25067 shutdown 2024-09-11T23:52:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 -- 172.21.15.62:0/1245408293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 msgr2=0x55ccef99ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef99ab60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40680a47a0 tx=0x7f40680cd2b0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 -- 172.21.15.62:0/1245408293 shutdown_connections 2024-09-11T23:52:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ccefa01cd0 0x55ccefa009e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccef982bb0 0x55ccef9961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccef9782f0 0x55ccef99b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f405e7fc640 0 client.25067 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:52:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 --2- 172.21.15.62:0/1245408293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ccef9779d0 0x55ccef99ab60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 -- 172.21.15.62:0/1245408293 >> 172.21.15.62:0/1245408293 conn(0x55ccef973f60 msgr2=0x55ccef992590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 -- 172.21.15.62:0/1245408293 shutdown_connections 2024-09-11T23:52:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.315+0000 7f407f8a3740 1 -- 172.21.15.62:0/1245408293 wait complete. 2024-09-11T23:52:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407f8a3740 20 client.25067 trim_cache size 0 max 16384 2024-09-11T23:52:30.332 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:52:30.332 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:52:30.332 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-09-11T23:52:30.332 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24469, 2024-09-11T23:52:30.332 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-09-11T23:52:30.333 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-09-11T23:52:30.333 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-09-11T23:52:30.333 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 130.45819861800001, 2024-09-11T23:52:30.333 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 25, 2024-09-11T23:52:30.333 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 84, 2024-09-11T23:52:30.333 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 83, 2024-09-11T23:52:30.333 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 163.52428595200001 2024-09-11T23:52:30.334 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:52:30.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 -- 172.21.15.62:0/2070206406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f40640a2e00 msgr2=0x7f40640a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f40640a2e00 0x7f40640a52c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4070006010 tx=0x7f407002a420 comp rx=0 tx=0).stop 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 -- 172.21.15.62:0/2070206406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4078102d30 msgr2=0x7f40781a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4078102d30 0x7f40781a03f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4068009d50 tx=0x7f406800c6a0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 -- 172.21.15.62:0/2070206406 shutdown_connections 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f40640a2e00 0x7f40640a52c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40781082a0 0x7f407819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40781036e0 0x7f40781a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 --2- 172.21.15.62:0/2070206406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4078102d30 0x7f40781a03f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 -- 172.21.15.62:0/2070206406 >> 172.21.15.62:0/2070206406 conn(0x7f40780fc7d0 msgr2=0x7f40780fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 -- 172.21.15.62:0/2070206406 shutdown_connections 2024-09-11T23:52:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:30.316+0000 7f407d4ca640 1 -- 172.21.15.62:0/2070206406 wait complete. 2024-09-11T23:52:31.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:52:31.377 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:31.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:31 smithi083 ceph-mon[34805]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:52:31.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:31 smithi062 ceph-mon[30958]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:52:31.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:31 smithi120 ceph-mon[35182]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:52:31.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.919+0000 7fb59f64a640 1 -- 172.21.15.62:0/3638975734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb598106f50 msgr2=0x7fb59810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:31.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.919+0000 7fb59f64a640 1 --2- 172.21.15.62:0/3638975734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb598106f50 0x7fb59810eaf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb590009cd0 tx=0x7fb59002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.919+0000 7fb59f64a640 1 -- 172.21.15.62:0/3638975734 shutdown_connections 2024-09-11T23:52:31.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.919+0000 7fb59f64a640 1 --2- 172.21.15.62:0/3638975734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb598106f50 0x7fb59810eaf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.919+0000 7fb59f64a640 1 --2- 172.21.15.62:0/3638975734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598100d00 0x7fb598106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.919+0000 7fb59f64a640 1 --2- 172.21.15.62:0/3638975734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb598100350 0x7fb598100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.919+0000 7fb59f64a640 1 -- 172.21.15.62:0/3638975734 >> 172.21.15.62:0/3638975734 conn(0x7fb598077210 msgr2=0x7fb5980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:31.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.919+0000 7fb59f64a640 1 -- 172.21.15.62:0/3638975734 shutdown_connections 2024-09-11T23:52:31.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.919+0000 7fb59f64a640 1 -- 172.21.15.62:0/3638975734 wait complete. 2024-09-11T23:52:31.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59f64a640 1 Processor -- start 2024-09-11T23:52:31.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59f64a640 1 -- start start 2024-09-11T23:52:31.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59f64a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb598100350 0x7fb598198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59f64a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb598100d00 0x7fb598198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59f64a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598106f50 0x7fb59819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59f64a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb598111fa0 con 0x7fb598100350 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59f64a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb598111e20 con 0x7fb598100d00 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59f64a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb598112120 con 0x7fb598106f50 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59d3e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb598100350 0x7fb598198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59d3e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb598100350 0x7fb598198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36136/0 (socket says 172.21.15.62:36136) 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59d3e6640 1 -- 172.21.15.62:0/461277435 learned_addr learned my addr 172.21.15.62:0/461277435 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb598100d00 0x7fb598198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59dbe7640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598106f50 0x7fb59819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59d3e6640 1 -- 172.21.15.62:0/461277435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598106f50 msgr2=0x7fb59819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598106f50 0x7fb59819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59d3e6640 1 -- 172.21.15.62:0/461277435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb598100d00 msgr2=0x7fb598198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb598100d00 0x7fb598198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59d3e6640 1 -- 172.21.15.62:0/461277435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59819d3f0 con 0x7fb598100350 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.920+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb598100d00 0x7fb598198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.921+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb598100350 0x7fb598198440 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb59400d950 tx=0x7fb59400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.921+0000 7fb58a7fc640 1 -- 172.21.15.62:0/461277435 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594017070 con 0x7fb598100350 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.921+0000 7fb59f64a640 1 -- 172.21.15.62:0/461277435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59819d6e0 con 0x7fb598100350 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.921+0000 7fb58a7fc640 1 -- 172.21.15.62:0/461277435 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb594009c30 con 0x7fb598100350 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.921+0000 7fb59f64a640 1 -- 172.21.15.62:0/461277435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5981a4fc0 con 0x7fb598100350 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.921+0000 7fb58a7fc640 1 -- 172.21.15.62:0/461277435 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594020060 con 0x7fb598100350 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.923+0000 7fb58a7fc640 1 -- 172.21.15.62:0/461277435 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fb5940109d0 con 0x7fb598100350 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.923+0000 7fb5a1a23740 1 Processor -- start 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb5a1a23740 1 -- start start 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a85adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a85b2f0 0x55a23a865670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a865bb0 0x55a23a867fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a23a7c1090 con 0x55a23a865bb0 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a23a768a70 con 0x55a23a85a9d0 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a23a7bd050 con 0x55a23a85b2f0 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59cbe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a865bb0 0x55a23a867fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59cbe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a865bb0 0x55a23a867fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36152/0 (socket says 172.21.15.62:36152) 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59cbe5640 1 -- 172.21.15.62:0/1990270627 learned_addr learned my addr 172.21.15.62:0/1990270627 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/1990270627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a85b2f0 0x55a23a865670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59dbe7640 1 --2- 172.21.15.62:0/1990270627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a85adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb58a7fc640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5840a2d80 0x7fb5840a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59cbe5640 1 -- 172.21.15.62:0/1990270627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a85b2f0 msgr2=0x55a23a865670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/1990270627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a85b2f0 0x55a23a865670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59cbe5640 1 -- 172.21.15.62:0/1990270627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 msgr2=0x55a23a85adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/1990270627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a85adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59cbe5640 1 -- 172.21.15.62:0/1990270627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a23a775130 con 0x55a23a865bb0 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.924+0000 7fb59dbe7640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5840a2d80 0x7fb5840a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.925+0000 7fb58a7fc640 1 -- 172.21.15.62:0/461277435 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb5940cf200 con 0x7fb598100350 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.925+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/1990270627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a865bb0 0x55a23a867fa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00aef0 tx=0x7fb58c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fc5ed8097db4bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.925+0000 7fb5837fe640 1 -- 172.21.15.62:0/1990270627 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c01d070 con 0x55a23a865bb0 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.925+0000 7fb5837fe640 1 -- 172.21.15.62:0/1990270627 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb58c010d20 con 0x55a23a865bb0 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.925+0000 7fb5837fe640 1 -- 172.21.15.62:0/1990270627 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c018e40 con 0x55a23a865bb0 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 -- 172.21.15.62:0/1990270627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a865bb0 msgr2=0x55a23a867fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/1990270627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a865bb0 0x55a23a867fa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00aef0 tx=0x7fb58c00f790 comp rx=0 tx=0).stop 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb59dbe7640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5840a2d80 0x7fb5840a5240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb590009a60 tx=0x7fb59003e040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 -- 172.21.15.62:0/1990270627 shutdown_connections 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/1990270627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a865bb0 0x55a23a867fa0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/1990270627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a85b2f0 0x55a23a865670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/1990270627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a85adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 -- 172.21.15.62:0/1990270627 >> 172.21.15.62:0/1990270627 conn(0x55a23a856f60 msgr2=0x55a23a857340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 -- 172.21.15.62:0/1990270627 shutdown_connections 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 -- 172.21.15.62:0/1990270627 wait complete. 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 -- start start 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.926+0000 7fb5a1a23740 1 Processor -- start 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a8653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a862720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a865bb0 0x55a23a862c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a23a868780 con 0x55a23a85b2f0 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a23a77bf20 con 0x55a23a85a9d0 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a23a868900 con 0x55a23a865bb0 2024-09-11T23:52:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb5a1a23740 10 client.0 _ll_register_callbacks cb 0x55a23a745030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59cbe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a862720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59cbe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a862720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36166/0 (socket says 172.21.15.62:36166) 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59cbe5640 1 -- 172.21.15.62:0/2368309033 learned_addr learned my addr 172.21.15.62:0/2368309033 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59dbe7640 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a865bb0 0x55a23a862c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a8653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59cbe5640 1 -- 172.21.15.62:0/2368309033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a865bb0 msgr2=0x55a23a862c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a865bb0 0x55a23a862c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59cbe5640 1 -- 172.21.15.62:0/2368309033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 msgr2=0x55a23a8653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a8653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59cbe5640 1 -- 172.21.15.62:0/2368309033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a23a5322b0 con 0x55a23a85b2f0 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a862720 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00aaf0 tx=0x7fb58c0186d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef6c3cc79890faf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb580ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb580ff9640 1 -- 172.21.15.62:0/2368309033 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c01d050 con 0x55a23a85b2f0 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb580ff9640 1 -- 172.21.15.62:0/2368309033 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb58c010a50 con 0x55a23a85b2f0 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.927+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2368309033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb58c010d20 con 0x55a23a85b2f0 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb580ff9640 1 -- 172.21.15.62:0/2368309033 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb58c032050 con 0x55a23a85b2f0 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb5a1a23740 10 client.15393 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2368309033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a23a874030 con 0x55a23a85b2f0 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb580ff9640 1 -- 172.21.15.62:0/2368309033 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb58c010d20 con 0x55a23a85b2f0 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb580ff9640 1 -- 172.21.15.62:0/2368309033 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb58c004a30 con 0x55a23a85b2f0 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb5a1a23740 10 client.15393 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb5a1a23740 10 client.15393 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb5a1a23740 20 client.15393 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a23a8762e0 0x55a23a877750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb5a1a23740 4 client.15393 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.928+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2368309033 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a23a7ce600 con 0x55a23a8762e0 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.929+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a23a8762e0 0x55a23a877750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.929+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a23a8762e0 0x55a23a877750 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3984398ac9ab817a server_cookie=ffd7a047cb5d0561 in_seq=0 out_seq=0 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.929+0000 7fb580ff9640 10 client.15393 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb580ff9640 1 -- 172.21.15.62:0/2368309033 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a23a7ce600 con 0x55a23a8762e0 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb580ff9640 10 client.15393 handle_command_reply: tid=0 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 client.15393 shutdown 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2368309033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 msgr2=0x55a23a862720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a862720 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00aaf0 tx=0x7fb58c0186d0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2368309033 shutdown_connections 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a23a8762e0 0x55a23a877750 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a865bb0 0x55a23a862c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a862720 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb580ff9640 0 client.15393 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:52:31.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2368309033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a8653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2368309033 >> 172.21.15.62:0/2368309033 conn(0x55a23a856f60 msgr2=0x55a23a85cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:31.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2368309033 shutdown_connections 2024-09-11T23:52:31.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2368309033 wait complete. 2024-09-11T23:52:31.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.930+0000 7fb5a1a23740 20 client.15393 trim_cache size 0 max 16384 2024-09-11T23:52:31.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.948+0000 7fb5a1a23740 1 Processor -- start 2024-09-11T23:52:31.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.948+0000 7fb5a1a23740 1 -- start start 2024-09-11T23:52:31.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.948+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a855ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.948+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a85f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.948+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a865bb0 0x55a23a85fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.948+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a23a875a90 con 0x55a23a85b2f0 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.948+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a23a876160 con 0x55a23a85a9d0 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.948+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a23a876830 con 0x55a23a865bb0 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59cbe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a85f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59cbe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a85f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36178/0 (socket says 172.21.15.62:36178) 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59cbe5640 1 -- 172.21.15.62:0/867026442 learned_addr learned my addr 172.21.15.62:0/867026442 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/867026442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a855ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59cbe5640 1 -- 172.21.15.62:0/867026442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a865bb0 msgr2=0x55a23a85fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59dbe7640 1 --2- 172.21.15.62:0/867026442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a865bb0 0x55a23a85fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/867026442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a865bb0 0x55a23a85fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59cbe5640 1 -- 172.21.15.62:0/867026442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 msgr2=0x55a23a855ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/867026442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a855ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59cbe5640 1 -- 172.21.15.62:0/867026442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a23a868300 con 0x55a23a85b2f0 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/867026442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a855ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/867026442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a85f980 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00f7d0 tx=0x7fb58c00a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23bbcacf6d382f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb582ffd640 1 -- 172.21.15.62:0/867026442 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c01d040 con 0x55a23a85b2f0 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb582ffd640 1 -- 172.21.15.62:0/867026442 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb58c029070 con 0x55a23a85b2f0 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.949+0000 7fb582ffd640 1 -- 172.21.15.62:0/867026442 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c024840 con 0x55a23a85b2f0 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 -- 172.21.15.62:0/867026442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 msgr2=0x55a23a85f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/867026442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a85f980 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00f7d0 tx=0x7fb58c00a940 comp rx=0 tx=0).stop 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 -- 172.21.15.62:0/867026442 shutdown_connections 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/867026442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a865bb0 0x55a23a85fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/867026442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85b2f0 0x55a23a85f980 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/867026442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a85a9d0 0x55a23a855ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 -- 172.21.15.62:0/867026442 >> 172.21.15.62:0/867026442 conn(0x55a23a856f60 msgr2=0x55a23a8557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 -- 172.21.15.62:0/867026442 shutdown_connections 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 -- 172.21.15.62:0/867026442 wait complete. 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 -- start start 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.950+0000 7fb5a1a23740 1 Processor -- start 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85a9d0 0x55a23a8801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a85b2f0 0x55a23a87b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb5a1a23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a865bb0 0x55a23a87ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a23a8775d0 con 0x55a23a85a9d0 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a23a876f00 con 0x55a23a865bb0 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb5a1a23740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a23a877ca0 con 0x55a23a85b2f0 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb5a1a23740 10 client.0 _ll_register_callbacks cb 0x7fb58c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb59d3e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85a9d0 0x55a23a8801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb59d3e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85a9d0 0x55a23a8801c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36180/0 (socket says 172.21.15.62:36180) 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb59d3e6640 1 -- 172.21.15.62:0/2300038624 learned_addr learned my addr 172.21.15.62:0/2300038624 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb59dbe7640 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a865bb0 0x55a23a87ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a85b2f0 0x55a23a87b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb59d3e6640 1 -- 172.21.15.62:0/2300038624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a85b2f0 msgr2=0x55a23a87b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a85b2f0 0x55a23a87b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb59d3e6640 1 -- 172.21.15.62:0/2300038624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a865bb0 msgr2=0x55a23a87ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a865bb0 0x55a23a87ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.951+0000 7fb59d3e6640 1 -- 172.21.15.62:0/2300038624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a23a87c260 con 0x55a23a85a9d0 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb59d3e6640 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85a9d0 0x55a23a8801c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb59409b390 tx=0x7fb59400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207ec69dc448a6a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb580ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb580ff9640 1 -- 172.21.15.62:0/2300038624 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5940b4060 con 0x55a23a85a9d0 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb580ff9640 1 -- 172.21.15.62:0/2300038624 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb5940ccb70 con 0x55a23a85a9d0 2024-09-11T23:52:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2300038624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb58c029070 con 0x55a23a85a9d0 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb580ff9640 1 -- 172.21.15.62:0/2300038624 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb59409aff0 con 0x55a23a85a9d0 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb5a1a23740 10 client.15399 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2300038624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a23a8e0ab0 con 0x55a23a85a9d0 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb580ff9640 1 -- 172.21.15.62:0/2300038624 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb58c029070 con 0x55a23a85a9d0 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb580ff9640 1 -- 172.21.15.62:0/2300038624 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb594023020 con 0x55a23a85a9d0 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb5a1a23740 10 client.15399 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb5a1a23740 10 client.15399 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb5a1a23740 20 client.15399 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a23a8e5610 0x55a23a8e4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb5a1a23740 4 client.15399 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.952+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2300038624 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "status"}) -- 0x55a23a7c8b10 con 0x55a23a8e5610 2024-09-11T23:52:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a23a8e5610 0x55a23a8e4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb59cbe5640 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a23a8e5610 0x55a23a8e4320 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b42aa1976506024a server_cookie=e6324ba8c744d677 in_seq=0 out_seq=0 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb580ff9640 10 client.15399 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb580ff9640 1 -- 172.21.15.62:0/2300038624 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a23a7c8b10 con 0x55a23a8e5610 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb580ff9640 10 client.15399 handle_command_reply: tid=0 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb5a1a23740 1 client.15399 shutdown 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2300038624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85a9d0 msgr2=0x55a23a8801c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85a9d0 0x55a23a8801c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb59409b390 tx=0x7fb59400bf60 comp rx=0 tx=0).stop 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2300038624 shutdown_connections 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a23a8e5610 0x55a23a8e4320 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a23a865bb0 0x55a23a87ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a23a85b2f0 0x55a23a87b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb580ff9640 0 client.15399 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:52:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb5a1a23740 1 --2- 172.21.15.62:0/2300038624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a23a85a9d0 0x55a23a8801c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.953+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2300038624 >> 172.21.15.62:0/2300038624 conn(0x55a23a856f60 msgr2=0x55a23a856000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2300038624 shutdown_connections 2024-09-11T23:52:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb5a1a23740 1 -- 172.21.15.62:0/2300038624 wait complete. 2024-09-11T23:52:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb5a1a23740 20 client.15399 trim_cache size 0 max 16384 2024-09-11T23:52:31.962 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:52:31.962 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:52:31.962 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-09-11T23:52:31.962 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24484, 2024-09-11T23:52:31.962 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-09-11T23:52:31.963 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-09-11T23:52:31.963 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-09-11T23:52:31.963 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 133.10127610699999, 2024-09-11T23:52:31.963 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 25, 2024-09-11T23:52:31.963 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 84, 2024-09-11T23:52:31.963 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 83, 2024-09-11T23:52:31.963 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 159.89084066800001 2024-09-11T23:52:31.963 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:52:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb59f64a640 1 -- 172.21.15.62:0/461277435 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5840a2d80 msgr2=0x7fb5840a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb59f64a640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5840a2d80 0x7fb5840a5240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb590009a60 tx=0x7fb59003e040 comp rx=0 tx=0).stop 2024-09-11T23:52:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb59f64a640 1 -- 172.21.15.62:0/461277435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb598100350 msgr2=0x7fb598198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb59f64a640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb598100350 0x7fb598198440 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb59400d950 tx=0x7fb59400de10 comp rx=0 tx=0).stop 2024-09-11T23:52:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb59f64a640 1 -- 172.21.15.62:0/461277435 shutdown_connections 2024-09-11T23:52:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb59f64a640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb5840a2d80 0x7fb5840a5240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb59f64a640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598106f50 0x7fb59819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb59f64a640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb598100d00 0x7fb598198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb59f64a640 1 --2- 172.21.15.62:0/461277435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb598100350 0x7fb598198440 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.954+0000 7fb59f64a640 1 -- 172.21.15.62:0/461277435 >> 172.21.15.62:0/461277435 conn(0x7fb598077210 msgr2=0x7fb598105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:31.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.955+0000 7fb59f64a640 1 -- 172.21.15.62:0/461277435 shutdown_connections 2024-09-11T23:52:31.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:31.955+0000 7fb59f64a640 1 -- 172.21.15.62:0/461277435 wait complete. 2024-09-11T23:52:32.685 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:52:32.944 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:33.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:33 smithi062 ceph-mon[30958]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:52:33.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:33 smithi083 ceph-mon[34805]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:52:33.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.427+0000 7fc80118c640 1 -- 172.21.15.62:0/1138428297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7fc102d40 msgr2=0x7fc7fc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:33.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.427+0000 7fc80118c640 1 --2- 172.21.15.62:0/1138428297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7fc102d40 0x7fc7fc103120 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009900 tx=0x7fc7ec02a480 comp rx=0 tx=0).stop 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.427+0000 7fc80118c640 1 -- 172.21.15.62:0/1138428297 shutdown_connections 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.427+0000 7fc80118c640 1 --2- 172.21.15.62:0/1138428297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc10c170 0x7fc7fc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.427+0000 7fc80118c640 1 --2- 172.21.15.62:0/1138428297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7fc103660 0x7fc7fc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.427+0000 7fc80118c640 1 --2- 172.21.15.62:0/1138428297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7fc102d40 0x7fc7fc103120 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.427+0000 7fc80118c640 1 -- 172.21.15.62:0/1138428297 >> 172.21.15.62:0/1138428297 conn(0x7fc7fc0fc7a0 msgr2=0x7fc7fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.427+0000 7fc80118c640 1 -- 172.21.15.62:0/1138428297 shutdown_connections 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.427+0000 7fc80118c640 1 -- 172.21.15.62:0/1138428297 wait complete. 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc80118c640 1 Processor -- start 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc80118c640 1 -- start start 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc80118c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc80118c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc103660 0x7fc7fc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc80118c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7fc10c170 0x7fc7fc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc80118c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc111f70 con 0x7fc7fc102d40 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc80118c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc111df0 con 0x7fc7fc10c170 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc80118c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1120f0 con 0x7fc7fc103660 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc7fad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc7fad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36192/0 (socket says 172.21.15.62:36192) 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc7fad9d640 1 -- 172.21.15.62:0/1324988933 learned_addr learned my addr 172.21.15.62:0/1324988933 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7fc10c170 0x7fc7fc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc7f3fff640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc103660 0x7fc7fc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc7fad9d640 1 -- 172.21.15.62:0/1324988933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc103660 msgr2=0x7fc7fc198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc103660 0x7fc7fc198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc7fad9d640 1 -- 172.21.15.62:0/1324988933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7fc10c170 msgr2=0x7fc7fc19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7fc10c170 0x7fc7fc19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.428+0000 7fc7fad9d640 1 -- 172.21.15.62:0/1324988933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7fc19d490 con 0x7fc7fc102d40 2024-09-11T23:52:33.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.429+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1984e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec006010 tx=0x7fc7ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.429+0000 7fc7f8d99640 1 -- 172.21.15.62:0/1324988933 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec03f030 con 0x7fc7fc102d40 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.429+0000 7fc80118c640 1 -- 172.21.15.62:0/1324988933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7fc19d720 con 0x7fc7fc102d40 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.429+0000 7fc7f8d99640 1 -- 172.21.15.62:0/1324988933 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ec003ce0 con 0x7fc7fc102d40 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.429+0000 7fc7f8d99640 1 -- 172.21.15.62:0/1324988933 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec035070 con 0x7fc7fc102d40 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.429+0000 7fc80118c640 1 -- 172.21.15.62:0/1324988933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7fc1a5060 con 0x7fc7fc102d40 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.431+0000 7fc7f8d99640 1 -- 172.21.15.62:0/1324988933 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fc7ec042030 con 0x7fc7fc102d40 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.431+0000 7fc803565740 1 Processor -- start 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.431+0000 7fc803565740 1 -- start start 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc803565740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 0x56475075cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc803565740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x564750767670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc803565740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750769fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc803565740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5647506c3090 con 0x564750767bb0 2024-09-11T23:52:33.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc803565740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56475066aa70 con 0x56475075c9d0 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc803565740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5647506bf050 con 0x56475075d2f0 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc7f8d99640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc7e80a2d80 0x7fc7e80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750769fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc7fb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 0x56475075cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750769fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36202/0 (socket says 172.21.15.62:36202) 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc7f3fff640 1 -- 172.21.15.62:0/347090694 learned_addr learned my addr 172.21.15.62:0/347090694 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc7e80a2d80 0x7fc7e80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/347090694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x564750767670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.432+0000 7fc7fb59e640 1 -- 172.21.15.62:0/347090694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 msgr2=0x564750767670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.433+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/347090694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x564750767670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.433+0000 7fc7fb59e640 1 -- 172.21.15.62:0/347090694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 msgr2=0x564750769fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.433+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/347090694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750769fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.433+0000 7fc7f8d99640 1 -- 172.21.15.62:0/1324988933 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc7ec0efbf0 con 0x7fc7fc102d40 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.433+0000 7fc7fb59e640 1 -- 172.21.15.62:0/347090694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564750677130 con 0x56475075c9d0 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.433+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/347090694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 0x56475075cdb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4005fd0 tx=0x7fc7f4010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61f3298fbeba2d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.433+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc7e80a2d80 0x7fc7e80a5240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4010c40 tx=0x7fc7f4017000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.433+0000 7fc7f17fa640 1 -- 172.21.15.62:0/347090694 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4024070 con 0x56475075c9d0 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.433+0000 7fc7f17fa640 1 -- 172.21.15.62:0/347090694 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f4009c50 con 0x56475075c9d0 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.433+0000 7fc7f17fa640 1 -- 172.21.15.62:0/347090694 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4005280 con 0x56475075c9d0 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 -- 172.21.15.62:0/347090694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 msgr2=0x56475075cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 --2- 172.21.15.62:0/347090694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 0x56475075cdb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4005fd0 tx=0x7fc7f4010740 comp rx=0 tx=0).stop 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 -- 172.21.15.62:0/347090694 shutdown_connections 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 --2- 172.21.15.62:0/347090694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750769fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 --2- 172.21.15.62:0/347090694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x564750767670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 --2- 172.21.15.62:0/347090694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 0x56475075cdb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 -- 172.21.15.62:0/347090694 >> 172.21.15.62:0/347090694 conn(0x564750758f60 msgr2=0x564750759340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 -- 172.21.15.62:0/347090694 shutdown_connections 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 -- 172.21.15.62:0/347090694 wait complete. 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 -- start start 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.434+0000 7fc803565740 1 Processor -- start 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc803565740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56475075c9d0 0x5647507674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc803565740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x564750764800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc803565740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564750767bb0 0x564750764d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc803565740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56475076a780 con 0x56475075c9d0 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc803565740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56475067df20 con 0x564750767bb0 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc803565740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56475076a900 con 0x56475075d2f0 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc803565740 10 client.0 _ll_register_callbacks cb 0x564750647030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56475075c9d0 0x5647507674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56475075c9d0 0x5647507674b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36212/0 (socket says 172.21.15.62:36212) 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fad9d640 1 -- 172.21.15.62:0/4112779861 learned_addr learned my addr 172.21.15.62:0/4112779861 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x564750764800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564750767bb0 0x564750764d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fad9d640 1 -- 172.21.15.62:0/4112779861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 msgr2=0x564750764800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x564750764800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fad9d640 1 -- 172.21.15.62:0/4112779861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564750767bb0 msgr2=0x564750764d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564750767bb0 0x564750764d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fad9d640 1 -- 172.21.15.62:0/4112779861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647504342b0 con 0x56475075c9d0 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564750767bb0 0x564750764d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56475075c9d0 0x5647507674b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0b84f0 tx=0x7fc7ec08b080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8314a04ef200d430 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.435+0000 7fc7e27fc640 1 -- 172.21.15.62:0/4112779861 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0bbbd0 con 0x56475075c9d0 2024-09-11T23:52:33.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc7e27fc640 1 -- 172.21.15.62:0/4112779861 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc7ec0c4870 con 0x56475075c9d0 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc803565740 1 -- 172.21.15.62:0/4112779861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f4009c50 con 0x56475075c9d0 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc7e27fc640 1 -- 172.21.15.62:0/4112779861 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7ec0dc0d0 con 0x56475075c9d0 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc803565740 10 client.15411 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc803565740 1 -- 172.21.15.62:0/4112779861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647507760f0 con 0x56475075c9d0 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc7e27fc640 1 -- 172.21.15.62:0/4112779861 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f4009c50 con 0x56475075c9d0 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc7e27fc640 1 -- 172.21.15.62:0/4112779861 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7ec089070 con 0x56475075c9d0 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc803565740 10 client.15411 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc803565740 10 client.15411 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc803565740 20 client.15411 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc803565740 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5647507787a0 0x564750779c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc803565740 4 client.15411 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:33.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc803565740 1 -- 172.21.15.62:0/4112779861 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647506d0600 con 0x5647507787a0 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.436+0000 7fc7f3fff640 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5647507787a0 0x564750779c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.437+0000 7fc7e27fc640 10 client.15411 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.437+0000 7fc7f3fff640 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5647507787a0 0x564750779c10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d449fc18b9671479 server_cookie=b21c4123f183044b in_seq=0 out_seq=0 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.437+0000 7fc7e27fc640 1 -- 172.21.15.62:0/4112779861 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647506d0600 con 0x5647507787a0 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.437+0000 7fc7e27fc640 10 client.15411 handle_command_reply: tid=0 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.437+0000 7fc803565740 1 client.15411 shutdown 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 1 -- 172.21.15.62:0/4112779861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56475075c9d0 msgr2=0x5647507674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56475075c9d0 0x5647507674b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0b84f0 tx=0x7fc7ec08b080 comp rx=0 tx=0).stop 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 1 -- 172.21.15.62:0/4112779861 shutdown_connections 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5647507787a0 0x564750779c10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564750767bb0 0x564750764d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x564750764800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc7e27fc640 0 client.15411 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:33.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 1 --2- 172.21.15.62:0/4112779861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56475075c9d0 0x5647507674b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 1 -- 172.21.15.62:0/4112779861 >> 172.21.15.62:0/4112779861 conn(0x564750758f60 msgr2=0x56475075ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:33.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 1 -- 172.21.15.62:0/4112779861 shutdown_connections 2024-09-11T23:52:33.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 1 -- 172.21.15.62:0/4112779861 wait complete. 2024-09-11T23:52:33.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.438+0000 7fc803565740 20 client.15411 trim_cache size 0 max 16384 2024-09-11T23:52:33.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc803565740 1 Processor -- start 2024-09-11T23:52:33.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc803565740 1 -- start start 2024-09-11T23:52:33.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc803565740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075c9d0 0x5647507640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc803565740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075d2f0 0x564750761ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc803565740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750762000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc803565740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564750777f50 con 0x564750767bb0 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc803565740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564750778620 con 0x56475075d2f0 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc803565740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564750778cf0 con 0x56475075c9d0 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc7fb59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750762000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc7fb59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750762000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36216/0 (socket says 172.21.15.62:36216) 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc7fb59e640 1 -- 172.21.15.62:0/1549050841 learned_addr learned my addr 172.21.15.62:0/1549050841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc7f3fff640 1 --2- 172.21.15.62:0/1549050841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075d2f0 0x564750761ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/1549050841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075c9d0 0x5647507640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc7fb59e640 1 -- 172.21.15.62:0/1549050841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075c9d0 msgr2=0x5647507640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/1549050841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075c9d0 0x5647507640c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc7fb59e640 1 -- 172.21.15.62:0/1549050841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075d2f0 msgr2=0x564750761ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.457+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/1549050841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075d2f0 0x564750761ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.458+0000 7fc7fb59e640 1 -- 172.21.15.62:0/1549050841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56475075a0f0 con 0x564750767bb0 2024-09-11T23:52:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.458+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/1549050841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750762000 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009960 tx=0x7fc7f400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9986f51cf0731a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.458+0000 7fc7f0ff9640 1 -- 172.21.15.62:0/1549050841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4024050 con 0x564750767bb0 2024-09-11T23:52:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.458+0000 7fc7f0ff9640 1 -- 172.21.15.62:0/1549050841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f4003e50 con 0x564750767bb0 2024-09-11T23:52:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.458+0000 7fc7f0ff9640 1 -- 172.21.15.62:0/1549050841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4005310 con 0x564750767bb0 2024-09-11T23:52:33.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 -- 172.21.15.62:0/1549050841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 msgr2=0x564750762000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 --2- 172.21.15.62:0/1549050841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750762000 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009960 tx=0x7fc7f400dbb0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 -- 172.21.15.62:0/1549050841 shutdown_connections 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 --2- 172.21.15.62:0/1549050841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x564750762000 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 --2- 172.21.15.62:0/1549050841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075d2f0 0x564750761ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 --2- 172.21.15.62:0/1549050841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075c9d0 0x5647507640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 -- 172.21.15.62:0/1549050841 >> 172.21.15.62:0/1549050841 conn(0x564750758f60 msgr2=0x564750777440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 -- 172.21.15.62:0/1549050841 shutdown_connections 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 -- 172.21.15.62:0/1549050841 wait complete. 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 -- start start 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 Processor -- start 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 0x564750782400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x56475077d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x56475077dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564750779a90 con 0x564750767bb0 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5647507793c0 con 0x56475075c9d0 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.459+0000 7fc803565740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56475077a160 con 0x56475075d2f0 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc803565740 10 client.0 _ll_register_callbacks cb 0x7fc7ec005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7fb59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x56475077dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7fb59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x56475077dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36218/0 (socket says 172.21.15.62:36218) 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7fb59e640 1 -- 172.21.15.62:0/1810103275 learned_addr learned my addr 172.21.15.62:0/1810103275 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 0x564750782400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7f3fff640 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x56475077d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7fb59e640 1 -- 172.21.15.62:0/1810103275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 msgr2=0x56475077d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x56475077d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7fb59e640 1 -- 172.21.15.62:0/1810103275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 msgr2=0x564750782400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:33.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 0x564750782400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7fb59e640 1 -- 172.21.15.62:0/1810103275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56475077e260 con 0x564750767bb0 2024-09-11T23:52:33.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7f3fff640 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x56475077d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:33.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7fb59e640 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x56475077dc70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4005010 tx=0x7fc7f40094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8decbe412cfe0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:33.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:33.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7e27fc640 1 -- 172.21.15.62:0/1810103275 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f403d060 con 0x564750767bb0 2024-09-11T23:52:33.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7e27fc640 1 -- 172.21.15.62:0/1810103275 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc7f4012df0 con 0x564750767bb0 2024-09-11T23:52:33.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc803565740 1 -- 172.21.15.62:0/1810103275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f4003e50 con 0x564750767bb0 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.460+0000 7fc7e27fc640 1 -- 172.21.15.62:0/1810103275 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7f4024020 con 0x564750767bb0 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc803565740 10 client.15417 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc803565740 1 -- 172.21.15.62:0/1810103275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647507e2c90 con 0x564750767bb0 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc7e27fc640 1 -- 172.21.15.62:0/1810103275 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f4003e50 con 0x564750767bb0 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc7e27fc640 1 -- 172.21.15.62:0/1810103275 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7f4041430 con 0x564750767bb0 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc803565740 10 client.15417 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc803565740 10 client.15417 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc803565740 20 client.15417 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc803565740 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5647507e70e0 0x5647507e5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc803565740 4 client.15417 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc803565740 1 -- 172.21.15.62:0/1810103275 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "status"}) -- 0x5647506cab10 con 0x5647507e70e0 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.461+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5647507e70e0 0x5647507e5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:33.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc7e27fc640 10 client.15417 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc7fad9d640 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5647507e70e0 0x5647507e5df0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f429c24ff286300 server_cookie=8fe6e0abbdb71339 in_seq=0 out_seq=0 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc7e27fc640 1 -- 172.21.15.62:0/1810103275 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5647506cab10 con 0x5647507e70e0 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc7e27fc640 10 client.15417 handle_command_reply: tid=0 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 client.15417 shutdown 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 -- 172.21.15.62:0/1810103275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 msgr2=0x56475077dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x56475077dc70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4005010 tx=0x7fc7f40094b0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 -- 172.21.15.62:0/1810103275 shutdown_connections 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5647507e70e0 0x5647507e5df0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc7e27fc640 0 client.15417 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564750767bb0 0x56475077dc70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56475075d2f0 0x56475077d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 --2- 172.21.15.62:0/1810103275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56475075c9d0 0x564750782400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 -- 172.21.15.62:0/1810103275 >> 172.21.15.62:0/1810103275 conn(0x564750758f60 msgr2=0x564750763110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:33.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 -- 172.21.15.62:0/1810103275 shutdown_connections 2024-09-11T23:52:33.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 1 -- 172.21.15.62:0/1810103275 wait complete. 2024-09-11T23:52:33.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.462+0000 7fc803565740 20 client.15417 trim_cache size 0 max 16384 2024-09-11T23:52:33.468 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:52:33.468 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "56675698-7097-11ef-bcea-c7b262605968", 2024-09-11T23:52:33.468 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-09-11T23:52:33.468 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24488, 2024-09-11T23:52:33.468 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-09-11T23:52:33.468 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-09-11T23:52:33.468 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-09-11T23:52:33.469 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 141.593090031, 2024-09-11T23:52:33.469 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 25, 2024-09-11T23:52:33.469 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 84, 2024-09-11T23:52:33.469 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 83, 2024-09-11T23:52:33.469 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 152.77474592799999 2024-09-11T23:52:33.469 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 -- 172.21.15.62:0/1324988933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc7e80a2d80 msgr2=0x7fc7e80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc7e80a2d80 0x7fc7e80a5240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4010c40 tx=0x7fc7f4017000 comp rx=0 tx=0).stop 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 -- 172.21.15.62:0/1324988933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7fc102d40 msgr2=0x7fc7fc1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1984e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec006010 tx=0x7fc7ec0042f0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 -- 172.21.15.62:0/1324988933 shutdown_connections 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc7e80a2d80 0x7fc7e80a5240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7fc10c170 0x7fc7fc19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7fc103660 0x7fc7fc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 --2- 172.21.15.62:0/1324988933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1984e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 -- 172.21.15.62:0/1324988933 >> 172.21.15.62:0/1324988933 conn(0x7fc7fc0fc7a0 msgr2=0x7fc7fc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:33.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 -- 172.21.15.62:0/1324988933 shutdown_connections 2024-09-11T23:52:33.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:33.463+0000 7fc80118c640 1 -- 172.21.15.62:0/1324988933 wait complete. 2024-09-11T23:52:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:33 smithi120 ceph-mon[35182]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:52:34.162 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-11T23:52:35.373 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:35 smithi120 ceph-mon[35182]: pgmap v17: 97 pgs: 97 active+clean; 2.2 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 83 KiB/s wr, 11 op/s 2024-09-11T23:52:35.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:35 smithi083 ceph-mon[34805]: pgmap v17: 97 pgs: 97 active+clean; 2.2 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 83 KiB/s wr, 11 op/s 2024-09-11T23:52:35.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:35 smithi062 ceph-mon[30958]: pgmap v17: 97 pgs: 97 active+clean; 2.2 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 83 KiB/s wr, 11 op/s 2024-09-11T23:52:36.318 INFO:teuthology.orchestra.run.smithi062.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-11T23:52:36.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-11T23:52:36.363 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/: 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 . 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 .. 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 data 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data: 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:total 35 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 . 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 .. 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 base 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 11 23:52 global 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 log 2024-09-11T23:52:36.364 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_commit_ts 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_dynshmem 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 11 23:52 pg_hba.conf 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 11 23:52 pg_ident.conf 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 pg_logical 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 pg_multixact 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_notify 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_replslot 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_serial 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_snapshots 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_stat 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_stat_tmp 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 pg_subtrans 2024-09-11T23:52:36.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_tblspc 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_twophase 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 PG_VERSION 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 pg_wal 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 pg_xact 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 11 23:52 postgresql.auto.conf 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 11 23:52 postgresql.conf 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/base: 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 . 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 11 23:52 1 2024-09-11T23:52:36.366 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 11 23:52 13493 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 11 23:52 13494 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:total 7777 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 11 23:52 . 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 .. 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 112 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 113 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 1247 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1247_fsm 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1247_vm 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 11 23:52 1249 2024-09-11T23:52:36.367 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1249_fsm 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1249_vm 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 11 23:52 1255 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1255_fsm 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1255_vm 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 11 23:52 1259 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1259_fsm 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1259_vm 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 11 23:52 13344 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13344_fsm 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13344_vm 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13346 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13348 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349 2024-09-11T23:52:36.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13349_fsm 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349_vm 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13351 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13353 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13354_fsm 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354_vm 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13356 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13358 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13359_fsm 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359_vm 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13361 2024-09-11T23:52:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13363 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1417 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1418 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 174 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 175 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2187 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2224 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2328 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2336 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2337 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2579 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2600 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2600_fsm 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2600_vm 2024-09-11T23:52:36.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2601_fsm 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601_vm 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2602 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2602_fsm 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2602_vm 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2603 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2603_fsm 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2603_vm 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2604 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2605 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2605_fsm 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2605_vm 2024-09-11T23:52:36.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2606_fsm 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606_vm 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2607 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2607_fsm 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2607_vm 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 11 23:52 2608 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2608_fsm 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2608_vm 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 11 23:52 2609 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2609_fsm 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2609_vm 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2610 2024-09-11T23:52:36.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2610_fsm 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2610_vm 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2611 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2612_fsm 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612_vm 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2613 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2615_fsm 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615_vm 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2616 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2616_fsm 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2616_vm 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 11 23:52 2617 2024-09-11T23:52:36.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2617_fsm 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2617_vm 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 11 23:52 2618 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2618_fsm 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2618_vm 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 11 23:52 2619 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2619_fsm 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2619_vm 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2620 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2650 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2651 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2652 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2653 2024-09-11T23:52:36.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2654 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2655 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2656 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2657 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 11 23:52 2658 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 2659 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2660 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2661 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2662 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2663 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2664 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2665 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2666 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2667 2024-09-11T23:52:36.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2668 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2669 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2670 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 11 23:52 2673 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 11 23:52 2674 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 11 23:52 2675 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2678 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2679 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2680 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2681 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2682 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2683 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2684 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2685 2024-09-11T23:52:36.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2686 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2687 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2688 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2689 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 11 23:52 2690 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 11 23:52 2691 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2692 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2693 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2696 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2699 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2701 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2702 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2703 2024-09-11T23:52:36.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2704 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2753 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2753_fsm 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2753_vm 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2754 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2755 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2756 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2757 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2830 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2831 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2832 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2833 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2834 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2835 2024-09-11T23:52:36.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2836 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2837 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 11 23:52 2838 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2838_fsm 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2838_vm 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2839 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840_fsm 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2840_vm 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2841 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2995 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2996 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3079_fsm 2024-09-11T23:52:36.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079_vm 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3080 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3081 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3085 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3118 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3119 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 11 23:52 3164 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3256 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3257 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3258 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3350 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3351 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3379 2024-09-11T23:52:36.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3380 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3381 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3394 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3394_fsm 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3394_vm 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3395 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3429 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3430 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3431 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3433 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3439 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3440 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3455 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 11 23:52 3456 2024-09-11T23:52:36.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3456_fsm 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3456_vm 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3466 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3467 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3468 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3501 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3502 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3503 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3534 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3541_fsm 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541_vm 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3542 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3574 2024-09-11T23:52:36.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3575 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3576 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3596 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3597 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3598 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3599 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3600_fsm 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600_vm 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3601_fsm 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601_vm 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602 2024-09-11T23:52:36.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3602_fsm 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602_vm 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603_fsm 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3603_vm 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3604 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3605 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3606 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3607 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3608 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 3609 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3712 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3764_fsm 2024-09-11T23:52:36.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764_vm 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3766 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3767 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3997 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4143 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4144 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4145 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4146 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4147 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4148 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4149 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4150 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4151 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4152 2024-09-11T23:52:36.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4153 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4154 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4155 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4156 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4157 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4158 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4159 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4160 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4161 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4162 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4163 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4164 2024-09-11T23:52:36.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4165 2024-09-11T23:52:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4166 2024-09-11T23:52:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4167 2024-09-11T23:52:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4168 2024-09-11T23:52:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4169 2024-09-11T23:52:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4170 2024-09-11T23:52:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4171 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4172 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4173 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4174 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 5002 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 548 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 549 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6102 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6104 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6106 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6110 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6111 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6112 2024-09-11T23:52:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6113 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6117 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 826 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 827 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 828 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 11 23:52 pg_filenode.map 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 PG_VERSION 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:total 7777 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 11 23:52 . 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 .. 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 112 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 113 2024-09-11T23:52:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 1247 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1247_fsm 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1247_vm 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 11 23:52 1249 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1249_fsm 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1249_vm 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 11 23:52 1255 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1255_fsm 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1255_vm 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 11 23:52 1259 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1259_fsm 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1259_vm 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 11 23:52 13344 2024-09-11T23:52:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13344_fsm 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13344_vm 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13346 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13348 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13349_fsm 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349_vm 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13351 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13353 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13354_fsm 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354_vm 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13356 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13358 2024-09-11T23:52:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13359_fsm 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359_vm 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13361 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13363 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1417 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1418 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 174 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 175 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2187 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2224 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2328 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2336 2024-09-11T23:52:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2337 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2579 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2600 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2600_fsm 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2600_vm 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2601_fsm 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601_vm 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2602 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2602_fsm 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2602_vm 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2603 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2603_fsm 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2603_vm 2024-09-11T23:52:36.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2604 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2605 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2605_fsm 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2605_vm 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2606_fsm 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606_vm 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2607 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2607_fsm 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2607_vm 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 11 23:52 2608 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2608_fsm 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2608_vm 2024-09-11T23:52:36.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 11 23:52 2609 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2609_fsm 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2609_vm 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2610 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2610_fsm 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2610_vm 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2611 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2612_fsm 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612_vm 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2613 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2615_fsm 2024-09-11T23:52:36.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615_vm 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2616 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2616_fsm 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2616_vm 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 11 23:52 2617 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2617_fsm 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2617_vm 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 11 23:52 2618 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2618_fsm 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2618_vm 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 11 23:52 2619 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2619_fsm 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2619_vm 2024-09-11T23:52:36.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2620 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2650 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2651 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2652 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2653 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2654 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2655 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2656 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2657 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 11 23:52 2658 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 2659 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2660 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2661 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2662 2024-09-11T23:52:36.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2663 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2664 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2665 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2666 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2667 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2668 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2669 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2670 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 11 23:52 2673 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 11 23:52 2674 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 11 23:52 2675 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2678 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2679 2024-09-11T23:52:36.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2680 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2681 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2682 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2683 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2684 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2685 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2686 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2687 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2688 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2689 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 11 23:52 2690 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 11 23:52 2691 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2692 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2693 2024-09-11T23:52:36.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2696 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2699 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2701 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2702 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2703 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2704 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2753 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2753_fsm 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2753_vm 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2754 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2755 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2756 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2757 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2830 2024-09-11T23:52:36.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2831 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2832 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2833 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2834 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2835 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2836 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2837 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 11 23:52 2838 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2838_fsm 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2838_vm 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2839 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840_fsm 2024-09-11T23:52:36.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2840_vm 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2841 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2995 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2996 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3079_fsm 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079_vm 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3080 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3081 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3085 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3118 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3119 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 11 23:52 3164 2024-09-11T23:52:36.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3256 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3257 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3258 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3350 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3351 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3379 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3380 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3381 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3394 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3394_fsm 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3394_vm 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3395 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3429 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3430 2024-09-11T23:52:36.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3431 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3433 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3439 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3440 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3455 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 11 23:52 3456 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3456_fsm 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3456_vm 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3466 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3467 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3468 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3501 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3502 2024-09-11T23:52:36.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3503 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3534 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3541_fsm 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541_vm 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3542 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3574 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3575 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3576 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3596 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3597 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3598 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3599 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600 2024-09-11T23:52:36.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3600_fsm 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600_vm 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3601_fsm 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601_vm 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3602_fsm 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602_vm 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603_fsm 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3603_vm 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3604 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3605 2024-09-11T23:52:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3606 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3607 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3608 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 3609 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3712 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3764_fsm 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764_vm 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3766 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3767 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3997 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4143 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4144 2024-09-11T23:52:36.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4145 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4146 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4147 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4148 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4149 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4150 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4151 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4152 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4153 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4154 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4155 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4156 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4157 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4158 2024-09-11T23:52:36.410 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4159 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4160 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4161 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4162 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4163 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4164 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4165 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4166 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4167 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4168 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4169 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4170 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4171 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4172 2024-09-11T23:52:36.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4173 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4174 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 5002 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 548 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 549 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6102 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6104 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6106 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6110 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6111 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6112 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6113 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6117 2024-09-11T23:52:36.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 826 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 827 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 828 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 11 23:52 pg_filenode.map 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 PG_VERSION 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:total 7777 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 11 23:52 . 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 .. 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 112 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 113 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 1247 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1247_fsm 2024-09-11T23:52:36.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1247_vm 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 11 23:52 1249 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1249_fsm 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1249_vm 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 11 23:52 1255 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1255_fsm 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1255_vm 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 11 23:52 1259 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1259_fsm 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1259_vm 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 11 23:52 13344 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13344_fsm 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13344_vm 2024-09-11T23:52:36.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13346 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13348 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13349_fsm 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349_vm 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13351 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13353 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13354_fsm 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354_vm 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13356 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13358 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13359_fsm 2024-09-11T23:52:36.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359_vm 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13361 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13363 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1417 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1418 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 174 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 175 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2187 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2224 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2328 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2336 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2337 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2579 2024-09-11T23:52:36.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2600 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2600_fsm 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2600_vm 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2601_fsm 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601_vm 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2602 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2602_fsm 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2602_vm 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2603 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2603_fsm 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2603_vm 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2604 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2605 2024-09-11T23:52:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2605_fsm 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2605_vm 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2606_fsm 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606_vm 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2607 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2607_fsm 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2607_vm 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 11 23:52 2608 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2608_fsm 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2608_vm 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 11 23:52 2609 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2609_fsm 2024-09-11T23:52:36.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2609_vm 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2610 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2610_fsm 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2610_vm 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2611 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2612_fsm 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612_vm 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2613 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2615_fsm 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615_vm 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2616 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2616_fsm 2024-09-11T23:52:36.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2616_vm 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 11 23:52 2617 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2617_fsm 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2617_vm 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 11 23:52 2618 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2618_fsm 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2618_vm 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 11 23:52 2619 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2619_fsm 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2619_vm 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2620 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2650 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2651 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2652 2024-09-11T23:52:36.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2653 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2654 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2655 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2656 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2657 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 11 23:52 2658 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 2659 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2660 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2661 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2662 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2663 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2664 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2665 2024-09-11T23:52:36.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2666 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2667 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2668 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2669 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2670 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 11 23:52 2673 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 11 23:52 2674 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 11 23:52 2675 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2678 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2679 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2680 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2681 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2682 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2683 2024-09-11T23:52:36.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2684 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2685 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2686 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2687 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2688 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2689 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 11 23:52 2690 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 11 23:52 2691 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2692 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2693 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2696 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2699 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2701 2024-09-11T23:52:36.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2702 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2703 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2704 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2753 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2753_fsm 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2753_vm 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2754 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2755 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2756 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2757 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2830 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2831 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2832 2024-09-11T23:52:36.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2833 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2834 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2835 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2836 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2837 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 11 23:52 2838 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2838_fsm 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2838_vm 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2839 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840_fsm 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2840_vm 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2841 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2995 2024-09-11T23:52:36.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2996 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3079_fsm 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079_vm 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3080 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3081 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3085 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3118 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3119 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 11 23:52 3164 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3256 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3257 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3258 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3350 2024-09-11T23:52:36.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3351 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3379 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3380 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3381 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3394 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3394_fsm 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3394_vm 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3395 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3429 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3430 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3431 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3433 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3439 2024-09-11T23:52:36.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3440 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3455 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 11 23:52 3456 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3456_fsm 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3456_vm 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3466 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3467 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3468 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3501 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3502 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3503 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3534 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541 2024-09-11T23:52:36.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3541_fsm 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541_vm 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3542 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3574 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3575 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3576 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3596 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3597 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3598 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3599 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3600_fsm 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600_vm 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601 2024-09-11T23:52:36.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3601_fsm 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601_vm 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3602_fsm 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602_vm 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603_fsm 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3603_vm 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3604 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3605 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3606 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3607 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3608 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 3609 2024-09-11T23:52:36.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3712 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3764_fsm 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764_vm 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3766 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3767 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3997 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4143 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4144 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4145 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4146 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4147 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4148 2024-09-11T23:52:36.431 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4149 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4150 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4151 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4152 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4153 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4154 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4155 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4156 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4157 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4158 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4159 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4160 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4161 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4162 2024-09-11T23:52:36.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4163 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4164 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4165 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4166 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4167 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4168 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4169 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4170 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4171 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4172 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4173 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4174 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 5002 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 548 2024-09-11T23:52:36.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 549 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6102 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6104 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6106 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6110 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6111 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6112 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6113 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6117 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 826 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 827 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 828 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 11 23:52 pg_filenode.map 2024-09-11T23:52:36.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 PG_VERSION 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/global: 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:total 537 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 11 23:52 . 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1213 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1213_fsm 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1213_vm 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1214 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1214_fsm 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1214_vm 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 1232 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 1233 2024-09-11T23:52:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1260 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1260_fsm 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1260_vm 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1261 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1261_fsm 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1261_vm 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1262 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1262_fsm 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1262_vm 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2396 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2396_fsm 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2396_vm 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2397 2024-09-11T23:52:36.436 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2671 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2672 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2676 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2677 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2694 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2695 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2697 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2698 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2846 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2847 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2964 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2965 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2966 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2967 2024-09-11T23:52:36.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3592 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3593 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4060 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4061 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4175 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4176 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4177 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4178 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4181 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4182 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4183 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4184 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4185 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4186 2024-09-11T23:52:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6000 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6001 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6002 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6100 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6114 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6115 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 pg_control 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 11 23:52 pg_filenode.map 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/log: 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.439 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.440 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-11T23:52:36.440 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.440 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.440 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.440 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.440 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-11T23:52:36.440 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.440 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.440 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.440 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:total 1 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 . 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 mappings 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 11 23:52 replorigin_checkpoint 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 snapshots 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 .. 2024-09-11T23:52:36.441 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 .. 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 . 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 members 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 offsets 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-11T23:52:36.442 INFO:teuthology.orchestra.run.smithi062.stdout:total 8 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 . 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 .. 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 0000 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:total 8 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 . 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 .. 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 0000 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.443 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-11T23:52:36.444 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.445 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:total 8 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 . 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 0000 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.446 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:total 16384 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 . 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 11 23:52 000000010000000000000001 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 archive_status 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 .. 2024-09-11T23:52:36.447 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:36.448 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-11T23:52:36.448 INFO:teuthology.orchestra.run.smithi062.stdout:total 8 2024-09-11T23:52:36.448 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 . 2024-09-11T23:52:36.448 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 11 23:52 .. 2024-09-11T23:52:36.448 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 0000 2024-09-11T23:52:36.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-11T23:52:36.562 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-11T23:52:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:dropping old tables... 2024-09-11T23:52:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-11T23:52:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-11T23:52:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-11T23:52:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-11T23:52:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:creating tables... 2024-09-11T23:52:36.618 INFO:teuthology.orchestra.run.smithi062.stderr:generating data (client-side)... 2024-09-11T23:52:36.715 INFO:teuthology.orchestra.run.smithi062.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.08 s, remaining 2.41 s) 2024-09-11T23:52:36.868 INFO:teuthology.orchestra.run.smithi062.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.23 s, remaining 3.45 s) 2024-09-11T23:52:36.965 INFO:teuthology.orchestra.run.smithi062.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.33 s, remaining 3.16 s) 2024-09-11T23:52:37.088 INFO:teuthology.orchestra.run.smithi062.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.45 s, remaining 3.15 s) 2024-09-11T23:52:37.231 INFO:teuthology.orchestra.run.smithi062.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.59 s, remaining 3.20 s) 2024-09-11T23:52:37.311 INFO:teuthology.orchestra.run.smithi062.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.67 s, remaining 2.92 s) 2024-09-11T23:52:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:37 smithi083 ceph-mon[34805]: pgmap v18: 97 pgs: 97 active+clean; 2.2 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 83 KiB/s wr, 8 op/s 2024-09-11T23:52:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.81 s, remaining 2.88 s) 2024-09-11T23:52:37.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:37 smithi062 ceph-mon[30958]: pgmap v18: 97 pgs: 97 active+clean; 2.2 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 83 KiB/s wr, 8 op/s 2024-09-11T23:52:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.96 s, remaining 2.89 s) 2024-09-11T23:52:37.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:37 smithi120 ceph-mon[35182]: pgmap v18: 97 pgs: 97 active+clean; 2.2 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 83 KiB/s wr, 8 op/s 2024-09-11T23:52:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.06 s, remaining 2.71 s) 2024-09-11T23:52:37.827 INFO:teuthology.orchestra.run.smithi062.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.19 s, remaining 2.62 s) 2024-09-11T23:52:37.981 INFO:teuthology.orchestra.run.smithi062.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.34 s, remaining 2.56 s) 2024-09-11T23:52:38.071 INFO:teuthology.orchestra.run.smithi062.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.43 s, remaining 2.39 s) 2024-09-11T23:52:38.214 INFO:teuthology.orchestra.run.smithi062.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.58 s, remaining 2.30 s) 2024-09-11T23:52:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.68 s, remaining 2.16 s) 2024-09-11T23:52:38.457 INFO:teuthology.orchestra.run.smithi062.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.82 s, remaining 2.06 s) 2024-09-11T23:52:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:1600000 of 3200000 tuples (50%) done (elapsed 1.96 s, remaining 1.96 s) 2024-09-11T23:52:38.739 INFO:teuthology.orchestra.run.smithi062.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.10 s, remaining 1.85 s) 2024-09-11T23:52:38.816 INFO:teuthology.orchestra.run.smithi062.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.18 s, remaining 1.69 s) 2024-09-11T23:52:38.967 INFO:teuthology.orchestra.run.smithi062.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.33 s, remaining 1.59 s) 2024-09-11T23:52:39.042 INFO:teuthology.orchestra.run.smithi062.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.40 s, remaining 1.44 s) 2024-09-11T23:52:39.183 INFO:teuthology.orchestra.run.smithi062.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.54 s, remaining 1.33 s) 2024-09-11T23:52:39.348 INFO:teuthology.orchestra.run.smithi062.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.71 s, remaining 1.23 s) 2024-09-11T23:52:39.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:39 smithi083 ceph-mon[34805]: pgmap v19: 97 pgs: 97 active+clean; 45 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 3.7 MiB/s wr, 113 op/s 2024-09-11T23:52:39.438 INFO:teuthology.orchestra.run.smithi062.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.80 s, remaining 1.10 s) 2024-09-11T23:52:39.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:39 smithi062 ceph-mon[30958]: pgmap v19: 97 pgs: 97 active+clean; 45 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 3.7 MiB/s wr, 113 op/s 2024-09-11T23:52:39.577 INFO:teuthology.orchestra.run.smithi062.stderr:2400000 of 3200000 tuples (75%) done (elapsed 2.94 s, remaining 0.98 s) 2024-09-11T23:52:39.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:39 smithi120 ceph-mon[35182]: pgmap v19: 97 pgs: 97 active+clean; 45 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 3.7 MiB/s wr, 113 op/s 2024-09-11T23:52:39.728 INFO:teuthology.orchestra.run.smithi062.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.09 s, remaining 0.87 s) 2024-09-11T23:52:39.821 INFO:teuthology.orchestra.run.smithi062.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.18 s, remaining 0.73 s) 2024-09-11T23:52:39.954 INFO:teuthology.orchestra.run.smithi062.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.32 s, remaining 0.61 s) 2024-09-11T23:52:40.046 INFO:teuthology.orchestra.run.smithi062.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.41 s, remaining 0.49 s) 2024-09-11T23:52:40.177 INFO:teuthology.orchestra.run.smithi062.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.54 s, remaining 0.37 s) 2024-09-11T23:52:40.346 INFO:teuthology.orchestra.run.smithi062.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.71 s, remaining 0.25 s) 2024-09-11T23:52:40.492 INFO:teuthology.orchestra.run.smithi062.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.85 s, remaining 0.12 s) 2024-09-11T23:52:40.573 INFO:teuthology.orchestra.run.smithi062.stderr:3200000 of 3200000 tuples (100%) done (elapsed 3.94 s, remaining 0.00 s) 2024-09-11T23:52:40.605 INFO:teuthology.orchestra.run.smithi062.stderr:vacuuming... 2024-09-11T23:52:41.205 INFO:teuthology.orchestra.run.smithi062.stderr:creating primary keys... 2024-09-11T23:52:41.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:41 smithi083 ceph-mon[34805]: pgmap v20: 97 pgs: 97 active+clean; 45 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 3.7 MiB/s wr, 111 op/s 2024-09-11T23:52:41.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:41 smithi062 ceph-mon[30958]: pgmap v20: 97 pgs: 97 active+clean; 45 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 3.7 MiB/s wr, 111 op/s 2024-09-11T23:52:41.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:41 smithi120 ceph-mon[35182]: pgmap v20: 97 pgs: 97 active+clean; 45 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 3.7 MiB/s wr, 111 op/s 2024-09-11T23:52:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:done in 5.78 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 3.99 s, vacuum 0.60 s, primary keys 1.18 s). 2024-09-11T23:52:42.390 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-11T23:52:42.444 INFO:teuthology.orchestra.run.smithi062.stderr:starting vacuum...end. 2024-09-11T23:52:43.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:43 smithi083 ceph-mon[34805]: pgmap v21: 97 pgs: 97 active+clean; 45 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 3.7 MiB/s wr, 111 op/s 2024-09-11T23:52:43.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:43 smithi062 ceph-mon[30958]: pgmap v21: 97 pgs: 97 active+clean; 45 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 3.7 MiB/s wr, 111 op/s 2024-09-11T23:52:43.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:43 smithi120 ceph-mon[35182]: pgmap v21: 97 pgs: 97 active+clean; 45 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 3.7 MiB/s wr, 111 op/s 2024-09-11T23:52:45.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:45 smithi083 ceph-mon[34805]: pgmap v22: 97 pgs: 97 active+clean; 381 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 59 MiB/s wr, 148 op/s 2024-09-11T23:52:45.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:45 smithi062 ceph-mon[30958]: pgmap v22: 97 pgs: 97 active+clean; 381 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 59 MiB/s wr, 148 op/s 2024-09-11T23:52:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:45 smithi120 ceph-mon[35182]: pgmap v22: 97 pgs: 97 active+clean; 381 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 59 MiB/s wr, 148 op/s 2024-09-11T23:52:45.960 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:52:46.335 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:46.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.911+0000 7fd8177d1640 1 -- 172.21.15.62:0/3831046588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd810106a70 msgr2=0x7fd810106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:46.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.911+0000 7fd8177d1640 1 --2- 172.21.15.62:0/3831046588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd810106a70 0x7fd810106e50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009890 tx=0x7fd80c02a370 comp rx=0 tx=0).stop 2024-09-11T23:52:46.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.912+0000 7fd8177d1640 1 -- 172.21.15.62:0/3831046588 shutdown_connections 2024-09-11T23:52:46.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.912+0000 7fd8177d1640 1 --2- 172.21.15.62:0/3831046588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd810101410 0x7fd81010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.912+0000 7fd8177d1640 1 --2- 172.21.15.62:0/3831046588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd810100a70 0x7fd810100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.912+0000 7fd8177d1640 1 --2- 172.21.15.62:0/3831046588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd810106a70 0x7fd810106e50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.912+0000 7fd8177d1640 1 -- 172.21.15.62:0/3831046588 >> 172.21.15.62:0/3831046588 conn(0x7fd8100fc7d0 msgr2=0x7fd8100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:46.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.914+0000 7fd8177d1640 1 -- 172.21.15.62:0/3831046588 shutdown_connections 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.914+0000 7fd8177d1640 1 -- 172.21.15.62:0/3831046588 wait complete. 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.915+0000 7fd8177d1640 1 Processor -- start 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.915+0000 7fd8177d1640 1 -- start start 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.915+0000 7fd8177d1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd810100a70 0x7fd8101960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.915+0000 7fd8177d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd810101410 0x7fd8101965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd8177d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd810106a70 0x7fd810196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd8177d1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd81010fd60 con 0x7fd810100a70 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd8177d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd81010fbe0 con 0x7fd810106a70 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd8177d1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd81010fee0 con 0x7fd810101410 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd81556d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd810100a70 0x7fd8101960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd814d6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd810101410 0x7fd8101965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd814d6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd810101410 0x7fd8101965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:45530/0 (socket says 172.21.15.62:45530) 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd814d6c640 1 -- 172.21.15.62:0/3012186948 learned_addr learned my addr 172.21.15.62:0/3012186948 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd815d6e640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd810106a70 0x7fd810196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd814d6c640 1 -- 172.21.15.62:0/3012186948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd810106a70 msgr2=0x7fd810196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd814d6c640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd810106a70 0x7fd810196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd814d6c640 1 -- 172.21.15.62:0/3012186948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd810100a70 msgr2=0x7fd8101960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd814d6c640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd810100a70 0x7fd8101960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.916+0000 7fd814d6c640 1 -- 172.21.15.62:0/3012186948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8101aad30 con 0x7fd810101410 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.917+0000 7fd814d6c640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd810101410 0x7fd8101965f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd800009750 tx=0x7fd800009c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.917+0000 7fd8067fc640 1 -- 172.21.15.62:0/3012186948 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800009e60 con 0x7fd810101410 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.917+0000 7fd8067fc640 1 -- 172.21.15.62:0/3012186948 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd800005020 con 0x7fd810101410 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.917+0000 7fd8067fc640 1 -- 172.21.15.62:0/3012186948 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800004ce0 con 0x7fd810101410 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.917+0000 7fd8177d1640 1 -- 172.21.15.62:0/3012186948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8101aafc0 con 0x7fd810101410 2024-09-11T23:52:46.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.918+0000 7fd8177d1640 1 -- 172.21.15.62:0/3012186948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8101ab520 con 0x7fd810101410 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.922+0000 7fd8067fc640 1 -- 172.21.15.62:0/3012186948 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fd800016410 con 0x7fd810101410 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.927+0000 7fd819baa740 1 Processor -- start 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.927+0000 7fd8067fc640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd7fc0a2e80 0x7fd7fc0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.927+0000 7fd819baa740 1 -- start start 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.927+0000 7fd819baa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e654a0 0x5602c2e65880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.927+0000 7fd819baa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e70140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.927+0000 7fd819baa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e72a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.927+0000 7fd819baa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5602c2dcb2e0 con 0x5602c2e65dc0 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.927+0000 7fd819baa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602c2d72ce0 con 0x5602c2e70680 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.927+0000 7fd819baa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5602c2dc72a0 con 0x5602c2e654a0 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.927+0000 7fd8067fc640 1 -- 172.21.15.62:0/3012186948 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fd8000cfb90 con 0x7fd810101410 2024-09-11T23:52:46.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd815d6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e72a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd815d6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e72a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:51172/0 (socket says 172.21.15.62:51172) 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd815d6e640 1 -- 172.21.15.62:0/1628736521 learned_addr learned my addr 172.21.15.62:0/1628736521 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd814d6c640 1 --2- 172.21.15.62:0/1628736521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e70140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd815d6e640 1 -- 172.21.15.62:0/1628736521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e654a0 msgr2=0x5602c2e65880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd815d6e640 1 --2- 172.21.15.62:0/1628736521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e654a0 0x5602c2e65880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd815d6e640 1 -- 172.21.15.62:0/1628736521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 msgr2=0x5602c2e70140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd815d6e640 1 --2- 172.21.15.62:0/1628736521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e70140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd815d6e640 1 -- 172.21.15.62:0/1628736521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c2d57440 con 0x5602c2e70680 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd815d6e640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd7fc0a2e80 0x7fd7fc0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.928+0000 7fd815d6e640 1 --2- 172.21.15.62:0/1628736521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e72a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd808012450 tx=0x7fd808012910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7307eb404fe790 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.929+0000 7fd815d6e640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd7fc0a2e80 0x7fd7fc0a5340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd808021000 tx=0x7fd80801e000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.929+0000 7fd7fb7fe640 1 -- 172.21.15.62:0/1628736521 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808010bb0 con 0x5602c2e70680 2024-09-11T23:52:46.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.930+0000 7fd7fb7fe640 1 -- 172.21.15.62:0/1628736521 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd80801cd80 con 0x5602c2e70680 2024-09-11T23:52:46.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.930+0000 7fd7fb7fe640 1 -- 172.21.15.62:0/1628736521 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80801b5a0 con 0x5602c2e70680 2024-09-11T23:52:46.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.931+0000 7fd819baa740 1 -- 172.21.15.62:0/1628736521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 msgr2=0x5602c2e72a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:46.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.931+0000 7fd819baa740 1 --2- 172.21.15.62:0/1628736521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e72a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd808012450 tx=0x7fd808012910 comp rx=0 tx=0).stop 2024-09-11T23:52:46.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.931+0000 7fd819baa740 1 -- 172.21.15.62:0/1628736521 shutdown_connections 2024-09-11T23:52:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.931+0000 7fd819baa740 1 --2- 172.21.15.62:0/1628736521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e72a70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.931+0000 7fd819baa740 1 --2- 172.21.15.62:0/1628736521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e70140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.931+0000 7fd819baa740 1 --2- 172.21.15.62:0/1628736521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e654a0 0x5602c2e65880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.931+0000 7fd819baa740 1 -- 172.21.15.62:0/1628736521 >> 172.21.15.62:0/1628736521 conn(0x5602c2e61a30 msgr2=0x5602c2e61e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.932+0000 7fd819baa740 1 -- 172.21.15.62:0/1628736521 shutdown_connections 2024-09-11T23:52:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.932+0000 7fd819baa740 1 -- 172.21.15.62:0/1628736521 wait complete. 2024-09-11T23:52:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.932+0000 7fd819baa740 1 -- start start 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.932+0000 7fd819baa740 1 Processor -- start 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.933+0000 7fd819baa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e654a0 0x5602c2e70030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.933+0000 7fd819baa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e6d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.933+0000 7fd819baa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 0x5602c2e6d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.934+0000 7fd819baa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5602c2e73250 con 0x5602c2e65dc0 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.934+0000 7fd819baa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602c2d86050 con 0x5602c2e654a0 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.934+0000 7fd819baa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5602c2e733d0 con 0x5602c2e70680 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.934+0000 7fd81556d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e654a0 0x5602c2e70030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.934+0000 7fd814d6c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e6d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.934+0000 7fd81556d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e654a0 0x5602c2e70030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:51184/0 (socket says 172.21.15.62:51184) 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.934+0000 7fd81556d640 1 -- 172.21.15.62:0/3074735411 learned_addr learned my addr 172.21.15.62:0/3074735411 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.934+0000 7fd815d6e640 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 0x5602c2e6d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.934+0000 7fd819baa740 10 client.0 _ll_register_callbacks cb 0x5602c2dc6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.935+0000 7fd81556d640 1 -- 172.21.15.62:0/3074735411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 msgr2=0x5602c2e6d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.935+0000 7fd81556d640 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 0x5602c2e6d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.935+0000 7fd81556d640 1 -- 172.21.15.62:0/3074735411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 msgr2=0x5602c2e6d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.935+0000 7fd81556d640 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e6d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.935+0000 7fd81556d640 1 -- 172.21.15.62:0/3074735411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c2b3c3c0 con 0x5602c2e654a0 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.935+0000 7fd81556d640 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e654a0 0x5602c2e70030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009890 tx=0x7fd80c009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a834cb06a329bd1b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.935+0000 7fd7f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.935+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/3074735411 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c037040 con 0x5602c2e654a0 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.936+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/3074735411 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fd80c030990 con 0x5602c2e654a0 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.936+0000 7fd819baa740 1 -- 172.21.15.62:0/3074735411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80801cd80 con 0x5602c2e654a0 2024-09-11T23:52:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.936+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/3074735411 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd80c04c050 con 0x5602c2e654a0 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.936+0000 7fd819baa740 10 client.25115 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.936+0000 7fd819baa740 1 -- 172.21.15.62:0/3074735411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5602c2e7eca0 con 0x5602c2e654a0 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.938+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/3074735411 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd80801cd80 con 0x5602c2e654a0 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.938+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/3074735411 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd80c03d440 con 0x5602c2e654a0 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.938+0000 7fd819baa740 10 client.25115 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.938+0000 7fd819baa740 10 client.25115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.938+0000 7fd819baa740 20 client.25115 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.938+0000 7fd819baa740 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c2e80200 0x5602c2e82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.938+0000 7fd819baa740 4 client.25115 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.938+0000 7fd819baa740 1 -- 172.21.15.62:0/3074735411 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602c2dd8850 con 0x5602c2e80200 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.938+0000 7fd814d6c640 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c2e80200 0x5602c2e82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.939+0000 7fd7f8ff9640 10 client.25115 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:46.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.939+0000 7fd814d6c640 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c2e80200 0x5602c2e82680 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd2fdb71cd2427ea server_cookie=837eaf3e564db458 in_seq=0 out_seq=0 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/3074735411 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602c2dd8850 con 0x5602c2e80200 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd7f8ff9640 10 client.25115 handle_command_reply: tid=0 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 client.25115 shutdown 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 -- 172.21.15.62:0/3074735411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e654a0 msgr2=0x5602c2e70030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e654a0 0x5602c2e70030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009890 tx=0x7fd80c009520 comp rx=0 tx=0).stop 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 -- 172.21.15.62:0/3074735411 shutdown_connections 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c2e80200 0x5602c2e82680 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 0x5602c2e6d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e6d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd7f8ff9640 0 client.25115 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 --2- 172.21.15.62:0/3074735411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e654a0 0x5602c2e70030 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 -- 172.21.15.62:0/3074735411 >> 172.21.15.62:0/3074735411 conn(0x5602c2e61a30 msgr2=0x5602c2e67760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 -- 172.21.15.62:0/3074735411 shutdown_connections 2024-09-11T23:52:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 1 -- 172.21.15.62:0/3074735411 wait complete. 2024-09-11T23:52:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.940+0000 7fd819baa740 20 client.25115 trim_cache size 0 max 16384 2024-09-11T23:52:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.967+0000 7fd819baa740 1 Processor -- start 2024-09-11T23:52:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.967+0000 7fd819baa740 1 -- start start 2024-09-11T23:52:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.967+0000 7fd819baa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e654a0 0x5602c2e7c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.967+0000 7fd819baa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e87f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.967+0000 7fd819baa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e88490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.967+0000 7fd819baa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4010f90 con 0x5602c2e65dc0 2024-09-11T23:52:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.967+0000 7fd819baa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f400e860 con 0x5602c2e70680 2024-09-11T23:52:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.967+0000 7fd819baa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f400c6b0 con 0x5602c2e654a0 2024-09-11T23:52:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd814d6c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e87f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd815d6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e88490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd815d6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e88490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:51192/0 (socket says 172.21.15.62:51192) 2024-09-11T23:52:46.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd815d6e640 1 -- 172.21.15.62:0/1821689635 learned_addr learned my addr 172.21.15.62:0/1821689635 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:46.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd814d6c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e87f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50626/0 (socket says 172.21.15.62:50626) 2024-09-11T23:52:46.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd81556d640 1 --2- 172.21.15.62:0/1821689635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e654a0 0x5602c2e7c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd815d6e640 1 -- 172.21.15.62:0/1821689635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e654a0 msgr2=0x5602c2e7c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:46.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd815d6e640 1 --2- 172.21.15.62:0/1821689635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e654a0 0x5602c2e7c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd815d6e640 1 -- 172.21.15.62:0/1821689635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 msgr2=0x5602c2e87f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:46.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd815d6e640 1 --2- 172.21.15.62:0/1821689635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e87f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd815d6e640 1 -- 172.21.15.62:0/1821689635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c2e72db0 con 0x5602c2e70680 2024-09-11T23:52:46.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.968+0000 7fd815d6e640 1 --2- 172.21.15.62:0/1821689635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e88490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd808000f80 tx=0x7fd808028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54405308c516f89f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.969+0000 7fd7faffd640 1 -- 172.21.15.62:0/1821689635 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808034070 con 0x5602c2e70680 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.969+0000 7fd7faffd640 1 -- 172.21.15.62:0/1821689635 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd80800ecb0 con 0x5602c2e70680 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.969+0000 7fd7faffd640 1 -- 172.21.15.62:0/1821689635 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80802a040 con 0x5602c2e70680 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.970+0000 7fd819baa740 1 -- 172.21.15.62:0/1821689635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 msgr2=0x5602c2e88490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.970+0000 7fd819baa740 1 --2- 172.21.15.62:0/1821689635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e88490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd808000f80 tx=0x7fd808028000 comp rx=0 tx=0).stop 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.970+0000 7fd819baa740 1 -- 172.21.15.62:0/1821689635 shutdown_connections 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.970+0000 7fd819baa740 1 --2- 172.21.15.62:0/1821689635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e70680 0x5602c2e88490 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.970+0000 7fd819baa740 1 --2- 172.21.15.62:0/1821689635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e65dc0 0x5602c2e87f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.970+0000 7fd819baa740 1 --2- 172.21.15.62:0/1821689635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e654a0 0x5602c2e7c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.970+0000 7fd819baa740 1 -- 172.21.15.62:0/1821689635 >> 172.21.15.62:0/1821689635 conn(0x5602c2e61a30 msgr2=0x5602c2e68870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.970+0000 7fd819baa740 1 -- 172.21.15.62:0/1821689635 shutdown_connections 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.970+0000 7fd819baa740 1 -- 172.21.15.62:0/1821689635 wait complete. 2024-09-11T23:52:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.970+0000 7fd819baa740 1 -- start start 2024-09-11T23:52:46.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.971+0000 7fd819baa740 1 Processor -- start 2024-09-11T23:52:46.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd819baa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e654a0 0x5602c2e746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd819baa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e65dc0 0x5602c2e74c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd819baa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 0x5602c2e75170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd819baa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f400cd20 con 0x5602c2e654a0 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd819baa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f400e190 con 0x5602c2e65dc0 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd819baa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f400d3f0 con 0x5602c2e70680 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd819baa740 10 client.0 _ll_register_callbacks cb 0x7fd8080093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd815d6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 0x5602c2e75170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd814d6c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e65dc0 0x5602c2e74c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd81556d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e654a0 0x5602c2e746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd815d6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 0x5602c2e75170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:45556/0 (socket says 172.21.15.62:45556) 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.972+0000 7fd815d6e640 1 -- 172.21.15.62:0/32326011 learned_addr learned my addr 172.21.15.62:0/32326011 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.973+0000 7fd814d6c640 1 -- 172.21.15.62:0/32326011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 msgr2=0x5602c2e75170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.973+0000 7fd814d6c640 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 0x5602c2e75170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.973+0000 7fd814d6c640 1 -- 172.21.15.62:0/32326011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e654a0 msgr2=0x5602c2e746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.973+0000 7fd814d6c640 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e654a0 0x5602c2e746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.973+0000 7fd814d6c640 1 -- 172.21.15.62:0/32326011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c2eea1f0 con 0x5602c2e65dc0 2024-09-11T23:52:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.973+0000 7fd814d6c640 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e65dc0 0x5602c2e74c10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd8000cd1f0 tx=0x7fd8000bc120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0a3817d3a892782 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.973+0000 7fd7f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:52:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.974+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/32326011 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000cd4a0 con 0x5602c2e65dc0 2024-09-11T23:52:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.974+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/32326011 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fd8000cd640 con 0x5602c2e65dc0 2024-09-11T23:52:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.974+0000 7fd819baa740 1 -- 172.21.15.62:0/32326011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80800ecb0 con 0x5602c2e65dc0 2024-09-11T23:52:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.974+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/32326011 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd8000bab50 con 0x5602c2e65dc0 2024-09-11T23:52:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.974+0000 7fd819baa740 10 client.25121 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.974+0000 7fd819baa740 1 -- 172.21.15.62:0/32326011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5602c2eeb560 con 0x5602c2e65dc0 2024-09-11T23:52:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.974+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/32326011 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd80800ecb0 con 0x5602c2e65dc0 2024-09-11T23:52:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.975+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/32326011 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd80009add0 con 0x5602c2e65dc0 2024-09-11T23:52:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.975+0000 7fd819baa740 10 client.25121 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.975+0000 7fd819baa740 10 client.25121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.975+0000 7fd819baa740 20 client.25121 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.975+0000 7fd819baa740 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c2eebc90 0x5602c2eee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.975+0000 7fd819baa740 4 client.25121 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:52:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.975+0000 7fd819baa740 1 -- 172.21.15.62:0/32326011 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5602c2dd2d60 con 0x5602c2eebc90 2024-09-11T23:52:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.975+0000 7fd81556d640 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c2eebc90 0x5602c2eee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd81556d640 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c2eebc90 0x5602c2eee0b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b529bfd4bb54476 server_cookie=cf50c4a0c1fa5d09 in_seq=0 out_seq=0 2024-09-11T23:52:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd7f8ff9640 10 client.25121 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd7f8ff9640 1 -- 172.21.15.62:0/32326011 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5602c2dd2d60 con 0x5602c2eebc90 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd7f8ff9640 10 client.25121 handle_command_reply: tid=0 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd819baa740 1 client.25121 shutdown 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd819baa740 1 -- 172.21.15.62:0/32326011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e65dc0 msgr2=0x5602c2e74c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd819baa740 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e65dc0 0x5602c2e74c10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd8000cd1f0 tx=0x7fd8000bc120 comp rx=0 tx=0).stop 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd819baa740 1 -- 172.21.15.62:0/32326011 shutdown_connections 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd819baa740 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c2eebc90 0x5602c2eee0b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd819baa740 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c2e70680 0x5602c2e75170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd819baa740 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c2e65dc0 0x5602c2e74c10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd819baa740 1 --2- 172.21.15.62:0/32326011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c2e654a0 0x5602c2e746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd819baa740 1 -- 172.21.15.62:0/32326011 >> 172.21.15.62:0/32326011 conn(0x5602c2e61a30 msgr2=0x5602c2e6eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.976+0000 7fd7f8ff9640 0 client.25121 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.977+0000 7fd819baa740 1 -- 172.21.15.62:0/32326011 shutdown_connections 2024-09-11T23:52:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.977+0000 7fd819baa740 1 -- 172.21.15.62:0/32326011 wait complete. 2024-09-11T23:52:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.977+0000 7fd819baa740 20 client.25121 trim_cache size 0 max 16384 2024-09-11T23:52:46.979 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:52:46.979 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-11T23:52:46.979 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-11T23:52:46.979 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:52:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.979+0000 7fd8177d1640 1 -- 172.21.15.62:0/3012186948 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd7fc0a2e80 msgr2=0x7fd7fc0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.979+0000 7fd8177d1640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd7fc0a2e80 0x7fd7fc0a5340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd808021000 tx=0x7fd80801e000 comp rx=0 tx=0).stop 2024-09-11T23:52:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.979+0000 7fd8177d1640 1 -- 172.21.15.62:0/3012186948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd810101410 msgr2=0x7fd8101965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.979+0000 7fd8177d1640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd810101410 0x7fd8101965f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd800009750 tx=0x7fd800009c10 comp rx=0 tx=0).stop 2024-09-11T23:52:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.979+0000 7fd8177d1640 1 -- 172.21.15.62:0/3012186948 shutdown_connections 2024-09-11T23:52:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.979+0000 7fd8177d1640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd7fc0a2e80 0x7fd7fc0a5340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.979+0000 7fd8177d1640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd810106a70 0x7fd810196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.979+0000 7fd8177d1640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd810101410 0x7fd8101965f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.979+0000 7fd8177d1640 1 --2- 172.21.15.62:0/3012186948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd810100a70 0x7fd8101960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.979+0000 7fd8177d1640 1 -- 172.21.15.62:0/3012186948 >> 172.21.15.62:0/3012186948 conn(0x7fd8100fc7d0 msgr2=0x7fd8100fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.980+0000 7fd8177d1640 1 -- 172.21.15.62:0/3012186948 shutdown_connections 2024-09-11T23:52:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:46.980+0000 7fd8177d1640 1 -- 172.21.15.62:0/3012186948 wait complete. 2024-09-11T23:52:47.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:47 smithi083 ceph-mon[34805]: pgmap v23: 97 pgs: 97 active+clean; 381 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 59 MiB/s wr, 142 op/s 2024-09-11T23:52:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 5.0 s, 3543.2 tps, lat 27.670 ms stddev 27.420 2024-09-11T23:52:47.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:47 smithi062 ceph-mon[30958]: pgmap v23: 97 pgs: 97 active+clean; 381 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 59 MiB/s wr, 142 op/s 2024-09-11T23:52:47.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:47 smithi120 ceph-mon[35182]: pgmap v23: 97 pgs: 97 active+clean; 381 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 59 MiB/s wr, 142 op/s 2024-09-11T23:52:47.672 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:52:47.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:52:48.043 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:48.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.566+0000 7f48592bf640 1 -- 172.21.15.62:0/2477195268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4854102570 msgr2=0x7f48541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:48.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.566+0000 7f48592bf640 1 --2- 172.21.15.62:0/2477195268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4854102570 0x7f48541029b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4848009930 tx=0x7f484802a4f0 comp rx=0 tx=0).stop 2024-09-11T23:52:48.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.567+0000 7f48592bf640 1 -- 172.21.15.62:0/2477195268 shutdown_connections 2024-09-11T23:52:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.567+0000 7f48592bf640 1 --2- 172.21.15.62:0/2477195268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4854102ef0 0x7f485410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.567+0000 7f48592bf640 1 --2- 172.21.15.62:0/2477195268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4854102570 0x7f48541029b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.567+0000 7f48592bf640 1 --2- 172.21.15.62:0/2477195268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4854100e20 0x7f4854101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.567+0000 7f48592bf640 1 -- 172.21.15.62:0/2477195268 >> 172.21.15.62:0/2477195268 conn(0x7f48540fc7d0 msgr2=0x7f48540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.567+0000 7f48592bf640 1 -- 172.21.15.62:0/2477195268 shutdown_connections 2024-09-11T23:52:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.567+0000 7f48592bf640 1 -- 172.21.15.62:0/2477195268 wait complete. 2024-09-11T23:52:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.568+0000 7f48592bf640 1 Processor -- start 2024-09-11T23:52:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.568+0000 7f48592bf640 1 -- start start 2024-09-11T23:52:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.568+0000 7f48592bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4854100e20 0x7f48541a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f48592bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4854102570 0x7f48541a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f48592bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4854102ef0 0x7f485419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f4852ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4854100e20 0x7f48541a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f48592bf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f485410ff20 con 0x7f4854100e20 2024-09-11T23:52:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f48592bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f485410fda0 con 0x7f4854102ef0 2024-09-11T23:52:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f48592bf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48541100a0 con 0x7f4854102570 2024-09-11T23:52:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f48527fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4854102570 0x7f48541a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f48527fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4854102570 0x7f48541a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:45566/0 (socket says 172.21.15.62:45566) 2024-09-11T23:52:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f48527fc640 1 -- 172.21.15.62:0/179357311 learned_addr learned my addr 172.21.15.62:0/179357311 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f48527fc640 1 -- 172.21.15.62:0/179357311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4854102ef0 msgr2=0x7f485419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.569+0000 7f48537fe640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4854102ef0 0x7f485419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.570+0000 7f48527fc640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4854102ef0 0x7f485419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.570+0000 7f48527fc640 1 -- 172.21.15.62:0/179357311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4854100e20 msgr2=0x7f48541a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.570+0000 7f48527fc640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4854100e20 0x7f48541a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.570+0000 7f48527fc640 1 -- 172.21.15.62:0/179357311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485419af00 con 0x7f4854102570 2024-09-11T23:52:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.570+0000 7f48537fe640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4854102ef0 0x7f485419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:52:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.570+0000 7f4852ffd640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4854100e20 0x7f48541a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:52:48.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.570+0000 7f48527fc640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4854102570 0x7f48541a0b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f484802aa60 tx=0x7f4848003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:48.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.571+0000 7f483bfff640 1 -- 172.21.15.62:0/179357311 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484803f070 con 0x7f4854102570 2024-09-11T23:52:48.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.571+0000 7f483bfff640 1 -- 172.21.15.62:0/179357311 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4848031c90 con 0x7f4854102570 2024-09-11T23:52:48.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.571+0000 7f483bfff640 1 -- 172.21.15.62:0/179357311 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848035030 con 0x7f4854102570 2024-09-11T23:52:48.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.571+0000 7f48592bf640 1 -- 172.21.15.62:0/179357311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485419b190 con 0x7f4854102570 2024-09-11T23:52:48.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.571+0000 7f48592bf640 1 -- 172.21.15.62:0/179357311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48541a7370 con 0x7f4854102570 2024-09-11T23:52:48.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.573+0000 7f48592bf640 1 -- 172.21.15.62:0/179357311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48541021b0 con 0x7f4854102570 2024-09-11T23:52:48.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.574+0000 7f483bfff640 1 -- 172.21.15.62:0/179357311 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f4848042070 con 0x7f4854102570 2024-09-11T23:52:48.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.576+0000 7f483bfff640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f483c0a2f20 0x7f483c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:48.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.576+0000 7f483bfff640 1 -- 172.21.15.62:0/179357311 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f48480efd80 con 0x7f4854102570 2024-09-11T23:52:48.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.577+0000 7f4852ffd640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f483c0a2f20 0x7f483c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:48.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.578+0000 7f4852ffd640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f483c0a2f20 0x7f483c0a53e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f484400a8b0 tx=0x7f4844008040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:48.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.585+0000 7f483bfff640 1 -- 172.21.15.62:0/179357311 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48480bc860 con 0x7f4854102570 2024-09-11T23:52:48.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.943+0000 7f48592bf640 1 -- 172.21.15.62:0/179357311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f485419bbc0 con 0x7f4854102570 2024-09-11T23:52:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.945+0000 7f483bfff640 1 -- 172.21.15.62:0/179357311 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f48480bc160 con 0x7f4854102570 2024-09-11T23:52:48.944 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:48.944 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"btime":"2024-09-11T23:52:04:422374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:04.372188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:52:48.945 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2024-09-11T23:52:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.948+0000 7f48592bf640 1 -- 172.21.15.62:0/179357311 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f483c0a2f20 msgr2=0x7f483c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.948+0000 7f48592bf640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f483c0a2f20 0x7f483c0a53e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f484400a8b0 tx=0x7f4844008040 comp rx=0 tx=0).stop 2024-09-11T23:52:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.948+0000 7f48592bf640 1 -- 172.21.15.62:0/179357311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4854102570 msgr2=0x7f48541a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.948+0000 7f48592bf640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4854102570 0x7f48541a0b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f484802aa60 tx=0x7f4848003b90 comp rx=0 tx=0).stop 2024-09-11T23:52:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.948+0000 7f48592bf640 1 -- 172.21.15.62:0/179357311 shutdown_connections 2024-09-11T23:52:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.948+0000 7f48592bf640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f483c0a2f20 0x7f483c0a53e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.948+0000 7f48592bf640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4854102ef0 0x7f485419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:48.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.948+0000 7f48592bf640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4854102570 0x7f48541a0b00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:48.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.948+0000 7f48592bf640 1 --2- 172.21.15.62:0/179357311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4854100e20 0x7f48541a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:48.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.948+0000 7f48592bf640 1 -- 172.21.15.62:0/179357311 >> 172.21.15.62:0/179357311 conn(0x7f48540fc7d0 msgr2=0x7f485410bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:48.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.949+0000 7f48592bf640 1 -- 172.21.15.62:0/179357311 shutdown_connections 2024-09-11T23:52:48.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:48.949+0000 7f48592bf640 1 -- 172.21.15.62:0/179357311 wait complete. 2024-09-11T23:52:49.163 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-11T23:52:49.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:52:49.201 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:49 smithi062 ceph-mon[30958]: pgmap v24: 97 pgs: 97 active+clean; 834 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 77 MiB/s wr, 287 op/s 2024-09-11T23:52:49.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:49 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/179357311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:49.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:49 smithi083 ceph-mon[34805]: pgmap v24: 97 pgs: 97 active+clean; 834 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 77 MiB/s wr, 287 op/s 2024-09-11T23:52:49.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:49 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/179357311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:49.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:49 smithi120 ceph-mon[35182]: pgmap v24: 97 pgs: 97 active+clean; 834 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 77 MiB/s wr, 287 op/s 2024-09-11T23:52:49.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:49 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/179357311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:49.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:52:49.893 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:50.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.469+0000 7f7c70457640 1 -- 172.21.15.62:0/1484934608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c68102d30 msgr2=0x7f7c68103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.469+0000 7f7c70457640 1 --2- 172.21.15.62:0/1484934608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c68102d30 0x7f7c68103110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009930 tx=0x7f7c6402a420 comp rx=0 tx=0).stop 2024-09-11T23:52:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.470+0000 7f7c70457640 1 -- 172.21.15.62:0/1484934608 shutdown_connections 2024-09-11T23:52:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.470+0000 7f7c70457640 1 --2- 172.21.15.62:0/1484934608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c681082a0 0x7f7c6810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.470+0000 7f7c70457640 1 --2- 172.21.15.62:0/1484934608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c681036e0 0x7f7c68107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.470+0000 7f7c70457640 1 --2- 172.21.15.62:0/1484934608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c68102d30 0x7f7c68103110 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.470+0000 7f7c70457640 1 -- 172.21.15.62:0/1484934608 >> 172.21.15.62:0/1484934608 conn(0x7f7c680fc7d0 msgr2=0x7f7c680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:50.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.471+0000 7f7c70457640 1 -- 172.21.15.62:0/1484934608 shutdown_connections 2024-09-11T23:52:50.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.471+0000 7f7c70457640 1 -- 172.21.15.62:0/1484934608 wait complete. 2024-09-11T23:52:50.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c70457640 1 Processor -- start 2024-09-11T23:52:50.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c70457640 1 -- start start 2024-09-11T23:52:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c70457640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c68102d30 0x7f7c681a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c70457640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c681036e0 0x7f7c681a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c70457640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c681082a0 0x7f7c6810dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c70457640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68112090 con 0x7f7c681036e0 2024-09-11T23:52:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c70457640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68111f10 con 0x7f7c68102d30 2024-09-11T23:52:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c70457640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68112210 con 0x7f7c681082a0 2024-09-11T23:52:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c6d9f2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c681036e0 0x7f7c681a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c6d9f2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c681036e0 0x7f7c681a4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50664/0 (socket says 172.21.15.62:50664) 2024-09-11T23:52:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c6d9f2640 1 -- 172.21.15.62:0/2492777645 learned_addr learned my addr 172.21.15.62:0/2492777645 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.472+0000 7f7c6e1f3640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c68102d30 0x7f7c681a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.473+0000 7f7c6e9f4640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c681082a0 0x7f7c6810dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.473+0000 7f7c6e1f3640 1 -- 172.21.15.62:0/2492777645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c681082a0 msgr2=0x7f7c6810dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.473+0000 7f7c6e1f3640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c681082a0 0x7f7c6810dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.473+0000 7f7c6e1f3640 1 -- 172.21.15.62:0/2492777645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c681036e0 msgr2=0x7f7c681a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.473+0000 7f7c6e1f3640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c681036e0 0x7f7c681a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.473+0000 7f7c6e1f3640 1 -- 172.21.15.62:0/2492777645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6810e490 con 0x7f7c68102d30 2024-09-11T23:52:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.473+0000 7f7c6d9f2640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c681036e0 0x7f7c681a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.473+0000 7f7c6e9f4640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c681082a0 0x7f7c6810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.474+0000 7f7c6e1f3640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c68102d30 0x7f7c681a4790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c640045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.475+0000 7f7c5b7fe640 1 -- 172.21.15.62:0/2492777645 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64035030 con 0x7f7c68102d30 2024-09-11T23:52:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.475+0000 7f7c5b7fe640 1 -- 172.21.15.62:0/2492777645 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c64003de0 con 0x7f7c68102d30 2024-09-11T23:52:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.475+0000 7f7c70457640 1 -- 172.21.15.62:0/2492777645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6810e720 con 0x7f7c68102d30 2024-09-11T23:52:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.475+0000 7f7c5b7fe640 1 -- 172.21.15.62:0/2492777645 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c640319b0 con 0x7f7c68102d30 2024-09-11T23:52:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.475+0000 7f7c70457640 1 -- 172.21.15.62:0/2492777645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c681ab660 con 0x7f7c68102d30 2024-09-11T23:52:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.480+0000 7f7c5b7fe640 1 -- 172.21.15.62:0/2492777645 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f7c64003900 con 0x7f7c68102d30 2024-09-11T23:52:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.482+0000 7f7c5b7fe640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7c540a2f50 0x7f7c540a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:50.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.482+0000 7f7c5b7fe640 1 -- 172.21.15.62:0/2492777645 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f7c640efd90 con 0x7f7c68102d30 2024-09-11T23:52:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.483+0000 7f7c6d9f2640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7c540a2f50 0x7f7c540a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.484+0000 7f7c70457640 1 -- 172.21.15.62:0/2492777645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c68103110 con 0x7f7c68102d30 2024-09-11T23:52:50.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.492+0000 7f7c6d9f2640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7c540a2f50 0x7f7c540a5410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c68100a90 tx=0x7f7c5c00b440 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:50.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.494+0000 7f7c5b7fe640 1 -- 172.21.15.62:0/2492777645 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c6404a030 con 0x7f7c68102d30 2024-09-11T23:52:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:50.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.898+0000 7f7c70457640 1 -- 172.21.15.62:0/2492777645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c68107270 con 0x7f7c68102d30 2024-09-11T23:52:50.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.899+0000 7f7c5b7fe640 1 -- 172.21.15.62:0/2492777645 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f7c64041020 con 0x7f7c68102d30 2024-09-11T23:52:50.899 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:52:50.899 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"btime":"2024-09-11T23:52:04:422374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:04.372188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:52:50.899 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2024-09-11T23:52:50.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.905+0000 7f7c70457640 1 -- 172.21.15.62:0/2492777645 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7c540a2f50 msgr2=0x7f7c540a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:50.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.905+0000 7f7c70457640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7c540a2f50 0x7f7c540a5410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c68100a90 tx=0x7f7c5c00b440 comp rx=0 tx=0).stop 2024-09-11T23:52:50.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.905+0000 7f7c70457640 1 -- 172.21.15.62:0/2492777645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c68102d30 msgr2=0x7f7c681a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:50.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.905+0000 7f7c70457640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c68102d30 0x7f7c681a4790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c640045e0 comp rx=0 tx=0).stop 2024-09-11T23:52:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.909+0000 7f7c70457640 1 -- 172.21.15.62:0/2492777645 shutdown_connections 2024-09-11T23:52:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.909+0000 7f7c70457640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7c540a2f50 0x7f7c540a5410 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.909+0000 7f7c70457640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c681082a0 0x7f7c6810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.909+0000 7f7c70457640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c681036e0 0x7f7c681a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.909+0000 7f7c70457640 1 --2- 172.21.15.62:0/2492777645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c68102d30 0x7f7c681a4790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.909+0000 7f7c70457640 1 -- 172.21.15.62:0/2492777645 >> 172.21.15.62:0/2492777645 conn(0x7f7c680fc7d0 msgr2=0x7f7c68101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.909+0000 7f7c70457640 1 -- 172.21.15.62:0/2492777645 shutdown_connections 2024-09-11T23:52:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:50.910+0000 7f7c70457640 1 -- 172.21.15.62:0/2492777645 wait complete. 2024-09-11T23:52:51.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.137+0000 7fe5caa13640 1 -- 172.21.15.62:0/3250242959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c4102570 msgr2=0x7fe5c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:51.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.137+0000 7fe5caa13640 1 --2- 172.21.15.62:0/3250242959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c4102570 0x7fe5c41029b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009900 tx=0x7fe5b802a480 comp rx=0 tx=0).stop 2024-09-11T23:52:51.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.137+0000 7fe5caa13640 1 -- 172.21.15.62:0/3250242959 shutdown_connections 2024-09-11T23:52:51.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.137+0000 7fe5caa13640 1 --2- 172.21.15.62:0/3250242959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5c4102ef0 0x7fe5c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.137+0000 7fe5caa13640 1 --2- 172.21.15.62:0/3250242959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c4102570 0x7fe5c41029b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.137+0000 7fe5caa13640 1 --2- 172.21.15.62:0/3250242959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4100e20 0x7fe5c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.137+0000 7fe5caa13640 1 -- 172.21.15.62:0/3250242959 >> 172.21.15.62:0/3250242959 conn(0x7fe5c40fc7d0 msgr2=0x7fe5c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:51.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.137+0000 7fe5caa13640 1 -- 172.21.15.62:0/3250242959 shutdown_connections 2024-09-11T23:52:51.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.137+0000 7fe5caa13640 1 -- 172.21.15.62:0/3250242959 wait complete. 2024-09-11T23:52:51.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.138+0000 7fe5caa13640 1 Processor -- start 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.138+0000 7fe5caa13640 1 -- start start 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5caa13640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c4100e20 0x7fe5c419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5caa13640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4102570 0x7fe5c419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5caa13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5c4102ef0 0x7fe5c419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5caa13640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c410fec0 con 0x7fe5c4100e20 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5caa13640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c410fd40 con 0x7fe5c4102ef0 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5caa13640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4110040 con 0x7fe5c4102570 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4102570 0x7fe5c419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4102570 0x7fe5c419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53308/0 (socket says 172.21.15.62:53308) 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5c37fe640 1 -- 172.21.15.62:0/2582328892 learned_addr learned my addr 172.21.15.62:0/2582328892 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c4100e20 0x7fe5c419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.139+0000 7fe5c8fb0640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5c4102ef0 0x7fe5c419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.140+0000 7fe5c37fe640 1 -- 172.21.15.62:0/2582328892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5c4102ef0 msgr2=0x7fe5c419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.140+0000 7fe5c37fe640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5c4102ef0 0x7fe5c419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.140+0000 7fe5c37fe640 1 -- 172.21.15.62:0/2582328892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c4100e20 msgr2=0x7fe5c419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.140+0000 7fe5c37fe640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c4100e20 0x7fe5c419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.140+0000 7fe5c37fe640 1 -- 172.21.15.62:0/2582328892 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c419f680 con 0x7fe5c4102570 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.140+0000 7fe5c8fb0640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5c4102ef0 0x7fe5c419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.140+0000 7fe5c37fe640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4102570 0x7fe5c419ac10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe5b802aa70 tx=0x7fe5b8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.141+0000 7fe5c17fa640 1 -- 172.21.15.62:0/2582328892 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b803f070 con 0x7fe5c4102570 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.141+0000 7fe5c17fa640 1 -- 172.21.15.62:0/2582328892 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b8031c30 con 0x7fe5c4102570 2024-09-11T23:52:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.141+0000 7fe5c17fa640 1 -- 172.21.15.62:0/2582328892 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8035030 con 0x7fe5c4102570 2024-09-11T23:52:51.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.141+0000 7fe5caa13640 1 -- 172.21.15.62:0/2582328892 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c419f910 con 0x7fe5c4102570 2024-09-11T23:52:51.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.141+0000 7fe5caa13640 1 -- 172.21.15.62:0/2582328892 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c419fce0 con 0x7fe5c4102570 2024-09-11T23:52:51.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.144+0000 7fe5c17fa640 1 -- 172.21.15.62:0/2582328892 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fe5b8042070 con 0x7fe5c4102570 2024-09-11T23:52:51.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.145+0000 7fe5ccdec740 1 Processor -- start 2024-09-11T23:52:51.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.146+0000 7fe5c17fa640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5ac0a2e80 0x7fe5ac0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.146+0000 7fe5c17fa640 1 -- 172.21.15.62:0/2582328892 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe5b80efd10 con 0x7fe5c4102570 2024-09-11T23:52:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.147+0000 7fe5ccdec740 1 -- start start 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.147+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1032d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.147+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 0x55b1032d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.147+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 0x55b1032edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.147+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b10323e2f0 con 0x55b1032d9b40 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.147+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b1031e5cf0 con 0x55b1032d85d0 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.147+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b10323a2b0 con 0x55b1032ebb80 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 0x55b1032d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c8fb0640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5ac0a2e80 0x7fe5ac0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c8fb0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 0x55b1032edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c8fb0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 0x55b1032edf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53314/0 (socket says 172.21.15.62:53314) 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c8fb0640 1 -- 172.21.15.62:0/3048824811 learned_addr learned my addr 172.21.15.62:0/3048824811 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/3048824811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1032d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c8fb0640 1 -- 172.21.15.62:0/3048824811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 msgr2=0x55b1032d8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c8fb0640 1 --2- 172.21.15.62:0/3048824811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 0x55b1032d8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c8fb0640 1 -- 172.21.15.62:0/3048824811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 msgr2=0x55b1032d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c8fb0640 1 --2- 172.21.15.62:0/3048824811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1032d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c8fb0640 1 -- 172.21.15.62:0/3048824811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1031ca3a0 con 0x55b1032ebb80 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.148+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/3048824811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1032d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.149+0000 7fe5c37fe640 1 --2- 172.21.15.62:0/3048824811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 0x55b1032d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.149+0000 7fe5c8fb0640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5ac0a2e80 0x7fe5ac0a5340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc013c70 tx=0x7fe5bc0137f0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.149+0000 7fe5c8fb0640 1 --2- 172.21.15.62:0/3048824811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 0x55b1032edf70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc011c40 tx=0x7fe5bc01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=136484118aa3771a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.149+0000 7fe5aa7fc640 1 -- 172.21.15.62:0/3048824811 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0138e0 con 0x55b1032ebb80 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.149+0000 7fe5aa7fc640 1 -- 172.21.15.62:0/3048824811 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5bc00fc60 con 0x55b1032ebb80 2024-09-11T23:52:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.149+0000 7fe5aa7fc640 1 -- 172.21.15.62:0/3048824811 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc004790 con 0x55b1032ebb80 2024-09-11T23:52:51.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.152+0000 7fe5ccdec740 1 -- 172.21.15.62:0/3048824811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 msgr2=0x55b1032edf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:51.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.152+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/3048824811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 0x55b1032edf70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc011c40 tx=0x7fe5bc01b000 comp rx=0 tx=0).stop 2024-09-11T23:52:51.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.152+0000 7fe5ccdec740 1 -- 172.21.15.62:0/3048824811 shutdown_connections 2024-09-11T23:52:51.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.153+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/3048824811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 0x55b1032edf70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.153+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/3048824811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 0x55b1032d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.153+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/3048824811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1032d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.153+0000 7fe5ccdec740 1 -- 172.21.15.62:0/3048824811 >> 172.21.15.62:0/3048824811 conn(0x55b1032a1310 msgr2=0x55b1032d4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:51.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.153+0000 7fe5ccdec740 1 -- 172.21.15.62:0/3048824811 shutdown_connections 2024-09-11T23:52:51.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.153+0000 7fe5ccdec740 1 -- 172.21.15.62:0/3048824811 wait complete. 2024-09-11T23:52:51.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.153+0000 7fe5ccdec740 1 -- start start 2024-09-11T23:52:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.153+0000 7fe5ccdec740 1 Processor -- start 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1032e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 0x55b1032e9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b1032ee820 con 0x55b1032d9b40 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b1031f9060 con 0x55b1032d85d0 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b1032ee9a0 con 0x55b1032ebb80 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5ccdec740 10 client.0 _ll_register_callbacks cb 0x55b102f4c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5c37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1032e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5c37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1032e88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41544/0 (socket says 172.21.15.62:41544) 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5c3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5c37fe640 1 -- 172.21.15.62:0/2254233656 learned_addr learned my addr 172.21.15.62:0/2254233656 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5c8fb0640 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 0x55b1032e9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5c3fff640 1 -- 172.21.15.62:0/2254233656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 msgr2=0x55b1032e9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:51.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 0x55b1032e9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5c3fff640 1 -- 172.21.15.62:0/2254233656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 msgr2=0x55b1032e88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:51.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1032e88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.154+0000 7fe5c3fff640 1 -- 172.21.15.62:0/2254233656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b102faf3c0 con 0x55b1032d85d0 2024-09-11T23:52:51.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.155+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400a530 tx=0x7fe5b400a9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96c530127c9cc9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:51.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.155+0000 7fe5a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:52:51.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.155+0000 7fe5a37fe640 1 -- 172.21.15.62:0/2254233656 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40035f0 con 0x55b1032d85d0 2024-09-11T23:52:51.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.155+0000 7fe5ccdec740 1 -- 172.21.15.62:0/2254233656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5bc00fc60 con 0x55b1032d85d0 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.155+0000 7fe5a37fe640 1 -- 172.21.15.62:0/2254233656 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe5b4003790 con 0x55b1032d85d0 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.155+0000 7fe5a37fe640 1 -- 172.21.15.62:0/2254233656 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe5b4025050 con 0x55b1032d85d0 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.155+0000 7fe5ccdec740 10 client.25133 fetch_fsmap learned FSMap version 25 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.155+0000 7fe5ccdec740 1 -- 172.21.15.62:0/2254233656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b1032fa370 con 0x55b1032d85d0 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.155+0000 7fe5a37fe640 1 -- 172.21.15.62:0/2254233656 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5bc00fc60 con 0x55b1032d85d0 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.155+0000 7fe5a37fe640 1 -- 172.21.15.62:0/2254233656 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe5b4017a40 con 0x55b1032d85d0 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.156+0000 7fe5ccdec740 10 client.25133 fetch_fsmap finished waiting for FSMap version 25 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.156+0000 7fe5ccdec740 10 client.25133 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.156+0000 7fe5ccdec740 20 client.25133 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.156+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1032fb7d0 0x55b1032fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.156+0000 7fe5ccdec740 4 client.25133 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.156+0000 7fe5ccdec740 1 -- 172.21.15.62:0/2254233656 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b10324b860 con 0x55b1032fb7d0 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.156+0000 7fe5c37fe640 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1032fb7d0 0x55b1032fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.159+0000 7fe5c37fe640 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1032fb7d0 0x55b1032fdc00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39abd05397477cb server_cookie=e8c106160decc53e in_seq=0 out_seq=0 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.159+0000 7fe5a37fe640 10 client.25133 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.159+0000 7fe5a37fe640 1 -- 172.21.15.62:0/2254233656 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b10324b860 con 0x55b1032fb7d0 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.159+0000 7fe5a37fe640 10 client.25133 handle_command_reply: tid=0 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.159+0000 7fe5ccdec740 1 client.25133 shutdown 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 1 -- 172.21.15.62:0/2254233656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 msgr2=0x55b1032eb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400a530 tx=0x7fe5b400a9f0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 1 -- 172.21.15.62:0/2254233656 shutdown_connections 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1032fb7d0 0x55b1032fdc00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5a37fe640 0 client.25133 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:52:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032ebb80 0x55b1032e9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1032e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/2254233656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 1 -- 172.21.15.62:0/2254233656 >> 172.21.15.62:0/2254233656 conn(0x55b1032a1310 msgr2=0x55b1032da960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 1 -- 172.21.15.62:0/2254233656 shutdown_connections 2024-09-11T23:52:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 1 -- 172.21.15.62:0/2254233656 wait complete. 2024-09-11T23:52:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.160+0000 7fe5ccdec740 20 client.25133 trim_cache size 0 max 16384 2024-09-11T23:52:51.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5ccdec740 1 Processor -- start 2024-09-11T23:52:51.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5ccdec740 1 -- start start 2024-09-11T23:52:51.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1033034d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032ebb80 0x55b103303a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41aded0 con 0x55b1032d9b40 2024-09-11T23:52:51.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41ab7a0 con 0x55b1032ebb80 2024-09-11T23:52:51.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41a9610 con 0x55b1032d85d0 2024-09-11T23:52:51.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:51 smithi062 ceph-mon[30958]: pgmap v25: 97 pgs: 97 active+clean; 834 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 74 MiB/s wr, 182 op/s 2024-09-11T23:52:51.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:51 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2492777645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5c8fb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032ebb80 0x55b103303a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5c3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.187+0000 7fe5c3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53342/0 (socket says 172.21.15.62:53342) 2024-09-11T23:52:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.188+0000 7fe5c3fff640 1 -- 172.21.15.62:0/1359591563 learned_addr learned my addr 172.21.15.62:0/1359591563 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.188+0000 7fe5c3fff640 1 -- 172.21.15.62:0/1359591563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032ebb80 msgr2=0x55b103303a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.188+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/1359591563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032ebb80 0x55b103303a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.188+0000 7fe5c3fff640 1 -- 172.21.15.62:0/1359591563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 msgr2=0x55b1033034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.188+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/1359591563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1033034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.188+0000 7fe5c3fff640 1 -- 172.21.15.62:0/1359591563 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1032d44f0 con 0x55b1032d85d0 2024-09-11T23:52:51.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.188+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/1359591563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4001eb0 tx=0x7fe5b4005980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39695381fe2e67c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:51.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.188+0000 7fe5a9ffb640 1 -- 172.21.15.62:0/1359591563 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40055a0 con 0x55b1032d85d0 2024-09-11T23:52:51.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.188+0000 7fe5a9ffb640 1 -- 172.21.15.62:0/1359591563 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b401c070 con 0x55b1032d85d0 2024-09-11T23:52:51.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.189+0000 7fe5a9ffb640 1 -- 172.21.15.62:0/1359591563 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4003d00 con 0x55b1032d85d0 2024-09-11T23:52:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.190+0000 7fe5ccdec740 1 -- 172.21.15.62:0/1359591563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 msgr2=0x55b1032eb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.190+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/1359591563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4001eb0 tx=0x7fe5b4005980 comp rx=0 tx=0).stop 2024-09-11T23:52:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.191+0000 7fe5ccdec740 1 -- 172.21.15.62:0/1359591563 shutdown_connections 2024-09-11T23:52:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.191+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/1359591563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032ebb80 0x55b103303a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.191+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/1359591563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032d9b40 0x55b1033034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.191+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/1359591563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032eb5c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.192+0000 7fe5ccdec740 1 -- 172.21.15.62:0/1359591563 >> 172.21.15.62:0/1359591563 conn(0x55b1032a1310 msgr2=0x55b1032cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.192+0000 7fe5ccdec740 1 -- 172.21.15.62:0/1359591563 shutdown_connections 2024-09-11T23:52:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.192+0000 7fe5ccdec740 1 -- 172.21.15.62:0/1359591563 wait complete. 2024-09-11T23:52:51.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.192+0000 7fe5ccdec740 1 -- start start 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5ccdec740 1 Processor -- start 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032f4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d9b40 0x55b1032e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5ccdec740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032ebb80 0x55b1032e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41a9c60 con 0x55b1032ebb80 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41ab0d0 con 0x55b1032d9b40 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5ccdec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41aa330 con 0x55b1032d85d0 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5ccdec740 10 client.0 _ll_register_callbacks cb 0x7fe5b4005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5c3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5c3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53346/0 (socket says 172.21.15.62:53346) 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5c3fff640 1 -- 172.21.15.62:0/360699130 learned_addr learned my addr 172.21.15.62:0/360699130 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5c8fb0640 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032ebb80 0x55b1032e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.193+0000 7fe5c37fe640 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d9b40 0x55b1032e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.194+0000 7fe5c3fff640 1 -- 172.21.15.62:0/360699130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d9b40 msgr2=0x55b1032e5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.194+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d9b40 0x55b1032e5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.194+0000 7fe5c3fff640 1 -- 172.21.15.62:0/360699130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032ebb80 msgr2=0x55b1032e58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.194+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032ebb80 0x55b1032e58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.194+0000 7fe5c3fff640 1 -- 172.21.15.62:0/360699130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1032e5e80 con 0x55b1032d85d0 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.194+0000 7fe5c3fff640 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032f4100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400bc50 tx=0x7fe5b40053a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=610ae9aaa2bb7692 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.194+0000 7fe5a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.194+0000 7fe5a37fe640 1 -- 172.21.15.62:0/360699130 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4000e60 con 0x55b1032d85d0 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.194+0000 7fe5a37fe640 1 -- 172.21.15.62:0/360699130 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe5b4003b80 con 0x55b1032d85d0 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.194+0000 7fe5ccdec740 1 -- 172.21.15.62:0/360699130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b401c070 con 0x55b1032d85d0 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.195+0000 7fe5a37fe640 1 -- 172.21.15.62:0/360699130 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe5b4003dc0 con 0x55b1032d85d0 2024-09-11T23:52:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.195+0000 7fe5ccdec740 10 client.25120 fetch_fsmap learned FSMap version 26 2024-09-11T23:52:51.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.195+0000 7fe5ccdec740 1 -- 172.21.15.62:0/360699130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b103366210 con 0x55b1032d85d0 2024-09-11T23:52:51.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.195+0000 7fe5a37fe640 1 -- 172.21.15.62:0/360699130 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b4017960 con 0x55b1032d85d0 2024-09-11T23:52:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.196+0000 7fe5a37fe640 1 -- 172.21.15.62:0/360699130 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fe5b4008ac0 con 0x55b1032d85d0 2024-09-11T23:52:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.196+0000 7fe5ccdec740 10 client.25120 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:52:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.196+0000 7fe5ccdec740 10 client.25120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:52:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.196+0000 7fe5ccdec740 20 client.25120 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.196+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1032fb7d0 0x55b1033688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.196+0000 7fe5ccdec740 4 client.25120 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:52:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.196+0000 7fe5ccdec740 1 -- 172.21.15.62:0/360699130 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe5b401c070 con 0x55b1032fb7d0 2024-09-11T23:52:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.196+0000 7fe5c37fe640 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1032fb7d0 0x55b1033688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.196+0000 7fe5c37fe640 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1032fb7d0 0x55b1033688d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ec5660c68494cec server_cookie=193fea29814d480c in_seq=0 out_seq=0 2024-09-11T23:52:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.196+0000 7fe5a37fe640 10 client.25120 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5a37fe640 1 -- 172.21.15.62:0/360699130 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe5b401c070 con 0x55b1032fb7d0 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5a37fe640 10 client.25120 handle_command_reply: tid=0 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 client.25120 shutdown 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 -- 172.21.15.62:0/360699130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 msgr2=0x55b1032f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032f4100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400bc50 tx=0x7fe5b40053a0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 -- 172.21.15.62:0/360699130 shutdown_connections 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1032fb7d0 0x55b1033688d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1032ebb80 0x55b1032e58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5a37fe640 0 client.25120 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1032d9b40 0x55b1032e5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 --2- 172.21.15.62:0/360699130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1032d85d0 0x55b1032f4100 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 -- 172.21.15.62:0/360699130 >> 172.21.15.62:0/360699130 conn(0x55b1032a1310 msgr2=0x55b1032d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 -- 172.21.15.62:0/360699130 shutdown_connections 2024-09-11T23:52:51.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 1 -- 172.21.15.62:0/360699130 wait complete. 2024-09-11T23:52:51.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.197+0000 7fe5ccdec740 20 client.25120 trim_cache size 0 max 16384 2024-09-11T23:52:51.200 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:52:51.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 -- 172.21.15.62:0/2582328892 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5ac0a2e80 msgr2=0x7fe5ac0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5ac0a2e80 0x7fe5ac0a5340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc013c70 tx=0x7fe5bc0137f0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 -- 172.21.15.62:0/2582328892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4102570 msgr2=0x7fe5c419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4102570 0x7fe5c419ac10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe5b802aa70 tx=0x7fe5b8003c10 comp rx=0 tx=0).stop 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 -- 172.21.15.62:0/2582328892 shutdown_connections 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5ac0a2e80 0x7fe5ac0a5340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5c4102ef0 0x7fe5c419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4102570 0x7fe5c419ac10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 --2- 172.21.15.62:0/2582328892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5c4100e20 0x7fe5c419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 -- 172.21.15.62:0/2582328892 >> 172.21.15.62:0/2582328892 conn(0x7fe5c40fc7d0 msgr2=0x7fe5c41068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 -- 172.21.15.62:0/2582328892 shutdown_connections 2024-09-11T23:52:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:51.198+0000 7fe5caa13640 1 -- 172.21.15.62:0/2582328892 wait complete. 2024-09-11T23:52:51.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:51 smithi120 ceph-mon[35182]: pgmap v25: 97 pgs: 97 active+clean; 834 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 74 MiB/s wr, 182 op/s 2024-09-11T23:52:51.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:51 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2492777645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:51.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:51 smithi083 ceph-mon[34805]: pgmap v25: 97 pgs: 97 active+clean; 834 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 74 MiB/s wr, 182 op/s 2024-09-11T23:52:51.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:51 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2492777645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:52:52.004 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:52:52.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:52 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:52:52.445 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-11T23:52:52.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-11T23:52:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 10.0 s, 3242.6 tps, lat 30.789 ms stddev 32.183 2024-09-11T23:52:52.525 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:52.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:52 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:52:52.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:52 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:52:53.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.074+0000 7fb6616c1640 1 -- 172.21.15.62:0/1689446624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c100e20 msgr2=0x7fb65c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:53.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.074+0000 7fb6616c1640 1 --2- 172.21.15.62:0/1689446624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c100e20 0x7fb65c101200 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009860 tx=0x7fb64c02a3d0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.075+0000 7fb6616c1640 1 -- 172.21.15.62:0/1689446624 shutdown_connections 2024-09-11T23:52:53.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.075+0000 7fb6616c1640 1 --2- 172.21.15.62:0/1689446624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb65c102ef0 0x7fb65c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.075+0000 7fb6616c1640 1 --2- 172.21.15.62:0/1689446624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65c102570 0x7fb65c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.075+0000 7fb6616c1640 1 --2- 172.21.15.62:0/1689446624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c100e20 0x7fb65c101200 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.075+0000 7fb6616c1640 1 -- 172.21.15.62:0/1689446624 >> 172.21.15.62:0/1689446624 conn(0x7fb65c0fc7d0 msgr2=0x7fb65c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:53.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.075+0000 7fb6616c1640 1 -- 172.21.15.62:0/1689446624 shutdown_connections 2024-09-11T23:52:53.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.075+0000 7fb6616c1640 1 -- 172.21.15.62:0/1689446624 wait complete. 2024-09-11T23:52:53.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.076+0000 7fb6616c1640 1 Processor -- start 2024-09-11T23:52:53.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb6616c1640 1 -- start start 2024-09-11T23:52:53.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb6616c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb65c100e20 0x7fb65c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb6616c1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c102570 0x7fb65c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb6616c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65c102ef0 0x7fb65c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb65a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c102570 0x7fb65c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb65affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb65c100e20 0x7fb65c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb65a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c102570 0x7fb65c197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41576/0 (socket says 172.21.15.62:41576) 2024-09-11T23:52:53.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb65a7fc640 1 -- 172.21.15.62:0/220713936 learned_addr learned my addr 172.21.15.62:0/220713936 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:53.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb6616c1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c110fa0 con 0x7fb65c102570 2024-09-11T23:52:53.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb6616c1640 1 -- 172.21.15.62:0/220713936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c110e20 con 0x7fb65c102ef0 2024-09-11T23:52:53.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.077+0000 7fb6616c1640 1 -- 172.21.15.62:0/220713936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c111120 con 0x7fb65c100e20 2024-09-11T23:52:53.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.078+0000 7fb65b7fe640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65c102ef0 0x7fb65c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.078+0000 7fb65affd640 1 -- 172.21.15.62:0/220713936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65c102ef0 msgr2=0x7fb65c19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:53.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.078+0000 7fb65affd640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65c102ef0 0x7fb65c19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.078+0000 7fb65affd640 1 -- 172.21.15.62:0/220713936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c102570 msgr2=0x7fb65c197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:53.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.078+0000 7fb65affd640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c102570 0x7fb65c197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.078+0000 7fb65affd640 1 -- 172.21.15.62:0/220713936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c19c390 con 0x7fb65c100e20 2024-09-11T23:52:53.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.079+0000 7fb65affd640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb65c100e20 0x7fb65c1973e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb64c02a8f0 tx=0x7fb64c0309c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:53.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.080+0000 7fb643fff640 1 -- 172.21.15.62:0/220713936 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c040070 con 0x7fb65c100e20 2024-09-11T23:52:53.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.080+0000 7fb643fff640 1 -- 172.21.15.62:0/220713936 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64c0055a0 con 0x7fb65c100e20 2024-09-11T23:52:53.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.080+0000 7fb6616c1640 1 -- 172.21.15.62:0/220713936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65c19c620 con 0x7fb65c100e20 2024-09-11T23:52:53.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.080+0000 7fb643fff640 1 -- 172.21.15.62:0/220713936 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c004920 con 0x7fb65c100e20 2024-09-11T23:52:53.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.081+0000 7fb6616c1640 1 -- 172.21.15.62:0/220713936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65c19ca40 con 0x7fb65c100e20 2024-09-11T23:52:53.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.083+0000 7fb663a9a740 1 Processor -- start 2024-09-11T23:52:53.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.083+0000 7fb663a9a740 1 -- start start 2024-09-11T23:52:53.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.083+0000 7fb643fff640 1 -- 172.21.15.62:0/220713936 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fb64c043070 con 0x7fb65c100e20 2024-09-11T23:52:53.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.084+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.084+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96dc5d0 0x556ac96dca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.084+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96f8380 0x556ac96fa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.084+0000 7fb663a9a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556ac96422f0 con 0x556ac96ddb40 2024-09-11T23:52:53.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.084+0000 7fb663a9a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ac95e9cf0 con 0x556ac96dc5d0 2024-09-11T23:52:53.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.084+0000 7fb663a9a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556ac963e2b0 con 0x556ac96f8380 2024-09-11T23:52:53.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.085+0000 7fb643fff640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb6440a2ed0 0x7fb6440a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.086+0000 7fb643fff640 1 -- 172.21.15.62:0/220713936 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb64c0f0260 con 0x7fb65c100e20 2024-09-11T23:52:53.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.086+0000 7fb65a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96f8380 0x556ac96fa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.086+0000 7fb65affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96dc5d0 0x556ac96dca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.086+0000 7fb65affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96dc5d0 0x556ac96dca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:49342/0 (socket says 172.21.15.62:49342) 2024-09-11T23:52:53.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.086+0000 7fb65affd640 1 -- 172.21.15.62:0/3887512435 learned_addr learned my addr 172.21.15.62:0/3887512435 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:53.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.087+0000 7fb65affd640 1 -- 172.21.15.62:0/3887512435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96f8380 msgr2=0x556ac96fa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:53.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.087+0000 7fb65affd640 1 --2- 172.21.15.62:0/3887512435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96f8380 0x556ac96fa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.087+0000 7fb65affd640 1 -- 172.21.15.62:0/3887512435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 msgr2=0x556ac96dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:53.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.087+0000 7fb65affd640 1 --2- 172.21.15.62:0/3887512435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.087+0000 7fb65affd640 1 -- 172.21.15.62:0/3887512435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ac95ce3a0 con 0x556ac96dc5d0 2024-09-11T23:52:53.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.087+0000 7fb65affd640 1 --2- 172.21.15.62:0/3887512435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96dc5d0 0x556ac96dca10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0ed950 tx=0x7fb64c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac7b2d4ce3dc23a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:53.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.087+0000 7fb6417fa640 1 -- 172.21.15.62:0/3887512435 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c04a040 con 0x556ac96dc5d0 2024-09-11T23:52:53.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.087+0000 7fb6417fa640 1 -- 172.21.15.62:0/3887512435 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64c0bc5e0 con 0x556ac96dc5d0 2024-09-11T23:52:53.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.087+0000 7fb6417fa640 1 -- 172.21.15.62:0/3887512435 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c031520 con 0x556ac96dc5d0 2024-09-11T23:52:53.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.087+0000 7fb65b7fe640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb6440a2ed0 0x7fb6440a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.088+0000 7fb663a9a740 1 -- 172.21.15.62:0/3887512435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96dc5d0 msgr2=0x556ac96dca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:53.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.088+0000 7fb663a9a740 1 --2- 172.21.15.62:0/3887512435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96dc5d0 0x556ac96dca10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0ed950 tx=0x7fb64c005f90 comp rx=0 tx=0).stop 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.088+0000 7fb65b7fe640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb6440a2ed0 0x7fb6440a5390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb654005fd0 tx=0x7fb654009bd0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.088+0000 7fb663a9a740 1 -- 172.21.15.62:0/3887512435 shutdown_connections 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.088+0000 7fb663a9a740 1 --2- 172.21.15.62:0/3887512435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96f8380 0x556ac96fa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.088+0000 7fb663a9a740 1 --2- 172.21.15.62:0/3887512435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96dc5d0 0x556ac96dca10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.088+0000 7fb663a9a740 1 --2- 172.21.15.62:0/3887512435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.088+0000 7fb663a9a740 1 -- 172.21.15.62:0/3887512435 >> 172.21.15.62:0/3887512435 conn(0x556ac96a5310 msgr2=0x556ac96d8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 1 -- 172.21.15.62:0/3887512435 shutdown_connections 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 1 -- 172.21.15.62:0/3887512435 wait complete. 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 1 -- start start 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 1 Processor -- start 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96f5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac96f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556ac95e9cf0 con 0x556ac96ddb40 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ac95fd060 con 0x556ac96f8380 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556ac96fae40 con 0x556ac96dc5d0 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.089+0000 7fb663a9a740 10 client.0 _ll_register_callbacks cb 0x556ac93508c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac96f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96f7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53382/0 (socket says 172.21.15.62:53382) 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65affd640 1 -- 172.21.15.62:0/4215407068 learned_addr learned my addr 172.21.15.62:0/4215407068 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65a7fc640 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65affd640 1 -- 172.21.15.62:0/4215407068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 msgr2=0x556ac96f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65affd640 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac96f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65affd640 1 -- 172.21.15.62:0/4215407068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 msgr2=0x556ac96f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65affd640 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65affd640 1 -- 172.21.15.62:0/4215407068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ac93b33c0 con 0x556ac96dc5d0 2024-09-11T23:52:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65a7fc640 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb65affd640 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96f7dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0b8fb0 tx=0x7fb64c0bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b510dd05f9b3c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.090+0000 7fb63eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:52:53.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.092+0000 7fb63eff5640 1 -- 172.21.15.62:0/4215407068 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0ca060 con 0x556ac96dc5d0 2024-09-11T23:52:53.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.092+0000 7fb663a9a740 1 -- 172.21.15.62:0/4215407068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c0bc5e0 con 0x556ac96dc5d0 2024-09-11T23:52:53.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.092+0000 7fb63eff5640 1 -- 172.21.15.62:0/4215407068 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb64c0d5050 con 0x556ac96dc5d0 2024-09-11T23:52:53.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.092+0000 7fb63eff5640 1 -- 172.21.15.62:0/4215407068 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb64c0d9030 con 0x556ac96dc5d0 2024-09-11T23:52:53.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.092+0000 7fb663a9a740 10 client.25132 fetch_fsmap learned FSMap version 26 2024-09-11T23:52:53.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.092+0000 7fb663a9a740 1 -- 172.21.15.62:0/4215407068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556ac97066a0 con 0x556ac96dc5d0 2024-09-11T23:52:53.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.093+0000 7fb63eff5640 1 -- 172.21.15.62:0/4215407068 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64c031c70 con 0x556ac96dc5d0 2024-09-11T23:52:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.093+0000 7fb63eff5640 1 -- 172.21.15.62:0/4215407068 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fb64c0edc00 con 0x556ac96dc5d0 2024-09-11T23:52:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.094+0000 7fb663a9a740 10 client.25132 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:52:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.094+0000 7fb663a9a740 10 client.25132 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:52:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.094+0000 7fb663a9a740 20 client.25132 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.094+0000 7fb663a9a740 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ac97078c0 0x556ac9709cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.094+0000 7fb663a9a740 4 client.25132 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.094+0000 7fb663a9a740 1 -- 172.21.15.62:0/4215407068 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb64c0bc5e0 con 0x556ac97078c0 2024-09-11T23:52:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.094+0000 7fb65a7fc640 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ac97078c0 0x556ac9709cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.095+0000 7fb65a7fc640 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ac97078c0 0x556ac9709cb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c3126a6ccff0d08 server_cookie=b2cd19fea39983a9 in_seq=0 out_seq=0 2024-09-11T23:52:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.095+0000 7fb63eff5640 10 client.25132 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:52:53.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb63eff5640 1 -- 172.21.15.62:0/4215407068 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb64c0bc5e0 con 0x556ac97078c0 2024-09-11T23:52:53.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb63eff5640 10 client.25132 handle_command_reply: tid=0 2024-09-11T23:52:53.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb663a9a740 1 client.25132 shutdown 2024-09-11T23:52:53.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb663a9a740 1 -- 172.21.15.62:0/4215407068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 msgr2=0x556ac96f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:53.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb663a9a740 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96f7dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0b8fb0 tx=0x7fb64c0bc040 comp rx=0 tx=0).stop 2024-09-11T23:52:53.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb663a9a740 1 -- 172.21.15.62:0/4215407068 shutdown_connections 2024-09-11T23:52:53.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb663a9a740 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ac97078c0 0x556ac9709cb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb663a9a740 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac96f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb663a9a740 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb663a9a740 1 --2- 172.21.15.62:0/4215407068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96f7dc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.096+0000 7fb663a9a740 1 -- 172.21.15.62:0/4215407068 >> 172.21.15.62:0/4215407068 conn(0x556ac96a5310 msgr2=0x556ac96de9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:53.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.097+0000 7fb63eff5640 0 client.25132 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:52:53.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.098+0000 7fb663a9a740 1 -- 172.21.15.62:0/4215407068 shutdown_connections 2024-09-11T23:52:53.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.098+0000 7fb663a9a740 1 -- 172.21.15.62:0/4215407068 wait complete. 2024-09-11T23:52:53.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.098+0000 7fb663a9a740 20 client.25132 trim_cache size 0 max 16384 2024-09-11T23:52:53.107 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.118+0000 7fb663a9a740 1 Processor -- start 2024-09-11T23:52:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.118+0000 7fb663a9a740 1 -- start start 2024-09-11T23:52:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.119+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.119+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac970f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.119+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac970fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.119+0000 7fb663a9a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1ade70 con 0x556ac96ddb40 2024-09-11T23:52:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.119+0000 7fb663a9a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1ab740 con 0x556ac96f8380 2024-09-11T23:52:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.120+0000 7fb663a9a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1a95b0 con 0x556ac96dc5d0 2024-09-11T23:52:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.120+0000 7fb65b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac970fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.120+0000 7fb65b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac970fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:49368/0 (socket says 172.21.15.62:49368) 2024-09-11T23:52:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.120+0000 7fb65b7fe640 1 -- 172.21.15.62:0/3719648861 learned_addr learned my addr 172.21.15.62:0/3719648861 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.120+0000 7fb65affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.120+0000 7fb65b7fe640 1 -- 172.21.15.62:0/3719648861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 msgr2=0x556ac96f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.120+0000 7fb65b7fe640 1 --2- 172.21.15.62:0/3719648861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.120+0000 7fb65b7fe640 1 -- 172.21.15.62:0/3719648861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 msgr2=0x556ac970f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.120+0000 7fb65b7fe640 1 --2- 172.21.15.62:0/3719648861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac970f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.120+0000 7fb65b7fe640 1 -- 172.21.15.62:0/3719648861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ac96d9950 con 0x556ac96f8380 2024-09-11T23:52:53.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.121+0000 7fb65b7fe640 1 --2- 172.21.15.62:0/3719648861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac970fa70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb65401c8d0 tx=0x7fb65400e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0e76f2130c758e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:53.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.121+0000 7fb640ff9640 1 -- 172.21.15.62:0/3719648861 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65402b070 con 0x556ac96f8380 2024-09-11T23:52:53.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.121+0000 7fb640ff9640 1 -- 172.21.15.62:0/3719648861 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb65401ce50 con 0x556ac96f8380 2024-09-11T23:52:53.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.121+0000 7fb640ff9640 1 -- 172.21.15.62:0/3719648861 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65401a6b0 con 0x556ac96f8380 2024-09-11T23:52:53.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.122+0000 7fb663a9a740 1 -- 172.21.15.62:0/3719648861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 msgr2=0x556ac970fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:53.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.122+0000 7fb663a9a740 1 --2- 172.21.15.62:0/3719648861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac970fa70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb65401c8d0 tx=0x7fb65400e000 comp rx=0 tx=0).stop 2024-09-11T23:52:53.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.122+0000 7fb663a9a740 1 -- 172.21.15.62:0/3719648861 shutdown_connections 2024-09-11T23:52:53.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.122+0000 7fb663a9a740 1 --2- 172.21.15.62:0/3719648861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac970fa70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.122+0000 7fb663a9a740 1 --2- 172.21.15.62:0/3719648861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac970f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.122+0000 7fb663a9a740 1 --2- 172.21.15.62:0/3719648861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.122+0000 7fb663a9a740 1 -- 172.21.15.62:0/3719648861 >> 172.21.15.62:0/3719648861 conn(0x556ac96a5310 msgr2=0x556ac96f3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:53.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.122+0000 7fb663a9a740 1 -- 172.21.15.62:0/3719648861 shutdown_connections 2024-09-11T23:52:53.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.123+0000 7fb663a9a740 1 -- 172.21.15.62:0/3719648861 wait complete. 2024-09-11T23:52:53.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.123+0000 7fb663a9a740 1 -- start start 2024-09-11T23:52:53.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.123+0000 7fb663a9a740 1 Processor -- start 2024-09-11T23:52:53.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb663a9a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac96fc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb65affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb663a9a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1a9c00 con 0x556ac96ddb40 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb663a9a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1ab070 con 0x556ac96f8380 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb663a9a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1aa2d0 con 0x556ac96dc5d0 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb65affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53402/0 (socket says 172.21.15.62:53402) 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb65affd640 1 -- 172.21.15.62:0/50360192 learned_addr learned my addr 172.21.15.62:0/50360192 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb65a7fc640 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb65b7fe640 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac96fc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb65affd640 1 -- 172.21.15.62:0/50360192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 msgr2=0x556ac96fc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb65affd640 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac96fc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb663a9a740 10 client.0 _ll_register_callbacks cb 0x7fb650009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb65affd640 1 -- 172.21.15.62:0/50360192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 msgr2=0x556ac96fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb65affd640 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.124+0000 7fb65affd640 1 -- 172.21.15.62:0/50360192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ac9771700 con 0x556ac96dc5d0 2024-09-11T23:52:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb663a9a740 1 -- 172.21.15.62:0/50360192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb65401ce50 con 0x556ac96dc5d0 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb65affd640 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96fbea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0c51c0 tx=0x7fb64c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35f6342295d9741f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb63eff5640 10 client.25138 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb63eff5640 1 -- 172.21.15.62:0/50360192 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c04a040 con 0x556ac96dc5d0 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb63eff5640 1 -- 172.21.15.62:0/50360192 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb64c0d5040 con 0x556ac96dc5d0 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb63eff5640 1 -- 172.21.15.62:0/50360192 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb64c0053f0 con 0x556ac96dc5d0 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb663a9a740 10 client.25138 fetch_fsmap learned FSMap version 26 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb663a9a740 1 -- 172.21.15.62:0/50360192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=26}) -- 0x556ac9771700 con 0x556ac96dc5d0 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb63eff5640 1 -- 172.21.15.62:0/50360192 <== mon.1 v2:172.21.15.120:3300/0 4 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fb64c0dc440 con 0x556ac96dc5d0 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb663a9a740 10 client.25138 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.125+0000 7fb663a9a740 10 client.25138 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.126+0000 7fb663a9a740 20 client.25138 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.126+0000 7fb663a9a740 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ac9772a00 0x556ac9774df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.126+0000 7fb663a9a740 4 client.25138 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:52:53.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.126+0000 7fb663a9a740 1 -- 172.21.15.62:0/50360192 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ac964f860 con 0x556ac9772a00 2024-09-11T23:52:53.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.126+0000 7fb65a7fc640 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ac9772a00 0x556ac9774df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:53.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.126+0000 7fb63eff5640 10 client.25138 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:52:53.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.126+0000 7fb65a7fc640 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ac9772a00 0x556ac9774df0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7aad5dc32d19eb18 server_cookie=2c38a04e4266f134 in_seq=0 out_seq=0 2024-09-11T23:52:53.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.126+0000 7fb63eff5640 1 -- 172.21.15.62:0/50360192 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ac964f860 con 0x556ac9772a00 2024-09-11T23:52:53.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.126+0000 7fb63eff5640 10 client.25138 handle_command_reply: tid=0 2024-09-11T23:52:53.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.126+0000 7fb663a9a740 1 client.25138 shutdown 2024-09-11T23:52:53.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 1 -- 172.21.15.62:0/50360192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 msgr2=0x556ac96fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:53.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96fbea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0c51c0 tx=0x7fb64c005f90 comp rx=0 tx=0).stop 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 1 -- 172.21.15.62:0/50360192 shutdown_connections 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ac9772a00 0x556ac9774df0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ac96f8380 0x556ac96fc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ac96ddb40 0x556ac96fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 1 --2- 172.21.15.62:0/50360192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ac96dc5d0 0x556ac96fbea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb63eff5640 0 client.25138 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 1 -- 172.21.15.62:0/50360192 >> 172.21.15.62:0/50360192 conn(0x556ac96a5310 msgr2=0x556ac96f4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 1 -- 172.21.15.62:0/50360192 shutdown_connections 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 1 -- 172.21.15.62:0/50360192 wait complete. 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.127+0000 7fb663a9a740 20 client.25138 trim_cache size 0 max 16384 2024-09-11T23:52:53.129 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:52:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.128+0000 7fb6616c1640 1 -- 172.21.15.62:0/220713936 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb6440a2ed0 msgr2=0x7fb6440a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.128+0000 7fb6616c1640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb6440a2ed0 0x7fb6440a5390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb654005fd0 tx=0x7fb654009bd0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.128+0000 7fb6616c1640 1 -- 172.21.15.62:0/220713936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb65c100e20 msgr2=0x7fb65c1973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.128+0000 7fb6616c1640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb65c100e20 0x7fb65c1973e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb64c02a8f0 tx=0x7fb64c0309c0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.129+0000 7fb6616c1640 1 -- 172.21.15.62:0/220713936 shutdown_connections 2024-09-11T23:52:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.129+0000 7fb6616c1640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb6440a2ed0 0x7fb6440a5390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.129+0000 7fb6616c1640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65c102ef0 0x7fb65c19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.129+0000 7fb6616c1640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c102570 0x7fb65c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.129+0000 7fb6616c1640 1 --2- 172.21.15.62:0/220713936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb65c100e20 0x7fb65c1973e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.129+0000 7fb6616c1640 1 -- 172.21.15.62:0/220713936 >> 172.21.15.62:0/220713936 conn(0x7fb65c0fc7d0 msgr2=0x7fb65c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.129+0000 7fb6616c1640 1 -- 172.21.15.62:0/220713936 shutdown_connections 2024-09-11T23:52:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:53.129+0000 7fb6616c1640 1 -- 172.21.15.62:0/220713936 wait complete. 2024-09-11T23:52:53.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:53 smithi062 ceph-mon[30958]: pgmap v26: 97 pgs: 97 active+clean; 834 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 74 MiB/s wr, 182 op/s 2024-09-11T23:52:53.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:53 smithi120 ceph-mon[35182]: pgmap v26: 97 pgs: 97 active+clean; 834 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 74 MiB/s wr, 182 op/s 2024-09-11T23:52:53.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:53 smithi083 ceph-mon[34805]: pgmap v26: 97 pgs: 97 active+clean; 834 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 74 MiB/s wr, 182 op/s 2024-09-11T23:52:54.145 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:52:54.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.412+0000 7f7e6a317640 1 -- 172.21.15.62:0/152756802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e64102790 msgr2=0x7f7e641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:54.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.412+0000 7f7e6a317640 1 --2- 172.21.15.62:0/152756802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e64102790 0x7f7e641067e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009860 tx=0x7f7e5402a340 comp rx=0 tx=0).stop 2024-09-11T23:52:54.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.412+0000 7f7e6a317640 1 -- 172.21.15.62:0/152756802 shutdown_connections 2024-09-11T23:52:54.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.412+0000 7f7e6a317640 1 --2- 172.21.15.62:0/152756802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e64106f10 0x7f7e6410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.412+0000 7f7e6a317640 1 --2- 172.21.15.62:0/152756802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e64102790 0x7f7e641067e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.412+0000 7f7e6a317640 1 --2- 172.21.15.62:0/152756802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e64101de0 0x7f7e641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.412+0000 7f7e6a317640 1 -- 172.21.15.62:0/152756802 >> 172.21.15.62:0/152756802 conn(0x7f7e640fd8c0 msgr2=0x7f7e640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.412+0000 7f7e6a317640 1 -- 172.21.15.62:0/152756802 shutdown_connections 2024-09-11T23:52:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.413+0000 7f7e6a317640 1 -- 172.21.15.62:0/152756802 wait complete. 2024-09-11T23:52:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.413+0000 7f7e6a317640 1 Processor -- start 2024-09-11T23:52:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.414+0000 7f7e6a317640 1 -- start start 2024-09-11T23:52:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.414+0000 7f7e6a317640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e64101de0 0x7f7e64075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.414+0000 7f7e6a317640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e64102790 0x7f7e64076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e6a317640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e64106f10 0x7f7e6406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e64102790 0x7f7e64076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e64101de0 0x7f7e64075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e64102790 0x7f7e64076040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:49392/0 (socket says 172.21.15.62:49392) 2024-09-11T23:52:54.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e64101de0 0x7f7e64075b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53416/0 (socket says 172.21.15.62:53416) 2024-09-11T23:52:54.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e688b4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e64106f10 0x7f7e6406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e637fe640 1 -- 172.21.15.62:0/1701333184 learned_addr learned my addr 172.21.15.62:0/1701333184 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:54.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7e64111f70 con 0x7f7e64106f10 2024-09-11T23:52:54.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e64111df0 con 0x7f7e64102790 2024-09-11T23:52:54.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e641120f0 con 0x7f7e64101de0 2024-09-11T23:52:54.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e63fff640 1 -- 172.21.15.62:0/1701333184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e64102790 msgr2=0x7f7e64076040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:54.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e63fff640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e64102790 0x7f7e64076040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e63fff640 1 -- 172.21.15.62:0/1701333184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e64106f10 msgr2=0x7f7e6406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:54.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e63fff640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e64106f10 0x7f7e6406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.415+0000 7f7e63fff640 1 -- 172.21.15.62:0/1701333184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6406d4a0 con 0x7f7e64101de0 2024-09-11T23:52:54.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.416+0000 7f7e63fff640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e64101de0 0x7f7e64075b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009fd0 tx=0x7f7e5c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:54.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.416+0000 7f7e617fa640 1 -- 172.21.15.62:0/1701333184 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c004b20 con 0x7f7e64101de0 2024-09-11T23:52:54.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.416+0000 7f7e617fa640 1 -- 172.21.15.62:0/1701333184 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e5c004cc0 con 0x7f7e64101de0 2024-09-11T23:52:54.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.416+0000 7f7e617fa640 1 -- 172.21.15.62:0/1701333184 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0107b0 con 0x7f7e64101de0 2024-09-11T23:52:54.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.416+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6406d790 con 0x7f7e64101de0 2024-09-11T23:52:54.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.417+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e64071720 con 0x7f7e64101de0 2024-09-11T23:52:54.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e6c6f0740 1 Processor -- start 2024-09-11T23:52:54.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e6c6f0740 1 -- start start 2024-09-11T23:52:54.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec73b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec7e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec80d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cfeebd92a0 con 0x55cfeec7e940 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cfeeb80b10 con 0x55cfeec73760 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cfeebd5260 con 0x55cfeec74080 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e688b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec73b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e688b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec73b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:49408/0 (socket says 172.21.15.62:49408) 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.421+0000 7f7e688b4640 1 -- 172.21.15.62:0/3331237950 learned_addr learned my addr 172.21.15.62:0/3331237950 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.422+0000 7f7e637fe640 1 --2- 172.21.15.62:0/3331237950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec80d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.422+0000 7f7e688b4640 1 -- 172.21.15.62:0/3331237950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 msgr2=0x55cfeec7e400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.422+0000 7f7e688b4640 1 --2- 172.21.15.62:0/3331237950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec7e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.422+0000 7f7e688b4640 1 -- 172.21.15.62:0/3331237950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 msgr2=0x55cfeec80d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.422+0000 7f7e688b4640 1 --2- 172.21.15.62:0/3331237950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec80d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.422+0000 7f7e688b4640 1 -- 172.21.15.62:0/3331237950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfee94ab90 con 0x55cfeec73760 2024-09-11T23:52:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.422+0000 7f7e637fe640 1 --2- 172.21.15.62:0/3331237950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec80d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:54.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.422+0000 7f7e688b4640 1 --2- 172.21.15.62:0/3331237950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec73b40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009be0 tx=0x7f7e58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbbf46b1a67b59d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:54.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.423+0000 7f7e4a7fc640 1 -- 172.21.15.62:0/3331237950 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58004230 con 0x55cfeec73760 2024-09-11T23:52:54.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.423+0000 7f7e4a7fc640 1 -- 172.21.15.62:0/3331237950 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e58009e50 con 0x55cfeec73760 2024-09-11T23:52:54.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.423+0000 7f7e4a7fc640 1 -- 172.21.15.62:0/3331237950 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58019960 con 0x55cfeec73760 2024-09-11T23:52:54.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.423+0000 7f7e617fa640 1 -- 172.21.15.62:0/1701333184 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f7e5c005020 con 0x7f7e64101de0 2024-09-11T23:52:54.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.424+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/3331237950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 msgr2=0x55cfeec73b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:54.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.424+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/3331237950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec73b40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009be0 tx=0x7f7e58011810 comp rx=0 tx=0).stop 2024-09-11T23:52:54.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.424+0000 7f7e617fa640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7e4c0a2ed0 0x7f7e4c0a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.425+0000 7f7e617fa640 1 -- 172.21.15.62:0/1701333184 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f7e5c0cfb60 con 0x7f7e64101de0 2024-09-11T23:52:54.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.425+0000 7f7e688b4640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7e4c0a2ed0 0x7f7e4c0a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.425+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/3331237950 shutdown_connections 2024-09-11T23:52:54.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.425+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/3331237950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec80d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.425+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/3331237950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec7e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.425+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/3331237950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec73b40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.426+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/3331237950 >> 172.21.15.62:0/3331237950 conn(0x55cfeec6fcf0 msgr2=0x55cfeec700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:54.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.426+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/3331237950 shutdown_connections 2024-09-11T23:52:54.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.426+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/3331237950 wait complete. 2024-09-11T23:52:54.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.426+0000 7f7e688b4640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7e4c0a2ed0 0x7f7e4c0a5390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009bb0 tx=0x7f7e58009790 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:54.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.426+0000 7f7e6c6f0740 1 -- start start 2024-09-11T23:52:54.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.427+0000 7f7e6c6f0740 1 Processor -- start 2024-09-11T23:52:54.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.428+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec7b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.428+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec7bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.428+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec7c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.428+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cfeec81510 con 0x55cfeec7e940 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.428+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cfeeb616e0 con 0x55cfeec73760 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.428+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cfeec81690 con 0x55cfeec74080 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.428+0000 7f7e6c6f0740 10 client.0 _ll_register_callbacks cb 0x55cfeeb85a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.428+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec7b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.428+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec7b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:49410/0 (socket says 172.21.15.62:49410) 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.428+0000 7f7e63fff640 1 -- 172.21.15.62:0/443269505 learned_addr learned my addr 172.21.15.62:0/443269505 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.429+0000 7f7e688b4640 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec7c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.429+0000 7f7e637fe640 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec7bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.429+0000 7f7e688b4640 1 -- 172.21.15.62:0/443269505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 msgr2=0x55cfeec7bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:54.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.429+0000 7f7e688b4640 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec7bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.429+0000 7f7e688b4640 1 -- 172.21.15.62:0/443269505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 msgr2=0x55cfeec7b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:54.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.429+0000 7f7e688b4640 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec7b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.429+0000 7f7e688b4640 1 -- 172.21.15.62:0/443269505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfeec8aab0 con 0x55cfeec7e940 2024-09-11T23:52:54.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.429+0000 7f7e688b4640 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec7c2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7e580099f0 tx=0x7f7e58003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8113d512f5c65eca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:54.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.430+0000 7f7e437fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:54.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.430+0000 7f7e437fe640 1 -- 172.21.15.62:0/443269505 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58033060 con 0x55cfeec7e940 2024-09-11T23:52:54.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.430+0000 7f7e437fe640 1 -- 172.21.15.62:0/443269505 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f7e58005450 con 0x55cfeec7e940 2024-09-11T23:52:54.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.431+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/443269505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e58009e50 con 0x55cfeec7e940 2024-09-11T23:52:54.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.431+0000 7f7e437fe640 1 -- 172.21.15.62:0/443269505 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e58005650 con 0x55cfeec7e940 2024-09-11T23:52:54.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.431+0000 7f7e6c6f0740 10 client.15483 fetch_fsmap learned FSMap version 26 2024-09-11T23:52:54.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.431+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/443269505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cfeec8d440 con 0x55cfeec7e940 2024-09-11T23:52:54.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.432+0000 7f7e437fe640 1 -- 172.21.15.62:0/443269505 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e58009e50 con 0x55cfeec7e940 2024-09-11T23:52:54.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.432+0000 7f7e437fe640 1 -- 172.21.15.62:0/443269505 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f7e580225e0 con 0x55cfeec7e940 2024-09-11T23:52:54.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.432+0000 7f7e6c6f0740 10 client.15483 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:52:54.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.432+0000 7f7e6c6f0740 10 client.15483 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:52:54.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.432+0000 7f7e6c6f0740 20 client.15483 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:54.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.432+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cfeec8e790 0x55cfeec90b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.432+0000 7f7e6c6f0740 4 client.15483 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:54.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.432+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/443269505 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfeebe69d0 con 0x55cfeec8e790 2024-09-11T23:52:54.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.432+0000 7f7e63fff640 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cfeec8e790 0x55cfeec90b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.438+0000 7f7e63fff640 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cfeec8e790 0x55cfeec90b80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95db0207e35b1392 server_cookie=7e8a73306cf8e790 in_seq=0 out_seq=0 2024-09-11T23:52:54.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.438+0000 7f7e437fe640 10 client.15483 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:54.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.438+0000 7f7e437fe640 1 -- 172.21.15.62:0/443269505 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cfeebe69d0 con 0x55cfeec8e790 2024-09-11T23:52:54.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.438+0000 7f7e437fe640 10 client.15483 handle_command_reply: tid=0 2024-09-11T23:52:54.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.438+0000 7f7e6c6f0740 1 client.15483 shutdown 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/443269505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 msgr2=0x55cfeec7c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec7c2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7e580099f0 tx=0x7f7e58003d70 comp rx=0 tx=0).stop 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/443269505 shutdown_connections 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cfeec8e790 0x55cfeec90b80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec7c2f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec7bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/443269505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec7b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/443269505 >> 172.21.15.62:0/443269505 conn(0x55cfeec6fcf0 msgr2=0x55cfeec6ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e437fe640 0 client.15483 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/443269505 shutdown_connections 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/443269505 wait complete. 2024-09-11T23:52:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.439+0000 7f7e6c6f0740 20 client.15483 trim_cache size 0 max 16384 2024-09-11T23:52:54.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.470+0000 7f7e6c6f0740 1 Processor -- start 2024-09-11T23:52:54.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.470+0000 7f7e6c6f0740 1 -- start start 2024-09-11T23:52:54.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.471+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec7c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.471+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec758c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.471+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec96340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.471+0000 7f7e637fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.471+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50011080 con 0x55cfeec7e940 2024-09-11T23:52:54.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.471+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5000e950 con 0x55cfeec73760 2024-09-11T23:52:54.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.471+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5000c7a0 con 0x55cfeec74080 2024-09-11T23:52:54.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.471+0000 7f7e637fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec758c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53434/0 (socket says 172.21.15.62:53434) 2024-09-11T23:52:54.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.471+0000 7f7e637fe640 1 -- 172.21.15.62:0/3411717311 learned_addr learned my addr 172.21.15.62:0/3411717311 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:54.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.471+0000 7f7e688b4640 1 --2- 172.21.15.62:0/3411717311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec96340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.472+0000 7f7e637fe640 1 -- 172.21.15.62:0/3411717311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 msgr2=0x55cfeec7c140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:52:54.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.472+0000 7f7e637fe640 1 --2- 172.21.15.62:0/3411717311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec7c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.472+0000 7f7e637fe640 1 -- 172.21.15.62:0/3411717311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 msgr2=0x55cfeec96340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:54.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.472+0000 7f7e637fe640 1 --2- 172.21.15.62:0/3411717311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec96340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.472+0000 7f7e637fe640 1 -- 172.21.15.62:0/3411717311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfeec6b260 con 0x55cfeec74080 2024-09-11T23:52:54.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.472+0000 7f7e637fe640 1 --2- 172.21.15.62:0/3411717311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec758c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e540057c0 tx=0x7f7e54009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ea2c341852b967 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:54.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.473+0000 7f7e49ffb640 1 -- 172.21.15.62:0/3411717311 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54037040 con 0x55cfeec74080 2024-09-11T23:52:54.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.473+0000 7f7e49ffb640 1 -- 172.21.15.62:0/3411717311 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e54030550 con 0x55cfeec74080 2024-09-11T23:52:54.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.473+0000 7f7e49ffb640 1 -- 172.21.15.62:0/3411717311 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54003ab0 con 0x55cfeec74080 2024-09-11T23:52:54.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.474+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/3411717311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 msgr2=0x55cfeec758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:54.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.474+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/3411717311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec758c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e540057c0 tx=0x7f7e54009570 comp rx=0 tx=0).stop 2024-09-11T23:52:54.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.474+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/3411717311 shutdown_connections 2024-09-11T23:52:54.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.474+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/3411717311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec96340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.474+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/3411717311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec74080 0x55cfeec758c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.474+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/3411717311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec73760 0x55cfeec7c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.474+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/3411717311 >> 172.21.15.62:0/3411717311 conn(0x55cfeec6fcf0 msgr2=0x55cfeec751b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:54.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.475+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/3411717311 shutdown_connections 2024-09-11T23:52:54.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.475+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/3411717311 wait complete. 2024-09-11T23:52:54.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.476+0000 7f7e6c6f0740 1 -- start start 2024-09-11T23:52:54.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.476+0000 7f7e6c6f0740 1 Processor -- start 2024-09-11T23:52:54.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.477+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec73760 0x55cfeec86b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.477+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec74080 0x55cfeec87060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.477+0000 7f7e6c6f0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec83910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.477+0000 7f7e637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec74080 0x55cfeec87060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.477+0000 7f7e637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec74080 0x55cfeec87060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:49416/0 (socket says 172.21.15.62:49416) 2024-09-11T23:52:54.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.477+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec73760 0x55cfeec86b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5000ce10 con 0x55cfeec7e940 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5000e280 con 0x55cfeec74080 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e6c6f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5000d4e0 con 0x55cfeec73760 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e6c6f0740 10 client.0 _ll_register_callbacks cb 0x7f7e58019b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e637fe640 1 -- 172.21.15.62:0/471453677 learned_addr learned my addr 172.21.15.62:0/471453677 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e688b4640 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec83910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e63fff640 1 -- 172.21.15.62:0/471453677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec74080 msgr2=0x55cfeec87060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e63fff640 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec74080 0x55cfeec87060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e63fff640 1 -- 172.21.15.62:0/471453677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 msgr2=0x55cfeec83910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e63fff640 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec83910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.478+0000 7f7e63fff640 1 -- 172.21.15.62:0/471453677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfeec83fd0 con 0x55cfeec73760 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.479+0000 7f7e63fff640 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec73760 0x55cfeec86b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0cd300 tx=0x7f7e5c09bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ec52cf7cb375e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:54.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.479+0000 7f7e437fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:52:54.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.479+0000 7f7e437fe640 1 -- 172.21.15.62:0/471453677 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0a9060 con 0x55cfeec73760 2024-09-11T23:52:54.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.479+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/471453677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e54030550 con 0x55cfeec73760 2024-09-11T23:52:54.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.479+0000 7f7e437fe640 1 -- 172.21.15.62:0/471453677 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f7e5c09a240 con 0x55cfeec73760 2024-09-11T23:52:54.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.480+0000 7f7e437fe640 1 -- 172.21.15.62:0/471453677 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e5c0a02f0 con 0x55cfeec73760 2024-09-11T23:52:54.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.480+0000 7f7e6c6f0740 10 client.25153 fetch_fsmap learned FSMap version 26 2024-09-11T23:52:54.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.480+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/471453677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cfeecf9ce0 con 0x55cfeec73760 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.480+0000 7f7e437fe640 1 -- 172.21.15.62:0/471453677 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e54030550 con 0x55cfeec73760 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.481+0000 7f7e437fe640 1 -- 172.21.15.62:0/471453677 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f7e5c09c1a0 con 0x55cfeec73760 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.482+0000 7f7e6c6f0740 10 client.25153 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.482+0000 7f7e6c6f0740 10 client.25153 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.482+0000 7f7e6c6f0740 20 client.25153 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.482+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cfeecfa400 0x55cfeecfc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.482+0000 7f7e6c6f0740 4 client.25153 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.482+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/471453677 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55cfeebe0d20 con 0x55cfeecfa400 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.482+0000 7f7e637fe640 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cfeecfa400 0x55cfeecfc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.482+0000 7f7e637fe640 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cfeecfa400 0x55cfeecfc820 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902ea4e68bdffd91 server_cookie=21d455726f1aaef3 in_seq=0 out_seq=0 2024-09-11T23:52:54.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:54.482+0000 7f7e437fe640 10 client.25153 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:54.535 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:55.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:55 smithi062 ceph-mon[30958]: pgmap v27: 97 pgs: 97 active+clean; 925 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 78 MiB/s wr, 495 op/s 2024-09-11T23:52:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.424+0000 7fdadef0f640 1 -- 172.21.15.62:0/3377715249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdad81002f0 msgr2=0x7fdad81006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.424+0000 7fdadef0f640 1 --2- 172.21.15.62:0/3377715249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdad81002f0 0x7fdad81006d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009930 tx=0x7fdac802a420 comp rx=0 tx=0).stop 2024-09-11T23:52:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.424+0000 7fdadef0f640 1 -- 172.21.15.62:0/3377715249 shutdown_connections 2024-09-11T23:52:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.424+0000 7fdadef0f640 1 --2- 172.21.15.62:0/3377715249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad8106f50 0x7fdad810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.424+0000 7fdadef0f640 1 --2- 172.21.15.62:0/3377715249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdad8100ca0 0x7fdad8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.424+0000 7fdadef0f640 1 --2- 172.21.15.62:0/3377715249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdad81002f0 0x7fdad81006d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.424+0000 7fdadef0f640 1 -- 172.21.15.62:0/3377715249 >> 172.21.15.62:0/3377715249 conn(0x7fdad8077210 msgr2=0x7fdad80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:55.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.424+0000 7fdadef0f640 1 -- 172.21.15.62:0/3377715249 shutdown_connections 2024-09-11T23:52:55.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.424+0000 7fdadef0f640 1 -- 172.21.15.62:0/3377715249 wait complete. 2024-09-11T23:52:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdadef0f640 1 Processor -- start 2024-09-11T23:52:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdadef0f640 1 -- start start 2024-09-11T23:52:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdadef0f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad81002f0 0x7fdad8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdadef0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdad8100ca0 0x7fdad8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdadef0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdad8106f50 0x7fdad819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdadef0f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8111fa0 con 0x7fdad8100ca0 2024-09-11T23:52:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdadef0f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8111e20 con 0x7fdad81002f0 2024-09-11T23:52:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdadef0f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8112120 con 0x7fdad8106f50 2024-09-11T23:52:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdad8100ca0 0x7fdad8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdadccab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad81002f0 0x7fdad8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdad8100ca0 0x7fdad8198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41684/0 (socket says 172.21.15.62:41684) 2024-09-11T23:52:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdad7fff640 1 -- 172.21.15.62:0/1582493411 learned_addr learned my addr 172.21.15.62:0/1582493411 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.425+0000 7fdadd4ac640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdad8106f50 0x7fdad819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdad7fff640 1 -- 172.21.15.62:0/1582493411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdad8106f50 msgr2=0x7fdad819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdad7fff640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdad8106f50 0x7fdad819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdad7fff640 1 -- 172.21.15.62:0/1582493411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad81002f0 msgr2=0x7fdad8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdad7fff640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad81002f0 0x7fdad8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdad7fff640 1 -- 172.21.15.62:0/1582493411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad819d4b0 con 0x7fdad8100ca0 2024-09-11T23:52:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdad7fff640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdad8100ca0 0x7fdad8198a40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e9f0 tx=0x7fdac000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdad5ffb640 1 -- 172.21.15.62:0/1582493411 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac000ccc0 con 0x7fdad8100ca0 2024-09-11T23:52:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdad5ffb640 1 -- 172.21.15.62:0/1582493411 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac0005500 con 0x7fdad8100ca0 2024-09-11T23:52:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdadef0f640 1 -- 172.21.15.62:0/1582493411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad819d7a0 con 0x7fdad8100ca0 2024-09-11T23:52:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdad5ffb640 1 -- 172.21.15.62:0/1582493411 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0010690 con 0x7fdad8100ca0 2024-09-11T23:52:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.426+0000 7fdadef0f640 1 -- 172.21.15.62:0/1582493411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad81a5080 con 0x7fdad8100ca0 2024-09-11T23:52:55.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdad5ffb640 1 -- 172.21.15.62:0/1582493411 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fdac00046c0 con 0x7fdad8100ca0 2024-09-11T23:52:55.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdae12e8740 1 Processor -- start 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdae12e8740 1 -- start start 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c093a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c093aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 0x55d6c0947760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdae12e8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d6c08a02f0 con 0x55d6c093bb40 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdae12e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6c0847cf0 con 0x55d6c093a5d0 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdae12e8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d6c089c2b0 con 0x55d6c0945370 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdadd4ac640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c093a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdad5ffb640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdac40a2e00 0x7fdac40a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdadccab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c093aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 0x55d6c0947760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdadccab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c093aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:49448/0 (socket says 172.21.15.62:49448) 2024-09-11T23:52:55.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdadd4ac640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c093a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41686/0 (socket says 172.21.15.62:41686) 2024-09-11T23:52:55.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.429+0000 7fdadccab640 1 -- 172.21.15.62:0/1564466742 learned_addr learned my addr 172.21.15.62:0/1564466742 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdadd4ac640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdac40a2e00 0x7fdac40a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdadccab640 1 -- 172.21.15.62:0/1564466742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 msgr2=0x55d6c0947760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdadccab640 1 --2- 172.21.15.62:0/1564466742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 0x55d6c0947760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdadccab640 1 -- 172.21.15.62:0/1564466742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 msgr2=0x55d6c093a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdadccab640 1 --2- 172.21.15.62:0/1564466742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c093a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdadccab640 1 -- 172.21.15.62:0/1564466742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c082c3a0 con 0x55d6c093a5d0 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdad5ffb640 1 -- 172.21.15.62:0/1582493411 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fdac00cf890 con 0x7fdad8100ca0 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdadccab640 1 --2- 172.21.15.62:0/1564466742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c093aa10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdac802a940 tx=0x7fdac8005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c2561c5e3d84c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdadd4ac640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdac40a2e00 0x7fdac40a52c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009fd0 tx=0x7fdad0004ab0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdaceffd640 1 -- 172.21.15.62:0/1564466742 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8035040 con 0x55d6c093a5d0 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdaceffd640 1 -- 172.21.15.62:0/1564466742 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac8031870 con 0x55d6c093a5d0 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.430+0000 7fdaceffd640 1 -- 172.21.15.62:0/1564466742 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8004a00 con 0x55d6c093a5d0 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.431+0000 7fdae12e8740 1 -- 172.21.15.62:0/1564466742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 msgr2=0x55d6c093aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.431+0000 7fdae12e8740 1 --2- 172.21.15.62:0/1564466742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c093aa10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdac802a940 tx=0x7fdac8005bd0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.431+0000 7fdae12e8740 1 -- 172.21.15.62:0/1564466742 shutdown_connections 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.431+0000 7fdae12e8740 1 --2- 172.21.15.62:0/1564466742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 0x55d6c0947760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.431+0000 7fdae12e8740 1 --2- 172.21.15.62:0/1564466742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c093aa10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.431+0000 7fdae12e8740 1 --2- 172.21.15.62:0/1564466742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c093a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.431+0000 7fdae12e8740 1 -- 172.21.15.62:0/1564466742 >> 172.21.15.62:0/1564466742 conn(0x55d6c0903310 msgr2=0x55d6c0936c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.431+0000 7fdae12e8740 1 -- 172.21.15.62:0/1564466742 shutdown_connections 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.431+0000 7fdae12e8740 1 -- 172.21.15.62:0/1564466742 wait complete. 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.431+0000 7fdae12e8740 1 -- start start 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdae12e8740 1 Processor -- start 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0944d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093bb40 0x55d6c0941ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c0945370 0x55d6c09425e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdae12e8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d6c0948010 con 0x55d6c0945370 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdae12e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6c085b060 con 0x55d6c093bb40 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdae12e8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d6c0948190 con 0x55d6c093a5d0 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdae12e8740 10 client.0 _ll_register_callbacks cb 0x55d6c05ae8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdadccab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0944d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdadd4ac640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c0945370 0x55d6c09425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093bb40 0x55d6c0941ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdadccab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0944d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53484/0 (socket says 172.21.15.62:53484) 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdadccab640 1 -- 172.21.15.62:0/2927206332 learned_addr learned my addr 172.21.15.62:0/2927206332 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdadd4ac640 1 -- 172.21.15.62:0/2927206332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 msgr2=0x55d6c0944d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdadd4ac640 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0944d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdadd4ac640 1 -- 172.21.15.62:0/2927206332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093bb40 msgr2=0x55d6c0941ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdadd4ac640 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093bb40 0x55d6c0941ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdadd4ac640 1 -- 172.21.15.62:0/2927206332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c06113c0 con 0x55d6c0945370 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdad7fff640 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093bb40 0x55d6c0941ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.432+0000 7fdadccab640 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0944d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.433+0000 7fdadd4ac640 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c0945370 0x55d6c09425e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdad0012040 tx=0x7fdad0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1316456debb5a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.433+0000 7fdab7fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.433+0000 7fdab7fff640 1 -- 172.21.15.62:0/2927206332 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0025070 con 0x55d6c0945370 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.433+0000 7fdab7fff640 1 -- 172.21.15.62:0/2927206332 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fdad0010d90 con 0x55d6c0945370 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.433+0000 7fdae12e8740 1 -- 172.21.15.62:0/2927206332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac8031870 con 0x55d6c0945370 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.433+0000 7fdab7fff640 1 -- 172.21.15.62:0/2927206332 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdad0004b20 con 0x55d6c0945370 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.433+0000 7fdae12e8740 10 client.15501 fetch_fsmap learned FSMap version 26 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.433+0000 7fdae12e8740 1 -- 172.21.15.62:0/2927206332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d6c0953ae0 con 0x55d6c0945370 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.433+0000 7fdab7fff640 1 -- 172.21.15.62:0/2927206332 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac8031870 con 0x55d6c0945370 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.433+0000 7fdab7fff640 1 -- 172.21.15.62:0/2927206332 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fdad0021440 con 0x55d6c0945370 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.434+0000 7fdae12e8740 10 client.15501 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.434+0000 7fdae12e8740 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.434+0000 7fdae12e8740 20 client.15501 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.434+0000 7fdae12e8740 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d6c0954ef0 0x55d6c0957360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.434+0000 7fdae12e8740 4 client.15501 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.434+0000 7fdae12e8740 1 -- 172.21.15.62:0/2927206332 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6c08ad860 con 0x55d6c0954ef0 2024-09-11T23:52:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.434+0000 7fdadccab640 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d6c0954ef0 0x55d6c0957360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.434+0000 7fdab7fff640 10 client.15501 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.434+0000 7fdadccab640 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d6c0954ef0 0x55d6c0957360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62da199e2e28ef50 server_cookie=da349e16801ea651 in_seq=0 out_seq=0 2024-09-11T23:52:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.437+0000 7fdab7fff640 1 -- 172.21.15.62:0/2927206332 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6c08ad860 con 0x55d6c0954ef0 2024-09-11T23:52:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.437+0000 7fdab7fff640 10 client.15501 handle_command_reply: tid=0 2024-09-11T23:52:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 client.15501 shutdown 2024-09-11T23:52:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 -- 172.21.15.62:0/2927206332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c0945370 msgr2=0x55d6c09425e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c0945370 0x55d6c09425e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdad0012040 tx=0x7fdad0004a40 comp rx=0 tx=0).stop 2024-09-11T23:52:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 -- 172.21.15.62:0/2927206332 shutdown_connections 2024-09-11T23:52:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d6c0954ef0 0x55d6c0957360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c0945370 0x55d6c09425e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093bb40 0x55d6c0941ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 --2- 172.21.15.62:0/2927206332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0944d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 -- 172.21.15.62:0/2927206332 >> 172.21.15.62:0/2927206332 conn(0x55d6c0903310 msgr2=0x55d6c093c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdab7fff640 0 client.15501 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 -- 172.21.15.62:0/2927206332 shutdown_connections 2024-09-11T23:52:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 1 -- 172.21.15.62:0/2927206332 wait complete. 2024-09-11T23:52:55.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.438+0000 7fdae12e8740 20 client.15501 trim_cache size 0 max 16384 2024-09-11T23:52:55.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.451+0000 7fdae12e8740 1 Processor -- start 2024-09-11T23:52:55.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.451+0000 7fdae12e8740 1 -- start start 2024-09-11T23:52:55.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0936e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c09512b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c0945370 0x55d6c09418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdae12e8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81adeb0 con 0x55d6c093bb40 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdae12e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81ab780 con 0x55d6c0945370 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdae12e8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81a95f0 con 0x55d6c093a5d0 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c09512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdadccab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0936e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdadd4ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c0945370 0x55d6c09418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdadccab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0936e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53492/0 (socket says 172.21.15.62:53492) 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c09512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41702/0 (socket says 172.21.15.62:41702) 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdadccab640 1 -- 172.21.15.62:0/715035074 learned_addr learned my addr 172.21.15.62:0/715035074 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdadccab640 1 -- 172.21.15.62:0/715035074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c0945370 msgr2=0x55d6c09418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdadccab640 1 --2- 172.21.15.62:0/715035074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c0945370 0x55d6c09418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdadccab640 1 -- 172.21.15.62:0/715035074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 msgr2=0x55d6c09512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdadccab640 1 --2- 172.21.15.62:0/715035074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c09512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdadccab640 1 -- 172.21.15.62:0/715035074 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c0940fa0 con 0x55d6c093a5d0 2024-09-11T23:52:55.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdad7fff640 1 --2- 172.21.15.62:0/715035074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c09512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:52:55.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.452+0000 7fdadccab640 1 --2- 172.21.15.62:0/715035074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0936e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdac80096c0 tx=0x7fdac8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9f9fcd9eeaa1518 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:55.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdace7fc640 1 -- 172.21.15.62:0/715035074 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8035040 con 0x55d6c093a5d0 2024-09-11T23:52:55.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdace7fc640 1 -- 172.21.15.62:0/715035074 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac803d760 con 0x55d6c093a5d0 2024-09-11T23:52:55.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdace7fc640 1 -- 172.21.15.62:0/715035074 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8003850 con 0x55d6c093a5d0 2024-09-11T23:52:55.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdae12e8740 1 -- 172.21.15.62:0/715035074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 msgr2=0x55d6c0936e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdae12e8740 1 --2- 172.21.15.62:0/715035074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0936e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdac80096c0 tx=0x7fdac8000bc0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdae12e8740 1 -- 172.21.15.62:0/715035074 shutdown_connections 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdae12e8740 1 --2- 172.21.15.62:0/715035074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c0945370 0x55d6c09418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdae12e8740 1 --2- 172.21.15.62:0/715035074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c09512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdae12e8740 1 --2- 172.21.15.62:0/715035074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c093a5d0 0x55d6c0936e10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdae12e8740 1 -- 172.21.15.62:0/715035074 >> 172.21.15.62:0/715035074 conn(0x55d6c0903310 msgr2=0x55d6c0936700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdae12e8740 1 -- 172.21.15.62:0/715035074 shutdown_connections 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.453+0000 7fdae12e8740 1 -- 172.21.15.62:0/715035074 wait complete. 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdae12e8740 1 -- start start 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdae12e8740 1 Processor -- start 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c0948540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c0948a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdae12e8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 0x55d6c09bdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdae12e8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81a9c40 con 0x55d6c093bb40 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdae12e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81ab0b0 con 0x55d6c093a5d0 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdae12e8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81aa310 con 0x55d6c0945370 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdae12e8740 10 client.0 _ll_register_callbacks cb 0x7fdad0009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c0948a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdadccab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c0948540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdadd4ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 0x55d6c09bdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdadccab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c0948540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:49480/0 (socket says 172.21.15.62:49480) 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdadccab640 1 -- 172.21.15.62:0/293001809 learned_addr learned my addr 172.21.15.62:0/293001809 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.454+0000 7fdadd4ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 0x55d6c09bdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:53498/0 (socket says 172.21.15.62:53498) 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdadccab640 1 -- 172.21.15.62:0/293001809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 msgr2=0x55d6c09bdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdadccab640 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 0x55d6c09bdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdadccab640 1 -- 172.21.15.62:0/293001809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 msgr2=0x55d6c0948a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdadccab640 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c0948a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdadccab640 1 -- 172.21.15.62:0/293001809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c09be130 con 0x55d6c093a5d0 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdadccab640 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c0948540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdac8050040 tx=0x7fdac8009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=161bc6b69ab7aff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdab7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdab7fff640 1 -- 172.21.15.62:0/293001809 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8035070 con 0x55d6c093a5d0 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdab7fff640 1 -- 172.21.15.62:0/293001809 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fdac8004030 con 0x55d6c093a5d0 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdae12e8740 1 -- 172.21.15.62:0/293001809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac803d760 con 0x55d6c093a5d0 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdab7fff640 1 -- 172.21.15.62:0/293001809 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdac8052040 con 0x55d6c093a5d0 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdae12e8740 10 client.25190 fetch_fsmap learned FSMap version 26 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.455+0000 7fdae12e8740 1 -- 172.21.15.62:0/293001809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d6c09bf310 con 0x55d6c093a5d0 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdab7fff640 1 -- 172.21.15.62:0/293001809 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac803d760 con 0x55d6c093a5d0 2024-09-11T23:52:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdab7fff640 1 -- 172.21.15.62:0/293001809 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fdac8060450 con 0x55d6c093a5d0 2024-09-11T23:52:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdae12e8740 10 client.25190 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:52:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdae12e8740 10 client.25190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdae12e8740 20 client.25190 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdae12e8740 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d6c09bf640 0x55d6c09c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdae12e8740 4 client.25190 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:52:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdae12e8740 1 -- 172.21.15.62:0/293001809 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6c08a7d70 con 0x55d6c09bf640 2024-09-11T23:52:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdad7fff640 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d6c09bf640 0x55d6c09c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdad7fff640 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d6c09bf640 0x55d6c09c1a30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7880866ed88b server_cookie=4b5d93c5abac1b7e in_seq=0 out_seq=0 2024-09-11T23:52:55.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.456+0000 7fdab7fff640 10 client.25190 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:55.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.459+0000 7fdab7fff640 1 -- 172.21.15.62:0/293001809 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6c08a7d70 con 0x55d6c09bf640 2024-09-11T23:52:55.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.459+0000 7fdab7fff640 10 client.25190 handle_command_reply: tid=0 2024-09-11T23:52:55.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.459+0000 7fdae12e8740 1 client.25190 shutdown 2024-09-11T23:52:55.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 1 -- 172.21.15.62:0/293001809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 msgr2=0x55d6c0948540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c0948540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdac8050040 tx=0x7fdac8009850 comp rx=0 tx=0).stop 2024-09-11T23:52:55.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 1 -- 172.21.15.62:0/293001809 shutdown_connections 2024-09-11T23:52:55.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d6c09bf640 0x55d6c09c1a30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d6c0945370 0x55d6c09bdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d6c093bb40 0x55d6c0948a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdab7fff640 0 client.25190 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:55.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 1 --2- 172.21.15.62:0/293001809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c093a5d0 0x55d6c0948540 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 1 -- 172.21.15.62:0/293001809 >> 172.21.15.62:0/293001809 conn(0x55d6c0903310 msgr2=0x55d6c09354e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:55.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 1 -- 172.21.15.62:0/293001809 shutdown_connections 2024-09-11T23:52:55.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 1 -- 172.21.15.62:0/293001809 wait complete. 2024-09-11T23:52:55.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdae12e8740 20 client.25190 trim_cache size 0 max 16384 2024-09-11T23:52:55.461 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:52:55.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdadef0f640 1 -- 172.21.15.62:0/1582493411 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdac40a2e00 msgr2=0x7fdac40a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdadef0f640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdac40a2e00 0x7fdac40a52c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009fd0 tx=0x7fdad0004ab0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdadef0f640 1 -- 172.21.15.62:0/1582493411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdad8100ca0 msgr2=0x7fdad8198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.460+0000 7fdadef0f640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdad8100ca0 0x7fdad8198a40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e9f0 tx=0x7fdac000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.461+0000 7fdadef0f640 1 -- 172.21.15.62:0/1582493411 shutdown_connections 2024-09-11T23:52:55.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.461+0000 7fdadef0f640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdac40a2e00 0x7fdac40a52c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.461+0000 7fdadef0f640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdad8106f50 0x7fdad819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.461+0000 7fdadef0f640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdad8100ca0 0x7fdad8198a40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.461+0000 7fdadef0f640 1 --2- 172.21.15.62:0/1582493411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad81002f0 0x7fdad8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.461+0000 7fdadef0f640 1 -- 172.21.15.62:0/1582493411 >> 172.21.15.62:0/1582493411 conn(0x7fdad8077210 msgr2=0x7fdad81052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.461+0000 7fdadef0f640 1 -- 172.21.15.62:0/1582493411 shutdown_connections 2024-09-11T23:52:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.461+0000 7fdadef0f640 1 -- 172.21.15.62:0/1582493411 wait complete. 2024-09-11T23:52:55.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:55 smithi120 ceph-mon[35182]: pgmap v27: 97 pgs: 97 active+clean; 925 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 78 MiB/s wr, 495 op/s 2024-09-11T23:52:55.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:55 smithi083 ceph-mon[34805]: pgmap v27: 97 pgs: 97 active+clean; 925 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 78 MiB/s wr, 495 op/s 2024-09-11T23:52:55.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.790+0000 7f7e437fe640 1 -- 172.21.15.62:0/471453677 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55cfeebe0d20 con 0x55cfeecfa400 2024-09-11T23:52:55.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.790+0000 7f7e437fe640 10 client.25153 handle_command_reply: tid=0 2024-09-11T23:52:55.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.790+0000 7f7e6c6f0740 1 client.25153 shutdown 2024-09-11T23:52:55.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.790+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/471453677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec73760 msgr2=0x55cfeec86b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.790+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec73760 0x55cfeec86b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0cd300 tx=0x7f7e5c09bb50 comp rx=0 tx=0).stop 2024-09-11T23:52:55.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/471453677 shutdown_connections 2024-09-11T23:52:55.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55cfeecfa400 0x55cfeecfc820 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cfeec7e940 0x55cfeec83910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfeec74080 0x55cfeec87060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6c6f0740 1 --2- 172.21.15.62:0/471453677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfeec73760 0x55cfeec86b20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e437fe640 0 client.25153 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:55.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/471453677 >> 172.21.15.62:0/471453677 conn(0x55cfeec6fcf0 msgr2=0x55cfeec6ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:55.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/471453677 shutdown_connections 2024-09-11T23:52:55.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6c6f0740 1 -- 172.21.15.62:0/471453677 wait complete. 2024-09-11T23:52:55.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6c6f0740 20 client.25153 trim_cache size 0 max 16384 2024-09-11T23:52:55.791 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:52:55.791 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 26, 2024-09-11T23:52:55.791 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:52:55.791 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 2, 2024-09-11T23:52:55.791 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "85ba7803": { 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 2, 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 144.5, 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:52:55.792 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:52:55.793 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:52:55.793 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:52:55.793 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:52:55.793 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:52:55.793 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:52:55.793 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:52:55.793 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:52:55.793 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7e4c0a2ed0 msgr2=0x7f7e4c0a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6a317640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7e4c0a2ed0 0x7f7e4c0a5390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009bb0 tx=0x7f7e58009790 comp rx=0 tx=0).stop 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e64101de0 msgr2=0x7f7e64075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.791+0000 7f7e6a317640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e64101de0 0x7f7e64075b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009fd0 tx=0x7f7e5c00ef90 comp rx=0 tx=0).stop 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.792+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 shutdown_connections 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.792+0000 7f7e6a317640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7e4c0a2ed0 0x7f7e4c0a5390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.792+0000 7f7e6a317640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e64106f10 0x7f7e6406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.792+0000 7f7e6a317640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e64102790 0x7f7e64076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.792+0000 7f7e6a317640 1 --2- 172.21.15.62:0/1701333184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e64101de0 0x7f7e64075b00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.792+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 >> 172.21.15.62:0/1701333184 conn(0x7f7e640fd8c0 msgr2=0x7f7e640febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.792+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 shutdown_connections 2024-09-11T23:52:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:55.792+0000 7f7e6a317640 1 -- 172.21.15.62:0/1701333184 wait complete. 2024-09-11T23:52:56.897 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '85ba7803', quiesce took 7.7 seconds. Will release after: 16.8 2024-09-11T23:52:57.448 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 15.0 s, 1439.8 tps, lat 28.671 ms stddev 26.600 2024-09-11T23:52:57.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:57 smithi062 ceph-mon[30958]: pgmap v28: 97 pgs: 97 active+clean; 925 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 457 op/s 2024-09-11T23:52:57.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:57 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:52:57.468 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:57 smithi083 ceph-mon[34805]: pgmap v28: 97 pgs: 97 active+clean; 925 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 457 op/s 2024-09-11T23:52:57.468 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:57 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:52:57.478 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:52:57.479 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:52:57.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:57 smithi120 ceph-mon[35182]: pgmap v28: 97 pgs: 97 active+clean; 925 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 457 op/s 2024-09-11T23:52:57.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:57 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:52:57.752 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:58.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.176+0000 7f06ed0ac640 1 -- 172.21.15.62:0/3845720827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e8102ef0 msgr2=0x7f06e810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:58.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.176+0000 7f06ed0ac640 1 --2- 172.21.15.62:0/3845720827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e8102ef0 0x7f06e810da40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009ca0 tx=0x7f06e002a950 comp rx=0 tx=0).stop 2024-09-11T23:52:58.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.176+0000 7f06ed0ac640 1 -- 172.21.15.62:0/3845720827 shutdown_connections 2024-09-11T23:52:58.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.176+0000 7f06ed0ac640 1 --2- 172.21.15.62:0/3845720827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e8102ef0 0x7f06e810da40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.176+0000 7f06ed0ac640 1 --2- 172.21.15.62:0/3845720827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06e8102570 0x7f06e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.176+0000 7f06ed0ac640 1 --2- 172.21.15.62:0/3845720827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e8100e20 0x7f06e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.176+0000 7f06ed0ac640 1 -- 172.21.15.62:0/3845720827 >> 172.21.15.62:0/3845720827 conn(0x7f06e80fc7d0 msgr2=0x7f06e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:58.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.176+0000 7f06ed0ac640 1 -- 172.21.15.62:0/3845720827 shutdown_connections 2024-09-11T23:52:58.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.176+0000 7f06ed0ac640 1 -- 172.21.15.62:0/3845720827 wait complete. 2024-09-11T23:52:58.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06ed0ac640 1 Processor -- start 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06ed0ac640 1 -- start start 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06ed0ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06e8100e20 0x7f06e80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06ed0ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e8102570 0x7f06e80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06ed0ac640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e8102ef0 0x7f06e80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06ed0ac640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06e8110f80 con 0x7f06e8102ef0 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06ed0ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06e8110e00 con 0x7f06e8102570 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06ed0ac640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06e8111100 con 0x7f06e8100e20 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06e759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e8102ef0 0x7f06e80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06e759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e8102ef0 0x7f06e80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41716/0 (socket says 172.21.15.62:41716) 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06e759e640 1 -- 172.21.15.62:0/1422985660 learned_addr learned my addr 172.21.15.62:0/1422985660 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06de59c640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e8102570 0x7f06e80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.177+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06e8100e20 0x7f06e80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06e759e640 1 -- 172.21.15.62:0/1422985660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06e8100e20 msgr2=0x7f06e80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06e759e640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06e8100e20 0x7f06e80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06e759e640 1 -- 172.21.15.62:0/1422985660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e8102570 msgr2=0x7f06e80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06e759e640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e8102570 0x7f06e80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06e759e640 1 -- 172.21.15.62:0/1422985660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e81a69e0 con 0x7f06e8102ef0 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06e8100e20 0x7f06e80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06e759e640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e8102ef0 0x7f06e80fd700 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f06e0006010 tx=0x7f06e002f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06e4d99640 1 -- 172.21.15.62:0/1422985660 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0034040 con 0x7f06e8102ef0 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06ed0ac640 1 -- 172.21.15.62:0/1422985660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e81a6c70 con 0x7f06e8102ef0 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06e4d99640 1 -- 172.21.15.62:0/1422985660 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e0003e00 con 0x7f06e8102ef0 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06e4d99640 1 -- 172.21.15.62:0/1422985660 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0045480 con 0x7f06e8102ef0 2024-09-11T23:52:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.178+0000 7f06ed0ac640 1 -- 172.21.15.62:0/1422985660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e81a7150 con 0x7f06e8102ef0 2024-09-11T23:52:58.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.180+0000 7f06e4d99640 1 -- 172.21.15.62:0/1422985660 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f06e0037020 con 0x7f06e8102ef0 2024-09-11T23:52:58.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06ef485740 1 Processor -- start 2024-09-11T23:52:58.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06ef485740 1 -- start start 2024-09-11T23:52:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06ef485740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff4b60 0x560df2ff4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06ef485740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df2fff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06ef485740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df3002130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06ef485740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560df2f5a280 con 0x560df2ff5480 2024-09-11T23:52:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06ef485740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560df2f01b10 con 0x560df2ff4b60 2024-09-11T23:52:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06ef485740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560df2f56240 con 0x560df2fffd40 2024-09-11T23:52:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06e6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df2fff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06e4d99640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f06d40a2d80 0x7f06d40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06e6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df2fff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41720/0 (socket says 172.21.15.62:41720) 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06e6d9d640 1 -- 172.21.15.62:0/2467157772 learned_addr learned my addr 172.21.15.62:0/2467157772 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.181+0000 7f06de59c640 1 --2- 172.21.15.62:0/2467157772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff4b60 0x560df2ff4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06e759e640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f06d40a2d80 0x7f06d40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06de59c640 1 --2- 172.21.15.62:0/2467157772 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df3002130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06e6d9d640 1 -- 172.21.15.62:0/2467157772 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 msgr2=0x560df3002130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/2467157772 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df3002130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06e4d99640 1 -- 172.21.15.62:0/1422985660 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f06e00f0630 con 0x7f06e8102ef0 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06e6d9d640 1 -- 172.21.15.62:0/2467157772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff4b60 msgr2=0x560df2ff4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/2467157772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff4b60 0x560df2ff4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06e6d9d640 1 -- 172.21.15.62:0/2467157772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df2f0e1f0 con 0x560df2ff5480 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/2467157772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df2fff800 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f06d800cca0 tx=0x7f06d800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe4d2918fe96cd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06e759e640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f06d40a2d80 0x7f06d40a5240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f06e00ede60 tx=0x7f06e002a950 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06dd59a640 1 -- 172.21.15.62:0/2467157772 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8005220 con 0x560df2ff5480 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06dd59a640 1 -- 172.21.15.62:0/2467157772 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d8004c80 con 0x560df2ff5480 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.182+0000 7f06dd59a640 1 -- 172.21.15.62:0/2467157772 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8010890 con 0x560df2ff5480 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.183+0000 7f06ef485740 1 -- 172.21.15.62:0/2467157772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 msgr2=0x560df2fff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.183+0000 7f06ef485740 1 --2- 172.21.15.62:0/2467157772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df2fff800 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f06d800cca0 tx=0x7f06d800e720 comp rx=0 tx=0).stop 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.183+0000 7f06ef485740 1 -- 172.21.15.62:0/2467157772 shutdown_connections 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.183+0000 7f06ef485740 1 --2- 172.21.15.62:0/2467157772 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df3002130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.183+0000 7f06ef485740 1 --2- 172.21.15.62:0/2467157772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df2fff800 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.183+0000 7f06ef485740 1 --2- 172.21.15.62:0/2467157772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff4b60 0x560df2ff4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.183+0000 7f06ef485740 1 -- 172.21.15.62:0/2467157772 >> 172.21.15.62:0/2467157772 conn(0x560df2ff10f0 msgr2=0x560df2ff14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.183+0000 7f06ef485740 1 -- 172.21.15.62:0/2467157772 shutdown_connections 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.183+0000 7f06ef485740 1 -- 172.21.15.62:0/2467157772 wait complete. 2024-09-11T23:52:58.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.183+0000 7f06ef485740 1 -- start start 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06ef485740 1 Processor -- start 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06ef485740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ffcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06ef485740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2ff5480 0x560df2ffd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06ef485740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2fffd40 0x560df2ffd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06ef485740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560df3002910 con 0x560df2ff4b60 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06ef485740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560df2f14f10 con 0x560df2fffd40 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06ef485740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560df3002a90 con 0x560df2ff5480 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06ef485740 10 client.0 _ll_register_callbacks cb 0x560df2edda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06de59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ffcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06de59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ffcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41728/0 (socket says 172.21.15.62:41728) 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06de59c640 1 -- 172.21.15.62:0/2042738764 learned_addr learned my addr 172.21.15.62:0/2042738764 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2ff5480 0x560df2ffd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06de59c640 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2fffd40 0x560df2ffd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06de59c640 1 -- 172.21.15.62:0/2042738764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2ff5480 msgr2=0x560df2ffd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06de59c640 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2ff5480 0x560df2ffd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06de59c640 1 -- 172.21.15.62:0/2042738764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2fffd40 msgr2=0x560df2ffd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06de59c640 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2fffd40 0x560df2ffd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.184+0000 7f06de59c640 1 -- 172.21.15.62:0/2042738764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df2ccb6a0 con 0x560df2ff4b60 2024-09-11T23:52:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06de59c640 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ffcb60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f06d0011e60 tx=0x7f06d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b58f3adec6ae530 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ce7fc640 1 -- 172.21.15.62:0/2042738764 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0004220 con 0x560df2ff4b60 2024-09-11T23:52:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ce7fc640 1 -- 172.21.15.62:0/2042738764 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f06d00043c0 con 0x560df2ff4b60 2024-09-11T23:52:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ef485740 1 -- 172.21.15.62:0/2042738764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d8004c80 con 0x560df2ff4b60 2024-09-11T23:52:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ce7fc640 1 -- 172.21.15.62:0/2042738764 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06d000ae20 con 0x560df2ff4b60 2024-09-11T23:52:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ef485740 10 client.15519 fetch_fsmap learned FSMap version 27 2024-09-11T23:52:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ef485740 1 -- 172.21.15.62:0/2042738764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560df300e520 con 0x560df2ff4b60 2024-09-11T23:52:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ce7fc640 1 -- 172.21.15.62:0/2042738764 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d8004c80 con 0x560df2ff4b60 2024-09-11T23:52:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ce7fc640 1 -- 172.21.15.62:0/2042738764 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f06d0004c90 con 0x560df2ff4b60 2024-09-11T23:52:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ef485740 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:52:58.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ef485740 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:58.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ef485740 20 client.15519 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:58.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ef485740 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560df300fd50 0x560df30121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ef485740 4 client.15519 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:58.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.185+0000 7f06ef485740 1 -- 172.21.15.62:0/2042738764 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560df2f677f0 con 0x560df300fd50 2024-09-11T23:52:58.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.186+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560df300fd50 0x560df30121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.186+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560df300fd50 0x560df30121d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfae4b831908f052 server_cookie=652a7dcefcaafada in_seq=0 out_seq=0 2024-09-11T23:52:58.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.186+0000 7f06ce7fc640 10 client.15519 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:58.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ce7fc640 1 -- 172.21.15.62:0/2042738764 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560df2f677f0 con 0x560df300fd50 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ce7fc640 10 client.15519 handle_command_reply: tid=0 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 client.15519 shutdown 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 -- 172.21.15.62:0/2042738764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 msgr2=0x560df2ffcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ffcb60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f06d0011e60 tx=0x7f06d0004730 comp rx=0 tx=0).stop 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 -- 172.21.15.62:0/2042738764 shutdown_connections 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560df300fd50 0x560df30121d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2fffd40 0x560df2ffd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2ff5480 0x560df2ffd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 --2- 172.21.15.62:0/2042738764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ffcb60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ce7fc640 0 client.15519 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 -- 172.21.15.62:0/2042738764 >> 172.21.15.62:0/2042738764 conn(0x560df2ff10f0 msgr2=0x560df2ff0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 -- 172.21.15.62:0/2042738764 shutdown_connections 2024-09-11T23:52:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.187+0000 7f06ef485740 1 -- 172.21.15.62:0/2042738764 wait complete. 2024-09-11T23:52:58.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.188+0000 7f06ef485740 20 client.15519 trim_cache size 0 max 16384 2024-09-11T23:52:58.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.205+0000 7f06ef485740 1 Processor -- start 2024-09-11T23:52:58.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.205+0000 7f06ef485740 1 -- start start 2024-09-11T23:52:58.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06ef485740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ff6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06ef485740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff5480 0x560df2ff9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06ef485740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df2ffa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06ef485740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81b22b0 con 0x560df2ff4b60 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06ef485740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81afb80 con 0x560df2ff5480 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06ef485740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81ad9d0 con 0x560df2fffd40 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06de59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ff6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06de59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ff6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41744/0 (socket says 172.21.15.62:41744) 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06de59c640 1 -- 172.21.15.62:0/1381404703 learned_addr learned my addr 172.21.15.62:0/1381404703 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06de59c640 1 --2- 172.21.15.62:0/1381404703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df2ffa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/1381404703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff5480 0x560df2ff9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06de59c640 1 -- 172.21.15.62:0/1381404703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 msgr2=0x560df2ffa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06de59c640 1 --2- 172.21.15.62:0/1381404703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df2ffa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06de59c640 1 -- 172.21.15.62:0/1381404703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff5480 msgr2=0x560df2ff9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06de59c640 1 --2- 172.21.15.62:0/1381404703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff5480 0x560df2ff9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06de59c640 1 -- 172.21.15.62:0/1381404703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df2ff52b0 con 0x560df2ff4b60 2024-09-11T23:52:58.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.206+0000 7f06de59c640 1 --2- 172.21.15.62:0/1381404703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ff6cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f06d0004700 tx=0x7f06d00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3050a9bd3fb761 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:58.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06dcd99640 1 -- 172.21.15.62:0/1381404703 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d002d030 con 0x560df2ff4b60 2024-09-11T23:52:58.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06dcd99640 1 -- 172.21.15.62:0/1381404703 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d0003860 con 0x560df2ff4b60 2024-09-11T23:52:58.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06dcd99640 1 -- 172.21.15.62:0/1381404703 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0027660 con 0x560df2ff4b60 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06ef485740 1 -- 172.21.15.62:0/1381404703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 msgr2=0x560df2ff6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06ef485740 1 --2- 172.21.15.62:0/1381404703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ff6cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f06d0004700 tx=0x7f06d00058a0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06ef485740 1 -- 172.21.15.62:0/1381404703 shutdown_connections 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06ef485740 1 --2- 172.21.15.62:0/1381404703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df2ffa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06ef485740 1 --2- 172.21.15.62:0/1381404703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff5480 0x560df2ff9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06ef485740 1 --2- 172.21.15.62:0/1381404703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff4b60 0x560df2ff6cc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06ef485740 1 -- 172.21.15.62:0/1381404703 >> 172.21.15.62:0/1381404703 conn(0x560df2ff10f0 msgr2=0x560df2ff65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06ef485740 1 -- 172.21.15.62:0/1381404703 shutdown_connections 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.207+0000 7f06ef485740 1 -- 172.21.15.62:0/1381404703 wait complete. 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06ef485740 1 -- start start 2024-09-11T23:52:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06ef485740 1 Processor -- start 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06ef485740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff4b60 0x560df3018100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06ef485740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df30154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06ef485740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df30159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06ef485740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81ae040 con 0x560df2ff5480 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06ef485740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81af4b0 con 0x560df2ff4b60 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06ef485740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81ae710 con 0x560df2fffd40 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06ef485740 10 client.0 _ll_register_callbacks cb 0x7f06d000a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06e6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df30154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06e6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df30154a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41754/0 (socket says 172.21.15.62:41754) 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06e6d9d640 1 -- 172.21.15.62:0/1947223908 learned_addr learned my addr 172.21.15.62:0/1947223908 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06de59c640 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff4b60 0x560df3018100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.208+0000 7f06de59c640 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df30159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06e6d9d640 1 -- 172.21.15.62:0/1947223908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 msgr2=0x560df30159e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df30159e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06e6d9d640 1 -- 172.21.15.62:0/1947223908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff4b60 msgr2=0x560df3018100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff4b60 0x560df3018100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06e6d9d640 1 -- 172.21.15.62:0/1947223908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df30161c0 con 0x560df2ff5480 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06e6d9d640 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df30154a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f06d8004f50 tx=0x7f06d8010570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da1c92c7ab69e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06ce7fc640 1 -- 172.21.15.62:0/1947223908 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d802b070 con 0x560df2ff5480 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06ce7fc640 1 -- 172.21.15.62:0/1947223908 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f06d8004a70 con 0x560df2ff5480 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06ef485740 1 -- 172.21.15.62:0/1947223908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d0003860 con 0x560df2ff5480 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06ce7fc640 1 -- 172.21.15.62:0/1947223908 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06d800c8b0 con 0x560df2ff5480 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06ef485740 10 client.15525 fetch_fsmap learned FSMap version 27 2024-09-11T23:52:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06ef485740 1 -- 172.21.15.62:0/1947223908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560df307a9f0 con 0x560df2ff5480 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.209+0000 7f06ce7fc640 1 -- 172.21.15.62:0/1947223908 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d0003860 con 0x560df2ff5480 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.210+0000 7f06ce7fc640 1 -- 172.21.15.62:0/1947223908 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f06d8010a30 con 0x560df2ff5480 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.210+0000 7f06ef485740 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.210+0000 7f06ef485740 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.210+0000 7f06ef485740 20 client.15525 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.210+0000 7f06ef485740 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560df307ae30 0x560df307d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.210+0000 7f06ef485740 4 client.15525 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.210+0000 7f06ef485740 1 -- 172.21.15.62:0/1947223908 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560df2f61d00 con 0x560df307ae30 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.210+0000 7f06de59c640 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560df307ae30 0x560df307d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.211+0000 7f06de59c640 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560df307ae30 0x560df307d220 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df3d7d1efef4644 server_cookie=dac0f248f7c4d4fa in_seq=0 out_seq=0 2024-09-11T23:52:58.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.211+0000 7f06ce7fc640 10 client.15525 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.211+0000 7f06ce7fc640 1 -- 172.21.15.62:0/1947223908 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560df2f61d00 con 0x560df307ae30 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.211+0000 7f06ce7fc640 10 client.15525 handle_command_reply: tid=0 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.211+0000 7f06ef485740 1 client.15525 shutdown 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 1 -- 172.21.15.62:0/1947223908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 msgr2=0x560df30154a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df30154a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f06d8004f50 tx=0x7f06d8010570 comp rx=0 tx=0).stop 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 1 -- 172.21.15.62:0/1947223908 shutdown_connections 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560df307ae30 0x560df307d220 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560df2fffd40 0x560df30159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560df2ff5480 0x560df30154a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 1 --2- 172.21.15.62:0/1947223908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560df2ff4b60 0x560df3018100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 1 -- 172.21.15.62:0/1947223908 >> 172.21.15.62:0/1947223908 conn(0x560df2ff10f0 msgr2=0x560df300b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ce7fc640 0 client.15525 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:58.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 1 -- 172.21.15.62:0/1947223908 shutdown_connections 2024-09-11T23:52:58.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 1 -- 172.21.15.62:0/1947223908 wait complete. 2024-09-11T23:52:58.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.212+0000 7f06ef485740 20 client.15525 trim_cache size 0 max 16384 2024-09-11T23:52:58.217 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:52:58.217 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:52:58.217 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "e3b9d6ac-1e5b-4481-b4c8-8d160af9fa95", 2024-09-11T23:52:58.217 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:52:58.217 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:52:58.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 -- 172.21.15.62:0/1422985660 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f06d40a2d80 msgr2=0x7f06d40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:58.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f06d40a2d80 0x7f06d40a5240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f06e00ede60 tx=0x7f06e002a950 comp rx=0 tx=0).stop 2024-09-11T23:52:58.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 -- 172.21.15.62:0/1422985660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e8102ef0 msgr2=0x7f06e80fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:58.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e8102ef0 0x7f06e80fd700 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f06e0006010 tx=0x7f06e002f6f0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 -- 172.21.15.62:0/1422985660 shutdown_connections 2024-09-11T23:52:58.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f06d40a2d80 0x7f06d40a5240 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e8102ef0 0x7f06e80fd700 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e8102570 0x7f06e80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 --2- 172.21.15.62:0/1422985660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06e8100e20 0x7f06e80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:58.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 -- 172.21.15.62:0/1422985660 >> 172.21.15.62:0/1422985660 conn(0x7f06e80fc7d0 msgr2=0x7f06e806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:58.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 -- 172.21.15.62:0/1422985660 shutdown_connections 2024-09-11T23:52:58.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:58.213+0000 7f06ed0ac640 1 -- 172.21.15.62:0/1422985660 wait complete. 2024-09-11T23:52:58.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:52:59.257 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:52:59.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:59 smithi062 ceph-mon[30958]: pgmap v29: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 25 MiB/s wr, 735 op/s 2024-09-11T23:52:59.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:59 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:52:59.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:59 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:59.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:59 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:52:59.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:59 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:52:59.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:59 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:59.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:52:59 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:59.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:59 smithi120 ceph-mon[35182]: pgmap v29: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 25 MiB/s wr, 735 op/s 2024-09-11T23:52:59.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:59 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:52:59.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:59 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:59.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:59 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:52:59.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:59 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:52:59.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:59 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:59.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:52:59 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:59.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:59 smithi083 ceph-mon[34805]: pgmap v29: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 25 MiB/s wr, 735 op/s 2024-09-11T23:52:59.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:59 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:52:59.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:59 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:59.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:59 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:52:59.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:59 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:52:59.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:59 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:59.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:52:59 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:52:59.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.732+0000 7fad36389640 1 -- 172.21.15.62:0/1546218605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad30106a80 msgr2=0x7fad30106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:59.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.732+0000 7fad36389640 1 --2- 172.21.15.62:0/1546218605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad30106a80 0x7fad30106e60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fad20009930 tx=0x7fad2002a440 comp rx=0 tx=0).stop 2024-09-11T23:52:59.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.732+0000 7fad36389640 1 -- 172.21.15.62:0/1546218605 shutdown_connections 2024-09-11T23:52:59.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.732+0000 7fad36389640 1 --2- 172.21.15.62:0/1546218605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad30101420 0x7fad3010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.732+0000 7fad36389640 1 --2- 172.21.15.62:0/1546218605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad30100a80 0x7fad30100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.732+0000 7fad36389640 1 --2- 172.21.15.62:0/1546218605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad30106a80 0x7fad30106e60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.732+0000 7fad36389640 1 -- 172.21.15.62:0/1546218605 >> 172.21.15.62:0/1546218605 conn(0x7fad300fc7a0 msgr2=0x7fad300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.732+0000 7fad36389640 1 -- 172.21.15.62:0/1546218605 shutdown_connections 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.732+0000 7fad36389640 1 -- 172.21.15.62:0/1546218605 wait complete. 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.733+0000 7fad36389640 1 Processor -- start 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.733+0000 7fad36389640 1 -- start start 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.733+0000 7fad36389640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad30100a80 0x7fad30102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.733+0000 7fad36389640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad30101420 0x7fad30102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.733+0000 7fad36389640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad30106a80 0x7fad301030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.733+0000 7fad36389640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad30110e80 con 0x7fad30106a80 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.733+0000 7fad36389640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad30110d00 con 0x7fad30100a80 2024-09-11T23:52:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.733+0000 7fad36389640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad30111000 con 0x7fad30101420 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad34926640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad30106a80 0x7fad301030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad34926640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad30106a80 0x7fad301030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41764/0 (socket says 172.21.15.62:41764) 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad34926640 1 -- 172.21.15.62:0/3658816433 learned_addr learned my addr 172.21.15.62:0/3658816433 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad2ffff640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad30100a80 0x7fad30102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad30101420 0x7fad30102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad34926640 1 -- 172.21.15.62:0/3658816433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad30101420 msgr2=0x7fad30102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad34926640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad30101420 0x7fad30102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad34926640 1 -- 172.21.15.62:0/3658816433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad30100a80 msgr2=0x7fad30102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad34926640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad30100a80 0x7fad30102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad34926640 1 -- 172.21.15.62:0/3658816433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad301a6890 con 0x7fad30106a80 2024-09-11T23:52:59.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad34926640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad30106a80 0x7fad301030d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fad2800cce0 tx=0x7fad2800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad2d7fa640 1 -- 172.21.15.62:0/3658816433 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2800f450 con 0x7fad30106a80 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad36389640 1 -- 172.21.15.62:0/3658816433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad301a6b80 con 0x7fad30106a80 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad2d7fa640 1 -- 172.21.15.62:0/3658816433 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad2800fad0 con 0x7fad30106a80 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad2d7fa640 1 -- 172.21.15.62:0/3658816433 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28018c10 con 0x7fad30106a80 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.734+0000 7fad36389640 1 -- 172.21.15.62:0/3658816433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad301a70c0 con 0x7fad30106a80 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.737+0000 7fad2d7fa640 1 -- 172.21.15.62:0/3658816433 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fad2800f5f0 con 0x7fad30106a80 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.737+0000 7fad38762740 1 Processor -- start 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.737+0000 7fad38762740 1 -- start start 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.737+0000 7fad38762740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.737+0000 7fad38762740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fc5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.737+0000 7fad38762740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.737+0000 7fad38762740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556d36f202e0 con 0x556d36fc5680 2024-09-11T23:52:59.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.737+0000 7fad38762740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556d36ec7ce0 con 0x556d36fbadc0 2024-09-11T23:52:59.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.737+0000 7fad38762740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556d36f1c2a0 con 0x556d36fba4a0 2024-09-11T23:52:59.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41776/0 (socket says 172.21.15.62:41776) 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2f7fe640 1 -- 172.21.15.62:0/3725163165 learned_addr learned my addr 172.21.15.62:0/3725163165 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2d7fa640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad180a2d80 0x7fad180a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/3725163165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2ffff640 1 --2- 172.21.15.62:0/3725163165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fc5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2f7fe640 1 -- 172.21.15.62:0/3725163165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 msgr2=0x556d36fba880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/3725163165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fba880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2f7fe640 1 -- 172.21.15.62:0/3725163165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 msgr2=0x556d36fc5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/3725163165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fc5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad34926640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad180a2d80 0x7fad180a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2f7fe640 1 -- 172.21.15.62:0/3725163165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d36eac440 con 0x556d36fc5680 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2ffff640 1 --2- 172.21.15.62:0/3725163165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fc5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2d7fa640 1 -- 172.21.15.62:0/3658816433 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fad280cfca0 con 0x7fad30106a80 2024-09-11T23:52:59.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/3725163165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc7a70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fad24013580 tx=0x7fad2400c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0d5d77deb1c910 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad167fc640 1 -- 172.21.15.62:0/3725163165 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24019030 con 0x556d36fc5680 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad34926640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad180a2d80 0x7fad180a5240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad280cd360 tx=0x7fad28016f60 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad167fc640 1 -- 172.21.15.62:0/3725163165 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad24011a00 con 0x556d36fc5680 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.738+0000 7fad167fc640 1 -- 172.21.15.62:0/3725163165 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2400fa70 con 0x556d36fc5680 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.739+0000 7fad38762740 1 -- 172.21.15.62:0/3725163165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 msgr2=0x556d36fc7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.739+0000 7fad38762740 1 --2- 172.21.15.62:0/3725163165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc7a70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fad24013580 tx=0x7fad2400c6b0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.739+0000 7fad38762740 1 -- 172.21.15.62:0/3725163165 shutdown_connections 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.739+0000 7fad38762740 1 --2- 172.21.15.62:0/3725163165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc7a70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.739+0000 7fad38762740 1 --2- 172.21.15.62:0/3725163165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fc5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.739+0000 7fad38762740 1 --2- 172.21.15.62:0/3725163165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fba880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.739+0000 7fad38762740 1 -- 172.21.15.62:0/3725163165 >> 172.21.15.62:0/3725163165 conn(0x556d36fb6a30 msgr2=0x556d36fb6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.739+0000 7fad38762740 1 -- 172.21.15.62:0/3725163165 shutdown_connections 2024-09-11T23:52:59.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.739+0000 7fad38762740 1 -- 172.21.15.62:0/3725163165 wait complete. 2024-09-11T23:52:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad38762740 1 -- start start 2024-09-11T23:52:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad38762740 1 Processor -- start 2024-09-11T23:52:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad38762740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fc5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad38762740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fc2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad38762740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad38762740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556d36fc8250 con 0x556d36fc5680 2024-09-11T23:52:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad38762740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556d36edb050 con 0x556d36fbadc0 2024-09-11T23:52:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad38762740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556d36fc83d0 con 0x556d36fba4a0 2024-09-11T23:52:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad38762740 10 client.0 _ll_register_callbacks cb 0x556d36f1b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41778/0 (socket says 172.21.15.62:41778) 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad2f7fe640 1 -- 172.21.15.62:0/1018019233 learned_addr learned my addr 172.21.15.62:0/1018019233 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.740+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fc2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fc5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad2f7fe640 1 -- 172.21.15.62:0/1018019233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 msgr2=0x556d36fc5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fc5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad2f7fe640 1 -- 172.21.15.62:0/1018019233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 msgr2=0x556d36fc2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fc2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad2f7fe640 1 -- 172.21.15.62:0/1018019233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d36c913c0 con 0x556d36fc5680 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc28f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fad24011f70 tx=0x7fad24015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0612cc2b60d475 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:59.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1018019233 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24019070 con 0x556d36fc5680 2024-09-11T23:52:59.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad38762740 1 -- 172.21.15.62:0/1018019233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad24011a00 con 0x556d36fc5680 2024-09-11T23:52:59.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1018019233 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fad240117d0 con 0x556d36fc5680 2024-09-11T23:52:59.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1018019233 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad24032050 con 0x556d36fc5680 2024-09-11T23:52:59.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad38762740 10 client.15537 fetch_fsmap learned FSMap version 27 2024-09-11T23:52:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad38762740 1 -- 172.21.15.62:0/1018019233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d36fd3c80 con 0x556d36fc5680 2024-09-11T23:52:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.741+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1018019233 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad24011a00 con 0x556d36fc5680 2024-09-11T23:52:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.742+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1018019233 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fad2401e8a0 con 0x556d36fc5680 2024-09-11T23:52:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.742+0000 7fad38762740 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:52:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.742+0000 7fad38762740 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.742+0000 7fad38762740 20 client.15537 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.742+0000 7fad38762740 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d36fd5030 0x556d36fd7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.742+0000 7fad38762740 4 client.15537 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:52:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.742+0000 7fad38762740 1 -- 172.21.15.62:0/1018019233 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d36f2d850 con 0x556d36fd5030 2024-09-11T23:52:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.742+0000 7fad2ffff640 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d36fd5030 0x556d36fd7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.742+0000 7fad0f7fe640 10 client.15537 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.742+0000 7fad2ffff640 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d36fd5030 0x556d36fd7420 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dd2785a526f8717 server_cookie=eb666912f362dc17 in_seq=0 out_seq=0 2024-09-11T23:52:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.743+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1018019233 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d36f2d850 con 0x556d36fd5030 2024-09-11T23:52:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.743+0000 7fad0f7fe640 10 client.15537 handle_command_reply: tid=0 2024-09-11T23:52:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.743+0000 7fad38762740 1 client.15537 shutdown 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.743+0000 7fad38762740 1 -- 172.21.15.62:0/1018019233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 msgr2=0x556d36fc28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.743+0000 7fad38762740 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc28f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fad24011f70 tx=0x7fad24015040 comp rx=0 tx=0).stop 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.743+0000 7fad38762740 1 -- 172.21.15.62:0/1018019233 shutdown_connections 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.743+0000 7fad38762740 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d36fd5030 0x556d36fd7420 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.743+0000 7fad38762740 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fc28f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.743+0000 7fad0f7fe640 0 client.15537 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.743+0000 7fad38762740 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fc2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.744+0000 7fad38762740 1 --2- 172.21.15.62:0/1018019233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fc5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.744+0000 7fad38762740 1 -- 172.21.15.62:0/1018019233 >> 172.21.15.62:0/1018019233 conn(0x556d36fb6a30 msgr2=0x556d36fbc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.744+0000 7fad38762740 1 -- 172.21.15.62:0/1018019233 shutdown_connections 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.744+0000 7fad38762740 1 -- 172.21.15.62:0/1018019233 wait complete. 2024-09-11T23:52:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.744+0000 7fad38762740 20 client.15537 trim_cache size 0 max 16384 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad38762740 1 Processor -- start 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad38762740 1 -- start start 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad38762740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fbdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad38762740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fd14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad38762740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fdc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad38762740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad301b2280 con 0x556d36fc5680 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad38762740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad301afb50 con 0x556d36fbadc0 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad38762740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad301ad9a0 con 0x556d36fba4a0 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fdc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fdc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41780/0 (socket says 172.21.15.62:41780) 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad2f7fe640 1 -- 172.21.15.62:0/3857095988 learned_addr learned my addr 172.21.15.62:0/3857095988 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad2ffff640 1 --2- 172.21.15.62:0/3857095988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fd14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.761+0000 7fad2f7fe640 1 -- 172.21.15.62:0/3857095988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 msgr2=0x556d36fbdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/3857095988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fbdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad2f7fe640 1 -- 172.21.15.62:0/3857095988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 msgr2=0x556d36fd14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/3857095988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fd14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad2f7fe640 1 -- 172.21.15.62:0/3857095988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d36fc53c0 con 0x556d36fc5680 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/3857095988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fdc9f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fad2400f5a0 tx=0x7fad2401d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807f2cd038b3b379 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad15ffb640 1 -- 172.21.15.62:0/3857095988 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24019070 con 0x556d36fc5680 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad15ffb640 1 -- 172.21.15.62:0/3857095988 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad2400cd60 con 0x556d36fc5680 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad15ffb640 1 -- 172.21.15.62:0/3857095988 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24011990 con 0x556d36fc5680 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad38762740 1 -- 172.21.15.62:0/3857095988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 msgr2=0x556d36fdc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad38762740 1 --2- 172.21.15.62:0/3857095988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fdc9f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fad2400f5a0 tx=0x7fad2401d3c0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad38762740 1 -- 172.21.15.62:0/3857095988 shutdown_connections 2024-09-11T23:52:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad38762740 1 --2- 172.21.15.62:0/3857095988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fdc9f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad38762740 1 --2- 172.21.15.62:0/3857095988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fbadc0 0x556d36fd14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad38762740 1 --2- 172.21.15.62:0/3857095988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fba4a0 0x556d36fbdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.762+0000 7fad38762740 1 -- 172.21.15.62:0/3857095988 >> 172.21.15.62:0/3857095988 conn(0x556d36fb6a30 msgr2=0x556d36fc4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 1 -- 172.21.15.62:0/3857095988 shutdown_connections 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 1 -- 172.21.15.62:0/3857095988 wait complete. 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 1 -- start start 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 1 Processor -- start 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fba4a0 0x556d36fbf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fbadc0 0x556d36fbf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fbfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad301ae010 con 0x556d36fc5680 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad301af480 con 0x556d36fba4a0 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad301ae6e0 con 0x556d36fbadc0 2024-09-11T23:52:59.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad38762740 10 client.0 _ll_register_callbacks cb 0x7fad24013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fbfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fbfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41796/0 (socket says 172.21.15.62:41796) 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.763+0000 7fad2f7fe640 1 -- 172.21.15.62:0/1514120799 learned_addr learned my addr 172.21.15.62:0/1514120799 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fba4a0 0x556d36fbf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad2ffff640 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fbadc0 0x556d36fbf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad2f7fe640 1 -- 172.21.15.62:0/1514120799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fbadc0 msgr2=0x556d36fbf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fbadc0 0x556d36fbf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad2f7fe640 1 -- 172.21.15.62:0/1514120799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fba4a0 msgr2=0x556d36fbf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fba4a0 0x556d36fbf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad2f7fe640 1 -- 172.21.15.62:0/1514120799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d3703ea70 con 0x556d36fc5680 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad2f7fe640 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fbfe10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fad24020040 tx=0x7fad24015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95148fc41d853f1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:52:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1514120799 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24019070 con 0x556d36fc5680 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad38762740 1 -- 172.21.15.62:0/1514120799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d36f27d60 con 0x556d36fc5680 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1514120799 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fad2401e410 con 0x556d36fc5680 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1514120799 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad240119f0 con 0x556d36fc5680 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad38762740 10 client.15543 fetch_fsmap learned FSMap version 27 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad38762740 1 -- 172.21.15.62:0/1514120799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d3703fd20 con 0x556d36fc5680 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.764+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1514120799 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556d36f27d60 con 0x556d36fc5680 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.765+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1514120799 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fad24004680 con 0x556d36fc5680 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.765+0000 7fad38762740 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.765+0000 7fad38762740 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:52:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.765+0000 7fad38762740 20 client.15543 populate_metadata read hostname 'smithi062' 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.765+0000 7fad38762740 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d36fd5030 0x556d37042390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.765+0000 7fad38762740 4 client.15543 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.765+0000 7fad38762740 1 -- 172.21.15.62:0/1514120799 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d36ea4210 con 0x556d36fd5030 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.765+0000 7fad2ffff640 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d36fd5030 0x556d37042390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.765+0000 7fad2ffff640 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d36fd5030 0x556d37042390 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b0935d6c768919 server_cookie=5acda4e07ad1ee22 in_seq=0 out_seq=0 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.765+0000 7fad0f7fe640 10 client.15543 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad0f7fe640 1 -- 172.21.15.62:0/1514120799 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556d36ea4210 con 0x556d36fd5030 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad0f7fe640 10 client.15543 handle_command_reply: tid=0 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 client.15543 shutdown 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 -- 172.21.15.62:0/1514120799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 msgr2=0x556d36fbfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fbfe10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fad24020040 tx=0x7fad24015040 comp rx=0 tx=0).stop 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 -- 172.21.15.62:0/1514120799 shutdown_connections 2024-09-11T23:52:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d36fd5030 0x556d37042390 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d36fc5680 0x556d36fbfe10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d36fbadc0 0x556d36fbf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 --2- 172.21.15.62:0/1514120799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d36fba4a0 0x556d36fbf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 -- 172.21.15.62:0/1514120799 >> 172.21.15.62:0/1514120799 conn(0x556d36fb6a30 msgr2=0x556d36fb5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad0f7fe640 0 client.15543 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:52:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 -- 172.21.15.62:0/1514120799 shutdown_connections 2024-09-11T23:52:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 1 -- 172.21.15.62:0/1514120799 wait complete. 2024-09-11T23:52:59.817 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stdout: "e3b9d6ac-1e5b-4481-b4c8-8d160af9fa95": { 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "e3b9d6ac-1e5b-4481-b4c8-8d160af9fa95", 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.766+0000 7fad38762740 20 client.15543 trim_cache size 0 max 16384 2024-09-11T23:52:59.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 -- 172.21.15.62:0/3658816433 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad180a2d80 msgr2=0x7fad180a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad180a2d80 0x7fad180a5240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad280cd360 tx=0x7fad28016f60 comp rx=0 tx=0).stop 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 -- 172.21.15.62:0/3658816433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad30106a80 msgr2=0x7fad301030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad30106a80 0x7fad301030d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fad2800cce0 tx=0x7fad2800a620 comp rx=0 tx=0).stop 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 -- 172.21.15.62:0/3658816433 shutdown_connections 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad180a2d80 0x7fad180a5240 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad30106a80 0x7fad301030d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad30101420 0x7fad30102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 --2- 172.21.15.62:0/3658816433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad30100a80 0x7fad30102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 -- 172.21.15.62:0/3658816433 >> 172.21.15.62:0/3658816433 conn(0x7fad300fc7a0 msgr2=0x7fad3006a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 -- 172.21.15.62:0/3658816433 shutdown_connections 2024-09-11T23:52:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:52:59.767+0000 7fad36389640 1 -- 172.21.15.62:0/3658816433 wait complete. 2024-09-11T23:53:00.923 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:53:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:53:00.740+0000 7f7759933640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:53:00.923 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:53:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:53:00.740+0000 7f7759933640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:53:00.923 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:53:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:53:00.740+0000 7f7759933640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:53:00.923 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:53:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:53:00.740+0000 7f7759933640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:53:00.923 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 11 23:53:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-11T23:53:00.740+0000 7f7759933640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:53:00.953 INFO:tasks.cephfs.filesystem:scrub status for tag:e3b9d6ac-1e5b-4481-b4c8-8d160af9fa95 - {'path': '/', 'tag': 'e3b9d6ac-1e5b-4481-b4c8-8d160af9fa95', 'options': 'recursive,force'} 2024-09-11T23:53:01.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:01 smithi062 ceph-mon[30958]: pgmap v30: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.7 MiB/s wr, 591 op/s 2024-09-11T23:53:01.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:01 smithi120 ceph-mon[35182]: pgmap v30: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.7 MiB/s wr, 591 op/s 2024-09-11T23:53:01.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:01 smithi083 ceph-mon[34805]: pgmap v30: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.7 MiB/s wr, 591 op/s 2024-09-11T23:53:02.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 20.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:53:03.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:03 smithi062 ceph-mon[30958]: pgmap v31: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.7 MiB/s wr, 591 op/s 2024-09-11T23:53:03.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:03 smithi062 ceph-mon[30958]: mgrmap e24: x(active, since 56s), standbys: z, y 2024-09-11T23:53:03.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:03 smithi120 ceph-mon[35182]: pgmap v31: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.7 MiB/s wr, 591 op/s 2024-09-11T23:53:03.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:03 smithi120 ceph-mon[35182]: mgrmap e24: x(active, since 56s), standbys: z, y 2024-09-11T23:53:03.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:03 smithi083 ceph-mon[34805]: pgmap v31: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.7 MiB/s wr, 591 op/s 2024-09-11T23:53:03.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:03 smithi083 ceph-mon[34805]: mgrmap e24: x(active, since 56s), standbys: z, y 2024-09-11T23:53:05.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:05 smithi062 ceph-mon[30958]: pgmap v32: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 6.7 MiB/s wr, 610 op/s 2024-09-11T23:53:05.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:05 smithi120 ceph-mon[35182]: pgmap v32: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 6.7 MiB/s wr, 610 op/s 2024-09-11T23:53:05.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:05 smithi083 ceph-mon[34805]: pgmap v32: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 6.7 MiB/s wr, 610 op/s 2024-09-11T23:53:06.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:06 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:53:06.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:06 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:53:06.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:06 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:53:06.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:06 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:53:06.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:06 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:53:06.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:06 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:53:07.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 25.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:53:07.531 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:07 smithi062 ceph-mon[30958]: pgmap v33: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 2.2 MiB/s wr, 297 op/s 2024-09-11T23:53:07.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:07 smithi120 ceph-mon[35182]: pgmap v33: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 2.2 MiB/s wr, 297 op/s 2024-09-11T23:53:07.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:07 smithi083 ceph-mon[34805]: pgmap v33: 97 pgs: 97 active+clean; 931 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 2.2 MiB/s wr, 297 op/s 2024-09-11T23:53:09.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:09 smithi120 ceph-mon[35182]: pgmap v34: 97 pgs: 97 active+clean; 958 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 6.6 MiB/s wr, 383 op/s 2024-09-11T23:53:09.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:09 smithi083 ceph-mon[34805]: pgmap v34: 97 pgs: 97 active+clean; 958 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 6.6 MiB/s wr, 383 op/s 2024-09-11T23:53:09.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:09 smithi062 ceph-mon[30958]: pgmap v34: 97 pgs: 97 active+clean; 958 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 6.6 MiB/s wr, 383 op/s 2024-09-11T23:53:11.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:11 smithi120 ceph-mon[35182]: pgmap v35: 97 pgs: 97 active+clean; 958 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 104 op/s 2024-09-11T23:53:11.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:11 smithi083 ceph-mon[34805]: pgmap v35: 97 pgs: 97 active+clean; 958 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 104 op/s 2024-09-11T23:53:11.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:11 smithi062 ceph-mon[30958]: pgmap v35: 97 pgs: 97 active+clean; 958 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 104 op/s 2024-09-11T23:53:12.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 30.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:53:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:13 smithi120 ceph-mon[35182]: pgmap v36: 97 pgs: 97 active+clean; 958 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 104 op/s 2024-09-11T23:53:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:13 smithi083 ceph-mon[34805]: pgmap v36: 97 pgs: 97 active+clean; 958 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 104 op/s 2024-09-11T23:53:13.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:53:13.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:13 smithi062 ceph-mon[30958]: pgmap v36: 97 pgs: 97 active+clean; 958 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 104 op/s 2024-09-11T23:53:13.972 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:53:14.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:14 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:53:14.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.461+0000 7fdf7458c640 1 -- 172.21.15.62:0/1607846365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf6c103740 msgr2=0x7fdf6c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:14.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.461+0000 7fdf7458c640 1 --2- 172.21.15.62:0/1607846365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf6c103740 0x7fdf6c107b80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009930 tx=0x7fdf5c02a420 comp rx=0 tx=0).stop 2024-09-11T23:53:14.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.461+0000 7fdf7458c640 1 -- 172.21.15.62:0/1607846365 shutdown_connections 2024-09-11T23:53:14.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.461+0000 7fdf7458c640 1 --2- 172.21.15.62:0/1607846365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c108290 0x7fdf6c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.461+0000 7fdf7458c640 1 --2- 172.21.15.62:0/1607846365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf6c103740 0x7fdf6c107b80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.461+0000 7fdf7458c640 1 --2- 172.21.15.62:0/1607846365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6c102d90 0x7fdf6c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.461+0000 7fdf7458c640 1 -- 172.21.15.62:0/1607846365 >> 172.21.15.62:0/1607846365 conn(0x7fdf6c0fc7d0 msgr2=0x7fdf6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.461+0000 7fdf7458c640 1 -- 172.21.15.62:0/1607846365 shutdown_connections 2024-09-11T23:53:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.461+0000 7fdf7458c640 1 -- 172.21.15.62:0/1607846365 wait complete. 2024-09-11T23:53:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf7458c640 1 Processor -- start 2024-09-11T23:53:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf7458c640 1 -- start start 2024-09-11T23:53:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf7458c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102d90 0x7fdf6c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf7458c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf6c103740 0x7fdf6c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf7458c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6c108290 0x7fdf6c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf7458c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c112090 con 0x7fdf6c103740 2024-09-11T23:53:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf7458c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c111f10 con 0x7fdf6c108290 2024-09-11T23:53:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf7458c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c112210 con 0x7fdf6c102d90 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf72328640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102d90 0x7fdf6c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf72b29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6c108290 0x7fdf6c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf72328640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102d90 0x7fdf6c1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:49836/0 (socket says 172.21.15.62:49836) 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.462+0000 7fdf72328640 1 -- 172.21.15.62:0/156207379 learned_addr learned my addr 172.21.15.62:0/156207379 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf71b27640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf6c103740 0x7fdf6c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf72b29640 1 -- 172.21.15.62:0/156207379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102d90 msgr2=0x7fdf6c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf72b29640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102d90 0x7fdf6c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf72b29640 1 -- 172.21.15.62:0/156207379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf6c103740 msgr2=0x7fdf6c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf72b29640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf6c103740 0x7fdf6c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf72b29640 1 -- 172.21.15.62:0/156207379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c19d580 con 0x7fdf6c108290 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf72b29640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6c108290 0x7fdf6c19cea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400e940 tx=0x7fdf6400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf637fe640 1 -- 172.21.15.62:0/156207379 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64004d20 con 0x7fdf6c108290 2024-09-11T23:53:14.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf637fe640 1 -- 172.21.15.62:0/156207379 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf64005020 con 0x7fdf6c108290 2024-09-11T23:53:14.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf637fe640 1 -- 172.21.15.62:0/156207379 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf640116b0 con 0x7fdf6c108290 2024-09-11T23:53:14.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf7458c640 1 -- 172.21.15.62:0/156207379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c19d870 con 0x7fdf6c108290 2024-09-11T23:53:14.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.463+0000 7fdf7458c640 1 -- 172.21.15.62:0/156207379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c1a5150 con 0x7fdf6c108290 2024-09-11T23:53:14.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.464+0000 7fdf7458c640 1 -- 172.21.15.62:0/156207379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6c103170 con 0x7fdf6c108290 2024-09-11T23:53:14.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.466+0000 7fdf637fe640 1 -- 172.21.15.62:0/156207379 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fdf640054d0 con 0x7fdf6c108290 2024-09-11T23:53:14.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.467+0000 7fdf637fe640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf580a2e90 0x7fdf580a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:14.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.467+0000 7fdf72328640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf580a2e90 0x7fdf580a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:14.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.467+0000 7fdf637fe640 1 -- 172.21.15.62:0/156207379 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fdf640cfa60 con 0x7fdf6c108290 2024-09-11T23:53:14.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.467+0000 7fdf72328640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf580a2e90 0x7fdf580a5350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009c90 tx=0x7fdf68005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:14.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.469+0000 7fdf637fe640 1 -- 172.21.15.62:0/156207379 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf64016030 con 0x7fdf6c108290 2024-09-11T23:53:14.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:14 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:53:14.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:14 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:53:14.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.688+0000 7fdf7458c640 1 -- 172.21.15.62:0/156207379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf6c199930 con 0x7fdf6c108290 2024-09-11T23:53:14.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.689+0000 7fdf637fe640 1 -- 172.21.15.62:0/156207379 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11912 (secure 0 0 0) 0x7fdf6c199930 con 0x7fdf6c108290 2024-09-11T23:53:14.689 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:53:14.689 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-09-11T23:52:56:186189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:55.695276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:53:14.690 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 -- 172.21.15.62:0/156207379 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf580a2e90 msgr2=0x7fdf580a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf580a2e90 0x7fdf580a5350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009c90 tx=0x7fdf68005f90 comp rx=0 tx=0).stop 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 -- 172.21.15.62:0/156207379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6c108290 msgr2=0x7fdf6c19cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6c108290 0x7fdf6c19cea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400e940 tx=0x7fdf6400ee00 comp rx=0 tx=0).stop 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 -- 172.21.15.62:0/156207379 shutdown_connections 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf580a2e90 0x7fdf580a5350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6c108290 0x7fdf6c19cea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf6c103740 0x7fdf6c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 --2- 172.21.15.62:0/156207379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102d90 0x7fdf6c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 -- 172.21.15.62:0/156207379 >> 172.21.15.62:0/156207379 conn(0x7fdf6c0fc7d0 msgr2=0x7fdf6c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:14.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 -- 172.21.15.62:0/156207379 shutdown_connections 2024-09-11T23:53:14.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:14.693+0000 7fdf7458c640 1 -- 172.21.15.62:0/156207379 wait complete. 2024-09-11T23:53:15.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:15 smithi062 ceph-mon[30958]: pgmap v37: 97 pgs: 97 active+clean; 934 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 118 op/s 2024-09-11T23:53:15.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:15 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:53:15.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:15 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:53:15.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:15 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/156207379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:53:15.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:15 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:53:15.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:15 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:53:15.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:15 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:53:15.495 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 85ba7803 --release --await' 2024-09-11T23:53:15.495 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 85ba7803 --release --await 2024-09-11T23:53:15.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:15 smithi120 ceph-mon[35182]: pgmap v37: 97 pgs: 97 active+clean; 934 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 118 op/s 2024-09-11T23:53:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:15 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:53:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:15 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:53:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:15 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/156207379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:53:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:15 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:53:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:15 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:53:15.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:15 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:53:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:15 smithi083 ceph-mon[34805]: pgmap v37: 97 pgs: 97 active+clean; 934 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 118 op/s 2024-09-11T23:53:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:15 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:53:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:15 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:53:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:15 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/156207379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:53:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:15 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:53:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:15 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:53:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:15 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:53:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:53:16.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.251+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2221269565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d78112b90 msgr2=0x7f8d78114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:16.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.251+0000 7f8d7fc32640 1 --2- 172.21.15.62:0/2221269565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d78112b90 0x7f8d78114f80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009ca0 tx=0x7f8d7002a950 comp rx=0 tx=0).stop 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.251+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2221269565 shutdown_connections 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.251+0000 7f8d7fc32640 1 --2- 172.21.15.62:0/2221269565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d78112b90 0x7f8d78114f80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.251+0000 7f8d7fc32640 1 --2- 172.21.15.62:0/2221269565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d78074eb0 0x7f8d78112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.251+0000 7f8d7fc32640 1 --2- 172.21.15.62:0/2221269565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d78074590 0x7f8d78074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.251+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2221269565 >> 172.21.15.62:0/2221269565 conn(0x7f8d780fd8c0 msgr2=0x7f8d780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.251+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2221269565 shutdown_connections 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.251+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2221269565 wait complete. 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7fc32640 1 Processor -- start 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7fc32640 1 -- start start 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7fc32640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d78074590 0x7f8d781a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7fc32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d78074eb0 0x7f8d781a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7fc32640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d78112b90 0x7f8d7819ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7fc32640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78118680 con 0x7f8d78112b90 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7fc32640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78118500 con 0x7f8d78074eb0 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7fc32640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78118800 con 0x7f8d78074590 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7e1cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d78112b90 0x7f8d7819ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7e1cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d78112b90 0x7f8d7819ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57552/0 (socket says 172.21.15.62:57552) 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7e1cf640 1 -- 172.21.15.62:0/2312905039 learned_addr learned my addr 172.21.15.62:0/2312905039 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.252+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d78074eb0 0x7f8d781a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d78074590 0x7f8d781a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d7e1cf640 1 -- 172.21.15.62:0/2312905039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d78074590 msgr2=0x7f8d781a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d7e1cf640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d78074590 0x7f8d781a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d7e1cf640 1 -- 172.21.15.62:0/2312905039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d78074eb0 msgr2=0x7f8d781a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d7e1cf640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d78074eb0 0x7f8d781a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d7e1cf640 1 -- 172.21.15.62:0/2312905039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7819f140 con 0x7f8d78112b90 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d7e1cf640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d78112b90 0x7f8d7819ea10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d7002af40 tx=0x7f8d70003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d6effd640 1 -- 172.21.15.62:0/2312905039 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70034040 con 0x7f8d78112b90 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d6effd640 1 -- 172.21.15.62:0/2312905039 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7002fe20 con 0x7f8d78112b90 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2312905039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7819f3d0 con 0x7f8d78112b90 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d6effd640 1 -- 172.21.15.62:0/2312905039 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70004750 con 0x7f8d78112b90 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.253+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2312905039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d781ab720 con 0x7f8d78112b90 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.255+0000 7f8d6effd640 1 -- 172.21.15.62:0/2312905039 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f8d70037020 con 0x7f8d78112b90 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d8200b740 1 Processor -- start 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d8200b740 1 -- start start 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886060 0x558c23886440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 0x558c23890d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23891240 0x558c23893630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d8200b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558c237eb620 con 0x558c23891240 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d8200b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558c23792e50 con 0x558c23886980 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d8200b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558c237e75e0 con 0x558c23886060 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d7d1cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23891240 0x558c23893630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d6effd640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8d640a2e90 0x7f8d640a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d7d1cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23891240 0x558c23893630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57558/0 (socket says 172.21.15.62:57558) 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/1598834648 learned_addr learned my addr 172.21.15.62:0/1598834648 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/1598834648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 0x558c23890d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/1598834648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886060 0x558c23886440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.256+0000 7f8d7e1cf640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8d640a2e90 0x7f8d640a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/1598834648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886060 msgr2=0x558c23886440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/1598834648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886060 0x558c23886440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/1598834648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 msgr2=0x558c23890d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/1598834648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 0x558c23890d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d6effd640 1 -- 172.21.15.62:0/2312905039 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f8d700f0660 con 0x7f8d78112b90 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/1598834648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c23783d40 con 0x558c23891240 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/1598834648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23891240 0x558c23893630 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009f20 tx=0x7f8d68009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5307a6a0eaa5746 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d7e1cf640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8d640a2e90 0x7f8d640a5350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d700eddd0 tx=0x7f8d700a7040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d63fff640 1 -- 172.21.15.62:0/1598834648 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68017470 con 0x558c23891240 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d63fff640 1 -- 172.21.15.62:0/1598834648 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6801c070 con 0x558c23891240 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.257+0000 7f8d63fff640 1 -- 172.21.15.62:0/1598834648 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68015860 con 0x558c23891240 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 -- 172.21.15.62:0/1598834648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23891240 msgr2=0x558c23893630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 --2- 172.21.15.62:0/1598834648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23891240 0x558c23893630 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009f20 tx=0x7f8d68009580 comp rx=0 tx=0).stop 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 -- 172.21.15.62:0/1598834648 shutdown_connections 2024-09-11T23:53:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 --2- 172.21.15.62:0/1598834648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23891240 0x558c23893630 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 --2- 172.21.15.62:0/1598834648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 0x558c23890d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 --2- 172.21.15.62:0/1598834648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886060 0x558c23886440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 -- 172.21.15.62:0/1598834648 >> 172.21.15.62:0/1598834648 conn(0x558c238825f0 msgr2=0x558c238829d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 -- 172.21.15.62:0/1598834648 shutdown_connections 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 -- 172.21.15.62:0/1598834648 wait complete. 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 -- start start 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.258+0000 7f8d8200b740 1 Processor -- start 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c2388e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 0x558c2388e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23891240 0x558c2388eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d8200b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558c23893e10 con 0x558c23886060 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d8200b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558c23773f10 con 0x558c23886980 2024-09-11T23:53:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d8200b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558c23893f90 con 0x558c23891240 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d8200b740 10 client.0 _ll_register_callbacks cb 0x558c23797db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d1cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c2388e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d1cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c2388e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57562/0 (socket says 172.21.15.62:57562) 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/2301687085 learned_addr learned my addr 172.21.15.62:0/2301687085 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 0x558c2388e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/2301687085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23891240 msgr2=0x558c2388eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23891240 0x558c2388eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/2301687085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 msgr2=0x558c2388e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 0x558c2388e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/2301687085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c234fa580 con 0x558c23886060 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 0x558c2388e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:53:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c2388e110 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d68010b70 tx=0x7f8d68006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e0c55585f4703d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.259+0000 7f8d617fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d617fa640 1 -- 172.21.15.62:0/2301687085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68017b70 con 0x558c23886060 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d617fa640 1 -- 172.21.15.62:0/2301687085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f8d680155e0 con 0x558c23886060 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d8200b740 1 -- 172.21.15.62:0/2301687085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d6801c070 con 0x558c23886060 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d617fa640 1 -- 172.21.15.62:0/2301687085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d68031050 con 0x558c23886060 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d8200b740 10 client.15567 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d8200b740 1 -- 172.21.15.62:0/2301687085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c235c4960 con 0x558c23886060 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d617fa640 1 -- 172.21.15.62:0/2301687085 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6801c070 con 0x558c23886060 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d617fa640 1 -- 172.21.15.62:0/2301687085 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d68020970 con 0x558c23886060 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d8200b740 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d8200b740 10 client.15567 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d8200b740 20 client.15567 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d8200b740 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558c238a0cb0 0x558c238a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d8200b740 4 client.15567 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d8200b740 1 -- 172.21.15.62:0/2301687085 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c237f8b90 con 0x558c238a0cb0 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.260+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558c238a0cb0 0x558c238a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.261+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558c238a0cb0 0x558c238a30a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d08c311a05c61b4 server_cookie=ee2b2278eb1fa86a in_seq=0 out_seq=0 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.261+0000 7f8d617fa640 10 client.15567 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d617fa640 1 -- 172.21.15.62:0/2301687085 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c237f8b90 con 0x558c238a0cb0 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d617fa640 10 client.15567 handle_command_reply: tid=0 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 client.15567 shutdown 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 -- 172.21.15.62:0/2301687085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 msgr2=0x558c2388e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c2388e110 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d68010b70 tx=0x7f8d68006a30 comp rx=0 tx=0).stop 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 -- 172.21.15.62:0/2301687085 shutdown_connections 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558c238a0cb0 0x558c238a30a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23891240 0x558c2388eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23886980 0x558c2388e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d617fa640 0 client.15567 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 --2- 172.21.15.62:0/2301687085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c2388e110 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 -- 172.21.15.62:0/2301687085 >> 172.21.15.62:0/2301687085 conn(0x558c238825f0 msgr2=0x558c238882d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 -- 172.21.15.62:0/2301687085 shutdown_connections 2024-09-11T23:53:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 1 -- 172.21.15.62:0/2301687085 wait complete. 2024-09-11T23:53:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.262+0000 7f8d8200b740 20 client.15567 trim_cache size 0 max 16384 2024-09-11T23:53:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.279+0000 7f8d8200b740 1 Processor -- start 2024-09-11T23:53:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.279+0000 7f8d8200b740 1 -- start start 2024-09-11T23:53:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.279+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c238881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.279+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 0x558c238a8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.279+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23891240 0x558c238a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.279+0000 7f8d8200b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b6830 con 0x558c23886060 2024-09-11T23:53:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.279+0000 7f8d8200b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b4100 con 0x558c23891240 2024-09-11T23:53:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.279+0000 7f8d8200b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b1f50 con 0x558c23886980 2024-09-11T23:53:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d1cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c238881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d1cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c238881c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57578/0 (socket says 172.21.15.62:57578) 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/2023705243 learned_addr learned my addr 172.21.15.62:0/2023705243 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/2023705243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 msgr2=0x558c238a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/2023705243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 0x558c238a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/2023705243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 0x558c238a8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/2023705243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23891240 msgr2=0x558c238a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/2023705243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23891240 0x558c238a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/2023705243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c23883280 con 0x558c23886060 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/2023705243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 0x558c238a8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/2023705243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c238881c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8d68017840 tx=0x7f8d68022670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dfc272c932db8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d637fe640 1 -- 172.21.15.62:0/2023705243 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68017b70 con 0x558c23886060 2024-09-11T23:53:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d637fe640 1 -- 172.21.15.62:0/2023705243 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6801c040 con 0x558c23886060 2024-09-11T23:53:16.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.280+0000 7f8d637fe640 1 -- 172.21.15.62:0/2023705243 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68020680 con 0x558c23886060 2024-09-11T23:53:16.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.281+0000 7f8d8200b740 1 -- 172.21.15.62:0/2023705243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 msgr2=0x558c238881c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.281+0000 7f8d8200b740 1 --2- 172.21.15.62:0/2023705243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c238881c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8d68017840 tx=0x7f8d68022670 comp rx=0 tx=0).stop 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.281+0000 7f8d8200b740 1 -- 172.21.15.62:0/2023705243 shutdown_connections 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.281+0000 7f8d8200b740 1 --2- 172.21.15.62:0/2023705243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23891240 0x558c238a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.281+0000 7f8d8200b740 1 --2- 172.21.15.62:0/2023705243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 0x558c238a8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.281+0000 7f8d8200b740 1 --2- 172.21.15.62:0/2023705243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c238881c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.281+0000 7f8d8200b740 1 -- 172.21.15.62:0/2023705243 >> 172.21.15.62:0/2023705243 conn(0x558c238825f0 msgr2=0x558c23887ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.281+0000 7f8d8200b740 1 -- 172.21.15.62:0/2023705243 shutdown_connections 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.281+0000 7f8d8200b740 1 -- 172.21.15.62:0/2023705243 wait complete. 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.281+0000 7f8d8200b740 1 -- start start 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d8200b740 1 Processor -- start 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c23895770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 0x558c23895cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d8200b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23891240 0x558c2390a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d8200b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b25c0 con 0x558c23886060 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d8200b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b3a30 con 0x558c23891240 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d8200b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b2c90 con 0x558c23886980 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d8200b740 10 client.0 _ll_register_callbacks cb 0x7f8d68017870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d7d1cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c23895770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d7d1cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c23895770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57586/0 (socket says 172.21.15.62:57586) 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/1880241677 learned_addr learned my addr 172.21.15.62:0/1880241677 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 0x558c23895cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23891240 0x558c2390a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/1880241677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 msgr2=0x558c23895cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 0x558c23895cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/1880241677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23891240 msgr2=0x558c2390a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23891240 0x558c2390a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.282+0000 7f8d7d1cd640 1 -- 172.21.15.62:0/1880241677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c2390ada0 con 0x558c23886060 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d7d1cd640 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c23895770 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009da0 tx=0x7f8d680226e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb9ab717a6acbd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d617fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d617fa640 1 -- 172.21.15.62:0/1880241677 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d680095e0 con 0x558c23886060 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d8200b740 1 -- 172.21.15.62:0/1880241677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d6801c040 con 0x558c23886060 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d617fa640 1 -- 172.21.15.62:0/1880241677 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f8d68005bd0 con 0x558c23886060 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d617fa640 1 -- 172.21.15.62:0/1880241677 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d68005dd0 con 0x558c23886060 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d8200b740 10 client.15573 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d8200b740 1 -- 172.21.15.62:0/1880241677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c2390c010 con 0x558c23886060 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d617fa640 1 -- 172.21.15.62:0/1880241677 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6801c040 con 0x558c23886060 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d617fa640 1 -- 172.21.15.62:0/1880241677 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d680207c0 con 0x558c23886060 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d8200b740 10 client.15573 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d8200b740 10 client.15573 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:53:16.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d8200b740 20 client.15573 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:16.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.283+0000 7f8d8200b740 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558c238a0cb0 0x558c2390e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:16.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.284+0000 7f8d8200b740 4 client.15573 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "85ba7803", "release": true, "await": true}] 2024-09-11T23:53:16.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.284+0000 7f8d8200b740 1 -- 172.21.15.62:0/1880241677 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "85ba7803", "release": true, "await": true}) -- 0x558c237f30a0 con 0x558c238a0cb0 2024-09-11T23:53:16.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.284+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558c238a0cb0 0x558c2390e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:16.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.284+0000 7f8d617fa640 10 client.15573 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:16.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.284+0000 7f8d7d9ce640 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558c238a0cb0 0x558c2390e630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1a0a00a6a5613de server_cookie=6a20d8607c39a280 in_seq=0 out_seq=0 2024-09-11T23:53:16.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d617fa640 1 -- 172.21.15.62:0/1880241677 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558c237f30a0 con 0x558c238a0cb0 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d617fa640 10 client.15573 handle_command_reply: tid=0 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 client.15573 shutdown 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 -- 172.21.15.62:0/1880241677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 msgr2=0x558c23895770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c23895770 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009da0 tx=0x7f8d680226e0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 -- 172.21.15.62:0/1880241677 shutdown_connections 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558c238a0cb0 0x558c2390e630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c23891240 0x558c2390a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c23886980 0x558c23895cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d617fa640 0 client.15573 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 --2- 172.21.15.62:0/1880241677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c23886060 0x558c23895770 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 -- 172.21.15.62:0/1880241677 >> 172.21.15.62:0/1880241677 conn(0x558c238825f0 msgr2=0x558c23881750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:16.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 -- 172.21.15.62:0/1880241677 shutdown_connections 2024-09-11T23:53:16.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.327+0000 7f8d8200b740 1 -- 172.21.15.62:0/1880241677 wait complete. 2024-09-11T23:53:16.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.329+0000 7f8d8200b740 20 client.15573 trim_cache size 0 max 16384 2024-09-11T23:53:16.330 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:53:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 27, 2024-09-11T23:53:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:53:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 4, 2024-09-11T23:53:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:53:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "85ba7803": { 2024-09-11T23:53:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 4, 2024-09-11T23:53:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:53:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:53:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:53:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:53:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:53:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:53:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 144.5, 2024-09-11T23:53:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:53:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:53:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:53:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:53:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:53:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:53:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:53:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:53:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:53:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:53:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:53:16.332 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:53:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2312905039 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8d640a2e90 msgr2=0x7f8d640a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8d640a2e90 0x7f8d640a5350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d700eddd0 tx=0x7f8d700a7040 comp rx=0 tx=0).stop 2024-09-11T23:53:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2312905039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d78112b90 msgr2=0x7f8d7819ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d78112b90 0x7f8d7819ea10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d7002af40 tx=0x7f8d70003f10 comp rx=0 tx=0).stop 2024-09-11T23:53:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2312905039 shutdown_connections 2024-09-11T23:53:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8d640a2e90 0x7f8d640a5350 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d78112b90 0x7f8d7819ea10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d78074eb0 0x7f8d781a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 --2- 172.21.15.62:0/2312905039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d78074590 0x7f8d781a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2312905039 >> 172.21.15.62:0/2312905039 conn(0x7f8d780fd8c0 msgr2=0x7f8d780ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2312905039 shutdown_connections 2024-09-11T23:53:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:16.332+0000 7f8d7fc32640 1 -- 172.21.15.62:0/2312905039 wait complete. 2024-09-11T23:53:17.059 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '85ba7803', total seconds elapsed: 27.9 2024-09-11T23:53:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 35.0 s, 208.0 tps, lat 2206.205 ms stddev 6405.190 2024-09-11T23:53:17.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:17 smithi062 ceph-mon[30958]: pgmap v38: 97 pgs: 97 active+clean; 934 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 MiB/s wr, 99 op/s 2024-09-11T23:53:17.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:17 smithi120 ceph-mon[35182]: pgmap v38: 97 pgs: 97 active+clean; 934 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 MiB/s wr, 99 op/s 2024-09-11T23:53:17.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:17 smithi083 ceph-mon[34805]: pgmap v38: 97 pgs: 97 active+clean; 934 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 MiB/s wr, 99 op/s 2024-09-11T23:53:19.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:19 smithi120 ceph-mon[35182]: pgmap v39: 97 pgs: 97 active+clean; 935 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 MiB/s wr, 183 op/s 2024-09-11T23:53:19.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:19 smithi083 ceph-mon[34805]: pgmap v39: 97 pgs: 97 active+clean; 935 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 MiB/s wr, 183 op/s 2024-09-11T23:53:19.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:19 smithi062 ceph-mon[30958]: pgmap v39: 97 pgs: 97 active+clean; 935 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 MiB/s wr, 183 op/s 2024-09-11T23:53:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:21 smithi120 ceph-mon[35182]: pgmap v40: 97 pgs: 97 active+clean; 935 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 MiB/s wr, 97 op/s 2024-09-11T23:53:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:21 smithi083 ceph-mon[34805]: pgmap v40: 97 pgs: 97 active+clean; 935 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 MiB/s wr, 97 op/s 2024-09-11T23:53:21.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:21 smithi062 ceph-mon[30958]: pgmap v40: 97 pgs: 97 active+clean; 935 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 MiB/s wr, 97 op/s 2024-09-11T23:53:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 40.0 s, 3620.0 tps, lat 27.634 ms stddev 30.942 2024-09-11T23:53:23.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:23 smithi120 ceph-mon[35182]: pgmap v41: 97 pgs: 97 active+clean; 944 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 MiB/s wr, 207 op/s 2024-09-11T23:53:23.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:23 smithi083 ceph-mon[34805]: pgmap v41: 97 pgs: 97 active+clean; 944 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 MiB/s wr, 207 op/s 2024-09-11T23:53:23.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:23 smithi062 ceph-mon[30958]: pgmap v41: 97 pgs: 97 active+clean; 944 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 MiB/s wr, 207 op/s 2024-09-11T23:53:25.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:25 smithi120 ceph-mon[35182]: pgmap v42: 97 pgs: 97 active+clean; 972 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 39 MiB/s wr, 687 op/s 2024-09-11T23:53:25.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:25 smithi083 ceph-mon[34805]: pgmap v42: 97 pgs: 97 active+clean; 972 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 39 MiB/s wr, 687 op/s 2024-09-11T23:53:25.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:25 smithi062 ceph-mon[30958]: pgmap v42: 97 pgs: 97 active+clean; 972 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 39 MiB/s wr, 687 op/s 2024-09-11T23:53:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 45.0 s, 3887.3 tps, lat 25.656 ms stddev 25.918 2024-09-11T23:53:27.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:27 smithi062 ceph-mon[30958]: pgmap v43: 97 pgs: 97 active+clean; 972 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 35 MiB/s wr, 673 op/s 2024-09-11T23:53:27.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:27 smithi120 ceph-mon[35182]: pgmap v43: 97 pgs: 97 active+clean; 972 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 35 MiB/s wr, 673 op/s 2024-09-11T23:53:27.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:27 smithi083 ceph-mon[34805]: pgmap v43: 97 pgs: 97 active+clean; 972 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 35 MiB/s wr, 673 op/s 2024-09-11T23:53:29.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:29 smithi120 ceph-mon[35182]: pgmap v44: 97 pgs: 97 active+clean; 1009 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 39 MiB/s wr, 1.00k op/s 2024-09-11T23:53:29.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:29 smithi083 ceph-mon[34805]: pgmap v44: 97 pgs: 97 active+clean; 1009 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 39 MiB/s wr, 1.00k op/s 2024-09-11T23:53:29.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:29 smithi062 ceph-mon[30958]: pgmap v44: 97 pgs: 97 active+clean; 1009 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 39 MiB/s wr, 1.00k op/s 2024-09-11T23:53:30.954 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:53:31.336 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:53:31.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:31 smithi062 ceph-mon[30958]: pgmap v45: 97 pgs: 97 active+clean; 1009 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 38 MiB/s wr, 921 op/s 2024-09-11T23:53:31.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:31 smithi120 ceph-mon[35182]: pgmap v45: 97 pgs: 97 active+clean; 1009 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 38 MiB/s wr, 921 op/s 2024-09-11T23:53:31.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:31 smithi083 ceph-mon[34805]: pgmap v45: 97 pgs: 97 active+clean; 1009 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 38 MiB/s wr, 921 op/s 2024-09-11T23:53:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.844+0000 7fb849b99640 1 -- 172.21.15.62:0/4052324179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb844105e00 msgr2=0x7fb84410d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.844+0000 7fb849b99640 1 --2- 172.21.15.62:0/4052324179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb844105e00 0x7fb84410d9a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009cd0 tx=0x7fb83c02a8f0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.844+0000 7fb849b99640 1 -- 172.21.15.62:0/4052324179 shutdown_connections 2024-09-11T23:53:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.844+0000 7fb849b99640 1 --2- 172.21.15.62:0/4052324179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb844105e00 0x7fb84410d9a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.844+0000 7fb849b99640 1 --2- 172.21.15.62:0/4052324179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8441021b0 0x7fb8441056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.844+0000 7fb849b99640 1 --2- 172.21.15.62:0/4052324179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb844101800 0x7fb844101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.844+0000 7fb849b99640 1 -- 172.21.15.62:0/4052324179 >> 172.21.15.62:0/4052324179 conn(0x7fb8440fba90 msgr2=0x7fb8440fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.845+0000 7fb849b99640 1 -- 172.21.15.62:0/4052324179 shutdown_connections 2024-09-11T23:53:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.845+0000 7fb849b99640 1 -- 172.21.15.62:0/4052324179 wait complete. 2024-09-11T23:53:31.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.846+0000 7fb849b99640 1 Processor -- start 2024-09-11T23:53:31.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.846+0000 7fb849b99640 1 -- start start 2024-09-11T23:53:31.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.847+0000 7fb849b99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb844101800 0x7fb8441a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.847+0000 7fb849b99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8441021b0 0x7fb8441a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.847+0000 7fb849b99640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb844105e00 0x7fb84419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.847+0000 7fb849b99640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb84410fd50 con 0x7fb844105e00 2024-09-11T23:53:31.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.847+0000 7fb842ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8441021b0 0x7fb8441a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.847+0000 7fb842ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8441021b0 0x7fb8441a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52192/0 (socket says 172.21.15.62:52192) 2024-09-11T23:53:31.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.847+0000 7fb842ffd640 1 -- 172.21.15.62:0/4193730975 learned_addr learned my addr 172.21.15.62:0/4193730975 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:31.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.847+0000 7fb849b99640 1 -- 172.21.15.62:0/4193730975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb84410fbd0 con 0x7fb844101800 2024-09-11T23:53:31.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.847+0000 7fb849b99640 1 -- 172.21.15.62:0/4193730975 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb84410fed0 con 0x7fb8441021b0 2024-09-11T23:53:31.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.848+0000 7fb843fff640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb844105e00 0x7fb84419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.848+0000 7fb842ffd640 1 -- 172.21.15.62:0/4193730975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb844101800 msgr2=0x7fb8441a46f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:53:31.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.848+0000 7fb842ffd640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb844101800 0x7fb8441a46f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.848+0000 7fb842ffd640 1 -- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb844105e00 msgr2=0x7fb84419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:31.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.849+0000 7fb842ffd640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb844105e00 0x7fb84419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.849+0000 7fb842ffd640 1 -- 172.21.15.62:0/4193730975 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84419efe0 con 0x7fb8441021b0 2024-09-11T23:53:31.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.849+0000 7fb843fff640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb844105e00 0x7fb84419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:31.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.849+0000 7fb842ffd640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8441021b0 0x7fb8441a4c30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb83800e9c0 tx=0x7fb83800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:31.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.851+0000 7fb840ff9640 1 -- 172.21.15.62:0/4193730975 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83800cd80 con 0x7fb8441021b0 2024-09-11T23:53:31.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.851+0000 7fb840ff9640 1 -- 172.21.15.62:0/4193730975 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb838005500 con 0x7fb8441021b0 2024-09-11T23:53:31.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.852+0000 7fb840ff9640 1 -- 172.21.15.62:0/4193730975 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838010690 con 0x7fb8441021b0 2024-09-11T23:53:31.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.852+0000 7fb849b99640 1 -- 172.21.15.62:0/4193730975 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84419f2d0 con 0x7fb8441021b0 2024-09-11T23:53:31.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.852+0000 7fb849b99640 1 -- 172.21.15.62:0/4193730975 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8441ab5c0 con 0x7fb8441021b0 2024-09-11T23:53:31.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.859+0000 7fb84bf72740 1 Processor -- start 2024-09-11T23:53:31.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb84bf72740 1 -- start start 2024-09-11T23:53:31.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e4e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e4ec80 0x555a87e6a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e6a690 0x555a87e6ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb84bf72740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555a87db42e0 con 0x555a87e4ec80 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb84bf72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a87d5bce0 con 0x555a87e4e360 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb84bf72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a87db02a0 con 0x555a87e6a690 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb843fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e4e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb843fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e4e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:47756/0 (socket says 172.21.15.62:47756) 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb843fff640 1 -- 172.21.15.62:0/1968467113 learned_addr learned my addr 172.21.15.62:0/1968467113 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.860+0000 7fb842ffd640 1 --2- 172.21.15.62:0/1968467113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e6a690 0x555a87e6ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.861+0000 7fb843fff640 1 -- 172.21.15.62:0/1968467113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e6a690 msgr2=0x555a87e6ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.861+0000 7fb843fff640 1 --2- 172.21.15.62:0/1968467113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e6a690 0x555a87e6ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.861+0000 7fb843fff640 1 -- 172.21.15.62:0/1968467113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e4ec80 msgr2=0x555a87e6a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.861+0000 7fb8437fe640 1 --2- 172.21.15.62:0/1968467113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e4ec80 0x555a87e6a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.861+0000 7fb843fff640 1 --2- 172.21.15.62:0/1968467113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e4ec80 0x555a87e6a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.861+0000 7fb843fff640 1 -- 172.21.15.62:0/1968467113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a87d40440 con 0x555a87e4e360 2024-09-11T23:53:31.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.861+0000 7fb843fff640 1 --2- 172.21.15.62:0/1968467113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e4e740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005fa0 tx=0x7fb83c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b0eac29745bca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:31.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.862+0000 7fb840ff9640 1 -- 172.21.15.62:0/4193730975 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fb838004640 con 0x7fb8441021b0 2024-09-11T23:53:31.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.862+0000 7fb829ffb640 1 -- 172.21.15.62:0/1968467113 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c030b30 con 0x555a87e4e360 2024-09-11T23:53:31.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.862+0000 7fb829ffb640 1 -- 172.21.15.62:0/1968467113 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83c036030 con 0x555a87e4e360 2024-09-11T23:53:31.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.862+0000 7fb829ffb640 1 -- 172.21.15.62:0/1968467113 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0047f0 con 0x555a87e4e360 2024-09-11T23:53:31.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.864+0000 7fb84bf72740 1 -- 172.21.15.62:0/1968467113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 msgr2=0x555a87e4e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:31.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.864+0000 7fb84bf72740 1 --2- 172.21.15.62:0/1968467113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e4e740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005fa0 tx=0x7fb83c009800 comp rx=0 tx=0).stop 2024-09-11T23:53:31.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.864+0000 7fb84bf72740 1 -- 172.21.15.62:0/1968467113 shutdown_connections 2024-09-11T23:53:31.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.864+0000 7fb84bf72740 1 --2- 172.21.15.62:0/1968467113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e6a690 0x555a87e6ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.864+0000 7fb84bf72740 1 --2- 172.21.15.62:0/1968467113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e4ec80 0x555a87e6a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.864+0000 7fb84bf72740 1 --2- 172.21.15.62:0/1968467113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e4e740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.864+0000 7fb84bf72740 1 -- 172.21.15.62:0/1968467113 >> 172.21.15.62:0/1968467113 conn(0x555a87e4aa30 msgr2=0x555a87e4ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:31.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.864+0000 7fb84bf72740 1 -- 172.21.15.62:0/1968467113 shutdown_connections 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.864+0000 7fb84bf72740 1 -- 172.21.15.62:0/1968467113 wait complete. 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.865+0000 7fb84bf72740 1 -- start start 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.865+0000 7fb84bf72740 1 Processor -- start 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.866+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e67b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.866+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4ec80 0x555a87e680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.866+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4ec80 0x555a87e680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.866+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4ec80 0x555a87e680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:47758/0 (socket says 172.21.15.62:47758) 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.866+0000 7fb843fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e67b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.866+0000 7fb843fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e67b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52208/0 (socket says 172.21.15.62:52208) 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.867+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e76080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.867+0000 7fb8437fe640 1 -- 172.21.15.62:0/2429698705 learned_addr learned my addr 172.21.15.62:0/2429698705 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.867+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555a87e512c0 con 0x555a87e6a690 2024-09-11T23:53:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.867+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a87d6f050 con 0x555a87e4ec80 2024-09-11T23:53:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.867+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a87e51440 con 0x555a87e4e360 2024-09-11T23:53:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.867+0000 7fb84bf72740 10 client.0 _ll_register_callbacks cb 0x555a87daf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.868+0000 7fb843fff640 1 -- 172.21.15.62:0/2429698705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4ec80 msgr2=0x555a87e680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.868+0000 7fb843fff640 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4ec80 0x555a87e680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.868+0000 7fb843fff640 1 -- 172.21.15.62:0/2429698705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 msgr2=0x555a87e76080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.868+0000 7fb843fff640 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e76080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.868+0000 7fb843fff640 1 -- 172.21.15.62:0/2429698705 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a87b253c0 con 0x555a87e4e360 2024-09-11T23:53:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.869+0000 7fb843fff640 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e67b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009870 tx=0x7fb83c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bab60c4f20f8873 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.869+0000 7fb840ff9640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb82c0a2d40 0x7fb82c0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.869+0000 7fb822ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:53:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.870+0000 7fb822ffd640 1 -- 172.21.15.62:0/2429698705 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c074020 con 0x555a87e4e360 2024-09-11T23:53:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.870+0000 7fb822ffd640 1 -- 172.21.15.62:0/2429698705 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb83c003970 con 0x555a87e4e360 2024-09-11T23:53:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.870+0000 7fb840ff9640 1 -- 172.21.15.62:0/4193730975 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb8380cfa70 con 0x7fb8441021b0 2024-09-11T23:53:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.870+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83c036030 con 0x555a87e4e360 2024-09-11T23:53:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.870+0000 7fb8437fe640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb82c0a2d40 0x7fb82c0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.871+0000 7fb822ffd640 1 -- 172.21.15.62:0/2429698705 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb83c07f050 con 0x555a87e4e360 2024-09-11T23:53:31.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.871+0000 7fb84bf72740 10 client.25231 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:31.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.871+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a87e512c0 con 0x555a87e4e360 2024-09-11T23:53:31.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.871+0000 7fb8437fe640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb82c0a2d40 0x7fb82c0a5200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x555a87e68f40 tx=0x7fb8340092a0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:31.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.871+0000 7fb822ffd640 1 -- 172.21.15.62:0/2429698705 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83c036030 con 0x555a87e4e360 2024-09-11T23:53:31.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.872+0000 7fb822ffd640 1 -- 172.21.15.62:0/2429698705 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb83c065070 con 0x555a87e4e360 2024-09-11T23:53:31.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.872+0000 7fb84bf72740 10 client.25231 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:31.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.872+0000 7fb84bf72740 10 client.25231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:53:31.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.872+0000 7fb84bf72740 20 client.25231 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:31.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.872+0000 7fb84bf72740 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a87e5a320 0x555a87e5c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.872+0000 7fb84bf72740 4 client.25231 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:53:31.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.872+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a87dc1850 con 0x555a87e5a320 2024-09-11T23:53:31.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.872+0000 7fb842ffd640 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a87e5a320 0x555a87e5c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.873+0000 7fb842ffd640 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a87e5a320 0x555a87e5c780 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4934ece7be88431b server_cookie=4eda52f2038f276f in_seq=0 out_seq=0 2024-09-11T23:53:31.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.873+0000 7fb822ffd640 10 client.25231 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:31.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.873+0000 7fb822ffd640 1 -- 172.21.15.62:0/2429698705 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a87dc1850 con 0x555a87e5a320 2024-09-11T23:53:31.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.873+0000 7fb822ffd640 10 client.25231 handle_command_reply: tid=0 2024-09-11T23:53:31.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.873+0000 7fb84bf72740 1 client.25231 shutdown 2024-09-11T23:53:31.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.876+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 msgr2=0x555a87e67b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:31.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.876+0000 7fb84bf72740 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e67b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009870 tx=0x7fb83c005dc0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.876+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 shutdown_connections 2024-09-11T23:53:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.876+0000 7fb84bf72740 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a87e5a320 0x555a87e5c780 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.876+0000 7fb84bf72740 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e76080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.876+0000 7fb84bf72740 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4ec80 0x555a87e680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.876+0000 7fb84bf72740 1 --2- 172.21.15.62:0/2429698705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e67b70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.876+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 >> 172.21.15.62:0/2429698705 conn(0x555a87e4aa30 msgr2=0x555a87e50790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.876+0000 7fb822ffd640 0 client.25231 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.877+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 shutdown_connections 2024-09-11T23:53:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.877+0000 7fb84bf72740 1 -- 172.21.15.62:0/2429698705 wait complete. 2024-09-11T23:53:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.877+0000 7fb84bf72740 20 client.25231 trim_cache size 0 max 16384 2024-09-11T23:53:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.904+0000 7fb84bf72740 1 Processor -- start 2024-09-11T23:53:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.904+0000 7fb84bf72740 1 -- start start 2024-09-11T23:53:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.904+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e75b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.905+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4ec80 0x555a87e71cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.905+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e75b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.905+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e75b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:47772/0 (socket says 172.21.15.62:47772) 2024-09-11T23:53:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.905+0000 7fb8437fe640 1 -- 172.21.15.62:0/632966637 learned_addr learned my addr 172.21.15.62:0/632966637 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.905+0000 7fb84bf72740 1 --2- 172.21.15.62:0/632966637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e72200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.905+0000 7fb842ffd640 1 --2- 172.21.15.62:0/632966637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4ec80 0x555a87e71cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.905+0000 7fb84bf72740 1 -- 172.21.15.62:0/632966637 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb844197760 con 0x555a87e6a690 2024-09-11T23:53:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.905+0000 7fb84bf72740 1 -- 172.21.15.62:0/632966637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb844074240 con 0x555a87e4e360 2024-09-11T23:53:31.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.905+0000 7fb84bf72740 1 -- 172.21.15.62:0/632966637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb844072090 con 0x555a87e4ec80 2024-09-11T23:53:31.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.906+0000 7fb843fff640 1 --2- 172.21.15.62:0/632966637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e72200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.906+0000 7fb8437fe640 1 -- 172.21.15.62:0/632966637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4ec80 msgr2=0x555a87e71cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.906+0000 7fb8437fe640 1 --2- 172.21.15.62:0/632966637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4ec80 0x555a87e71cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.906+0000 7fb8437fe640 1 -- 172.21.15.62:0/632966637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 msgr2=0x555a87e72200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.906+0000 7fb8437fe640 1 --2- 172.21.15.62:0/632966637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e72200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.906+0000 7fb8437fe640 1 -- 172.21.15.62:0/632966637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a87e4eab0 con 0x555a87e4e360 2024-09-11T23:53:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.906+0000 7fb8437fe640 1 --2- 172.21.15.62:0/632966637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e75b60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb834011e80 tx=0x7fb834014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e57c0e4ed4c1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.907+0000 7fb8297fa640 1 -- 172.21.15.62:0/632966637 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83401f420 con 0x555a87e4e360 2024-09-11T23:53:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.907+0000 7fb8297fa640 1 -- 172.21.15.62:0/632966637 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83401faa0 con 0x555a87e4e360 2024-09-11T23:53:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.907+0000 7fb8297fa640 1 -- 172.21.15.62:0/632966637 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834027cc0 con 0x555a87e4e360 2024-09-11T23:53:31.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.908+0000 7fb84bf72740 1 -- 172.21.15.62:0/632966637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 msgr2=0x555a87e75b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:31.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.908+0000 7fb84bf72740 1 --2- 172.21.15.62:0/632966637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e75b60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb834011e80 tx=0x7fb834014000 comp rx=0 tx=0).stop 2024-09-11T23:53:31.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.908+0000 7fb84bf72740 1 -- 172.21.15.62:0/632966637 shutdown_connections 2024-09-11T23:53:31.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.908+0000 7fb84bf72740 1 --2- 172.21.15.62:0/632966637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e72200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.908+0000 7fb84bf72740 1 --2- 172.21.15.62:0/632966637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4ec80 0x555a87e71cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.908+0000 7fb84bf72740 1 --2- 172.21.15.62:0/632966637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4e360 0x555a87e75b60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.908+0000 7fb84bf72740 1 -- 172.21.15.62:0/632966637 >> 172.21.15.62:0/632966637 conn(0x555a87e4aa30 msgr2=0x555a87e49b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:31.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.908+0000 7fb84bf72740 1 -- 172.21.15.62:0/632966637 shutdown_connections 2024-09-11T23:53:31.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.908+0000 7fb84bf72740 1 -- 172.21.15.62:0/632966637 wait complete. 2024-09-11T23:53:31.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.909+0000 7fb84bf72740 1 -- start start 2024-09-11T23:53:31.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.909+0000 7fb84bf72740 1 Processor -- start 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e72450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4ec80 0x555a87e6e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb84bf72740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e6ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb84bf72740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb844072700 con 0x555a87e6a690 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb84bf72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb844073b70 con 0x555a87e4ec80 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb84bf72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb844072dd0 con 0x555a87e4e360 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb84bf72740 10 client.0 _ll_register_callbacks cb 0x7fb8380d3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e72450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e72450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52232/0 (socket says 172.21.15.62:52232) 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb8437fe640 1 -- 172.21.15.62:0/333847181 learned_addr learned my addr 172.21.15.62:0/333847181 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb843fff640 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e6ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb8437fe640 1 -- 172.21.15.62:0/333847181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4ec80 msgr2=0x555a87e6e790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:53:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb8437fe640 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4ec80 0x555a87e6e790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb8437fe640 1 -- 172.21.15.62:0/333847181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 msgr2=0x555a87e6ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:31.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb8437fe640 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e6ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.910+0000 7fb8437fe640 1 -- 172.21.15.62:0/333847181 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a87e6f4d0 con 0x555a87e4e360 2024-09-11T23:53:31.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.911+0000 7fb8437fe640 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e72450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb834012010 tx=0x7fb834027580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f34cf1a1b3e78a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.912+0000 7fb822ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:53:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.912+0000 7fb822ffd640 1 -- 172.21.15.62:0/333847181 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83401f830 con 0x555a87e4e360 2024-09-11T23:53:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.912+0000 7fb822ffd640 1 -- 172.21.15.62:0/333847181 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb834004020 con 0x555a87e4e360 2024-09-11T23:53:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.912+0000 7fb84bf72740 1 -- 172.21.15.62:0/333847181 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83401faa0 con 0x555a87e4e360 2024-09-11T23:53:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb822ffd640 1 -- 172.21.15.62:0/333847181 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb834004220 con 0x555a87e4e360 2024-09-11T23:53:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb84bf72740 10 client.25237 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb84bf72740 1 -- 172.21.15.62:0/333847181 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a87ef5770 con 0x555a87e4e360 2024-09-11T23:53:31.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb822ffd640 1 -- 172.21.15.62:0/333847181 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83401faa0 con 0x555a87e4e360 2024-09-11T23:53:31.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb822ffd640 1 -- 172.21.15.62:0/333847181 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb83400f040 con 0x555a87e4e360 2024-09-11T23:53:31.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb84bf72740 10 client.25237 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:31.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb84bf72740 10 client.25237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:53:31.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb84bf72740 20 client.25237 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:31.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb84bf72740 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a87ef5d60 0x555a87ef8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:31.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb84bf72740 4 client.25237 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:53:31.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.913+0000 7fb84bf72740 1 -- 172.21.15.62:0/333847181 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a87dbbd60 con 0x555a87ef5d60 2024-09-11T23:53:31.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.915+0000 7fb842ffd640 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a87ef5d60 0x555a87ef8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:31.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.915+0000 7fb842ffd640 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a87ef5d60 0x555a87ef8180 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a986cd9cfc3725c server_cookie=4ecd25a6c1926d0e in_seq=0 out_seq=0 2024-09-11T23:53:31.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.915+0000 7fb822ffd640 10 client.25237 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:31.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb822ffd640 1 -- 172.21.15.62:0/333847181 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555a87dbbd60 con 0x555a87ef5d60 2024-09-11T23:53:31.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb822ffd640 10 client.25237 handle_command_reply: tid=0 2024-09-11T23:53:31.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb84bf72740 1 client.25237 shutdown 2024-09-11T23:53:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb84bf72740 1 -- 172.21.15.62:0/333847181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 msgr2=0x555a87e72450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb84bf72740 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e72450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb834012010 tx=0x7fb834027580 comp rx=0 tx=0).stop 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb84bf72740 1 -- 172.21.15.62:0/333847181 shutdown_connections 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb84bf72740 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a87ef5d60 0x555a87ef8180 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb84bf72740 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a87e6a690 0x555a87e6ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb84bf72740 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a87e4ec80 0x555a87e6e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb84bf72740 1 --2- 172.21.15.62:0/333847181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a87e4e360 0x555a87e72450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.916+0000 7fb84bf72740 1 -- 172.21.15.62:0/333847181 >> 172.21.15.62:0/333847181 conn(0x555a87e4aa30 msgr2=0x555a87e49350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.917+0000 7fb822ffd640 0 client.25237 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.917+0000 7fb84bf72740 1 -- 172.21.15.62:0/333847181 shutdown_connections 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.917+0000 7fb84bf72740 1 -- 172.21.15.62:0/333847181 wait complete. 2024-09-11T23:53:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.917+0000 7fb84bf72740 20 client.25237 trim_cache size 0 max 16384 2024-09-11T23:53:31.918 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:53:31.918 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-11T23:53:31.918 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-11T23:53:31.918 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:53:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 -- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb82c0a2d40 msgr2=0x7fb82c0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb82c0a2d40 0x7fb82c0a5200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x555a87e68f40 tx=0x7fb8340092a0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 -- 172.21.15.62:0/4193730975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8441021b0 msgr2=0x7fb8441a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8441021b0 0x7fb8441a4c30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb83800e9c0 tx=0x7fb83800ee80 comp rx=0 tx=0).stop 2024-09-11T23:53:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 -- 172.21.15.62:0/4193730975 shutdown_connections 2024-09-11T23:53:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb82c0a2d40 0x7fb82c0a5200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb844105e00 0x7fb84419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8441021b0 0x7fb8441a4c30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 --2- 172.21.15.62:0/4193730975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb844101800 0x7fb8441a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 -- 172.21.15.62:0/4193730975 >> 172.21.15.62:0/4193730975 conn(0x7fb8440fba90 msgr2=0x7fb8440fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 -- 172.21.15.62:0/4193730975 shutdown_connections 2024-09-11T23:53:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:31.918+0000 7fb849b99640 1 -- 172.21.15.62:0/4193730975 wait complete. 2024-09-11T23:53:32.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 50.0 s, 3683.8 tps, lat 27.207 ms stddev 28.470 2024-09-11T23:53:32.745 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:53:32.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:53:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:53:33.509 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:33 smithi062 ceph-mon[30958]: pgmap v46: 97 pgs: 97 active+clean; 1014 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 39 MiB/s wr, 972 op/s 2024-09-11T23:53:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:33 smithi120 ceph-mon[35182]: pgmap v46: 97 pgs: 97 active+clean; 1014 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 39 MiB/s wr, 972 op/s 2024-09-11T23:53:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.634+0000 7ffab892b640 1 -- 172.21.15.62:0/578811554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4108e00 msgr2=0x7ffab410ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.634+0000 7ffab892b640 1 --2- 172.21.15.62:0/578811554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4108e00 0x7ffab410ce00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009900 tx=0x7ffaa802a410 comp rx=0 tx=0).stop 2024-09-11T23:53:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.634+0000 7ffab892b640 1 -- 172.21.15.62:0/578811554 shutdown_connections 2024-09-11T23:53:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.634+0000 7ffab892b640 1 --2- 172.21.15.62:0/578811554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab410d530 0x7ffab41150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.634+0000 7ffab892b640 1 --2- 172.21.15.62:0/578811554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4108e00 0x7ffab410ce00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.634+0000 7ffab892b640 1 --2- 172.21.15.62:0/578811554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffab4108450 0x7ffab4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.634+0000 7ffab892b640 1 -- 172.21.15.62:0/578811554 >> 172.21.15.62:0/578811554 conn(0x7ffab4077ba0 msgr2=0x7ffab4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.634+0000 7ffab892b640 1 -- 172.21.15.62:0/578811554 shutdown_connections 2024-09-11T23:53:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.634+0000 7ffab892b640 1 -- 172.21.15.62:0/578811554 wait complete. 2024-09-11T23:53:33.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab892b640 1 Processor -- start 2024-09-11T23:53:33.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab892b640 1 -- start start 2024-09-11T23:53:33.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab892b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4108450 0x7ffab419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab892b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffab4108e00 0x7ffab419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab892b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab410d530 0x7ffab41a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab892b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4118550 con 0x7ffab410d530 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab892b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41183d0 con 0x7ffab4108450 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab892b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41186d0 con 0x7ffab4108e00 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab410d530 0x7ffab41a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffab4108e00 0x7ffab419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab410d530 0x7ffab41a33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33626/0 (socket says 172.21.15.62:33626) 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab2d9d640 1 -- 172.21.15.62:0/2025052214 learned_addr learned my addr 172.21.15.62:0/2025052214 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.635+0000 7ffab259c640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4108450 0x7ffab419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.636+0000 7ffab1d9b640 1 -- 172.21.15.62:0/2025052214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4108450 msgr2=0x7ffab419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:33.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.636+0000 7ffab1d9b640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4108450 0x7ffab419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:33.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.636+0000 7ffab1d9b640 1 -- 172.21.15.62:0/2025052214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab410d530 msgr2=0x7ffab41a33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:33.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.636+0000 7ffab1d9b640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab410d530 0x7ffab41a33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:33.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.636+0000 7ffab1d9b640 1 -- 172.21.15.62:0/2025052214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab41a3b00 con 0x7ffab4108e00 2024-09-11T23:53:33.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.636+0000 7ffab2d9d640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab410d530 0x7ffab41a33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:33.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.637+0000 7ffab1d9b640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffab4108e00 0x7ffab419f010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005940 tx=0x7ffaa8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.637+0000 7ffaa37fe640 1 -- 172.21.15.62:0/2025052214 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa803f030 con 0x7ffab4108e00 2024-09-11T23:53:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.638+0000 7ffaa37fe640 1 -- 172.21.15.62:0/2025052214 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa8003ad0 con 0x7ffab4108e00 2024-09-11T23:53:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.638+0000 7ffab892b640 1 -- 172.21.15.62:0/2025052214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab41a3d90 con 0x7ffab4108e00 2024-09-11T23:53:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.638+0000 7ffab892b640 1 -- 172.21.15.62:0/2025052214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab41ab680 con 0x7ffab4108e00 2024-09-11T23:53:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.638+0000 7ffaa37fe640 1 -- 172.21.15.62:0/2025052214 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8035070 con 0x7ffab4108e00 2024-09-11T23:53:33.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.639+0000 7ffab892b640 1 -- 172.21.15.62:0/2025052214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab4108830 con 0x7ffab4108e00 2024-09-11T23:53:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.641+0000 7ffaa37fe640 1 -- 172.21.15.62:0/2025052214 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7ffaa8042030 con 0x7ffab4108e00 2024-09-11T23:53:33.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.641+0000 7ffaa37fe640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa9c0a2e90 0x7ffa9c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:33.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.642+0000 7ffaa37fe640 1 -- 172.21.15.62:0/2025052214 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7ffaa80ef790 con 0x7ffab4108e00 2024-09-11T23:53:33.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.642+0000 7ffab259c640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa9c0a2e90 0x7ffa9c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.642+0000 7ffab259c640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa9c0a2e90 0x7ffa9c0a5350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009bd0 tx=0x7ffaa4005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:33.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.646+0000 7ffaa37fe640 1 -- 172.21.15.62:0/2025052214 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffaa80bbc10 con 0x7ffab4108e00 2024-09-11T23:53:33.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:33 smithi083 ceph-mon[34805]: pgmap v46: 97 pgs: 97 active+clean; 1014 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 39 MiB/s wr, 972 op/s 2024-09-11T23:53:33.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.984+0000 7ffab892b640 1 -- 172.21.15.62:0/2025052214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffab4110770 con 0x7ffab4108e00 2024-09-11T23:53:33.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.986+0000 7ffaa37fe640 1 -- 172.21.15.62:0/2025052214 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11912 (secure 0 0 0) 0x7ffaa80bff10 con 0x7ffab4108e00 2024-09-11T23:53:33.986 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:53:33.986 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-09-11T23:52:56:186189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:55.695276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:53:33.986 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-09-11T23:53:33.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 -- 172.21.15.62:0/2025052214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa9c0a2e90 msgr2=0x7ffa9c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:33.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa9c0a2e90 0x7ffa9c0a5350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009bd0 tx=0x7ffaa4005f90 comp rx=0 tx=0).stop 2024-09-11T23:53:33.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 -- 172.21.15.62:0/2025052214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffab4108e00 msgr2=0x7ffab419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:33.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffab4108e00 0x7ffab419f010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005940 tx=0x7ffaa8004270 comp rx=0 tx=0).stop 2024-09-11T23:53:33.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 -- 172.21.15.62:0/2025052214 shutdown_connections 2024-09-11T23:53:33.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa9c0a2e90 0x7ffa9c0a5350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:33.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab410d530 0x7ffab41a33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:33.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffab4108e00 0x7ffab419f010 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:33.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 --2- 172.21.15.62:0/2025052214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4108450 0x7ffab419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:33.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 -- 172.21.15.62:0/2025052214 >> 172.21.15.62:0/2025052214 conn(0x7ffab4077ba0 msgr2=0x7ffab4078ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:33.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.989+0000 7ffab892b640 1 -- 172.21.15.62:0/2025052214 shutdown_connections 2024-09-11T23:53:33.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:33.990+0000 7ffab892b640 1 -- 172.21.15.62:0/2025052214 wait complete. 2024-09-11T23:53:34.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:34 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2025052214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:53:34.648 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:34 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2025052214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:53:34.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:34 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2025052214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:53:34.689 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:53:35.079 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:53:35.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:35 smithi062 ceph-mon[30958]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 34 MiB/s wr, 1.13k op/s 2024-09-11T23:53:35.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:35 smithi120 ceph-mon[35182]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 34 MiB/s wr, 1.13k op/s 2024-09-11T23:53:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.632+0000 7f193ed6e640 1 -- 172.21.15.62:0/165876197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1938102ef0 msgr2=0x7f193810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.632+0000 7f193ed6e640 1 --2- 172.21.15.62:0/165876197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1938102ef0 0x7f193810da40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1930009ca0 tx=0x7f193002a950 comp rx=0 tx=0).stop 2024-09-11T23:53:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.632+0000 7f193ed6e640 1 -- 172.21.15.62:0/165876197 shutdown_connections 2024-09-11T23:53:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.632+0000 7f193ed6e640 1 --2- 172.21.15.62:0/165876197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1938102ef0 0x7f193810da40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.632+0000 7f193ed6e640 1 --2- 172.21.15.62:0/165876197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1938102570 0x7f19381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.632+0000 7f193ed6e640 1 --2- 172.21.15.62:0/165876197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938100e20 0x7f1938101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.632+0000 7f193ed6e640 1 -- 172.21.15.62:0/165876197 >> 172.21.15.62:0/165876197 conn(0x7f19380fc7d0 msgr2=0x7f19380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.632+0000 7f193ed6e640 1 -- 172.21.15.62:0/165876197 shutdown_connections 2024-09-11T23:53:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.632+0000 7f193ed6e640 1 -- 172.21.15.62:0/165876197 wait complete. 2024-09-11T23:53:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.633+0000 7f193ed6e640 1 Processor -- start 2024-09-11T23:53:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.633+0000 7f193ed6e640 1 -- start start 2024-09-11T23:53:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.633+0000 7f193ed6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1938100e20 0x7f193819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.633+0000 7f193ed6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938102570 0x7f193819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.634+0000 7f193ed6e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1938102ef0 0x7f193819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.634+0000 7f193ed6e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f193810fe80 con 0x7f1938102ef0 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.634+0000 7f193ed6e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f193810fd00 con 0x7f1938102570 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.634+0000 7f193ed6e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1938110000 con 0x7f1938100e20 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.634+0000 7f193d30b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1938102ef0 0x7f193819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.634+0000 7f193d30b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1938102ef0 0x7f193819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33642/0 (socket says 172.21.15.62:33642) 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.634+0000 7f193d30b640 1 -- 172.21.15.62:0/3651145887 learned_addr learned my addr 172.21.15.62:0/3651145887 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.634+0000 7f193d30b640 1 -- 172.21.15.62:0/3651145887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1938100e20 msgr2=0x7f193819a680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.634+0000 7f193d30b640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1938100e20 0x7f193819a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.635+0000 7f193d30b640 1 -- 172.21.15.62:0/3651145887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938102570 msgr2=0x7f193819abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.635+0000 7f193d30b640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938102570 0x7f193819abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.635+0000 7f193d30b640 1 -- 172.21.15.62:0/3651145887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193819f630 con 0x7f1938102ef0 2024-09-11T23:53:35.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.635+0000 7f193d30b640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1938102ef0 0x7f193819ef50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1930004af0 tx=0x7f193002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.636+0000 7f1935ffb640 1 -- 172.21.15.62:0/3651145887 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930034040 con 0x7f1938102ef0 2024-09-11T23:53:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.636+0000 7f1935ffb640 1 -- 172.21.15.62:0/3651145887 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1930030920 con 0x7f1938102ef0 2024-09-11T23:53:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.636+0000 7f193ed6e640 1 -- 172.21.15.62:0/3651145887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193819f8c0 con 0x7f1938102ef0 2024-09-11T23:53:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.636+0000 7f1935ffb640 1 -- 172.21.15.62:0/3651145887 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19300047f0 con 0x7f1938102ef0 2024-09-11T23:53:35.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.637+0000 7f193ed6e640 1 -- 172.21.15.62:0/3651145887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19381a7200 con 0x7f1938102ef0 2024-09-11T23:53:35.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.641+0000 7f1935ffb640 1 -- 172.21.15.62:0/3651145887 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f1930037020 con 0x7f1938102ef0 2024-09-11T23:53:35.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.642+0000 7f1941147740 1 Processor -- start 2024-09-11T23:53:35.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.642+0000 7f1941147740 1 -- start start 2024-09-11T23:53:35.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.642+0000 7f1941147740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561159666b50 0x561159665090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.642+0000 7f1941147740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x561159665a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.642+0000 7f1941147740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159678ba0 0x56115967af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.642+0000 7f1941147740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5611595cb2f0 con 0x5611596655d0 2024-09-11T23:53:35.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.642+0000 7f1941147740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561159572cf0 con 0x561159678ba0 2024-09-11T23:53:35.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.642+0000 7f1941147740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5611595c72b0 con 0x561159666b50 2024-09-11T23:53:35.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.643+0000 7f193cb0a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x561159665a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.643+0000 7f193cb0a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x561159665a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33646/0 (socket says 172.21.15.62:33646) 2024-09-11T23:53:35.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.643+0000 7f193cb0a640 1 -- 172.21.15.62:0/3176307155 learned_addr learned my addr 172.21.15.62:0/3176307155 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:35.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.644+0000 7f193cb0a640 1 -- 172.21.15.62:0/3176307155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561159666b50 msgr2=0x561159665090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:35.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.644+0000 7f193cb0a640 1 --2- 172.21.15.62:0/3176307155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561159666b50 0x561159665090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.644+0000 7f193cb0a640 1 -- 172.21.15.62:0/3176307155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159678ba0 msgr2=0x56115967af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:35.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.644+0000 7f193cb0a640 1 --2- 172.21.15.62:0/3176307155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159678ba0 0x56115967af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.644+0000 7f193cb0a640 1 -- 172.21.15.62:0/3176307155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611595573a0 con 0x5611596655d0 2024-09-11T23:53:35.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.644+0000 7f1935ffb640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19280a2f60 0x7f19280a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.644+0000 7f1935ffb640 1 -- 172.21.15.62:0/3651145887 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f19300f04f0 con 0x7f1938102ef0 2024-09-11T23:53:35.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.644+0000 7f1937fff640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19280a2f60 0x7f19280a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.645+0000 7f193cb0a640 1 --2- 172.21.15.62:0/3176307155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x561159665a30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f192c00db40 tx=0x7f192c009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508448aaf76f659 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:35.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.646+0000 7f1922ffd640 1 -- 172.21.15.62:0/3176307155 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c009b20 con 0x5611596655d0 2024-09-11T23:53:35.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.646+0000 7f1922ffd640 1 -- 172.21.15.62:0/3176307155 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f192c009cc0 con 0x5611596655d0 2024-09-11T23:53:35.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.646+0000 7f1922ffd640 1 -- 172.21.15.62:0/3176307155 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c016410 con 0x5611596655d0 2024-09-11T23:53:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.647+0000 7f1941147740 1 -- 172.21.15.62:0/3176307155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 msgr2=0x561159665a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.647+0000 7f1941147740 1 --2- 172.21.15.62:0/3176307155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x561159665a30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f192c00db40 tx=0x7f192c009370 comp rx=0 tx=0).stop 2024-09-11T23:53:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.647+0000 7f1937fff640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19280a2f60 0x7f19280a5420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1924005fd0 tx=0x7f1924005ea0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.647+0000 7f1941147740 1 -- 172.21.15.62:0/3176307155 shutdown_connections 2024-09-11T23:53:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.647+0000 7f1941147740 1 --2- 172.21.15.62:0/3176307155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159678ba0 0x56115967af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.647+0000 7f1941147740 1 --2- 172.21.15.62:0/3176307155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x561159665a30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.647+0000 7f1941147740 1 --2- 172.21.15.62:0/3176307155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561159666b50 0x561159665090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.647+0000 7f1941147740 1 -- 172.21.15.62:0/3176307155 >> 172.21.15.62:0/3176307155 conn(0x56115962e310 msgr2=0x561159661ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.647+0000 7f1941147740 1 -- 172.21.15.62:0/3176307155 shutdown_connections 2024-09-11T23:53:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.647+0000 7f1941147740 1 -- 172.21.15.62:0/3176307155 wait complete. 2024-09-11T23:53:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f1941147740 1 -- start start 2024-09-11T23:53:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f1941147740 1 Processor -- start 2024-09-11T23:53:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f1941147740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159678580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f1941147740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x561159675830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f1941147740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561159678ba0 0x561159675e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f1941147740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56115968c8e0 con 0x561159678ba0 2024-09-11T23:53:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f1941147740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561159586060 con 0x561159666b50 2024-09-11T23:53:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f1941147740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56115968ca60 con 0x5611596655d0 2024-09-11T23:53:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f1941147740 10 client.0 _ll_register_callbacks cb 0x5611592d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f193d30b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561159678ba0 0x561159675e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f193d30b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561159678ba0 0x561159675e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33658/0 (socket says 172.21.15.62:33658) 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f193d30b640 1 -- 172.21.15.62:0/2623673246 learned_addr learned my addr 172.21.15.62:0/2623673246 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.648+0000 7f193cb0a640 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159678580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.649+0000 7f1937fff640 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x561159675830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.649+0000 7f193d30b640 1 -- 172.21.15.62:0/2623673246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 msgr2=0x561159678580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.649+0000 7f193d30b640 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159678580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.649+0000 7f193d30b640 1 -- 172.21.15.62:0/2623673246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 msgr2=0x561159675830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.649+0000 7f193d30b640 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x561159675830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.649+0000 7f193d30b640 1 -- 172.21.15.62:0/2623673246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115933c3c0 con 0x561159678ba0 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.649+0000 7f1937fff640 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x561159675830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.649+0000 7f193cb0a640 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159678580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.649+0000 7f193d30b640 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561159678ba0 0x561159675e00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f19300c4fe0 tx=0x7f19300bc6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2346149b0f075e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.650+0000 7f1917fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:53:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.650+0000 7f1917fff640 1 -- 172.21.15.62:0/2623673246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19300bcd00 con 0x561159678ba0 2024-09-11T23:53:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.650+0000 7f1917fff640 1 -- 172.21.15.62:0/2623673246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f19300c5080 con 0x561159678ba0 2024-09-11T23:53:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.650+0000 7f1941147740 1 -- 172.21.15.62:0/2623673246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f192c009cc0 con 0x561159678ba0 2024-09-11T23:53:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.651+0000 7f1917fff640 1 -- 172.21.15.62:0/2623673246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19300bcfa0 con 0x561159678ba0 2024-09-11T23:53:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.651+0000 7f1941147740 10 client.15603 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.651+0000 7f1941147740 1 -- 172.21.15.62:0/2623673246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56115968c8e0 con 0x561159678ba0 2024-09-11T23:53:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.653+0000 7f1917fff640 1 -- 172.21.15.62:0/2623673246 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1930043e40 con 0x561159678ba0 2024-09-11T23:53:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.653+0000 7f1917fff640 1 -- 172.21.15.62:0/2623673246 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f19300bdbd0 con 0x561159678ba0 2024-09-11T23:53:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.653+0000 7f1941147740 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.653+0000 7f1941147740 10 client.15603 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:53:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.653+0000 7f1941147740 20 client.15603 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.653+0000 7f1941147740 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5611596997d0 0x56115969bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.653+0000 7f1941147740 4 client.15603 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:53:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.653+0000 7f1941147740 1 -- 172.21.15.62:0/2623673246 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f192c009cc0 con 0x5611596997d0 2024-09-11T23:53:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.654+0000 7f193cb0a640 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5611596997d0 0x56115969bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.655+0000 7f1917fff640 10 client.15603 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:53:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.655+0000 7f193cb0a640 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5611596997d0 0x56115969bc00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92ffe9337251a91 server_cookie=c2dea51c42dc6678 in_seq=0 out_seq=0 2024-09-11T23:53:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.656+0000 7f1917fff640 1 -- 172.21.15.62:0/2623673246 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f192c009cc0 con 0x5611596997d0 2024-09-11T23:53:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.656+0000 7f1917fff640 10 client.15603 handle_command_reply: tid=0 2024-09-11T23:53:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.656+0000 7f1941147740 1 client.15603 shutdown 2024-09-11T23:53:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.657+0000 7f1941147740 1 -- 172.21.15.62:0/2623673246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561159678ba0 msgr2=0x561159675e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.657+0000 7f1941147740 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561159678ba0 0x561159675e00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f19300c4fe0 tx=0x7f19300bc6f0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.657+0000 7f1941147740 1 -- 172.21.15.62:0/2623673246 shutdown_connections 2024-09-11T23:53:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.657+0000 7f1941147740 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5611596997d0 0x56115969bc00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.657+0000 7f1941147740 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561159678ba0 0x561159675e00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.657+0000 7f1941147740 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x561159675830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.657+0000 7f1917fff640 0 client.15603 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:53:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.658+0000 7f1941147740 1 --2- 172.21.15.62:0/2623673246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159678580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.658+0000 7f1941147740 1 -- 172.21.15.62:0/2623673246 >> 172.21.15.62:0/2623673246 conn(0x56115962e310 msgr2=0x561159667970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.658+0000 7f1941147740 1 -- 172.21.15.62:0/2623673246 shutdown_connections 2024-09-11T23:53:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.660+0000 7f1941147740 1 -- 172.21.15.62:0/2623673246 wait complete. 2024-09-11T23:53:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.660+0000 7f1941147740 20 client.15603 trim_cache size 0 max 16384 2024-09-11T23:53:35.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:35 smithi083 ceph-mon[34805]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 34 MiB/s wr, 1.13k op/s 2024-09-11T23:53:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.688+0000 7f1941147740 1 Processor -- start 2024-09-11T23:53:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.688+0000 7f1941147740 1 -- start start 2024-09-11T23:53:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.688+0000 7f1941147740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159695b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.689+0000 7f1941147740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x5611596777a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.689+0000 7f1941147740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596997d0 0x5611596a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.689+0000 7f193cb0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159695b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.689+0000 7f193cb0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159695b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52302/0 (socket says 172.21.15.62:52302) 2024-09-11T23:53:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.689+0000 7f1937fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x5611596777a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.689+0000 7f1937fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x5611596777a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:47820/0 (socket says 172.21.15.62:47820) 2024-09-11T23:53:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.689+0000 7f193d30b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596997d0 0x5611596a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.690+0000 7f193d30b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596997d0 0x5611596a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33662/0 (socket says 172.21.15.62:33662) 2024-09-11T23:53:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.690+0000 7f193cb0a640 1 -- 172.21.15.62:0/1611540041 learned_addr learned my addr 172.21.15.62:0/1611540041 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.690+0000 7f1941147740 1 -- 172.21.15.62:0/1611540041 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1938072620 con 0x5611596997d0 2024-09-11T23:53:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.690+0000 7f1941147740 1 -- 172.21.15.62:0/1611540041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f193806fef0 con 0x561159666b50 2024-09-11T23:53:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.690+0000 7f1941147740 1 -- 172.21.15.62:0/1611540041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f193806dd40 con 0x5611596655d0 2024-09-11T23:53:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f1937fff640 1 -- 172.21.15.62:0/1611540041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 msgr2=0x561159695b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f1937fff640 1 --2- 172.21.15.62:0/1611540041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159695b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f1937fff640 1 -- 172.21.15.62:0/1611540041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596997d0 msgr2=0x5611596a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f1937fff640 1 --2- 172.21.15.62:0/1611540041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596997d0 0x5611596a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f1937fff640 1 -- 172.21.15.62:0/1611540041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611596628a0 con 0x561159666b50 2024-09-11T23:53:35.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f193d30b640 1 --2- 172.21.15.62:0/1611540041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596997d0 0x5611596a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:35.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f193cb0a640 1 --2- 172.21.15.62:0/1611540041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159695b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:35.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f1937fff640 1 --2- 172.21.15.62:0/1611540041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x5611596777a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19240058c0 tx=0x7f1924010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e59ef4bfb7347b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:35.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f19227fc640 1 -- 172.21.15.62:0/1611540041 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924010d30 con 0x561159666b50 2024-09-11T23:53:35.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f19227fc640 1 -- 172.21.15.62:0/1611540041 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1924016030 con 0x561159666b50 2024-09-11T23:53:35.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.691+0000 7f19227fc640 1 -- 172.21.15.62:0/1611540041 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192400f620 con 0x561159666b50 2024-09-11T23:53:35.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.692+0000 7f1941147740 1 -- 172.21.15.62:0/1611540041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 msgr2=0x5611596777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:35.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.692+0000 7f1941147740 1 --2- 172.21.15.62:0/1611540041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x5611596777a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19240058c0 tx=0x7f1924010000 comp rx=0 tx=0).stop 2024-09-11T23:53:35.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.692+0000 7f1941147740 1 -- 172.21.15.62:0/1611540041 shutdown_connections 2024-09-11T23:53:35.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.692+0000 7f1941147740 1 --2- 172.21.15.62:0/1611540041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596997d0 0x5611596a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.692+0000 7f1941147740 1 --2- 172.21.15.62:0/1611540041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x5611596777a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.692+0000 7f1941147740 1 --2- 172.21.15.62:0/1611540041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596655d0 0x561159695b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.693+0000 7f1941147740 1 -- 172.21.15.62:0/1611540041 >> 172.21.15.62:0/1611540041 conn(0x56115962e310 msgr2=0x561159677090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:35.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.693+0000 7f1941147740 1 -- 172.21.15.62:0/1611540041 shutdown_connections 2024-09-11T23:53:35.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.693+0000 7f1941147740 1 -- 172.21.15.62:0/1611540041 wait complete. 2024-09-11T23:53:35.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.693+0000 7f1941147740 1 -- start start 2024-09-11T23:53:35.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.693+0000 7f1941147740 1 Processor -- start 2024-09-11T23:53:35.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.694+0000 7f1941147740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x5611596728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.694+0000 7f1941147740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x561159672e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.694+0000 7f1941147740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596997d0 0x561159673350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.694+0000 7f1941147740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f193806e3b0 con 0x5611596655d0 2024-09-11T23:53:35.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.694+0000 7f1941147740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f193806f820 con 0x561159666b50 2024-09-11T23:53:35.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.694+0000 7f1941147740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f193806ea80 con 0x5611596997d0 2024-09-11T23:53:35.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.694+0000 7f1941147740 10 client.0 _ll_register_callbacks cb 0x7f1930004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:35.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.694+0000 7f193cb0a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x5611596728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.694+0000 7f193cb0a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x5611596728d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33674/0 (socket says 172.21.15.62:33674) 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.694+0000 7f193cb0a640 1 -- 172.21.15.62:0/331564468 learned_addr learned my addr 172.21.15.62:0/331564468 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.695+0000 7f193d30b640 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596997d0 0x561159673350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.695+0000 7f1937fff640 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x561159672e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.695+0000 7f193cb0a640 1 -- 172.21.15.62:0/331564468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596997d0 msgr2=0x561159673350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.695+0000 7f193cb0a640 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596997d0 0x561159673350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.695+0000 7f193cb0a640 1 -- 172.21.15.62:0/331564468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 msgr2=0x561159672e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.695+0000 7f193cb0a640 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x561159672e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.695+0000 7f193cb0a640 1 -- 172.21.15.62:0/331564468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561159690380 con 0x5611596655d0 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.695+0000 7f1937fff640 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x561159672e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.695+0000 7f193cb0a640 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x5611596728d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f192c000c00 tx=0x7f192c004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08ce8a223c8e71d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.695+0000 7f1917fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:53:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.696+0000 7f1917fff640 1 -- 172.21.15.62:0/331564468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c0050d0 con 0x5611596655d0 2024-09-11T23:53:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.696+0000 7f1917fff640 1 -- 172.21.15.62:0/331564468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f192c013040 con 0x5611596655d0 2024-09-11T23:53:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.696+0000 7f1941147740 1 -- 172.21.15.62:0/331564468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1924016030 con 0x5611596655d0 2024-09-11T23:53:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.696+0000 7f1917fff640 1 -- 172.21.15.62:0/331564468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f192c005270 con 0x5611596655d0 2024-09-11T23:53:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.696+0000 7f1941147740 10 client.15609 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.696+0000 7f1941147740 1 -- 172.21.15.62:0/331564468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5611597048a0 con 0x5611596655d0 2024-09-11T23:53:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.697+0000 7f1917fff640 1 -- 172.21.15.62:0/331564468 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f192c038070 con 0x5611596655d0 2024-09-11T23:53:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.697+0000 7f1917fff640 1 -- 172.21.15.62:0/331564468 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f192c016be0 con 0x5611596655d0 2024-09-11T23:53:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.697+0000 7f1941147740 10 client.15609 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.697+0000 7f1941147740 10 client.15609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:53:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.697+0000 7f1941147740 20 client.15609 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.697+0000 7f1941147740 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x561159704ce0 0x5611597070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.697+0000 7f1941147740 4 client.15609 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:53:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.697+0000 7f1941147740 1 -- 172.21.15.62:0/331564468 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611595d8860 con 0x561159704ce0 2024-09-11T23:53:35.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.698+0000 7f1937fff640 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x561159704ce0 0x5611597070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:35.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.699+0000 7f1937fff640 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x561159704ce0 0x5611597070d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e884478a66dc4e2d server_cookie=4381c24669ed4613 in_seq=0 out_seq=0 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.699+0000 7f1917fff640 10 client.15609 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.699+0000 7f1917fff640 1 -- 172.21.15.62:0/331564468 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611595d8860 con 0x561159704ce0 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.699+0000 7f1917fff640 10 client.15609 handle_command_reply: tid=0 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.699+0000 7f1941147740 1 client.15609 shutdown 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.699+0000 7f1941147740 1 -- 172.21.15.62:0/331564468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 msgr2=0x5611596728d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.699+0000 7f1941147740 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x5611596728d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f192c000c00 tx=0x7f192c004a10 comp rx=0 tx=0).stop 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.699+0000 7f1941147740 1 -- 172.21.15.62:0/331564468 shutdown_connections 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.699+0000 7f1941147740 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x561159704ce0 0x5611597070d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.700+0000 7f1917fff640 0 client.15609 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.700+0000 7f1941147740 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611596997d0 0x561159673350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.700+0000 7f1941147740 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561159666b50 0x561159672e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.700+0000 7f1941147740 1 --2- 172.21.15.62:0/331564468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611596655d0 0x5611596728d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.700+0000 7f1941147740 1 -- 172.21.15.62:0/331564468 >> 172.21.15.62:0/331564468 conn(0x56115962e310 msgr2=0x561159673ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:35.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.700+0000 7f1941147740 1 -- 172.21.15.62:0/331564468 shutdown_connections 2024-09-11T23:53:35.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.700+0000 7f1941147740 1 -- 172.21.15.62:0/331564468 wait complete. 2024-09-11T23:53:35.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.700+0000 7f1941147740 20 client.15609 trim_cache size 0 max 16384 2024-09-11T23:53:35.703 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:53:35.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 -- 172.21.15.62:0/3651145887 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19280a2f60 msgr2=0x7f19280a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:35.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19280a2f60 0x7f19280a5420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1924005fd0 tx=0x7f1924005ea0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 -- 172.21.15.62:0/3651145887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1938102ef0 msgr2=0x7f193819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:35.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1938102ef0 0x7f193819ef50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1930004af0 tx=0x7f193002f5b0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 -- 172.21.15.62:0/3651145887 shutdown_connections 2024-09-11T23:53:35.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19280a2f60 0x7f19280a5420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1938102ef0 0x7f193819ef50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938102570 0x7f193819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 --2- 172.21.15.62:0/3651145887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1938100e20 0x7f193819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:35.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 -- 172.21.15.62:0/3651145887 >> 172.21.15.62:0/3651145887 conn(0x7f19380fc7d0 msgr2=0x7f1938106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:35.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 -- 172.21.15.62:0/3651145887 shutdown_connections 2024-09-11T23:53:35.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:35.701+0000 7f193ed6e640 1 -- 172.21.15.62:0/3651145887 wait complete. 2024-09-11T23:53:36.486 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:53:36.891 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:53:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 55.0 s, 3712.0 tps, lat 26.920 ms stddev 27.016 2024-09-11T23:53:37.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:37 smithi062 ceph-mon[30958]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.0 MiB/s wr, 651 op/s 2024-09-11T23:53:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:37 smithi120 ceph-mon[35182]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.0 MiB/s wr, 651 op/s 2024-09-11T23:53:37.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:37 smithi083 ceph-mon[34805]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.0 MiB/s wr, 651 op/s 2024-09-11T23:53:38.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.047+0000 7f8848c33640 1 -- 172.21.15.62:0/2209294636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8844108290 msgr2=0x7f884410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:38.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.047+0000 7f8848c33640 1 --2- 172.21.15.62:0/2209294636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8844108290 0x7f884410ec30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f883c009c10 tx=0x7f883c02a6e0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.047+0000 7f8848c33640 1 -- 172.21.15.62:0/2209294636 shutdown_connections 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.047+0000 7f8848c33640 1 --2- 172.21.15.62:0/2209294636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8844108290 0x7f884410ec30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.047+0000 7f8848c33640 1 --2- 172.21.15.62:0/2209294636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8844103740 0x7f8844107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.047+0000 7f8848c33640 1 --2- 172.21.15.62:0/2209294636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8844102d90 0x7f8844103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.047+0000 7f8848c33640 1 -- 172.21.15.62:0/2209294636 >> 172.21.15.62:0/2209294636 conn(0x7f88440fc7d0 msgr2=0x7f88440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.047+0000 7f8848c33640 1 -- 172.21.15.62:0/2209294636 shutdown_connections 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.047+0000 7f8848c33640 1 -- 172.21.15.62:0/2209294636 wait complete. 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.048+0000 7f8848c33640 1 Processor -- start 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.048+0000 7f8848c33640 1 -- start start 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.048+0000 7f8848c33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8844102d90 0x7f8844198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.048+0000 7f8848c33640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8844103740 0x7f8844198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8848c33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8844108290 0x7f884419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8848c33640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8844112090 con 0x7f8844103740 2024-09-11T23:53:38.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8848c33640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8844111f10 con 0x7f8844102d90 2024-09-11T23:53:38.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8848c33640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8844112210 con 0x7f8844108290 2024-09-11T23:53:38.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8841d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8844103740 0x7f8844198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8842d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8844108290 0x7f884419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8842d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8844108290 0x7f884419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52334/0 (socket says 172.21.15.62:52334) 2024-09-11T23:53:38.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8842d9d640 1 -- 172.21.15.62:0/4229042559 learned_addr learned my addr 172.21.15.62:0/4229042559 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:38.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f884259c640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8844102d90 0x7f8844198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8842d9d640 1 -- 172.21.15.62:0/4229042559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8844102d90 msgr2=0x7f8844198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8842d9d640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8844102d90 0x7f8844198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8842d9d640 1 -- 172.21.15.62:0/4229042559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8844103740 msgr2=0x7f8844198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8842d9d640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8844103740 0x7f8844198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f8842d9d640 1 -- 172.21.15.62:0/4229042559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884419d530 con 0x7f8844108290 2024-09-11T23:53:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.049+0000 7f884259c640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8844102d90 0x7f8844198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:53:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.050+0000 7f8842d9d640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8844108290 0x7f884419ce50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f883c034040 tx=0x7f883c032f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.050+0000 7f88337fe640 1 -- 172.21.15.62:0/4229042559 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c036040 con 0x7f8844108290 2024-09-11T23:53:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.050+0000 7f8848c33640 1 -- 172.21.15.62:0/4229042559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884419d7c0 con 0x7f8844108290 2024-09-11T23:53:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.050+0000 7f88337fe640 1 -- 172.21.15.62:0/4229042559 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883c003e50 con 0x7f8844108290 2024-09-11T23:53:38.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.050+0000 7f88337fe640 1 -- 172.21.15.62:0/4229042559 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c03ca90 con 0x7f8844108290 2024-09-11T23:53:38.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.050+0000 7f8848c33640 1 -- 172.21.15.62:0/4229042559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8844100920 con 0x7f8844108290 2024-09-11T23:53:38.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.054+0000 7f884b00c740 1 Processor -- start 2024-09-11T23:53:38.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.055+0000 7f884b00c740 1 -- start start 2024-09-11T23:53:38.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.055+0000 7f884b00c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x557565049090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.055+0000 7f884b00c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575650495d0 0x557565049a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f884b00c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557565054370 0x557565056760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f884b00c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557564faf2f0 con 0x55756504ab40 2024-09-11T23:53:38.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f884b00c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557564f56cf0 con 0x5575650495d0 2024-09-11T23:53:38.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f884b00c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557564fab2b0 con 0x557565054370 2024-09-11T23:53:38.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8841d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x557565049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8841d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x557565049090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33706/0 (socket says 172.21.15.62:33706) 2024-09-11T23:53:38.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8841d9b640 1 -- 172.21.15.62:0/328708286 learned_addr learned my addr 172.21.15.62:0/328708286 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:38.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8841d9b640 1 --2- 172.21.15.62:0/328708286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557565054370 0x557565056760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f884259c640 1 --2- 172.21.15.62:0/328708286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575650495d0 0x557565049a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f88337fe640 1 -- 172.21.15.62:0/4229042559 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f883c03ccf0 con 0x7f8844108290 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8841d9b640 1 -- 172.21.15.62:0/328708286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575650495d0 msgr2=0x557565049a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8841d9b640 1 --2- 172.21.15.62:0/328708286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575650495d0 0x557565049a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8841d9b640 1 -- 172.21.15.62:0/328708286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 msgr2=0x557565049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8841d9b640 1 --2- 172.21.15.62:0/328708286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x557565049090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8841d9b640 1 -- 172.21.15.62:0/328708286 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557564f3b3a0 con 0x557565054370 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8841d9b640 1 --2- 172.21.15.62:0/328708286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557565054370 0x557565056760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88380137a0 tx=0x7f8838004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76c619e9b489c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.056+0000 7f8830ff9640 1 -- 172.21.15.62:0/328708286 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838013110 con 0x557565054370 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.057+0000 7f8830ff9640 1 -- 172.21.15.62:0/328708286 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8838005500 con 0x557565054370 2024-09-11T23:53:38.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.057+0000 7f8830ff9640 1 -- 172.21.15.62:0/328708286 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838004de0 con 0x557565054370 2024-09-11T23:53:38.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.057+0000 7f88337fe640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f882c0a2ed0 0x7f882c0a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.057+0000 7f884b00c740 1 -- 172.21.15.62:0/328708286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557565054370 msgr2=0x557565056760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:38.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.057+0000 7f884b00c740 1 --2- 172.21.15.62:0/328708286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557565054370 0x557565056760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88380137a0 tx=0x7f8838004730 comp rx=0 tx=0).stop 2024-09-11T23:53:38.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.057+0000 7f88337fe640 1 -- 172.21.15.62:0/4229042559 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f883c0f3b80 con 0x7f8844108290 2024-09-11T23:53:38.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.058+0000 7f884259c640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f882c0a2ed0 0x7f882c0a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.058+0000 7f884b00c740 1 -- 172.21.15.62:0/328708286 shutdown_connections 2024-09-11T23:53:38.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.058+0000 7f884b00c740 1 --2- 172.21.15.62:0/328708286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557565054370 0x557565056760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.058+0000 7f884b00c740 1 --2- 172.21.15.62:0/328708286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575650495d0 0x557565049a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.058+0000 7f884b00c740 1 --2- 172.21.15.62:0/328708286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x557565049090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.058+0000 7f884b00c740 1 -- 172.21.15.62:0/328708286 >> 172.21.15.62:0/328708286 conn(0x557565012310 msgr2=0x557565045c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:38.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.058+0000 7f884b00c740 1 -- 172.21.15.62:0/328708286 shutdown_connections 2024-09-11T23:53:38.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.058+0000 7f884b00c740 1 -- 172.21.15.62:0/328708286 wait complete. 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.059+0000 7f884b00c740 1 -- start start 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.059+0000 7f884b00c740 1 Processor -- start 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f884b00c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5575650495d0 0x557565053d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f884b00c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55756504ab40 0x557565051010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f884b00c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x557565051600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f884b00c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557565057010 con 0x5575650495d0 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f884b00c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557564f6a060 con 0x557565054370 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f884b00c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557565057190 con 0x55756504ab40 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f884b00c740 10 client.0 _ll_register_callbacks cb 0x557564cbd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f884259c640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f882c0a2ed0 0x7f882c0a5390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x557565046ac0 tx=0x7f8834014040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f8842d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x557565051600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f8842d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x557565051600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:47866/0 (socket says 172.21.15.62:47866) 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f8842d9d640 1 -- 172.21.15.62:0/1659430677 learned_addr learned my addr 172.21.15.62:0/1659430677 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f8841d9b640 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55756504ab40 0x557565051010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.060+0000 7f884259c640 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5575650495d0 0x557565053d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f8842d9d640 1 -- 172.21.15.62:0/1659430677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55756504ab40 msgr2=0x557565051010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f8842d9d640 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55756504ab40 0x557565051010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f8842d9d640 1 -- 172.21.15.62:0/1659430677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5575650495d0 msgr2=0x557565053d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f8842d9d640 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5575650495d0 0x557565053d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f8842d9d640 1 -- 172.21.15.62:0/1659430677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557564d203c0 con 0x557565054370 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f8842d9d640 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x557565051600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f883c0bca30 tx=0x7f883c031b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc7999f554f555e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f8829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:53:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f8829ffb640 1 -- 172.21.15.62:0/1659430677 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c0c4410 con 0x557565054370 2024-09-11T23:53:38.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f8829ffb640 1 -- 172.21.15.62:0/1659430677 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f883c0c40d0 con 0x557565054370 2024-09-11T23:53:38.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f884b00c740 1 -- 172.21.15.62:0/1659430677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8838005500 con 0x557565054370 2024-09-11T23:53:38.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f8829ffb640 1 -- 172.21.15.62:0/1659430677 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f883c0c8cd0 con 0x557565054370 2024-09-11T23:53:38.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f884b00c740 10 client.25286 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:38.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.061+0000 7f884b00c740 1 -- 172.21.15.62:0/1659430677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557565062b10 con 0x557565054370 2024-09-11T23:53:38.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.062+0000 7f8829ffb640 1 -- 172.21.15.62:0/1659430677 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8838005500 con 0x557565054370 2024-09-11T23:53:38.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.062+0000 7f8829ffb640 1 -- 172.21.15.62:0/1659430677 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f883c04e030 con 0x557565054370 2024-09-11T23:53:38.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.062+0000 7f884b00c740 10 client.25286 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:38.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.062+0000 7f884b00c740 10 client.25286 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:53:38.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.062+0000 7f884b00c740 20 client.25286 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:38.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.062+0000 7f884b00c740 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557565063f70 0x5575650663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.062+0000 7f884b00c740 4 client.25286 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:53:38.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.062+0000 7f884b00c740 1 -- 172.21.15.62:0/1659430677 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557564fbc860 con 0x557565063f70 2024-09-11T23:53:38.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.063+0000 7f884259c640 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557565063f70 0x5575650663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.063+0000 7f884259c640 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557565063f70 0x5575650663a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f45740104704c30a server_cookie=e197e3d7ef82dc54 in_seq=0 out_seq=0 2024-09-11T23:53:38.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.063+0000 7f8829ffb640 10 client.25286 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:53:38.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.064+0000 7f8829ffb640 1 -- 172.21.15.62:0/1659430677 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557564fbc860 con 0x557565063f70 2024-09-11T23:53:38.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.064+0000 7f8829ffb640 10 client.25286 handle_command_reply: tid=0 2024-09-11T23:53:38.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.064+0000 7f884b00c740 1 client.25286 shutdown 2024-09-11T23:53:38.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.064+0000 7f884b00c740 1 -- 172.21.15.62:0/1659430677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 msgr2=0x557565051600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:38.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.064+0000 7f884b00c740 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x557565051600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f883c0bca30 tx=0x7f883c031b20 comp rx=0 tx=0).stop 2024-09-11T23:53:38.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.065+0000 7f884b00c740 1 -- 172.21.15.62:0/1659430677 shutdown_connections 2024-09-11T23:53:38.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.065+0000 7f884b00c740 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557565063f70 0x5575650663a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.065+0000 7f884b00c740 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x557565051600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.065+0000 7f884b00c740 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55756504ab40 0x557565051010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.065+0000 7f8829ffb640 0 client.25286 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:53:38.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.065+0000 7f884b00c740 1 --2- 172.21.15.62:0/1659430677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5575650495d0 0x557565053d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.065+0000 7f884b00c740 1 -- 172.21.15.62:0/1659430677 >> 172.21.15.62:0/1659430677 conn(0x557565012310 msgr2=0x55756504b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:38.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.065+0000 7f884b00c740 1 -- 172.21.15.62:0/1659430677 shutdown_connections 2024-09-11T23:53:38.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.067+0000 7f884b00c740 1 -- 172.21.15.62:0/1659430677 wait complete. 2024-09-11T23:53:38.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.067+0000 7f884b00c740 20 client.25286 trim_cache size 0 max 16384 2024-09-11T23:53:38.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.085+0000 7f884b00c740 1 Processor -- start 2024-09-11T23:53:38.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884b00c740 1 -- start start 2024-09-11T23:53:38.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884b00c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 0x557565052f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884b00c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x55756504c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884b00c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x55756504c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884b00c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88441adec0 con 0x55756504ab40 2024-09-11T23:53:38.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884b00c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88441ab790 con 0x557565054370 2024-09-11T23:53:38.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884b00c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88441a95e0 con 0x5575650495d0 2024-09-11T23:53:38.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 0x557565052f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 0x557565052f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52362/0 (socket says 172.21.15.62:52362) 2024-09-11T23:53:38.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884259c640 1 -- 172.21.15.62:0/3814859981 learned_addr learned my addr 172.21.15.62:0/3814859981 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:38.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f8841d9b640 1 --2- 172.21.15.62:0/3814859981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x55756504c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f8842d9d640 1 --2- 172.21.15.62:0/3814859981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x55756504c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884259c640 1 -- 172.21.15.62:0/3814859981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 msgr2=0x55756504c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:38.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.086+0000 7f884259c640 1 --2- 172.21.15.62:0/3814859981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x55756504c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.087+0000 7f884259c640 1 -- 172.21.15.62:0/3814859981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 msgr2=0x55756504c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:38.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.087+0000 7f884259c640 1 --2- 172.21.15.62:0/3814859981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x55756504c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.087+0000 7f884259c640 1 -- 172.21.15.62:0/3814859981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756504ffe0 con 0x5575650495d0 2024-09-11T23:53:38.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.087+0000 7f884259c640 1 --2- 172.21.15.62:0/3814859981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 0x557565052f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8834014010 tx=0x7f8834016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ec0f8ec032a0f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:38.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.088+0000 7f882bfff640 1 -- 172.21.15.62:0/3814859981 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834021190 con 0x5575650495d0 2024-09-11T23:53:38.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.088+0000 7f882bfff640 1 -- 172.21.15.62:0/3814859981 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8834033420 con 0x5575650495d0 2024-09-11T23:53:38.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.088+0000 7f882bfff640 1 -- 172.21.15.62:0/3814859981 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883401f420 con 0x5575650495d0 2024-09-11T23:53:38.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 -- 172.21.15.62:0/3814859981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 msgr2=0x557565052f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:38.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 --2- 172.21.15.62:0/3814859981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 0x557565052f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8834014010 tx=0x7f8834016000 comp rx=0 tx=0).stop 2024-09-11T23:53:38.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 -- 172.21.15.62:0/3814859981 shutdown_connections 2024-09-11T23:53:38.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 --2- 172.21.15.62:0/3814859981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x55756504c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 --2- 172.21.15.62:0/3814859981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x55756504c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 --2- 172.21.15.62:0/3814859981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 0x557565052f80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 -- 172.21.15.62:0/3814859981 >> 172.21.15.62:0/3814859981 conn(0x557565012310 msgr2=0x557565052870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:38.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 -- 172.21.15.62:0/3814859981 shutdown_connections 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 -- 172.21.15.62:0/3814859981 wait complete. 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 -- start start 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.089+0000 7f884b00c740 1 Processor -- start 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f884b00c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 0x55756504df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f884b00c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x55756504e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f884b00c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x55756504ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f884b00c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88441a9c50 con 0x55756504ab40 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f884b00c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88441ab0c0 con 0x557565054370 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f884b00c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88441aa320 con 0x5575650495d0 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f884b00c740 10 client.0 _ll_register_callbacks cb 0x7f883c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f8841d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x55756504e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f8841d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x55756504e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33732/0 (socket says 172.21.15.62:33732) 2024-09-11T23:53:38.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f8841d9b640 1 -- 172.21.15.62:0/3733107209 learned_addr learned my addr 172.21.15.62:0/3733107209 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:38.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f884259c640 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 0x55756504df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f8842d9d640 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x55756504ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f8841d9b640 1 -- 172.21.15.62:0/3733107209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 msgr2=0x55756504df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:38.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f8841d9b640 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 0x55756504df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f8841d9b640 1 -- 172.21.15.62:0/3733107209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 msgr2=0x55756504ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:38.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f8841d9b640 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x55756504ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.090+0000 7f8841d9b640 1 -- 172.21.15.62:0/3733107209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575650cdc40 con 0x55756504ab40 2024-09-11T23:53:38.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.091+0000 7f8841d9b640 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x55756504e4c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f88380137a0 tx=0x7f88380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bcc49bdf2e5cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:38.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.091+0000 7f8829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.091+0000 7f8829ffb640 1 -- 172.21.15.62:0/3733107209 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883802f060 con 0x55756504ab40 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.091+0000 7f884b00c740 1 -- 172.21.15.62:0/3733107209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8834033420 con 0x55756504ab40 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.091+0000 7f8829ffb640 1 -- 172.21.15.62:0/3733107209 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f8838009aa0 con 0x55756504ab40 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.091+0000 7f8829ffb640 1 -- 172.21.15.62:0/3733107209 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8838017050 con 0x55756504ab40 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.091+0000 7f884b00c740 10 client.15627 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.091+0000 7f884b00c740 1 -- 172.21.15.62:0/3733107209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575650cee80 con 0x55756504ab40 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.091+0000 7f8829ffb640 1 -- 172.21.15.62:0/3733107209 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8834033420 con 0x55756504ab40 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.092+0000 7f8829ffb640 1 -- 172.21.15.62:0/3733107209 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f88380216a0 con 0x55756504ab40 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.092+0000 7f884b00c740 10 client.15627 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.092+0000 7f884b00c740 10 client.15627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:53:38.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.092+0000 7f884b00c740 20 client.15627 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:38.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.092+0000 7f884b00c740 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557565063f70 0x5575650d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:38.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.092+0000 7f884b00c740 4 client.15627 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:53:38.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.092+0000 7f884b00c740 1 -- 172.21.15.62:0/3733107209 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557564fb6d70 con 0x557565063f70 2024-09-11T23:53:38.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.093+0000 7f884259c640 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557565063f70 0x5575650d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:38.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f8829ffb640 10 client.15627 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:53:38.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884259c640 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557565063f70 0x5575650d14c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9676a6a8b5970dbc server_cookie=f44944ff1b803b6c in_seq=0 out_seq=0 2024-09-11T23:53:38.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f8829ffb640 1 -- 172.21.15.62:0/3733107209 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557564fb6d70 con 0x557565063f70 2024-09-11T23:53:38.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f8829ffb640 10 client.15627 handle_command_reply: tid=0 2024-09-11T23:53:38.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 client.15627 shutdown 2024-09-11T23:53:38.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 -- 172.21.15.62:0/3733107209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 msgr2=0x55756504e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:38.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x55756504e4c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f88380137a0 tx=0x7f88380046c0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 -- 172.21.15.62:0/3733107209 shutdown_connections 2024-09-11T23:53:38.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x557565063f70 0x5575650d14c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557565054370 0x55756504ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756504ab40 0x55756504e4c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 --2- 172.21.15.62:0/3733107209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575650495d0 0x55756504df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f8829ffb640 0 client.15627 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:53:38.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 -- 172.21.15.62:0/3733107209 >> 172.21.15.62:0/3733107209 conn(0x557565012310 msgr2=0x557565053090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:38.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 -- 172.21.15.62:0/3733107209 shutdown_connections 2024-09-11T23:53:38.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 1 -- 172.21.15.62:0/3733107209 wait complete. 2024-09-11T23:53:38.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.094+0000 7f884b00c740 20 client.15627 trim_cache size 0 max 16384 2024-09-11T23:53:38.101 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:53:38.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 -- 172.21.15.62:0/4229042559 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f882c0a2ed0 msgr2=0x7f882c0a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:38.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f882c0a2ed0 0x7f882c0a5390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x557565046ac0 tx=0x7f8834014040 comp rx=0 tx=0).stop 2024-09-11T23:53:38.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 -- 172.21.15.62:0/4229042559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8844108290 msgr2=0x7f884419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:38.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8844108290 0x7f884419ce50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f883c034040 tx=0x7f883c032f70 comp rx=0 tx=0).stop 2024-09-11T23:53:38.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 -- 172.21.15.62:0/4229042559 shutdown_connections 2024-09-11T23:53:38.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f882c0a2ed0 0x7f882c0a5390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8844108290 0x7f884419ce50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8844103740 0x7f8844198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 --2- 172.21.15.62:0/4229042559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8844102d90 0x7f8844198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:38.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 -- 172.21.15.62:0/4229042559 >> 172.21.15.62:0/4229042559 conn(0x7f88440fc7d0 msgr2=0x7f8844105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:38.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.095+0000 7f8848c33640 1 -- 172.21.15.62:0/4229042559 shutdown_connections 2024-09-11T23:53:38.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:38.096+0000 7f8848c33640 1 -- 172.21.15.62:0/4229042559 wait complete. 2024-09-11T23:53:38.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:53:39.198 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:53:39.576 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:39 smithi062 ceph-mon[30958]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 971 op/s 2024-09-11T23:53:39.576 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:39 smithi062 ceph-mon[30958]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-11T23:53:39.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:39 smithi120 ceph-mon[35182]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 971 op/s 2024-09-11T23:53:39.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:39 smithi120 ceph-mon[35182]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-11T23:53:39.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:39 smithi083 ceph-mon[34805]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 971 op/s 2024-09-11T23:53:39.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:39 smithi083 ceph-mon[34805]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-11T23:53:39.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.713+0000 7f6d10c3e640 1 -- 172.21.15.62:0/936164536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0c103670 msgr2=0x7f6d0c10bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:39.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.713+0000 7f6d10c3e640 1 --2- 172.21.15.62:0/936164536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0c103670 0x7f6d0c10bb60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009900 tx=0x7f6cf402a480 comp rx=0 tx=0).stop 2024-09-11T23:53:39.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.713+0000 7f6d10c3e640 1 -- 172.21.15.62:0/936164536 shutdown_connections 2024-09-11T23:53:39.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.713+0000 7f6d10c3e640 1 --2- 172.21.15.62:0/936164536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0c10c180 0x7f6d0c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.713+0000 7f6d10c3e640 1 --2- 172.21.15.62:0/936164536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0c103670 0x7f6d0c10bb60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.713+0000 7f6d10c3e640 1 --2- 172.21.15.62:0/936164536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d0c102d50 0x7f6d0c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.713+0000 7f6d10c3e640 1 -- 172.21.15.62:0/936164536 >> 172.21.15.62:0/936164536 conn(0x7f6d0c0fc7d0 msgr2=0x7f6d0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:39.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.713+0000 7f6d10c3e640 1 -- 172.21.15.62:0/936164536 shutdown_connections 2024-09-11T23:53:39.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.713+0000 7f6d10c3e640 1 -- 172.21.15.62:0/936164536 wait complete. 2024-09-11T23:53:39.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.714+0000 7f6d10c3e640 1 Processor -- start 2024-09-11T23:53:39.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.714+0000 7f6d10c3e640 1 -- start start 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d10c3e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0c102d50 0x7f6d0c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d10c3e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0c103670 0x7f6d0c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d10c3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d0c10c180 0x7f6d0c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d10c3e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c111fd0 con 0x7f6d0c103670 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d10c3e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c111e50 con 0x7f6d0c102d50 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d10c3e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c112150 con 0x7f6d0c10c180 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d03fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0c103670 0x7f6d0c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0c102d50 0x7f6d0c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0c102d50 0x7f6d0c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:47914/0 (socket says 172.21.15.62:47914) 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0a59c640 1 -- 172.21.15.62:0/3403821325 learned_addr learned my addr 172.21.15.62:0/3403821325 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d0c10c180 0x7f6d0c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0a59c640 1 -- 172.21.15.62:0/3403821325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d0c10c180 msgr2=0x7f6d0c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:39.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d0c10c180 0x7f6d0c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0a59c640 1 -- 172.21.15.62:0/3403821325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0c103670 msgr2=0x7f6d0c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:39.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0c103670 0x7f6d0c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0a59c640 1 -- 172.21.15.62:0/3403821325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c19d440 con 0x7f6d0c102d50 2024-09-11T23:53:39.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d03fff640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0c103670 0x7f6d0c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:53:39.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d0c10c180 0x7f6d0c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:53:39.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.715+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0c102d50 0x7f6d0c198490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00e9f0 tx=0x7f6cfc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:39.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.716+0000 7f6d037fe640 1 -- 172.21.15.62:0/3403821325 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00cd80 con 0x7f6d0c102d50 2024-09-11T23:53:39.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.716+0000 7f6d037fe640 1 -- 172.21.15.62:0/3403821325 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cfc010910 con 0x7f6d0c102d50 2024-09-11T23:53:39.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.716+0000 7f6d037fe640 1 -- 172.21.15.62:0/3403821325 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc018990 con 0x7f6d0c102d50 2024-09-11T23:53:39.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.716+0000 7f6d10c3e640 1 -- 172.21.15.62:0/3403821325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0c19d730 con 0x7f6d0c102d50 2024-09-11T23:53:39.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.716+0000 7f6d10c3e640 1 -- 172.21.15.62:0/3403821325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0c19db30 con 0x7f6d0c102d50 2024-09-11T23:53:39.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.721+0000 7f6d037fe640 1 -- 172.21.15.62:0/3403821325 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6cfc010ac0 con 0x7f6d0c102d50 2024-09-11T23:53:39.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.721+0000 7f6d13017740 1 Processor -- start 2024-09-11T23:53:39.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.722+0000 7f6d13017740 1 -- start start 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.722+0000 7f6d13017740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c5d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.722+0000 7f6d13017740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 0x55ce10c5e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.722+0000 7f6d037fe640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6cec0a2e40 0x7f6cec0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d03fff640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6cec0a2e40 0x7f6cec0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d037fe640 1 -- 172.21.15.62:0/3403821325 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6cfc0cf850 con 0x7f6d0c102d50 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d0ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c5d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d0ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c5d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52406/0 (socket says 172.21.15.62:52406) 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d13017740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 0x55ce10c6af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d13017740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ce10bbb2f0 con 0x55ce10c5dde0 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d13017740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ce10b62cf0 con 0x55ce10c68b80 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d13017740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ce10bb72b0 con 0x55ce10c5f350 2024-09-11T23:53:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d0ad9d640 1 -- 172.21.15.62:0/3857961601 learned_addr learned my addr 172.21.15.62:0/3857961601 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/3857961601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 0x55ce10c5e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/3857961601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 0x55ce10c6af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d0ad9d640 1 -- 172.21.15.62:0/3857961601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 msgr2=0x55ce10c6af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/3857961601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 0x55ce10c6af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d0ad9d640 1 -- 172.21.15.62:0/3857961601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 msgr2=0x55ce10c5e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/3857961601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 0x55ce10c5e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.723+0000 7f6d0ad9d640 1 -- 172.21.15.62:0/3857961601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce10b473a0 con 0x55ce10c5f350 2024-09-11T23:53:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.724+0000 7f6d03fff640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6cec0a2e40 0x7f6cec0a5300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4000c00 tx=0x7f6cf402a980 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.724+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/3857961601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c5d8a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d04005c40 tx=0x7f6d04009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d348427b0d3da2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.724+0000 7f6d00ff9640 1 -- 172.21.15.62:0/3857961601 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0401a070 con 0x55ce10c5f350 2024-09-11T23:53:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.724+0000 7f6d00ff9640 1 -- 172.21.15.62:0/3857961601 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d04010e30 con 0x55ce10c5f350 2024-09-11T23:53:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.724+0000 7f6d00ff9640 1 -- 172.21.15.62:0/3857961601 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04011cc0 con 0x55ce10c5f350 2024-09-11T23:53:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.725+0000 7f6d13017740 1 -- 172.21.15.62:0/3857961601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 msgr2=0x55ce10c5d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.725+0000 7f6d13017740 1 --2- 172.21.15.62:0/3857961601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c5d8a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d04005c40 tx=0x7f6d04009c80 comp rx=0 tx=0).stop 2024-09-11T23:53:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.725+0000 7f6d13017740 1 -- 172.21.15.62:0/3857961601 shutdown_connections 2024-09-11T23:53:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.725+0000 7f6d13017740 1 --2- 172.21.15.62:0/3857961601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 0x55ce10c6af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.726+0000 7f6d13017740 1 --2- 172.21.15.62:0/3857961601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 0x55ce10c5e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.726+0000 7f6d13017740 1 --2- 172.21.15.62:0/3857961601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c5d8a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.726+0000 7f6d13017740 1 -- 172.21.15.62:0/3857961601 >> 172.21.15.62:0/3857961601 conn(0x55ce10c1e310 msgr2=0x55ce10c5a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.727+0000 7f6d13017740 1 -- 172.21.15.62:0/3857961601 shutdown_connections 2024-09-11T23:53:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.727+0000 7f6d13017740 1 -- 172.21.15.62:0/3857961601 wait complete. 2024-09-11T23:53:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.727+0000 7f6d13017740 1 -- start start 2024-09-11T23:53:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.727+0000 7f6d13017740 1 Processor -- start 2024-09-11T23:53:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d13017740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c685a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d13017740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c65850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d13017740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c68b80 0x55ce10c65e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d13017740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ce10c6b880 con 0x55ce10c68b80 2024-09-11T23:53:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d13017740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ce10b76060 con 0x55ce10c5dde0 2024-09-11T23:53:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d13017740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ce10c6ba00 con 0x55ce10c5f350 2024-09-11T23:53:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d0a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d0a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c685a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:47920/0 (socket says 172.21.15.62:47920) 2024-09-11T23:53:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d0a59c640 1 -- 172.21.15.62:0/1110785782 learned_addr learned my addr 172.21.15.62:0/1110785782 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d13017740 10 client.0 _ll_register_callbacks cb 0x55ce108c98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d03fff640 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c65850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.728+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c68b80 0x55ce10c65e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.729+0000 7f6d03fff640 1 -- 172.21.15.62:0/1110785782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 msgr2=0x55ce10c685a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.729+0000 7f6d03fff640 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c685a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.729+0000 7f6d03fff640 1 -- 172.21.15.62:0/1110785782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c68b80 msgr2=0x55ce10c65e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.729+0000 7f6d03fff640 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c68b80 0x55ce10c65e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.729+0000 7f6d03fff640 1 -- 172.21.15.62:0/1110785782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1092c3c0 con 0x55ce10c5f350 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.729+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c68b80 0x55ce10c65e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.729+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.730+0000 7f6d03fff640 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c65850 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cf402aa70 tx=0x7f6cf4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b881206668ea9b93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.730+0000 7f6cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.730+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/1110785782 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf404c020 con 0x55ce10c5f350 2024-09-11T23:53:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.730+0000 7f6d13017740 1 -- 172.21.15.62:0/1110785782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d04010e30 con 0x55ce10c5f350 2024-09-11T23:53:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.730+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/1110785782 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6cf4004ad0 con 0x55ce10c5f350 2024-09-11T23:53:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.730+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/1110785782 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6cf4057050 con 0x55ce10c5f350 2024-09-11T23:53:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6d13017740 10 client.25285 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6d13017740 1 -- 172.21.15.62:0/1110785782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ce10c773a0 con 0x55ce10c5f350 2024-09-11T23:53:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/1110785782 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d04010e30 con 0x55ce10c5f350 2024-09-11T23:53:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/1110785782 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6cf40035f0 con 0x55ce10c5f350 2024-09-11T23:53:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6d13017740 10 client.25285 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6d13017740 10 client.25285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:53:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6d13017740 20 client.25285 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6d13017740 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ce10c78bf0 0x55ce10c7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6d13017740 4 client.25285 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6d13017740 1 -- 172.21.15.62:0/1110785782 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce10bc8860 con 0x55ce10c78bf0 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.731+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ce10c78bf0 0x55ce10c7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.732+0000 7f6cf9ffb640 10 client.25285 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.732+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ce10c78bf0 0x55ce10c7b090 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6932ed1d7846b4b server_cookie=73719dd1029b5013 in_seq=0 out_seq=0 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/1110785782 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce10bc8860 con 0x55ce10c78bf0 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6cf9ffb640 10 client.25285 handle_command_reply: tid=0 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6d13017740 1 client.25285 shutdown 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6d13017740 1 -- 172.21.15.62:0/1110785782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 msgr2=0x55ce10c65850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6d13017740 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c65850 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cf402aa70 tx=0x7f6cf4009260 comp rx=0 tx=0).stop 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6d13017740 1 -- 172.21.15.62:0/1110785782 shutdown_connections 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6d13017740 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ce10c78bf0 0x55ce10c7b090 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6d13017740 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c68b80 0x55ce10c65e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6d13017740 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c5f350 0x55ce10c65850 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6d13017740 1 --2- 172.21.15.62:0/1110785782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6cf9ffb640 0 client.25285 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:39.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6d13017740 1 -- 172.21.15.62:0/1110785782 >> 172.21.15.62:0/1110785782 conn(0x55ce10c1e310 msgr2=0x55ce10c60170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:39.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.733+0000 7f6d13017740 1 -- 172.21.15.62:0/1110785782 shutdown_connections 2024-09-11T23:53:39.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.734+0000 7f6d13017740 1 -- 172.21.15.62:0/1110785782 wait complete. 2024-09-11T23:53:39.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.734+0000 7f6d13017740 20 client.25285 trim_cache size 0 max 16384 2024-09-11T23:53:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d13017740 1 Processor -- start 2024-09-11T23:53:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d13017740 1 -- start start 2024-09-11T23:53:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d13017740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c74b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d13017740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c68b80 0x55ce10c809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d13017740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c64830 0x55ce10c80ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d13017740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1adfd0 con 0x55ce10c64830 2024-09-11T23:53:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d13017740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1ab8a0 con 0x55ce10c5dde0 2024-09-11T23:53:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d13017740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1a96f0 con 0x55ce10c68b80 2024-09-11T23:53:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d0a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c74b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d0ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c64830 0x55ce10c80ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d03fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c68b80 0x55ce10c809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d0a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c74b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:47934/0 (socket says 172.21.15.62:47934) 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.752+0000 7f6d0a59c640 1 -- 172.21.15.62:0/352487986 learned_addr learned my addr 172.21.15.62:0/352487986 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6d0a59c640 1 -- 172.21.15.62:0/352487986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c68b80 msgr2=0x55ce10c809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/352487986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c68b80 0x55ce10c809a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6d0a59c640 1 -- 172.21.15.62:0/352487986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c64830 msgr2=0x55ce10c80ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/352487986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c64830 0x55ce10c80ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6d0a59c640 1 -- 172.21.15.62:0/352487986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce10c661d0 con 0x55ce10c5dde0 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6d03fff640 1 --2- 172.21.15.62:0/352487986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c68b80 0x55ce10c809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/352487986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c64830 0x55ce10c80ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/352487986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c74b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc000c00 tx=0x7f6cfc0cf760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4e55dd94956862e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6cfbfff640 1 -- 172.21.15.62:0/352487986 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0a00f0 con 0x55ce10c5dde0 2024-09-11T23:53:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6cfbfff640 1 -- 172.21.15.62:0/352487986 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cfc0a49b0 con 0x55ce10c5dde0 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.753+0000 7f6cfbfff640 1 -- 172.21.15.62:0/352487986 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc09b190 con 0x55ce10c5dde0 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.754+0000 7f6d13017740 1 -- 172.21.15.62:0/352487986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 msgr2=0x55ce10c74b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.754+0000 7f6d13017740 1 --2- 172.21.15.62:0/352487986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c74b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc000c00 tx=0x7f6cfc0cf760 comp rx=0 tx=0).stop 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.754+0000 7f6d13017740 1 -- 172.21.15.62:0/352487986 shutdown_connections 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.754+0000 7f6d13017740 1 --2- 172.21.15.62:0/352487986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c64830 0x55ce10c80ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.754+0000 7f6d13017740 1 --2- 172.21.15.62:0/352487986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c68b80 0x55ce10c809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.754+0000 7f6d13017740 1 --2- 172.21.15.62:0/352487986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c5dde0 0x55ce10c74b20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.754+0000 7f6d13017740 1 -- 172.21.15.62:0/352487986 >> 172.21.15.62:0/352487986 conn(0x55ce10c1e310 msgr2=0x55ce10c64120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.754+0000 7f6d13017740 1 -- 172.21.15.62:0/352487986 shutdown_connections 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.754+0000 7f6d13017740 1 -- 172.21.15.62:0/352487986 wait complete. 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.754+0000 7f6d13017740 1 -- start start 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d13017740 1 Processor -- start 2024-09-11T23:53:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d13017740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 0x55ce10c6cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d13017740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c64830 0x55ce10c6d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d13017740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 0x55ce10c6d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d13017740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1a9d60 con 0x55ce10c5dde0 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d13017740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1ab1d0 con 0x55ce10c68b80 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d0a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 0x55ce10c6cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d0a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 0x55ce10c6cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33798/0 (socket says 172.21.15.62:33798) 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d0a59c640 1 -- 172.21.15.62:0/3645947123 learned_addr learned my addr 172.21.15.62:0/3645947123 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 0x55ce10c6d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d03fff640 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c64830 0x55ce10c6d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.755+0000 7f6d13017740 1 -- 172.21.15.62:0/3645947123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1aa430 con 0x55ce10c64830 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6d13017740 10 client.0 _ll_register_callbacks cb 0x7f6cf403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6d0a59c640 1 -- 172.21.15.62:0/3645947123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c64830 msgr2=0x55ce10c6d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c64830 0x55ce10c6d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6d0a59c640 1 -- 172.21.15.62:0/3645947123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 msgr2=0x55ce10c6d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 0x55ce10c6d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6d0a59c640 1 -- 172.21.15.62:0/3645947123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce10ce2b70 con 0x55ce10c5dde0 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6d03fff640 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c64830 0x55ce10c6d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6d0ad9d640 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 0x55ce10c6d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6d0a59c640 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 0x55ce10c6cbb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc098710 tx=0x7f6cfc0a00f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7119432e9ee3c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/3645947123 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0b8020 con 0x55ce10c5dde0 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/3645947123 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6cfc018bd0 con 0x55ce10c5dde0 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.756+0000 7f6d13017740 1 -- 172.21.15.62:0/3645947123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cfc0a49b0 con 0x55ce10c5dde0 2024-09-11T23:53:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.757+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/3645947123 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6cfc09b190 con 0x55ce10c5dde0 2024-09-11T23:53:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.757+0000 7f6d13017740 10 client.15642 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.757+0000 7f6d13017740 1 -- 172.21.15.62:0/3645947123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ce10ce3df0 con 0x55ce10c5dde0 2024-09-11T23:53:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.757+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/3645947123 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cfc09b330 con 0x55ce10c5dde0 2024-09-11T23:53:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.757+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/3645947123 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6cfc09ad00 con 0x55ce10c5dde0 2024-09-11T23:53:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.757+0000 7f6d13017740 10 client.15642 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.757+0000 7f6d13017740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:53:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.757+0000 7f6d13017740 20 client.15642 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.758+0000 7f6d13017740 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ce10ce4430 0x55ce10ce6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.758+0000 7f6d13017740 4 client.15642 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:53:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.758+0000 7f6d13017740 1 -- 172.21.15.62:0/3645947123 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6cfc0a49b0 con 0x55ce10ce4430 2024-09-11T23:53:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.758+0000 7f6d03fff640 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ce10ce4430 0x55ce10ce6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6cf9ffb640 10 client.15642 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6d03fff640 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ce10ce4430 0x55ce10ce6850 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882bd4674a5e71fe server_cookie=aac440f5c9fb23d1 in_seq=0 out_seq=0 2024-09-11T23:53:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6cf9ffb640 1 -- 172.21.15.62:0/3645947123 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6cfc0a49b0 con 0x55ce10ce4430 2024-09-11T23:53:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6cf9ffb640 10 client.15642 handle_command_reply: tid=0 2024-09-11T23:53:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6d13017740 1 client.15642 shutdown 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6d13017740 1 -- 172.21.15.62:0/3645947123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 msgr2=0x55ce10c6cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6d13017740 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 0x55ce10c6cbb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc098710 tx=0x7f6cfc0a00f0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6d13017740 1 -- 172.21.15.62:0/3645947123 shutdown_connections 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6d13017740 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ce10ce4430 0x55ce10ce6850 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6d13017740 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce10c68b80 0x55ce10c6d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6d13017740 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ce10c64830 0x55ce10c6d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6d13017740 1 --2- 172.21.15.62:0/3645947123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce10c5dde0 0x55ce10c6cbb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.759+0000 7f6d13017740 1 -- 172.21.15.62:0/3645947123 >> 172.21.15.62:0/3645947123 conn(0x55ce10c1e310 msgr2=0x55ce10c670b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6cf9ffb640 0 client.15642 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d13017740 1 -- 172.21.15.62:0/3645947123 shutdown_connections 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d13017740 1 -- 172.21.15.62:0/3645947123 wait complete. 2024-09-11T23:53:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d13017740 20 client.15642 trim_cache size 0 max 16384 2024-09-11T23:53:39.765 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:53:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d10c3e640 1 -- 172.21.15.62:0/3403821325 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6cec0a2e40 msgr2=0x7f6cec0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d10c3e640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6cec0a2e40 0x7f6cec0a5300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4000c00 tx=0x7f6cf402a980 comp rx=0 tx=0).stop 2024-09-11T23:53:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d10c3e640 1 -- 172.21.15.62:0/3403821325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0c102d50 msgr2=0x7f6d0c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d10c3e640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0c102d50 0x7f6d0c198490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00e9f0 tx=0x7f6cfc00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d10c3e640 1 -- 172.21.15.62:0/3403821325 shutdown_connections 2024-09-11T23:53:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d10c3e640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6cec0a2e40 0x7f6cec0a5300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d10c3e640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d0c10c180 0x7f6d0c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d10c3e640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0c103670 0x7f6d0c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d10c3e640 1 --2- 172.21.15.62:0/3403821325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0c102d50 0x7f6d0c198490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.760+0000 7f6d10c3e640 1 -- 172.21.15.62:0/3403821325 >> 172.21.15.62:0/3403821325 conn(0x7f6d0c0fc7d0 msgr2=0x7f6d0c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.761+0000 7f6d10c3e640 1 -- 172.21.15.62:0/3403821325 shutdown_connections 2024-09-11T23:53:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:39.761+0000 7f6d10c3e640 1 -- 172.21.15.62:0/3403821325 wait complete. 2024-09-11T23:53:41.424 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:53:41.425 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:53:41.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:41 smithi120 ceph-mon[35182]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.8 MiB/s wr, 639 op/s 2024-09-11T23:53:41.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:41 smithi083 ceph-mon[34805]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.8 MiB/s wr, 639 op/s 2024-09-11T23:53:41.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:41 smithi062 ceph-mon[30958]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.8 MiB/s wr, 639 op/s 2024-09-11T23:53:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:53:42.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.326+0000 7f83acdf5640 1 -- 172.21.15.62:0/1014680488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83a8104000 msgr2=0x7f83a81043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:42.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.326+0000 7f83acdf5640 1 --2- 172.21.15.62:0/1014680488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83a8104000 0x7f83a81043e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8398009900 tx=0x7f839802a410 comp rx=0 tx=0).stop 2024-09-11T23:53:42.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.326+0000 7f83acdf5640 1 -- 172.21.15.62:0/1014680488 shutdown_connections 2024-09-11T23:53:42.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.326+0000 7f83acdf5640 1 --2- 172.21.15.62:0/1014680488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a8109130 0x7f83a8110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.326+0000 7f83acdf5640 1 --2- 172.21.15.62:0/1014680488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81049b0 0x7f83a8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.326+0000 7f83acdf5640 1 --2- 172.21.15.62:0/1014680488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83a8104000 0x7f83a81043e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.326+0000 7f83acdf5640 1 -- 172.21.15.62:0/1014680488 >> 172.21.15.62:0/1014680488 conn(0x7f83a80ffa20 msgr2=0x7f83a8101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:42.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.326+0000 7f83acdf5640 1 -- 172.21.15.62:0/1014680488 shutdown_connections 2024-09-11T23:53:42.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.326+0000 7f83acdf5640 1 -- 172.21.15.62:0/1014680488 wait complete. 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.327+0000 7f83acdf5640 1 Processor -- start 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.327+0000 7f83acdf5640 1 -- start start 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83acdf5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a8104000 0x7f83a819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83acdf5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81049b0 0x7f83a819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83acdf5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83a8109130 0x7f83a819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83acdf5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8114180 con 0x7f83a8104000 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83acdf5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8114000 con 0x7f83a81049b0 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83acdf5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8114300 con 0x7f83a8109130 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a8104000 0x7f83a819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81049b0 0x7f83a819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81049b0 0x7f83a819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:46192/0 (socket says 172.21.15.62:46192) 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a5d9b640 1 -- 172.21.15.62:0/539422762 learned_addr learned my addr 172.21.15.62:0/539422762 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a6d9d640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83a8109130 0x7f83a819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a5d9b640 1 -- 172.21.15.62:0/539422762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83a8109130 msgr2=0x7f83a819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83a8109130 0x7f83a819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a5d9b640 1 -- 172.21.15.62:0/539422762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a8104000 msgr2=0x7f83a819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a8104000 0x7f83a819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a5d9b640 1 -- 172.21.15.62:0/539422762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a819f650 con 0x7f83a81049b0 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81049b0 0x7f83a819abe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f839c00e9f0 tx=0x7f839c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.328+0000 7f83977fe640 1 -- 172.21.15.62:0/539422762 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c00cd80 con 0x7f83a81049b0 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.329+0000 7f83977fe640 1 -- 172.21.15.62:0/539422762 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f839c010910 con 0x7f83a81049b0 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.329+0000 7f83acdf5640 1 -- 172.21.15.62:0/539422762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a819f940 con 0x7f83a81049b0 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.329+0000 7f83977fe640 1 -- 172.21.15.62:0/539422762 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c018990 con 0x7f83a81049b0 2024-09-11T23:53:42.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.329+0000 7f83acdf5640 1 -- 172.21.15.62:0/539422762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a81a7220 con 0x7f83a81049b0 2024-09-11T23:53:42.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.334+0000 7f83af1ce740 1 Processor -- start 2024-09-11T23:53:42.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.334+0000 7f83af1ce740 1 -- start start 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.334+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a44f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.334+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 0x556311a4f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.334+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a52130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.334+0000 7f83af1ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5563119aa280 con 0x556311a4fd40 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.334+0000 7f83af1ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556311951b10 con 0x556311a45480 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.334+0000 7f83af1ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5563119a6240 con 0x556311a44b60 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.334+0000 7f83977fe640 1 -- 172.21.15.62:0/539422762 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f839c018b30 con 0x7f83a81049b0 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.335+0000 7f83a659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 0x556311a4f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.335+0000 7f83a659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 0x556311a4f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:46194/0 (socket says 172.21.15.62:46194) 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.335+0000 7f83a659c640 1 -- 172.21.15.62:0/2338522776 learned_addr learned my addr 172.21.15.62:0/2338522776 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.335+0000 7f83a6d9d640 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a44f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.335+0000 7f83a659c640 1 -- 172.21.15.62:0/2338522776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 msgr2=0x556311a44f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:42.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.335+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a52130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.336+0000 7f83a659c640 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a44f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.336+0000 7f83a659c640 1 -- 172.21.15.62:0/2338522776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 msgr2=0x556311a52130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.336+0000 7f83a659c640 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a52130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.336+0000 7f83a659c640 1 -- 172.21.15.62:0/2338522776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55631195e1f0 con 0x556311a45480 2024-09-11T23:53:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.336+0000 7f83a6d9d640 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a44f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:53:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.336+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a52130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:53:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.336+0000 7f83a659c640 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 0x556311a4f800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8398009900 tx=0x7f8398005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31063741da1374aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.337+0000 7f8394ff9640 1 -- 172.21.15.62:0/2338522776 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398035040 con 0x556311a45480 2024-09-11T23:53:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.337+0000 7f8394ff9640 1 -- 172.21.15.62:0/2338522776 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8398004400 con 0x556311a45480 2024-09-11T23:53:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.337+0000 7f83977fe640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83900a2d70 0x7f83900a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.337+0000 7f8394ff9640 1 -- 172.21.15.62:0/2338522776 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980035f0 con 0x556311a45480 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.338+0000 7f83977fe640 1 -- 172.21.15.62:0/539422762 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f839c0cf6f0 con 0x7f83a81049b0 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.338+0000 7f83af1ce740 1 -- 172.21.15.62:0/2338522776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 msgr2=0x556311a4f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.338+0000 7f83af1ce740 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 0x556311a4f800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8398009900 tx=0x7f8398005de0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.338+0000 7f83a6d9d640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83900a2d70 0x7f83900a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.338+0000 7f83af1ce740 1 -- 172.21.15.62:0/2338522776 shutdown_connections 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.338+0000 7f83af1ce740 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a52130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.338+0000 7f83af1ce740 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 0x556311a4f800 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.338+0000 7f83af1ce740 1 --2- 172.21.15.62:0/2338522776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a44f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.338+0000 7f83af1ce740 1 -- 172.21.15.62:0/2338522776 >> 172.21.15.62:0/2338522776 conn(0x556311a410f0 msgr2=0x556311a414d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.339+0000 7f83af1ce740 1 -- 172.21.15.62:0/2338522776 shutdown_connections 2024-09-11T23:53:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.339+0000 7f83af1ce740 1 -- 172.21.15.62:0/2338522776 wait complete. 2024-09-11T23:53:42.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.339+0000 7f83a6d9d640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83900a2d70 0x7f83900a5230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005fd0 tx=0x7f83a0009500 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:42.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.339+0000 7f83af1ce740 1 -- start start 2024-09-11T23:53:42.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.339+0000 7f83af1ce740 1 Processor -- start 2024-09-11T23:53:42.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a44b60 0x556311a4f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a45480 0x556311a4cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a4d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83a659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a44b60 0x556311a4f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83a659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a44b60 0x556311a4f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:46210/0 (socket says 172.21.15.62:46210) 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83a659c640 1 -- 172.21.15.62:0/2809434693 learned_addr learned my addr 172.21.15.62:0/2809434693 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556311a52960 con 0x556311a4fd40 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a45480 0x556311a4cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556311964f10 con 0x556311a44b60 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556311a52ae0 con 0x556311a45480 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83a6d9d640 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a4d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.340+0000 7f83af1ce740 10 client.0 _ll_register_callbacks cb 0x55631192da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f83a659c640 1 -- 172.21.15.62:0/2809434693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a45480 msgr2=0x556311a4cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f83a659c640 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a45480 0x556311a4cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f83a659c640 1 -- 172.21.15.62:0/2809434693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 msgr2=0x556311a4d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f83a659c640 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a4d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f83a659c640 1 -- 172.21.15.62:0/2809434693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55631171b6a0 con 0x556311a44b60 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f83a659c640 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a44b60 0x556311a4f7b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8398004ce0 tx=0x7f8398000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517a44219e36305b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f838dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f838dffb640 1 -- 172.21.15.62:0/2809434693 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398035050 con 0x556311a44b60 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8398004400 con 0x556311a44b60 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f838dffb640 1 -- 172.21.15.62:0/2809434693 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f8398004200 con 0x556311a44b60 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f838dffb640 1 -- 172.21.15.62:0/2809434693 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8398009520 con 0x556311a44b60 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f83af1ce740 10 client.25322 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556311a66c20 con 0x556311a44b60 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.341+0000 7f838dffb640 1 -- 172.21.15.62:0/2809434693 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8398004400 con 0x556311a44b60 2024-09-11T23:53:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.342+0000 7f838dffb640 1 -- 172.21.15.62:0/2809434693 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8398031b70 con 0x556311a44b60 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.342+0000 7f83af1ce740 10 client.25322 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.342+0000 7f83af1ce740 10 client.25322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.342+0000 7f83af1ce740 20 client.25322 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.342+0000 7f83af1ce740 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556311a67f70 0x556311a6a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.342+0000 7f83af1ce740 4 client.25322 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.342+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563119b77f0 con 0x556311a67f70 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.342+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556311a67f70 0x556311a6a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.342+0000 7f838dffb640 10 client.25322 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.342+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556311a67f70 0x556311a6a360 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8afd5089cda5cb6 server_cookie=c388fe4c9fbe00ac in_seq=0 out_seq=0 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.343+0000 7f838dffb640 1 -- 172.21.15.62:0/2809434693 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563119b77f0 con 0x556311a67f70 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.343+0000 7f838dffb640 10 client.25322 handle_command_reply: tid=0 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.343+0000 7f83af1ce740 1 client.25322 shutdown 2024-09-11T23:53:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.343+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a44b60 msgr2=0x556311a4f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.343+0000 7f83af1ce740 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a44b60 0x556311a4f7b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8398004ce0 tx=0x7f8398000bc0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.343+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 shutdown_connections 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.343+0000 7f83af1ce740 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556311a67f70 0x556311a6a360 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.344+0000 7f838dffb640 0 client.25322 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.344+0000 7f83af1ce740 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a4d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.344+0000 7f83af1ce740 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a45480 0x556311a4cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.344+0000 7f83af1ce740 1 --2- 172.21.15.62:0/2809434693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a44b60 0x556311a4f7b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.344+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 >> 172.21.15.62:0/2809434693 conn(0x556311a410f0 msgr2=0x556311a46dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.344+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 shutdown_connections 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.344+0000 7f83af1ce740 1 -- 172.21.15.62:0/2809434693 wait complete. 2024-09-11T23:53:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.344+0000 7f83af1ce740 20 client.25322 trim_cache size 0 max 16384 2024-09-11T23:53:42.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83af1ce740 1 Processor -- start 2024-09-11T23:53:42.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83af1ce740 1 -- start start 2024-09-11T23:53:42.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 0x556311a4b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a6d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83af1ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81adef0 con 0x556311a4fd40 2024-09-11T23:53:42.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83af1ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81ab7c0 con 0x556311a45480 2024-09-11T23:53:42.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83af1ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81a9610 con 0x556311a44b60 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83a659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83a659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a5bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:48000/0 (socket says 172.21.15.62:48000) 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83a659c640 1 -- 172.21.15.62:0/3095386142 learned_addr learned my addr 172.21.15.62:0/3095386142 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.370+0000 7f83a6d9d640 1 --2- 172.21.15.62:0/3095386142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a6d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.371+0000 7f83a659c640 1 -- 172.21.15.62:0/3095386142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 msgr2=0x556311a4b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.371+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/3095386142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 0x556311a4b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.371+0000 7f83a659c640 1 --2- 172.21.15.62:0/3095386142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 0x556311a4b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.371+0000 7f83a659c640 1 -- 172.21.15.62:0/3095386142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 msgr2=0x556311a6d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.371+0000 7f83a659c640 1 --2- 172.21.15.62:0/3095386142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a6d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.371+0000 7f83a659c640 1 -- 172.21.15.62:0/3095386142 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556311a40f20 con 0x556311a44b60 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.371+0000 7f83a6d9d640 1 --2- 172.21.15.62:0/3095386142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.371+0000 7f83a659c640 1 --2- 172.21.15.62:0/3095386142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a5bd90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8398031550 tx=0x7f83980038c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d237929c34199d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:42.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.371+0000 7f838ffff640 1 -- 172.21.15.62:0/3095386142 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398035030 con 0x556311a44b60 2024-09-11T23:53:42.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.372+0000 7f838ffff640 1 -- 172.21.15.62:0/3095386142 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8398004200 con 0x556311a44b60 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.372+0000 7f838ffff640 1 -- 172.21.15.62:0/3095386142 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980053b0 con 0x556311a44b60 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 -- 172.21.15.62:0/3095386142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 msgr2=0x556311a5bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 --2- 172.21.15.62:0/3095386142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a5bd90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8398031550 tx=0x7f83980038c0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 -- 172.21.15.62:0/3095386142 shutdown_connections 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 --2- 172.21.15.62:0/3095386142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a4fd40 0x556311a6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 --2- 172.21.15.62:0/3095386142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a45480 0x556311a4b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 --2- 172.21.15.62:0/3095386142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a5bd90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 -- 172.21.15.62:0/3095386142 >> 172.21.15.62:0/3095386142 conn(0x556311a410f0 msgr2=0x556311a4b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 -- 172.21.15.62:0/3095386142 shutdown_connections 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 -- 172.21.15.62:0/3095386142 wait complete. 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 -- start start 2024-09-11T23:53:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.373+0000 7f83af1ce740 1 Processor -- start 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a69d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a45480 0x556311a6a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83af1ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a4fd40 0x556311aca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83af1ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81a9c80 con 0x556311a45480 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83af1ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81ab0f0 con 0x556311a4fd40 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83af1ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81aa350 con 0x556311a44b60 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a45480 0x556311a6a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a69d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a69d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:48002/0 (socket says 172.21.15.62:48002) 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a659c640 1 -- 172.21.15.62:0/1739184817 learned_addr learned my addr 172.21.15.62:0/1739184817 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83af1ce740 10 client.0 _ll_register_callbacks cb 0x7f8398009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a6d9d640 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a4fd40 0x556311aca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a659c640 1 -- 172.21.15.62:0/1739184817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a4fd40 msgr2=0x556311aca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a659c640 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a4fd40 0x556311aca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a659c640 1 -- 172.21.15.62:0/1739184817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a45480 msgr2=0x556311a6a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a659c640 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a45480 0x556311a6a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a659c640 1 -- 172.21.15.62:0/1739184817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556311acb090 con 0x556311a44b60 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.374+0000 7f83a659c640 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a69d10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8398005ec0 tx=0x7f8398031480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9605a313c016c536 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.375+0000 7f838dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.375+0000 7f838dffb640 1 -- 172.21.15.62:0/1739184817 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398035040 con 0x556311a44b60 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.375+0000 7f838dffb640 1 -- 172.21.15.62:0/1739184817 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f8398003c70 con 0x556311a44b60 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.375+0000 7f83af1ce740 1 -- 172.21.15.62:0/1739184817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8398004200 con 0x556311a44b60 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.375+0000 7f838dffb640 1 -- 172.21.15.62:0/1739184817 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8398003e70 con 0x556311a44b60 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f83af1ce740 10 client.25306 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f83af1ce740 1 -- 172.21.15.62:0/1739184817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556311ad3400 con 0x556311a44b60 2024-09-11T23:53:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f838dffb640 1 -- 172.21.15.62:0/1739184817 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8398005510 con 0x556311a44b60 2024-09-11T23:53:42.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f838dffb640 1 -- 172.21.15.62:0/1739184817 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f839804e470 con 0x556311a44b60 2024-09-11T23:53:42.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f83af1ce740 10 client.25306 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:42.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f83af1ce740 10 client.25306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:53:42.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f83af1ce740 20 client.25306 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:42.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f83af1ce740 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556311a67f70 0x556311ad5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:42.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f83af1ce740 4 client.25306 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:53:42.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f83af1ce740 1 -- 172.21.15.62:0/1739184817 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f8398004200 con 0x556311a67f70 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.376+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556311a67f70 0x556311ad5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.377+0000 7f838dffb640 10 client.25306 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.377+0000 7f83a5d9b640 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556311a67f70 0x556311ad5a10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c8f02c20fc0c315 server_cookie=fa0029b35966baf6 in_seq=0 out_seq=0 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.377+0000 7f838dffb640 1 -- 172.21.15.62:0/1739184817 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f8398004200 con 0x556311a67f70 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.377+0000 7f838dffb640 10 client.25306 handle_command_reply: tid=0 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.377+0000 7f83af1ce740 1 client.25306 shutdown 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 1 -- 172.21.15.62:0/1739184817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 msgr2=0x556311a69d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a69d10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8398005ec0 tx=0x7f8398031480 comp rx=0 tx=0).stop 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 1 -- 172.21.15.62:0/1739184817 shutdown_connections 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556311a67f70 0x556311ad5a10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556311a4fd40 0x556311aca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556311a45480 0x556311a6a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 1 --2- 172.21.15.62:0/1739184817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556311a44b60 0x556311a69d10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f838dffb640 0 client.25306 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:42.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 1 -- 172.21.15.62:0/1739184817 >> 172.21.15.62:0/1739184817 conn(0x556311a410f0 msgr2=0x556311a40270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:42.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 1 -- 172.21.15.62:0/1739184817 shutdown_connections 2024-09-11T23:53:42.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 1 -- 172.21.15.62:0/1739184817 wait complete. 2024-09-11T23:53:42.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.378+0000 7f83af1ce740 20 client.25306 trim_cache size 0 max 16384 2024-09-11T23:53:42.381 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:53:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:53:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "fd639421-75f2-45f7-97f0-db979134cf56", 2024-09-11T23:53:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:53:42.381 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:53:42.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 -- 172.21.15.62:0/539422762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83900a2d70 msgr2=0x7f83900a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:42.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83900a2d70 0x7f83900a5230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005fd0 tx=0x7f83a0009500 comp rx=0 tx=0).stop 2024-09-11T23:53:42.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 -- 172.21.15.62:0/539422762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81049b0 msgr2=0x7f83a819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:42.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81049b0 0x7f83a819abe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f839c00e9f0 tx=0x7f839c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 -- 172.21.15.62:0/539422762 shutdown_connections 2024-09-11T23:53:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83900a2d70 0x7f83900a5230 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83a8109130 0x7f83a819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a81049b0 0x7f83a819abe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 --2- 172.21.15.62:0/539422762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a8104000 0x7f83a819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 -- 172.21.15.62:0/539422762 >> 172.21.15.62:0/539422762 conn(0x7f83a80ffa20 msgr2=0x7f83a8100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 -- 172.21.15.62:0/539422762 shutdown_connections 2024-09-11T23:53:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:42.379+0000 7f83acdf5640 1 -- 172.21.15.62:0/539422762 wait complete. 2024-09-11T23:53:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 60.0 s, 3641.6 tps, lat 27.354 ms stddev 27.641 2024-09-11T23:53:43.232 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:53:43.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:43 smithi120 ceph-mon[35182]: pgmap v51: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.8 MiB/s wr, 779 op/s 2024-09-11T23:53:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:43 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:53:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:43 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:53:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:43 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:53:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:43 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:53:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:43 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:53:43.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:43 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:53:43.624 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:53:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:43 smithi083 ceph-mon[34805]: pgmap v51: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.8 MiB/s wr, 779 op/s 2024-09-11T23:53:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:43 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:53:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:43 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:53:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:43 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:53:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:43 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:53:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:43 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:53:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:43 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:53:43.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:43 smithi062 ceph-mon[30958]: pgmap v51: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.8 MiB/s wr, 779 op/s 2024-09-11T23:53:43.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:43 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:53:43.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:43 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:53:43.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:43 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:53:43.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:43 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:53:43.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:43 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:53:43.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:43 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:53:44.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.159+0000 7f072f9d1640 1 -- 172.21.15.62:0/2775563274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0728101360 msgr2=0x7f072810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:44.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.159+0000 7f072f9d1640 1 --2- 172.21.15.62:0/2775563274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0728101360 0x7f072810df90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0724009900 tx=0x7f072402a410 comp rx=0 tx=0).stop 2024-09-11T23:53:44.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.159+0000 7f072f9d1640 1 -- 172.21.15.62:0/2775563274 shutdown_connections 2024-09-11T23:53:44.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.159+0000 7f072f9d1640 1 --2- 172.21.15.62:0/2775563274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072810e4d0 0x7f07281108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.159+0000 7f072f9d1640 1 --2- 172.21.15.62:0/2775563274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0728101360 0x7f072810df90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.159+0000 7f072f9d1640 1 --2- 172.21.15.62:0/2775563274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07281009b0 0x7f0728100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.159+0000 7f072f9d1640 1 -- 172.21.15.62:0/2775563274 >> 172.21.15.62:0/2775563274 conn(0x7f07280fc7d0 msgr2=0x7f07280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:44.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.159+0000 7f072f9d1640 1 -- 172.21.15.62:0/2775563274 shutdown_connections 2024-09-11T23:53:44.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.159+0000 7f072f9d1640 1 -- 172.21.15.62:0/2775563274 wait complete. 2024-09-11T23:53:44.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.160+0000 7f072f9d1640 1 Processor -- start 2024-09-11T23:53:44.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.160+0000 7f072f9d1640 1 -- start start 2024-09-11T23:53:44.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.161+0000 7f072f9d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07281009b0 0x7f07281a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.161+0000 7f072f9d1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0728101360 0x7f07281a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.161+0000 7f072f9d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f072810e4d0 0x7f072819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.161+0000 7f072f9d1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0728114250 con 0x7f0728101360 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.161+0000 7f072f9d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07281140d0 con 0x7f07281009b0 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.161+0000 7f072f9d1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07281143d0 con 0x7f072810e4d0 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.162+0000 7f072cf6c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0728101360 0x7f07281a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.162+0000 7f072cf6c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0728101360 0x7f07281a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37436/0 (socket says 172.21.15.62:37436) 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.162+0000 7f072cf6c640 1 -- 172.21.15.62:0/3032806649 learned_addr learned my addr 172.21.15.62:0/3032806649 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.162+0000 7f072d76d640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07281009b0 0x7f07281a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.162+0000 7f072d76d640 1 -- 172.21.15.62:0/3032806649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f072810e4d0 msgr2=0x7f072819a5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.162+0000 7f072d76d640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f072810e4d0 0x7f072819a5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.162+0000 7f072d76d640 1 -- 172.21.15.62:0/3032806649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0728101360 msgr2=0x7f07281a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.162+0000 7f072d76d640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0728101360 0x7f07281a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.162+0000 7f072d76d640 1 -- 172.21.15.62:0/3032806649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072819acd0 con 0x7f07281009b0 2024-09-11T23:53:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.163+0000 7f072d76d640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07281009b0 0x7f07281a03e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f071c00e9f0 tx=0x7f071c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.163+0000 7f071a7fc640 1 -- 172.21.15.62:0/3032806649 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c00ccc0 con 0x7f07281009b0 2024-09-11T23:53:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.164+0000 7f071a7fc640 1 -- 172.21.15.62:0/3032806649 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071c010910 con 0x7f07281009b0 2024-09-11T23:53:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.164+0000 7f071a7fc640 1 -- 172.21.15.62:0/3032806649 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c0188e0 con 0x7f07281009b0 2024-09-11T23:53:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.164+0000 7f072f9d1640 1 -- 172.21.15.62:0/3032806649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072819afc0 con 0x7f07281009b0 2024-09-11T23:53:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.164+0000 7f072f9d1640 1 -- 172.21.15.62:0/3032806649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07281a72b0 con 0x7f07281009b0 2024-09-11T23:53:44.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.167+0000 7f071a7fc640 1 -- 172.21.15.62:0/3032806649 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f071c010ac0 con 0x7f07281009b0 2024-09-11T23:53:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.167+0000 7f071a7fc640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f07140a2e40 0x7f07140a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.168+0000 7f072cf6c640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f07140a2e40 0x7f07140a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.168+0000 7f071a7fc640 1 -- 172.21.15.62:0/3032806649 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f071c0cf850 con 0x7f07281009b0 2024-09-11T23:53:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.168+0000 7f0731daa740 1 Processor -- start 2024-09-11T23:53:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.168+0000 7f0731daa740 1 -- start start 2024-09-11T23:53:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.168+0000 7f0731daa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e82370880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.168+0000 7f0731daa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e8237b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.168+0000 7f0731daa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 0x562e8237da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f0731daa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562e822d62e0 con 0x562e823704a0 2024-09-11T23:53:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f0731daa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562e8227dce0 con 0x562e8237b680 2024-09-11T23:53:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f0731daa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562e822d22a0 con 0x562e82370dc0 2024-09-11T23:53:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f072df6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 0x562e8237da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f072df6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 0x562e8237da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:46258/0 (socket says 172.21.15.62:46258) 2024-09-11T23:53:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f072df6e640 1 -- 172.21.15.62:0/1764985425 learned_addr learned my addr 172.21.15.62:0/1764985425 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f072cf6c640 1 --2- 172.21.15.62:0/1764985425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e8237b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f072cf6c640 1 -- 172.21.15.62:0/1764985425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 msgr2=0x562e8237da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f072cf6c640 1 --2- 172.21.15.62:0/1764985425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 0x562e8237da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f072cf6c640 1 -- 172.21.15.62:0/1764985425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 msgr2=0x562e82370880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.169+0000 7f072cf6c640 1 --2- 172.21.15.62:0/1764985425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e82370880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.170+0000 7f072cf6c640 1 -- 172.21.15.62:0/1764985425 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e82262440 con 0x562e82370dc0 2024-09-11T23:53:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.170+0000 7f072cf6c640 1 --2- 172.21.15.62:0/1764985425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e8237b140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0724000c00 tx=0x7f0724046000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d24e2eb2cc82810 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.170+0000 7f072cf6c640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f07140a2e40 0x7f07140a5300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0728102340 tx=0x7f0724004350 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.170+0000 7f07137fe640 1 -- 172.21.15.62:0/1764985425 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724042070 con 0x562e82370dc0 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.170+0000 7f07137fe640 1 -- 172.21.15.62:0/1764985425 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0724003d00 con 0x562e82370dc0 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.170+0000 7f07137fe640 1 -- 172.21.15.62:0/1764985425 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724035040 con 0x562e82370dc0 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.171+0000 7f0731daa740 1 -- 172.21.15.62:0/1764985425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 msgr2=0x562e8237b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.171+0000 7f0731daa740 1 --2- 172.21.15.62:0/1764985425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e8237b140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0724000c00 tx=0x7f0724046000 comp rx=0 tx=0).stop 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.171+0000 7f0731daa740 1 -- 172.21.15.62:0/1764985425 shutdown_connections 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.171+0000 7f0731daa740 1 --2- 172.21.15.62:0/1764985425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 0x562e8237da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.171+0000 7f0731daa740 1 --2- 172.21.15.62:0/1764985425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e8237b140 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.171+0000 7f0731daa740 1 --2- 172.21.15.62:0/1764985425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e82370880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.171+0000 7f0731daa740 1 -- 172.21.15.62:0/1764985425 >> 172.21.15.62:0/1764985425 conn(0x562e8236ca30 msgr2=0x562e8236ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.171+0000 7f0731daa740 1 -- 172.21.15.62:0/1764985425 shutdown_connections 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.171+0000 7f0731daa740 1 -- 172.21.15.62:0/1764985425 wait complete. 2024-09-11T23:53:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.171+0000 7f0731daa740 1 -- start start 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f0731daa740 1 Processor -- start 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f0731daa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e8237b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f0731daa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e82370dc0 0x562e823782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f0731daa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e8237b680 0x562e82378830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f0731daa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562e8237e2f0 con 0x562e823704a0 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f0731daa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562e82291050 con 0x562e82370dc0 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f0731daa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562e8237e470 con 0x562e8237b680 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f0731daa740 10 client.0 _ll_register_callbacks cb 0x562e822d1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f072d76d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e8237b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f072d76d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e8237b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37456/0 (socket says 172.21.15.62:37456) 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f072d76d640 1 -- 172.21.15.62:0/414530316 learned_addr learned my addr 172.21.15.62:0/414530316 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f072cf6c640 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e82370dc0 0x562e823782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.172+0000 7f072df6e640 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e8237b680 0x562e82378830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.173+0000 7f072cf6c640 1 -- 172.21.15.62:0/414530316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e8237b680 msgr2=0x562e82378830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.173+0000 7f072cf6c640 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e8237b680 0x562e82378830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.173+0000 7f072cf6c640 1 -- 172.21.15.62:0/414530316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 msgr2=0x562e8237b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.173+0000 7f072cf6c640 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e8237b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.173+0000 7f072cf6c640 1 -- 172.21.15.62:0/414530316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e820473c0 con 0x562e82370dc0 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.173+0000 7f072cf6c640 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e82370dc0 0x562e823782b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0724009850 tx=0x7f0724009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=447398cef09d65a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.173+0000 7f0710ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.173+0000 7f0710ff9640 1 -- 172.21.15.62:0/414530316 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724042040 con 0x562e82370dc0 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.173+0000 7f0710ff9640 1 -- 172.21.15.62:0/414530316 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f07240317c0 con 0x562e82370dc0 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.173+0000 7f0731daa740 1 -- 172.21.15.62:0/414530316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0724003d00 con 0x562e82370dc0 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.174+0000 7f0710ff9640 1 -- 172.21.15.62:0/414530316 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0724061050 con 0x562e82370dc0 2024-09-11T23:53:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.175+0000 7f0731daa740 10 client.25337 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.175+0000 7f0731daa740 1 -- 172.21.15.62:0/414530316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e8237e2f0 con 0x562e82370dc0 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.175+0000 7f0710ff9640 1 -- 172.21.15.62:0/414530316 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0724058070 con 0x562e82370dc0 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.175+0000 7f0710ff9640 1 -- 172.21.15.62:0/414530316 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f07240535e0 con 0x562e82370dc0 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.175+0000 7f0731daa740 10 client.25337 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.175+0000 7f0731daa740 10 client.25337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.175+0000 7f0731daa740 20 client.25337 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.175+0000 7f0731daa740 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562e8238b1c0 0x562e8238d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.175+0000 7f0731daa740 4 client.25337 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.175+0000 7f0731daa740 1 -- 172.21.15.62:0/414530316 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0724003d00 con 0x562e8238b1c0 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.176+0000 7f072d76d640 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562e8238b1c0 0x562e8238d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.176+0000 7f0710ff9640 10 client.25337 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.176+0000 7f072d76d640 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562e8238b1c0 0x562e8238d5e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de796ef072e297b6 server_cookie=b7829f91e36c9fa1 in_seq=0 out_seq=0 2024-09-11T23:53:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0710ff9640 1 -- 172.21.15.62:0/414530316 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0724003d00 con 0x562e8238b1c0 2024-09-11T23:53:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0710ff9640 10 client.25337 handle_command_reply: tid=0 2024-09-11T23:53:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0731daa740 1 client.25337 shutdown 2024-09-11T23:53:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0731daa740 1 -- 172.21.15.62:0/414530316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e82370dc0 msgr2=0x562e823782b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0731daa740 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e82370dc0 0x562e823782b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0724009850 tx=0x7f0724009260 comp rx=0 tx=0).stop 2024-09-11T23:53:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0731daa740 1 -- 172.21.15.62:0/414530316 shutdown_connections 2024-09-11T23:53:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0731daa740 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562e8238b1c0 0x562e8238d5e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0731daa740 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e8237b680 0x562e82378830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0710ff9640 0 client.25337 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0731daa740 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e82370dc0 0x562e823782b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.177+0000 7f0731daa740 1 --2- 172.21.15.62:0/414530316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e8237b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.178+0000 7f0731daa740 1 -- 172.21.15.62:0/414530316 >> 172.21.15.62:0/414530316 conn(0x562e8236ca30 msgr2=0x562e82372710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.178+0000 7f0731daa740 1 -- 172.21.15.62:0/414530316 shutdown_connections 2024-09-11T23:53:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.178+0000 7f0731daa740 1 -- 172.21.15.62:0/414530316 wait complete. 2024-09-11T23:53:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.178+0000 7f0731daa740 20 client.25337 trim_cache size 0 max 16384 2024-09-11T23:53:44.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.204+0000 7f0731daa740 1 Processor -- start 2024-09-11T23:53:44.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.204+0000 7f0731daa740 1 -- start start 2024-09-11T23:53:44.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.205+0000 7f0731daa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e823704a0 0x562e82375510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.205+0000 7f0731daa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e82375a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.205+0000 7f0731daa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e8237b680 0x562e82379b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.205+0000 7f0731daa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07281b24a0 con 0x562e8237b680 2024-09-11T23:53:44.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.205+0000 7f0731daa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07281afd70 con 0x562e823704a0 2024-09-11T23:53:44.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.205+0000 7f072df6e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e8237b680 0x562e82379b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.205+0000 7f072df6e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e8237b680 0x562e82379b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37462/0 (socket says 172.21.15.62:37462) 2024-09-11T23:53:44.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.205+0000 7f072df6e640 1 -- 172.21.15.62:0/2631061780 learned_addr learned my addr 172.21.15.62:0/2631061780 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:44.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.205+0000 7f0731daa740 1 -- 172.21.15.62:0/2631061780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07281adbc0 con 0x562e82370dc0 2024-09-11T23:53:44.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.205+0000 7f072d76d640 1 --2- 172.21.15.62:0/2631061780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e823704a0 0x562e82375510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.206+0000 7f072d76d640 1 -- 172.21.15.62:0/2631061780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 msgr2=0x562e82375a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:44.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.206+0000 7f072d76d640 1 --2- 172.21.15.62:0/2631061780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e82375a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.206+0000 7f072d76d640 1 -- 172.21.15.62:0/2631061780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e8237b680 msgr2=0x562e82379b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:44.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.206+0000 7f072d76d640 1 --2- 172.21.15.62:0/2631061780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e8237b680 0x562e82379b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.206+0000 7f072d76d640 1 -- 172.21.15.62:0/2631061780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e82376800 con 0x562e823704a0 2024-09-11T23:53:44.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.206+0000 7f072d76d640 1 --2- 172.21.15.62:0/2631061780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e823704a0 0x562e82375510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f071c0981a0 tx=0x7f071c009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd338fea13c68b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:44.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.206+0000 7f0712ffd640 1 -- 172.21.15.62:0/2631061780 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c0b9070 con 0x562e823704a0 2024-09-11T23:53:44.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.206+0000 7f0712ffd640 1 -- 172.21.15.62:0/2631061780 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071c0a49b0 con 0x562e823704a0 2024-09-11T23:53:44.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.206+0000 7f0712ffd640 1 -- 172.21.15.62:0/2631061780 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c09b3f0 con 0x562e823704a0 2024-09-11T23:53:44.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.208+0000 7f0731daa740 1 -- 172.21.15.62:0/2631061780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e823704a0 msgr2=0x562e82375510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:44.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.208+0000 7f0731daa740 1 --2- 172.21.15.62:0/2631061780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e823704a0 0x562e82375510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f071c0981a0 tx=0x7f071c009700 comp rx=0 tx=0).stop 2024-09-11T23:53:44.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.208+0000 7f0731daa740 1 -- 172.21.15.62:0/2631061780 shutdown_connections 2024-09-11T23:53:44.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.208+0000 7f0731daa740 1 --2- 172.21.15.62:0/2631061780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e8237b680 0x562e82379b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.208+0000 7f0731daa740 1 --2- 172.21.15.62:0/2631061780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e82375a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.208+0000 7f0731daa740 1 --2- 172.21.15.62:0/2631061780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e823704a0 0x562e82375510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.208+0000 7f0731daa740 1 -- 172.21.15.62:0/2631061780 >> 172.21.15.62:0/2631061780 conn(0x562e8236ca30 msgr2=0x562e823773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.208+0000 7f0731daa740 1 -- 172.21.15.62:0/2631061780 shutdown_connections 2024-09-11T23:53:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.208+0000 7f0731daa740 1 -- 172.21.15.62:0/2631061780 wait complete. 2024-09-11T23:53:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.209+0000 7f0731daa740 1 -- start start 2024-09-11T23:53:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.209+0000 7f0731daa740 1 Processor -- start 2024-09-11T23:53:44.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.209+0000 7f0731daa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e823925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.209+0000 7f0731daa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e82392b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.209+0000 7f0731daa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 0x562e823ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.209+0000 7f0731daa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07281ae230 con 0x562e823704a0 2024-09-11T23:53:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.209+0000 7f0731daa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07281af6a0 con 0x562e8237b680 2024-09-11T23:53:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.209+0000 7f0731daa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07281ae900 con 0x562e82370dc0 2024-09-11T23:53:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.209+0000 7f0731daa740 10 client.0 _ll_register_callbacks cb 0x7f07200098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:53:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072cf6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e82392b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072d76d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e823925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072cf6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e82392b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:48040/0 (socket says 172.21.15.62:48040) 2024-09-11T23:53:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072cf6c640 1 -- 172.21.15.62:0/1032353351 learned_addr learned my addr 172.21.15.62:0/1032353351 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:53:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072df6e640 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 0x562e823ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072cf6c640 1 -- 172.21.15.62:0/1032353351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 msgr2=0x562e823ecd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072cf6c640 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 0x562e823ecd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072cf6c640 1 -- 172.21.15.62:0/1032353351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 msgr2=0x562e823925d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:53:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072cf6c640 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e823925d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072cf6c640 1 -- 172.21.15.62:0/1032353351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e823ed3a0 con 0x562e82370dc0 2024-09-11T23:53:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072df6e640 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 0x562e823ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:53:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072d76d640 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e823925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:53:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f072cf6c640 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e82392b10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f072403ec80 tx=0x7f07240035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=127e1f31626bad6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:53:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.210+0000 7f0710ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:53:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.211+0000 7f0710ff9640 1 -- 172.21.15.62:0/1032353351 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724035040 con 0x562e82370dc0 2024-09-11T23:53:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.211+0000 7f0710ff9640 1 -- 172.21.15.62:0/1032353351 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f0724003d00 con 0x562e82370dc0 2024-09-11T23:53:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.211+0000 7f0731daa740 1 -- 172.21.15.62:0/1032353351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f071c0a49b0 con 0x562e82370dc0 2024-09-11T23:53:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.212+0000 7f0710ff9640 1 -- 172.21.15.62:0/1032353351 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0724053440 con 0x562e82370dc0 2024-09-11T23:53:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.212+0000 7f0731daa740 10 client.25318 fetch_fsmap learned FSMap version 27 2024-09-11T23:53:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.212+0000 7f0731daa740 1 -- 172.21.15.62:0/1032353351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e823f5940 con 0x562e82370dc0 2024-09-11T23:53:44.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.212+0000 7f0710ff9640 1 -- 172.21.15.62:0/1032353351 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0724058040 con 0x562e82370dc0 2024-09-11T23:53:44.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.212+0000 7f0710ff9640 1 -- 172.21.15.62:0/1032353351 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f072405dab0 con 0x562e82370dc0 2024-09-11T23:53:44.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.212+0000 7f0731daa740 10 client.25318 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.212+0000 7f0731daa740 10 client.25318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.212+0000 7f0731daa740 20 client.25318 populate_metadata read hostname 'smithi062' 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.212+0000 7f0731daa740 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562e8238b1c0 0x562e823f8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.213+0000 7f0731daa740 4 client.25318 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.213+0000 7f0731daa740 1 -- 172.21.15.62:0/1032353351 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f071c0a49b0 con 0x562e8238b1c0 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.213+0000 7f072d76d640 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562e8238b1c0 0x562e823f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.214+0000 7f072d76d640 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562e8238b1c0 0x562e823f8070 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=684fdff906b4dc38 server_cookie=95e3f007f7b9df9b in_seq=0 out_seq=0 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.214+0000 7f0710ff9640 10 client.25318 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.214+0000 7f0710ff9640 1 -- 172.21.15.62:0/1032353351 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f071c0a49b0 con 0x562e8238b1c0 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.214+0000 7f0710ff9640 10 client.25318 handle_command_reply: tid=0 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.214+0000 7f0731daa740 1 client.25318 shutdown 2024-09-11T23:53:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 1 -- 172.21.15.62:0/1032353351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 msgr2=0x562e82392b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e82392b10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f072403ec80 tx=0x7f07240035f0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 1 -- 172.21.15.62:0/1032353351 shutdown_connections 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562e8238b1c0 0x562e823f8070 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e8237b680 0x562e823ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e82370dc0 0x562e82392b10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 1 --2- 172.21.15.62:0/1032353351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e823704a0 0x562e823925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 1 -- 172.21.15.62:0/1032353351 >> 172.21.15.62:0/1032353351 conn(0x562e8236ca30 msgr2=0x562e82379b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0710ff9640 0 client.25318 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 1 -- 172.21.15.62:0/1032353351 shutdown_connections 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 1 -- 172.21.15.62:0/1032353351 wait complete. 2024-09-11T23:53:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.215+0000 7f0731daa740 20 client.25318 trim_cache size 0 max 16384 2024-09-11T23:53:44.218 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:53:44.218 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:53:44.218 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:53:44.218 INFO:teuthology.orchestra.run.smithi062.stdout: "fd639421-75f2-45f7-97f0-db979134cf56": { 2024-09-11T23:53:44.218 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:53:44.218 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "fd639421-75f2-45f7-97f0-db979134cf56", 2024-09-11T23:53:44.219 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:53:44.219 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:53:44.220 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:53:44.220 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.216+0000 7f072f9d1640 1 -- 172.21.15.62:0/3032806649 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f07140a2e40 msgr2=0x7f07140a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.216+0000 7f072f9d1640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f07140a2e40 0x7f07140a5300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0728102340 tx=0x7f0724004350 comp rx=0 tx=0).stop 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.216+0000 7f072f9d1640 1 -- 172.21.15.62:0/3032806649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07281009b0 msgr2=0x7f07281a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.216+0000 7f072f9d1640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07281009b0 0x7f07281a03e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f071c00e9f0 tx=0x7f071c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.217+0000 7f072f9d1640 1 -- 172.21.15.62:0/3032806649 shutdown_connections 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.217+0000 7f072f9d1640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f07140a2e40 0x7f07140a5300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.217+0000 7f072f9d1640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f072810e4d0 0x7f072819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.217+0000 7f072f9d1640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0728101360 0x7f07281a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.217+0000 7f072f9d1640 1 --2- 172.21.15.62:0/3032806649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07281009b0 0x7f07281a03e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.217+0000 7f072f9d1640 1 -- 172.21.15.62:0/3032806649 >> 172.21.15.62:0/3032806649 conn(0x7f07280fc7d0 msgr2=0x7f072810f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.217+0000 7f072f9d1640 1 -- 172.21.15.62:0/3032806649 shutdown_connections 2024-09-11T23:53:44.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:53:44.217+0000 7f072f9d1640 1 -- 172.21.15.62:0/3032806649 wait complete. 2024-09-11T23:53:44.954 INFO:tasks.cephfs.filesystem:scrub status for tag:fd639421-75f2-45f7-97f0-db979134cf56 - {'path': '/', 'tag': 'fd639421-75f2-45f7-97f0-db979134cf56', 'options': 'recursive,force'} 2024-09-11T23:53:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:45 smithi120 ceph-mon[35182]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 12 MiB/s wr, 935 op/s 2024-09-11T23:53:45.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:45 smithi083 ceph-mon[34805]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 12 MiB/s wr, 935 op/s 2024-09-11T23:53:45.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:45 smithi062 ceph-mon[30958]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 12 MiB/s wr, 935 op/s 2024-09-11T23:53:46.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:46 smithi120 ceph-mon[35182]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.7 MiB/s wr, 668 op/s 2024-09-11T23:53:46.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:46 smithi083 ceph-mon[34805]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.7 MiB/s wr, 668 op/s 2024-09-11T23:53:46.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:46 smithi062 ceph-mon[30958]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.7 MiB/s wr, 668 op/s 2024-09-11T23:53:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 65.0 s, 3543.6 tps, lat 28.278 ms stddev 28.555 2024-09-11T23:53:48.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:48 smithi120 ceph-mon[35182]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-11T23:53:48.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:48 smithi083 ceph-mon[34805]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-11T23:53:48.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:48 smithi062 ceph-mon[30958]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-11T23:53:50.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:50 smithi120 ceph-mon[35182]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 9.7 MiB/s wr, 688 op/s 2024-09-11T23:53:50.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:50 smithi083 ceph-mon[34805]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 9.7 MiB/s wr, 688 op/s 2024-09-11T23:53:50.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:50 smithi062 ceph-mon[30958]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 9.7 MiB/s wr, 688 op/s 2024-09-11T23:53:51.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:51 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:53:51.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:51 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:53:51.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:51 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:53:51.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:51 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:53:51.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:51 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:53:51.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:51 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:53:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 70.0 s, 3399.2 tps, lat 29.452 ms stddev 42.470 2024-09-11T23:53:52.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:52 smithi120 ceph-mon[35182]: pgmap v56: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 23 MiB/s wr, 1.13k op/s 2024-09-11T23:53:52.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:52 smithi083 ceph-mon[34805]: pgmap v56: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 23 MiB/s wr, 1.13k op/s 2024-09-11T23:53:52.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:52 smithi062 ceph-mon[30958]: pgmap v56: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 23 MiB/s wr, 1.13k op/s 2024-09-11T23:53:54.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:54 smithi120 ceph-mon[35182]: pgmap v57: 97 pgs: 97 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 39 MiB/s wr, 1.62k op/s 2024-09-11T23:53:54.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:54 smithi083 ceph-mon[34805]: pgmap v57: 97 pgs: 97 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 39 MiB/s wr, 1.62k op/s 2024-09-11T23:53:54.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:54 smithi062 ceph-mon[30958]: pgmap v57: 97 pgs: 97 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 39 MiB/s wr, 1.62k op/s 2024-09-11T23:53:56.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:56 smithi120 ceph-mon[35182]: pgmap v58: 97 pgs: 97 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 36 MiB/s wr, 1.41k op/s 2024-09-11T23:53:56.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:56 smithi083 ceph-mon[34805]: pgmap v58: 97 pgs: 97 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 36 MiB/s wr, 1.41k op/s 2024-09-11T23:53:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:56 smithi062 ceph-mon[30958]: pgmap v58: 97 pgs: 97 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 36 MiB/s wr, 1.41k op/s 2024-09-11T23:53:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 75.0 s, 3587.4 tps, lat 27.879 ms stddev 28.939 2024-09-11T23:53:58.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:53:58 smithi120 ceph-mon[35182]: pgmap v59: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 43 MiB/s wr, 1.83k op/s 2024-09-11T23:53:58.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:53:58 smithi083 ceph-mon[34805]: pgmap v59: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 43 MiB/s wr, 1.83k op/s 2024-09-11T23:53:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:53:58 smithi062 ceph-mon[30958]: pgmap v59: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 43 MiB/s wr, 1.83k op/s 2024-09-11T23:54:00.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:00 smithi120 ceph-mon[35182]: pgmap v60: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 39 MiB/s wr, 1.49k op/s 2024-09-11T23:54:00.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:00 smithi083 ceph-mon[34805]: pgmap v60: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 39 MiB/s wr, 1.49k op/s 2024-09-11T23:54:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:00 smithi062 ceph-mon[30958]: pgmap v60: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 39 MiB/s wr, 1.49k op/s 2024-09-11T23:54:02.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 80.0 s, 2633.4 tps, lat 37.274 ms stddev 40.448 2024-09-11T23:54:02.560 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-11T23:54:02.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:54:02.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:02 smithi120 ceph-mon[35182]: pgmap v61: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 MiB/s wr, 1.66k op/s 2024-09-11T23:54:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:02 smithi083 ceph-mon[34805]: pgmap v61: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 MiB/s wr, 1.66k op/s 2024-09-11T23:54:02.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:02 smithi062 ceph-mon[30958]: pgmap v61: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 MiB/s wr, 1.66k op/s 2024-09-11T23:54:03.023 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:03.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.599+0000 7fe6975f6640 1 -- 172.21.15.62:0/1559344413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690105ec0 msgr2=0x7fe690106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:03.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.599+0000 7fe6975f6640 1 --2- 172.21.15.62:0/1559344413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690105ec0 0x7fe690106340 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe688009ca0 tx=0x7fe68802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.605+0000 7fe6975f6640 1 -- 172.21.15.62:0/1559344413 shutdown_connections 2024-09-11T23:54:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.605+0000 7fe6975f6640 1 --2- 172.21.15.62:0/1559344413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690105ec0 0x7fe690106340 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.605+0000 7fe6975f6640 1 --2- 172.21.15.62:0/1559344413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6901011e0 0x7fe690105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.605+0000 7fe6975f6640 1 --2- 172.21.15.62:0/1559344413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6901008c0 0x7fe690100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.605+0000 7fe6975f6640 1 -- 172.21.15.62:0/1559344413 >> 172.21.15.62:0/1559344413 conn(0x7fe6900fc7d0 msgr2=0x7fe6900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.605+0000 7fe6975f6640 1 -- 172.21.15.62:0/1559344413 shutdown_connections 2024-09-11T23:54:03.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.605+0000 7fe6975f6640 1 -- 172.21.15.62:0/1559344413 wait complete. 2024-09-11T23:54:03.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.606+0000 7fe6975f6640 1 Processor -- start 2024-09-11T23:54:03.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.606+0000 7fe6975f6640 1 -- start start 2024-09-11T23:54:03.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.606+0000 7fe6975f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6901008c0 0x7fe69010efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:03.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.606+0000 7fe6975f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6901011e0 0x7fe690106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:03.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.606+0000 7fe6975f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690105ec0 0x7fe6901074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:03.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.606+0000 7fe6975f6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe690118750 con 0x7fe690105ec0 2024-09-11T23:54:03.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.606+0000 7fe6975f6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901185d0 con 0x7fe6901011e0 2024-09-11T23:54:03.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.606+0000 7fe6975f6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901188d0 con 0x7fe6901008c0 2024-09-11T23:54:03.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.607+0000 7fe694b91640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6901011e0 0x7fe690106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:03.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.607+0000 7fe694b91640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6901011e0 0x7fe690106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:45076/0 (socket says 172.21.15.62:45076) 2024-09-11T23:54:03.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.607+0000 7fe694b91640 1 -- 172.21.15.62:0/2858459643 learned_addr learned my addr 172.21.15.62:0/2858459643 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:03.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.607+0000 7fe695b93640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690105ec0 0x7fe6901074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.608+0000 7fe695b93640 1 -- 172.21.15.62:0/2858459643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6901008c0 msgr2=0x7fe69010efe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:54:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.608+0000 7fe695b93640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6901008c0 0x7fe69010efe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.608+0000 7fe695b93640 1 -- 172.21.15.62:0/2858459643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6901011e0 msgr2=0x7fe690106fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.608+0000 7fe695b93640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6901011e0 0x7fe690106fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.608+0000 7fe695b93640 1 -- 172.21.15.62:0/2858459643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe690107d10 con 0x7fe690105ec0 2024-09-11T23:54:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.608+0000 7fe695b93640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690105ec0 0x7fe6901074f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe68802ae60 tx=0x7fe688031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.609+0000 7fe6867fc640 1 -- 172.21.15.62:0/2858459643 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688004040 con 0x7fe690105ec0 2024-09-11T23:54:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.609+0000 7fe6867fc640 1 -- 172.21.15.62:0/2858459643 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68802d070 con 0x7fe690105ec0 2024-09-11T23:54:03.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.609+0000 7fe6867fc640 1 -- 172.21.15.62:0/2858459643 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68802fce0 con 0x7fe690105ec0 2024-09-11T23:54:03.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.609+0000 7fe6975f6640 1 -- 172.21.15.62:0/2858459643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6901b3820 con 0x7fe690105ec0 2024-09-11T23:54:03.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.609+0000 7fe6975f6640 1 -- 172.21.15.62:0/2858459643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6901b3ce0 con 0x7fe690105ec0 2024-09-11T23:54:03.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.620+0000 7fe6867fc640 1 -- 172.21.15.62:0/2858459643 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe6880041e0 con 0x7fe690105ec0 2024-09-11T23:54:03.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.621+0000 7fe6867fc640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe67c0a2e80 0x7fe67c0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:03.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.621+0000 7fe6975f6640 1 -- 172.21.15.62:0/2858459643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69006a850 con 0x7fe690105ec0 2024-09-11T23:54:03.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.622+0000 7fe6867fc640 1 -- 172.21.15.62:0/2858459643 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe6880f0b30 con 0x7fe690105ec0 2024-09-11T23:54:03.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.622+0000 7fe695392640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe67c0a2e80 0x7fe67c0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:03.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.624+0000 7fe695392640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe67c0a2e80 0x7fe67c0a5340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe680009670 tx=0x7fe680006bd0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:03.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.631+0000 7fe6867fc640 1 -- 172.21.15.62:0/2858459643 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6880bd830 con 0x7fe690105ec0 2024-09-11T23:54:03.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.975+0000 7fe6975f6640 1 -- 172.21.15.62:0/2858459643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6901b4000 con 0x7fe690105ec0 2024-09-11T23:54:03.977 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:54:03.977 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-09-11T23:52:56:186189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:55.695276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:54:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-09-11T23:54:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.976+0000 7fe6867fc640 1 -- 172.21.15.62:0/2858459643 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11912 (secure 0 0 0) 0x7fe6880bd130 con 0x7fe690105ec0 2024-09-11T23:54:03.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.980+0000 7fe6975f6640 1 -- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe67c0a2e80 msgr2=0x7fe67c0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:03.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.980+0000 7fe6975f6640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe67c0a2e80 0x7fe67c0a5340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe680009670 tx=0x7fe680006bd0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.980+0000 7fe6975f6640 1 -- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690105ec0 msgr2=0x7fe6901074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:03.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.980+0000 7fe6975f6640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690105ec0 0x7fe6901074f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe68802ae60 tx=0x7fe688031690 comp rx=0 tx=0).stop 2024-09-11T23:54:03.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.980+0000 7fe6975f6640 1 -- 172.21.15.62:0/2858459643 shutdown_connections 2024-09-11T23:54:03.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.980+0000 7fe6975f6640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe67c0a2e80 0x7fe67c0a5340 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe680009670 tx=0x7fe680006bd0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.980+0000 7fe6975f6640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690105ec0 0x7fe6901074f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.980+0000 7fe6975f6640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6901011e0 0x7fe690106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.980+0000 7fe6975f6640 1 --2- 172.21.15.62:0/2858459643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6901008c0 0x7fe69010efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:03.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.980+0000 7fe6975f6640 1 -- 172.21.15.62:0/2858459643 >> 172.21.15.62:0/2858459643 conn(0x7fe6900fc7d0 msgr2=0x7fe690103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:03.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.981+0000 7fe6975f6640 1 -- 172.21.15.62:0/2858459643 shutdown_connections 2024-09-11T23:54:03.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:03.981+0000 7fe6975f6640 1 -- 172.21.15.62:0/2858459643 wait complete. 2024-09-11T23:54:04.677 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-11T23:54:04.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-11T23:54:04.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:04 smithi120 ceph-mon[35182]: pgmap v62: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 1.38k op/s 2024-09-11T23:54:04.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:04 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2858459643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:04.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:04 smithi083 ceph-mon[34805]: pgmap v62: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 1.38k op/s 2024-09-11T23:54:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:04 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2858459643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:04.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:04 smithi062 ceph-mon[30958]: pgmap v62: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 1.38k op/s 2024-09-11T23:54:04.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:04 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2858459643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.485+0000 7fcde9773640 1 -- 172.21.15.62:0/2542607063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde4109b10 msgr2=0x7fcde4114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.485+0000 7fcde9773640 1 --2- 172.21.15.62:0/2542607063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde4109b10 0x7fcde4114570 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009900 tx=0x7fcdcc02a480 comp rx=0 tx=0).stop 2024-09-11T23:54:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.485+0000 7fcde9773640 1 -- 172.21.15.62:0/2542607063 shutdown_connections 2024-09-11T23:54:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.485+0000 7fcde9773640 1 --2- 172.21.15.62:0/2542607063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcde4114ab0 0x7fcde4116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.485+0000 7fcde9773640 1 --2- 172.21.15.62:0/2542607063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde4109b10 0x7fcde4114570 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.485+0000 7fcde9773640 1 --2- 172.21.15.62:0/2542607063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcde41091f0 0x7fcde41095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.485+0000 7fcde9773640 1 -- 172.21.15.62:0/2542607063 >> 172.21.15.62:0/2542607063 conn(0x7fcde406e140 msgr2=0x7fcde406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.485+0000 7fcde9773640 1 -- 172.21.15.62:0/2542607063 shutdown_connections 2024-09-11T23:54:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.485+0000 7fcde9773640 1 -- 172.21.15.62:0/2542607063 wait complete. 2024-09-11T23:54:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde9773640 1 Processor -- start 2024-09-11T23:54:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde9773640 1 -- start start 2024-09-11T23:54:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde9773640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcde41091f0 0x7fcde4111820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde9773640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde4109b10 0x7fcde4111d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde9773640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcde4114ab0 0x7fcde41122a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde9773640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcde410c0b0 con 0x7fcde4109b10 2024-09-11T23:54:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde9773640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcde410bf30 con 0x7fcde41091f0 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde9773640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcde410c230 con 0x7fcde4114ab0 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcde4114ab0 0x7fcde41122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcde4114ab0 0x7fcde41122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:43766/0 (socket says 172.21.15.62:43766) 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde37fe640 1 -- 172.21.15.62:0/3312002833 learned_addr learned my addr 172.21.15.62:0/3312002833 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcddbfff640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde4109b10 0x7fcde4111d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.486+0000 7fcde2ffd640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcde41091f0 0x7fcde4111820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde37fe640 1 -- 172.21.15.62:0/3312002833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcde41091f0 msgr2=0x7fcde4111820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde37fe640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcde41091f0 0x7fcde4111820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde37fe640 1 -- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde4109b10 msgr2=0x7fcde4111d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde37fe640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde4109b10 0x7fcde4111d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde37fe640 1 -- 172.21.15.62:0/3312002833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde41b3570 con 0x7fcde4114ab0 2024-09-11T23:54:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcddbfff640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde4109b10 0x7fcde4111d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:54:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde37fe640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcde4114ab0 0x7fcde41122a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00eb40 tx=0x7fcddc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde0ff9640 1 -- 172.21.15.62:0/3312002833 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc004b20 con 0x7fcde4114ab0 2024-09-11T23:54:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde0ff9640 1 -- 172.21.15.62:0/3312002833 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcddc004cc0 con 0x7fcde4114ab0 2024-09-11T23:54:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde9773640 1 -- 172.21.15.62:0/3312002833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde41b3860 con 0x7fcde4114ab0 2024-09-11T23:54:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde0ff9640 1 -- 172.21.15.62:0/3312002833 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0107b0 con 0x7fcde4114ab0 2024-09-11T23:54:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.487+0000 7fcde9773640 1 -- 172.21.15.62:0/3312002833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde41b3d70 con 0x7fcde4114ab0 2024-09-11T23:54:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.490+0000 7fcdebb4c740 1 Processor -- start 2024-09-11T23:54:06.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.491+0000 7fcde0ff9640 1 -- 172.21.15.62:0/3312002833 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcddc005020 con 0x7fcde4114ab0 2024-09-11T23:54:06.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.491+0000 7fcdebb4c740 1 -- start start 2024-09-11T23:54:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.492+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 0x5644c80b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.493+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b9080 0x5644c80c3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.493+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80c3940 0x5644c80c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.493+0000 7fcdebb4c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5644c801e2a0 con 0x5644c80c3940 2024-09-11T23:54:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.493+0000 7fcdebb4c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5644c7fc5b10 con 0x5644c80b8760 2024-09-11T23:54:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.493+0000 7fcdebb4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5644c801a260 con 0x5644c80b9080 2024-09-11T23:54:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.493+0000 7fcde2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b9080 0x5644c80c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.493+0000 7fcde2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b9080 0x5644c80c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:43774/0 (socket says 172.21.15.62:43774) 2024-09-11T23:54:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.493+0000 7fcde2ffd640 1 -- 172.21.15.62:0/37141579 learned_addr learned my addr 172.21.15.62:0/37141579 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.493+0000 7fcddbfff640 1 --2- 172.21.15.62:0/37141579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 0x5644c80b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.493+0000 7fcde0ff9640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcdd00a2cf0 0x7fcdd00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.494+0000 7fcde0ff9640 1 -- 172.21.15.62:0/3312002833 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fcddc0cfb60 con 0x7fcde4114ab0 2024-09-11T23:54:06.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.494+0000 7fcde37fe640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcdd00a2cf0 0x7fcdd00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.494+0000 7fcddbfff640 1 --2- 172.21.15.62:0/37141579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80c3940 0x5644c80c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.494+0000 7fcddbfff640 1 -- 172.21.15.62:0/37141579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b9080 msgr2=0x5644c80c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:06.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.494+0000 7fcddbfff640 1 --2- 172.21.15.62:0/37141579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b9080 0x5644c80c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.494+0000 7fcddbfff640 1 -- 172.21.15.62:0/37141579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80c3940 msgr2=0x5644c80c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:06.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.494+0000 7fcddbfff640 1 --2- 172.21.15.62:0/37141579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80c3940 0x5644c80c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.494+0000 7fcddbfff640 1 -- 172.21.15.62:0/37141579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c7d8fb90 con 0x5644c80b8760 2024-09-11T23:54:06.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.495+0000 7fcddbfff640 1 --2- 172.21.15.62:0/37141579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 0x5644c80b8b40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc02aa70 tx=0x7fcdcc031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5170c3e43ed7e7b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:06.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.495+0000 7fcdd97fa640 1 -- 172.21.15.62:0/37141579 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc035070 con 0x5644c80b8760 2024-09-11T23:54:06.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.496+0000 7fcdd97fa640 1 -- 172.21.15.62:0/37141579 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc009520 con 0x5644c80b8760 2024-09-11T23:54:06.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.496+0000 7fcdd97fa640 1 -- 172.21.15.62:0/37141579 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc003850 con 0x5644c80b8760 2024-09-11T23:54:06.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.496+0000 7fcde37fe640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcdd00a2cf0 0x7fcdd00a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcddc098a80 tx=0x7fcddc0cfe60 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:06.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.497+0000 7fcdebb4c740 1 -- 172.21.15.62:0/37141579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 msgr2=0x5644c80b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:06.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.497+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/37141579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 0x5644c80b8b40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc02aa70 tx=0x7fcdcc031400 comp rx=0 tx=0).stop 2024-09-11T23:54:06.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.497+0000 7fcdebb4c740 1 -- 172.21.15.62:0/37141579 shutdown_connections 2024-09-11T23:54:06.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.497+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/37141579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80c3940 0x5644c80c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.497+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/37141579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b9080 0x5644c80c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.497+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/37141579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 0x5644c80b8b40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.497+0000 7fcdebb4c740 1 -- 172.21.15.62:0/37141579 >> 172.21.15.62:0/37141579 conn(0x5644c80b4cf0 msgr2=0x5644c80b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:06.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.497+0000 7fcdebb4c740 1 -- 172.21.15.62:0/37141579 shutdown_connections 2024-09-11T23:54:06.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.497+0000 7fcdebb4c740 1 -- 172.21.15.62:0/37141579 wait complete. 2024-09-11T23:54:06.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcdebb4c740 1 -- start start 2024-09-11T23:54:06.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcdebb4c740 1 Processor -- start 2024-09-11T23:54:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80c3940 0x5644c80c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcdebb4c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5644c80c6510 con 0x5644c80b9080 2024-09-11T23:54:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcdebb4c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5644c7fa66e0 con 0x5644c80c3940 2024-09-11T23:54:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcdebb4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5644c80c6690 con 0x5644c80b8760 2024-09-11T23:54:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcdebb4c740 10 client.0 _ll_register_callbacks cb 0x5644c7fcaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcde2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcde2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80c0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43092/0 (socket says 172.21.15.62:43092) 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcde2ffd640 1 -- 172.21.15.62:0/2027778817 learned_addr learned my addr 172.21.15.62:0/2027778817 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcddbfff640 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80c3940 0x5644c80c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.498+0000 7fcddbfff640 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.499+0000 7fcddbfff640 1 -- 172.21.15.62:0/2027778817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80c3940 msgr2=0x5644c80c12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.499+0000 7fcddbfff640 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80c3940 0x5644c80c12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.499+0000 7fcddbfff640 1 -- 172.21.15.62:0/2027778817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 msgr2=0x5644c80c0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.499+0000 7fcddbfff640 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80c0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.499+0000 7fcddbfff640 1 -- 172.21.15.62:0/2027778817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c80cfa50 con 0x5644c80b8760 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.499+0000 7fcddbfff640 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80c0870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc005ab0 tx=0x7fcdcc02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de989491ec251d05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.499+0000 7fcdca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.500+0000 7fcdca7fc640 1 -- 172.21.15.62:0/2027778817 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc050070 con 0x5644c80b8760 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.500+0000 7fcdca7fc640 1 -- 172.21.15.62:0/2027778817 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fcdcc005500 con 0x5644c80b8760 2024-09-11T23:54:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.500+0000 7fcdebb4c740 1 -- 172.21.15.62:0/2027778817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdcc009520 con 0x5644c80b8760 2024-09-11T23:54:06.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.501+0000 7fcdca7fc640 1 -- 172.21.15.62:0/2027778817 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc035070 con 0x5644c80b8760 2024-09-11T23:54:06.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.501+0000 7fcdebb4c740 10 client.25330 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:06.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.501+0000 7fcdebb4c740 1 -- 172.21.15.62:0/2027778817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644c80c6510 con 0x5644c80b8760 2024-09-11T23:54:06.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.501+0000 7fcdca7fc640 1 -- 172.21.15.62:0/2027778817 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc009520 con 0x5644c80b8760 2024-09-11T23:54:06.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.501+0000 7fcdca7fc640 1 -- 172.21.15.62:0/2027778817 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcdcc057020 con 0x5644c80b8760 2024-09-11T23:54:06.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.502+0000 7fcdebb4c740 10 client.25330 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:06.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.502+0000 7fcdebb4c740 10 client.25330 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:54:06.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.502+0000 7fcdebb4c740 20 client.25330 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:06.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.502+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5644c80d3560 0x5644c80d5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.502+0000 7fcdebb4c740 4 client.25330 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:54:06.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.502+0000 7fcdebb4c740 1 -- 172.21.15.62:0/2027778817 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644c802b9d0 con 0x5644c80d3560 2024-09-11T23:54:06.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.502+0000 7fcde2ffd640 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5644c80d3560 0x5644c80d5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.502+0000 7fcdca7fc640 10 client.25330 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.502+0000 7fcde2ffd640 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5644c80d3560 0x5644c80d5950 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a491d0062f9be9bb server_cookie=9e66edfef58cb6bc in_seq=0 out_seq=0 2024-09-11T23:54:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.503+0000 7fcdca7fc640 1 -- 172.21.15.62:0/2027778817 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644c802b9d0 con 0x5644c80d3560 2024-09-11T23:54:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.503+0000 7fcdca7fc640 10 client.25330 handle_command_reply: tid=0 2024-09-11T23:54:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.503+0000 7fcdebb4c740 1 client.25330 shutdown 2024-09-11T23:54:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.503+0000 7fcdebb4c740 1 -- 172.21.15.62:0/2027778817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 msgr2=0x5644c80c0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.503+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80c0870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc005ab0 tx=0x7fcdcc02a480 comp rx=0 tx=0).stop 2024-09-11T23:54:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.504+0000 7fcdebb4c740 1 -- 172.21.15.62:0/2027778817 shutdown_connections 2024-09-11T23:54:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.504+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5644c80d3560 0x5644c80d5950 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.504+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80c3940 0x5644c80c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.504+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80c0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.504+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/2027778817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80c0870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.504+0000 7fcdebb4c740 1 -- 172.21.15.62:0/2027778817 >> 172.21.15.62:0/2027778817 conn(0x5644c80b4cf0 msgr2=0x5644c80b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.504+0000 7fcdca7fc640 0 client.25330 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.504+0000 7fcdebb4c740 1 -- 172.21.15.62:0/2027778817 shutdown_connections 2024-09-11T23:54:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.504+0000 7fcdebb4c740 1 -- 172.21.15.62:0/2027778817 wait complete. 2024-09-11T23:54:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.505+0000 7fcdebb4c740 20 client.25330 trim_cache size 0 max 16384 2024-09-11T23:54:06.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.529+0000 7fcdebb4c740 1 Processor -- start 2024-09-11T23:54:06.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.529+0000 7fcdebb4c740 1 -- start start 2024-09-11T23:54:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.529+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.529+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.530+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.530+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80bb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:43792/0 (socket says 172.21.15.62:43792) 2024-09-11T23:54:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.530+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80c3940 0x5644c80db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.530+0000 7fcddbfff640 1 -- 172.21.15.62:0/1474539553 learned_addr learned my addr 172.21.15.62:0/1474539553 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.530+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1474539553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41bef70 con 0x5644c80b9080 2024-09-11T23:54:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.530+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1474539553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41bc840 con 0x5644c80c3940 2024-09-11T23:54:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.530+0000 7fcde2ffd640 1 --2- 172.21.15.62:0/1474539553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.530+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1474539553 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41ba690 con 0x5644c80b8760 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.530+0000 7fcddbfff640 1 --2- 172.21.15.62:0/1474539553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80c3940 0x5644c80db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.531+0000 7fcde2ffd640 1 -- 172.21.15.62:0/1474539553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 msgr2=0x5644c80bb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.531+0000 7fcde2ffd640 1 --2- 172.21.15.62:0/1474539553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80bb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.531+0000 7fcde2ffd640 1 -- 172.21.15.62:0/1474539553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80c3940 msgr2=0x5644c80db4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.531+0000 7fcde2ffd640 1 --2- 172.21.15.62:0/1474539553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80c3940 0x5644c80db4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.531+0000 7fcde2ffd640 1 -- 172.21.15.62:0/1474539553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c80b0260 con 0x5644c80b9080 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.531+0000 7fcddbfff640 1 --2- 172.21.15.62:0/1474539553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80bb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.531+0000 7fcde2ffd640 1 --2- 172.21.15.62:0/1474539553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80dafb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x5644c80d5f20 tx=0x7fcdd4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9957c16cbb5b1958 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.531+0000 7fcdd8ff9640 1 -- 172.21.15.62:0/1474539553 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40038b0 con 0x5644c80b9080 2024-09-11T23:54:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.531+0000 7fcdd8ff9640 1 -- 172.21.15.62:0/1474539553 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5644c802b9d0 con 0x5644c80b9080 2024-09-11T23:54:06.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.532+0000 7fcdd8ff9640 1 -- 172.21.15.62:0/1474539553 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4029480 con 0x5644c80b9080 2024-09-11T23:54:06.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.532+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1474539553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 msgr2=0x5644c80dafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:06.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.532+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/1474539553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80dafb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x5644c80d5f20 tx=0x7fcdd4014040 comp rx=0 tx=0).stop 2024-09-11T23:54:06.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.533+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1474539553 shutdown_connections 2024-09-11T23:54:06.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.533+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/1474539553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80c3940 0x5644c80db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.533+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/1474539553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80dafb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.533+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/1474539553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80b8760 0x5644c80bb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.533+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1474539553 >> 172.21.15.62:0/1474539553 conn(0x5644c80b4cf0 msgr2=0x5644c80bf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.533+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1474539553 shutdown_connections 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.533+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1474539553 wait complete. 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.533+0000 7fcdebb4c740 1 -- start start 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcdebb4c740 1 Processor -- start 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 0x5644c80cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcdebb4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80c3940 0x5644c80c8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcdebb4c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41bad00 con 0x5644c80b9080 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcdebb4c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41bc170 con 0x5644c80b8760 2024-09-11T23:54:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcdebb4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41bb3d0 con 0x5644c80c3940 2024-09-11T23:54:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 0x5644c80cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcdebb4c740 10 client.0 _ll_register_callbacks cb 0x7fcdcc0049f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcde2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 0x5644c80cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:45122/0 (socket says 172.21.15.62:45122) 2024-09-11T23:54:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.534+0000 7fcddbfff640 1 -- 172.21.15.62:0/1550970093 learned_addr learned my addr 172.21.15.62:0/1550970093 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.535+0000 7fcddbfff640 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80c3940 0x5644c80c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.535+0000 7fcde2ffd640 1 -- 172.21.15.62:0/1550970093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80c3940 msgr2=0x5644c80c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:06.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.535+0000 7fcde2ffd640 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80c3940 0x5644c80c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.535+0000 7fcde2ffd640 1 -- 172.21.15.62:0/1550970093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 msgr2=0x5644c80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:06.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.535+0000 7fcde2ffd640 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 0x5644c80cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.535+0000 7fcde2ffd640 1 -- 172.21.15.62:0/1550970093 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c80c9140 con 0x5644c80b9080 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.535+0000 7fcddbfff640 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80c3940 0x5644c80c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.536+0000 7fcde2ffd640 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80cc120 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5644c80641b0 tx=0x7fcdd4005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7f556703847172 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.536+0000 7fcdca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.536+0000 7fcdca7fc640 1 -- 172.21.15.62:0/1550970093 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd402e070 con 0x5644c80b9080 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.536+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1550970093 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644c802b9d0 con 0x5644c80b9080 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.536+0000 7fcdca7fc640 1 -- 172.21.15.62:0/1550970093 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fcdd40048c0 con 0x5644c80b9080 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.536+0000 7fcdca7fc640 1 -- 172.21.15.62:0/1550970093 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcdd4004ac0 con 0x5644c80b9080 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.536+0000 7fcdebb4c740 10 client.15696 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.536+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1550970093 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644c813eaf0 con 0x5644c80b9080 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.537+0000 7fcdca7fc640 1 -- 172.21.15.62:0/1550970093 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5644c802b9d0 con 0x5644c80b9080 2024-09-11T23:54:06.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.537+0000 7fcdca7fc640 1 -- 172.21.15.62:0/1550970093 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcdd4012540 con 0x5644c80b9080 2024-09-11T23:54:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.537+0000 7fcdebb4c740 10 client.15696 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.537+0000 7fcdebb4c740 10 client.15696 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:54:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.537+0000 7fcdebb4c740 20 client.15696 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.537+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5644c813f1a0 0x5644c81415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.537+0000 7fcdebb4c740 4 client.15696 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-11T23:54:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.537+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1550970093 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x5644c8025d20 con 0x5644c813f1a0 2024-09-11T23:54:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.537+0000 7fcddbfff640 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5644c813f1a0 0x5644c81415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.538+0000 7fcdca7fc640 10 client.15696 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.538+0000 7fcddbfff640 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5644c813f1a0 0x5644c81415c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd366e0b4c0f2539 server_cookie=b8a36dab005eff03 in_seq=0 out_seq=0 2024-09-11T23:54:06.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:06 smithi062 ceph-mon[30958]: pgmap v63: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-09-11T23:54:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.773+0000 7fcdca7fc640 1 -- 172.21.15.62:0/1550970093 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5644c8025d20 con 0x5644c813f1a0 2024-09-11T23:54:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.773+0000 7fcdca7fc640 10 client.15696 handle_command_reply: tid=0 2024-09-11T23:54:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.773+0000 7fcdebb4c740 1 client.15696 shutdown 2024-09-11T23:54:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.773+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1550970093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 msgr2=0x5644c80cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.773+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80cc120 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5644c80641b0 tx=0x7fcdd4005430 comp rx=0 tx=0).stop 2024-09-11T23:54:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1550970093 shutdown_connections 2024-09-11T23:54:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5644c813f1a0 0x5644c81415c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644c80c3940 0x5644c80c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c80b9080 0x5644c80cc120 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcdebb4c740 1 --2- 172.21.15.62:0/1550970093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644c80b8760 0x5644c80cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcdca7fc640 0 client.15696 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1550970093 >> 172.21.15.62:0/1550970093 conn(0x5644c80b4cf0 msgr2=0x5644c80b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1550970093 shutdown_connections 2024-09-11T23:54:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcdebb4c740 1 -- 172.21.15.62:0/1550970093 wait complete. 2024-09-11T23:54:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcdebb4c740 20 client.15696 trim_cache size 0 max 16384 2024-09-11T23:54:06.775 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:54:06.775 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 27, 2024-09-11T23:54:06.775 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 6, 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "d27ad26f": { 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 6, 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 133.4, 2024-09-11T23:54:06.776 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:06.777 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:54:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcde9773640 1 -- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcdd00a2cf0 msgr2=0x7fcdd00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcde9773640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcdd00a2cf0 0x7fcdd00a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcddc098a80 tx=0x7fcddc0cfe60 comp rx=0 tx=0).stop 2024-09-11T23:54:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcde9773640 1 -- 172.21.15.62:0/3312002833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcde4114ab0 msgr2=0x7fcde41122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcde9773640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcde4114ab0 0x7fcde41122a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00eb40 tx=0x7fcddc00c2d0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcde9773640 1 -- 172.21.15.62:0/3312002833 shutdown_connections 2024-09-11T23:54:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcde9773640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcdd00a2cf0 0x7fcdd00a51b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcde9773640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcde4114ab0 0x7fcde41122a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcde9773640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde4109b10 0x7fcde4111d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcde9773640 1 --2- 172.21.15.62:0/3312002833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcde41091f0 0x7fcde4111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:06.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.774+0000 7fcde9773640 1 -- 172.21.15.62:0/3312002833 >> 172.21.15.62:0/3312002833 conn(0x7fcde406e140 msgr2=0x7fcde40708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:06.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.775+0000 7fcde9773640 1 -- 172.21.15.62:0/3312002833 shutdown_connections 2024-09-11T23:54:06.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:06.775+0000 7fcde9773640 1 -- 172.21.15.62:0/3312002833 wait complete. 2024-09-11T23:54:06.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:06 smithi120 ceph-mon[35182]: pgmap v63: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-09-11T23:54:06.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:06 smithi083 ceph-mon[34805]: pgmap v63: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-09-11T23:54:07.446 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 85.0 s, 2334.4 tps, lat 36.039 ms stddev 69.535 2024-09-11T23:54:07.497 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd27ad26f', quiesce took 4.9 seconds. Will release after: 8.5 2024-09-11T23:54:08.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:08 smithi120 ceph-mon[35182]: pgmap v64: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 2.16k op/s 2024-09-11T23:54:08.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:08 smithi083 ceph-mon[34805]: pgmap v64: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 2.16k op/s 2024-09-11T23:54:08.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:08 smithi062 ceph-mon[30958]: pgmap v64: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 2.16k op/s 2024-09-11T23:54:10.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:10 smithi120 ceph-mon[35182]: pgmap v65: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-09-11T23:54:10.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:10 smithi083 ceph-mon[34805]: pgmap v65: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-09-11T23:54:10.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:10 smithi062 ceph-mon[30958]: pgmap v65: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-09-11T23:54:12.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 90.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:54:12.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:12 smithi120 ceph-mon[35182]: pgmap v66: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 MiB/s wr, 1.81k op/s 2024-09-11T23:54:12.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:12 smithi083 ceph-mon[34805]: pgmap v66: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 MiB/s wr, 1.81k op/s 2024-09-11T23:54:12.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:12 smithi062 ceph-mon[30958]: pgmap v66: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 MiB/s wr, 1.81k op/s 2024-09-11T23:54:14.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:14 smithi120 ceph-mon[35182]: pgmap v67: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2024-09-11T23:54:14.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:14 smithi083 ceph-mon[34805]: pgmap v67: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2024-09-11T23:54:14.955 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:54:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:14 smithi062 ceph-mon[30958]: pgmap v67: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2024-09-11T23:54:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:15.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:15 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:54:15.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:15 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:15 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:15 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:15 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1607214990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4101de0 msgr2=0x7ff3a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 --2- 172.21.15.62:0/1607214990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4101de0 0x7ff3a41021c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff398009900 tx=0x7ff39802a410 comp rx=0 tx=0).stop 2024-09-11T23:54:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1607214990 shutdown_connections 2024-09-11T23:54:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 --2- 172.21.15.62:0/1607214990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a4106f10 0x7ff3a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 --2- 172.21.15.62:0/1607214990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3a4102790 0x7ff3a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 --2- 172.21.15.62:0/1607214990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4101de0 0x7ff3a41021c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1607214990 >> 172.21.15.62:0/1607214990 conn(0x7ff3a40fd8c0 msgr2=0x7ff3a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1607214990 shutdown_connections 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1607214990 wait complete. 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 Processor -- start 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.724+0000 7ff3aae4e640 1 -- start start 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3aae4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3a4101de0 0x7ff3a40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3aae4e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4102790 0x7ff3a4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3aae4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a4106f10 0x7ff3a4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3aae4e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a4111f70 con 0x7ff3a4102790 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3aae4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a4111df0 con 0x7ff3a4106f10 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3aae4e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41120f0 con 0x7ff3a4101de0 2024-09-11T23:54:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4102790 0x7ff3a4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4102790 0x7ff3a4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36336/0 (socket says 172.21.15.62:36336) 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a3fff640 1 -- 172.21.15.62:0/1813392033 learned_addr learned my addr 172.21.15.62:0/1813392033 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a4106f10 0x7ff3a4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a8bea640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3a4101de0 0x7ff3a40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a3fff640 1 -- 172.21.15.62:0/1813392033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3a4101de0 msgr2=0x7ff3a40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3a4101de0 0x7ff3a40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a3fff640 1 -- 172.21.15.62:0/1813392033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a4106f10 msgr2=0x7ff3a4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a4106f10 0x7ff3a4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a3fff640 1 -- 172.21.15.62:0/1813392033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a41a6990 con 0x7ff3a4102790 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4102790 0x7ff3a4074a30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff39000e9f0 tx=0x7ff39000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a1ffb640 1 -- 172.21.15.62:0/1813392033 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39000cd80 con 0x7ff3a4102790 2024-09-11T23:54:15.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3a1ffb640 1 -- 172.21.15.62:0/1813392033 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff390005500 con 0x7ff3a4102790 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.725+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1813392033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a41a6c80 con 0x7ff3a4102790 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.726+0000 7ff3a1ffb640 1 -- 172.21.15.62:0/1813392033 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390010690 con 0x7ff3a4102790 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.726+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1813392033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a41a7170 con 0x7ff3a4102790 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.728+0000 7ff3a1ffb640 1 -- 172.21.15.62:0/1813392033 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff3900046c0 con 0x7ff3a4102790 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.728+0000 7ff3ad227740 1 Processor -- start 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.728+0000 7ff3ad227740 1 -- start start 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 0x55ecb23af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3ad227740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ecb230a2e0 con 0x55ecb23a44a0 2024-09-11T23:54:15.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3ad227740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ecb22b1ce0 con 0x55ecb23af680 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3ad227740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ecb23062a0 con 0x55ecb23a4dc0 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a93eb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a93eb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36350/0 (socket says 172.21.15.62:36350) 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a93eb640 1 -- 172.21.15.62:0/543177218 learned_addr learned my addr 172.21.15.62:0/543177218 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a8bea640 1 --2- 172.21.15.62:0/543177218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 0x55ecb23af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a1ffb640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3940a2cf0 0x7ff3940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/543177218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a93eb640 1 -- 172.21.15.62:0/543177218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 msgr2=0x55ecb23af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/543177218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 0x55ecb23af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a93eb640 1 -- 172.21.15.62:0/543177218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 msgr2=0x55ecb23b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:15.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/543177218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a93eb640 1 -- 172.21.15.62:0/543177218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb2296440 con 0x55ecb23a44a0 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3940a2cf0 0x7ff3940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a1ffb640 1 -- 172.21.15.62:0/1813392033 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7ff3900cf8a0 con 0x7ff3a4102790 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.729+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/543177218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23a4880 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff39c011590 tx=0x7ff39c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf61c54fda3dcaa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff38effd640 1 -- 172.21.15.62:0/543177218 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c017050 con 0x55ecb23a44a0 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff38effd640 1 -- 172.21.15.62:0/543177218 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39c01bb90 con 0x55ecb23a44a0 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff38effd640 1 -- 172.21.15.62:0/543177218 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00e3b0 con 0x55ecb23a44a0 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3940a2cf0 0x7ff3940a51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff39c022000 tx=0x7ff39c023000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff3ad227740 1 -- 172.21.15.62:0/543177218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 msgr2=0x55ecb23a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff3ad227740 1 --2- 172.21.15.62:0/543177218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23a4880 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff39c011590 tx=0x7ff39c011a50 comp rx=0 tx=0).stop 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff3ad227740 1 -- 172.21.15.62:0/543177218 shutdown_connections 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff3ad227740 1 --2- 172.21.15.62:0/543177218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff3ad227740 1 --2- 172.21.15.62:0/543177218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 0x55ecb23af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff3ad227740 1 --2- 172.21.15.62:0/543177218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23a4880 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.730+0000 7ff3ad227740 1 -- 172.21.15.62:0/543177218 >> 172.21.15.62:0/543177218 conn(0x55ecb23a0a30 msgr2=0x55ecb23a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 1 -- 172.21.15.62:0/543177218 shutdown_connections 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 1 -- 172.21.15.62:0/543177218 wait complete. 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 1 -- start start 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 1 Processor -- start 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 0x55ecb23ac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ecb23b2250 con 0x55ecb23a44a0 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ecb22c5050 con 0x55ecb23af680 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ecb23b23d0 con 0x55ecb23a4dc0 2024-09-11T23:54:15.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3ad227740 10 client.0 _ll_register_callbacks cb 0x55ecb2305160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.731+0000 7ff3a8bea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 0x55ecb23ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a8bea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36364/0 (socket says 172.21.15.62:36364) 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a8bea640 1 -- 172.21.15.62:0/557999106 learned_addr learned my addr 172.21.15.62:0/557999106 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a3fff640 1 -- 172.21.15.62:0/557999106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 msgr2=0x55ecb23ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a3fff640 1 -- 172.21.15.62:0/557999106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 msgr2=0x55ecb23aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a3fff640 1 -- 172.21.15.62:0/557999106 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb207b3c0 con 0x55ecb23a4dc0 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a8bea640 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:15.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 0x55ecb23ac280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff3900ccfc0 tx=0x7ff39009b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97d8914db722a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff383fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff383fff640 1 -- 172.21.15.62:0/557999106 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3900cd440 con 0x55ecb23a4dc0 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff383fff640 1 -- 172.21.15.62:0/557999106 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7ff3900a4570 con 0x55ecb23a4dc0 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3ad227740 1 -- 172.21.15.62:0/557999106 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff39c01bb90 con 0x55ecb23a4dc0 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff383fff640 1 -- 172.21.15.62:0/557999106 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff39009c490 con 0x55ecb23a4dc0 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3ad227740 10 client.25348 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.732+0000 7ff3ad227740 1 -- 172.21.15.62:0/557999106 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ecb23bdc20 con 0x55ecb23a4dc0 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff383fff640 1 -- 172.21.15.62:0/557999106 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39c01bb90 con 0x55ecb23a4dc0 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff383fff640 1 -- 172.21.15.62:0/557999106 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff39009b370 con 0x55ecb23a4dc0 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff3ad227740 10 client.25348 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff3ad227740 10 client.25348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff3ad227740 20 client.25348 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff3ad227740 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ecb23bf130 0x55ecb23c15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff3ad227740 4 client.25348 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff3ad227740 1 -- 172.21.15.62:0/557999106 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecb2317850 con 0x55ecb23bf130 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff3a8bea640 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ecb23bf130 0x55ecb23c15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff383fff640 10 client.25348 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.733+0000 7ff3a8bea640 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ecb23bf130 0x55ecb23c15b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84a2b35598ffa41 server_cookie=63d61293e76487c9 in_seq=0 out_seq=0 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.734+0000 7ff383fff640 1 -- 172.21.15.62:0/557999106 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ecb2317850 con 0x55ecb23bf130 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.734+0000 7ff383fff640 10 client.25348 handle_command_reply: tid=0 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.734+0000 7ff3ad227740 1 client.25348 shutdown 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 1 -- 172.21.15.62:0/557999106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 msgr2=0x55ecb23ac280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 0x55ecb23ac280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff3900ccfc0 tx=0x7ff39009b6f0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 1 -- 172.21.15.62:0/557999106 shutdown_connections 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ecb23bf130 0x55ecb23c15b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a4dc0 0x55ecb23ac280 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff383fff640 0 client.25348 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:15.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 1 --2- 172.21.15.62:0/557999106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a44a0 0x55ecb23aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 1 -- 172.21.15.62:0/557999106 >> 172.21.15.62:0/557999106 conn(0x55ecb23a0a30 msgr2=0x55ecb23a6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:15.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 1 -- 172.21.15.62:0/557999106 shutdown_connections 2024-09-11T23:54:15.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 1 -- 172.21.15.62:0/557999106 wait complete. 2024-09-11T23:54:15.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.735+0000 7ff3ad227740 20 client.25348 trim_cache size 0 max 16384 2024-09-11T23:54:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3ad227740 1 Processor -- start 2024-09-11T23:54:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3ad227740 1 -- start start 2024-09-11T23:54:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a44a0 0x55ecb23bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a4dc0 0x55ecb23c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3ad227740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41b2280 con 0x55ecb23a4dc0 2024-09-11T23:54:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3ad227740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41afb50 con 0x55ecb23af680 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3ad227740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41ad9a0 con 0x55ecb23a44a0 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3a3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a4dc0 0x55ecb23c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3a3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a4dc0 0x55ecb23c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36378/0 (socket says 172.21.15.62:36378) 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3a3fff640 1 -- 172.21.15.62:0/3211870804 learned_addr learned my addr 172.21.15.62:0/3211870804 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3a8bea640 1 --2- 172.21.15.62:0/3211870804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a44a0 0x55ecb23bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/3211870804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3a3fff640 1 -- 172.21.15.62:0/3211870804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a44a0 msgr2=0x55ecb23bb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/3211870804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a44a0 0x55ecb23bb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3a3fff640 1 -- 172.21.15.62:0/3211870804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 msgr2=0x55ecb23c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/3211870804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.753+0000 7ff3a3fff640 1 -- 172.21.15.62:0/3211870804 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb23aa7d0 con 0x55ecb23a4dc0 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/3211870804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a4dc0 0x55ecb23c6e80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff390098760 tx=0x7ff39009b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba928f64edfcf08 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff38e7fc640 1 -- 172.21.15.62:0/3211870804 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39009bb60 con 0x55ecb23a4dc0 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff38e7fc640 1 -- 172.21.15.62:0/3211870804 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3900cd440 con 0x55ecb23a4dc0 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff38e7fc640 1 -- 172.21.15.62:0/3211870804 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39009b1e0 con 0x55ecb23a4dc0 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff3ad227740 1 -- 172.21.15.62:0/3211870804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a4dc0 msgr2=0x55ecb23c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff3ad227740 1 --2- 172.21.15.62:0/3211870804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a4dc0 0x55ecb23c6e80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff390098760 tx=0x7ff39009b820 comp rx=0 tx=0).stop 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff3ad227740 1 -- 172.21.15.62:0/3211870804 shutdown_connections 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff3ad227740 1 --2- 172.21.15.62:0/3211870804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23af680 0x55ecb23c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff3ad227740 1 --2- 172.21.15.62:0/3211870804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23a4dc0 0x55ecb23c6e80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff3ad227740 1 --2- 172.21.15.62:0/3211870804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a44a0 0x55ecb23bb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.754+0000 7ff3ad227740 1 -- 172.21.15.62:0/3211870804 >> 172.21.15.62:0/3211870804 conn(0x55ecb23a0a30 msgr2=0x55ecb23ab370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 1 -- 172.21.15.62:0/3211870804 shutdown_connections 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 1 -- 172.21.15.62:0/3211870804 wait complete. 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 1 -- start start 2024-09-11T23:54:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 1 Processor -- start 2024-09-11T23:54:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a44a0 0x55ecb23b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23a4dc0 0x55ecb23b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23af680 0x55ecb23b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41ae010 con 0x55ecb23af680 2024-09-11T23:54:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41af480 con 0x55ecb23a4dc0 2024-09-11T23:54:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41ae6e0 con 0x55ecb23a44a0 2024-09-11T23:54:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.755+0000 7ff3ad227740 10 client.0 _ll_register_callbacks cb 0x7ff39000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a93eb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23af680 0x55ecb23b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a8bea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a44a0 0x55ecb23b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a93eb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23af680 0x55ecb23b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36386/0 (socket says 172.21.15.62:36386) 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a93eb640 1 -- 172.21.15.62:0/2419711362 learned_addr learned my addr 172.21.15.62:0/2419711362 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23a4dc0 0x55ecb23b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a93eb640 1 -- 172.21.15.62:0/2419711362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a44a0 msgr2=0x55ecb23b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a44a0 0x55ecb23b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a93eb640 1 -- 172.21.15.62:0/2419711362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23a4dc0 msgr2=0x55ecb23b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23a4dc0 0x55ecb23b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a93eb640 1 -- 172.21.15.62:0/2419711362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb23b5880 con 0x55ecb23af680 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a3fff640 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23a4dc0 0x55ecb23b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3a93eb640 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23af680 0x55ecb23b5340 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff39c005fd0 tx=0x7ff39c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528e5bd53626277f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff383fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff383fff640 1 -- 172.21.15.62:0/2419711362 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c02a040 con 0x55ecb23af680 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff383fff640 1 -- 172.21.15.62:0/2419711362 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7ff39c004bc0 con 0x55ecb23af680 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff3ad227740 1 -- 172.21.15.62:0/2419711362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3900cd440 con 0x55ecb23af680 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.756+0000 7ff383fff640 1 -- 172.21.15.62:0/2419711362 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff39c017020 con 0x55ecb23af680 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff3ad227740 10 client.15714 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff3ad227740 1 -- 172.21.15.62:0/2419711362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ecb2429b60 con 0x55ecb23af680 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff383fff640 1 -- 172.21.15.62:0/2419711362 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3900cd440 con 0x55ecb23af680 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff383fff640 1 -- 172.21.15.62:0/2419711362 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff39c025430 con 0x55ecb23af680 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff3ad227740 10 client.15714 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff3ad227740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff3ad227740 20 client.15714 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff3ad227740 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ecb242a100 0x55ecb242c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff3ad227740 4 client.15714 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff3ad227740 1 -- 172.21.15.62:0/2419711362 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ecb2311d60 con 0x55ecb242a100 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff3a8bea640 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ecb242a100 0x55ecb242c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff3a8bea640 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ecb242a100 0x55ecb242c520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=137fda818fc5c396 server_cookie=baa3e8fdd39e200f in_seq=0 out_seq=0 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.757+0000 7ff383fff640 10 client.15714 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff383fff640 1 -- 172.21.15.62:0/2419711362 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ecb2311d60 con 0x55ecb242a100 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff383fff640 10 client.15714 handle_command_reply: tid=0 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 client.15714 shutdown 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 -- 172.21.15.62:0/2419711362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23af680 msgr2=0x55ecb23b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23af680 0x55ecb23b5340 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff39c005fd0 tx=0x7ff39c009b30 comp rx=0 tx=0).stop 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 -- 172.21.15.62:0/2419711362 shutdown_connections 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ecb242a100 0x55ecb242c520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecb23af680 0x55ecb23b5340 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecb23a4dc0 0x55ecb23b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 --2- 172.21.15.62:0/2419711362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecb23a44a0 0x55ecb23b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 -- 172.21.15.62:0/2419711362 >> 172.21.15.62:0/2419711362 conn(0x55ecb23a0a30 msgr2=0x55ecb239f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff383fff640 0 client.15714 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 -- 172.21.15.62:0/2419711362 shutdown_connections 2024-09-11T23:54:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 1 -- 172.21.15.62:0/2419711362 wait complete. 2024-09-11T23:54:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.758+0000 7ff3ad227740 20 client.15714 trim_cache size 0 max 16384 2024-09-11T23:54:15.768 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:54:15.768 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-11T23:54:15.768 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-11T23:54:15.768 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:54:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1813392033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3940a2cf0 msgr2=0x7ff3940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3940a2cf0 0x7ff3940a51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff39c022000 tx=0x7ff39c023000 comp rx=0 tx=0).stop 2024-09-11T23:54:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1813392033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4102790 msgr2=0x7ff3a4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4102790 0x7ff3a4074a30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff39000e9f0 tx=0x7ff39000edf0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1813392033 shutdown_connections 2024-09-11T23:54:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3940a2cf0 0x7ff3940a51b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a4106f10 0x7ff3a4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4102790 0x7ff3a4074a30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 --2- 172.21.15.62:0/1813392033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3a4101de0 0x7ff3a40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1813392033 >> 172.21.15.62:0/1813392033 conn(0x7ff3a40fd8c0 msgr2=0x7ff3a40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1813392033 shutdown_connections 2024-09-11T23:54:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:15.759+0000 7ff3aae4e640 1 -- 172.21.15.62:0/1813392033 wait complete. 2024-09-11T23:54:15.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:15 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:54:15.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:15 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:15 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:15 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:15 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:15 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:54:15.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:15 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:15 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:15 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:15 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:15.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:54:16.512 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:54:16.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:54:16.594 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:16.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:16 smithi062 ceph-mon[30958]: pgmap v68: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-11T23:54:16.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:16 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:54:16.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:16 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:54:16.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:16 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:16.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:16 smithi120 ceph-mon[35182]: pgmap v68: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-11T23:54:16.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:16 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:54:16.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:16 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:54:16.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:16 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:16 smithi083 ceph-mon[34805]: pgmap v68: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-11T23:54:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:16 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:54:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:16 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:54:16.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:16 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:54:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.115+0000 7ff3c3405640 1 -- 172.21.15.62:0/3057708856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3bc106f10 msgr2=0x7ff3bc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.115+0000 7ff3c3405640 1 --2- 172.21.15.62:0/3057708856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3bc106f10 0x7ff3bc10ea90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009ca0 tx=0x7ff3b402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.115+0000 7ff3c3405640 1 -- 172.21.15.62:0/3057708856 shutdown_connections 2024-09-11T23:54:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.115+0000 7ff3c3405640 1 --2- 172.21.15.62:0/3057708856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3bc106f10 0x7ff3bc10ea90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.115+0000 7ff3c3405640 1 --2- 172.21.15.62:0/3057708856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3bc102790 0x7ff3bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.115+0000 7ff3c3405640 1 --2- 172.21.15.62:0/3057708856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3bc101de0 0x7ff3bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.115+0000 7ff3c3405640 1 -- 172.21.15.62:0/3057708856 >> 172.21.15.62:0/3057708856 conn(0x7ff3bc0fd8c0 msgr2=0x7ff3bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.115+0000 7ff3c3405640 1 -- 172.21.15.62:0/3057708856 shutdown_connections 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.115+0000 7ff3c3405640 1 -- 172.21.15.62:0/3057708856 wait complete. 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c3405640 1 Processor -- start 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c3405640 1 -- start start 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c3405640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3bc101de0 0x7ff3bc1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c3405640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3bc102790 0x7ff3bc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c3405640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3bc106f10 0x7ff3bc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c3405640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc111f70 con 0x7ff3bc102790 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c3405640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc111df0 con 0x7ff3bc101de0 2024-09-11T23:54:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c3405640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1120f0 con 0x7ff3bc106f10 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c09a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3bc102790 0x7ff3bc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c09a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3bc102790 0x7ff3bc1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36404/0 (socket says 172.21.15.62:36404) 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.116+0000 7ff3c09a0640 1 -- 172.21.15.62:0/106666570 learned_addr learned my addr 172.21.15.62:0/106666570 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c11a1640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3bc101de0 0x7ff3bc1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c19a2640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3bc106f10 0x7ff3bc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c09a0640 1 -- 172.21.15.62:0/106666570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3bc106f10 msgr2=0x7ff3bc19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c09a0640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3bc106f10 0x7ff3bc19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c09a0640 1 -- 172.21.15.62:0/106666570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3bc101de0 msgr2=0x7ff3bc1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c09a0640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3bc101de0 0x7ff3bc1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c09a0640 1 -- 172.21.15.62:0/106666570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3bc19d360 con 0x7ff3bc102790 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c19a2640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3bc106f10 0x7ff3bc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c09a0640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3bc102790 0x7ff3bc1988f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00e9f0 tx=0x7ff3ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3b27fc640 1 -- 172.21.15.62:0/106666570 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac00cd80 con 0x7ff3bc102790 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c3405640 1 -- 172.21.15.62:0/106666570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3bc19d650 con 0x7ff3bc102790 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3b27fc640 1 -- 172.21.15.62:0/106666570 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3ac010910 con 0x7ff3bc102790 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3b27fc640 1 -- 172.21.15.62:0/106666570 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0188f0 con 0x7ff3bc102790 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.117+0000 7ff3c3405640 1 -- 172.21.15.62:0/106666570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3bc1a4f30 con 0x7ff3bc102790 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.118+0000 7ff3c3405640 1 -- 172.21.15.62:0/106666570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3bc1021c0 con 0x7ff3bc102790 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.119+0000 7ff3b27fc640 1 -- 172.21.15.62:0/106666570 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff3ac010ac0 con 0x7ff3bc102790 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.120+0000 7ff3b27fc640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3a80a2cf0 0x7ff3a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.120+0000 7ff3c11a1640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3a80a2cf0 0x7ff3a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.120+0000 7ff3b27fc640 1 -- 172.21.15.62:0/106666570 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7ff3ac0cfbe0 con 0x7ff3bc102790 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.121+0000 7ff3c11a1640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3a80a2cf0 0x7ff3a80a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800bbe0 tx=0x7ff3b800b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.123+0000 7ff3b27fc640 1 -- 172.21.15.62:0/106666570 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3ac09c830 con 0x7ff3bc102790 2024-09-11T23:54:17.181 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:17.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.352+0000 7ff3c3405640 1 -- 172.21.15.62:0/106666570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3bc02d7c0 con 0x7ff3bc102790 2024-09-11T23:54:17.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.353+0000 7ff3b27fc640 1 -- 172.21.15.62:0/106666570 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11912 (secure 0 0 0) 0x7ff3ac09c130 con 0x7ff3bc102790 2024-09-11T23:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:54:17.354 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-09-11T23:52:56:186189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:55.695276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:54:17.356 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-09-11T23:54:17.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 -- 172.21.15.62:0/106666570 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3a80a2cf0 msgr2=0x7ff3a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:17.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3a80a2cf0 0x7ff3a80a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800bbe0 tx=0x7ff3b800b390 comp rx=0 tx=0).stop 2024-09-11T23:54:17.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 -- 172.21.15.62:0/106666570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3bc102790 msgr2=0x7ff3bc1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:17.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3bc102790 0x7ff3bc1988f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00e9f0 tx=0x7ff3ac00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 -- 172.21.15.62:0/106666570 shutdown_connections 2024-09-11T23:54:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3a80a2cf0 0x7ff3a80a51b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3bc106f10 0x7ff3bc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3bc102790 0x7ff3bc1988f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 --2- 172.21.15.62:0/106666570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3bc101de0 0x7ff3bc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 -- 172.21.15.62:0/106666570 >> 172.21.15.62:0/106666570 conn(0x7ff3bc0fd8c0 msgr2=0x7ff3bc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:17.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 -- 172.21.15.62:0/106666570 shutdown_connections 2024-09-11T23:54:17.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.355+0000 7ff3c3405640 1 -- 172.21.15.62:0/106666570 wait complete. 2024-09-11T23:54:17.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 95.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:54:17.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.810+0000 7fe001281640 1 -- 172.21.15.62:0/3619049500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdffc10c170 msgr2=0x7fdffc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:17.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.810+0000 7fe001281640 1 --2- 172.21.15.62:0/3619049500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdffc10c170 0x7fdffc10e610 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009ca0 tx=0x7fdff402a950 comp rx=0 tx=0).stop 2024-09-11T23:54:17.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.810+0000 7fe001281640 1 -- 172.21.15.62:0/3619049500 shutdown_connections 2024-09-11T23:54:17.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.810+0000 7fe001281640 1 --2- 172.21.15.62:0/3619049500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdffc10c170 0x7fdffc10e610 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.810+0000 7fe001281640 1 --2- 172.21.15.62:0/3619049500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdffc103660 0x7fdffc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.810+0000 7fe001281640 1 --2- 172.21.15.62:0/3619049500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc102d40 0x7fdffc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.810+0000 7fe001281640 1 -- 172.21.15.62:0/3619049500 >> 172.21.15.62:0/3619049500 conn(0x7fdffc0fc7a0 msgr2=0x7fdffc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.810+0000 7fe001281640 1 -- 172.21.15.62:0/3619049500 shutdown_connections 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.810+0000 7fe001281640 1 -- 172.21.15.62:0/3619049500 wait complete. 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fe001281640 1 Processor -- start 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fe001281640 1 -- start start 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fe001281640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc102d40 0x7fdffc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fe001281640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdffc103660 0x7fdffc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fe001281640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdffc10c170 0x7fdffc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fe001281640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc111fa0 con 0x7fdffc10c170 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fe001281640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc111e20 con 0x7fdffc102d40 2024-09-11T23:54:17.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fe001281640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc112120 con 0x7fdffc103660 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fdffb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdffc10c170 0x7fdffc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fdffb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdffc10c170 0x7fdffc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36422/0 (socket says 172.21.15.62:36422) 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.811+0000 7fdffb7fe640 1 -- 172.21.15.62:0/377131118 learned_addr learned my addr 172.21.15.62:0/377131118 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdffaffd640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc102d40 0x7fdffc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdff3fff640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdffc103660 0x7fdffc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdffb7fe640 1 -- 172.21.15.62:0/377131118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdffc103660 msgr2=0x7fdffc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdffb7fe640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdffc103660 0x7fdffc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdffb7fe640 1 -- 172.21.15.62:0/377131118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc102d40 msgr2=0x7fdffc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdffb7fe640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc102d40 0x7fdffc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdffb7fe640 1 -- 172.21.15.62:0/377131118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc19d380 con 0x7fdffc10c170 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdff3fff640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdffc103660 0x7fdffc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdffb7fe640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdffc10c170 0x7fdffc19cca0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009dc0 tx=0x7fdff4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:17.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdff8ff9640 1 -- 172.21.15.62:0/377131118 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4034040 con 0x7fdffc10c170 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdff8ff9640 1 -- 172.21.15.62:0/377131118 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff402f940 con 0x7fdffc10c170 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fe001281640 1 -- 172.21.15.62:0/377131118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdffc19d610 con 0x7fdffc10c170 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fdff8ff9640 1 -- 172.21.15.62:0/377131118 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40046a0 con 0x7fdffc10c170 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.812+0000 7fe001281640 1 -- 172.21.15.62:0/377131118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdffc1a4f50 con 0x7fdffc10c170 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.813+0000 7fe001281640 1 -- 172.21.15.62:0/377131118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdffc103120 con 0x7fdffc10c170 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.814+0000 7fdff8ff9640 1 -- 172.21.15.62:0/377131118 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdff4037020 con 0x7fdffc10c170 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.815+0000 7fdff8ff9640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdfe80a2d70 0x7fdfe80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.815+0000 7fdffaffd640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdfe80a2d70 0x7fdfe80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.815+0000 7fdff8ff9640 1 -- 172.21.15.62:0/377131118 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fdff40f0080 con 0x7fdffc10c170 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.816+0000 7fdffaffd640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdfe80a2d70 0x7fdfe80a5230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009bd0 tx=0x7fdfec005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:17.818+0000 7fdff8ff9640 1 -- 172.21.15.62:0/377131118 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdff40bce70 con 0x7fdffc10c170 2024-09-11T23:54:17.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:17 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/106666570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:17.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:17 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/106666570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:17.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:17 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/106666570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:18.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.034+0000 7fe001281640 1 -- 172.21.15.62:0/377131118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdffc199730 con 0x7fdffc10c170 2024-09-11T23:54:18.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.035+0000 7fdff8ff9640 1 -- 172.21.15.62:0/377131118 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11912 (secure 0 0 0) 0x7fdffc199730 con 0x7fdffc10c170 2024-09-11T23:54:18.035 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:54:18.035 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-09-11T23:52:56:186189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:55.695276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:54:18.035 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-09-11T23:54:18.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.037+0000 7fe001281640 1 -- 172.21.15.62:0/377131118 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdfe80a2d70 msgr2=0x7fdfe80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdfe80a2d70 0x7fdfe80a5230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009bd0 tx=0x7fdfec005f90 comp rx=0 tx=0).stop 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 -- 172.21.15.62:0/377131118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdffc10c170 msgr2=0x7fdffc19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdffc10c170 0x7fdffc19cca0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009dc0 tx=0x7fdff4003c10 comp rx=0 tx=0).stop 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 -- 172.21.15.62:0/377131118 shutdown_connections 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdfe80a2d70 0x7fdfe80a5230 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdffc10c170 0x7fdffc19cca0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdffc103660 0x7fdffc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 --2- 172.21.15.62:0/377131118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc102d40 0x7fdffc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 -- 172.21.15.62:0/377131118 >> 172.21.15.62:0/377131118 conn(0x7fdffc0fc7a0 msgr2=0x7fdffc0fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 -- 172.21.15.62:0/377131118 shutdown_connections 2024-09-11T23:54:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:18.038+0000 7fe001281640 1 -- 172.21.15.62:0/377131118 wait complete. 2024-09-11T23:54:18.351 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id d27ad26f --release --await' 2024-09-11T23:54:18.352 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id d27ad26f --release --await 2024-09-11T23:54:18.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:18 smithi120 ceph-mon[35182]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 1.58k op/s 2024-09-11T23:54:18.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:18 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/377131118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:18.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:18 smithi083 ceph-mon[34805]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 1.58k op/s 2024-09-11T23:54:18.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:18 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/377131118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:18.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:18 smithi062 ceph-mon[30958]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 1.58k op/s 2024-09-11T23:54:18.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:18 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/377131118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:19.006 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:54:19.009 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.498+0000 7f4f4259e640 1 -- 172.21.15.62:0/3245729957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3c107af0 msgr2=0x7f4f3c107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.498+0000 7f4f4259e640 1 --2- 172.21.15.62:0/3245729957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3c107af0 0x7f4f3c107ed0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009930 tx=0x7f4f3802a420 comp rx=0 tx=0).stop 2024-09-11T23:54:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.498+0000 7f4f4259e640 1 -- 172.21.15.62:0/3245729957 shutdown_connections 2024-09-11T23:54:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.498+0000 7f4f4259e640 1 --2- 172.21.15.62:0/3245729957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f3c102380 0x7f4f3c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.498+0000 7f4f4259e640 1 --2- 172.21.15.62:0/3245729957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f3c1019e0 0x7f4f3c101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.498+0000 7f4f4259e640 1 --2- 172.21.15.62:0/3245729957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3c107af0 0x7f4f3c107ed0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.498+0000 7f4f4259e640 1 -- 172.21.15.62:0/3245729957 >> 172.21.15.62:0/3245729957 conn(0x7f4f3c0fd700 msgr2=0x7f4f3c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.498+0000 7f4f4259e640 1 -- 172.21.15.62:0/3245729957 shutdown_connections 2024-09-11T23:54:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.498+0000 7f4f4259e640 1 -- 172.21.15.62:0/3245729957 wait complete. 2024-09-11T23:54:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f4259e640 1 Processor -- start 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f4259e640 1 -- start start 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f4259e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f3c1019e0 0x7f4f3c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f4259e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3c102380 0x7f4f3c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f4259e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f3c107af0 0x7f4f3c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f4259e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c111ef0 con 0x7f4f3c102380 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f4259e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c111d70 con 0x7f4f3c107af0 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f4259e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c112070 con 0x7f4f3c1019e0 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f40d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3c102380 0x7f4f3c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f40d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3c102380 0x7f4f3c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36436/0 (socket says 172.21.15.62:36436) 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f40d9b640 1 -- 172.21.15.62:0/3634571833 learned_addr learned my addr 172.21.15.62:0/3634571833 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f41d9d640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f3c107af0 0x7f4f3c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f4159c640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f3c1019e0 0x7f4f3c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f40d9b640 1 -- 172.21.15.62:0/3634571833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f3c1019e0 msgr2=0x7f4f3c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f3c1019e0 0x7f4f3c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f40d9b640 1 -- 172.21.15.62:0/3634571833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f3c107af0 msgr2=0x7f4f3c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f3c107af0 0x7f4f3c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f40d9b640 1 -- 172.21.15.62:0/3634571833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c1a68c0 con 0x7f4f3c102380 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.499+0000 7f4f41d9d640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f3c107af0 0x7f4f3c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.500+0000 7f4f4159c640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f3c1019e0 0x7f4f3c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.500+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3c102380 0x7f4f3c074a30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00d950 tx=0x7f4f2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.500+0000 7f4f327fc640 1 -- 172.21.15.62:0/3634571833 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c017070 con 0x7f4f3c102380 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.500+0000 7f4f327fc640 1 -- 172.21.15.62:0/3634571833 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f2c009c30 con 0x7f4f3c102380 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.500+0000 7f4f4259e640 1 -- 172.21.15.62:0/3634571833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c1a6bb0 con 0x7f4f3c102380 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.500+0000 7f4f327fc640 1 -- 172.21.15.62:0/3634571833 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c00f6b0 con 0x7f4f3c102380 2024-09-11T23:54:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.500+0000 7f4f4259e640 1 -- 172.21.15.62:0/3634571833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c1a70f0 con 0x7f4f3c102380 2024-09-11T23:54:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.502+0000 7f4f327fc640 1 -- 172.21.15.62:0/3634571833 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4f2c0109d0 con 0x7f4f3c102380 2024-09-11T23:54:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f45bd9740 1 Processor -- start 2024-09-11T23:54:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f45bd9740 1 -- start start 2024-09-11T23:54:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f327fc640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4f280a2cf0 0x7f4f280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f327fc640 1 -- 172.21.15.62:0/3634571833 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f4f2c0cf120 con 0x7f4f3c102380 2024-09-11T23:54:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f41d9d640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4f280a2cf0 0x7f4f280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44b210 0x564f0c44b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44bb30 0x564f0c455eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c4587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f45bd9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564f0c3b0620 con 0x564f0c44bb30 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f45bd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564f0c357e50 con 0x564f0c44b210 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.503+0000 7f4f45bd9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f0c3ac5e0 con 0x564f0c4563f0 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f40d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44bb30 0x564f0c455eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f40d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44bb30 0x564f0c455eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36440/0 (socket says 172.21.15.62:36440) 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f40d9b640 1 -- 172.21.15.62:0/2925722925 learned_addr learned my addr 172.21.15.62:0/2925722925 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f41d9d640 1 --2- 172.21.15.62:0/2925722925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c4587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f4159c640 1 --2- 172.21.15.62:0/2925722925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44b210 0x564f0c44b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f40d9b640 1 -- 172.21.15.62:0/2925722925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 msgr2=0x564f0c4587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/2925722925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c4587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f40d9b640 1 -- 172.21.15.62:0/2925722925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44b210 msgr2=0x564f0c44b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/2925722925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44b210 0x564f0c44b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f40d9b640 1 -- 172.21.15.62:0/2925722925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f0c348d40 con 0x564f0c44bb30 2024-09-11T23:54:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f41d9d640 1 --2- 172.21.15.62:0/2925722925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c4587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:54:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f41d9d640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4f280a2cf0 0x7f4f280a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4f34005cc0 tx=0x7f4f3400e720 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/2925722925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44bb30 0x564f0c455eb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c097ad0 tx=0x7f4f2c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30e6fe4e571f02d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.504+0000 7f4f277fe640 1 -- 172.21.15.62:0/2925722925 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c020040 con 0x564f0c44bb30 2024-09-11T23:54:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.505+0000 7f4f277fe640 1 -- 172.21.15.62:0/2925722925 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f2c09fa20 con 0x564f0c44bb30 2024-09-11T23:54:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.505+0000 7f4f277fe640 1 -- 172.21.15.62:0/2925722925 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c084bb0 con 0x564f0c44bb30 2024-09-11T23:54:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.505+0000 7f4f45bd9740 1 -- 172.21.15.62:0/2925722925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44bb30 msgr2=0x564f0c455eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.505+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/2925722925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44bb30 0x564f0c455eb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c097ad0 tx=0x7f4f2c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.505+0000 7f4f45bd9740 1 -- 172.21.15.62:0/2925722925 shutdown_connections 2024-09-11T23:54:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.505+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/2925722925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c4587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.505+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/2925722925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44bb30 0x564f0c455eb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.505+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/2925722925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44b210 0x564f0c44b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.505+0000 7f4f45bd9740 1 -- 172.21.15.62:0/2925722925 >> 172.21.15.62:0/2925722925 conn(0x564f0c4477a0 msgr2=0x564f0c447b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.506+0000 7f4f45bd9740 1 -- 172.21.15.62:0/2925722925 shutdown_connections 2024-09-11T23:54:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.506+0000 7f4f45bd9740 1 -- 172.21.15.62:0/2925722925 wait complete. 2024-09-11T23:54:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.506+0000 7f4f45bd9740 1 -- start start 2024-09-11T23:54:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.506+0000 7f4f45bd9740 1 Processor -- start 2024-09-11T23:54:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44bb30 0x564f0c453830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c453d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f45bd9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564f0c458fc0 con 0x564f0c44b210 2024-09-11T23:54:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f45bd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564f0c338f10 con 0x564f0c44bb30 2024-09-11T23:54:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f45bd9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f0c459140 con 0x564f0c4563f0 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f45bd9740 10 client.0 _ll_register_callbacks cb 0x564f0c35cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f4159c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f4159c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36444/0 (socket says 172.21.15.62:36444) 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f4159c640 1 -- 172.21.15.62:0/1856259326 learned_addr learned my addr 172.21.15.62:0/1856259326 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44bb30 0x564f0c453830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f41d9d640 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c453d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f4159c640 1 -- 172.21.15.62:0/1856259326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 msgr2=0x564f0c453d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f4159c640 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c453d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f4159c640 1 -- 172.21.15.62:0/1856259326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44bb30 msgr2=0x564f0c453830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f4159c640 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44bb30 0x564f0c453830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f4159c640 1 -- 172.21.15.62:0/1856259326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f0c0bf580 con 0x564f0c44b210 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f4159c640 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4532f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4f38033000 tx=0x7f4f38031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445a0301d18fefa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.507+0000 7f4f24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f24ff9640 1 -- 172.21.15.62:0/1856259326 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3803f070 con 0x564f0c44b210 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f24ff9640 1 -- 172.21.15.62:0/1856259326 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f4f38004450 con 0x564f0c44b210 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f45bd9740 1 -- 172.21.15.62:0/1856259326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f2c09fa20 con 0x564f0c44b210 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f24ff9640 1 -- 172.21.15.62:0/1856259326 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f38035070 con 0x564f0c44b210 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f45bd9740 10 client.15738 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f45bd9740 1 -- 172.21.15.62:0/1856259326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564f0c189960 con 0x564f0c44b210 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f24ff9640 1 -- 172.21.15.62:0/1856259326 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f2c09fa20 con 0x564f0c44b210 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f24ff9640 1 -- 172.21.15.62:0/1856259326 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4f38004c90 con 0x564f0c44b210 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f45bd9740 10 client.15738 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f45bd9740 10 client.15738 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f45bd9740 20 client.15738 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564f0c465f70 0x564f0c468430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f45bd9740 4 client.15738 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:54:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.508+0000 7f4f45bd9740 1 -- 172.21.15.62:0/1856259326 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f0c3bdb90 con 0x564f0c465f70 2024-09-11T23:54:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.509+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564f0c465f70 0x564f0c468430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.509+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564f0c465f70 0x564f0c468430 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31db9faacc315edf server_cookie=72f29bd0dfdebb91 in_seq=0 out_seq=0 2024-09-11T23:54:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.509+0000 7f4f24ff9640 10 client.15738 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f24ff9640 1 -- 172.21.15.62:0/1856259326 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f0c3bdb90 con 0x564f0c465f70 2024-09-11T23:54:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f24ff9640 10 client.15738 handle_command_reply: tid=0 2024-09-11T23:54:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 client.15738 shutdown 2024-09-11T23:54:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 -- 172.21.15.62:0/1856259326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 msgr2=0x564f0c4532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4532f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4f38033000 tx=0x7f4f38031920 comp rx=0 tx=0).stop 2024-09-11T23:54:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 -- 172.21.15.62:0/1856259326 shutdown_connections 2024-09-11T23:54:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564f0c465f70 0x564f0c468430 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c453d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44bb30 0x564f0c453830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f24ff9640 0 client.15738 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/1856259326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4532f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 -- 172.21.15.62:0/1856259326 >> 172.21.15.62:0/1856259326 conn(0x564f0c4477a0 msgr2=0x564f0c44d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 -- 172.21.15.62:0/1856259326 shutdown_connections 2024-09-11T23:54:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.510+0000 7f4f45bd9740 1 -- 172.21.15.62:0/1856259326 wait complete. 2024-09-11T23:54:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.511+0000 7f4f45bd9740 20 client.15738 trim_cache size 0 max 16384 2024-09-11T23:54:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f45bd9740 1 Processor -- start 2024-09-11T23:54:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f45bd9740 1 -- start start 2024-09-11T23:54:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4467f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c44bb30 0x564f0c46dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c4563f0 0x564f0c46e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f45bd9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1b2130 con 0x564f0c44b210 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f45bd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1afa00 con 0x564f0c4563f0 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f45bd9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1ad850 con 0x564f0c44bb30 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f4159c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f4159c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36460/0 (socket says 172.21.15.62:36460) 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f4159c640 1 -- 172.21.15.62:0/3791780499 learned_addr learned my addr 172.21.15.62:0/3791780499 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.532+0000 7f4f41d9d640 1 --2- 172.21.15.62:0/3791780499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c4563f0 0x564f0c46e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.533+0000 7f4f4159c640 1 -- 172.21.15.62:0/3791780499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c44bb30 msgr2=0x564f0c46dd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.533+0000 7f4f4159c640 1 --2- 172.21.15.62:0/3791780499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c44bb30 0x564f0c46dd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.533+0000 7f4f4159c640 1 -- 172.21.15.62:0/3791780499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c4563f0 msgr2=0x564f0c46e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.533+0000 7f4f4159c640 1 --2- 172.21.15.62:0/3791780499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c4563f0 0x564f0c46e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.533+0000 7f4f4159c640 1 -- 172.21.15.62:0/3791780499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f0c445d60 con 0x564f0c44b210 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.533+0000 7f4f4159c640 1 --2- 172.21.15.62:0/3791780499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4467f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4f38001eb0 tx=0x7f4f3802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2655e9ff0f3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.533+0000 7f4f26ffd640 1 -- 172.21.15.62:0/3791780499 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38050040 con 0x564f0c44b210 2024-09-11T23:54:19.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.533+0000 7f4f26ffd640 1 -- 172.21.15.62:0/3791780499 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38003a00 con 0x564f0c44b210 2024-09-11T23:54:19.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.533+0000 7f4f26ffd640 1 -- 172.21.15.62:0/3791780499 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3803f040 con 0x564f0c44b210 2024-09-11T23:54:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.534+0000 7f4f45bd9740 1 -- 172.21.15.62:0/3791780499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 msgr2=0x564f0c4467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.534+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/3791780499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4467f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4f38001eb0 tx=0x7f4f3802a420 comp rx=0 tx=0).stop 2024-09-11T23:54:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.534+0000 7f4f45bd9740 1 -- 172.21.15.62:0/3791780499 shutdown_connections 2024-09-11T23:54:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.534+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/3791780499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c4563f0 0x564f0c46e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.534+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/3791780499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c44bb30 0x564f0c46dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.534+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/3791780499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c4467f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.534+0000 7f4f45bd9740 1 -- 172.21.15.62:0/3791780499 >> 172.21.15.62:0/3791780499 conn(0x564f0c4477a0 msgr2=0x564f0c4460e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.534+0000 7f4f45bd9740 1 -- 172.21.15.62:0/3791780499 shutdown_connections 2024-09-11T23:54:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.534+0000 7f4f45bd9740 1 -- 172.21.15.62:0/3791780499 wait complete. 2024-09-11T23:54:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.535+0000 7f4f45bd9740 1 -- start start 2024-09-11T23:54:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.535+0000 7f4f45bd9740 1 Processor -- start 2024-09-11T23:54:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.535+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c45a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.535+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44bb30 0x564f0c45acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.535+0000 7f4f45bd9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c4cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.535+0000 7f4f4159c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c45a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f4159c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c45a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36470/0 (socket says 172.21.15.62:36470) 2024-09-11T23:54:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f45bd9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1adec0 con 0x564f0c44b210 2024-09-11T23:54:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f4159c640 1 -- 172.21.15.62:0/752696477 learned_addr learned my addr 172.21.15.62:0/752696477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f45bd9740 1 -- 172.21.15.62:0/752696477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1af330 con 0x564f0c44bb30 2024-09-11T23:54:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f45bd9740 1 -- 172.21.15.62:0/752696477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1ae590 con 0x564f0c4563f0 2024-09-11T23:54:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44bb30 0x564f0c45acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f45bd9740 10 client.0 _ll_register_callbacks cb 0x7f4f380315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f41d9d640 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c4cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f4159c640 1 -- 172.21.15.62:0/752696477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 msgr2=0x564f0c4cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f4159c640 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c4cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f4159c640 1 -- 172.21.15.62:0/752696477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44bb30 msgr2=0x564f0c45acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f4159c640 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44bb30 0x564f0c45acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f4159c640 1 -- 172.21.15.62:0/752696477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f0c4d0210 con 0x564f0c44b210 2024-09-11T23:54:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f4159c640 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c45a7b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4f38005c90 tx=0x7f4f38005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb6caaf568ec456 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.536+0000 7f4f24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:54:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.537+0000 7f4f24ff9640 1 -- 172.21.15.62:0/752696477 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38061070 con 0x564f0c44b210 2024-09-11T23:54:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.537+0000 7f4f24ff9640 1 -- 172.21.15.62:0/752696477 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f4f380055c0 con 0x564f0c44b210 2024-09-11T23:54:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.537+0000 7f4f45bd9740 1 -- 172.21.15.62:0/752696477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f38003a00 con 0x564f0c44b210 2024-09-11T23:54:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.537+0000 7f4f24ff9640 1 -- 172.21.15.62:0/752696477 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f3806d050 con 0x564f0c44b210 2024-09-11T23:54:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.537+0000 7f4f45bd9740 10 client.15744 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.537+0000 7f4f45bd9740 1 -- 172.21.15.62:0/752696477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564f0c4d1490 con 0x564f0c44b210 2024-09-11T23:54:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.538+0000 7f4f24ff9640 1 -- 172.21.15.62:0/752696477 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38031900 con 0x564f0c44b210 2024-09-11T23:54:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.538+0000 7f4f24ff9640 1 -- 172.21.15.62:0/752696477 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4f38066440 con 0x564f0c44b210 2024-09-11T23:54:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.538+0000 7f4f45bd9740 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.538+0000 7f4f45bd9740 10 client.15744 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:54:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.538+0000 7f4f45bd9740 20 client.15744 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.538+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564f0c4d1b00 0x564f0c4d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.538+0000 7f4f45bd9740 4 client.15744 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d27ad26f", "release": true, "await": true}] 2024-09-11T23:54:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.538+0000 7f4f45bd9740 1 -- 172.21.15.62:0/752696477 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d27ad26f", "release": true, "await": true}) -- 0x7f4f38003a00 con 0x564f0c4d1b00 2024-09-11T23:54:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.539+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564f0c4d1b00 0x564f0c4d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.539+0000 7f4f40d9b640 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564f0c4d1b00 0x564f0c4d3f20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27fecca4f297a72 server_cookie=b505a5a6376baeb in_seq=0 out_seq=0 2024-09-11T23:54:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.539+0000 7f4f24ff9640 10 client.15744 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:19.544 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:19.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.582+0000 7f4f24ff9640 1 -- 172.21.15.62:0/752696477 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f4f38003a00 con 0x564f0c4d1b00 2024-09-11T23:54:19.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.582+0000 7f4f24ff9640 10 client.15744 handle_command_reply: tid=0 2024-09-11T23:54:19.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.582+0000 7f4f45bd9740 1 client.15744 shutdown 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f45bd9740 1 -- 172.21.15.62:0/752696477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 msgr2=0x564f0c45a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c45a7b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4f38005c90 tx=0x7f4f38005cc0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f45bd9740 1 -- 172.21.15.62:0/752696477 shutdown_connections 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564f0c4d1b00 0x564f0c4d3f20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f0c4563f0 0x564f0c4cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f0c44bb30 0x564f0c45acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f45bd9740 1 --2- 172.21.15.62:0/752696477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f0c44b210 0x564f0c45a7b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f24ff9640 0 client.15744 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f45bd9740 1 -- 172.21.15.62:0/752696477 >> 172.21.15.62:0/752696477 conn(0x564f0c4477a0 msgr2=0x564f0c44d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f45bd9740 1 -- 172.21.15.62:0/752696477 shutdown_connections 2024-09-11T23:54:19.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.584+0000 7f4f45bd9740 1 -- 172.21.15.62:0/752696477 wait complete. 2024-09-11T23:54:19.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.585+0000 7f4f45bd9740 20 client.15744 trim_cache size 0 max 16384 2024-09-11T23:54:19.587 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:54:19.587 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 27, 2024-09-11T23:54:19.587 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:54:19.587 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 8, 2024-09-11T23:54:19.587 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:54:19.587 INFO:teuthology.orchestra.run.smithi062.stdout: "d27ad26f": { 2024-09-11T23:54:19.587 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 8, 2024-09-11T23:54:19.587 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 133.4, 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:54:19.588 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:19.589 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:19.589 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:19.589 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:19.589 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:19.589 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:54:19.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 -- 172.21.15.62:0/3634571833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4f280a2cf0 msgr2=0x7f4f280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:19.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4f280a2cf0 0x7f4f280a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4f34005cc0 tx=0x7f4f3400e720 comp rx=0 tx=0).stop 2024-09-11T23:54:19.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 -- 172.21.15.62:0/3634571833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3c102380 msgr2=0x7f4f3c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:19.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3c102380 0x7f4f3c074a30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00d950 tx=0x7f4f2c00de10 comp rx=0 tx=0).stop 2024-09-11T23:54:19.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 -- 172.21.15.62:0/3634571833 shutdown_connections 2024-09-11T23:54:19.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4f280a2cf0 0x7f4f280a51b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f3c107af0 0x7f4f3c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f3c102380 0x7f4f3c074a30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 --2- 172.21.15.62:0/3634571833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f3c1019e0 0x7f4f3c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:19.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 -- 172.21.15.62:0/3634571833 >> 172.21.15.62:0/3634571833 conn(0x7f4f3c0fd700 msgr2=0x7f4f3c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:19.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 -- 172.21.15.62:0/3634571833 shutdown_connections 2024-09-11T23:54:19.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:19.587+0000 7f4f4259e640 1 -- 172.21.15.62:0/3634571833 wait complete. 2024-09-11T23:54:20.422 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd27ad26f', total seconds elapsed: 17.9 2024-09-11T23:54:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.600+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2037375682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4a4100a70 msgr2=0x7fd4a4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.600+0000 7fd4aa64e640 1 --2- 172.21.15.62:0/2037375682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4a4100a70 0x7fd4a4100ed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd498009890 tx=0x7fd49802a360 comp rx=0 tx=0).stop 2024-09-11T23:54:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.600+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2037375682 shutdown_connections 2024-09-11T23:54:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.600+0000 7fd4aa64e640 1 --2- 172.21.15.62:0/2037375682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a4101410 0x7fd4a410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.600+0000 7fd4aa64e640 1 --2- 172.21.15.62:0/2037375682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4a4100a70 0x7fd4a4100ed0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.600+0000 7fd4aa64e640 1 --2- 172.21.15.62:0/2037375682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a4106a70 0x7fd4a4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.600+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2037375682 >> 172.21.15.62:0/2037375682 conn(0x7fd4a40fc790 msgr2=0x7fd4a40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.600+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2037375682 shutdown_connections 2024-09-11T23:54:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.601+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2037375682 wait complete. 2024-09-11T23:54:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.602+0000 7fd4aa64e640 1 Processor -- start 2024-09-11T23:54:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.602+0000 7fd4aa64e640 1 -- start start 2024-09-11T23:54:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.602+0000 7fd4aa64e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a4100a70 0x7fd4a410f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.602+0000 7fd4aa64e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4a4101410 0x7fd4a410f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.602+0000 7fd4aa64e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a4106a70 0x7fd4a410fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.602+0000 7fd4aa64e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a4111f70 con 0x7fd4a4100a70 2024-09-11T23:54:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.602+0000 7fd4aa64e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a4111df0 con 0x7fd4a4106a70 2024-09-11T23:54:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.602+0000 7fd4aa64e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a41120f0 con 0x7fd4a4101410 2024-09-11T23:54:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.603+0000 7fd4a37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4a4101410 0x7fd4a410f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.603+0000 7fd4a3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a4100a70 0x7fd4a410f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.603+0000 7fd4a37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4a4101410 0x7fd4a410f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:50026/0 (socket says 172.21.15.62:50026) 2024-09-11T23:54:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.603+0000 7fd4a37fe640 1 -- 172.21.15.62:0/2306144762 learned_addr learned my addr 172.21.15.62:0/2306144762 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.603+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a4106a70 0x7fd4a410fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.604+0000 7fd4a8beb640 1 -- 172.21.15.62:0/2306144762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4a4101410 msgr2=0x7fd4a410f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.604+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4a4101410 0x7fd4a410f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.604+0000 7fd4a8beb640 1 -- 172.21.15.62:0/2306144762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a4100a70 msgr2=0x7fd4a410f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.604+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a4100a70 0x7fd4a410f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.604+0000 7fd4a8beb640 1 -- 172.21.15.62:0/2306144762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a41a6950 con 0x7fd4a4106a70 2024-09-11T23:54:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.605+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a4106a70 0x7fd4a410fc60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009fd0 tx=0x7fd49c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.607+0000 7fd4a17fa640 1 -- 172.21.15.62:0/2306144762 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00ce20 con 0x7fd4a4106a70 2024-09-11T23:54:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.607+0000 7fd4a17fa640 1 -- 172.21.15.62:0/2306144762 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49c005020 con 0x7fd4a4106a70 2024-09-11T23:54:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.607+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2306144762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a41a6c40 con 0x7fd4a4106a70 2024-09-11T23:54:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.607+0000 7fd4a17fa640 1 -- 172.21.15.62:0/2306144762 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00ce20 con 0x7fd4a4106a70 2024-09-11T23:54:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.607+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2306144762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a41a7130 con 0x7fd4a4106a70 2024-09-11T23:54:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.612+0000 7fd4aca27740 1 Processor -- start 2024-09-11T23:54:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.613+0000 7fd4a17fa640 1 -- 172.21.15.62:0/2306144762 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd49c0104b0 con 0x7fd4a4106a70 2024-09-11T23:54:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.613+0000 7fd4aca27740 1 -- start start 2024-09-11T23:54:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.613+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 0x558a494a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.613+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 0x558a494a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.613+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.613+0000 7fd4aca27740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558a4940e2f0 con 0x558a494a85d0 2024-09-11T23:54:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.613+0000 7fd4aca27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558a493b5cf0 con 0x558a494b3370 2024-09-11T23:54:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.613+0000 7fd4aca27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a4940a2b0 con 0x558a494a9b40 2024-09-11T23:54:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 0x558a494a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 0x558a494a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 0x558a494a8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:50032/0 (socket says 172.21.15.62:50032) 2024-09-11T23:54:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a37fe640 1 -- 172.21.15.62:0/2090094117 learned_addr learned my addr 172.21.15.62:0/2090094117 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a37fe640 1 --2- 172.21.15.62:0/2090094117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a37fe640 1 -- 172.21.15.62:0/2090094117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 msgr2=0x558a494b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a37fe640 1 --2- 172.21.15.62:0/2090094117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a37fe640 1 -- 172.21.15.62:0/2090094117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 msgr2=0x558a494a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a37fe640 1 --2- 172.21.15.62:0/2090094117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 0x558a494a8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a37fe640 1 -- 172.21.15.62:0/2090094117 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a4939a3a0 con 0x558a494a9b40 2024-09-11T23:54:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a17fa640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd48c0a30f0 0x7fd48c0a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a37fe640 1 --2- 172.21.15.62:0/2090094117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 0x558a494a8090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd498031000 tx=0x7fd498005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5664e181988fd326 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a3fff640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd48c0a30f0 0x7fd48c0a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.614+0000 7fd4a17fa640 1 -- 172.21.15.62:0/2306144762 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fd49c00cc20 con 0x7fd4a4106a70 2024-09-11T23:54:20.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.616+0000 7fd48a7fc640 1 -- 172.21.15.62:0/2090094117 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498047070 con 0x558a494a9b40 2024-09-11T23:54:20.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.616+0000 7fd48a7fc640 1 -- 172.21.15.62:0/2090094117 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd498004320 con 0x558a494a9b40 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.616+0000 7fd48a7fc640 1 -- 172.21.15.62:0/2090094117 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498047070 con 0x558a494a9b40 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.617+0000 7fd4aca27740 1 -- 172.21.15.62:0/2090094117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 msgr2=0x558a494a8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.617+0000 7fd4aca27740 1 --2- 172.21.15.62:0/2090094117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 0x558a494a8090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd498031000 tx=0x7fd498005d20 comp rx=0 tx=0).stop 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.617+0000 7fd4a3fff640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd48c0a30f0 0x7fd48c0a55b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x558a494a5ac0 tx=0x7fd494009260 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.617+0000 7fd4aca27740 1 -- 172.21.15.62:0/2090094117 shutdown_connections 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.617+0000 7fd4aca27740 1 --2- 172.21.15.62:0/2090094117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.617+0000 7fd4aca27740 1 --2- 172.21.15.62:0/2090094117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 0x558a494a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.617+0000 7fd4aca27740 1 --2- 172.21.15.62:0/2090094117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 0x558a494a8090 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.617+0000 7fd4aca27740 1 -- 172.21.15.62:0/2090094117 >> 172.21.15.62:0/2090094117 conn(0x558a49471310 msgr2=0x558a494a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.618+0000 7fd4aca27740 1 -- 172.21.15.62:0/2090094117 shutdown_connections 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.618+0000 7fd4aca27740 1 -- 172.21.15.62:0/2090094117 wait complete. 2024-09-11T23:54:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.618+0000 7fd4aca27740 1 -- start start 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.618+0000 7fd4aca27740 1 Processor -- start 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494a85d0 0x558a494b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a9b40 0x558a494b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494b3370 0x558a494b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4aca27740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558a494be820 con 0x558a494a9b40 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4aca27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558a493c9060 con 0x558a494a85d0 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4aca27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a494be9a0 con 0x558a494b3370 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4aca27740 10 client.0 _ll_register_callbacks cb 0x558a4911c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4a8beb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494b3370 0x558a494b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4a37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a9b40 0x558a494b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4a8beb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494b3370 0x558a494b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:50038/0 (socket says 172.21.15.62:50038) 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4a8beb640 1 -- 172.21.15.62:0/868638281 learned_addr learned my addr 172.21.15.62:0/868638281 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4a3fff640 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494a85d0 0x558a494b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4a8beb640 1 -- 172.21.15.62:0/868638281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494a85d0 msgr2=0x558a494b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494a85d0 0x558a494b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4a8beb640 1 -- 172.21.15.62:0/868638281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a9b40 msgr2=0x558a494b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a9b40 0x558a494b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.619+0000 7fd4a8beb640 1 -- 172.21.15.62:0/868638281 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a4917f3c0 con 0x558a494b3370 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.620+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494b3370 0x558a494b05d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd49c09b770 tx=0x7fd49c09bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c961d4e548ba1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.620+0000 7fd4837fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.620+0000 7fd4837fe640 1 -- 172.21.15.62:0/868638281 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0a4450 con 0x558a494b3370 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.620+0000 7fd4aca27740 1 -- 172.21.15.62:0/868638281 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd498004320 con 0x558a494b3370 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.620+0000 7fd4837fe640 1 -- 172.21.15.62:0/868638281 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fd49c0a3fc0 con 0x558a494b3370 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.620+0000 7fd4837fe640 1 -- 172.21.15.62:0/868638281 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd49c0a41c0 con 0x558a494b3370 2024-09-11T23:54:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.620+0000 7fd4aca27740 10 client.25387 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.620+0000 7fd4aca27740 1 -- 172.21.15.62:0/868638281 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a494ca360 con 0x558a494b3370 2024-09-11T23:54:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.620+0000 7fd4837fe640 1 -- 172.21.15.62:0/868638281 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd498004320 con 0x558a494b3370 2024-09-11T23:54:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.621+0000 7fd4837fe640 1 -- 172.21.15.62:0/868638281 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd49c0975c0 con 0x558a494b3370 2024-09-11T23:54:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.621+0000 7fd4aca27740 10 client.25387 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.621+0000 7fd4aca27740 10 client.25387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:54:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.621+0000 7fd4aca27740 20 client.25387 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.621+0000 7fd4aca27740 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x558a494cbc40 0x558a494ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.621+0000 7fd4aca27740 4 client.25387 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:54:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.621+0000 7fd4aca27740 1 -- 172.21.15.62:0/868638281 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a4941b860 con 0x558a494cbc40 2024-09-11T23:54:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.621+0000 7fd4a3fff640 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x558a494cbc40 0x558a494ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.623+0000 7fd4837fe640 10 client.25387 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:54:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.623+0000 7fd4a3fff640 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x558a494cbc40 0x558a494ce090 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7ca690bf446ef4e server_cookie=bc003f65788611df in_seq=0 out_seq=0 2024-09-11T23:54:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.624+0000 7fd4837fe640 1 -- 172.21.15.62:0/868638281 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a4941b860 con 0x558a494cbc40 2024-09-11T23:54:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.624+0000 7fd4837fe640 10 client.25387 handle_command_reply: tid=0 2024-09-11T23:54:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.624+0000 7fd4aca27740 1 client.25387 shutdown 2024-09-11T23:54:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 1 -- 172.21.15.62:0/868638281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494b3370 msgr2=0x558a494b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494b3370 0x558a494b05d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd49c09b770 tx=0x7fd49c09bee0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 1 -- 172.21.15.62:0/868638281 shutdown_connections 2024-09-11T23:54:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x558a494cbc40 0x558a494ce090 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494b3370 0x558a494b05d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a9b40 0x558a494b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4837fe640 0 client.25387 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:54:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 1 --2- 172.21.15.62:0/868638281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494a85d0 0x558a494b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 1 -- 172.21.15.62:0/868638281 >> 172.21.15.62:0/868638281 conn(0x558a49471310 msgr2=0x558a494aa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 1 -- 172.21.15.62:0/868638281 shutdown_connections 2024-09-11T23:54:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 1 -- 172.21.15.62:0/868638281 wait complete. 2024-09-11T23:54:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.625+0000 7fd4aca27740 20 client.25387 trim_cache size 0 max 16384 2024-09-11T23:54:20.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.646+0000 7fd4aca27740 1 Processor -- start 2024-09-11T23:54:20.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.646+0000 7fd4aca27740 1 -- start start 2024-09-11T23:54:20.646 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:20 smithi062 ceph-mon[30958]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-11T23:54:20.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.646+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 0x558a494c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.646+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 0x558a494af890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.646+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.646+0000 7fd4aca27740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd490011050 con 0x558a494a85d0 2024-09-11T23:54:20.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.646+0000 7fd4aca27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd49000e920 con 0x558a494b3370 2024-09-11T23:54:20.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.646+0000 7fd4aca27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd49000c770 con 0x558a494a9b40 2024-09-11T23:54:20.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.646+0000 7fd4a3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 0x558a494c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.646+0000 7fd4a3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 0x558a494c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36520/0 (socket says 172.21.15.62:36520) 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.647+0000 7fd4a3fff640 1 -- 172.21.15.62:0/1448980586 learned_addr learned my addr 172.21.15.62:0/1448980586 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.647+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/1448980586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.647+0000 7fd4a8beb640 1 -- 172.21.15.62:0/1448980586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 msgr2=0x558a494af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.647+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/1448980586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 0x558a494af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.647+0000 7fd4a8beb640 1 -- 172.21.15.62:0/1448980586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 msgr2=0x558a494c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.647+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/1448980586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 0x558a494c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.647+0000 7fd4a8beb640 1 -- 172.21.15.62:0/1448980586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a494ce710 con 0x558a494b3370 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.647+0000 7fd4a3fff640 1 --2- 172.21.15.62:0/1448980586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 0x558a494c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.647+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/1448980586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494afdd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd49c09b9a0 tx=0x7fd49c0ba2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71a9219267462f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.648+0000 7fd489ffb640 1 -- 172.21.15.62:0/1448980586 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0a4450 con 0x558a494b3370 2024-09-11T23:54:20.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.648+0000 7fd489ffb640 1 -- 172.21.15.62:0/1448980586 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49c09a6a0 con 0x558a494b3370 2024-09-11T23:54:20.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.648+0000 7fd489ffb640 1 -- 172.21.15.62:0/1448980586 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c09b140 con 0x558a494b3370 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.649+0000 7fd4aca27740 1 -- 172.21.15.62:0/1448980586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 msgr2=0x558a494afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.649+0000 7fd4aca27740 1 --2- 172.21.15.62:0/1448980586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494afdd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd49c09b9a0 tx=0x7fd49c0ba2e0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.649+0000 7fd4aca27740 1 -- 172.21.15.62:0/1448980586 shutdown_connections 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.649+0000 7fd4aca27740 1 --2- 172.21.15.62:0/1448980586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494afdd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.649+0000 7fd4aca27740 1 --2- 172.21.15.62:0/1448980586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a9b40 0x558a494af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.649+0000 7fd4aca27740 1 --2- 172.21.15.62:0/1448980586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a85d0 0x558a494c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.649+0000 7fd4aca27740 1 -- 172.21.15.62:0/1448980586 >> 172.21.15.62:0/1448980586 conn(0x558a49471310 msgr2=0x558a494ab2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.650+0000 7fd4aca27740 1 -- 172.21.15.62:0/1448980586 shutdown_connections 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.650+0000 7fd4aca27740 1 -- 172.21.15.62:0/1448980586 wait complete. 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.651+0000 7fd4aca27740 1 -- start start 2024-09-11T23:54:20.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.651+0000 7fd4aca27740 1 Processor -- start 2024-09-11T23:54:20.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.652+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a85d0 0x558a494c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.652+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a9b40 0x558a494ace00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.652+0000 7fd4a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a85d0 0x558a494c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.652+0000 7fd4a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a85d0 0x558a494c44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:50064/0 (socket says 172.21.15.62:50064) 2024-09-11T23:54:20.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.652+0000 7fd4aca27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.652+0000 7fd4a3fff640 1 -- 172.21.15.62:0/3829549033 learned_addr learned my addr 172.21.15.62:0/3829549033 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:20.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.652+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd49000cde0 con 0x558a494a9b40 2024-09-11T23:54:20.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.652+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd49000e250 con 0x558a494b3370 2024-09-11T23:54:20.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.652+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd49000d4b0 con 0x558a494a85d0 2024-09-11T23:54:20.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.653+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.653+0000 7fd4a8beb640 1 -- 172.21.15.62:0/3829549033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a85d0 msgr2=0x558a494c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.653+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a85d0 0x558a494c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.653+0000 7fd4a8beb640 1 -- 172.21.15.62:0/3829549033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a9b40 msgr2=0x558a494ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.653+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a9b40 0x558a494ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.653+0000 7fd4a8beb640 1 -- 172.21.15.62:0/3829549033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a494a4510 con 0x558a494b3370 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.653+0000 7fd4a8beb640 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494ad3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd49c005990 tx=0x7fd49c097bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b802702e70323e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.653+0000 7fd4837fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.654+0000 7fd4aca27740 10 client.0 _ll_register_callbacks cb 0x7fd49c097360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.654+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd49c09a6a0 con 0x558a494b3370 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.654+0000 7fd4837fe640 1 -- 172.21.15.62:0/3829549033 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c09b140 con 0x558a494b3370 2024-09-11T23:54:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.655+0000 7fd4837fe640 1 -- 172.21.15.62:0/3829549033 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fd49c097530 con 0x558a494b3370 2024-09-11T23:54:20.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.655+0000 7fd4837fe640 1 -- 172.21.15.62:0/3829549033 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd49c0a4450 con 0x558a494b3370 2024-09-11T23:54:20.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.655+0000 7fd4aca27740 10 client.25430 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:20.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.655+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=27}) -- 0x558a495365e0 con 0x558a494b3370 2024-09-11T23:54:20.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.656+0000 7fd4837fe640 1 -- 172.21.15.62:0/3829549033 <== mon.2 v2:172.21.15.83:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd49c09ac30 con 0x558a494b3370 2024-09-11T23:54:20.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.656+0000 7fd4aca27740 10 client.25430 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:20.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.656+0000 7fd4aca27740 10 client.25430 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:54:20.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.657+0000 7fd4aca27740 20 client.25430 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:20.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.657+0000 7fd4aca27740 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x558a49536970 0x558a49538df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:20.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.657+0000 7fd4aca27740 4 client.25430 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:54:20.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.657+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd49c09a6a0 con 0x558a49536970 2024-09-11T23:54:20.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.657+0000 7fd4a3fff640 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x558a49536970 0x558a49538df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:20.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.660+0000 7fd4a3fff640 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x558a49536970 0x558a49538df0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69c80e58732d23d5 server_cookie=f2cf455bc06bebcb in_seq=0 out_seq=0 2024-09-11T23:54:20.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.660+0000 7fd4837fe640 10 client.25430 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:54:20.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.661+0000 7fd4837fe640 1 -- 172.21.15.62:0/3829549033 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd49c09a6a0 con 0x558a49536970 2024-09-11T23:54:20.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.661+0000 7fd4837fe640 10 client.25430 handle_command_reply: tid=0 2024-09-11T23:54:20.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.661+0000 7fd4aca27740 1 client.25430 shutdown 2024-09-11T23:54:20.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.662+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 msgr2=0x558a494ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:20.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.662+0000 7fd4aca27740 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494ad3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd49c005990 tx=0x7fd49c097bb0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.662+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 shutdown_connections 2024-09-11T23:54:20.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.662+0000 7fd4aca27740 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x558a49536970 0x558a49538df0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.662+0000 7fd4aca27740 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a494b3370 0x558a494ad3f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.662+0000 7fd4aca27740 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a494a9b40 0x558a494ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.662+0000 7fd4aca27740 1 --2- 172.21.15.62:0/3829549033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a494a85d0 0x558a494c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.662+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 >> 172.21.15.62:0/3829549033 conn(0x558a49471310 msgr2=0x558a494af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:20.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.668+0000 7fd4837fe640 0 client.25430 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:54:20.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.668+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 shutdown_connections 2024-09-11T23:54:20.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.672+0000 7fd4aca27740 1 -- 172.21.15.62:0/3829549033 wait complete. 2024-09-11T23:54:20.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.673+0000 7fd4aca27740 20 client.25430 trim_cache size 0 max 16384 2024-09-11T23:54:20.675 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:54:20.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.676+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2306144762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd48c0a30f0 msgr2=0x7fd48c0a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:20.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.676+0000 7fd4aa64e640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd48c0a30f0 0x7fd48c0a55b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x558a494a5ac0 tx=0x7fd494009260 comp rx=0 tx=0).stop 2024-09-11T23:54:20.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.676+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2306144762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a4106a70 msgr2=0x7fd4a410fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:20.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.676+0000 7fd4aa64e640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a4106a70 0x7fd4a410fc60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009fd0 tx=0x7fd49c00ef10 comp rx=0 tx=0).stop 2024-09-11T23:54:20.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.677+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2306144762 shutdown_connections 2024-09-11T23:54:20.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.677+0000 7fd4aa64e640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd48c0a30f0 0x7fd48c0a55b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.677+0000 7fd4aa64e640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a4106a70 0x7fd4a410fc60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.677+0000 7fd4aa64e640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4a4101410 0x7fd4a410f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.677+0000 7fd4aa64e640 1 --2- 172.21.15.62:0/2306144762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4a4100a70 0x7fd4a410f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.677+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2306144762 >> 172.21.15.62:0/2306144762 conn(0x7fd4a40fc790 msgr2=0x7fd4a40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.677+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2306144762 shutdown_connections 2024-09-11T23:54:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:20.677+0000 7fd4aa64e640 1 -- 172.21.15.62:0/2306144762 wait complete. 2024-09-11T23:54:20.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:20 smithi120 ceph-mon[35182]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-11T23:54:20.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:20 smithi083 ceph-mon[34805]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-09-11T23:54:21.885 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:54:22.277 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 100.0 s, 2399.0 tps, lat 132.595 ms stddev 1183.110 2024-09-11T23:54:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.793+0000 7f6ee77a7640 1 -- 172.21.15.62:0/571995598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0105ec0 msgr2=0x7f6ee0106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.793+0000 7f6ee77a7640 1 --2- 172.21.15.62:0/571995598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0105ec0 0x7f6ee0106340 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009be0 tx=0x7f6ed802a6e0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.794+0000 7f6ee77a7640 1 -- 172.21.15.62:0/571995598 shutdown_connections 2024-09-11T23:54:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.794+0000 7f6ee77a7640 1 --2- 172.21.15.62:0/571995598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0105ec0 0x7f6ee0106340 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.794+0000 7f6ee77a7640 1 --2- 172.21.15.62:0/571995598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee01011e0 0x7f6ee0105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.794+0000 7f6ee77a7640 1 --2- 172.21.15.62:0/571995598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ee01008c0 0x7f6ee0100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.794+0000 7f6ee77a7640 1 -- 172.21.15.62:0/571995598 >> 172.21.15.62:0/571995598 conn(0x7f6ee00fc7d0 msgr2=0x7f6ee00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.794+0000 7f6ee77a7640 1 -- 172.21.15.62:0/571995598 shutdown_connections 2024-09-11T23:54:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.794+0000 7f6ee77a7640 1 -- 172.21.15.62:0/571995598 wait complete. 2024-09-11T23:54:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee77a7640 1 Processor -- start 2024-09-11T23:54:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee77a7640 1 -- start start 2024-09-11T23:54:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee77a7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ee01008c0 0x7f6ee01a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee77a7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee01011e0 0x7f6ee01a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee77a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee0105ec0 0x7f6ee019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee77a7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee01186d0 con 0x7f6ee01011e0 2024-09-11T23:54:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee77a7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0118550 con 0x7f6ee01008c0 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee77a7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0118850 con 0x7f6ee0105ec0 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee5d44640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee0105ec0 0x7f6ee019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee5d44640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee0105ec0 0x7f6ee019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40776/0 (socket says 172.21.15.62:40776) 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee5d44640 1 -- 172.21.15.62:0/3408698162 learned_addr learned my addr 172.21.15.62:0/3408698162 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.795+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee01011e0 0x7f6ee01a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.796+0000 7f6ee5543640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ee01008c0 0x7f6ee01a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.796+0000 7f6ee5d44640 1 -- 172.21.15.62:0/3408698162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ee01008c0 msgr2=0x7f6ee01a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.796+0000 7f6ee5d44640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ee01008c0 0x7f6ee01a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.796+0000 7f6ee5d44640 1 -- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee01011e0 msgr2=0x7f6ee01a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.796+0000 7f6ee5d44640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee01011e0 0x7f6ee01a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.796+0000 7f6ee5d44640 1 -- 172.21.15.62:0/3408698162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee019f0c0 con 0x7f6ee0105ec0 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.796+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee01011e0 0x7f6ee01a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:54:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.796+0000 7f6ee5d44640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee0105ec0 0x7f6ee019e990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009b30 tx=0x7f6ed8030d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:22.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.797+0000 7f6ed27fc640 1 -- 172.21.15.62:0/3408698162 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8040020 con 0x7f6ee0105ec0 2024-09-11T23:54:22.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.797+0000 7f6ee77a7640 1 -- 172.21.15.62:0/3408698162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee019f350 con 0x7f6ee0105ec0 2024-09-11T23:54:22.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.797+0000 7f6ee77a7640 1 -- 172.21.15.62:0/3408698162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee01ab6a0 con 0x7f6ee0105ec0 2024-09-11T23:54:22.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.797+0000 7f6ed27fc640 1 -- 172.21.15.62:0/3408698162 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ed8031920 con 0x7f6ee0105ec0 2024-09-11T23:54:22.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.797+0000 7f6ed27fc640 1 -- 172.21.15.62:0/3408698162 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8036040 con 0x7f6ee0105ec0 2024-09-11T23:54:22.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.801+0000 7f6ed27fc640 1 -- 172.21.15.62:0/3408698162 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6ed8043020 con 0x7f6ee0105ec0 2024-09-11T23:54:22.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.802+0000 7f6ee9b80740 1 Processor -- start 2024-09-11T23:54:22.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.802+0000 7f6ee9b80740 1 -- start start 2024-09-11T23:54:22.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 0x563a50421090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50421a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a5042e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee9b80740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563a503872f0 con 0x563a50422b40 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee9b80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a5032ecf0 con 0x563a504215d0 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee9b80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563a503832b0 con 0x563a5042c370 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ed27fc640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6ecc0a2df0 0x7f6ecc0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee4d42640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 0x563a50421090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee5543640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50421a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee4d42640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a5042e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee5543640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50421a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:36240/0 (socket says 172.21.15.62:36240) 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ee5543640 1 -- 172.21.15.62:0/3824628464 learned_addr learned my addr 172.21.15.62:0/3824628464 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.803+0000 7f6ed27fc640 1 -- 172.21.15.62:0/3408698162 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6ed80f01d0 con 0x7f6ee0105ec0 2024-09-11T23:54:22.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.804+0000 7f6ee4d42640 1 -- 172.21.15.62:0/3824628464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 msgr2=0x563a50421a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:22.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.804+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/3824628464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50421a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.804+0000 7f6ee4d42640 1 -- 172.21.15.62:0/3824628464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 msgr2=0x563a50421090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:22.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.804+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/3824628464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 0x563a50421090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.804+0000 7f6ee5d44640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6ecc0a2df0 0x7f6ecc0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.804+0000 7f6ee4d42640 1 -- 172.21.15.62:0/3824628464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a503133a0 con 0x563a5042c370 2024-09-11T23:54:22.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.804+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/3824628464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a5042e760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ee01a0180 tx=0x7f6ed4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da73591aabf4035f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:22.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.804+0000 7f6ecb7fe640 1 -- 172.21.15.62:0/3824628464 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4012110 con 0x563a5042c370 2024-09-11T23:54:22.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.804+0000 7f6ecb7fe640 1 -- 172.21.15.62:0/3824628464 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ed4005500 con 0x563a5042c370 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.805+0000 7f6ecb7fe640 1 -- 172.21.15.62:0/3824628464 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4012110 con 0x563a5042c370 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.805+0000 7f6ee9b80740 1 -- 172.21.15.62:0/3824628464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 msgr2=0x563a5042e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.805+0000 7f6ee5d44640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6ecc0a2df0 0x7f6ecc0a52b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8006010 tx=0x7f6ed8030f40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.805+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/3824628464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a5042e760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ee01a0180 tx=0x7f6ed4009260 comp rx=0 tx=0).stop 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.806+0000 7f6ee9b80740 1 -- 172.21.15.62:0/3824628464 shutdown_connections 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.806+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/3824628464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a5042e760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.806+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/3824628464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50421a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.806+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/3824628464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 0x563a50421090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.806+0000 7f6ee9b80740 1 -- 172.21.15.62:0/3824628464 >> 172.21.15.62:0/3824628464 conn(0x563a503ea310 msgr2=0x563a5041dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.806+0000 7f6ee9b80740 1 -- 172.21.15.62:0/3824628464 shutdown_connections 2024-09-11T23:54:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.806+0000 7f6ee9b80740 1 -- 172.21.15.62:0/3824628464 wait complete. 2024-09-11T23:54:22.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.807+0000 7f6ee9b80740 1 -- start start 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.807+0000 7f6ee9b80740 1 Processor -- start 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.808+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a5042bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.808+0000 7f6ee4d42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a5042bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.808+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a50422b40 0x563a50428e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.808+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a5042c370 0x563a504293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.808+0000 7f6ee4d42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a5042bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:36254/0 (socket says 172.21.15.62:36254) 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.808+0000 7f6ee4d42640 1 -- 172.21.15.62:0/2804193597 learned_addr learned my addr 172.21.15.62:0/2804193597 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.808+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563a5042f0c0 con 0x563a5042c370 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.808+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a50342060 con 0x563a504215d0 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.808+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563a5042f240 con 0x563a50422b40 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.809+0000 7f6ee9b80740 10 client.0 _ll_register_callbacks cb 0x563a500958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:22.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.809+0000 7f6ee5543640 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a50422b40 0x563a50428e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.809+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a5042c370 0x563a504293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.809+0000 7f6ee4d42640 1 -- 172.21.15.62:0/2804193597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a50422b40 msgr2=0x563a50428e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.809+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a50422b40 0x563a50428e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.809+0000 7f6ee4d42640 1 -- 172.21.15.62:0/2804193597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a5042c370 msgr2=0x563a504293c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.809+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a5042c370 0x563a504293c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.809+0000 7f6ee4d42640 1 -- 172.21.15.62:0/2804193597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a500f83c0 con 0x563a504215d0 2024-09-11T23:54:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.810+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a5042bbd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4001eb0 tx=0x7f6ed4009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea97cf41fef471d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.810+0000 7f6ec8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:54:22.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.810+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/2804193597 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4012110 con 0x563a504215d0 2024-09-11T23:54:22.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.810+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/2804193597 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6ed4005230 con 0x563a504215d0 2024-09-11T23:54:22.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.810+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ed4005500 con 0x563a504215d0 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.810+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/2804193597 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6ed4031050 con 0x563a504215d0 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.810+0000 7f6ee9b80740 10 client.25439 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.810+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a5043aad0 con 0x563a504215d0 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.811+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/2804193597 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ed4028070 con 0x563a504215d0 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.811+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/2804193597 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6ed4020680 con 0x563a504215d0 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.811+0000 7f6ee9b80740 10 client.25439 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.811+0000 7f6ee9b80740 10 client.25439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.811+0000 7f6ee9b80740 20 client.25439 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.811+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563a5043be70 0x563a5043e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.811+0000 7f6ee9b80740 4 client.25439 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:54:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.811+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ed4005500 con 0x563a5043be70 2024-09-11T23:54:22.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.811+0000 7f6ee5543640 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563a5043be70 0x563a5043e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.812+0000 7f6ee5543640 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563a5043be70 0x563a5043e260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8dc965e30b5cf999 server_cookie=cb47852fdbd770e1 in_seq=0 out_seq=0 2024-09-11T23:54:22.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.812+0000 7f6ec8ff9640 10 client.25439 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:54:22.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.813+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/2804193597 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6ed4005500 con 0x563a5043be70 2024-09-11T23:54:22.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.813+0000 7f6ec8ff9640 10 client.25439 handle_command_reply: tid=0 2024-09-11T23:54:22.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.813+0000 7f6ee9b80740 1 client.25439 shutdown 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.813+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 msgr2=0x563a5042bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.813+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a5042bbd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4001eb0 tx=0x7f6ed4009480 comp rx=0 tx=0).stop 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.814+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 shutdown_connections 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.814+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563a5043be70 0x563a5043e260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.814+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a5042c370 0x563a504293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.814+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a50422b40 0x563a50428e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.814+0000 7f6ec8ff9640 0 client.25439 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.814+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/2804193597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a5042bbd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.814+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 >> 172.21.15.62:0/2804193597 conn(0x563a503ea310 msgr2=0x563a50423930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.814+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 shutdown_connections 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.815+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2804193597 wait complete. 2024-09-11T23:54:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.815+0000 7f6ee9b80740 20 client.25439 trim_cache size 0 max 16384 2024-09-11T23:54:22.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee9b80740 1 Processor -- start 2024-09-11T23:54:22.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee9b80740 1 -- start start 2024-09-11T23:54:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a504215d0 0x563a50424120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a50422b40 0x563a50424660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a5042d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee9b80740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec40110d0 con 0x563a504215d0 2024-09-11T23:54:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee9b80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec400e9a0 con 0x563a50422b40 2024-09-11T23:54:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee9b80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec400c7f0 con 0x563a5042c370 2024-09-11T23:54:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee5543640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a50422b40 0x563a50424660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee5543640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a50422b40 0x563a50424660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:36268/0 (socket says 172.21.15.62:36268) 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee5543640 1 -- 172.21.15.62:0/2247933522 learned_addr learned my addr 172.21.15.62:0/2247933522 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.836+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/2247933522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a504215d0 0x563a50424120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.837+0000 7f6ee5543640 1 -- 172.21.15.62:0/2247933522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 msgr2=0x563a5042d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.837+0000 7f6ee5543640 1 --2- 172.21.15.62:0/2247933522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a5042d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.837+0000 7f6ee5543640 1 -- 172.21.15.62:0/2247933522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a504215d0 msgr2=0x563a50424120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.837+0000 7f6ee5543640 1 --2- 172.21.15.62:0/2247933522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a504215d0 0x563a50424120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.837+0000 7f6ee5543640 1 -- 172.21.15.62:0/2247933522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a504273d0 con 0x563a50422b40 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.837+0000 7f6ee5543640 1 --2- 172.21.15.62:0/2247933522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a50422b40 0x563a50424660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x563a5041eac0 tx=0x7f6edc016800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d9bb0ad9a76fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.838+0000 7f6ecaffd640 1 -- 172.21.15.62:0/2247933522 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00dc60 con 0x563a50422b40 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.838+0000 7f6ecaffd640 1 -- 172.21.15.62:0/2247933522 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6edc00de00 con 0x563a50422b40 2024-09-11T23:54:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.838+0000 7f6ecaffd640 1 -- 172.21.15.62:0/2247933522 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc01c340 con 0x563a50422b40 2024-09-11T23:54:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.839+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2247933522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a50422b40 msgr2=0x563a50424660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.839+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/2247933522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a50422b40 0x563a50424660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x563a5041eac0 tx=0x7f6edc016800 comp rx=0 tx=0).stop 2024-09-11T23:54:22.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.839+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2247933522 shutdown_connections 2024-09-11T23:54:22.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.839+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/2247933522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a5042d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.839+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/2247933522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a50422b40 0x563a50424660 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.839+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/2247933522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a504215d0 0x563a50424120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.840+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2247933522 >> 172.21.15.62:0/2247933522 conn(0x563a503ea310 msgr2=0x563a5041cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.840+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2247933522 shutdown_connections 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.840+0000 7f6ee9b80740 1 -- 172.21.15.62:0/2247933522 wait complete. 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.840+0000 7f6ee9b80740 1 -- start start 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.841+0000 7f6ee9b80740 1 Processor -- start 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.842+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50434dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 0x563a50443390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee9b80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a50443a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee4d42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50434dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee4d42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50434dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:36280/0 (socket says 172.21.15.62:36280) 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee4d42640 1 -- 172.21.15.62:0/4073797639 learned_addr learned my addr 172.21.15.62:0/4073797639 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec400ce60 con 0x563a50422b40 2024-09-11T23:54:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec400e2d0 con 0x563a504215d0 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec400d530 con 0x563a5042c370 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a50443a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee9b80740 10 client.0 _ll_register_callbacks cb 0x7f6ed4009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee4d42640 1 -- 172.21.15.62:0/4073797639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 msgr2=0x563a50443a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a50443a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee5543640 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 0x563a50443390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee4d42640 1 -- 172.21.15.62:0/4073797639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 msgr2=0x563a50443390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 0x563a50443390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee4d42640 1 -- 172.21.15.62:0/4073797639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a504328a0 con 0x563a504215d0 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6edc00de00 con 0x563a504215d0 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee5543640 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 0x563a50443390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:54:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ee4d42640 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50434dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4010450 tx=0x7f6ed4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea03924678e61e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.843+0000 7f6ec8ff9640 10 client.25445 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/4073797639 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed403d060 con 0x563a504215d0 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/4073797639 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6ed4020440 con 0x563a504215d0 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/4073797639 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6ed4020640 con 0x563a504215d0 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ee9b80740 10 client.25445 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=27}) -- 0x563a504a6a20 con 0x563a504215d0 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/4073797639 <== mon.2 v2:172.21.15.83:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6ed400f790 con 0x563a504215d0 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ee9b80740 10 client.25445 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ee9b80740 10 client.25445 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ee9b80740 20 client.25445 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563a5043be70 0x563a504a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ee9b80740 4 client.25445 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:54:22.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.844+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a50394860 con 0x563a5043be70 2024-09-11T23:54:22.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.845+0000 7f6ee5543640 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563a5043be70 0x563a504a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:22.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.845+0000 7f6ee5543640 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563a5043be70 0x563a504a9120 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9edc504c28c8ca1c server_cookie=4f9bdc9444ef18af in_seq=0 out_seq=0 2024-09-11T23:54:22.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.845+0000 7f6ec8ff9640 10 client.25445 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:54:22.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.845+0000 7f6ec8ff9640 1 -- 172.21.15.62:0/4073797639 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a50394860 con 0x563a5043be70 2024-09-11T23:54:22.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.845+0000 7f6ec8ff9640 10 client.25445 handle_command_reply: tid=0 2024-09-11T23:54:22.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 client.25445 shutdown 2024-09-11T23:54:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 msgr2=0x563a50434dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50434dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4010450 tx=0x7f6ed4005fd0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 shutdown_connections 2024-09-11T23:54:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563a5043be70 0x563a504a9120 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563a5042c370 0x563a50443a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ec8ff9640 0 client.25445 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:54:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a50422b40 0x563a50443390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 --2- 172.21.15.62:0/4073797639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a504215d0 0x563a50434dc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 >> 172.21.15.62:0/4073797639 conn(0x563a503ea310 msgr2=0x563a5042a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:22.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 shutdown_connections 2024-09-11T23:54:22.851 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:54:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.846+0000 7f6ee9b80740 1 -- 172.21.15.62:0/4073797639 wait complete. 2024-09-11T23:54:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.847+0000 7f6ee9b80740 20 client.25445 trim_cache size 0 max 16384 2024-09-11T23:54:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.848+0000 7f6ee77a7640 1 -- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6ecc0a2df0 msgr2=0x7f6ecc0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.848+0000 7f6ee77a7640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6ecc0a2df0 0x7f6ecc0a52b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8006010 tx=0x7f6ed8030f40 comp rx=0 tx=0).stop 2024-09-11T23:54:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.848+0000 7f6ee77a7640 1 -- 172.21.15.62:0/3408698162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee0105ec0 msgr2=0x7f6ee019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.848+0000 7f6ee77a7640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee0105ec0 0x7f6ee019e990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009b30 tx=0x7f6ed8030d90 comp rx=0 tx=0).stop 2024-09-11T23:54:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.848+0000 7f6ee77a7640 1 -- 172.21.15.62:0/3408698162 shutdown_connections 2024-09-11T23:54:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.848+0000 7f6ee77a7640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6ecc0a2df0 0x7f6ecc0a52b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.848+0000 7f6ee77a7640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee0105ec0 0x7f6ee019e990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.848+0000 7f6ee77a7640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee01011e0 0x7f6ee01a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.848+0000 7f6ee77a7640 1 --2- 172.21.15.62:0/3408698162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ee01008c0 0x7f6ee01a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.848+0000 7f6ee77a7640 1 -- 172.21.15.62:0/3408698162 >> 172.21.15.62:0/3408698162 conn(0x7f6ee00fc7d0 msgr2=0x7f6ee010f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.849+0000 7f6ee77a7640 1 -- 172.21.15.62:0/3408698162 shutdown_connections 2024-09-11T23:54:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:22.849+0000 7f6ee77a7640 1 -- 172.21.15.62:0/3408698162 wait complete. 2024-09-11T23:54:23.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:23 smithi083 ceph-mon[34805]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-09-11T23:54:23.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:23 smithi062 ceph-mon[30958]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-09-11T23:54:23.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:23 smithi120 ceph-mon[35182]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-09-11T23:54:23.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:54:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.699+0000 7f9087b99640 1 -- 172.21.15.62:0/2472348028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9080076720 msgr2=0x7f9080076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.699+0000 7f9087b99640 1 --2- 172.21.15.62:0/2472348028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9080076720 0x7f9080076b00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f907c009900 tx=0x7f907c02a410 comp rx=0 tx=0).stop 2024-09-11T23:54:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.699+0000 7f9087b99640 1 -- 172.21.15.62:0/2472348028 shutdown_connections 2024-09-11T23:54:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.699+0000 7f9087b99640 1 --2- 172.21.15.62:0/2472348028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9080112a30 0x7f9080114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.699+0000 7f9087b99640 1 --2- 172.21.15.62:0/2472348028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9080077040 0x7f90801124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.699+0000 7f9087b99640 1 --2- 172.21.15.62:0/2472348028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9080076720 0x7f9080076b00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.699+0000 7f9087b99640 1 -- 172.21.15.62:0/2472348028 >> 172.21.15.62:0/2472348028 conn(0x7f90800ffab0 msgr2=0x7f9080101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.699+0000 7f9087b99640 1 -- 172.21.15.62:0/2472348028 shutdown_connections 2024-09-11T23:54:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.699+0000 7f9087b99640 1 -- 172.21.15.62:0/2472348028 wait complete. 2024-09-11T23:54:24.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.700+0000 7f9087b99640 1 Processor -- start 2024-09-11T23:54:24.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.701+0000 7f9087b99640 1 -- start start 2024-09-11T23:54:24.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.701+0000 7f9087b99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9080076720 0x7f90801a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.701+0000 7f9087b99640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9080077040 0x7f90801a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.701+0000 7f9085935640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9080076720 0x7f90801a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.701+0000 7f9085935640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9080076720 0x7f90801a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:36296/0 (socket says 172.21.15.62:36296) 2024-09-11T23:54:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.701+0000 7f9087b99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9080112a30 0x7f908019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.701+0000 7f9085935640 1 -- 172.21.15.62:0/1496195930 learned_addr learned my addr 172.21.15.62:0/1496195930 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.701+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9080118610 con 0x7f9080077040 2024-09-11T23:54:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.701+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9080118490 con 0x7f9080076720 2024-09-11T23:54:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.701+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9080118790 con 0x7f9080112a30 2024-09-11T23:54:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.702+0000 7f9086136640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9080112a30 0x7f908019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.702+0000 7f9085134640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9080077040 0x7f90801a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.702+0000 7f9085935640 1 -- 172.21.15.62:0/1496195930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9080112a30 msgr2=0x7f908019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.702+0000 7f9085935640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9080112a30 0x7f908019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.702+0000 7f9085935640 1 -- 172.21.15.62:0/1496195930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9080077040 msgr2=0x7f90801a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.702+0000 7f9085935640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9080077040 0x7f90801a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.702+0000 7f9085935640 1 -- 172.21.15.62:0/1496195930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908019efb0 con 0x7f9080076720 2024-09-11T23:54:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.702+0000 7f9085935640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9080076720 0x7f90801a46c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f907c02a980 tx=0x7f907c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.703+0000 7f9076ffd640 1 -- 172.21.15.62:0/1496195930 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c03f070 con 0x7f9080076720 2024-09-11T23:54:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.703+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908019f240 con 0x7f9080076720 2024-09-11T23:54:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.703+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90801ab590 con 0x7f9080076720 2024-09-11T23:54:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.703+0000 7f9076ffd640 1 -- 172.21.15.62:0/1496195930 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f907c031c90 con 0x7f9080076720 2024-09-11T23:54:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.703+0000 7f9076ffd640 1 -- 172.21.15.62:0/1496195930 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c035030 con 0x7f9080076720 2024-09-11T23:54:24.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.705+0000 7f9076ffd640 1 -- 172.21.15.62:0/1496195930 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f907c042070 con 0x7f9080076720 2024-09-11T23:54:24.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.706+0000 7f9089f72740 1 Processor -- start 2024-09-11T23:54:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.707+0000 7f9089f72740 1 -- start start 2024-09-11T23:54:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.707+0000 7f9076ffd640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f906c0a2dc0 0x7f906c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.707+0000 7f9086136640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f906c0a2dc0 0x7f906c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.707+0000 7f9089f72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc09090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.707+0000 7f9089f72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc09a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.707+0000 7f9089f72740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc14370 0x55e7ffc16760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.707+0000 7f9089f72740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffb6f2f0 con 0x55e7ffc14370 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.707+0000 7f9089f72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffb16cf0 con 0x55e7ffc0ab40 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.707+0000 7f9089f72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffb6b2b0 con 0x55e7ffc095d0 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9085134640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc09a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9085935640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc09090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9085134640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc09a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40838/0 (socket says 172.21.15.62:40838) 2024-09-11T23:54:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9085134640 1 -- 172.21.15.62:0/1821513113 learned_addr learned my addr 172.21.15.62:0/1821513113 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9085134640 1 -- 172.21.15.62:0/1821513113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 msgr2=0x55e7ffc09090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9085134640 1 --2- 172.21.15.62:0/1821513113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc09090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9085134640 1 -- 172.21.15.62:0/1821513113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc14370 msgr2=0x55e7ffc16760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9085134640 1 --2- 172.21.15.62:0/1821513113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc14370 0x55e7ffc16760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9085134640 1 -- 172.21.15.62:0/1821513113 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ffafb3a0 con 0x55e7ffc095d0 2024-09-11T23:54:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9076ffd640 1 -- 172.21.15.62:0/1496195930 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f907c0efac0 con 0x7f9080076720 2024-09-11T23:54:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.708+0000 7f9085134640 1 --2- 172.21.15.62:0/1821513113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc09a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9070009bf0 tx=0x7f9070010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e30c1098ce1000c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.709+0000 7f906bfff640 1 -- 172.21.15.62:0/1821513113 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070004020 con 0x55e7ffc095d0 2024-09-11T23:54:24.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.709+0000 7f906bfff640 1 -- 172.21.15.62:0/1821513113 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90700041c0 con 0x55e7ffc095d0 2024-09-11T23:54:24.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.709+0000 7f906bfff640 1 -- 172.21.15.62:0/1821513113 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070012690 con 0x55e7ffc095d0 2024-09-11T23:54:24.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.710+0000 7f9086136640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f906c0a2dc0 0x7f906c0a5280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55e7ffc0a070 tx=0x7f9078009a70 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:24.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.710+0000 7f9089f72740 1 -- 172.21.15.62:0/1821513113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 msgr2=0x55e7ffc09a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:24.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.710+0000 7f9089f72740 1 --2- 172.21.15.62:0/1821513113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc09a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9070009bf0 tx=0x7f9070010740 comp rx=0 tx=0).stop 2024-09-11T23:54:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.710+0000 7f9089f72740 1 -- 172.21.15.62:0/1821513113 shutdown_connections 2024-09-11T23:54:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.710+0000 7f9089f72740 1 --2- 172.21.15.62:0/1821513113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc14370 0x55e7ffc16760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.711+0000 7f9089f72740 1 --2- 172.21.15.62:0/1821513113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc09a10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.711+0000 7f9089f72740 1 --2- 172.21.15.62:0/1821513113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc09090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.711+0000 7f9089f72740 1 -- 172.21.15.62:0/1821513113 >> 172.21.15.62:0/1821513113 conn(0x55e7ffbd2310 msgr2=0x55e7ffc05c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.711+0000 7f9089f72740 1 -- 172.21.15.62:0/1821513113 shutdown_connections 2024-09-11T23:54:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.711+0000 7f9089f72740 1 -- 172.21.15.62:0/1821513113 wait complete. 2024-09-11T23:54:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.711+0000 7f9089f72740 1 -- start start 2024-09-11T23:54:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.711+0000 7f9089f72740 1 Processor -- start 2024-09-11T23:54:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9089f72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc13d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9089f72740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc11010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9089f72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc14370 0x55e7ffc11600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9089f72740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffc17070 con 0x55e7ffc0ab40 2024-09-11T23:54:24.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9089f72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffb2a060 con 0x55e7ffc14370 2024-09-11T23:54:24.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9089f72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffc171f0 con 0x55e7ffc095d0 2024-09-11T23:54:24.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9085935640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc13d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9085935640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc13d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40852/0 (socket says 172.21.15.62:40852) 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9085935640 1 -- 172.21.15.62:0/845420860 learned_addr learned my addr 172.21.15.62:0/845420860 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9086136640 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc14370 0x55e7ffc11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.712+0000 7f9089f72740 10 client.0 _ll_register_callbacks cb 0x55e7ff87d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f9086136640 1 -- 172.21.15.62:0/845420860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 msgr2=0x55e7ffc13d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f9086136640 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc13d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f9086136640 1 -- 172.21.15.62:0/845420860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc0ab40 msgr2=0x55e7ffc11010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f9086136640 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc11010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f9086136640 1 -- 172.21.15.62:0/845420860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ff8e03c0 con 0x55e7ffc14370 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f9085935640 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc13d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f9086136640 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc14370 0x55e7ffc11600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f908007aa10 tx=0x7f9078004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8670c30390657ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f90697fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f90697fa640 1 -- 172.21.15.62:0/845420860 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907800eb30 con 0x55e7ffc14370 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f90697fa640 1 -- 172.21.15.62:0/845420860 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f907800ecd0 con 0x55e7ffc14370 2024-09-11T23:54:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.713+0000 7f9089f72740 1 -- 172.21.15.62:0/845420860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90700041c0 con 0x55e7ffc14370 2024-09-11T23:54:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f90697fa640 1 -- 172.21.15.62:0/845420860 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9078026470 con 0x55e7ffc14370 2024-09-11T23:54:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f9089f72740 10 client.25457 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f9089f72740 1 -- 172.21.15.62:0/845420860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e7ffc22c30 con 0x55e7ffc14370 2024-09-11T23:54:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f90697fa640 1 -- 172.21.15.62:0/845420860 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90700041c0 con 0x55e7ffc14370 2024-09-11T23:54:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f90697fa640 1 -- 172.21.15.62:0/845420860 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9078019030 con 0x55e7ffc14370 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f9089f72740 10 client.25457 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f9089f72740 10 client.25457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f9089f72740 20 client.25457 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f9089f72740 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e7ffc24090 0x55e7ffc26550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f9089f72740 4 client.25457 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f9089f72740 1 -- 172.21.15.62:0/845420860 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ffb7c860 con 0x55e7ffc24090 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.714+0000 7f9085935640 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e7ffc24090 0x55e7ffc26550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.715+0000 7f9085935640 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e7ffc24090 0x55e7ffc26550 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19976495de6911c server_cookie=536b3b0cfc3814ec in_seq=0 out_seq=0 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.715+0000 7f90697fa640 10 client.25457 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.716+0000 7f90697fa640 1 -- 172.21.15.62:0/845420860 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7ffb7c860 con 0x55e7ffc24090 2024-09-11T23:54:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.716+0000 7f90697fa640 10 client.25457 handle_command_reply: tid=0 2024-09-11T23:54:24.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.716+0000 7f9089f72740 1 client.25457 shutdown 2024-09-11T23:54:24.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 1 -- 172.21.15.62:0/845420860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc14370 msgr2=0x55e7ffc11600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:24.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc14370 0x55e7ffc11600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f908007aa10 tx=0x7f9078004ab0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 1 -- 172.21.15.62:0/845420860 shutdown_connections 2024-09-11T23:54:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e7ffc24090 0x55e7ffc26550 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc14370 0x55e7ffc11600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f90697fa640 0 client.25457 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc11010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 1 --2- 172.21.15.62:0/845420860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc095d0 0x55e7ffc13d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 1 -- 172.21.15.62:0/845420860 >> 172.21.15.62:0/845420860 conn(0x55e7ffbd2310 msgr2=0x55e7ffc0b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 1 -- 172.21.15.62:0/845420860 shutdown_connections 2024-09-11T23:54:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 1 -- 172.21.15.62:0/845420860 wait complete. 2024-09-11T23:54:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.717+0000 7f9089f72740 20 client.25457 trim_cache size 0 max 16384 2024-09-11T23:54:24.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9089f72740 1 Processor -- start 2024-09-11T23:54:24.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9089f72740 1 -- start start 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9089f72740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc095d0 0x55e7ffc203a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9089f72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc0fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9089f72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc14370 0x55e7ffc15760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9089f72740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9064010fd0 con 0x55e7ffc095d0 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9089f72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f906400e8a0 con 0x55e7ffc0ab40 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9089f72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f906400c6f0 con 0x55e7ffc14370 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085935640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc095d0 0x55e7ffc203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085935640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc095d0 0x55e7ffc203a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54462/0 (socket says 172.21.15.62:54462) 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085134640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc0fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085935640 1 -- 172.21.15.62:0/88716247 learned_addr learned my addr 172.21.15.62:0/88716247 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9086136640 1 --2- 172.21.15.62:0/88716247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc14370 0x55e7ffc15760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085935640 1 -- 172.21.15.62:0/88716247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc14370 msgr2=0x55e7ffc15760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085935640 1 --2- 172.21.15.62:0/88716247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc14370 0x55e7ffc15760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085935640 1 -- 172.21.15.62:0/88716247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 msgr2=0x55e7ffc0fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085935640 1 --2- 172.21.15.62:0/88716247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc0fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085935640 1 -- 172.21.15.62:0/88716247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ffc117f0 con 0x55e7ffc095d0 2024-09-11T23:54:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085134640 1 --2- 172.21.15.62:0/88716247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc0fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.737+0000 7f9085935640 1 --2- 172.21.15.62:0/88716247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc095d0 0x55e7ffc203a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f907c006010 tx=0x7f907c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0508047a8ec6d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.738+0000 7f906b7fe640 1 -- 172.21.15.62:0/88716247 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c049040 con 0x55e7ffc095d0 2024-09-11T23:54:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.738+0000 7f906b7fe640 1 -- 172.21.15.62:0/88716247 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f907c0bd000 con 0x55e7ffc095d0 2024-09-11T23:54:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.738+0000 7f906b7fe640 1 -- 172.21.15.62:0/88716247 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c0b9670 con 0x55e7ffc095d0 2024-09-11T23:54:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.739+0000 7f9089f72740 1 -- 172.21.15.62:0/88716247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc095d0 msgr2=0x55e7ffc203a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.739+0000 7f9089f72740 1 --2- 172.21.15.62:0/88716247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc095d0 0x55e7ffc203a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f907c006010 tx=0x7f907c0dc040 comp rx=0 tx=0).stop 2024-09-11T23:54:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.740+0000 7f9089f72740 1 -- 172.21.15.62:0/88716247 shutdown_connections 2024-09-11T23:54:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.740+0000 7f9089f72740 1 --2- 172.21.15.62:0/88716247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc14370 0x55e7ffc15760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.740+0000 7f9089f72740 1 --2- 172.21.15.62:0/88716247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc0fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.740+0000 7f9089f72740 1 --2- 172.21.15.62:0/88716247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc095d0 0x55e7ffc203a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.740+0000 7f9089f72740 1 -- 172.21.15.62:0/88716247 >> 172.21.15.62:0/88716247 conn(0x55e7ffbd2310 msgr2=0x55e7ffc0f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:24.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.741+0000 7f9089f72740 1 -- 172.21.15.62:0/88716247 shutdown_connections 2024-09-11T23:54:24.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.741+0000 7f9089f72740 1 -- 172.21.15.62:0/88716247 wait complete. 2024-09-11T23:54:24.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.741+0000 7f9089f72740 1 -- start start 2024-09-11T23:54:24.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.741+0000 7f9089f72740 1 Processor -- start 2024-09-11T23:54:24.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9089f72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc095d0 0x55e7ffc2b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9089f72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc2b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9085935640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc095d0 0x55e7ffc2b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9085935640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc095d0 0x55e7ffc2b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:36336/0 (socket says 172.21.15.62:36336) 2024-09-11T23:54:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9085935640 1 -- 172.21.15.62:0/1936582641 learned_addr learned my addr 172.21.15.62:0/1936582641 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9085134640 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc2b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9089f72740 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc14370 0x55e7ffc1b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f906400cd60 con 0x55e7ffc14370 2024-09-11T23:54:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f906400e1d0 con 0x55e7ffc095d0 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f906400d430 con 0x55e7ffc0ab40 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.742+0000 7f9089f72740 10 client.0 _ll_register_callbacks cb 0x7f9078012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.743+0000 7f9085935640 1 -- 172.21.15.62:0/1936582641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc0ab40 msgr2=0x55e7ffc2b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.743+0000 7f9085935640 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc2b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.743+0000 7f9085935640 1 -- 172.21.15.62:0/1936582641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc14370 msgr2=0x55e7ffc1b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.743+0000 7f9085935640 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc14370 0x55e7ffc1b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.743+0000 7f9085935640 1 -- 172.21.15.62:0/1936582641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ffc1ba50 con 0x55e7ffc095d0 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.743+0000 7f9085134640 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc2b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.743+0000 7f9085935640 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc095d0 0x55e7ffc2b3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f907c02aa00 tx=0x7f907c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7684932c19a3d088 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.743+0000 7f90697fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.743+0000 7f90697fa640 1 -- 172.21.15.62:0/1936582641 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c049050 con 0x55e7ffc095d0 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.743+0000 7f90697fa640 1 -- 172.21.15.62:0/1936582641 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f907c0bb0a0 con 0x55e7ffc095d0 2024-09-11T23:54:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f907c0bd000 con 0x55e7ffc095d0 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f90697fa640 1 -- 172.21.15.62:0/1936582641 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f907c03b460 con 0x55e7ffc095d0 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f9089f72740 10 client.25463 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e7ffc1ccd0 con 0x55e7ffc095d0 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f90697fa640 1 -- 172.21.15.62:0/1936582641 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f907c0bd000 con 0x55e7ffc095d0 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f90697fa640 1 -- 172.21.15.62:0/1936582641 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f907c0bd260 con 0x55e7ffc095d0 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f9089f72740 10 client.25463 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f9089f72740 10 client.25463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f9089f72740 20 client.25463 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f9089f72740 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e7ffc24090 0x55e7ffc90eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f9089f72740 4 client.25463 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.744+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7ffb76d70 con 0x55e7ffc24090 2024-09-11T23:54:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.745+0000 7f9085134640 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e7ffc24090 0x55e7ffc90eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:24.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f9085134640 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e7ffc24090 0x55e7ffc90eb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480aea4fe90ffb9f server_cookie=d85443efa722e1c4 in_seq=0 out_seq=0 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f90697fa640 10 client.25463 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f90697fa640 1 -- 172.21.15.62:0/1936582641 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7ffb76d70 con 0x55e7ffc24090 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f90697fa640 10 client.25463 handle_command_reply: tid=0 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f9089f72740 1 client.25463 shutdown 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc095d0 msgr2=0x55e7ffc2b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f9089f72740 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc095d0 0x55e7ffc2b3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f907c02aa00 tx=0x7f907c009260 comp rx=0 tx=0).stop 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 shutdown_connections 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f9089f72740 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e7ffc24090 0x55e7ffc90eb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f9089f72740 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7ffc14370 0x55e7ffc1b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f9089f72740 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7ffc0ab40 0x55e7ffc2b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f9089f72740 1 --2- 172.21.15.62:0/1936582641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ffc095d0 0x55e7ffc2b3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.747+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 >> 172.21.15.62:0/1936582641 conn(0x55e7ffbd2310 msgr2=0x55e7ffc10100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:24.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.748+0000 7f90697fa640 0 client.25463 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:24.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.748+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 shutdown_connections 2024-09-11T23:54:24.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.748+0000 7f9089f72740 1 -- 172.21.15.62:0/1936582641 wait complete. 2024-09-11T23:54:24.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.748+0000 7f9089f72740 20 client.25463 trim_cache size 0 max 16384 2024-09-11T23:54:24.749 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:54:24.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f906c0a2dc0 msgr2=0x7f906c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:24.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f906c0a2dc0 0x7f906c0a5280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55e7ffc0a070 tx=0x7f9078009a70 comp rx=0 tx=0).stop 2024-09-11T23:54:24.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9080076720 msgr2=0x7f90801a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:24.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9080076720 0x7f90801a46c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f907c02a980 tx=0x7f907c003b90 comp rx=0 tx=0).stop 2024-09-11T23:54:24.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 shutdown_connections 2024-09-11T23:54:24.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f906c0a2dc0 0x7f906c0a5280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9080112a30 0x7f908019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9080077040 0x7f90801a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 --2- 172.21.15.62:0/1496195930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9080076720 0x7f90801a46c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:24.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 >> 172.21.15.62:0/1496195930 conn(0x7f90800ffab0 msgr2=0x7f9080101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:24.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 shutdown_connections 2024-09-11T23:54:24.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:24.749+0000 7f9087b99640 1 -- 172.21.15.62:0/1496195930 wait complete. 2024-09-11T23:54:25.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:25 smithi083 ceph-mon[34805]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-09-11T23:54:25.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:25 smithi062 ceph-mon[30958]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-09-11T23:54:25.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:25 smithi120 ceph-mon[35182]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-09-11T23:54:26.570 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:54:26.570 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:54:26.944 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:27.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:27 smithi062 ceph-mon[30958]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-09-11T23:54:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:27 smithi083 ceph-mon[34805]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-09-11T23:54:27.421 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:27 smithi120 ceph-mon[35182]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-09-11T23:54:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 105.0 s, 3338.2 tps, lat 29.764 ms stddev 30.035 2024-09-11T23:54:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.463+0000 7f931da4e640 1 -- 172.21.15.62:0/890276612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9318105ec0 msgr2=0x7f9318106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.463+0000 7f931da4e640 1 --2- 172.21.15.62:0/890276612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9318105ec0 0x7f9318106340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9310009cd0 tx=0x7f931002a8f0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.463+0000 7f931da4e640 1 -- 172.21.15.62:0/890276612 shutdown_connections 2024-09-11T23:54:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.463+0000 7f931da4e640 1 --2- 172.21.15.62:0/890276612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9318105ec0 0x7f9318106340 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.463+0000 7f931da4e640 1 --2- 172.21.15.62:0/890276612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93181011e0 0x7f9318105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.463+0000 7f931da4e640 1 --2- 172.21.15.62:0/890276612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93181008c0 0x7f9318100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.463+0000 7f931da4e640 1 -- 172.21.15.62:0/890276612 >> 172.21.15.62:0/890276612 conn(0x7f93180fc7d0 msgr2=0x7f93180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.463+0000 7f931da4e640 1 -- 172.21.15.62:0/890276612 shutdown_connections 2024-09-11T23:54:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.463+0000 7f931da4e640 1 -- 172.21.15.62:0/890276612 wait complete. 2024-09-11T23:54:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.464+0000 7f931da4e640 1 Processor -- start 2024-09-11T23:54:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.464+0000 7f931da4e640 1 -- start start 2024-09-11T23:54:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.464+0000 7f931da4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93181008c0 0x7f93181a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.464+0000 7f931da4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93181011e0 0x7f93181a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.464+0000 7f931da4e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9318105ec0 0x7f931819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.464+0000 7f931da4e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9318118660 con 0x7f9318105ec0 2024-09-11T23:54:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.464+0000 7f931da4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93181184e0 con 0x7f93181011e0 2024-09-11T23:54:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.464+0000 7f931da4e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93181187e0 con 0x7f93181008c0 2024-09-11T23:54:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.465+0000 7f9316ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93181008c0 0x7f93181a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.465+0000 7f9316ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93181008c0 0x7f93181a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40880/0 (socket says 172.21.15.62:40880) 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.465+0000 7f9316ffd640 1 -- 172.21.15.62:0/923618545 learned_addr learned my addr 172.21.15.62:0/923618545 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.465+0000 7f93167fc640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93181011e0 0x7f93181a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.465+0000 7f93167fc640 1 -- 172.21.15.62:0/923618545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93181008c0 msgr2=0x7f93181a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.465+0000 7f93167fc640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93181008c0 0x7f93181a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.465+0000 7f93167fc640 1 -- 172.21.15.62:0/923618545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9318105ec0 msgr2=0x7f931819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.465+0000 7f93167fc640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9318105ec0 0x7f931819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.465+0000 7f93167fc640 1 -- 172.21.15.62:0/923618545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931819f060 con 0x7f93181011e0 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.465+0000 7f93167fc640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93181011e0 0x7f93181a4cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f930c00d6d0 tx=0x7f930c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.466+0000 7f931ca4c640 1 -- 172.21.15.62:0/923618545 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c017070 con 0x7f93181011e0 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.466+0000 7f931ca4c640 1 -- 172.21.15.62:0/923618545 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c009820 con 0x7f93181011e0 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.466+0000 7f931ca4c640 1 -- 172.21.15.62:0/923618545 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c004d00 con 0x7f93181011e0 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.466+0000 7f931da4e640 1 -- 172.21.15.62:0/923618545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931819f350 con 0x7f93181011e0 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.467+0000 7f931da4e640 1 -- 172.21.15.62:0/923618545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93181ab640 con 0x7f93181011e0 2024-09-11T23:54:27.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.470+0000 7f931fe27740 1 Processor -- start 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.471+0000 7f931ca4c640 1 -- 172.21.15.62:0/923618545 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f930c005220 con 0x7f93181011e0 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.471+0000 7f931fe27740 1 -- start start 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.471+0000 7f931fe27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e42f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.471+0000 7f931fe27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.471+0000 7f931fe27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e50130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.471+0000 7f931fe27740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562b96da8280 con 0x562b96e4dd40 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.471+0000 7f931fe27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562b96d4fb10 con 0x562b96e43480 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.471+0000 7f931fe27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562b96da4240 con 0x562b96e42b60 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e42f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e42f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40896/0 (socket says 172.21.15.62:40896) 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93177fe640 1 -- 172.21.15.62:0/451051990 learned_addr learned my addr 172.21.15.62:0/451051990 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:27.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f9316ffd640 1 --2- 172.21.15.62:0/451051990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93177fe640 1 -- 172.21.15.62:0/451051990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 msgr2=0x562b96e4d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93177fe640 1 --2- 172.21.15.62:0/451051990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93177fe640 1 -- 172.21.15.62:0/451051990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 msgr2=0x562b96e50130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93167fc640 1 --2- 172.21.15.62:0/451051990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e50130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93177fe640 1 --2- 172.21.15.62:0/451051990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e50130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93177fe640 1 -- 172.21.15.62:0/451051990 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b96d5c1f0 con 0x562b96e42b60 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93167fc640 1 --2- 172.21.15.62:0/451051990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e50130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.472+0000 7f93177fe640 1 --2- 172.21.15.62:0/451051990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e42f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9310004af0 tx=0x7f9310009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23d79252ac312b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.473+0000 7f92fdffb640 1 -- 172.21.15.62:0/451051990 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310030620 con 0x562b96e42b60 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.473+0000 7f92fdffb640 1 -- 172.21.15.62:0/451051990 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931003f070 con 0x562b96e42b60 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.473+0000 7f92fdffb640 1 -- 172.21.15.62:0/451051990 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93100042a0 con 0x562b96e42b60 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.473+0000 7f931ca4c640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f93000a2ec0 0x7f93000a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f931ca4c640 1 -- 172.21.15.62:0/923618545 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f930c0cee90 con 0x7f93181011e0 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f931fe27740 1 -- 172.21.15.62:0/451051990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 msgr2=0x562b96e42f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f931fe27740 1 --2- 172.21.15.62:0/451051990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e42f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9310004af0 tx=0x7f9310009a20 comp rx=0 tx=0).stop 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f931fe27740 1 -- 172.21.15.62:0/451051990 shutdown_connections 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f931fe27740 1 --2- 172.21.15.62:0/451051990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e50130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f931fe27740 1 --2- 172.21.15.62:0/451051990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f931fe27740 1 --2- 172.21.15.62:0/451051990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e42f40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f931fe27740 1 -- 172.21.15.62:0/451051990 >> 172.21.15.62:0/451051990 conn(0x562b96e3f0f0 msgr2=0x562b96e3f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f931fe27740 1 -- 172.21.15.62:0/451051990 shutdown_connections 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f931fe27740 1 -- 172.21.15.62:0/451051990 wait complete. 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.474+0000 7f9316ffd640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f93000a2ec0 0x7f93000a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f931fe27740 1 -- start start 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f9316ffd640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f93000a2ec0 0x7f93000a5380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x562b96e3ff50 tx=0x7f9308005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f931fe27740 1 Processor -- start 2024-09-11T23:54:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f931fe27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e4abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f931fe27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f931fe27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e4b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f931fe27740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562b96e50960 con 0x562b96e4dd40 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f931fe27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562b96d62f10 con 0x562b96e43480 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f931fe27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562b96e50ae0 con 0x562b96e42b60 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f931fe27740 10 client.0 _ll_register_callbacks cb 0x562b96d2ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f9316ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e4abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f93167fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f93167fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:36370/0 (socket says 172.21.15.62:36370) 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f93167fc640 1 -- 172.21.15.62:0/1237008359 learned_addr learned my addr 172.21.15.62:0/1237008359 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.476+0000 7f9316ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e4abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40908/0 (socket says 172.21.15.62:40908) 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.477+0000 7f93177fe640 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e4b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.477+0000 7f93167fc640 1 -- 172.21.15.62:0/1237008359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 msgr2=0x562b96e4abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.477+0000 7f93167fc640 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e4abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.477+0000 7f93167fc640 1 -- 172.21.15.62:0/1237008359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 msgr2=0x562b96e4b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.477+0000 7f93167fc640 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e4b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.477+0000 7f93167fc640 1 -- 172.21.15.62:0/1237008359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b96b196a0 con 0x562b96e43480 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.478+0000 7f93167fc640 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4b100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f930c009610 tx=0x7f930c09b240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac1587b242ac0d28 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.478+0000 7f92f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.479+0000 7f92f6ffd640 1 -- 172.21.15.62:0/1237008359 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c020040 con 0x562b96e43480 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.479+0000 7f92f6ffd640 1 -- 172.21.15.62:0/1237008359 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f930c0cc830 con 0x562b96e43480 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.479+0000 7f931fe27740 1 -- 172.21.15.62:0/1237008359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f931003f070 con 0x562b96e43480 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.479+0000 7f92f6ffd640 1 -- 172.21.15.62:0/1237008359 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f930c09b380 con 0x562b96e43480 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.479+0000 7f931fe27740 10 client.25475 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.479+0000 7f931fe27740 1 -- 172.21.15.62:0/1237008359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b96e5c4b0 con 0x562b96e43480 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.479+0000 7f92f6ffd640 1 -- 172.21.15.62:0/1237008359 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c09f730 con 0x562b96e43480 2024-09-11T23:54:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.480+0000 7f92f6ffd640 1 -- 172.21.15.62:0/1237008359 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f930c0099d0 con 0x562b96e43480 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.480+0000 7f931fe27740 10 client.25475 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.480+0000 7f931fe27740 10 client.25475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.480+0000 7f931fe27740 20 client.25475 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.480+0000 7f931fe27740 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562b96e5d910 0x562b96e5fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.480+0000 7f931fe27740 4 client.25475 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.480+0000 7f931fe27740 1 -- 172.21.15.62:0/1237008359 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f931003f070 con 0x562b96e5d910 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.480+0000 7f9316ffd640 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562b96e5d910 0x562b96e5fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.481+0000 7f9316ffd640 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562b96e5d910 0x562b96e5fd30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697e45499bf65833 server_cookie=a0846fe62b8dbc53 in_seq=0 out_seq=0 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.481+0000 7f92f6ffd640 10 client.25475 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.482+0000 7f92f6ffd640 1 -- 172.21.15.62:0/1237008359 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f931003f070 con 0x562b96e5d910 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.482+0000 7f92f6ffd640 10 client.25475 handle_command_reply: tid=0 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.482+0000 7f931fe27740 1 client.25475 shutdown 2024-09-11T23:54:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.482+0000 7f931fe27740 1 -- 172.21.15.62:0/1237008359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 msgr2=0x562b96e4b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.483+0000 7f931fe27740 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4b100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f930c009610 tx=0x7f930c09b240 comp rx=0 tx=0).stop 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.483+0000 7f931fe27740 1 -- 172.21.15.62:0/1237008359 shutdown_connections 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.483+0000 7f931fe27740 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562b96e5d910 0x562b96e5fd30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.483+0000 7f931fe27740 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e4b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.483+0000 7f931fe27740 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4b100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.483+0000 7f931fe27740 1 --2- 172.21.15.62:0/1237008359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e4abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.483+0000 7f92f6ffd640 0 client.25475 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.483+0000 7f931fe27740 1 -- 172.21.15.62:0/1237008359 >> 172.21.15.62:0/1237008359 conn(0x562b96e3f0f0 msgr2=0x562b96e44e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.483+0000 7f931fe27740 1 -- 172.21.15.62:0/1237008359 shutdown_connections 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.484+0000 7f931fe27740 1 -- 172.21.15.62:0/1237008359 wait complete. 2024-09-11T23:54:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.484+0000 7f931fe27740 20 client.25475 trim_cache size 0 max 16384 2024-09-11T23:54:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.507+0000 7f931fe27740 1 Processor -- start 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.507+0000 7f931fe27740 1 -- start start 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.507+0000 7f931fe27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e4b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.507+0000 7f931fe27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.507+0000 7f931fe27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e65600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f931fe27740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f93181b6890 con 0x562b96e4dd40 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f931fe27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93181b4160 con 0x562b96e43480 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f931fe27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93181b1fb0 con 0x562b96e42b60 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f93177fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f93177fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e65600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54522/0 (socket says 172.21.15.62:54522) 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f93177fe640 1 -- 172.21.15.62:0/271683314 learned_addr learned my addr 172.21.15.62:0/271683314 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f9316ffd640 1 --2- 172.21.15.62:0/271683314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e4b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f93167fc640 1 --2- 172.21.15.62:0/271683314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f93177fe640 1 -- 172.21.15.62:0/271683314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 msgr2=0x562b96e4b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f93177fe640 1 --2- 172.21.15.62:0/271683314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e4b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f93177fe640 1 -- 172.21.15.62:0/271683314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 msgr2=0x562b96e4ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f93177fe640 1 --2- 172.21.15.62:0/271683314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f93177fe640 1 -- 172.21.15.62:0/271683314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b96e3ef20 con 0x562b96e4dd40 2024-09-11T23:54:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.508+0000 7f93177fe640 1 --2- 172.21.15.62:0/271683314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e65600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9310043000 tx=0x7f9310030450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad37f991b6fe30c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.509+0000 7f92fd7fa640 1 -- 172.21.15.62:0/271683314 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310003b70 con 0x562b96e4dd40 2024-09-11T23:54:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.509+0000 7f92fd7fa640 1 -- 172.21.15.62:0/271683314 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931003f040 con 0x562b96e4dd40 2024-09-11T23:54:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.509+0000 7f92fd7fa640 1 -- 172.21.15.62:0/271683314 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93100045c0 con 0x562b96e4dd40 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.510+0000 7f931fe27740 1 -- 172.21.15.62:0/271683314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 msgr2=0x562b96e65600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.510+0000 7f931fe27740 1 --2- 172.21.15.62:0/271683314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e65600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9310043000 tx=0x7f9310030450 comp rx=0 tx=0).stop 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.510+0000 7f931fe27740 1 -- 172.21.15.62:0/271683314 shutdown_connections 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.510+0000 7f931fe27740 1 --2- 172.21.15.62:0/271683314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e4dd40 0x562b96e65600 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.510+0000 7f931fe27740 1 --2- 172.21.15.62:0/271683314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e43480 0x562b96e4ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.510+0000 7f931fe27740 1 --2- 172.21.15.62:0/271683314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e42b60 0x562b96e4b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.510+0000 7f931fe27740 1 -- 172.21.15.62:0/271683314 >> 172.21.15.62:0/271683314 conn(0x562b96e3f0f0 msgr2=0x562b96e4c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.511+0000 7f931fe27740 1 -- 172.21.15.62:0/271683314 shutdown_connections 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.511+0000 7f931fe27740 1 -- 172.21.15.62:0/271683314 wait complete. 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.511+0000 7f931fe27740 1 -- start start 2024-09-11T23:54:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.511+0000 7f931fe27740 1 Processor -- start 2024-09-11T23:54:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.511+0000 7f931fe27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e42b60 0x562b96e55f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.511+0000 7f931fe27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e43480 0x562b96e56480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f931fe27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e4dd40 0x562b96e52d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f931fe27740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f93181b2620 con 0x562b96e43480 2024-09-11T23:54:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f931fe27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93181b3a90 con 0x562b96e42b60 2024-09-11T23:54:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f931fe27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93181b2cf0 con 0x562b96e4dd40 2024-09-11T23:54:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f931fe27740 10 client.0 _ll_register_callbacks cb 0x7f930c005810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f93167fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e43480 0x562b96e56480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f93167fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e43480 0x562b96e56480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54536/0 (socket says 172.21.15.62:54536) 2024-09-11T23:54:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f93167fc640 1 -- 172.21.15.62:0/3122293393 learned_addr learned my addr 172.21.15.62:0/3122293393 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f9316ffd640 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e42b60 0x562b96e55f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f93177fe640 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e4dd40 0x562b96e52d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f9316ffd640 1 -- 172.21.15.62:0/3122293393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e4dd40 msgr2=0x562b96e52d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f9316ffd640 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e4dd40 0x562b96e52d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f9316ffd640 1 -- 172.21.15.62:0/3122293393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e43480 msgr2=0x562b96e56480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f9316ffd640 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e43480 0x562b96e56480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f9316ffd640 1 -- 172.21.15.62:0/3122293393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b96e53560 con 0x562b96e42b60 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f93177fe640 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e4dd40 0x562b96e52d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f93167fc640 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e43480 0x562b96e56480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f9316ffd640 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e42b60 0x562b96e55f40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x562b96e5dcf0 tx=0x7f93080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=388a20c286b1bd04 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f92f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f92f6ffd640 1 -- 172.21.15.62:0/3122293393 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308003850 con 0x562b96e42b60 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.512+0000 7f92f6ffd640 1 -- 172.21.15.62:0/3122293393 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f93080039f0 con 0x562b96e42b60 2024-09-11T23:54:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f931fe27740 1 -- 172.21.15.62:0/3122293393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f931003f040 con 0x562b96e42b60 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f92f6ffd640 1 -- 172.21.15.62:0/3122293393 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9308003bf0 con 0x562b96e42b60 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f931fe27740 10 client.25481 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f931fe27740 1 -- 172.21.15.62:0/3122293393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b96ec90e0 con 0x562b96e42b60 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f92f6ffd640 1 -- 172.21.15.62:0/3122293393 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931003f040 con 0x562b96e42b60 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f92f6ffd640 1 -- 172.21.15.62:0/3122293393 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9308035430 con 0x562b96e42b60 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f931fe27740 10 client.25481 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f931fe27740 10 client.25481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f931fe27740 20 client.25481 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f931fe27740 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562b96ec97e0 0x562b96ecbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f931fe27740 4 client.25481 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f931fe27740 1 -- 172.21.15.62:0/3122293393 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562b96db57f0 con 0x562b96ec97e0 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.513+0000 7f93167fc640 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562b96ec97e0 0x562b96ecbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.514+0000 7f92f6ffd640 10 client.25481 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.514+0000 7f93167fc640 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562b96ec97e0 0x562b96ecbc00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc26fb3595eeda1 server_cookie=8503bf605e05c5bb in_seq=0 out_seq=0 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.515+0000 7f92f6ffd640 1 -- 172.21.15.62:0/3122293393 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562b96db57f0 con 0x562b96ec97e0 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.515+0000 7f92f6ffd640 10 client.25481 handle_command_reply: tid=0 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 client.25481 shutdown 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 -- 172.21.15.62:0/3122293393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e42b60 msgr2=0x562b96e55f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e42b60 0x562b96e55f40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x562b96e5dcf0 tx=0x7f93080046c0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 -- 172.21.15.62:0/3122293393 shutdown_connections 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562b96ec97e0 0x562b96ecbc00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b96e4dd40 0x562b96e52d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f92f6ffd640 0 client.25481 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b96e43480 0x562b96e56480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 --2- 172.21.15.62:0/3122293393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b96e42b60 0x562b96e55f40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 -- 172.21.15.62:0/3122293393 >> 172.21.15.62:0/3122293393 conn(0x562b96e3f0f0 msgr2=0x562b96e59400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 -- 172.21.15.62:0/3122293393 shutdown_connections 2024-09-11T23:54:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.516+0000 7f931fe27740 1 -- 172.21.15.62:0/3122293393 wait complete. 2024-09-11T23:54:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931fe27740 20 client.25481 trim_cache size 0 max 16384 2024-09-11T23:54:27.522 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:54:27.522 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:54:27.522 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "47edb652-470c-445b-8618-bdf26f24671a", 2024-09-11T23:54:27.522 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:54:27.522 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:54:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931da4e640 1 -- 172.21.15.62:0/923618545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f93000a2ec0 msgr2=0x7f93000a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931da4e640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f93000a2ec0 0x7f93000a5380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x562b96e3ff50 tx=0x7f9308005f90 comp rx=0 tx=0).stop 2024-09-11T23:54:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931da4e640 1 -- 172.21.15.62:0/923618545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93181011e0 msgr2=0x7f93181a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931da4e640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93181011e0 0x7f93181a4cb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f930c00d6d0 tx=0x7f930c00db90 comp rx=0 tx=0).stop 2024-09-11T23:54:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931da4e640 1 -- 172.21.15.62:0/923618545 shutdown_connections 2024-09-11T23:54:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931da4e640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f93000a2ec0 0x7f93000a5380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931da4e640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9318105ec0 0x7f931819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931da4e640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93181011e0 0x7f93181a4cb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931da4e640 1 --2- 172.21.15.62:0/923618545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93181008c0 0x7f93181a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.517+0000 7f931da4e640 1 -- 172.21.15.62:0/923618545 >> 172.21.15.62:0/923618545 conn(0x7f93180fc7d0 msgr2=0x7f9318103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.518+0000 7f931da4e640 1 -- 172.21.15.62:0/923618545 shutdown_connections 2024-09-11T23:54:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:27.518+0000 7f931da4e640 1 -- 172.21.15.62:0/923618545 wait complete. 2024-09-11T23:54:28.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:54:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:28 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:54:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:28 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:28 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:54:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:28 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:54:28.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:28 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:28 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:54:28.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:28 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:28 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:28 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:54:28.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:28 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:28 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:54:28.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:28 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:54:28.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:28 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:28 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:54:28.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:28 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:28 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.561 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:28.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:28 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:54:28.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:28 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:28 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:54:28.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:28 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:54:28.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:28 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:28 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:54:28.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:28 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:28.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:28 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:54:29.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.122+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1076854702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1b81091f0 msgr2=0x7fc1b81095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.122+0000 7fc1bc9e0640 1 --2- 172.21.15.62:0/1076854702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1b81091f0 0x7fc1b81095d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009900 tx=0x7fc1a002a480 comp rx=0 tx=0).stop 2024-09-11T23:54:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.123+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1076854702 shutdown_connections 2024-09-11T23:54:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.123+0000 7fc1bc9e0640 1 --2- 172.21.15.62:0/1076854702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b8114ab0 0x7fc1b8116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.123+0000 7fc1bc9e0640 1 --2- 172.21.15.62:0/1076854702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8109b10 0x7fc1b8114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.123+0000 7fc1bc9e0640 1 --2- 172.21.15.62:0/1076854702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1b81091f0 0x7fc1b81095d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.123+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1076854702 >> 172.21.15.62:0/1076854702 conn(0x7fc1b806e140 msgr2=0x7fc1b806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.123+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1076854702 shutdown_connections 2024-09-11T23:54:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.123+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1076854702 wait complete. 2024-09-11T23:54:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.124+0000 7fc1bc9e0640 1 Processor -- start 2024-09-11T23:54:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.124+0000 7fc1bc9e0640 1 -- start start 2024-09-11T23:54:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.124+0000 7fc1bc9e0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b81091f0 0x7fc1b81108a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.124+0000 7fc1bc9e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8109b10 0x7fc1b8110de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.124+0000 7fc1bc9e0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1b8114ab0 0x7fc1b8111320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.124+0000 7fc1bc9e0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b810b100 con 0x7fc1b8114ab0 2024-09-11T23:54:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.124+0000 7fc1bc9e0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b810af80 con 0x7fc1b8109b10 2024-09-11T23:54:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.124+0000 7fc1bc9e0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b810b280 con 0x7fc1b81091f0 2024-09-11T23:54:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.125+0000 7fc1b659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b81091f0 0x7fc1b81108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.125+0000 7fc1b659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b81091f0 0x7fc1b81108a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40944/0 (socket says 172.21.15.62:40944) 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.125+0000 7fc1b659c640 1 -- 172.21.15.62:0/1659380455 learned_addr learned my addr 172.21.15.62:0/1659380455 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.125+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1b8114ab0 0x7fc1b8111320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.125+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8109b10 0x7fc1b8110de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.125+0000 7fc1b659c640 1 -- 172.21.15.62:0/1659380455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8109b10 msgr2=0x7fc1b8110de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.125+0000 7fc1b659c640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8109b10 0x7fc1b8110de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.125+0000 7fc1b659c640 1 -- 172.21.15.62:0/1659380455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1b8114ab0 msgr2=0x7fc1b8111320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.125+0000 7fc1b659c640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1b8114ab0 0x7fc1b8111320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.125+0000 7fc1b659c640 1 -- 172.21.15.62:0/1659380455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b81b3570 con 0x7fc1b81091f0 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.126+0000 7fc1b659c640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b81091f0 0x7fc1b81108a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0000c00 tx=0x7fc1a0032ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.126+0000 7fc1af7fe640 1 -- 172.21.15.62:0/1659380455 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a003f030 con 0x7fc1b81091f0 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.126+0000 7fc1af7fe640 1 -- 172.21.15.62:0/1659380455 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a0003ad0 con 0x7fc1b81091f0 2024-09-11T23:54:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.126+0000 7fc1af7fe640 1 -- 172.21.15.62:0/1659380455 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0035070 con 0x7fc1b81091f0 2024-09-11T23:54:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.126+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1659380455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b81b3800 con 0x7fc1b81091f0 2024-09-11T23:54:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.127+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1659380455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b81b3c60 con 0x7fc1b81091f0 2024-09-11T23:54:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.129+0000 7fc1bedb9740 1 Processor -- start 2024-09-11T23:54:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.129+0000 7fc1af7fe640 1 -- 172.21.15.62:0/1659380455 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc1a0042030 con 0x7fc1b81091f0 2024-09-11T23:54:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.129+0000 7fc1bedb9740 1 -- start start 2024-09-11T23:54:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 0x5594c87a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a4dc0 0x5594c87af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87af680 0x5594c87b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5594c870a2e0 con 0x5594c87af680 2024-09-11T23:54:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594c86b1ce0 con 0x5594c87a44a0 2024-09-11T23:54:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5594c87062a0 con 0x5594c87a4dc0 2024-09-11T23:54:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1b6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 0x5594c87a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1b6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 0x5594c87a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:36418/0 (socket says 172.21.15.62:36418) 2024-09-11T23:54:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1b6d9d640 1 -- 172.21.15.62:0/1135517302 learned_addr learned my addr 172.21.15.62:0/1135517302 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:29.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/1135517302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87af680 0x5594c87b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.130+0000 7fc1b659c640 1 --2- 172.21.15.62:0/1135517302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a4dc0 0x5594c87af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.131+0000 7fc1b6d9d640 1 -- 172.21.15.62:0/1135517302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a4dc0 msgr2=0x5594c87af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:29.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.131+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/1135517302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a4dc0 0x5594c87af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.131+0000 7fc1b6d9d640 1 -- 172.21.15.62:0/1135517302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87af680 msgr2=0x5594c87b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:29.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.131+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/1135517302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87af680 0x5594c87b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.131+0000 7fc1b6d9d640 1 -- 172.21.15.62:0/1135517302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c8696440 con 0x5594c87a44a0 2024-09-11T23:54:29.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.131+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/1135517302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 0x5594c87a4880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009f30 tx=0x7fc1b00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb380f826e5e435 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:29.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.132+0000 7fc1acff9640 1 -- 172.21.15.62:0/1135517302 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0004d20 con 0x5594c87a44a0 2024-09-11T23:54:29.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.132+0000 7fc1acff9640 1 -- 172.21.15.62:0/1135517302 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b0004050 con 0x5594c87a44a0 2024-09-11T23:54:29.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.132+0000 7fc1acff9640 1 -- 172.21.15.62:0/1135517302 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0005280 con 0x5594c87a44a0 2024-09-11T23:54:29.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.132+0000 7fc1af7fe640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc1a40a2cf0 0x7fc1a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.132+0000 7fc1b659c640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc1a40a2cf0 0x7fc1a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.132+0000 7fc1af7fe640 1 -- 172.21.15.62:0/1659380455 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc1a00ef880 con 0x7fc1b81091f0 2024-09-11T23:54:29.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 -- 172.21.15.62:0/1135517302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 msgr2=0x5594c87a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/1135517302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 0x5594c87a4880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009f30 tx=0x7fc1b00118d0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 -- 172.21.15.62:0/1135517302 shutdown_connections 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/1135517302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87af680 0x5594c87b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/1135517302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a4dc0 0x5594c87af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/1135517302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 0x5594c87a4880 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 -- 172.21.15.62:0/1135517302 >> 172.21.15.62:0/1135517302 conn(0x5594c87a0a30 msgr2=0x5594c87a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1b659c640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc1a40a2cf0 0x7fc1a40a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00ed0b0 tx=0x7fc1a002a480 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 -- 172.21.15.62:0/1135517302 shutdown_connections 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 -- 172.21.15.62:0/1135517302 wait complete. 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 -- start start 2024-09-11T23:54:29.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.133+0000 7fc1bedb9740 1 Processor -- start 2024-09-11T23:54:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 0x5594c87aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87af680 0x5594c87ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5594c87b2250 con 0x5594c87a4dc0 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594c86c5050 con 0x5594c87a44a0 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1b5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1b5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ac220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54576/0 (socket says 172.21.15.62:54576) 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1b5d9b640 1 -- 172.21.15.62:0/27650282 learned_addr learned my addr 172.21.15.62:0/27650282 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1bedb9740 1 -- 172.21.15.62:0/27650282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5594c87b23d0 con 0x5594c87af680 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 0x5594c87aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1bedb9740 10 client.0 _ll_register_callbacks cb 0x5594c8705160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.134+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87af680 0x5594c87ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.135+0000 7fc1b5d9b640 1 -- 172.21.15.62:0/27650282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87af680 msgr2=0x5594c87ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.135+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87af680 0x5594c87ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.135+0000 7fc1b5d9b640 1 -- 172.21.15.62:0/27650282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 msgr2=0x5594c87aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.135+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 0x5594c87aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.135+0000 7fc1b5d9b640 1 -- 172.21.15.62:0/27650282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c847b3c0 con 0x5594c87a4dc0 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.136+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ac220 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800bdd0 tx=0x7fc1a80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc2b452b2906136 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.136+0000 7fc19dffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.136+0000 7fc19dffb640 1 -- 172.21.15.62:0/27650282 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8004230 con 0x5594c87a4dc0 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.136+0000 7fc19dffb640 1 -- 172.21.15.62:0/27650282 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc1a80043d0 con 0x5594c87a4dc0 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.136+0000 7fc1bedb9740 1 -- 172.21.15.62:0/27650282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1b0004050 con 0x5594c87a4dc0 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.137+0000 7fc19dffb640 1 -- 172.21.15.62:0/27650282 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc1a80035f0 con 0x5594c87a4dc0 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.137+0000 7fc1bedb9740 10 client.15804 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.137+0000 7fc1bedb9740 1 -- 172.21.15.62:0/27650282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594c87bdc00 con 0x5594c87a4dc0 2024-09-11T23:54:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.137+0000 7fc19dffb640 1 -- 172.21.15.62:0/27650282 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b0004050 con 0x5594c87a4dc0 2024-09-11T23:54:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.137+0000 7fc19dffb640 1 -- 172.21.15.62:0/27650282 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc1a801b8d0 con 0x5594c87a4dc0 2024-09-11T23:54:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.138+0000 7fc1bedb9740 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.138+0000 7fc1bedb9740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:54:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.138+0000 7fc1bedb9740 20 client.15804 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.138+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5594c87bf180 0x5594c87c15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.138+0000 7fc1bedb9740 4 client.15804 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:54:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.138+0000 7fc1bedb9740 1 -- 172.21.15.62:0/27650282 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594c8717850 con 0x5594c87bf180 2024-09-11T23:54:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.138+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5594c87bf180 0x5594c87c15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.138+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5594c87bf180 0x5594c87c15b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb8fed6ade41a6d server_cookie=496643da8c49be8c in_seq=0 out_seq=0 2024-09-11T23:54:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.138+0000 7fc19dffb640 10 client.15804 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc19dffb640 1 -- 172.21.15.62:0/27650282 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594c8717850 con 0x5594c87bf180 2024-09-11T23:54:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc19dffb640 10 client.15804 handle_command_reply: tid=0 2024-09-11T23:54:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 client.15804 shutdown 2024-09-11T23:54:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 -- 172.21.15.62:0/27650282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 msgr2=0x5594c87ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ac220 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800bdd0 tx=0x7fc1a80117b0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 -- 172.21.15.62:0/27650282 shutdown_connections 2024-09-11T23:54:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5594c87bf180 0x5594c87c15b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc19dffb640 0 client.15804 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87af680 0x5594c87ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ac220 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/27650282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a44a0 0x5594c87aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 -- 172.21.15.62:0/27650282 >> 172.21.15.62:0/27650282 conn(0x5594c87a0a30 msgr2=0x5594c87a6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:29.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 -- 172.21.15.62:0/27650282 shutdown_connections 2024-09-11T23:54:29.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.139+0000 7fc1bedb9740 1 -- 172.21.15.62:0/27650282 wait complete. 2024-09-11T23:54:29.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.140+0000 7fc1bedb9740 20 client.15804 trim_cache size 0 max 16384 2024-09-11T23:54:29.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.165+0000 7fc1bedb9740 1 Processor -- start 2024-09-11T23:54:29.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.165+0000 7fc1bedb9740 1 -- start start 2024-09-11T23:54:29.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.165+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 0x5594c87bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.165+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.165+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87af680 0x5594c87ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.165+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b81beef0 con 0x5594c87a4dc0 2024-09-11T23:54:29.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.165+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b81bc7c0 con 0x5594c87af680 2024-09-11T23:54:29.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.165+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b81ba610 con 0x5594c87a44a0 2024-09-11T23:54:29.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.166+0000 7fc1b5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.166+0000 7fc1b6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 0x5594c87bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.166+0000 7fc1b5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54578/0 (socket says 172.21.15.62:54578) 2024-09-11T23:54:29.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.166+0000 7fc1b5d9b640 1 -- 172.21.15.62:0/3566796431 learned_addr learned my addr 172.21.15.62:0/3566796431 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:29.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.166+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/3566796431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87af680 0x5594c87ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.166+0000 7fc1b5d9b640 1 -- 172.21.15.62:0/3566796431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 msgr2=0x5594c87bb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.166+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/3566796431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 0x5594c87bb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.166+0000 7fc1b5d9b640 1 -- 172.21.15.62:0/3566796431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87af680 msgr2=0x5594c87ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.166+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/3566796431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87af680 0x5594c87ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.166+0000 7fc1b5d9b640 1 -- 172.21.15.62:0/3566796431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c87a1ae0 con 0x5594c87a4dc0 2024-09-11T23:54:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.167+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/3566796431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ae190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc1a801bcf0 tx=0x7fc1a800b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb37d8f43248537 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.167+0000 7fc19ffff640 1 -- 172.21.15.62:0/3566796431 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8028070 con 0x5594c87a4dc0 2024-09-11T23:54:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.167+0000 7fc19ffff640 1 -- 172.21.15.62:0/3566796431 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a8029040 con 0x5594c87a4dc0 2024-09-11T23:54:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.167+0000 7fc19ffff640 1 -- 172.21.15.62:0/3566796431 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a80048e0 con 0x5594c87a4dc0 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.169+0000 7fc1bedb9740 1 -- 172.21.15.62:0/3566796431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 msgr2=0x5594c87ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.169+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/3566796431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ae190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc1a801bcf0 tx=0x7fc1a800b9c0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.169+0000 7fc1bedb9740 1 -- 172.21.15.62:0/3566796431 shutdown_connections 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.169+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/3566796431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87af680 0x5594c87ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.169+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/3566796431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87a4dc0 0x5594c87ae190 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.169+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/3566796431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 0x5594c87bb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.169+0000 7fc1bedb9740 1 -- 172.21.15.62:0/3566796431 >> 172.21.15.62:0/3566796431 conn(0x5594c87a0a30 msgr2=0x5594c87ada80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.169+0000 7fc1bedb9740 1 -- 172.21.15.62:0/3566796431 shutdown_connections 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.169+0000 7fc1bedb9740 1 -- 172.21.15.62:0/3566796431 wait complete. 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1bedb9740 1 -- start start 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1bedb9740 1 Processor -- start 2024-09-11T23:54:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 0x5594c87b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a4dc0 0x5594c87b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1bedb9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87af680 0x5594c8828e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b81bac80 con 0x5594c87af680 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b81bc0f0 con 0x5594c87a4dc0 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1bedb9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b81bb350 con 0x5594c87a44a0 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1bedb9740 10 client.0 _ll_register_callbacks cb 0x7fc1a8005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1b6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87af680 0x5594c8828e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1b6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 0x5594c87b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1b6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 0x5594c87b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40970/0 (socket says 172.21.15.62:40970) 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1b6d9d640 1 -- 172.21.15.62:0/393061737 learned_addr learned my addr 172.21.15.62:0/393061737 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.170+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a4dc0 0x5594c87b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.171+0000 7fc1b6d9d640 1 -- 172.21.15.62:0/393061737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a4dc0 msgr2=0x5594c87b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.171+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a4dc0 0x5594c87b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.171+0000 7fc1b6d9d640 1 -- 172.21.15.62:0/393061737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87af680 msgr2=0x5594c8828e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.171+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87af680 0x5594c8828e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.171+0000 7fc1b6d9d640 1 -- 172.21.15.62:0/393061737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c88293a0 con 0x5594c87a44a0 2024-09-11T23:54:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.171+0000 7fc1b6d9d640 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 0x5594c87b5760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009600 tx=0x7fc1b00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82285cb8471f463d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.171+0000 7fc19dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:54:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.171+0000 7fc19dffb640 1 -- 172.21.15.62:0/393061737 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0032040 con 0x5594c87a44a0 2024-09-11T23:54:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.171+0000 7fc1bedb9740 1 -- 172.21.15.62:0/393061737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a8029040 con 0x5594c87a44a0 2024-09-11T23:54:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.171+0000 7fc19dffb640 1 -- 172.21.15.62:0/393061737 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc1b0004c00 con 0x5594c87a44a0 2024-09-11T23:54:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.172+0000 7fc19dffb640 1 -- 172.21.15.62:0/393061737 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc1b0004e00 con 0x5594c87a44a0 2024-09-11T23:54:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.172+0000 7fc1bedb9740 10 client.25459 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.172+0000 7fc1bedb9740 1 -- 172.21.15.62:0/393061737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594c882a510 con 0x5594c87a44a0 2024-09-11T23:54:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.173+0000 7fc19dffb640 1 -- 172.21.15.62:0/393061737 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a8029040 con 0x5594c87a44a0 2024-09-11T23:54:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.173+0000 7fc19dffb640 1 -- 172.21.15.62:0/393061737 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc1b0025040 con 0x5594c87a44a0 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.173+0000 7fc1bedb9740 10 client.25459 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.173+0000 7fc1bedb9740 10 client.25459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.173+0000 7fc1bedb9740 20 client.25459 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.173+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5594c87bf180 0x5594c882ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.173+0000 7fc1bedb9740 4 client.25459 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.173+0000 7fc1bedb9740 1 -- 172.21.15.62:0/393061737 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594c8711d60 con 0x5594c87bf180 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.173+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5594c87bf180 0x5594c882ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1b5d9b640 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5594c87bf180 0x5594c882ca30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711368491dfb3c55 server_cookie=575d321e71977887 in_seq=0 out_seq=0 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc19dffb640 10 client.25459 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc19dffb640 1 -- 172.21.15.62:0/393061737 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5594c8711d60 con 0x5594c87bf180 2024-09-11T23:54:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc19dffb640 10 client.25459 handle_command_reply: tid=0 2024-09-11T23:54:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 client.25459 shutdown 2024-09-11T23:54:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 -- 172.21.15.62:0/393061737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 msgr2=0x5594c87b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 0x5594c87b5760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009600 tx=0x7fc1b00118d0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 -- 172.21.15.62:0/393061737 shutdown_connections 2024-09-11T23:54:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5594c87bf180 0x5594c882ca30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594c87af680 0x5594c8828e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c87a4dc0 0x5594c87b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 --2- 172.21.15.62:0/393061737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594c87a44a0 0x5594c87b5760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc19dffb640 0 client.25459 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:29.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 -- 172.21.15.62:0/393061737 >> 172.21.15.62:0/393061737 conn(0x5594c87a0a30 msgr2=0x5594c87ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:29.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 -- 172.21.15.62:0/393061737 shutdown_connections 2024-09-11T23:54:29.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.174+0000 7fc1bedb9740 1 -- 172.21.15.62:0/393061737 wait complete. 2024-09-11T23:54:29.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bedb9740 20 client.25459 trim_cache size 0 max 16384 2024-09-11T23:54:29.180 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:54:29.180 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:54:29.180 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:54:29.181 INFO:teuthology.orchestra.run.smithi062.stdout: "47edb652-470c-445b-8618-bdf26f24671a": { 2024-09-11T23:54:29.181 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:54:29.181 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "47edb652-470c-445b-8618-bdf26f24671a", 2024-09-11T23:54:29.181 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:54:29.181 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:29.181 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:29.181 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:54:29.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1659380455 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc1a40a2cf0 msgr2=0x7fc1a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:29.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc1a40a2cf0 0x7fc1a40a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00ed0b0 tx=0x7fc1a002a480 comp rx=0 tx=0).stop 2024-09-11T23:54:29.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1659380455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b81091f0 msgr2=0x7fc1b81108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:29.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b81091f0 0x7fc1b81108a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0000c00 tx=0x7fc1a0032ab0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1659380455 shutdown_connections 2024-09-11T23:54:29.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc1a40a2cf0 0x7fc1a40a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1b8114ab0 0x7fc1b8111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8109b10 0x7fc1b8110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 --2- 172.21.15.62:0/1659380455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b81091f0 0x7fc1b81108a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:29.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1659380455 >> 172.21.15.62:0/1659380455 conn(0x7fc1b806e140 msgr2=0x7fc1b8116bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:29.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1659380455 shutdown_connections 2024-09-11T23:54:29.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:29.175+0000 7fc1bc9e0640 1 -- 172.21.15.62:0/1659380455 wait complete. 2024-09-11T23:54:29.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:29 smithi083 ceph-mon[34805]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 509 op/s 2024-09-11T23:54:29.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:29 smithi062 ceph-mon[30958]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 509 op/s 2024-09-11T23:54:29.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:29 smithi120 ceph-mon[35182]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 509 op/s 2024-09-11T23:54:30.018 INFO:tasks.cephfs.filesystem:scrub status for tag:47edb652-470c-445b-8618-bdf26f24671a - {'path': '/', 'tag': '47edb652-470c-445b-8618-bdf26f24671a', 'options': 'recursive,force'} 2024-09-11T23:54:31.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:31 smithi083 ceph-mon[34805]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 453 op/s 2024-09-11T23:54:31.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:31 smithi062 ceph-mon[30958]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 453 op/s 2024-09-11T23:54:31.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:31 smithi120 ceph-mon[35182]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 453 op/s 2024-09-11T23:54:32.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 110.0 s, 3018.4 tps, lat 33.276 ms stddev 32.382 2024-09-11T23:54:33.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:33 smithi083 ceph-mon[34805]: pgmap v76: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.8 MiB/s wr, 713 op/s 2024-09-11T23:54:33.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:33 smithi062 ceph-mon[30958]: pgmap v76: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.8 MiB/s wr, 713 op/s 2024-09-11T23:54:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:33 smithi120 ceph-mon[35182]: pgmap v76: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.8 MiB/s wr, 713 op/s 2024-09-11T23:54:35.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:35 smithi083 ceph-mon[34805]: pgmap v77: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 636 op/s 2024-09-11T23:54:35.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:35 smithi062 ceph-mon[30958]: pgmap v77: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 636 op/s 2024-09-11T23:54:35.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:35 smithi120 ceph-mon[35182]: pgmap v77: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 636 op/s 2024-09-11T23:54:36.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:36 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:54:36.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:36 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:54:36.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:36 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:54:36.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:36 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:54:36.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:36 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:54:36.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:36 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:54:36.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:36 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:54:36.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:36 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:54:36.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:36 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:54:37.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:37 smithi083 ceph-mon[34805]: pgmap v78: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 620 op/s 2024-09-11T23:54:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 115.0 s, 3253.6 tps, lat 30.711 ms stddev 32.100 2024-09-11T23:54:37.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:37 smithi062 ceph-mon[30958]: pgmap v78: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 620 op/s 2024-09-11T23:54:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:37 smithi120 ceph-mon[35182]: pgmap v78: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 620 op/s 2024-09-11T23:54:39.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:39 smithi062 ceph-mon[30958]: pgmap v79: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 920 op/s 2024-09-11T23:54:39.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:39 smithi120 ceph-mon[35182]: pgmap v79: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 920 op/s 2024-09-11T23:54:39.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:39 smithi083 ceph-mon[34805]: pgmap v79: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 920 op/s 2024-09-11T23:54:41.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:41 smithi062 ceph-mon[30958]: pgmap v80: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.8 MiB/s wr, 596 op/s 2024-09-11T23:54:41.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:41 smithi120 ceph-mon[35182]: pgmap v80: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.8 MiB/s wr, 596 op/s 2024-09-11T23:54:41.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:41 smithi083 ceph-mon[34805]: pgmap v80: 97 pgs: 97 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.8 MiB/s wr, 596 op/s 2024-09-11T23:54:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 120.0 s, 3323.4 tps, lat 30.031 ms stddev 29.965 2024-09-11T23:54:43.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:43 smithi062 ceph-mon[30958]: pgmap v81: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.7 MiB/s wr, 854 op/s 2024-09-11T23:54:43.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:43 smithi120 ceph-mon[35182]: pgmap v81: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.7 MiB/s wr, 854 op/s 2024-09-11T23:54:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:43 smithi083 ceph-mon[34805]: pgmap v81: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.7 MiB/s wr, 854 op/s 2024-09-11T23:54:45.309 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-11T23:54:45.309 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:54:45.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:45 smithi062 ceph-mon[30958]: pgmap v82: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.3 MiB/s wr, 682 op/s 2024-09-11T23:54:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:45 smithi120 ceph-mon[35182]: pgmap v82: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.3 MiB/s wr, 682 op/s 2024-09-11T23:54:45.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:45 smithi083 ceph-mon[34805]: pgmap v82: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.3 MiB/s wr, 682 op/s 2024-09-11T23:54:45.698 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.254+0000 7f08a15f3640 1 -- 172.21.15.62:0/1917669326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f089c100aa0 msgr2=0x7f089c10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.254+0000 7f08a15f3640 1 --2- 172.21.15.62:0/1917669326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f089c100aa0 0x7f089c10ea20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0894009ca0 tx=0x7f089402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.255+0000 7f08a15f3640 1 -- 172.21.15.62:0/1917669326 shutdown_connections 2024-09-11T23:54:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.255+0000 7f08a15f3640 1 --2- 172.21.15.62:0/1917669326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f089c100aa0 0x7f089c10ea20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.255+0000 7f08a15f3640 1 --2- 172.21.15.62:0/1917669326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c100100 0x7f089c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.255+0000 7f08a15f3640 1 --2- 172.21.15.62:0/1917669326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c106b80 0x7f089c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.255+0000 7f08a15f3640 1 -- 172.21.15.62:0/1917669326 >> 172.21.15.62:0/1917669326 conn(0x7f089c077230 msgr2=0x7f089c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.256+0000 7f08a15f3640 1 -- 172.21.15.62:0/1917669326 shutdown_connections 2024-09-11T23:54:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.256+0000 7f08a15f3640 1 -- 172.21.15.62:0/1917669326 wait complete. 2024-09-11T23:54:46.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.256+0000 7f08a15f3640 1 Processor -- start 2024-09-11T23:54:46.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f08a15f3640 1 -- start start 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f08a15f3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c100100 0x7f089c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f08a15f3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c100aa0 0x7f089c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f08a15f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f089c106b80 0x7f089c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f08a15f3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f089c112050 con 0x7f089c106b80 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f08a15f3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f089c111ed0 con 0x7f089c100aa0 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f08a15f3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1121d0 con 0x7f089c100100 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f089b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f089c106b80 0x7f089c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f089b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f089c106b80 0x7f089c19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35742/0 (socket says 172.21.15.62:35742) 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f089b7fe640 1 -- 172.21.15.62:0/911717557 learned_addr learned my addr 172.21.15.62:0/911717557 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f089affd640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c100100 0x7f089c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.257+0000 7f089a7fc640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c100aa0 0x7f089c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f089affd640 1 -- 172.21.15.62:0/911717557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c100aa0 msgr2=0x7f089c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f089affd640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c100aa0 0x7f089c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f089affd640 1 -- 172.21.15.62:0/911717557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f089c106b80 msgr2=0x7f089c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f089affd640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f089c106b80 0x7f089c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f089affd640 1 -- 172.21.15.62:0/911717557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c19d3c0 con 0x7f089c100100 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f089a7fc640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c100aa0 0x7f089c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f089b7fe640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f089c106b80 0x7f089c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f089affd640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c100100 0x7f089c198410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f088c00c970 tx=0x7f088c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f0883fff640 1 -- 172.21.15.62:0/911717557 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c007bb0 con 0x7f089c100100 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f08a15f3640 1 -- 172.21.15.62:0/911717557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c19d6b0 con 0x7f089c100100 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f0883fff640 1 -- 172.21.15.62:0/911717557 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f088c007d50 con 0x7f089c100100 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f0883fff640 1 -- 172.21.15.62:0/911717557 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c018820 con 0x7f089c100100 2024-09-11T23:54:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.258+0000 7f08a15f3640 1 -- 172.21.15.62:0/911717557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c101210 con 0x7f089c100100 2024-09-11T23:54:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.259+0000 7f08a15f3640 1 -- 172.21.15.62:0/911717557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f089c199770 con 0x7f089c100100 2024-09-11T23:54:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.262+0000 7f0883fff640 1 -- 172.21.15.62:0/911717557 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f088c00f450 con 0x7f089c100100 2024-09-11T23:54:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.262+0000 7f0883fff640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f08840a7180 0x7f08840a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.262+0000 7f0883fff640 1 -- 172.21.15.62:0/911717557 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f088c0cf3a0 con 0x7f089c100100 2024-09-11T23:54:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.262+0000 7f089a7fc640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f08840a7180 0x7f08840a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.263+0000 7f089a7fc640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f08840a7180 0x7f08840a9640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f089c0ff780 tx=0x7f089000a2d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:46.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.267+0000 7f0883fff640 1 -- 172.21.15.62:0/911717557 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f088c014030 con 0x7f089c100100 2024-09-11T23:54:46.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.611+0000 7f08a15f3640 1 -- 172.21.15.62:0/911717557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f089c101a10 con 0x7f089c100100 2024-09-11T23:54:46.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.612+0000 7f0883fff640 1 -- 172.21.15.62:0/911717557 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11912 (secure 0 0 0) 0x7f088c09bff0 con 0x7f089c100100 2024-09-11T23:54:46.611 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:54:46.611 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-09-11T23:52:56:186189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:55.695276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:54:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-09-11T23:54:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.616+0000 7f08a15f3640 1 -- 172.21.15.62:0/911717557 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f08840a7180 msgr2=0x7f08840a9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.616+0000 7f08a15f3640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f08840a7180 0x7f08840a9640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f089c0ff780 tx=0x7f089000a2d0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.616+0000 7f08a15f3640 1 -- 172.21.15.62:0/911717557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c100100 msgr2=0x7f089c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.616+0000 7f08a15f3640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c100100 0x7f089c198410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f088c00c970 tx=0x7f088c00ce30 comp rx=0 tx=0).stop 2024-09-11T23:54:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.616+0000 7f08a15f3640 1 -- 172.21.15.62:0/911717557 shutdown_connections 2024-09-11T23:54:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.616+0000 7f08a15f3640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f08840a7180 0x7f08840a9640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.616+0000 7f08a15f3640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f089c106b80 0x7f089c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.616+0000 7f08a15f3640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c100aa0 0x7f089c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.616+0000 7f08a15f3640 1 --2- 172.21.15.62:0/911717557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f089c100100 0x7f089c198410 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.616+0000 7f08a15f3640 1 -- 172.21.15.62:0/911717557 >> 172.21.15.62:0/911717557 conn(0x7f089c077230 msgr2=0x7f089c104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.617+0000 7f08a15f3640 1 -- 172.21.15.62:0/911717557 shutdown_connections 2024-09-11T23:54:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:46.617+0000 7f08a15f3640 1 -- 172.21.15.62:0/911717557 wait complete. 2024-09-11T23:54:47.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:47 smithi062 ceph-mon[30958]: pgmap v83: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.0 MiB/s wr, 647 op/s 2024-09-11T23:54:47.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:47 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/911717557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:47.399 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-11T23:54:47.400 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-11T23:54:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 125.0 s, 3208.6 tps, lat 31.200 ms stddev 31.506 2024-09-11T23:54:47.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:47 smithi120 ceph-mon[35182]: pgmap v83: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.0 MiB/s wr, 647 op/s 2024-09-11T23:54:47.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:47 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/911717557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:47 smithi083 ceph-mon[34805]: pgmap v83: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.0 MiB/s wr, 647 op/s 2024-09-11T23:54:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:47 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/911717557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:54:47.807 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:54:48.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.358+0000 7f861d7a0640 1 -- 172.21.15.62:0/1323210579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8618109450 msgr2=0x7f8618109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:48.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.358+0000 7f861d7a0640 1 --2- 172.21.15.62:0/1323210579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8618109450 0x7f8618109830 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8610009ca0 tx=0x7f861002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.359+0000 7f861d7a0640 1 -- 172.21.15.62:0/1323210579 shutdown_connections 2024-09-11T23:54:48.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.359+0000 7f861d7a0640 1 --2- 172.21.15.62:0/1323210579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861810ea80 0x7f86181154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.359+0000 7f861d7a0640 1 --2- 172.21.15.62:0/1323210579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8618109e00 0x7f861810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.359+0000 7f861d7a0640 1 --2- 172.21.15.62:0/1323210579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8618109450 0x7f8618109830 secure :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8610009ca0 tx=0x7f861002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.359+0000 7f861d7a0640 1 -- 172.21.15.62:0/1323210579 >> 172.21.15.62:0/1323210579 conn(0x7f861806e2d0 msgr2=0x7f861806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.359+0000 7f861d7a0640 1 -- 172.21.15.62:0/1323210579 shutdown_connections 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.360+0000 7f861d7a0640 1 -- 172.21.15.62:0/1323210579 wait complete. 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.360+0000 7f861d7a0640 1 Processor -- start 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.362+0000 7f861d7a0640 1 -- start start 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.362+0000 7f861d7a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8618109e00 0x7f86181a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.362+0000 7f861d7a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861810ea80 0x7f86181a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.362+0000 7f861d7a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86181a4c80 0x7f86181a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.362+0000 7f861d7a0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f861811a790 con 0x7f8618109e00 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.362+0000 7f861d7a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f861811a610 con 0x7f86181a4c80 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.362+0000 7f861d7a0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f861811a910 con 0x7f861810ea80 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.362+0000 7f86177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86181a4c80 0x7f86181a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.362+0000 7f86177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86181a4c80 0x7f86181a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35348/0 (socket says 172.21.15.62:35348) 2024-09-11T23:54:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.363+0000 7f86177fe640 1 -- 172.21.15.62:0/1310728459 learned_addr learned my addr 172.21.15.62:0/1310728459 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.363+0000 7f86167fc640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861810ea80 0x7f86181a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.363+0000 7f8616ffd640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8618109e00 0x7f86181a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.363+0000 7f86177fe640 1 -- 172.21.15.62:0/1310728459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861810ea80 msgr2=0x7f86181a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.363+0000 7f86177fe640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861810ea80 0x7f86181a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.363+0000 7f86177fe640 1 -- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8618109e00 msgr2=0x7f86181a7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.363+0000 7f86177fe640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8618109e00 0x7f86181a7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.363+0000 7f86177fe640 1 -- 172.21.15.62:0/1310728459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8618114900 con 0x7f86181a4c80 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.363+0000 7f86167fc640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861810ea80 0x7f86181a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.363+0000 7f8616ffd640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8618109e00 0x7f86181a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.364+0000 7f86177fe640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86181a4c80 0x7f86181a5130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f860800d950 tx=0x7f860800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.364+0000 7f85fffff640 1 -- 172.21.15.62:0/1310728459 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608017030 con 0x7f86181a4c80 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.364+0000 7f85fffff640 1 -- 172.21.15.62:0/1310728459 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608004020 con 0x7f86181a4c80 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.364+0000 7f861d7a0640 1 -- 172.21.15.62:0/1310728459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8618114b90 con 0x7f86181a4c80 2024-09-11T23:54:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.364+0000 7f861d7a0640 1 -- 172.21.15.62:0/1310728459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8618115080 con 0x7f86181a4c80 2024-09-11T23:54:48.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.365+0000 7f85fffff640 1 -- 172.21.15.62:0/1310728459 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608003c00 con 0x7f86181a4c80 2024-09-11T23:54:48.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.368+0000 7f861fb79740 1 Processor -- start 2024-09-11T23:54:48.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.368+0000 7f85fffff640 1 -- 172.21.15.62:0/1310728459 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f860801a070 con 0x7f86181a4c80 2024-09-11T23:54:48.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.368+0000 7f861fb79740 1 -- start start 2024-09-11T23:54:48.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f861fb79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 0x5586ef5ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f861fb79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5cf080 0x5586ef5d9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f861fb79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 0x5586ef5dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f861fb79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5586ef5342a0 con 0x5586ef5ce760 2024-09-11T23:54:48.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f861fb79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5586ef4dbb10 con 0x5586ef5d9940 2024-09-11T23:54:48.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f861fb79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5586ef530260 con 0x5586ef5cf080 2024-09-11T23:54:48.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f86167fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 0x5586ef5ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f86167fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 0x5586ef5ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35780/0 (socket says 172.21.15.62:35780) 2024-09-11T23:54:48.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f86167fc640 1 -- 172.21.15.62:0/694639034 learned_addr learned my addr 172.21.15.62:0/694639034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:48.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f86167fc640 1 --2- 172.21.15.62:0/694639034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 0x5586ef5dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f86167fc640 1 -- 172.21.15.62:0/694639034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5cf080 msgr2=0x5586ef5d9400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:54:48.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f86167fc640 1 --2- 172.21.15.62:0/694639034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5cf080 0x5586ef5d9400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f86167fc640 1 -- 172.21.15.62:0/694639034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 msgr2=0x5586ef5ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:48.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f86167fc640 1 --2- 172.21.15.62:0/694639034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 0x5586ef5ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.369+0000 7f86167fc640 1 -- 172.21.15.62:0/694639034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ef2a5b90 con 0x5586ef5d9940 2024-09-11T23:54:48.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.370+0000 7f86167fc640 1 --2- 172.21.15.62:0/694639034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 0x5586ef5dbd30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f860c013a40 tx=0x7f860c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b27f8664daa9ea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:48.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.370+0000 7f85fd7fa640 1 -- 172.21.15.62:0/694639034 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c004030 con 0x5586ef5d9940 2024-09-11T23:54:48.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.371+0000 7f85fd7fa640 1 -- 172.21.15.62:0/694639034 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f860c00fde0 con 0x5586ef5d9940 2024-09-11T23:54:48.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.371+0000 7f85fd7fa640 1 -- 172.21.15.62:0/694639034 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c003b80 con 0x5586ef5d9940 2024-09-11T23:54:48.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f861fb79740 1 -- 172.21.15.62:0/694639034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 msgr2=0x5586ef5dbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:48.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f861fb79740 1 --2- 172.21.15.62:0/694639034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 0x5586ef5dbd30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f860c013a40 tx=0x7f860c004730 comp rx=0 tx=0).stop 2024-09-11T23:54:48.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f861fb79740 1 -- 172.21.15.62:0/694639034 shutdown_connections 2024-09-11T23:54:48.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f861fb79740 1 --2- 172.21.15.62:0/694639034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 0x5586ef5dbd30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f861fb79740 1 --2- 172.21.15.62:0/694639034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5cf080 0x5586ef5d9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f861fb79740 1 --2- 172.21.15.62:0/694639034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 0x5586ef5ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f861fb79740 1 -- 172.21.15.62:0/694639034 >> 172.21.15.62:0/694639034 conn(0x5586ef5cacf0 msgr2=0x5586ef5cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:48.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f85fffff640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86000a2ec0 0x7f86000a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f861fb79740 1 -- 172.21.15.62:0/694639034 shutdown_connections 2024-09-11T23:54:48.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f8616ffd640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86000a2ec0 0x7f86000a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.372+0000 7f85fffff640 1 -- 172.21.15.62:0/1310728459 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f86080ced80 con 0x7f86181a4c80 2024-09-11T23:54:48.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.373+0000 7f861fb79740 1 -- 172.21.15.62:0/694639034 wait complete. 2024-09-11T23:54:48.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.373+0000 7f8616ffd640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86000a2ec0 0x7f86000a5380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8610006010 tx=0x7f8610009360 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:48.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.374+0000 7f861fb79740 1 -- start start 2024-09-11T23:54:48.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.374+0000 7f861fb79740 1 Processor -- start 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.375+0000 7f861fb79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 0x5586ef5d6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.375+0000 7f861fb79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5cf080 0x5586ef5d6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f861fb79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 0x5586ef5d71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f861fb79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5586ef5dc510 con 0x5586ef5ce760 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f861fb79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5586ef4bc6e0 con 0x5586ef5cf080 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 0x5586ef5d6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f861fb79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5586ef5dc690 con 0x5586ef5d9940 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f86167fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5cf080 0x5586ef5d6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f86167fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5cf080 0x5586ef5d6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35354/0 (socket says 172.21.15.62:35354) 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f86167fc640 1 -- 172.21.15.62:0/2894663553 learned_addr learned my addr 172.21.15.62:0/2894663553 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f861fb79740 10 client.0 _ll_register_callbacks cb 0x5586ef4e0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f86177fe640 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 0x5586ef5d71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f86167fc640 1 -- 172.21.15.62:0/2894663553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 msgr2=0x5586ef5d71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f86167fc640 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 0x5586ef5d71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f86167fc640 1 -- 172.21.15.62:0/2894663553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 msgr2=0x5586ef5d6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f86167fc640 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 0x5586ef5d6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f86167fc640 1 -- 172.21.15.62:0/2894663553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ef5e59e0 con 0x5586ef5cf080 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.376+0000 7f86177fe640 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 0x5586ef5d71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.377+0000 7f86167fc640 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5cf080 0x5586ef5d6cb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f860c016040 tx=0x7f860c0133e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f687b2e8b69eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.377+0000 7f85faff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.377+0000 7f85faff5640 1 -- 172.21.15.62:0/2894663553 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c02c060 con 0x5586ef5cf080 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.377+0000 7f85faff5640 1 -- 172.21.15.62:0/2894663553 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f860c01f840 con 0x5586ef5cf080 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.377+0000 7f861fb79740 1 -- 172.21.15.62:0/2894663553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f860c00fde0 con 0x5586ef5cf080 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.377+0000 7f85faff5640 1 -- 172.21.15.62:0/2894663553 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f860c013620 con 0x5586ef5cf080 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.377+0000 7f861fb79740 10 client.25514 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.377+0000 7f861fb79740 1 -- 172.21.15.62:0/2894663553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586ef5e8320 con 0x5586ef5cf080 2024-09-11T23:54:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.377+0000 7f85faff5640 1 -- 172.21.15.62:0/2894663553 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f860c031480 con 0x5586ef5cf080 2024-09-11T23:54:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.377+0000 7f85faff5640 1 -- 172.21.15.62:0/2894663553 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f860c010430 con 0x5586ef5cf080 2024-09-11T23:54:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.378+0000 7f861fb79740 10 client.25514 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.378+0000 7f861fb79740 10 client.25514 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:54:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.378+0000 7f861fb79740 20 client.25514 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.378+0000 7f861fb79740 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5586ef5e9720 0x5586ef5ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.378+0000 7f861fb79740 4 client.25514 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:54:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.378+0000 7f861fb79740 1 -- 172.21.15.62:0/2894663553 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f860c00fde0 con 0x5586ef5e9720 2024-09-11T23:54:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.378+0000 7f8616ffd640 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5586ef5e9720 0x5586ef5ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.379+0000 7f8616ffd640 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5586ef5e9720 0x5586ef5ebb10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab5f0be1ac9090a8 server_cookie=72ec165907afbeda in_seq=0 out_seq=0 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.379+0000 7f85faff5640 10 client.25514 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.379+0000 7f85faff5640 1 -- 172.21.15.62:0/2894663553 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f860c00fde0 con 0x5586ef5e9720 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.379+0000 7f85faff5640 10 client.25514 handle_command_reply: tid=0 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.379+0000 7f861fb79740 1 client.25514 shutdown 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 1 -- 172.21.15.62:0/2894663553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5cf080 msgr2=0x5586ef5d6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5cf080 0x5586ef5d6cb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f860c016040 tx=0x7f860c0133e0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 1 -- 172.21.15.62:0/2894663553 shutdown_connections 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5586ef5e9720 0x5586ef5ebb10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 0x5586ef5d71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5cf080 0x5586ef5d6cb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 1 --2- 172.21.15.62:0/2894663553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5ce760 0x5586ef5d6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 1 -- 172.21.15.62:0/2894663553 >> 172.21.15.62:0/2894663553 conn(0x5586ef5cacf0 msgr2=0x5586ef5d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f85faff5640 0 client.25514 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 1 -- 172.21.15.62:0/2894663553 shutdown_connections 2024-09-11T23:54:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 1 -- 172.21.15.62:0/2894663553 wait complete. 2024-09-11T23:54:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.380+0000 7f861fb79740 20 client.25514 trim_cache size 0 max 16384 2024-09-11T23:54:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f861fb79740 1 Processor -- start 2024-09-11T23:54:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f861fb79740 1 -- start start 2024-09-11T23:54:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f861fb79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5ce760 0x5586ef5d7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f861fb79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f861fb79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 0x5586ef5f12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f861fb79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8604010f40 con 0x5586ef5cf080 2024-09-11T23:54:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f861fb79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f860400e810 con 0x5586ef5ce760 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f861fb79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f860400c660 con 0x5586ef5d9940 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f86167fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f86167fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35788/0 (socket says 172.21.15.62:35788) 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f86177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 0x5586ef5f12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f86167fc640 1 -- 172.21.15.62:0/2216869133 learned_addr learned my addr 172.21.15.62:0/2216869133 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.403+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5ce760 0x5586ef5d7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.404+0000 7f8616ffd640 1 -- 172.21.15.62:0/2216869133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 msgr2=0x5586ef5f12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.404+0000 7f8616ffd640 1 --2- 172.21.15.62:0/2216869133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 0x5586ef5f12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.404+0000 7f8616ffd640 1 -- 172.21.15.62:0/2216869133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 msgr2=0x5586ef5d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.404+0000 7f8616ffd640 1 --2- 172.21.15.62:0/2216869133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.404+0000 7f8616ffd640 1 -- 172.21.15.62:0/2216869133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ef5cab20 con 0x5586ef5ce760 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.404+0000 7f86167fc640 1 --2- 172.21.15.62:0/2216869133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.404+0000 7f8616ffd640 1 --2- 172.21.15.62:0/2216869133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5ce760 0x5586ef5d7040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8610008000 tx=0x7f861003e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b57a1e9bf2f9746f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:48.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.404+0000 7f85fcff9640 1 -- 172.21.15.62:0/2216869133 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610055070 con 0x5586ef5ce760 2024-09-11T23:54:48.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.404+0000 7f85fcff9640 1 -- 172.21.15.62:0/2216869133 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8610036070 con 0x5586ef5ce760 2024-09-11T23:54:48.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.404+0000 7f85fcff9640 1 -- 172.21.15.62:0/2216869133 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100042a0 con 0x5586ef5ce760 2024-09-11T23:54:48.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.405+0000 7f861fb79740 1 -- 172.21.15.62:0/2216869133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5ce760 msgr2=0x5586ef5d7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:48.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.405+0000 7f861fb79740 1 --2- 172.21.15.62:0/2216869133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5ce760 0x5586ef5d7040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8610008000 tx=0x7f861003e000 comp rx=0 tx=0).stop 2024-09-11T23:54:48.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.405+0000 7f861fb79740 1 -- 172.21.15.62:0/2216869133 shutdown_connections 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.405+0000 7f861fb79740 1 --2- 172.21.15.62:0/2216869133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5d9940 0x5586ef5f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.405+0000 7f861fb79740 1 --2- 172.21.15.62:0/2216869133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.405+0000 7f861fb79740 1 --2- 172.21.15.62:0/2216869133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5ce760 0x5586ef5d7040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.405+0000 7f861fb79740 1 -- 172.21.15.62:0/2216869133 >> 172.21.15.62:0/2216869133 conn(0x5586ef5cacf0 msgr2=0x5586ef5d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.405+0000 7f861fb79740 1 -- 172.21.15.62:0/2216869133 shutdown_connections 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.405+0000 7f861fb79740 1 -- 172.21.15.62:0/2216869133 wait complete. 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.406+0000 7f861fb79740 1 -- start start 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.406+0000 7f861fb79740 1 Processor -- start 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f861fb79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5ce760 0x5586ef5dd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f861fb79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5ddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f861fb79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 0x5586ef5de290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f861fb79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f860400ccd0 con 0x5586ef5cf080 2024-09-11T23:54:48.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f861fb79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f860400e140 con 0x5586ef5d9940 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f861fb79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f860400d3a0 con 0x5586ef5ce760 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f861fb79740 10 client.0 _ll_register_callbacks cb 0x7f860c00ff90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5ce760 0x5586ef5dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5ce760 0x5586ef5dd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:34260/0 (socket says 172.21.15.62:34260) 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f8616ffd640 1 -- 172.21.15.62:0/414529678 learned_addr learned my addr 172.21.15.62:0/414529678 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f86167fc640 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5ddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.407+0000 7f86177fe640 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 0x5586ef5de290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.408+0000 7f86167fc640 1 -- 172.21.15.62:0/414529678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5ce760 msgr2=0x5586ef5dd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.408+0000 7f86167fc640 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5ce760 0x5586ef5dd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.408+0000 7f86167fc640 1 -- 172.21.15.62:0/414529678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 msgr2=0x5586ef5de290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.408+0000 7f86167fc640 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 0x5586ef5de290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.408+0000 7f86167fc640 1 -- 172.21.15.62:0/414529678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ef5c6260 con 0x5586ef5cf080 2024-09-11T23:54:48.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.408+0000 7f8616ffd640 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5ce760 0x5586ef5dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:54:48.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.408+0000 7f86167fc640 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5ddd50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f860c010800 tx=0x7f860c011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39780fc30cc6159d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:54:48.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.408+0000 7f85faff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:54:48.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.408+0000 7f85faff5640 1 -- 172.21.15.62:0/414529678 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c02c040 con 0x5586ef5cf080 2024-09-11T23:54:48.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.409+0000 7f85faff5640 1 -- 172.21.15.62:0/414529678 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f860c0050d0 con 0x5586ef5cf080 2024-09-11T23:54:48.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.409+0000 7f861fb79740 1 -- 172.21.15.62:0/414529678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8610036070 con 0x5586ef5cf080 2024-09-11T23:54:48.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.409+0000 7f85faff5640 1 -- 172.21.15.62:0/414529678 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f860c040050 con 0x5586ef5cf080 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.409+0000 7f861fb79740 10 client.15834 fetch_fsmap learned FSMap version 27 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.409+0000 7f861fb79740 1 -- 172.21.15.62:0/414529678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586ef5ce590 con 0x5586ef5cf080 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.410+0000 7f85faff5640 1 -- 172.21.15.62:0/414529678 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8610036070 con 0x5586ef5cf080 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.410+0000 7f85faff5640 1 -- 172.21.15.62:0/414529678 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f860c003b40 con 0x5586ef5cf080 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.410+0000 7f861fb79740 10 client.15834 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.410+0000 7f861fb79740 10 client.15834 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.410+0000 7f861fb79740 20 client.15834 populate_metadata read hostname 'smithi062' 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.410+0000 7f861fb79740 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5586ef654ab0 0x5586ef656ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.410+0000 7f861fb79740 4 client.15834 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.410+0000 7f861fb79740 1 -- 172.21.15.62:0/414529678 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x5586ef5419d0 con 0x5586ef654ab0 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.411+0000 7f8616ffd640 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5586ef654ab0 0x5586ef656ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.411+0000 7f85faff5640 10 client.15834 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:48.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.411+0000 7f8616ffd640 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5586ef654ab0 0x5586ef656ed0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=147cc6c1c5c710bf server_cookie=ab62e707a6d25ba8 in_seq=0 out_seq=0 2024-09-11T23:54:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.663+0000 7f85faff5640 1 -- 172.21.15.62:0/414529678 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5586ef5419d0 con 0x5586ef654ab0 2024-09-11T23:54:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.663+0000 7f85faff5640 10 client.15834 handle_command_reply: tid=0 2024-09-11T23:54:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.663+0000 7f861fb79740 1 client.15834 shutdown 2024-09-11T23:54:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 1 -- 172.21.15.62:0/414529678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 msgr2=0x5586ef5ddd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5ddd50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f860c010800 tx=0x7f860c011fb0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 1 -- 172.21.15.62:0/414529678 shutdown_connections 2024-09-11T23:54:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5586ef654ab0 0x5586ef656ed0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586ef5d9940 0x5586ef5de290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586ef5cf080 0x5586ef5ddd50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 1 --2- 172.21.15.62:0/414529678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586ef5ce760 0x5586ef5dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 1 -- 172.21.15.62:0/414529678 >> 172.21.15.62:0/414529678 conn(0x5586ef5cacf0 msgr2=0x5586ef5c9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f85faff5640 0 client.15834 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:54:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 1 -- 172.21.15.62:0/414529678 shutdown_connections 2024-09-11T23:54:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 1 -- 172.21.15.62:0/414529678 wait complete. 2024-09-11T23:54:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.664+0000 7f861fb79740 20 client.15834 trim_cache size 0 max 16384 2024-09-11T23:54:48.664 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:54:48.664 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 27, 2024-09-11T23:54:48.664 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:54:48.664 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 10, 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "63cb6b6e": { 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 10, 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 144.7, 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:54:48.665 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:54:48.666 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:54:48.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:54:48.666 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:54:48.666 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:48.666 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:48.666 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:48.666 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:48.666 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:54:48.666 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 -- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86000a2ec0 msgr2=0x7f86000a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86000a2ec0 0x7f86000a5380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8610006010 tx=0x7f8610009360 comp rx=0 tx=0).stop 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 -- 172.21.15.62:0/1310728459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86181a4c80 msgr2=0x7f86181a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86181a4c80 0x7f86181a5130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f860800d950 tx=0x7f860800de10 comp rx=0 tx=0).stop 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 -- 172.21.15.62:0/1310728459 shutdown_connections 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86000a2ec0 0x7f86000a5380 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86181a4c80 0x7f86181a5130 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861810ea80 0x7f86181a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 --2- 172.21.15.62:0/1310728459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8618109e00 0x7f86181a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 -- 172.21.15.62:0/1310728459 >> 172.21.15.62:0/1310728459 conn(0x7f861806e2d0 msgr2=0x7f8618117910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 -- 172.21.15.62:0/1310728459 shutdown_connections 2024-09-11T23:54:48.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:54:48.665+0000 7f861d7a0640 1 -- 172.21.15.62:0/1310728459 wait complete. 2024-09-11T23:54:49.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:49 smithi062 ceph-mon[30958]: pgmap v84: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-09-11T23:54:49.365 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '63cb6b6e', quiesce took 4.1 seconds. Will release after: 20.6 2024-09-11T23:54:49.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:49 smithi120 ceph-mon[35182]: pgmap v84: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-09-11T23:54:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:49 smithi083 ceph-mon[34805]: pgmap v84: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-09-11T23:54:51.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:51 smithi062 ceph-mon[30958]: pgmap v85: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-09-11T23:54:51.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:51 smithi120 ceph-mon[35182]: pgmap v85: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-09-11T23:54:51.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:51 smithi083 ceph-mon[34805]: pgmap v85: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-09-11T23:54:52.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 130.0 s, 561.3 tps, lat 35.144 ms stddev 35.937 2024-09-11T23:54:53.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:53 smithi062 ceph-mon[30958]: pgmap v86: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.8 MiB/s wr, 777 op/s 2024-09-11T23:54:53.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:53 smithi120 ceph-mon[35182]: pgmap v86: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.8 MiB/s wr, 777 op/s 2024-09-11T23:54:53.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:53 smithi083 ceph-mon[34805]: pgmap v86: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.8 MiB/s wr, 777 op/s 2024-09-11T23:54:55.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:55 smithi062 ceph-mon[30958]: pgmap v87: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.3 MiB/s wr, 585 op/s 2024-09-11T23:54:55.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:55 smithi120 ceph-mon[35182]: pgmap v87: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.3 MiB/s wr, 585 op/s 2024-09-11T23:54:55.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:55 smithi083 ceph-mon[34805]: pgmap v87: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.3 MiB/s wr, 585 op/s 2024-09-11T23:54:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 135.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:54:57.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:57 smithi062 ceph-mon[30958]: pgmap v88: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 495 op/s 2024-09-11T23:54:57.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:57 smithi120 ceph-mon[35182]: pgmap v88: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 495 op/s 2024-09-11T23:54:57.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:57 smithi083 ceph-mon[34805]: pgmap v88: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 495 op/s 2024-09-11T23:54:59.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:54:59 smithi062 ceph-mon[30958]: pgmap v89: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 500 op/s 2024-09-11T23:54:59.468 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:54:59 smithi083 ceph-mon[34805]: pgmap v89: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 500 op/s 2024-09-11T23:54:59.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:54:59 smithi120 ceph-mon[35182]: pgmap v89: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 500 op/s 2024-09-11T23:55:00.020 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:55:00.292 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:00.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.739+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/244163583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59c102dc0 msgr2=0x7fe59c1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:00.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.739+0000 7fe5a3cd9640 1 --2- 172.21.15.62:0/244163583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59c102dc0 0x7fe59c1031a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe59802a480 comp rx=0 tx=0).stop 2024-09-11T23:55:00.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.739+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/244163583 shutdown_connections 2024-09-11T23:55:00.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.739+0000 7fe5a3cd9640 1 --2- 172.21.15.62:0/244163583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe59c10c1a0 0x7fe59c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.739+0000 7fe5a3cd9640 1 --2- 172.21.15.62:0/244163583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe59c1036e0 0x7fe59c10bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.739+0000 7fe5a3cd9640 1 --2- 172.21.15.62:0/244163583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59c102dc0 0x7fe59c1031a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.739+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/244163583 >> 172.21.15.62:0/244163583 conn(0x7fe59c0fc7a0 msgr2=0x7fe59c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.740+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/244163583 shutdown_connections 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.740+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/244163583 wait complete. 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.740+0000 7fe5a3cd9640 1 Processor -- start 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.740+0000 7fe5a3cd9640 1 -- start start 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.740+0000 7fe5a3cd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe59c102dc0 0x7fe59c100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.740+0000 7fe5a3cd9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe59c1036e0 0x7fe59c100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.740+0000 7fe5a3cd9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59c10c1a0 0x7fe59c1014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.740+0000 7fe5a3cd9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c111f70 con 0x7fe59c10c1a0 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.740+0000 7fe5a3cd9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c111df0 con 0x7fe59c1036e0 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.740+0000 7fe5a3cd9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1120f0 con 0x7fe59c102dc0 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a2276640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59c10c1a0 0x7fe59c1014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a2276640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59c10c1a0 0x7fe59c1014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44964/0 (socket says 172.21.15.62:44964) 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a2276640 1 -- 172.21.15.62:0/622336668 learned_addr learned my addr 172.21.15.62:0/622336668 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a1274640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe59c1036e0 0x7fe59c100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe59c102dc0 0x7fe59c100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a2276640 1 -- 172.21.15.62:0/622336668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe59c102dc0 msgr2=0x7fe59c100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a2276640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe59c102dc0 0x7fe59c100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a2276640 1 -- 172.21.15.62:0/622336668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe59c1036e0 msgr2=0x7fe59c100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a2276640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe59c1036e0 0x7fe59c100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a2276640 1 -- 172.21.15.62:0/622336668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c1a6930 con 0x7fe59c10c1a0 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a2276640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59c10c1a0 0x7fe59c1014a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe59400eaa0 tx=0x7fe59400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe58effd640 1 -- 172.21.15.62:0/622336668 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59400cdb0 con 0x7fe59c10c1a0 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe58effd640 1 -- 172.21.15.62:0/622336668 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe594005020 con 0x7fe59c10c1a0 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/622336668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59c1a6c20 con 0x7fe59c10c1a0 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe58effd640 1 -- 172.21.15.62:0/622336668 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940116b0 con 0x7fe59c10c1a0 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.741+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/622336668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59c1a7110 con 0x7fe59c10c1a0 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe58effd640 1 -- 172.21.15.62:0/622336668 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe594005560 con 0x7fe59c10c1a0 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a60b2740 1 Processor -- start 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a60b2740 1 -- start start 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x563050981880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x56305098c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x56305098ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5630508e72e0 con 0x563050981dc0 2024-09-11T23:55:00.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56305088ece0 con 0x5630509814a0 2024-09-11T23:55:00.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5630508e32a0 con 0x56305098c680 2024-09-11T23:55:00.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a1a75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x56305098c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a1a75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x56305098c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44972/0 (socket says 172.21.15.62:44972) 2024-09-11T23:55:00.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a1a75640 1 -- 172.21.15.62:0/1531683020 learned_addr learned my addr 172.21.15.62:0/1531683020 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:00.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe58effd640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5880a2cf0 0x7fe5880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.744+0000 7fe5a1274640 1 --2- 172.21.15.62:0/1531683020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x563050981880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe5a1274640 1 --2- 172.21.15.62:0/1531683020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x56305098ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe5a2276640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5880a2cf0 0x7fe5880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe5a1a75640 1 -- 172.21.15.62:0/1531683020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 msgr2=0x56305098ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:00.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/1531683020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x56305098ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe5a1a75640 1 -- 172.21.15.62:0/1531683020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 msgr2=0x563050981880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/1531683020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x563050981880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe58effd640 1 -- 172.21.15.62:0/622336668 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe5940cf630 con 0x7fe59c10c1a0 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe5a1a75640 1 -- 172.21.15.62:0/1531683020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563050873440 con 0x563050981dc0 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe5a1274640 1 --2- 172.21.15.62:0/1531683020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x563050981880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe5a2276640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5880a2cf0 0x7fe5880a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe59400efd0 tx=0x7fe59406aab0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/1531683020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x56305098c140 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe598005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812e04f763da2472 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe587fff640 1 -- 172.21.15.62:0/1531683020 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59803f070 con 0x563050981dc0 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe587fff640 1 -- 172.21.15.62:0/1531683020 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe598004030 con 0x563050981dc0 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.745+0000 7fe587fff640 1 -- 172.21.15.62:0/1531683020 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598035070 con 0x563050981dc0 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.746+0000 7fe5a60b2740 1 -- 172.21.15.62:0/1531683020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 msgr2=0x56305098c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.746+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/1531683020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x56305098c140 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe598005e80 comp rx=0 tx=0).stop 2024-09-11T23:55:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.746+0000 7fe5a60b2740 1 -- 172.21.15.62:0/1531683020 shutdown_connections 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.746+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/1531683020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x56305098ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.746+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/1531683020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x56305098c140 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.746+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/1531683020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x563050981880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.746+0000 7fe5a60b2740 1 -- 172.21.15.62:0/1531683020 >> 172.21.15.62:0/1531683020 conn(0x56305097da30 msgr2=0x56305097de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.746+0000 7fe5a60b2740 1 -- 172.21.15.62:0/1531683020 shutdown_connections 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.746+0000 7fe5a60b2740 1 -- 172.21.15.62:0/1531683020 wait complete. 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a60b2740 1 -- start start 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a60b2740 1 Processor -- start 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x56305098bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563050981dc0 0x5630509891f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56305098c680 0x563050989730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56305098f250 con 0x56305098c680 2024-09-11T23:55:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5630508a2050 con 0x5630509814a0 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56305098f3d0 con 0x563050981dc0 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a60b2740 10 client.0 _ll_register_callbacks cb 0x5630508e2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a1274640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56305098c680 0x563050989730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a1274640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56305098c680 0x563050989730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44984/0 (socket says 172.21.15.62:44984) 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a1274640 1 -- 172.21.15.62:0/583489146 learned_addr learned my addr 172.21.15.62:0/583489146 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.747+0000 7fe5a1274640 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x56305098bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563050981dc0 0x5630509891f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5a1274640 1 -- 172.21.15.62:0/583489146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563050981dc0 msgr2=0x5630509891f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5a1274640 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563050981dc0 0x5630509891f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5a1274640 1 -- 172.21.15.62:0/583489146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 msgr2=0x56305098bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5a1274640 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x56305098bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5a1274640 1 -- 172.21.15.62:0/583489146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630506583c0 con 0x56305098c680 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5a1274640 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56305098c680 0x563050989730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe590012d50 tx=0x7fe590004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedd5abf7170116b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5857fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5857fa640 1 -- 172.21.15.62:0/583489146 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59000edf0 con 0x56305098c680 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5857fa640 1 -- 172.21.15.62:0/583489146 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe590004030 con 0x56305098c680 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5a60b2740 1 -- 172.21.15.62:0/583489146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe598004030 con 0x56305098c680 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5857fa640 1 -- 172.21.15.62:0/583489146 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe59002b050 con 0x56305098c680 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5a60b2740 10 client.15846 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5a60b2740 1 -- 172.21.15.62:0/583489146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56305099ac30 con 0x56305098c680 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.748+0000 7fe5857fa640 1 -- 172.21.15.62:0/583489146 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe598004030 con 0x56305098c680 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.749+0000 7fe5857fa640 1 -- 172.21.15.62:0/583489146 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe590010430 con 0x56305098c680 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.749+0000 7fe5a60b2740 10 client.15846 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.749+0000 7fe5a60b2740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:00.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.749+0000 7fe5a60b2740 20 client.15846 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.749+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56305099c0f0 0x56305099e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.749+0000 7fe5a60b2740 4 client.15846 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.749+0000 7fe5a60b2740 1 -- 172.21.15.62:0/583489146 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630508f4850 con 0x56305099c0f0 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.749+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56305099c0f0 0x56305099e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.749+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56305099c0f0 0x56305099e550 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc07e8488629a03b server_cookie=f4ca2ed7386d9e06 in_seq=0 out_seq=0 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.749+0000 7fe5857fa640 10 client.15846 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5857fa640 1 -- 172.21.15.62:0/583489146 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630508f4850 con 0x56305099c0f0 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5857fa640 10 client.15846 handle_command_reply: tid=0 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 client.15846 shutdown 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 -- 172.21.15.62:0/583489146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56305098c680 msgr2=0x563050989730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56305098c680 0x563050989730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe590012d50 tx=0x7fe590004730 comp rx=0 tx=0).stop 2024-09-11T23:55:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 -- 172.21.15.62:0/583489146 shutdown_connections 2024-09-11T23:55:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56305099c0f0 0x56305099e550 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56305098c680 0x563050989730 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5857fa640 0 client.15846 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563050981dc0 0x5630509891f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/583489146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x56305098bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 -- 172.21.15.62:0/583489146 >> 172.21.15.62:0/583489146 conn(0x56305097da30 msgr2=0x56305097cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 -- 172.21.15.62:0/583489146 shutdown_connections 2024-09-11T23:55:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 1 -- 172.21.15.62:0/583489146 wait complete. 2024-09-11T23:55:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.750+0000 7fe5a60b2740 20 client.15846 trim_cache size 0 max 16384 2024-09-11T23:55:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.767+0000 7fe5a60b2740 1 Processor -- start 2024-09-11T23:55:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a60b2740 1 -- start start 2024-09-11T23:55:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x5630509984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x5630509a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x5630509a4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1b22c0 con 0x563050981dc0 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1afb90 con 0x5630509814a0 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1ad9e0 con 0x56305098c680 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1a75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x5630509a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1a75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x5630509a3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44994/0 (socket says 172.21.15.62:44994) 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1a75640 1 -- 172.21.15.62:0/3397663314 learned_addr learned my addr 172.21.15.62:0/3397663314 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1274640 1 --2- 172.21.15.62:0/3397663314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x5630509a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1274640 1 --2- 172.21.15.62:0/3397663314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x5630509984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1a75640 1 -- 172.21.15.62:0/3397663314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 msgr2=0x5630509a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/3397663314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x5630509a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1a75640 1 -- 172.21.15.62:0/3397663314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 msgr2=0x5630509984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/3397663314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x5630509984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1a75640 1 -- 172.21.15.62:0/3397663314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563050982330 con 0x563050981dc0 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.768+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/3397663314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x5630509a3e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe598005170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d91c867e5e8046 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5877fe640 1 -- 172.21.15.62:0/3397663314 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598035050 con 0x563050981dc0 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5877fe640 1 -- 172.21.15.62:0/3397663314 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe598004040 con 0x563050981dc0 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5877fe640 1 -- 172.21.15.62:0/3397663314 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59803f040 con 0x563050981dc0 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5a60b2740 1 -- 172.21.15.62:0/3397663314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 msgr2=0x5630509a3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/3397663314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x5630509a3e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe598005170 comp rx=0 tx=0).stop 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5a60b2740 1 -- 172.21.15.62:0/3397663314 shutdown_connections 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/3397663314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x5630509a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/3397663314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563050981dc0 0x5630509a3e20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/3397663314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630509814a0 0x5630509984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5a60b2740 1 -- 172.21.15.62:0/3397663314 >> 172.21.15.62:0/3397663314 conn(0x56305097da30 msgr2=0x5630509847e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5a60b2740 1 -- 172.21.15.62:0/3397663314 shutdown_connections 2024-09-11T23:55:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.769+0000 7fe5a60b2740 1 -- 172.21.15.62:0/3397663314 wait complete. 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a60b2740 1 -- start start 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a60b2740 1 Processor -- start 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630509814a0 0x563050990720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563050981dc0 0x563050990c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a60b2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x563050a05df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1ae050 con 0x5630509814a0 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1af4c0 con 0x563050981dc0 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a60b2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1ae720 con 0x56305098c680 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a60b2740 10 client.0 _ll_register_callbacks cb 0x7fe590012990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a1274640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630509814a0 0x563050990720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a1274640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630509814a0 0x563050990720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45004/0 (socket says 172.21.15.62:45004) 2024-09-11T23:55:00.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.770+0000 7fe5a1274640 1 -- 172.21.15.62:0/2986980503 learned_addr learned my addr 172.21.15.62:0/2986980503 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563050981dc0 0x563050990c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a1274640 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x563050a05df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a1274640 1 -- 172.21.15.62:0/2986980503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 msgr2=0x563050a05df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a1274640 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x563050a05df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a1274640 1 -- 172.21.15.62:0/2986980503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563050981dc0 msgr2=0x563050990c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a1274640 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563050981dc0 0x563050990c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a1274640 1 -- 172.21.15.62:0/2986980503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563050991540 con 0x5630509814a0 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563050981dc0 0x563050990c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a1274640 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630509814a0 0x563050990720 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x56305097e6f0 tx=0x7fe590004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fe9797f92b432f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5857fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5857fa640 1 -- 172.21.15.62:0/2986980503 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59000b8e0 con 0x5630509814a0 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a60b2740 1 -- 172.21.15.62:0/2986980503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe598004040 con 0x5630509814a0 2024-09-11T23:55:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5857fa640 1 -- 172.21.15.62:0/2986980503 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe590010a70 con 0x5630509814a0 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5857fa640 1 -- 172.21.15.62:0/2986980503 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe590010c70 con 0x5630509814a0 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a60b2740 10 client.15852 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.771+0000 7fe5a60b2740 1 -- 172.21.15.62:0/2986980503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563050a07450 con 0x5630509814a0 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5857fa640 1 -- 172.21.15.62:0/2986980503 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe598004040 con 0x5630509814a0 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5857fa640 1 -- 172.21.15.62:0/2986980503 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe5900037b0 con 0x5630509814a0 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5a60b2740 10 client.15852 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5a60b2740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5a60b2740 20 client.15852 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563050a07b40 0x563050a09f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5a60b2740 4 client.15852 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5a60b2740 1 -- 172.21.15.62:0/2986980503 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630508eed60 con 0x563050a07b40 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563050a07b40 0x563050a09f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5a1a75640 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563050a07b40 0x563050a09f60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7107fd86430cd891 server_cookie=afd01c1450895cf5 in_seq=0 out_seq=0 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.772+0000 7fe5857fa640 10 client.15852 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5857fa640 1 -- 172.21.15.62:0/2986980503 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5630508eed60 con 0x563050a07b40 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5857fa640 10 client.15852 handle_command_reply: tid=0 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 client.15852 shutdown 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 -- 172.21.15.62:0/2986980503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630509814a0 msgr2=0x563050990720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630509814a0 0x563050990720 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x56305097e6f0 tx=0x7fe590004730 comp rx=0 tx=0).stop 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 -- 172.21.15.62:0/2986980503 shutdown_connections 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563050a07b40 0x563050a09f60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56305098c680 0x563050a05df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563050981dc0 0x563050990c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5857fa640 0 client.15852 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 --2- 172.21.15.62:0/2986980503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630509814a0 0x563050990720 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 -- 172.21.15.62:0/2986980503 >> 172.21.15.62:0/2986980503 conn(0x56305097da30 msgr2=0x56305098aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:00.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 -- 172.21.15.62:0/2986980503 shutdown_connections 2024-09-11T23:55:00.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 1 -- 172.21.15.62:0/2986980503 wait complete. 2024-09-11T23:55:00.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.773+0000 7fe5a60b2740 20 client.15852 trim_cache size 0 max 16384 2024-09-11T23:55:00.780 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:55:00.780 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-11T23:55:00.780 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-11T23:55:00.780 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:55:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/622336668 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5880a2cf0 msgr2=0x7fe5880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5880a2cf0 0x7fe5880a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe59400efd0 tx=0x7fe59406aab0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/622336668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59c10c1a0 msgr2=0x7fe59c1014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59c10c1a0 0x7fe59c1014a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe59400eaa0 tx=0x7fe59400ef60 comp rx=0 tx=0).stop 2024-09-11T23:55:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/622336668 shutdown_connections 2024-09-11T23:55:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5880a2cf0 0x7fe5880a51b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59c10c1a0 0x7fe59c1014a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe59c1036e0 0x7fe59c100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 --2- 172.21.15.62:0/622336668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe59c102dc0 0x7fe59c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/622336668 >> 172.21.15.62:0/622336668 conn(0x7fe59c0fc7a0 msgr2=0x7fe59c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/622336668 shutdown_connections 2024-09-11T23:55:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:00.774+0000 7fe5a3cd9640 1 -- 172.21.15.62:0/622336668 wait complete. 2024-09-11T23:55:01.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:01 smithi062 ceph-mon[30958]: pgmap v90: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 194 op/s 2024-09-11T23:55:01.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:01 smithi120 ceph-mon[35182]: pgmap v90: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 194 op/s 2024-09-11T23:55:01.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:01 smithi083 ceph-mon[34805]: pgmap v90: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 194 op/s 2024-09-11T23:55:01.672 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:55:01.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:55:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:02.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.414+0000 7feb64e74640 1 -- 172.21.15.62:0/3517522565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb60102ef0 msgr2=0x7feb6010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:02.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.414+0000 7feb64e74640 1 --2- 172.21.15.62:0/3517522565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb60102ef0 0x7feb6010da40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7feb58009ca0 tx=0x7feb5802a950 comp rx=0 tx=0).stop 2024-09-11T23:55:02.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.414+0000 7feb64e74640 1 -- 172.21.15.62:0/3517522565 shutdown_connections 2024-09-11T23:55:02.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.414+0000 7feb64e74640 1 --2- 172.21.15.62:0/3517522565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb60102ef0 0x7feb6010da40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:02.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.414+0000 7feb64e74640 1 --2- 172.21.15.62:0/3517522565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb60102570 0x7feb601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:02.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.414+0000 7feb64e74640 1 --2- 172.21.15.62:0/3517522565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb60100e20 0x7feb60101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:02.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.414+0000 7feb64e74640 1 -- 172.21.15.62:0/3517522565 >> 172.21.15.62:0/3517522565 conn(0x7feb600fc7d0 msgr2=0x7feb600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:02.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.414+0000 7feb64e74640 1 -- 172.21.15.62:0/3517522565 shutdown_connections 2024-09-11T23:55:02.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.414+0000 7feb64e74640 1 -- 172.21.15.62:0/3517522565 wait complete. 2024-09-11T23:55:02.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb64e74640 1 Processor -- start 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb64e74640 1 -- start start 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb64e74640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb60100e20 0x7feb6019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb64e74640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb60102570 0x7feb6019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb64e74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb60102ef0 0x7feb6019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb64e74640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb60110fd0 con 0x7feb60100e20 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb64e74640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb60110e50 con 0x7feb60102ef0 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb64e74640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feb60111150 con 0x7feb60102570 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb5e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb60100e20 0x7feb6019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb5e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb60100e20 0x7feb6019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52648/0 (socket says 172.21.15.62:52648) 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb5e59c640 1 -- 172.21.15.62:0/3218023316 learned_addr learned my addr 172.21.15.62:0/3218023316 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb5ed9d640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb60102ef0 0x7feb6019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:02.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.415+0000 7feb57fff640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb60102570 0x7feb6019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:02.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.416+0000 7feb5e59c640 1 -- 172.21.15.62:0/3218023316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb60102570 msgr2=0x7feb6019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:02.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.416+0000 7feb5e59c640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb60102570 0x7feb6019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:02.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.416+0000 7feb5e59c640 1 -- 172.21.15.62:0/3218023316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb60102ef0 msgr2=0x7feb6019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:02.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.416+0000 7feb5e59c640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb60102ef0 0x7feb6019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:02.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.416+0000 7feb5e59c640 1 -- 172.21.15.62:0/3218023316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6019f5b0 con 0x7feb60100e20 2024-09-11T23:55:02.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.416+0000 7feb5e59c640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb60100e20 0x7feb6019a600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7feb5000d950 tx=0x7feb5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:02.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.416+0000 7feb577fe640 1 -- 172.21.15.62:0/3218023316 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50017070 con 0x7feb60100e20 2024-09-11T23:55:02.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.416+0000 7feb64e74640 1 -- 172.21.15.62:0/3218023316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6019f8a0 con 0x7feb60100e20 2024-09-11T23:55:02.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.416+0000 7feb64e74640 1 -- 172.21.15.62:0/3218023316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb601a7180 con 0x7feb60100e20 2024-09-11T23:55:02.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.417+0000 7feb577fe640 1 -- 172.21.15.62:0/3218023316 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb50009c30 con 0x7feb60100e20 2024-09-11T23:55:02.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.417+0000 7feb577fe640 1 -- 172.21.15.62:0/3218023316 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5000f6b0 con 0x7feb60100e20 2024-09-11T23:55:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.418+0000 7feb64e74640 1 -- 172.21.15.62:0/3218023316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb40005290 con 0x7feb60100e20 2024-09-11T23:55:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.419+0000 7feb577fe640 1 -- 172.21.15.62:0/3218023316 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feb5000f850 con 0x7feb60100e20 2024-09-11T23:55:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.420+0000 7feb577fe640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feb4c0a2d70 0x7feb4c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.420+0000 7feb57fff640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feb4c0a2d70 0x7feb4c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.420+0000 7feb577fe640 1 -- 172.21.15.62:0/3218023316 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7feb500cf2d0 con 0x7feb60100e20 2024-09-11T23:55:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.420+0000 7feb57fff640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feb4c0a2d70 0x7feb4c0a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb48005e10 tx=0x7feb48005da0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:02.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.423+0000 7feb577fe640 1 -- 172.21.15.62:0/3218023316 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb5009bf40 con 0x7feb60100e20 2024-09-11T23:55:02.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 140.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:55:02.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.636+0000 7feb64e74640 1 -- 172.21.15.62:0/3218023316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb40005580 con 0x7feb60100e20 2024-09-11T23:55:02.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.637+0000 7feb577fe640 1 -- 172.21.15.62:0/3218023316 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11912 (secure 0 0 0) 0x7feb50009de0 con 0x7feb60100e20 2024-09-11T23:55:02.637 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:55:02.637 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-09-11T23:52:56:186189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:55.695276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:55:02.637 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-09-11T23:55:02.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 -- 172.21.15.62:0/3218023316 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feb4c0a2d70 msgr2=0x7feb4c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:02.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feb4c0a2d70 0x7feb4c0a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb48005e10 tx=0x7feb48005da0 comp rx=0 tx=0).stop 2024-09-11T23:55:02.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 -- 172.21.15.62:0/3218023316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb60100e20 msgr2=0x7feb6019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:02.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb60100e20 0x7feb6019a600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7feb5000d950 tx=0x7feb5000de10 comp rx=0 tx=0).stop 2024-09-11T23:55:02.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 -- 172.21.15.62:0/3218023316 shutdown_connections 2024-09-11T23:55:02.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feb4c0a2d70 0x7feb4c0a5230 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:02.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb60102ef0 0x7feb6019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:02.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb60102570 0x7feb6019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:02.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 --2- 172.21.15.62:0/3218023316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb60100e20 0x7feb6019a600 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:02.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 -- 172.21.15.62:0/3218023316 >> 172.21.15.62:0/3218023316 conn(0x7feb600fc7d0 msgr2=0x7feb6006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:02.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 -- 172.21.15.62:0/3218023316 shutdown_connections 2024-09-11T23:55:02.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:02.639+0000 7feb64e74640 1 -- 172.21.15.62:0/3218023316 wait complete. 2024-09-11T23:55:03.354 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:55:03.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:03 smithi062 ceph-mon[30958]: pgmap v91: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.1 MiB/s wr, 215 op/s 2024-09-11T23:55:03.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:03 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3218023316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:55:03.616 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:03.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:03 smithi120 ceph-mon[35182]: pgmap v91: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.1 MiB/s wr, 215 op/s 2024-09-11T23:55:03.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:03 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3218023316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:55:03.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:03 smithi083 ceph-mon[34805]: pgmap v91: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.1 MiB/s wr, 215 op/s 2024-09-11T23:55:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:03 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3218023316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:55:04.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.100+0000 7f18fc009640 1 -- 172.21.15.62:0/1177721724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18f41036c0 msgr2=0x7f18f410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:04.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.100+0000 7f18fc009640 1 --2- 172.21.15.62:0/1177721724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18f41036c0 0x7f18f410bbb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009900 tx=0x7f18f002a480 comp rx=0 tx=0).stop 2024-09-11T23:55:04.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.100+0000 7f18fc009640 1 -- 172.21.15.62:0/1177721724 shutdown_connections 2024-09-11T23:55:04.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.100+0000 7f18fc009640 1 --2- 172.21.15.62:0/1177721724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410c180 0x7f18f410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.100+0000 7f18fc009640 1 --2- 172.21.15.62:0/1177721724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18f41036c0 0x7f18f410bbb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.100+0000 7f18fc009640 1 --2- 172.21.15.62:0/1177721724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f4102da0 0x7f18f4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.100+0000 7f18fc009640 1 -- 172.21.15.62:0/1177721724 >> 172.21.15.62:0/1177721724 conn(0x7f18f40fc7a0 msgr2=0x7f18f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.100+0000 7f18fc009640 1 -- 172.21.15.62:0/1177721724 shutdown_connections 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.100+0000 7f18fc009640 1 -- 172.21.15.62:0/1177721724 wait complete. 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.100+0000 7f18fc009640 1 Processor -- start 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18fc009640 1 -- start start 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18fc009640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f4102da0 0x7f18f4100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18fc009640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18f41036c0 0x7f18f4100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18fc009640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410c180 0x7f18f4101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18fc009640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18f4111fa0 con 0x7f18f41036c0 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18fc009640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18f4111e20 con 0x7f18f4102da0 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18fc009640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f18f4112120 con 0x7f18f410c180 2024-09-11T23:55:04.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18f95a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18f41036c0 0x7f18f4100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18f95a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18f41036c0 0x7f18f4100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52666/0 (socket says 172.21.15.62:52666) 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18f95a4640 1 -- 172.21.15.62:0/398311111 learned_addr learned my addr 172.21.15.62:0/398311111 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18f9da5640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f4102da0 0x7f18f4100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410c180 0x7f18f4101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18f95a4640 1 -- 172.21.15.62:0/398311111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410c180 msgr2=0x7f18f4101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18f95a4640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410c180 0x7f18f4101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18f95a4640 1 -- 172.21.15.62:0/398311111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f4102da0 msgr2=0x7f18f4100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18f95a4640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f4102da0 0x7f18f4100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18f95a4640 1 -- 172.21.15.62:0/398311111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f41a6970 con 0x7f18f41036c0 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410c180 0x7f18f4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.101+0000 7f18f95a4640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18f41036c0 0x7f18f4100f40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f18f0006010 tx=0x7f18f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:04.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.102+0000 7f18eaffd640 1 -- 172.21.15.62:0/398311111 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f003f030 con 0x7f18f41036c0 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.102+0000 7f18fc009640 1 -- 172.21.15.62:0/398311111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f41a6c00 con 0x7f18f41036c0 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.102+0000 7f18eaffd640 1 -- 172.21.15.62:0/398311111 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f0003ce0 con 0x7f18f41036c0 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.102+0000 7f18eaffd640 1 -- 172.21.15.62:0/398311111 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0035070 con 0x7f18f41036c0 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.102+0000 7f18fc009640 1 -- 172.21.15.62:0/398311111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f41a70e0 con 0x7f18f41036c0 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.104+0000 7f18eaffd640 1 -- 172.21.15.62:0/398311111 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f18f0042030 con 0x7f18f41036c0 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.104+0000 7f18fe3e2740 1 Processor -- start 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18fe3e2740 1 -- start start 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356cb40 0x56313356b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 0x56313356ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133578760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631334d12f0 con 0x563133576370 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563133478cf0 con 0x56313356cb40 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5631334cd2b0 con 0x56313356b5d0 2024-09-11T23:55:04.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18f95a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133578760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18f95a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133578760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52676/0 (socket says 172.21.15.62:52676) 2024-09-11T23:55:04.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18f95a4640 1 -- 172.21.15.62:0/1868858033 learned_addr learned my addr 172.21.15.62:0/1868858033 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18f9da5640 1 --2- 172.21.15.62:0/1868858033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 0x56313356ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/1868858033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356cb40 0x56313356b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18eaffd640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f18e00a2cf0 0x7f18e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18f95a4640 1 -- 172.21.15.62:0/1868858033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 msgr2=0x56313356ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18f95a4640 1 --2- 172.21.15.62:0/1868858033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 0x56313356ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18f95a4640 1 -- 172.21.15.62:0/1868858033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356cb40 msgr2=0x56313356b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18f95a4640 1 --2- 172.21.15.62:0/1868858033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356cb40 0x56313356b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18f95a4640 1 -- 172.21.15.62:0/1868858033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56313345d3a0 con 0x563133576370 2024-09-11T23:55:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.105+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f18e00a2cf0 0x7f18e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.106+0000 7f18f95a4640 1 --2- 172.21.15.62:0/1868858033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133578760 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f18f00ed020 tx=0x7f18f00e6a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de160daffb64792b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.106+0000 7f18eaffd640 1 -- 172.21.15.62:0/398311111 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f18f00ef930 con 0x7f18f41036c0 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.106+0000 7f18dffff640 1 -- 172.21.15.62:0/1868858033 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f00bba80 con 0x563133576370 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.106+0000 7f18dffff640 1 -- 172.21.15.62:0/1868858033 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f003b580 con 0x563133576370 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.106+0000 7f18dffff640 1 -- 172.21.15.62:0/1868858033 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f00044e0 con 0x563133576370 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1868858033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 msgr2=0x563133578760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f18e00a2cf0 0x7f18e00a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00f4f0 tx=0x7f18ec004b50 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/1868858033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133578760 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f18f00ed020 tx=0x7f18f00e6a10 comp rx=0 tx=0).stop 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1868858033 shutdown_connections 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/1868858033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133578760 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/1868858033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 0x56313356ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/1868858033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356cb40 0x56313356b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1868858033 >> 172.21.15.62:0/1868858033 conn(0x563133534310 msgr2=0x563133567c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:04.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1868858033 shutdown_connections 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1868858033 wait complete. 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 -- start start 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 Processor -- start 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356b5d0 0x563133575d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356cb40 0x563133572fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.107+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133573530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563133578e30 con 0x563133576370 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56313348c060 con 0x56313356b5d0 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563133578fb0 con 0x56313356cb40 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fe3e2740 10 client.0 _ll_register_callbacks cb 0x5631331df8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fa5a6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133573530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fa5a6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133573530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52680/0 (socket says 172.21.15.62:52680) 2024-09-11T23:55:04.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fa5a6640 1 -- 172.21.15.62:0/1629186039 learned_addr learned my addr 172.21.15.62:0/1629186039 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18f9da5640 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356b5d0 0x563133575d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18f95a4640 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356cb40 0x563133572fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fa5a6640 1 -- 172.21.15.62:0/1629186039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356cb40 msgr2=0x563133572fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356cb40 0x563133572fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fa5a6640 1 -- 172.21.15.62:0/1629186039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356b5d0 msgr2=0x563133575d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356b5d0 0x563133575d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fa5a6640 1 -- 172.21.15.62:0/1629186039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631332423c0 con 0x563133576370 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133573530 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f18ec010010 tx=0x7f18ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91db1304c4116f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18dd7fa640 1 -- 172.21.15.62:0/1629186039 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec01a450 con 0x563133576370 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18dd7fa640 1 -- 172.21.15.62:0/1629186039 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f18ec01a5f0 con 0x563133576370 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.108+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1629186039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5631334de860 con 0x563133576370 2024-09-11T23:55:04.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18dd7fa640 1 -- 172.21.15.62:0/1629186039 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18ec01a7f0 con 0x563133576370 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18fe3e2740 10 client.15870 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1629186039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631335847f0 con 0x563133576370 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18dd7fa640 1 -- 172.21.15.62:0/1629186039 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5631334de860 con 0x563133576370 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18dd7fa640 1 -- 172.21.15.62:0/1629186039 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f18ec023070 con 0x563133576370 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18fe3e2740 10 client.15870 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18fe3e2740 10 client.15870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18fe3e2740 20 client.15870 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x563133585cb0 0x5631335880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18fe3e2740 4 client.15870 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1629186039 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631334d8d70 con 0x563133585cb0 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.109+0000 7f18f9da5640 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x563133585cb0 0x5631335880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.110+0000 7f18dd7fa640 10 client.15870 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:55:04.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.110+0000 7f18f9da5640 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x563133585cb0 0x5631335880c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e0b6f790959c618 server_cookie=1da0a301c3dc56d in_seq=0 out_seq=0 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18dd7fa640 1 -- 172.21.15.62:0/1629186039 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631334d8d70 con 0x563133585cb0 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18dd7fa640 10 client.15870 handle_command_reply: tid=0 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 client.15870 shutdown 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1629186039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 msgr2=0x563133573530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133573530 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f18ec010010 tx=0x7f18ec004a40 comp rx=0 tx=0).stop 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1629186039 shutdown_connections 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x563133585cb0 0x5631335880c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563133576370 0x563133573530 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356cb40 0x563133572fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/1629186039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356b5d0 0x563133575d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18dd7fa640 0 client.15870 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1629186039 >> 172.21.15.62:0/1629186039 conn(0x563133534310 msgr2=0x56313356d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1629186039 shutdown_connections 2024-09-11T23:55:04.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 1 -- 172.21.15.62:0/1629186039 wait complete. 2024-09-11T23:55:04.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.111+0000 7f18fe3e2740 20 client.15870 trim_cache size 0 max 16384 2024-09-11T23:55:04.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18fe3e2740 1 Processor -- start 2024-09-11T23:55:04.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18fe3e2740 1 -- start start 2024-09-11T23:55:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 0x563133572870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133566bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563133576370 0x56313356e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18f41b2350 con 0x56313356cb40 2024-09-11T23:55:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18f41afc20 con 0x563133576370 2024-09-11T23:55:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f18f41ada70 con 0x56313356b5d0 2024-09-11T23:55:04.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18f95a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133566bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18f95a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133566bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52692/0 (socket says 172.21.15.62:52692) 2024-09-11T23:55:04.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.124+0000 7f18f95a4640 1 -- 172.21.15.62:0/3012993282 learned_addr learned my addr 172.21.15.62:0/3012993282 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:04.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18f9da5640 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 0x563133572870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563133576370 0x56313356e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18f95a4640 1 -- 172.21.15.62:0/3012993282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 msgr2=0x563133572870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18f95a4640 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 0x563133572870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18f95a4640 1 -- 172.21.15.62:0/3012993282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563133576370 msgr2=0x56313356e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18f95a4640 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563133576370 0x56313356e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18f95a4640 1 -- 172.21.15.62:0/3012993282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563133571f60 con 0x56313356cb40 2024-09-11T23:55:04.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18f9da5640 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 0x563133572870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:55:04.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563133576370 0x56313356e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:55:04.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18f95a4640 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133566bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f18f0005d00 tx=0x7f18f0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf5413f5f4ac766 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:04.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18df7fe640 1 -- 172.21.15.62:0/3012993282 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0049040 con 0x56313356cb40 2024-09-11T23:55:04.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18df7fe640 1 -- 172.21.15.62:0/3012993282 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f00bc640 con 0x56313356cb40 2024-09-11T23:55:04.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.125+0000 7f18df7fe640 1 -- 172.21.15.62:0/3012993282 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f003bcb0 con 0x56313356cb40 2024-09-11T23:55:04.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 -- 172.21.15.62:0/3012993282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 msgr2=0x563133566bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:04.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133566bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f18f0005d00 tx=0x7f18f0005c90 comp rx=0 tx=0).stop 2024-09-11T23:55:04.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 -- 172.21.15.62:0/3012993282 shutdown_connections 2024-09-11T23:55:04.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563133576370 0x56313356e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133566bf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/3012993282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56313356b5d0 0x563133572870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 -- 172.21.15.62:0/3012993282 >> 172.21.15.62:0/3012993282 conn(0x563133534310 msgr2=0x5631335664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 -- 172.21.15.62:0/3012993282 shutdown_connections 2024-09-11T23:55:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 -- 172.21.15.62:0/3012993282 wait complete. 2024-09-11T23:55:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 -- start start 2024-09-11T23:55:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.126+0000 7f18fe3e2740 1 Processor -- start 2024-09-11T23:55:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356b5d0 0x56313357e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133579a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18fe3e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563133576370 0x563133579f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18f41ae0e0 con 0x56313356cb40 2024-09-11T23:55:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18f41af550 con 0x56313356b5d0 2024-09-11T23:55:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18fe3e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f18f41ae7b0 con 0x563133576370 2024-09-11T23:55:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18fe3e2740 10 client.0 _ll_register_callbacks cb 0x7f18ec007d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18f95a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133579a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18f95a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133579a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52704/0 (socket says 172.21.15.62:52704) 2024-09-11T23:55:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18f95a4640 1 -- 172.21.15.62:0/82905974 learned_addr learned my addr 172.21.15.62:0/82905974 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563133576370 0x563133579f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18f9da5640 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356b5d0 0x56313357e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18f95a4640 1 -- 172.21.15.62:0/82905974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563133576370 msgr2=0x563133579f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18f95a4640 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563133576370 0x563133579f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18f95a4640 1 -- 172.21.15.62:0/82905974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356b5d0 msgr2=0x56313357e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18f95a4640 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356b5d0 0x56313357e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18f95a4640 1 -- 172.21.15.62:0/82905974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56313357a600 con 0x56313356cb40 2024-09-11T23:55:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18fa5a6640 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563133576370 0x563133579f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18f95a4640 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133579a20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f18f00c4680 tx=0x7f18f0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71e185f552eb321 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.127+0000 7f18dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:55:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18dd7fa640 1 -- 172.21.15.62:0/82905974 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f00c02f0 con 0x56313356cb40 2024-09-11T23:55:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18dd7fa640 1 -- 172.21.15.62:0/82905974 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f18f00ed330 con 0x56313356cb40 2024-09-11T23:55:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18fe3e2740 1 -- 172.21.15.62:0/82905974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f00bc640 con 0x56313356cb40 2024-09-11T23:55:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18dd7fa640 1 -- 172.21.15.62:0/82905974 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18f00c1480 con 0x56313356cb40 2024-09-11T23:55:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18fe3e2740 10 client.15876 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18fe3e2740 1 -- 172.21.15.62:0/82905974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631335f0760 con 0x56313356cb40 2024-09-11T23:55:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18dd7fa640 1 -- 172.21.15.62:0/82905974 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f00bc640 con 0x56313356cb40 2024-09-11T23:55:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18dd7fa640 1 -- 172.21.15.62:0/82905974 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f18f00b85b0 con 0x56313356cb40 2024-09-11T23:55:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18fe3e2740 10 client.15876 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18fe3e2740 10 client.15876 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:55:04.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18fe3e2740 20 client.15876 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:04.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5631335f0b40 0x5631335f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:04.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18fe3e2740 4 client.15876 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:55:04.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18fe3e2740 1 -- 172.21.15.62:0/82905974 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563133455170 con 0x5631335f0b40 2024-09-11T23:55:04.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.128+0000 7f18f9da5640 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5631335f0b40 0x5631335f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:04.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18dd7fa640 10 client.15876 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:55:04.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18f9da5640 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5631335f0b40 0x5631335f2f30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc47921c0c786501 server_cookie=cabce8f545d41644 in_seq=0 out_seq=0 2024-09-11T23:55:04.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18dd7fa640 1 -- 172.21.15.62:0/82905974 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563133455170 con 0x5631335f0b40 2024-09-11T23:55:04.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18dd7fa640 10 client.15876 handle_command_reply: tid=0 2024-09-11T23:55:04.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 client.15876 shutdown 2024-09-11T23:55:04.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 -- 172.21.15.62:0/82905974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 msgr2=0x563133579a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133579a20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f18f00c4680 tx=0x7f18f0003f80 comp rx=0 tx=0).stop 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 -- 172.21.15.62:0/82905974 shutdown_connections 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5631335f0b40 0x5631335f2f30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563133576370 0x563133579f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56313356cb40 0x563133579a20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18dd7fa640 0 client.15876 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 --2- 172.21.15.62:0/82905974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56313356b5d0 0x56313357e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 -- 172.21.15.62:0/82905974 >> 172.21.15.62:0/82905974 conn(0x563133534310 msgr2=0x563133566d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 -- 172.21.15.62:0/82905974 shutdown_connections 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 1 -- 172.21.15.62:0/82905974 wait complete. 2024-09-11T23:55:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.129+0000 7f18fe3e2740 20 client.15876 trim_cache size 0 max 16384 2024-09-11T23:55:04.157 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 -- 172.21.15.62:0/398311111 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f18e00a2cf0 msgr2=0x7f18e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f18e00a2cf0 0x7f18e00a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00f4f0 tx=0x7f18ec004b50 comp rx=0 tx=0).stop 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 -- 172.21.15.62:0/398311111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18f41036c0 msgr2=0x7f18f4100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18f41036c0 0x7f18f4100f40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f18f0006010 tx=0x7f18f00042f0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 -- 172.21.15.62:0/398311111 shutdown_connections 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f18e00a2cf0 0x7f18e00a51b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410c180 0x7f18f4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18f41036c0 0x7f18f4100f40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 --2- 172.21.15.62:0/398311111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f4102da0 0x7f18f4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 -- 172.21.15.62:0/398311111 >> 172.21.15.62:0/398311111 conn(0x7f18f40fc7a0 msgr2=0x7f18f40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 -- 172.21.15.62:0/398311111 shutdown_connections 2024-09-11T23:55:04.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:04.130+0000 7f18fc009640 1 -- 172.21.15.62:0/398311111 wait complete. 2024-09-11T23:55:05.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:05 smithi062 ceph-mon[30958]: pgmap v92: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.3 MiB/s wr, 89 op/s 2024-09-11T23:55:05.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:05 smithi120 ceph-mon[35182]: pgmap v92: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.3 MiB/s wr, 89 op/s 2024-09-11T23:55:05.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:05 smithi083 ceph-mon[34805]: pgmap v92: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.3 MiB/s wr, 89 op/s 2024-09-11T23:55:05.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:55:05.943 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:06.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.420+0000 7fbbee29f640 1 -- 172.21.15.62:0/1082238553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe8101de0 msgr2=0x7fbbe81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:06.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.420+0000 7fbbee29f640 1 --2- 172.21.15.62:0/1082238553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe8101de0 0x7fbbe81021c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009900 tx=0x7fbbd802a410 comp rx=0 tx=0).stop 2024-09-11T23:55:06.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.420+0000 7fbbee29f640 1 -- 172.21.15.62:0/1082238553 shutdown_connections 2024-09-11T23:55:06.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.420+0000 7fbbee29f640 1 --2- 172.21.15.62:0/1082238553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe8106f10 0x7fbbe810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.420+0000 7fbbee29f640 1 --2- 172.21.15.62:0/1082238553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe8102790 0x7fbbe81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.420+0000 7fbbee29f640 1 --2- 172.21.15.62:0/1082238553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe8101de0 0x7fbbe81021c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.420+0000 7fbbee29f640 1 -- 172.21.15.62:0/1082238553 >> 172.21.15.62:0/1082238553 conn(0x7fbbe80fd8c0 msgr2=0x7fbbe80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:06.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.420+0000 7fbbee29f640 1 -- 172.21.15.62:0/1082238553 shutdown_connections 2024-09-11T23:55:06.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.420+0000 7fbbee29f640 1 -- 172.21.15.62:0/1082238553 wait complete. 2024-09-11T23:55:06.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.421+0000 7fbbee29f640 1 Processor -- start 2024-09-11T23:55:06.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.421+0000 7fbbee29f640 1 -- start start 2024-09-11T23:55:06.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.421+0000 7fbbee29f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe8101de0 0x7fbbe80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.421+0000 7fbbee29f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe8102790 0x7fbbe8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.421+0000 7fbbee29f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe8106f10 0x7fbbe8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.421+0000 7fbbee29f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8111f70 con 0x7fbbe8101de0 2024-09-11T23:55:06.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.421+0000 7fbbee29f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8111df0 con 0x7fbbe8102790 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.421+0000 7fbbee29f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81120f0 con 0x7fbbe8106f10 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe8101de0 0x7fbbe80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe8101de0 0x7fbbe80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52718/0 (socket says 172.21.15.62:52718) 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe7fff640 1 -- 172.21.15.62:0/1521669026 learned_addr learned my addr 172.21.15.62:0/1521669026 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbec83c640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe8106f10 0x7fbbe8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe77fe640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe8102790 0x7fbbe8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe7fff640 1 -- 172.21.15.62:0/1521669026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe8106f10 msgr2=0x7fbbe8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe8106f10 0x7fbbe8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe7fff640 1 -- 172.21.15.62:0/1521669026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe8102790 msgr2=0x7fbbe8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe8102790 0x7fbbe8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe7fff640 1 -- 172.21.15.62:0/1521669026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe81a68f0 con 0x7fbbe8101de0 2024-09-11T23:55:06.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe77fe640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe8102790 0x7fbbe8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe8101de0 0x7fbbe80744f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8006010 tx=0x7fbbd80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe57fa640 1 -- 172.21.15.62:0/1521669026 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd803f030 con 0x7fbbe8101de0 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe57fa640 1 -- 172.21.15.62:0/1521669026 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd8003ce0 con 0x7fbbe8101de0 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbee29f640 1 -- 172.21.15.62:0/1521669026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe81a6b80 con 0x7fbbe8101de0 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbe57fa640 1 -- 172.21.15.62:0/1521669026 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8035070 con 0x7fbbe8101de0 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.422+0000 7fbbee29f640 1 -- 172.21.15.62:0/1521669026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe81a7060 con 0x7fbbe8101de0 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbe57fa640 1 -- 172.21.15.62:0/1521669026 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbbd8042030 con 0x7fbbe8101de0 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbf0678740 1 Processor -- start 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbf0678740 1 -- start start 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbe57fa640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbd00a2cf0 0x7fbbd00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690eb40 0x56490690d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690d5d0 0x56490690da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490691a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbf0678740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5649068732f0 con 0x56490690eb40 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbf0678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56490681acf0 con 0x56490690d5d0 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbf0678740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56490686f2b0 con 0x564906918370 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbec83c640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbd00a2cf0 0x7fbbd00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690eb40 0x56490690d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbe57fa640 1 -- 172.21.15.62:0/1521669026 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fbbd80ef930 con 0x7fbbe8101de0 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690eb40 0x56490690d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52726/0 (socket says 172.21.15.62:52726) 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.425+0000 7fbbe7fff640 1 -- 172.21.15.62:0/4256470894 learned_addr learned my addr 172.21.15.62:0/4256470894 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbe77fe640 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690d5d0 0x56490690da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbec83c640 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490691a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbe7fff640 1 -- 172.21.15.62:0/4256470894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 msgr2=0x56490691a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:06.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490691a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbe7fff640 1 -- 172.21.15.62:0/4256470894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690d5d0 msgr2=0x56490690da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690d5d0 0x56490690da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbe7fff640 1 -- 172.21.15.62:0/4256470894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649067ff3a0 con 0x56490690eb40 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbec83c640 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490691a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbe77fe640 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690d5d0 0x56490690da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690eb40 0x56490690d090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80ed6b0 tx=0x7fbbd80bbd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ae0d286d0d440f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbec83c640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbd00a2cf0 0x7fbbd00a51b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x56490690e070 tx=0x7fbbe0009a60 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbce7fc640 1 -- 172.21.15.62:0/4256470894 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8049070 con 0x56490690eb40 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbce7fc640 1 -- 172.21.15.62:0/4256470894 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd803b580 con 0x56490690eb40 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.426+0000 7fbbce7fc640 1 -- 172.21.15.62:0/4256470894 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8087ef0 con 0x56490690eb40 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 -- 172.21.15.62:0/4256470894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690eb40 msgr2=0x56490690d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690eb40 0x56490690d090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80ed6b0 tx=0x7fbbd80bbd10 comp rx=0 tx=0).stop 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 -- 172.21.15.62:0/4256470894 shutdown_connections 2024-09-11T23:55:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490691a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690d5d0 0x56490690da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 --2- 172.21.15.62:0/4256470894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690eb40 0x56490690d090 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 -- 172.21.15.62:0/4256470894 >> 172.21.15.62:0/4256470894 conn(0x5649068d6310 msgr2=0x564906909c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 -- 172.21.15.62:0/4256470894 shutdown_connections 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 -- 172.21.15.62:0/4256470894 wait complete. 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 -- start start 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.427+0000 7fbbf0678740 1 Processor -- start 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56490690d5d0 0x564906917da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x564906915050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564906918370 0x5649069157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbf0678740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56490691ae30 con 0x564906918370 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbf0678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56490682e060 con 0x56490690eb40 2024-09-11T23:55:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbf0678740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56490691afb0 con 0x56490690d5d0 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbf0678740 10 client.0 _ll_register_callbacks cb 0x5649065818c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbec83c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564906918370 0x5649069157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbec83c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564906918370 0x5649069157e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52732/0 (socket says 172.21.15.62:52732) 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbec83c640 1 -- 172.21.15.62:0/2068010817 learned_addr learned my addr 172.21.15.62:0/2068010817 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbe77fe640 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x564906915050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56490690d5d0 0x564906917da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbec83c640 1 -- 172.21.15.62:0/2068010817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56490690d5d0 msgr2=0x564906917da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbec83c640 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56490690d5d0 0x564906917da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbec83c640 1 -- 172.21.15.62:0/2068010817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 msgr2=0x564906915050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbec83c640 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x564906915050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbec83c640 1 -- 172.21.15.62:0/2068010817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649065e43c0 con 0x564906918370 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.428+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56490690d5d0 0x564906917da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbec83c640 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564906918370 0x5649069157e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0013c90 tx=0x7fbbe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a749b1c71151a96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:06.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:55:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbc77fe640 1 -- 172.21.15.62:0/2068010817 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0027060 con 0x564906918370 2024-09-11T23:55:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbc77fe640 1 -- 172.21.15.62:0/2068010817 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fbbe0022670 con 0x564906918370 2024-09-11T23:55:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbf0678740 1 -- 172.21.15.62:0/2068010817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564906880860 con 0x564906918370 2024-09-11T23:55:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbc77fe640 1 -- 172.21.15.62:0/2068010817 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbe001b070 con 0x564906918370 2024-09-11T23:55:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbf0678740 10 client.15888 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbf0678740 1 -- 172.21.15.62:0/2068010817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564906926820 con 0x564906918370 2024-09-11T23:55:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbc77fe640 1 -- 172.21.15.62:0/2068010817 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564906880860 con 0x564906918370 2024-09-11T23:55:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbc77fe640 1 -- 172.21.15.62:0/2068010817 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbbe000f040 con 0x564906918370 2024-09-11T23:55:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbf0678740 10 client.15888 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbf0678740 10 client.15888 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbf0678740 20 client.15888 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbf0678740 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564906927ce0 0x56490692a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbf0678740 4 client.15888 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.429+0000 7fbbf0678740 1 -- 172.21.15.62:0/2068010817 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56490687ad70 con 0x564906927ce0 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.430+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564906927ce0 0x56490692a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.430+0000 7fbbc77fe640 10 client.15888 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.430+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564906927ce0 0x56490692a130 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53980b6df7cdf424 server_cookie=8f0fbb54e76b03dc in_seq=0 out_seq=0 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbc77fe640 1 -- 172.21.15.62:0/2068010817 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56490687ad70 con 0x564906927ce0 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbc77fe640 10 client.15888 handle_command_reply: tid=0 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 client.15888 shutdown 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 -- 172.21.15.62:0/2068010817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564906918370 msgr2=0x5649069157e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564906918370 0x5649069157e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0013c90 tx=0x7fbbe0004a40 comp rx=0 tx=0).stop 2024-09-11T23:55:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 -- 172.21.15.62:0/2068010817 shutdown_connections 2024-09-11T23:55:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564906927ce0 0x56490692a130 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564906918370 0x5649069157e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x564906915050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 --2- 172.21.15.62:0/2068010817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56490690d5d0 0x564906917da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 -- 172.21.15.62:0/2068010817 >> 172.21.15.62:0/2068010817 conn(0x5649068d6310 msgr2=0x56490690f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbc77fe640 0 client.15888 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:55:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 -- 172.21.15.62:0/2068010817 shutdown_connections 2024-09-11T23:55:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 1 -- 172.21.15.62:0/2068010817 wait complete. 2024-09-11T23:55:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.431+0000 7fbbf0678740 20 client.15888 trim_cache size 0 max 16384 2024-09-11T23:55:06.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.444+0000 7fbbf0678740 1 Processor -- start 2024-09-11T23:55:06.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.444+0000 7fbbf0678740 1 -- start start 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906910180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x5649069106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490692fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbf0678740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b22e0 con 0x56490690d5d0 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbf0678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81afbb0 con 0x56490690eb40 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbf0678740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81ada00 con 0x564906918370 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906910180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906910180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52746/0 (socket says 172.21.15.62:52746) 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbe7fff640 1 -- 172.21.15.62:0/2514463386 learned_addr learned my addr 172.21.15.62:0/2514463386 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbec83c640 1 --2- 172.21.15.62:0/2514463386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490692fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbe77fe640 1 --2- 172.21.15.62:0/2514463386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x5649069106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbe7fff640 1 -- 172.21.15.62:0/2514463386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 msgr2=0x56490692fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/2514463386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490692fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbe7fff640 1 -- 172.21.15.62:0/2514463386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 msgr2=0x5649069106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/2514463386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x5649069106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbe7fff640 1 -- 172.21.15.62:0/2514463386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649069244f0 con 0x56490690d5d0 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbec83c640 1 --2- 172.21.15.62:0/2514463386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490692fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/2514463386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906910180 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbbd802a450 tx=0x7fbbd80efb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be97d7a34b3fd074 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.445+0000 7fbbcdffb640 1 -- 172.21.15.62:0/2514463386 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8049040 con 0x56490690d5d0 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbcdffb640 1 -- 172.21.15.62:0/2514463386 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd80bbf40 con 0x56490690d5d0 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbcdffb640 1 -- 172.21.15.62:0/2514463386 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80bc1a0 con 0x56490690d5d0 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbf0678740 1 -- 172.21.15.62:0/2514463386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 msgr2=0x564906910180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbf0678740 1 --2- 172.21.15.62:0/2514463386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906910180 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbbd802a450 tx=0x7fbbd80efb30 comp rx=0 tx=0).stop 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbf0678740 1 -- 172.21.15.62:0/2514463386 shutdown_connections 2024-09-11T23:55:06.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbf0678740 1 --2- 172.21.15.62:0/2514463386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490692fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbf0678740 1 --2- 172.21.15.62:0/2514463386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x5649069106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbf0678740 1 --2- 172.21.15.62:0/2514463386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906910180 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbf0678740 1 -- 172.21.15.62:0/2514463386 >> 172.21.15.62:0/2514463386 conn(0x5649068d6310 msgr2=0x564906929cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbf0678740 1 -- 172.21.15.62:0/2514463386 shutdown_connections 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbf0678740 1 -- 172.21.15.62:0/2514463386 wait complete. 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.446+0000 7fbbf0678740 1 -- start start 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbf0678740 1 Processor -- start 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906920810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x56490691bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbf0678740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490691c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbf0678740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81ae070 con 0x56490690d5d0 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbf0678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81af4e0 con 0x56490690eb40 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbf0678740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81ae740 con 0x564906918370 2024-09-11T23:55:06.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbf0678740 10 client.0 _ll_register_callbacks cb 0x7fbbe0013910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906920810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906920810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52754/0 (socket says 172.21.15.62:52754) 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbe7fff640 1 -- 172.21.15.62:0/794764336 learned_addr learned my addr 172.21.15.62:0/794764336 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbe77fe640 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x56490691bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbec83c640 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490691c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbe7fff640 1 -- 172.21.15.62:0/794764336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 msgr2=0x56490691c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.447+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490691c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbe7fff640 1 -- 172.21.15.62:0/794764336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 msgr2=0x56490691bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x56490691bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbe7fff640 1 -- 172.21.15.62:0/794764336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56490691c770 con 0x56490690d5d0 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbe7fff640 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906920810 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8000c00 tx=0x7fbbd8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61724b85a7e8356 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:55:06.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbc77fe640 1 -- 172.21.15.62:0/794764336 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8049040 con 0x56490690d5d0 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbc77fe640 1 -- 172.21.15.62:0/794764336 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fbbd80b85b0 con 0x56490690d5d0 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbf0678740 1 -- 172.21.15.62:0/794764336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd80bbf40 con 0x56490690d5d0 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbc77fe640 1 -- 172.21.15.62:0/794764336 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbd80c02f0 con 0x56490690d5d0 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbf0678740 10 client.15894 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbf0678740 1 -- 172.21.15.62:0/794764336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564906992890 con 0x56490690d5d0 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbc77fe640 1 -- 172.21.15.62:0/794764336 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd80bbf40 con 0x56490690d5d0 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbc77fe640 1 -- 172.21.15.62:0/794764336 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbbd80c4720 con 0x56490690d5d0 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbf0678740 10 client.15894 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbf0678740 10 client.15894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbf0678740 20 client.15894 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbf0678740 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564906992b70 0x564906994f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbf0678740 4 client.15894 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:55:06.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.448+0000 7fbbf0678740 1 -- 172.21.15.62:0/794764336 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649067f7170 con 0x564906992b70 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.449+0000 7fbbe77fe640 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564906992b70 0x564906994f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.449+0000 7fbbe77fe640 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564906992b70 0x564906994f60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71927e67cb097ece server_cookie=b1a69cb31c5ba828 in_seq=0 out_seq=0 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.449+0000 7fbbc77fe640 10 client.15894 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.449+0000 7fbbc77fe640 1 -- 172.21.15.62:0/794764336 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649067f7170 con 0x564906992b70 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.449+0000 7fbbc77fe640 10 client.15894 handle_command_reply: tid=0 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.449+0000 7fbbf0678740 1 client.15894 shutdown 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.449+0000 7fbbf0678740 1 -- 172.21.15.62:0/794764336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 msgr2=0x564906920810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.449+0000 7fbbf0678740 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906920810 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8000c00 tx=0x7fbbd8005c90 comp rx=0 tx=0).stop 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbf0678740 1 -- 172.21.15.62:0/794764336 shutdown_connections 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbf0678740 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564906992b70 0x564906994f60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbf0678740 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564906918370 0x56490691c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbf0678740 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56490690eb40 0x56490691bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbc77fe640 0 client.15894 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:55:06.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbf0678740 1 --2- 172.21.15.62:0/794764336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490690d5d0 0x564906920810 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbf0678740 1 -- 172.21.15.62:0/794764336 >> 172.21.15.62:0/794764336 conn(0x5649068d6310 msgr2=0x5649069168b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:06.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbf0678740 1 -- 172.21.15.62:0/794764336 shutdown_connections 2024-09-11T23:55:06.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbf0678740 1 -- 172.21.15.62:0/794764336 wait complete. 2024-09-11T23:55:06.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbf0678740 20 client.15894 trim_cache size 0 max 16384 2024-09-11T23:55:06.458 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:55:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbee29f640 1 -- 172.21.15.62:0/1521669026 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbd00a2cf0 msgr2=0x7fbbd00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbee29f640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbd00a2cf0 0x7fbbd00a51b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x56490690e070 tx=0x7fbbe0009a60 comp rx=0 tx=0).stop 2024-09-11T23:55:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbee29f640 1 -- 172.21.15.62:0/1521669026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe8101de0 msgr2=0x7fbbe80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbee29f640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe8101de0 0x7fbbe80744f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8006010 tx=0x7fbbd80042f0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbee29f640 1 -- 172.21.15.62:0/1521669026 shutdown_connections 2024-09-11T23:55:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbee29f640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbd00a2cf0 0x7fbbd00a51b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbee29f640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe8106f10 0x7fbbe8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbee29f640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe8102790 0x7fbbe8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbee29f640 1 --2- 172.21.15.62:0/1521669026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe8101de0 0x7fbbe80744f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:06.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.450+0000 7fbbee29f640 1 -- 172.21.15.62:0/1521669026 >> 172.21.15.62:0/1521669026 conn(0x7fbbe80fd8c0 msgr2=0x7fbbe80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:06.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.451+0000 7fbbee29f640 1 -- 172.21.15.62:0/1521669026 shutdown_connections 2024-09-11T23:55:06.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:06.451+0000 7fbbee29f640 1 -- 172.21.15.62:0/1521669026 wait complete. 2024-09-11T23:55:07.136 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:55:07.392 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:07.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 145.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:55:07.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:07 smithi062 ceph-mon[30958]: pgmap v93: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 891 KiB/s wr, 24 op/s 2024-09-11T23:55:07.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:07 smithi120 ceph-mon[35182]: pgmap v93: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 891 KiB/s wr, 24 op/s 2024-09-11T23:55:07.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:07 smithi083 ceph-mon[34805]: pgmap v93: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 891 KiB/s wr, 24 op/s 2024-09-11T23:55:07.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.872+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2132778814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9e4101420 msgr2=0x7fa9e410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:07.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.872+0000 7fa9e9cd8640 1 --2- 172.21.15.62:0/2132778814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9e4101420 0x7fa9e410d950 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009cd0 tx=0x7fa9dc02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.872+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2132778814 shutdown_connections 2024-09-11T23:55:07.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.872+0000 7fa9e9cd8640 1 --2- 172.21.15.62:0/2132778814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9e4101420 0x7fa9e410d950 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.872+0000 7fa9e9cd8640 1 --2- 172.21.15.62:0/2132778814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9e4100a80 0x7fa9e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.872+0000 7fa9e9cd8640 1 --2- 172.21.15.62:0/2132778814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4106a80 0x7fa9e4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.872+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2132778814 >> 172.21.15.62:0/2132778814 conn(0x7fa9e40fc7a0 msgr2=0x7fa9e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.872+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2132778814 shutdown_connections 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.872+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2132778814 wait complete. 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e9cd8640 1 Processor -- start 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e9cd8640 1 -- start start 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e9cd8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9e4100a80 0x7fa9e41970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e9cd8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9e4101420 0x7fa9e41975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e9cd8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4106a80 0x7fa9e4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e9cd8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4110e80 con 0x7fa9e4100a80 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e9cd8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4110d00 con 0x7fa9e4106a80 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e9cd8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4111000 con 0x7fa9e4101420 2024-09-11T23:55:07.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9e4100a80 0x7fa9e41970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9e4100a80 0x7fa9e41970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52768/0 (socket says 172.21.15.62:52768) 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e37fe640 1 -- 172.21.15.62:0/2935948648 learned_addr learned my addr 172.21.15.62:0/2935948648 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.873+0000 7fa9e3fff640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4106a80 0x7fa9e4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9e4101420 0x7fa9e41975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e37fe640 1 -- 172.21.15.62:0/2935948648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9e4101420 msgr2=0x7fa9e41975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9e4101420 0x7fa9e41975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e37fe640 1 -- 172.21.15.62:0/2935948648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4106a80 msgr2=0x7fa9e4197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4106a80 0x7fa9e4197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e37fe640 1 -- 172.21.15.62:0/2935948648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e41a6910 con 0x7fa9e4100a80 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e3fff640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4106a80 0x7fa9e4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9e4100a80 0x7fa9e41970a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400d950 tx=0x7fa9d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:07.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e0ff9640 1 -- 172.21.15.62:0/2935948648 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4017030 con 0x7fa9e4100a80 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e0ff9640 1 -- 172.21.15.62:0/2935948648 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d4004020 con 0x7fa9e4100a80 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2935948648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e41a6c00 con 0x7fa9e4100a80 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e0ff9640 1 -- 172.21.15.62:0/2935948648 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40047a0 con 0x7fa9e4100a80 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.874+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2935948648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e41a7190 con 0x7fa9e4100a80 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.877+0000 7fa9e0ff9640 1 -- 172.21.15.62:0/2935948648 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa9d40041d0 con 0x7fa9e4100a80 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.877+0000 7fa9e0ff9640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9cc0a2d70 0x7fa9cc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.877+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9cc0a2d70 0x7fa9cc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.877+0000 7fa9e0ff9640 1 -- 172.21.15.62:0/2935948648 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fa9d40ced90 con 0x7fa9e4100a80 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.877+0000 7fa9ec0b1740 1 Processor -- start 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9ec0b1740 1 -- start start 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e35d0 0x55eee93e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 0x55eee93f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9cc0a2d70 0x7fa9cc0a5230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005fd0 tx=0x7fa9d8009470 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55eee93492f0 con 0x55eee93e4b40 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eee92f0cf0 con 0x55eee93ee370 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55eee93452b0 con 0x55eee93e35d0 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52770/0 (socket says 172.21.15.62:52770) 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e37fe640 1 -- 172.21.15.62:0/3194555454 learned_addr learned my addr 172.21.15.62:0/3194555454 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e3fff640 1 --2- 172.21.15.62:0/3194555454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 0x55eee93f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/3194555454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e35d0 0x55eee93e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e37fe640 1 -- 172.21.15.62:0/3194555454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e35d0 msgr2=0x55eee93e3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/3194555454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e35d0 0x55eee93e3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e37fe640 1 -- 172.21.15.62:0/3194555454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 msgr2=0x55eee93f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/3194555454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 0x55eee93f0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e37fe640 1 -- 172.21.15.62:0/3194555454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee92d53a0 con 0x55eee93e4b40 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.878+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/3194555454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e3090 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4097cb0 tx=0x7fa9d409aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db05b0b93933ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9c9ffb640 1 -- 172.21.15.62:0/3194555454 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d409f690 con 0x55eee93e4b40 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9c9ffb640 1 -- 172.21.15.62:0/3194555454 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d4095e50 con 0x55eee93e4b40 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9c9ffb640 1 -- 172.21.15.62:0/3194555454 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40a3ce0 con 0x55eee93e4b40 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/3194555454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 msgr2=0x55eee93e3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/3194555454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e3090 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4097cb0 tx=0x7fa9d409aee0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/3194555454 shutdown_connections 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/3194555454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 0x55eee93f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/3194555454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e35d0 0x55eee93e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/3194555454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e3090 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/3194555454 >> 172.21.15.62:0/3194555454 conn(0x55eee93ac310 msgr2=0x55eee93dfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:07.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/3194555454 shutdown_connections 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.879+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/3194555454 wait complete. 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.880+0000 7fa9ec0b1740 1 -- start start 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.880+0000 7fa9ec0b1740 1 Processor -- start 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.880+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 0x55eee93edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.880+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e4b40 0x55eee93eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.880+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93ee370 0x55eee93eb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.880+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55eee93f1070 con 0x55eee93ee370 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.880+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eee9304060 con 0x55eee93e35d0 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.880+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55eee93f11f0 con 0x55eee93e4b40 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.880+0000 7fa9ec0b1740 10 client.0 _ll_register_callbacks cb 0x55eee90578c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93ee370 0x55eee93eb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e4b40 0x55eee93eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93ee370 0x55eee93eb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52786/0 (socket says 172.21.15.62:52786) 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e3fff640 1 -- 172.21.15.62:0/4294721402 learned_addr learned my addr 172.21.15.62:0/4294721402 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 0x55eee93edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e2ffd640 1 -- 172.21.15.62:0/4294721402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 msgr2=0x55eee93edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 0x55eee93edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e2ffd640 1 -- 172.21.15.62:0/4294721402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93ee370 msgr2=0x55eee93eb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93ee370 0x55eee93eb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e2ffd640 1 -- 172.21.15.62:0/4294721402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee90ba3c0 con 0x55eee93e4b40 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e3fff640 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93ee370 0x55eee93eb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 0x55eee93edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e4b40 0x55eee93eb040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4199200 tx=0x7fa9d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5bf1d1675244755 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/4294721402 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8023070 con 0x55eee93e4b40 2024-09-11T23:55:07.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/4294721402 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fa9d8010960 con 0x55eee93e4b40 2024-09-11T23:55:07.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.881+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/4294721402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d4095e50 con 0x55eee93e4b40 2024-09-11T23:55:07.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/4294721402 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9d8010b60 con 0x55eee93e4b40 2024-09-11T23:55:07.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9ec0b1740 10 client.25525 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:07.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/4294721402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eee93fcba0 con 0x55eee93e4b40 2024-09-11T23:55:07.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/4294721402 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d4095e50 con 0x55eee93e4b40 2024-09-11T23:55:07.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/4294721402 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa9d800f040 con 0x55eee93e4b40 2024-09-11T23:55:07.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9ec0b1740 10 client.25525 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9ec0b1740 10 client.25525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9ec0b1740 20 client.25525 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55eee93fe070 0x55eee9400480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9ec0b1740 4 client.25525 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/4294721402 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eee9356860 con 0x55eee93fe070 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.882+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55eee93fe070 0x55eee9400480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.883+0000 7fa9c2ffd640 10 client.25525 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.883+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55eee93fe070 0x55eee9400480 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f32dcb50ba4b03fe server_cookie=24a424c89645e457 in_seq=0 out_seq=0 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.883+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/4294721402 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eee9356860 con 0x55eee93fe070 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.883+0000 7fa9c2ffd640 10 client.25525 handle_command_reply: tid=0 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.883+0000 7fa9ec0b1740 1 client.25525 shutdown 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/4294721402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e4b40 msgr2=0x55eee93eb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:07.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e4b40 0x55eee93eb040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4199200 tx=0x7fa9d80046c0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/4294721402 shutdown_connections 2024-09-11T23:55:07.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55eee93fe070 0x55eee9400480 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93ee370 0x55eee93eb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e4b40 0x55eee93eb040 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9c2ffd640 0 client.25525 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:07.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/4294721402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 0x55eee93edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/4294721402 >> 172.21.15.62:0/4294721402 conn(0x55eee93ac310 msgr2=0x55eee93e5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:07.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/4294721402 shutdown_connections 2024-09-11T23:55:07.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/4294721402 wait complete. 2024-09-11T23:55:07.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.884+0000 7fa9ec0b1740 20 client.25525 trim_cache size 0 max 16384 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9ec0b1740 1 Processor -- start 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9ec0b1740 1 -- start start 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e35d0 0x55eee93dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 0x55eee93fa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41b2450 con 0x55eee93e4b40 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41afd20 con 0x55eee93ee370 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41adb70 con 0x55eee93e35d0 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52790/0 (socket says 172.21.15.62:52790) 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e2ffd640 1 -- 172.21.15.62:0/2973097213 learned_addr learned my addr 172.21.15.62:0/2973097213 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e3fff640 1 --2- 172.21.15.62:0/2973097213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 0x55eee93fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/2973097213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e35d0 0x55eee93dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e2ffd640 1 -- 172.21.15.62:0/2973097213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e35d0 msgr2=0x55eee93dfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/2973097213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e35d0 0x55eee93dfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e2ffd640 1 -- 172.21.15.62:0/2973097213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 msgr2=0x55eee93fa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/2973097213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 0x55eee93fa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e2ffd640 1 -- 172.21.15.62:0/2973097213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee93ebfe0 con 0x55eee93e4b40 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.897+0000 7fa9e3fff640 1 --2- 172.21.15.62:0/2973097213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 0x55eee93fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.898+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/2973097213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e4300 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80052d0 tx=0x7fa9d8010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58098dcb5653803a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.898+0000 7fa9c97fa640 1 -- 172.21.15.62:0/2973097213 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8038070 con 0x55eee93e4b40 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.898+0000 7fa9c97fa640 1 -- 172.21.15.62:0/2973097213 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8017040 con 0x55eee93e4b40 2024-09-11T23:55:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.898+0000 7fa9c97fa640 1 -- 172.21.15.62:0/2973097213 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8023070 con 0x55eee93e4b40 2024-09-11T23:55:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.898+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2973097213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 msgr2=0x55eee93e4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.898+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/2973097213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e4300 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80052d0 tx=0x7fa9d8010f60 comp rx=0 tx=0).stop 2024-09-11T23:55:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2973097213 shutdown_connections 2024-09-11T23:55:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/2973097213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93ee370 0x55eee93fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/2973097213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93e4300 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/2973097213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93e35d0 0x55eee93dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2973097213 >> 172.21.15.62:0/2973097213 conn(0x55eee93ac310 msgr2=0x55eee93daa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2973097213 shutdown_connections 2024-09-11T23:55:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2973097213 wait complete. 2024-09-11T23:55:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 -- start start 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 Processor -- start 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 0x55eee93f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93ee370 0x55eee93e7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41ae1e0 con 0x55eee93e4b40 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41af650 con 0x55eee93e35d0 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41ae8b0 con 0x55eee93ee370 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.899+0000 7fa9ec0b1740 10 client.0 _ll_register_callbacks cb 0x7fa9d800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93ee370 0x55eee93e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 0x55eee93f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93ee370 0x55eee93e7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:49998/0 (socket says 172.21.15.62:49998) 2024-09-11T23:55:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e3fff640 1 -- 172.21.15.62:0/2747070463 learned_addr learned my addr 172.21.15.62:0/2747070463 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e37fe640 1 -- 172.21.15.62:0/2747070463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93ee370 msgr2=0x55eee93e7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93ee370 0x55eee93e7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e37fe640 1 -- 172.21.15.62:0/2747070463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 msgr2=0x55eee93f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e37fe640 1 -- 172.21.15.62:0/2747070463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee93f7080 con 0x55eee93e35d0 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e3fff640 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93ee370 0x55eee93e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9e37fe640 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 0x55eee93f6480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4097cb0 tx=0x7fa9d4097610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa348ed5984b45fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/2747070463 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d409f690 con 0x55eee93e35d0 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/2747070463 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fa9d40a3ac0 con 0x55eee93e35d0 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2747070463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d8017040 con 0x55eee93e35d0 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/2747070463 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9d40a3f50 con 0x55eee93e35d0 2024-09-11T23:55:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9ec0b1740 10 client.25592 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.900+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2747070463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eee9468320 con 0x55eee93e35d0 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/2747070463 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8017040 con 0x55eee93e35d0 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/2747070463 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa9d409f170 con 0x55eee93e35d0 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9ec0b1740 10 client.25592 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9ec0b1740 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9ec0b1740 20 client.25592 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55eee93fe070 0x55eee946a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9ec0b1740 4 client.25592 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2747070463 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eee9350d70 con 0x55eee93fe070 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55eee93fe070 0x55eee946a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9e2ffd640 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55eee93fe070 0x55eee946a560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd0e5e9c20948feb server_cookie=f94c08b9f875c75 in_seq=0 out_seq=0 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.901+0000 7fa9c2ffd640 10 client.25592 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:07.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9c2ffd640 1 -- 172.21.15.62:0/2747070463 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eee9350d70 con 0x55eee93fe070 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9c2ffd640 10 client.25592 handle_command_reply: tid=0 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 client.25592 shutdown 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2747070463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 msgr2=0x55eee93f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 0x55eee93f6480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4097cb0 tx=0x7fa9d4097610 comp rx=0 tx=0).stop 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2747070463 shutdown_connections 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55eee93fe070 0x55eee946a560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eee93ee370 0x55eee93e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eee93e4b40 0x55eee93f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9c2ffd640 0 client.25592 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 --2- 172.21.15.62:0/2747070463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eee93e35d0 0x55eee93f6480 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2747070463 >> 172.21.15.62:0/2747070463 conn(0x55eee93ac310 msgr2=0x55eee93e9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2747070463 shutdown_connections 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 1 -- 172.21.15.62:0/2747070463 wait complete. 2024-09-11T23:55:07.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.902+0000 7fa9ec0b1740 20 client.25592 trim_cache size 0 max 16384 2024-09-11T23:55:07.908 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2935948648 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9cc0a2d70 msgr2=0x7fa9cc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9cc0a2d70 0x7fa9cc0a5230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005fd0 tx=0x7fa9d8009470 comp rx=0 tx=0).stop 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2935948648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9e4100a80 msgr2=0x7fa9e41970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9e4100a80 0x7fa9e41970a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400d950 tx=0x7fa9d400de10 comp rx=0 tx=0).stop 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2935948648 shutdown_connections 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9cc0a2d70 0x7fa9cc0a5230 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e4106a80 0x7fa9e4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9e4101420 0x7fa9e41975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 --2- 172.21.15.62:0/2935948648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9e4100a80 0x7fa9e41970a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2935948648 >> 172.21.15.62:0/2935948648 conn(0x7fa9e40fc7a0 msgr2=0x7fa9e40fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2935948648 shutdown_connections 2024-09-11T23:55:07.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:07.903+0000 7fa9e9cd8640 1 -- 172.21.15.62:0/2935948648 wait complete. 2024-09-11T23:55:09.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:09 smithi062 ceph-mon[30958]: pgmap v94: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 894 KiB/s wr, 28 op/s 2024-09-11T23:55:09.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:09 smithi120 ceph-mon[35182]: pgmap v94: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 894 KiB/s wr, 28 op/s 2024-09-11T23:55:09.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:09 smithi083 ceph-mon[34805]: pgmap v94: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 894 KiB/s wr, 28 op/s 2024-09-11T23:55:09.966 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:55:10.030 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:55:10.031 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:55:10.243 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:10.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.964+0000 7efd01e07640 1 -- 172.21.15.62:0/2174087311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcfc101de0 msgr2=0x7efcfc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.964+0000 7efd01e07640 1 --2- 172.21.15.62:0/2174087311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcfc101de0 0x7efcfc1021c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efcec009900 tx=0x7efcec02a410 comp rx=0 tx=0).stop 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.964+0000 7efd01e07640 1 -- 172.21.15.62:0/2174087311 shutdown_connections 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.964+0000 7efd01e07640 1 --2- 172.21.15.62:0/2174087311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcfc106f10 0x7efcfc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.964+0000 7efd01e07640 1 --2- 172.21.15.62:0/2174087311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcfc102790 0x7efcfc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.964+0000 7efd01e07640 1 --2- 172.21.15.62:0/2174087311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcfc101de0 0x7efcfc1021c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.964+0000 7efd01e07640 1 -- 172.21.15.62:0/2174087311 >> 172.21.15.62:0/2174087311 conn(0x7efcfc0fd8c0 msgr2=0x7efcfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.964+0000 7efd01e07640 1 -- 172.21.15.62:0/2174087311 shutdown_connections 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.964+0000 7efd01e07640 1 -- 172.21.15.62:0/2174087311 wait complete. 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.965+0000 7efd01e07640 1 Processor -- start 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.965+0000 7efd01e07640 1 -- start start 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.965+0000 7efd01e07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcfc101de0 0x7efcfc1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.965+0000 7efd01e07640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcfc102790 0x7efcfc1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.965+0000 7efd01e07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcfc106f10 0x7efcfc19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.965+0000 7efd01e07640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc111f70 con 0x7efcfc102790 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.965+0000 7efd01e07640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc111df0 con 0x7efcfc106f10 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.965+0000 7efd01e07640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1120f0 con 0x7efcfc101de0 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.965+0000 7efcfaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcfc102790 0x7efcfc1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.965+0000 7efcfaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcfc102790 0x7efcfc1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52824/0 (socket says 172.21.15.62:52824) 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcfaffd640 1 -- 172.21.15.62:0/1710749613 learned_addr learned my addr 172.21.15.62:0/1710749613 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcfb7fe640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcfc101de0 0x7efcfc1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcfbfff640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcfc106f10 0x7efcfc19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcfaffd640 1 -- 172.21.15.62:0/1710749613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcfc101de0 msgr2=0x7efcfc1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcfaffd640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcfc101de0 0x7efcfc1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcfaffd640 1 -- 172.21.15.62:0/1710749613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcfc106f10 msgr2=0x7efcfc19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcfaffd640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcfc106f10 0x7efcfc19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcfaffd640 1 -- 172.21.15.62:0/1710749613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc19d460 con 0x7efcfc102790 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcfaffd640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcfc102790 0x7efcfc1989f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efcf000d770 tx=0x7efcf000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcf8ff9640 1 -- 172.21.15.62:0/1710749613 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0017070 con 0x7efcfc102790 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efd01e07640 1 -- 172.21.15.62:0/1710749613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc19d750 con 0x7efcfc102790 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcf8ff9640 1 -- 172.21.15.62:0/1710749613 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf0005500 con 0x7efcfc102790 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efcf8ff9640 1 -- 172.21.15.62:0/1710749613 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0004e00 con 0x7efcfc102790 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.966+0000 7efd01e07640 1 -- 172.21.15.62:0/1710749613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc1a5030 con 0x7efcfc102790 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.967+0000 7efd01e07640 1 -- 172.21.15.62:0/1710749613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcfc1021c0 con 0x7efcfc102790 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.968+0000 7efcf8ff9640 1 -- 172.21.15.62:0/1710749613 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efcf0004fa0 con 0x7efcfc102790 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.969+0000 7efcf8ff9640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efce40a2d70 0x7efce40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.969+0000 7efcf8ff9640 1 -- 172.21.15.62:0/1710749613 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7efcf00ced40 con 0x7efcfc102790 2024-09-11T23:55:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.969+0000 7efcfb7fe640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efce40a2d70 0x7efce40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:10.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.970+0000 7efcfb7fe640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efce40a2d70 0x7efce40a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efcec009230 tx=0x7efcec0058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:10.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:10.974+0000 7efcf8ff9640 1 -- 172.21.15.62:0/1710749613 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcf009b340 con 0x7efcfc102790 2024-09-11T23:55:11.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.187+0000 7efd01e07640 1 -- 172.21.15.62:0/1710749613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcfc109b60 con 0x7efcfc102790 2024-09-11T23:55:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.188+0000 7efcf8ff9640 1 -- 172.21.15.62:0/1710749613 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11912 (secure 0 0 0) 0x7efcf009f640 con 0x7efcfc102790 2024-09-11T23:55:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:55:11.188 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-09-11T23:52:56:186189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:52:55.695276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:55:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-09-11T23:55:11.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.190+0000 7efd01e07640 1 -- 172.21.15.62:0/1710749613 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efce40a2d70 msgr2=0x7efce40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:11.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.190+0000 7efd01e07640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efce40a2d70 0x7efce40a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efcec009230 tx=0x7efcec0058d0 comp rx=0 tx=0).stop 2024-09-11T23:55:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.190+0000 7efd01e07640 1 -- 172.21.15.62:0/1710749613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcfc102790 msgr2=0x7efcfc1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.190+0000 7efd01e07640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcfc102790 0x7efcfc1989f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efcf000d770 tx=0x7efcf000dc30 comp rx=0 tx=0).stop 2024-09-11T23:55:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.190+0000 7efd01e07640 1 -- 172.21.15.62:0/1710749613 shutdown_connections 2024-09-11T23:55:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.190+0000 7efd01e07640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efce40a2d70 0x7efce40a5230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.190+0000 7efd01e07640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcfc106f10 0x7efcfc19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.190+0000 7efd01e07640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcfc102790 0x7efcfc1989f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.190+0000 7efd01e07640 1 --2- 172.21.15.62:0/1710749613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efcfc101de0 0x7efcfc1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.190+0000 7efd01e07640 1 -- 172.21.15.62:0/1710749613 >> 172.21.15.62:0/1710749613 conn(0x7efcfc0fd8c0 msgr2=0x7efcfc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.191+0000 7efd01e07640 1 -- 172.21.15.62:0/1710749613 shutdown_connections 2024-09-11T23:55:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:11.191+0000 7efd01e07640 1 -- 172.21.15.62:0/1710749613 wait complete. 2024-09-11T23:55:11.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:11 smithi062 ceph-mon[30958]: pgmap v95: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 643 KiB/s wr, 23 op/s 2024-09-11T23:55:11.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:11 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1710749613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:55:11.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:11 smithi120 ceph-mon[35182]: pgmap v95: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 643 KiB/s wr, 23 op/s 2024-09-11T23:55:11.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:11 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1710749613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:55:11.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:11 smithi083 ceph-mon[34805]: pgmap v95: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 643 KiB/s wr, 23 op/s 2024-09-11T23:55:11.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:11 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1710749613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:55:11.929 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 63cb6b6e --release --await' 2024-09-11T23:55:11.930 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 63cb6b6e --release --await 2024-09-11T23:55:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.122+0000 7f57048c6640 1 -- 172.21.15.62:0/3267751713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700105560 msgr2=0x7f570010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.122+0000 7f57048c6640 1 --2- 172.21.15.62:0/3267751713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700105560 0x7f570010d0e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009ca0 tx=0x7f56f802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.123+0000 7f57048c6640 1 -- 172.21.15.62:0/3267751713 shutdown_connections 2024-09-11T23:55:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.123+0000 7f57048c6640 1 --2- 172.21.15.62:0/3267751713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700105560 0x7f570010d0e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.123+0000 7f57048c6640 1 --2- 172.21.15.62:0/3267751713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5700100de0 0x7f5700104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.123+0000 7f57048c6640 1 --2- 172.21.15.62:0/3267751713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5700100430 0x7f5700100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.123+0000 7f57048c6640 1 -- 172.21.15.62:0/3267751713 >> 172.21.15.62:0/3267751713 conn(0x7f5700077230 msgr2=0x7f57000fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:12.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.123+0000 7f57048c6640 1 -- 172.21.15.62:0/3267751713 shutdown_connections 2024-09-11T23:55:12.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.123+0000 7f57048c6640 1 -- 172.21.15.62:0/3267751713 wait complete. 2024-09-11T23:55:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.124+0000 7f57048c6640 1 Processor -- start 2024-09-11T23:55:12.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.124+0000 7f57048c6640 1 -- start start 2024-09-11T23:55:12.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f57048c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5700100430 0x7f570010cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f57048c6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700100de0 0x7f570010d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f57048c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5700105560 0x7f5700109e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f57048c6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f570010fd80 con 0x7f5700100430 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f57048c6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f570010fc00 con 0x7f5700100de0 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f57048c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f570010ff00 con 0x7f5700105560 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f56fe59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5700100430 0x7f570010cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f56fe59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5700100430 0x7f570010cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41256/0 (socket says 172.21.15.62:41256) 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f56fe59c640 1 -- 172.21.15.62:0/808017012 learned_addr learned my addr 172.21.15.62:0/808017012 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f56fed9d640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5700105560 0x7f5700109e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700100de0 0x7f570010d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f56fed9d640 1 -- 172.21.15.62:0/808017012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700100de0 msgr2=0x7f570010d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f56fed9d640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700100de0 0x7f570010d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f56fed9d640 1 -- 172.21.15.62:0/808017012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5700100430 msgr2=0x7f570010cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f56fed9d640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5700100430 0x7f570010cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.125+0000 7f56fed9d640 1 -- 172.21.15.62:0/808017012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570010a670 con 0x7f5700105560 2024-09-11T23:55:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.126+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700100de0 0x7f570010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.126+0000 7f56fed9d640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5700105560 0x7f5700109e40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f56f8005e30 tx=0x7f56f8031720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.126+0000 7f56ef7fe640 1 -- 172.21.15.62:0/808017012 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f802fa50 con 0x7f5700105560 2024-09-11T23:55:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.126+0000 7f57048c6640 1 -- 172.21.15.62:0/808017012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570010a900 con 0x7f5700105560 2024-09-11T23:55:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.126+0000 7f57048c6640 1 -- 172.21.15.62:0/808017012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57001ab4a0 con 0x7f5700105560 2024-09-11T23:55:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.127+0000 7f56ef7fe640 1 -- 172.21.15.62:0/808017012 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f802d070 con 0x7f5700105560 2024-09-11T23:55:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.127+0000 7f56ef7fe640 1 -- 172.21.15.62:0/808017012 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f803b660 con 0x7f5700105560 2024-09-11T23:55:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.130+0000 7f5706c9f740 1 Processor -- start 2024-09-11T23:55:12.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.130+0000 7f56ef7fe640 1 -- 172.21.15.62:0/808017012 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f56f802fbf0 con 0x7f5700105560 2024-09-11T23:55:12.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.130+0000 7f5706c9f740 1 -- start start 2024-09-11T23:55:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.131+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 0x55ada6346f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.131+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6347480 0x55ada6351800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.131+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6351d40 0x55ada6354130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.131+0000 7f5706c9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ada62ac280 con 0x55ada6351d40 2024-09-11T23:55:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.131+0000 7f5706c9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ada6253b10 con 0x55ada6347480 2024-09-11T23:55:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.131+0000 7f5706c9f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ada62a8240 con 0x55ada6346b60 2024-09-11T23:55:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.131+0000 7f56fe59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6347480 0x55ada6351800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.131+0000 7f56fdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6351d40 0x55ada6354130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.131+0000 7f56fdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6351d40 0x55ada6354130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41272/0 (socket says 172.21.15.62:41272) 2024-09-11T23:55:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.131+0000 7f56fdd9b640 1 -- 172.21.15.62:0/2858074988 learned_addr learned my addr 172.21.15.62:0/2858074988 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.132+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/2858074988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 0x55ada6346f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.132+0000 7f56ef7fe640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f56e80a2df0 0x7f56e80a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.132+0000 7f56fdd9b640 1 -- 172.21.15.62:0/2858074988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 msgr2=0x55ada6346f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.132+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/2858074988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 0x55ada6346f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.132+0000 7f56fdd9b640 1 -- 172.21.15.62:0/2858074988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6347480 msgr2=0x55ada6351800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.132+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/2858074988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6347480 0x55ada6351800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.132+0000 7f56fdd9b640 1 -- 172.21.15.62:0/2858074988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ada62601f0 con 0x55ada6351d40 2024-09-11T23:55:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.132+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/2858074988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6351d40 0x55ada6354130 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f56f4012780 tx=0x7f56f4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76243344acf1a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.132+0000 7f56fed9d640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f56e80a2df0 0x7f56e80a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.132+0000 7f56ecff9640 1 -- 172.21.15.62:0/2858074988 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4012d00 con 0x55ada6351d40 2024-09-11T23:55:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.133+0000 7f56ecff9640 1 -- 172.21.15.62:0/2858074988 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f4010cb0 con 0x55ada6351d40 2024-09-11T23:55:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.133+0000 7f56ecff9640 1 -- 172.21.15.62:0/2858074988 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f401ebe0 con 0x55ada6351d40 2024-09-11T23:55:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.133+0000 7f56ef7fe640 1 -- 172.21.15.62:0/808017012 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f56f80f03f0 con 0x7f5700105560 2024-09-11T23:55:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.134+0000 7f56fed9d640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f56e80a2df0 0x7f56e80a52b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f56f80c5130 tx=0x7f56f80b8560 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.134+0000 7f5706c9f740 1 -- 172.21.15.62:0/2858074988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6351d40 msgr2=0x55ada6354130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.134+0000 7f5706c9f740 1 --2- 172.21.15.62:0/2858074988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6351d40 0x55ada6354130 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f56f4012780 tx=0x7f56f4009680 comp rx=0 tx=0).stop 2024-09-11T23:55:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.134+0000 7f5706c9f740 1 -- 172.21.15.62:0/2858074988 shutdown_connections 2024-09-11T23:55:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.134+0000 7f5706c9f740 1 --2- 172.21.15.62:0/2858074988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6351d40 0x55ada6354130 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.134+0000 7f5706c9f740 1 --2- 172.21.15.62:0/2858074988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6347480 0x55ada6351800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.134+0000 7f5706c9f740 1 --2- 172.21.15.62:0/2858074988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 0x55ada6346f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.134+0000 7f5706c9f740 1 -- 172.21.15.62:0/2858074988 >> 172.21.15.62:0/2858074988 conn(0x55ada63430f0 msgr2=0x55ada63434d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.134+0000 7f5706c9f740 1 -- 172.21.15.62:0/2858074988 shutdown_connections 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.134+0000 7f5706c9f740 1 -- 172.21.15.62:0/2858074988 wait complete. 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.135+0000 7f5706c9f740 1 -- start start 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.135+0000 7f5706c9f740 1 Processor -- start 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.135+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6346b60 0x55ada634ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.135+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6347480 0x55ada634f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.135+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 0x55ada634f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f5706c9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ada6354910 con 0x55ada6346b60 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f56fdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6346b60 0x55ada634ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f56fe59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6347480 0x55ada634f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f56fdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 0x55ada634f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f56fdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 0x55ada634f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:46218/0 (socket says 172.21.15.62:46218) 2024-09-11T23:55:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f56fdd9b640 1 -- 172.21.15.62:0/3932527659 learned_addr learned my addr 172.21.15.62:0/3932527659 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f5706c9f740 1 -- 172.21.15.62:0/3932527659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ada6266f10 con 0x55ada6347480 2024-09-11T23:55:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f5706c9f740 1 -- 172.21.15.62:0/3932527659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ada6354a90 con 0x55ada6351d40 2024-09-11T23:55:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f56fdd9b640 1 -- 172.21.15.62:0/3932527659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6347480 msgr2=0x55ada634f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6347480 0x55ada634f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f56fdd9b640 1 -- 172.21.15.62:0/3932527659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6346b60 msgr2=0x55ada634ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.136+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6346b60 0x55ada634ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.137+0000 7f56fdd9b640 1 -- 172.21.15.62:0/3932527659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ada601d6a0 con 0x55ada6351d40 2024-09-11T23:55:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.137+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 0x55ada634f690 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56f4006010 tx=0x7f56f401e840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd294335cb55e56d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.137+0000 7f56e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.137+0000 7f56e5ffb640 1 -- 172.21.15.62:0/3932527659 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f401eaf0 con 0x55ada6351d40 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.137+0000 7f56e5ffb640 1 -- 172.21.15.62:0/3932527659 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f56f4017070 con 0x55ada6351d40 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.137+0000 7f5706c9f740 10 client.0 _ll_register_callbacks cb 0x55ada622fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.137+0000 7f5706c9f740 1 -- 172.21.15.62:0/3932527659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56f4010cb0 con 0x55ada6351d40 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.137+0000 7f56e5ffb640 1 -- 172.21.15.62:0/3932527659 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56f401ec90 con 0x55ada6351d40 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.137+0000 7f5706c9f740 10 client.25549 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.138+0000 7f5706c9f740 1 -- 172.21.15.62:0/3932527659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55ada63602a0 con 0x55ada6351d40 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.138+0000 7f56e5ffb640 1 -- 172.21.15.62:0/3932527659 <== mon.1 v2:172.21.15.120:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f56f402a040 con 0x55ada6351d40 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.138+0000 7f5706c9f740 10 client.25549 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.138+0000 7f5706c9f740 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.138+0000 7f5706c9f740 20 client.25549 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.138+0000 7f5706c9f740 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ada63615f0 0x55ada63639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.138+0000 7f5706c9f740 4 client.25549 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.138+0000 7f5706c9f740 1 -- 172.21.15.62:0/3932527659 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56f4010cb0 con 0x55ada63615f0 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.138+0000 7f56fe59c640 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ada63615f0 0x55ada63639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.139+0000 7f56fe59c640 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ada63615f0 0x55ada63639e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81dbc9a501e0c26 server_cookie=564f32c7f9e264f1 in_seq=0 out_seq=0 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.139+0000 7f56e5ffb640 10 client.25549 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.139+0000 7f56e5ffb640 1 -- 172.21.15.62:0/3932527659 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f56f4010cb0 con 0x55ada63615f0 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.139+0000 7f56e5ffb640 10 client.25549 handle_command_reply: tid=0 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 client.25549 shutdown 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 -- 172.21.15.62:0/3932527659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 msgr2=0x55ada634f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 0x55ada634f690 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56f4006010 tx=0x7f56f401e840 comp rx=0 tx=0).stop 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 -- 172.21.15.62:0/3932527659 shutdown_connections 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ada63615f0 0x55ada63639e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 0x55ada634f690 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6347480 0x55ada634f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 --2- 172.21.15.62:0/3932527659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6346b60 0x55ada634ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 -- 172.21.15.62:0/3932527659 >> 172.21.15.62:0/3932527659 conn(0x55ada63430f0 msgr2=0x55ada6348dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f56e5ffb640 0 client.25549 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 -- 172.21.15.62:0/3932527659 shutdown_connections 2024-09-11T23:55:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 1 -- 172.21.15.62:0/3932527659 wait complete. 2024-09-11T23:55:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.140+0000 7f5706c9f740 20 client.25549 trim_cache size 0 max 16384 2024-09-11T23:55:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.158+0000 7f5706c9f740 1 Processor -- start 2024-09-11T23:55:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.158+0000 7f5706c9f740 1 -- start start 2024-09-11T23:55:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6346b60 0x55ada634f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 0x55ada6348cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 0x55ada63692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f5706c9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f56e0010f00 con 0x55ada6347480 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f5706c9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f56e000e7d0 con 0x55ada6346b60 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f5706c9f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56e000c620 con 0x55ada6351d40 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fe59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 0x55ada6348cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fe59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 0x55ada6348cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41292/0 (socket says 172.21.15.62:41292) 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fe59c640 1 -- 172.21.15.62:0/1187040476 learned_addr learned my addr 172.21.15.62:0/1187040476 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/1187040476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6346b60 0x55ada634f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/1187040476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 0x55ada63692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fe59c640 1 -- 172.21.15.62:0/1187040476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 msgr2=0x55ada63692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fe59c640 1 --2- 172.21.15.62:0/1187040476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 0x55ada63692d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fe59c640 1 -- 172.21.15.62:0/1187040476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6346b60 msgr2=0x55ada634f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fe59c640 1 --2- 172.21.15.62:0/1187040476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6346b60 0x55ada634f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fe59c640 1 -- 172.21.15.62:0/1187040476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ada63472b0 con 0x55ada6347480 2024-09-11T23:55:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.159+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/1187040476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6346b60 0x55ada634f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:55:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.160+0000 7f56fe59c640 1 --2- 172.21.15.62:0/1187040476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 0x55ada6348cc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55ada634e4a0 tx=0x7f56f0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d69707bf55019 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.160+0000 7f56e7fff640 1 -- 172.21.15.62:0/1187040476 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0005020 con 0x55ada6347480 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.160+0000 7f56e7fff640 1 -- 172.21.15.62:0/1187040476 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f00047a0 con 0x55ada6347480 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.160+0000 7f56e7fff640 1 -- 172.21.15.62:0/1187040476 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0010690 con 0x55ada6347480 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.160+0000 7f5706c9f740 1 -- 172.21.15.62:0/1187040476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 msgr2=0x55ada6348cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.160+0000 7f5706c9f740 1 --2- 172.21.15.62:0/1187040476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 0x55ada6348cc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55ada634e4a0 tx=0x7f56f0005f90 comp rx=0 tx=0).stop 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.161+0000 7f5706c9f740 1 -- 172.21.15.62:0/1187040476 shutdown_connections 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.161+0000 7f5706c9f740 1 --2- 172.21.15.62:0/1187040476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6351d40 0x55ada63692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.161+0000 7f5706c9f740 1 --2- 172.21.15.62:0/1187040476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 0x55ada6348cc0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.161+0000 7f5706c9f740 1 --2- 172.21.15.62:0/1187040476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6346b60 0x55ada634f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.161+0000 7f5706c9f740 1 -- 172.21.15.62:0/1187040476 >> 172.21.15.62:0/1187040476 conn(0x55ada63430f0 msgr2=0x55ada63485b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.161+0000 7f5706c9f740 1 -- 172.21.15.62:0/1187040476 shutdown_connections 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.161+0000 7f5706c9f740 1 -- 172.21.15.62:0/1187040476 wait complete. 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.161+0000 7f5706c9f740 1 -- start start 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.161+0000 7f5706c9f740 1 Processor -- start 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 0x55ada635a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 0x55ada63559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f5706c9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6351d40 0x55ada6356150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f5706c9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f56e000cc90 con 0x55ada6347480 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f56fdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 0x55ada635a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f56fdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 0x55ada635a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:46234/0 (socket says 172.21.15.62:46234) 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f56fdd9b640 1 -- 172.21.15.62:0/1654576857 learned_addr learned my addr 172.21.15.62:0/1654576857 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f56fe59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 0x55ada63559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f5706c9f740 1 -- 172.21.15.62:0/1654576857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f56e000e100 con 0x55ada6351d40 2024-09-11T23:55:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f5706c9f740 1 -- 172.21.15.62:0/1654576857 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56e000d360 con 0x55ada6346b60 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6351d40 0x55ada6356150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.162+0000 7f5706c9f740 10 client.0 _ll_register_callbacks cb 0x7f56f4009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f56fdd9b640 1 -- 172.21.15.62:0/1654576857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6351d40 msgr2=0x55ada6356150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6351d40 0x55ada6356150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f56fdd9b640 1 -- 172.21.15.62:0/1654576857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 msgr2=0x55ada63559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 0x55ada63559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f56fdd9b640 1 -- 172.21.15.62:0/1654576857 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ada63cb2a0 con 0x55ada6346b60 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f56fdd9b640 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 0x55ada635a690 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f56f4012f50 tx=0x7f56f401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4322689a037a55fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f56e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f56e5ffb640 1 -- 172.21.15.62:0/1654576857 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f403c040 con 0x55ada6346b60 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f56e5ffb640 1 -- 172.21.15.62:0/1654576857 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f56f4024040 con 0x55ada6346b60 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f5706c9f740 1 -- 172.21.15.62:0/1654576857 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56f00047a0 con 0x55ada6346b60 2024-09-11T23:55:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f56e5ffb640 1 -- 172.21.15.62:0/1654576857 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56f4010b60 con 0x55ada6346b60 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f5706c9f740 10 client.25555 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.163+0000 7f5706c9f740 1 -- 172.21.15.62:0/1654576857 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ada63cc480 con 0x55ada6346b60 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f56e5ffb640 1 -- 172.21.15.62:0/1654576857 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f00047a0 con 0x55ada6346b60 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f56e5ffb640 1 -- 172.21.15.62:0/1654576857 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f56f4040030 con 0x55ada6346b60 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f5706c9f740 10 client.25555 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f5706c9f740 10 client.25555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f5706c9f740 20 client.25555 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f5706c9f740 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ada63ccae0 0x55ada63cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f5706c9f740 4 client.25555 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f5706c9f740 1 -- 172.21.15.62:0/1654576857 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ada62b97f0 con 0x55ada63ccae0 2024-09-11T23:55:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f56fe59c640 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ada63ccae0 0x55ada63cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f56fe59c640 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ada63ccae0 0x55ada63cef00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9ef29bca1819a24 server_cookie=6edf05162c137454 in_seq=0 out_seq=0 2024-09-11T23:55:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.164+0000 7f56e5ffb640 10 client.25555 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f56e5ffb640 1 -- 172.21.15.62:0/1654576857 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f56f002c080 con 0x55ada63ccae0 2024-09-11T23:55:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f56e5ffb640 10 client.25555 handle_command_reply: tid=0 2024-09-11T23:55:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 client.25555 shutdown 2024-09-11T23:55:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 -- 172.21.15.62:0/1654576857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 msgr2=0x55ada635a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 0x55ada635a690 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f56f4012f50 tx=0x7f56f401a000 comp rx=0 tx=0).stop 2024-09-11T23:55:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 -- 172.21.15.62:0/1654576857 shutdown_connections 2024-09-11T23:55:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ada63ccae0 0x55ada63cef00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f56e5ffb640 0 client.25555 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ada6351d40 0x55ada6356150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ada6347480 0x55ada63559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 --2- 172.21.15.62:0/1654576857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ada6346b60 0x55ada635a690 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 -- 172.21.15.62:0/1654576857 >> 172.21.15.62:0/1654576857 conn(0x55ada63430f0 msgr2=0x55ada6348dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 -- 172.21.15.62:0/1654576857 shutdown_connections 2024-09-11T23:55:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 1 -- 172.21.15.62:0/1654576857 wait complete. 2024-09-11T23:55:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.166+0000 7f5706c9f740 20 client.25555 trim_cache size 0 max 16384 2024-09-11T23:55:12.170 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:55:12.170 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:55:12.170 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "1a2b97c3-57e3-4e35-9f51-81ffb230fbfd", 2024-09-11T23:55:12.171 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:55:12.171 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 -- 172.21.15.62:0/808017012 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f56e80a2df0 msgr2=0x7f56e80a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f56e80a2df0 0x7f56e80a52b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f56f80c5130 tx=0x7f56f80b8560 comp rx=0 tx=0).stop 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 -- 172.21.15.62:0/808017012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5700105560 msgr2=0x7f5700109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5700105560 0x7f5700109e40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f56f8005e30 tx=0x7f56f8031720 comp rx=0 tx=0).stop 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 -- 172.21.15.62:0/808017012 shutdown_connections 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f56e80a2df0 0x7f56e80a52b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5700105560 0x7f5700109e40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700100de0 0x7f570010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 --2- 172.21.15.62:0/808017012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5700100430 0x7f570010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 -- 172.21.15.62:0/808017012 >> 172.21.15.62:0/808017012 conn(0x7f5700077230 msgr2=0x7f5700103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 -- 172.21.15.62:0/808017012 shutdown_connections 2024-09-11T23:55:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:12.167+0000 7f57048c6640 1 -- 172.21.15.62:0/808017012 wait complete. 2024-09-11T23:55:12.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 150.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:55:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:13.013 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:55:13.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.424+0000 7f170e673640 1 -- 172.21.15.62:0/2422606136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1708101de0 msgr2=0x7f17081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:13.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.424+0000 7f170e673640 1 --2- 172.21.15.62:0/2422606136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1708101de0 0x7f17081021c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009900 tx=0x7f16f802a410 comp rx=0 tx=0).stop 2024-09-11T23:55:13.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.424+0000 7f170e673640 1 -- 172.21.15.62:0/2422606136 shutdown_connections 2024-09-11T23:55:13.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.424+0000 7f170e673640 1 --2- 172.21.15.62:0/2422606136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1708106f10 0x7f170810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.424+0000 7f170e673640 1 --2- 172.21.15.62:0/2422606136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708102790 0x7f17081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.425+0000 7f170e673640 1 --2- 172.21.15.62:0/2422606136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1708101de0 0x7f17081021c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.425+0000 7f170e673640 1 -- 172.21.15.62:0/2422606136 >> 172.21.15.62:0/2422606136 conn(0x7f17080fd8c0 msgr2=0x7f17080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:13.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.425+0000 7f170e673640 1 -- 172.21.15.62:0/2422606136 shutdown_connections 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.425+0000 7f170e673640 1 -- 172.21.15.62:0/2422606136 wait complete. 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.425+0000 7f170e673640 1 Processor -- start 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.425+0000 7f170e673640 1 -- start start 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f170e673640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 0x7f1708198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f170e673640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1708102790 0x7f1708198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f170e673640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1708106f10 0x7f170819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f170e673640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1708111fb0 con 0x7f1708102790 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f170e673640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1708111e30 con 0x7f1708106f10 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f170e673640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1708112130 con 0x7f1708101de0 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f17077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1708102790 0x7f1708198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f17077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1708102790 0x7f1708198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41318/0 (socket says 172.21.15.62:41318) 2024-09-11T23:55:13.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f17077fe640 1 -- 172.21.15.62:0/3601606070 learned_addr learned my addr 172.21.15.62:0/3601606070 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:13.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f1707fff640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 0x7f1708198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f170cc10640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1708106f10 0x7f170819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f17077fe640 1 -- 172.21.15.62:0/3601606070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 msgr2=0x7f1708198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:13.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f17077fe640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 0x7f1708198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f17077fe640 1 -- 172.21.15.62:0/3601606070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1708106f10 msgr2=0x7f170819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:13.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f17077fe640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1708106f10 0x7f170819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f17077fe640 1 -- 172.21.15.62:0/3601606070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170819d4e0 con 0x7f1708102790 2024-09-11T23:55:13.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f17077fe640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1708102790 0x7f1708198a70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c970 tx=0x7f16fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:13.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f17057fa640 1 -- 172.21.15.62:0/3601606070 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc007bb0 con 0x7f1708102790 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.426+0000 7f170e673640 1 -- 172.21.15.62:0/3601606070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170819d7d0 con 0x7f1708102790 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.427+0000 7f17057fa640 1 -- 172.21.15.62:0/3601606070 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16fc007d50 con 0x7f1708102790 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.427+0000 7f17057fa640 1 -- 172.21.15.62:0/3601606070 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00f450 con 0x7f1708102790 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.427+0000 7f170e673640 1 -- 172.21.15.62:0/3601606070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17081a50b0 con 0x7f1708102790 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.429+0000 7f17057fa640 1 -- 172.21.15.62:0/3601606070 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f16fc005020 con 0x7f1708102790 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f1710a4c740 1 Processor -- start 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f1710a4c740 1 -- start start 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0980 0x55794e1aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 0x55794e1ad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f1710a4c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55794e105620 con 0x55794e1a0060 2024-09-11T23:55:13.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f1710a4c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55794e0ace50 con 0x55794e1ab240 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f1710a4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55794e1015e0 con 0x55794e1a0980 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f170cc10640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f170cc10640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41330/0 (socket says 172.21.15.62:41330) 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f170cc10640 1 -- 172.21.15.62:0/1512703192 learned_addr learned my addr 172.21.15.62:0/1512703192 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f1707fff640 1 --2- 172.21.15.62:0/1512703192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0980 0x55794e1aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f17077fe640 1 --2- 172.21.15.62:0/1512703192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 0x55794e1ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f17057fa640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f16f00a2cf0 0x7f16f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f170cc10640 1 -- 172.21.15.62:0/1512703192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0980 msgr2=0x55794e1aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f170cc10640 1 --2- 172.21.15.62:0/1512703192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0980 0x55794e1aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f170cc10640 1 -- 172.21.15.62:0/1512703192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 msgr2=0x55794e1ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:13.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f170cc10640 1 --2- 172.21.15.62:0/1512703192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 0x55794e1ad630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f170cc10640 1 -- 172.21.15.62:0/1512703192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794e09dd40 con 0x55794e1a0060 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.430+0000 7f1707fff640 1 --2- 172.21.15.62:0/1512703192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0980 0x55794e1aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.431+0000 7f17057fa640 1 -- 172.21.15.62:0/3601606070 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f16fc0cf1b0 con 0x7f1708102790 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.431+0000 7f170cc10640 1 --2- 172.21.15.62:0/1512703192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1a0440 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f170000a430 tx=0x7f1700009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76297a0ee174b6c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.431+0000 7f170cc10640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f16f00a2cf0 0x7f16f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.431+0000 7f16ee7fc640 1 -- 172.21.15.62:0/1512703192 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700010830 con 0x55794e1a0060 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.431+0000 7f16ee7fc640 1 -- 172.21.15.62:0/1512703192 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1700010cd0 con 0x55794e1a0060 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.431+0000 7f16ee7fc640 1 -- 172.21.15.62:0/1512703192 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700020900 con 0x55794e1a0060 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 -- 172.21.15.62:0/1512703192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 msgr2=0x55794e1a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f170cc10640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f16f00a2cf0 0x7f16f00a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1700019000 tx=0x7f1700013750 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 --2- 172.21.15.62:0/1512703192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1a0440 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f170000a430 tx=0x7f1700009650 comp rx=0 tx=0).stop 2024-09-11T23:55:13.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 -- 172.21.15.62:0/1512703192 shutdown_connections 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 --2- 172.21.15.62:0/1512703192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 0x55794e1ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 --2- 172.21.15.62:0/1512703192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0980 0x55794e1aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 --2- 172.21.15.62:0/1512703192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1a0440 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 -- 172.21.15.62:0/1512703192 >> 172.21.15.62:0/1512703192 conn(0x55794e19c5f0 msgr2=0x55794e19c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 -- 172.21.15.62:0/1512703192 shutdown_connections 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 -- 172.21.15.62:0/1512703192 wait complete. 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 -- start start 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.432+0000 7f1710a4c740 1 Processor -- start 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1a0060 0x55794e1a8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e1a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 0x55794e1a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f1710a4c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55794e1ade10 con 0x55794e1a0980 2024-09-11T23:55:13.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f1710a4c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55794e08df10 con 0x55794e1a0060 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f1710a4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55794e1adf90 con 0x55794e1ab240 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f1710a4c740 10 client.0 _ll_register_callbacks cb 0x55794e0b1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f17077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e1a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f17077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e1a85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41342/0 (socket says 172.21.15.62:41342) 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f17077fe640 1 -- 172.21.15.62:0/2669286266 learned_addr learned my addr 172.21.15.62:0/2669286266 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f170cc10640 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 0x55794e1a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f1707fff640 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1a0060 0x55794e1a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f17077fe640 1 -- 172.21.15.62:0/2669286266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 msgr2=0x55794e1a8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f17077fe640 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 0x55794e1a8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f17077fe640 1 -- 172.21.15.62:0/2669286266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1a0060 msgr2=0x55794e1a8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f17077fe640 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1a0060 0x55794e1a8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f17077fe640 1 -- 172.21.15.62:0/2669286266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794de14580 con 0x55794e1a0980 2024-09-11T23:55:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f170cc10640 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 0x55794e1a8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f17077fe640 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e1a85c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0cc930 tx=0x7f16fc09b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f890b6cdeaf334 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.433+0000 7f16e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f16e77fe640 1 -- 172.21.15.62:0/2669286266 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc01d040 con 0x55794e1a0980 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f16e77fe640 1 -- 172.21.15.62:0/2669286266 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f16fc005310 con 0x55794e1a0980 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f1710a4c740 1 -- 172.21.15.62:0/2669286266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1700010cd0 con 0x55794e1a0980 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f16e77fe640 1 -- 172.21.15.62:0/2669286266 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16fc09fa50 con 0x55794e1a0980 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f1710a4c740 10 client.15948 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f1710a4c740 1 -- 172.21.15.62:0/2669286266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55794dede960 con 0x55794e1a0980 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f16e77fe640 1 -- 172.21.15.62:0/2669286266 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1700010cd0 con 0x55794e1a0980 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f16e77fe640 1 -- 172.21.15.62:0/2669286266 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f16fc0d1110 con 0x55794e1a0980 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f1710a4c740 10 client.15948 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f1710a4c740 10 client.15948 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:55:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f1710a4c740 20 client.15948 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55794e1bacb0 0x55794e1bd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f1710a4c740 4 client.15948 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f1710a4c740 1 -- 172.21.15.62:0/2669286266 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55794e112b90 con 0x55794e1bacb0 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.434+0000 7f1707fff640 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55794e1bacb0 0x55794e1bd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.435+0000 7f16e77fe640 10 client.15948 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.435+0000 7f1707fff640 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55794e1bacb0 0x55794e1bd0c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b1a5f0414776c2 server_cookie=3ac1a42712fa27d7 in_seq=0 out_seq=0 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f16e77fe640 1 -- 172.21.15.62:0/2669286266 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55794e112b90 con 0x55794e1bacb0 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f16e77fe640 10 client.15948 handle_command_reply: tid=0 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 client.15948 shutdown 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 -- 172.21.15.62:0/2669286266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 msgr2=0x55794e1a85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e1a85c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0cc930 tx=0x7f16fc09b290 comp rx=0 tx=0).stop 2024-09-11T23:55:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 -- 172.21.15.62:0/2669286266 shutdown_connections 2024-09-11T23:55:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55794e1bacb0 0x55794e1bd0c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 0x55794e1a8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e1a85c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f16e77fe640 0 client.15948 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2669286266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1a0060 0x55794e1a8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 -- 172.21.15.62:0/2669286266 >> 172.21.15.62:0/2669286266 conn(0x55794e19c5f0 msgr2=0x55794e19b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 -- 172.21.15.62:0/2669286266 shutdown_connections 2024-09-11T23:55:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 1 -- 172.21.15.62:0/2669286266 wait complete. 2024-09-11T23:55:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.436+0000 7f1710a4c740 20 client.15948 trim_cache size 0 max 16384 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.453+0000 7f1710a4c740 1 Processor -- start 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f1710a4c740 1 -- start start 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0060 0x55794e1a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e19b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 0x55794e1c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f1710a4c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f17081b0180 con 0x55794e1a0980 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f1710a4c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17081ada50 con 0x55794e1ab240 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f1710a4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17081ab8a0 con 0x55794e1a0060 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f17077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f17077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e19b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41348/0 (socket says 172.21.15.62:41348) 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f17077fe640 1 -- 172.21.15.62:0/1637391676 learned_addr learned my addr 172.21.15.62:0/1637391676 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f1707fff640 1 --2- 172.21.15.62:0/1637391676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0060 0x55794e1a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f17077fe640 1 -- 172.21.15.62:0/1637391676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0060 msgr2=0x55794e1a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f17077fe640 1 --2- 172.21.15.62:0/1637391676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0060 0x55794e1a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f170cc10640 1 --2- 172.21.15.62:0/1637391676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 0x55794e1c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f17077fe640 1 -- 172.21.15.62:0/1637391676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 msgr2=0x55794e1c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f17077fe640 1 --2- 172.21.15.62:0/1637391676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 0x55794e1c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f17077fe640 1 -- 172.21.15.62:0/1637391676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794e1aa9f0 con 0x55794e1a0980 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.454+0000 7f170cc10640 1 --2- 172.21.15.62:0/1637391676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 0x55794e1c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.455+0000 7f17077fe640 1 --2- 172.21.15.62:0/1637391676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e19b640 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0049d0 tx=0x7f16fc09b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638bb21f42890850 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.455+0000 7f16edffb640 1 -- 172.21.15.62:0/1637391676 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc01d070 con 0x55794e1a0980 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.455+0000 7f16edffb640 1 -- 172.21.15.62:0/1637391676 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16fc0a4310 con 0x55794e1a0980 2024-09-11T23:55:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.455+0000 7f16edffb640 1 -- 172.21.15.62:0/1637391676 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0afbc0 con 0x55794e1a0980 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 -- 172.21.15.62:0/1637391676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 msgr2=0x55794e19b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 --2- 172.21.15.62:0/1637391676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e19b640 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0049d0 tx=0x7f16fc09b3c0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 -- 172.21.15.62:0/1637391676 shutdown_connections 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 --2- 172.21.15.62:0/1637391676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1ab240 0x55794e1c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 --2- 172.21.15.62:0/1637391676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0980 0x55794e19b640 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 --2- 172.21.15.62:0/1637391676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1a0060 0x55794e1a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 -- 172.21.15.62:0/1637391676 >> 172.21.15.62:0/1637391676 conn(0x55794e19c5f0 msgr2=0x55794e19af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 -- 172.21.15.62:0/1637391676 shutdown_connections 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 -- 172.21.15.62:0/1637391676 wait complete. 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 -- start start 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.456+0000 7f1710a4c740 1 Processor -- start 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1a0980 0x55794e1b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1710a4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 0x55794e1ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1710a4c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f17081abf10 con 0x55794e1a0060 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1710a4c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17081ad380 con 0x55794e1a0980 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1710a4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17081ac5e0 con 0x55794e1ab240 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1710a4c740 10 client.0 _ll_register_callbacks cb 0x7f16fc00a4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1707fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1707fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41362/0 (socket says 172.21.15.62:41362) 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1707fff640 1 -- 172.21.15.62:0/2893502801 learned_addr learned my addr 172.21.15.62:0/2893502801 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f170cc10640 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 0x55794e1ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f17077fe640 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1a0980 0x55794e1b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1707fff640 1 -- 172.21.15.62:0/2893502801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 msgr2=0x55794e1ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1707fff640 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 0x55794e1ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1707fff640 1 -- 172.21.15.62:0/2893502801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1a0980 msgr2=0x55794e1b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1707fff640 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1a0980 0x55794e1b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1707fff640 1 -- 172.21.15.62:0/2893502801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794e1ae8e0 con 0x55794e1a0060 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f1707fff640 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1b33d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f16f802ac00 tx=0x7f16f8031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5562f34053a3de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.457+0000 7f16e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f16e77fe640 1 -- 172.21.15.62:0/2893502801 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f8035040 con 0x55794e1a0060 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f16e77fe640 1 -- 172.21.15.62:0/2893502801 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f16f8004450 con 0x55794e1a0060 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f1710a4c740 1 -- 172.21.15.62:0/2893502801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16fc0a4310 con 0x55794e1a0060 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f16e77fe640 1 -- 172.21.15.62:0/2893502801 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16f8005040 con 0x55794e1a0060 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f1710a4c740 10 client.15954 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f1710a4c740 1 -- 172.21.15.62:0/2893502801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55794e2263b0 con 0x55794e1a0060 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f16e77fe640 1 -- 172.21.15.62:0/2893502801 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16fc0a4310 con 0x55794e1a0060 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f16e77fe640 1 -- 172.21.15.62:0/2893502801 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f16f80035f0 con 0x55794e1a0060 2024-09-11T23:55:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f1710a4c740 10 client.15954 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:13.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f1710a4c740 10 client.15954 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:55:13.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f1710a4c740 20 client.15954 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:13.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55794e1bacb0 0x55794e228b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:13.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f1710a4c740 4 client.15954 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "63cb6b6e", "release": true, "await": true}] 2024-09-11T23:55:13.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.458+0000 7f1710a4c740 1 -- 172.21.15.62:0/2893502801 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "63cb6b6e", "release": true, "await": true}) -- 0x55794e10d0a0 con 0x55794e1bacb0 2024-09-11T23:55:13.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.459+0000 7f17077fe640 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55794e1bacb0 0x55794e228b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:13.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.459+0000 7f16e77fe640 10 client.15954 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:13.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.459+0000 7f17077fe640 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55794e1bacb0 0x55794e228b50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a76894cfe8642d09 server_cookie=ed6c912fafb10be7 in_seq=0 out_seq=0 2024-09-11T23:55:13.483 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:13.516 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:13 smithi062 ceph-mon[30958]: pgmap v96: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 518 op/s 2024-09-11T23:55:13.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:13 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:55:13.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:13 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:13 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:55:13.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:13 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:55:13.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:13 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:13 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:55:13.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:13 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:13 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.517 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 27, 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 12, 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "63cb6b6e": { 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 12, 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:55:13.518 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 144.7, 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:55:13.519 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:55:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.511+0000 7f16e77fe640 1 -- 172.21.15.62:0/2893502801 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55794e10d0a0 con 0x55794e1bacb0 2024-09-11T23:55:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.511+0000 7f16e77fe640 10 client.15954 handle_command_reply: tid=0 2024-09-11T23:55:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.511+0000 7f1710a4c740 1 client.15954 shutdown 2024-09-11T23:55:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.511+0000 7f1710a4c740 1 -- 172.21.15.62:0/2893502801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 msgr2=0x55794e1b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.511+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1b33d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f16f802ac00 tx=0x7f16f8031ab0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.512+0000 7f1710a4c740 1 -- 172.21.15.62:0/2893502801 shutdown_connections 2024-09-11T23:55:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.512+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55794e1bacb0 0x55794e228b50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.512+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55794e1ab240 0x55794e1ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.512+0000 7f16e77fe640 0 client.15954 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.512+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55794e1a0980 0x55794e1b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.512+0000 7f1710a4c740 1 --2- 172.21.15.62:0/2893502801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55794e1a0060 0x55794e1b33d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.512+0000 7f1710a4c740 1 -- 172.21.15.62:0/2893502801 >> 172.21.15.62:0/2893502801 conn(0x55794e19c5f0 msgr2=0x55794e1a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.512+0000 7f1710a4c740 1 -- 172.21.15.62:0/2893502801 shutdown_connections 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.512+0000 7f1710a4c740 1 -- 172.21.15.62:0/2893502801 wait complete. 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.512+0000 7f1710a4c740 20 client.15954 trim_cache size 0 max 16384 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.513+0000 7f170e673640 1 -- 172.21.15.62:0/3601606070 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f16f00a2cf0 msgr2=0x7f16f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.513+0000 7f170e673640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f16f00a2cf0 0x7f16f00a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1700019000 tx=0x7f1700013750 comp rx=0 tx=0).stop 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.513+0000 7f170e673640 1 -- 172.21.15.62:0/3601606070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1708102790 msgr2=0x7f1708198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.513+0000 7f170e673640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1708102790 0x7f1708198a70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c970 tx=0x7f16fc00ce30 comp rx=0 tx=0).stop 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.513+0000 7f170e673640 1 -- 172.21.15.62:0/3601606070 shutdown_connections 2024-09-11T23:55:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.513+0000 7f170e673640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f16f00a2cf0 0x7f16f00a51b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.513+0000 7f170e673640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1708106f10 0x7f170819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.513+0000 7f170e673640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1708102790 0x7f1708198a70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.513+0000 7f170e673640 1 --2- 172.21.15.62:0/3601606070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 0x7f1708198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:13.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.513+0000 7f170e673640 1 -- 172.21.15.62:0/3601606070 >> 172.21.15.62:0/3601606070 conn(0x7f17080fd8c0 msgr2=0x7f17080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:13.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.517+0000 7f170e673640 1 -- 172.21.15.62:0/3601606070 shutdown_connections 2024-09-11T23:55:13.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:13.518+0000 7f170e673640 1 -- 172.21.15.62:0/3601606070 wait complete. 2024-09-11T23:55:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:13 smithi120 ceph-mon[35182]: pgmap v96: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 518 op/s 2024-09-11T23:55:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:13 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:55:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:13 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:13 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:55:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:13 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:55:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:13 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:13 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:55:13.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:13 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:13 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:13 smithi083 ceph-mon[34805]: pgmap v96: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 518 op/s 2024-09-11T23:55:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:13 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:55:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:13 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:13 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:55:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:13 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:55:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:13 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:13 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:55:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:13 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:13.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:13 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:55:14.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.896+0000 7f55b95f1640 1 -- 172.21.15.62:0/1625932465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4104000 msgr2=0x7f55b41043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:14.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.896+0000 7f55b95f1640 1 --2- 172.21.15.62:0/1625932465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4104000 0x7f55b41043e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009900 tx=0x7f55a402a410 comp rx=0 tx=0).stop 2024-09-11T23:55:14.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.896+0000 7f55b95f1640 1 -- 172.21.15.62:0/1625932465 shutdown_connections 2024-09-11T23:55:14.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.896+0000 7f55b95f1640 1 --2- 172.21.15.62:0/1625932465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55b4109130 0x7f55b4110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.896+0000 7f55b95f1640 1 --2- 172.21.15.62:0/1625932465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b41049b0 0x7f55b4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.896+0000 7f55b95f1640 1 --2- 172.21.15.62:0/1625932465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4104000 0x7f55b41043e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.896+0000 7f55b95f1640 1 -- 172.21.15.62:0/1625932465 >> 172.21.15.62:0/1625932465 conn(0x7f55b40ffa20 msgr2=0x7f55b4101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:14.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.896+0000 7f55b95f1640 1 -- 172.21.15.62:0/1625932465 shutdown_connections 2024-09-11T23:55:14.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.896+0000 7f55b95f1640 1 -- 172.21.15.62:0/1625932465 wait complete. 2024-09-11T23:55:14.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.898+0000 7f55b95f1640 1 Processor -- start 2024-09-11T23:55:14.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.898+0000 7f55b95f1640 1 -- start start 2024-09-11T23:55:14.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b95f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b4104000 0x7f55b419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b95f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55b41049b0 0x7f55b419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b95f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4109130 0x7f55b419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b4104000 0x7f55b419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b95f1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4114180 con 0x7f55b4109130 2024-09-11T23:55:14.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b95f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4114000 con 0x7f55b4104000 2024-09-11T23:55:14.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b95f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4114300 con 0x7f55b41049b0 2024-09-11T23:55:14.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b4104000 0x7f55b419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:33358/0 (socket says 172.21.15.62:33358) 2024-09-11T23:55:14.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b2ffd640 1 -- 172.21.15.62:0/3376872826 learned_addr learned my addr 172.21.15.62:0/3376872826 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:14.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b37fe640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4109130 0x7f55b419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55b41049b0 0x7f55b419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b2ffd640 1 -- 172.21.15.62:0/3376872826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55b41049b0 msgr2=0x7f55b419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:14.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55b41049b0 0x7f55b419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b2ffd640 1 -- 172.21.15.62:0/3376872826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4109130 msgr2=0x7f55b419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:14.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.899+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4109130 0x7f55b419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.900+0000 7f55b2ffd640 1 -- 172.21.15.62:0/3376872826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b419f3e0 con 0x7f55b4104000 2024-09-11T23:55:14.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.900+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b4104000 0x7f55b419a430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009850 tx=0x7f55a4030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:14.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.901+0000 7f55b0ff9640 1 -- 172.21.15.62:0/3376872826 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a403f070 con 0x7f55b4104000 2024-09-11T23:55:14.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.901+0000 7f55b0ff9640 1 -- 172.21.15.62:0/3376872826 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a4004460 con 0x7f55b4104000 2024-09-11T23:55:14.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.901+0000 7f55b0ff9640 1 -- 172.21.15.62:0/3376872826 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4035030 con 0x7f55b4104000 2024-09-11T23:55:14.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.901+0000 7f55b95f1640 1 -- 172.21.15.62:0/3376872826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b419f670 con 0x7f55b4104000 2024-09-11T23:55:14.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.901+0000 7f55b95f1640 1 -- 172.21.15.62:0/3376872826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b419fa90 con 0x7f55b4104000 2024-09-11T23:55:14.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.904+0000 7f55b0ff9640 1 -- 172.21.15.62:0/3376872826 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f55a4042070 con 0x7f55b4104000 2024-09-11T23:55:14.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.904+0000 7f55b0ff9640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55a00a2e40 0x7f55a00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55b0ff9640 1 -- 172.21.15.62:0/3376872826 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f55a40efc00 con 0x7f55b4104000 2024-09-11T23:55:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55a00a2e40 0x7f55a00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55bb9ca740 1 Processor -- start 2024-09-11T23:55:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55bb9ca740 1 -- start start 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55a00a2e40 0x7f55a00a5300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f55b4100220 tx=0x7f559c005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d57174a0 0x55c2d5717880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d5717dc0 0x55c2d5722140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d5724a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55bb9ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c2d567d2e0 con 0x55c2d5717dc0 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55bb9ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5624ce0 con 0x55c2d57174a0 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.905+0000 7f55bb9ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c2d56792a0 con 0x55c2d5722680 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.906+0000 7f55b37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d5724a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.906+0000 7f55b37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d5724a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:46328/0 (socket says 172.21.15.62:46328) 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.906+0000 7f55b37fe640 1 -- 172.21.15.62:0/1427214199 learned_addr learned my addr 172.21.15.62:0/1427214199 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.906+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/1427214199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d57174a0 0x55c2d5717880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.906+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/1427214199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d5717dc0 0x55c2d5722140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.907+0000 7f55b2ffd640 1 -- 172.21.15.62:0/1427214199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 msgr2=0x55c2d5724a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.907+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/1427214199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d5724a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.907+0000 7f55b2ffd640 1 -- 172.21.15.62:0/1427214199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d5717dc0 msgr2=0x55c2d5722140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.907+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/1427214199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d5717dc0 0x55c2d5722140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.907+0000 7f55b2ffd640 1 -- 172.21.15.62:0/1427214199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d5609440 con 0x55c2d57174a0 2024-09-11T23:55:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.907+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/1427214199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d57174a0 0x55c2d5717880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f55a40bce90 tx=0x7f55a40bcd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f39a940c8826bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.907+0000 7f55a97fa640 1 -- 172.21.15.62:0/1427214199 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4049040 con 0x55c2d57174a0 2024-09-11T23:55:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.907+0000 7f55a97fa640 1 -- 172.21.15.62:0/1427214199 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a40c1490 con 0x55c2d57174a0 2024-09-11T23:55:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.907+0000 7f55a97fa640 1 -- 172.21.15.62:0/1427214199 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a40ed680 con 0x55c2d57174a0 2024-09-11T23:55:14.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.910+0000 7f55bb9ca740 1 -- 172.21.15.62:0/1427214199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d57174a0 msgr2=0x55c2d5717880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.910+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/1427214199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d57174a0 0x55c2d5717880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f55a40bce90 tx=0x7f55a40bcd90 comp rx=0 tx=0).stop 2024-09-11T23:55:14.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.910+0000 7f55bb9ca740 1 -- 172.21.15.62:0/1427214199 shutdown_connections 2024-09-11T23:55:14.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.910+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/1427214199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d5724a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.910+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/1427214199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d5717dc0 0x55c2d5722140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.910+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/1427214199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d57174a0 0x55c2d5717880 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.911+0000 7f55bb9ca740 1 -- 172.21.15.62:0/1427214199 >> 172.21.15.62:0/1427214199 conn(0x55c2d5713a30 msgr2=0x55c2d5713e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:14.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.911+0000 7f55bb9ca740 1 -- 172.21.15.62:0/1427214199 shutdown_connections 2024-09-11T23:55:14.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.911+0000 7f55bb9ca740 1 -- 172.21.15.62:0/1427214199 wait complete. 2024-09-11T23:55:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.912+0000 7f55bb9ca740 1 -- start start 2024-09-11T23:55:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.912+0000 7f55bb9ca740 1 Processor -- start 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 0x55c2d5722060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5717dc0 0x55c2d571f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5722680 0x55c2d571f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55b2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 0x55c2d5722060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55b2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 0x55c2d5722060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41402/0 (socket says 172.21.15.62:41402) 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55b2ffd640 1 -- 172.21.15.62:0/2358466969 learned_addr learned my addr 172.21.15.62:0/2358466969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5725010 con 0x55c2d57174a0 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5638050 con 0x55c2d5722680 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c2d5725190 con 0x55c2d5717dc0 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5717dc0 0x55c2d571f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55bb9ca740 10 client.0 _ll_register_callbacks cb 0x55c2d5678160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:14.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55b37fe640 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5722680 0x55c2d571f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55aa7fc640 1 -- 172.21.15.62:0/2358466969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5722680 msgr2=0x55c2d571f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:14.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5722680 0x55c2d571f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55aa7fc640 1 -- 172.21.15.62:0/2358466969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 msgr2=0x55c2d5722060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:14.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 0x55c2d5722060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55aa7fc640 1 -- 172.21.15.62:0/2358466969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d53ee3c0 con 0x55c2d5717dc0 2024-09-11T23:55:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5717dc0 0x55c2d571f310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f559c009a90 tx=0x7f559c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e7157c7ccb125e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.913+0000 7f559a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:55:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f559a7fc640 1 -- 172.21.15.62:0/2358466969 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c00e6a0 con 0x55c2d5717dc0 2024-09-11T23:55:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f559a7fc640 1 -- 172.21.15.62:0/2358466969 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f559c00e840 con 0x55c2d5717dc0 2024-09-11T23:55:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55a40c1490 con 0x55c2d5717dc0 2024-09-11T23:55:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f559a7fc640 1 -- 172.21.15.62:0/2358466969 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f559c00ea40 con 0x55c2d5717dc0 2024-09-11T23:55:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f55bb9ca740 10 client.25582 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2d5730a20 con 0x55c2d5717dc0 2024-09-11T23:55:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f559a7fc640 1 -- 172.21.15.62:0/2358466969 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a40c1490 con 0x55c2d5717dc0 2024-09-11T23:55:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f559a7fc640 1 -- 172.21.15.62:0/2358466969 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f559c027c60 con 0x55c2d5717dc0 2024-09-11T23:55:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f55bb9ca740 10 client.25582 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f55bb9ca740 10 client.25582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f55bb9ca740 20 client.25582 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c2d5731fa0 0x55c2d5734390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f55bb9ca740 4 client.25582 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.914+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2d568a850 con 0x55c2d5731fa0 2024-09-11T23:55:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.915+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c2d5731fa0 0x55c2d5734390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.915+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c2d5731fa0 0x55c2d5734390 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b331d945ce9a653c server_cookie=d2dfd0a37c79628e in_seq=0 out_seq=0 2024-09-11T23:55:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.915+0000 7f559a7fc640 10 client.25582 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.916+0000 7f559a7fc640 1 -- 172.21.15.62:0/2358466969 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2d568a850 con 0x55c2d5731fa0 2024-09-11T23:55:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.916+0000 7f559a7fc640 10 client.25582 handle_command_reply: tid=0 2024-09-11T23:55:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.916+0000 7f55bb9ca740 1 client.25582 shutdown 2024-09-11T23:55:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.917+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5717dc0 msgr2=0x55c2d571f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.917+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5717dc0 0x55c2d571f310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f559c009a90 tx=0x7f559c004730 comp rx=0 tx=0).stop 2024-09-11T23:55:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.917+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 shutdown_connections 2024-09-11T23:55:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.917+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c2d5731fa0 0x55c2d5734390 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.917+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5722680 0x55c2d571f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.917+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5717dc0 0x55c2d571f310 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f559c009a90 tx=0x7f559c004730 comp rx=0 tx=0).stop 2024-09-11T23:55:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.917+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/2358466969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 0x55c2d5722060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.917+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 >> 172.21.15.62:0/2358466969 conn(0x55c2d5713a30 msgr2=0x55c2d5719760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.917+0000 7f559a7fc640 0 client.25582 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.918+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 shutdown_connections 2024-09-11T23:55:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.918+0000 7f55bb9ca740 1 -- 172.21.15.62:0/2358466969 wait complete. 2024-09-11T23:55:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.918+0000 7f55bb9ca740 20 client.25582 trim_cache size 0 max 16384 2024-09-11T23:55:14.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.949+0000 7f55bb9ca740 1 Processor -- start 2024-09-11T23:55:14.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.952+0000 7f55bb9ca740 1 -- start start 2024-09-11T23:55:14.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.952+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d57174a0 0x55c2d572e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.952+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d5722680 0x55c2d5713e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.952+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5739ca0 0x55c2d573a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.952+0000 7f55bb9ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55b41b22c0 con 0x55c2d5722680 2024-09-11T23:55:14.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.952+0000 7f55bb9ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55b41afb90 con 0x55c2d5739ca0 2024-09-11T23:55:14.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.952+0000 7f55bb9ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55b41ad9e0 con 0x55c2d57174a0 2024-09-11T23:55:14.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.953+0000 7f55b37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5739ca0 0x55c2d573a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.953+0000 7f55b37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5739ca0 0x55c2d573a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:33390/0 (socket says 172.21.15.62:33390) 2024-09-11T23:55:14.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.953+0000 7f55b37fe640 1 -- 172.21.15.62:0/3572357549 learned_addr learned my addr 172.21.15.62:0/3572357549 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:14.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.954+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/3572357549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d57174a0 0x55c2d572e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.956+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/3572357549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d5722680 0x55c2d5713e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.960+0000 7f55b37fe640 1 -- 172.21.15.62:0/3572357549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d57174a0 msgr2=0x55c2d572e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.960+0000 7f55b37fe640 1 --2- 172.21.15.62:0/3572357549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d57174a0 0x55c2d572e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.960+0000 7f55b37fe640 1 -- 172.21.15.62:0/3572357549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d5722680 msgr2=0x55c2d5713e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:14.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.960+0000 7f55b37fe640 1 --2- 172.21.15.62:0/3572357549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d5722680 0x55c2d5713e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.960+0000 7f55b37fe640 1 -- 172.21.15.62:0/3572357549 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d5722390 con 0x55c2d5739ca0 2024-09-11T23:55:14.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.960+0000 7f55b37fe640 1 --2- 172.21.15.62:0/3572357549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5739ca0 0x55c2d573a100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f55ac003cc0 tx=0x7f55ac004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4261e9e2cafdcda9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:14.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.961+0000 7f55a8ff9640 1 -- 172.21.15.62:0/3572357549 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac004b20 con 0x55c2d5739ca0 2024-09-11T23:55:14.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.961+0000 7f55a8ff9640 1 -- 172.21.15.62:0/3572357549 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ac004cc0 con 0x55c2d5739ca0 2024-09-11T23:55:14.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.962+0000 7f55bb9ca740 1 -- 172.21.15.62:0/3572357549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5739ca0 msgr2=0x55c2d573a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:14.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.962+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/3572357549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5739ca0 0x55c2d573a100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f55ac003cc0 tx=0x7f55ac004410 comp rx=0 tx=0).stop 2024-09-11T23:55:14.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.962+0000 7f55bb9ca740 1 -- 172.21.15.62:0/3572357549 shutdown_connections 2024-09-11T23:55:14.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.962+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/3572357549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d5739ca0 0x55c2d573a100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.962+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/3572357549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d5722680 0x55c2d5713e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.962+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/3572357549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d57174a0 0x55c2d572e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.962+0000 7f55bb9ca740 1 -- 172.21.15.62:0/3572357549 >> 172.21.15.62:0/3572357549 conn(0x55c2d5713a30 msgr2=0x55c2d571e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:14.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.966+0000 7f55bb9ca740 1 -- 172.21.15.62:0/3572357549 shutdown_connections 2024-09-11T23:55:14.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.967+0000 7f55bb9ca740 1 -- 172.21.15.62:0/3572357549 wait complete. 2024-09-11T23:55:14.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.967+0000 7f55bb9ca740 1 -- start start 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.967+0000 7f55bb9ca740 1 Processor -- start 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 0x55c2d57a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d572a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55bb9ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d572a880 0x55c2d579e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55bb9ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55b41ae050 con 0x55c2d57174a0 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55bb9ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55b41af4c0 con 0x55c2d572a880 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55bb9ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55b41ae720 con 0x55c2d5722680 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55bb9ca740 10 client.0 _ll_register_callbacks cb 0x7f559c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d572a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55b2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 0x55c2d57a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d572a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:46352/0 (socket says 172.21.15.62:46352) 2024-09-11T23:55:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55aa7fc640 1 -- 172.21.15.62:0/4231609266 learned_addr learned my addr 172.21.15.62:0/4231609266 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:14.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.968+0000 7f55b37fe640 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d572a880 0x55c2d579e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.969+0000 7f55aa7fc640 1 -- 172.21.15.62:0/4231609266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d572a880 msgr2=0x55c2d579e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:14.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.969+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d572a880 0x55c2d579e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.969+0000 7f55aa7fc640 1 -- 172.21.15.62:0/4231609266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 msgr2=0x55c2d57a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.969+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 0x55c2d57a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.969+0000 7f55aa7fc640 1 -- 172.21.15.62:0/4231609266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d572af90 con 0x55c2d5722680 2024-09-11T23:55:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.969+0000 7f55b37fe640 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d572a880 0x55c2d579e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.969+0000 7f55aa7fc640 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d572a2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f559c0118d0 tx=0x7f559c00e6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2594036c1a89eb10 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.969+0000 7f559a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:55:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.970+0000 7f559a7fc640 1 -- 172.21.15.62:0/4231609266 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c03d060 con 0x55c2d5722680 2024-09-11T23:55:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.970+0000 7f55bb9ca740 1 -- 172.21.15.62:0/4231609266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c2d5684d60 con 0x55c2d5722680 2024-09-11T23:55:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.970+0000 7f559a7fc640 1 -- 172.21.15.62:0/4231609266 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f559c009040 con 0x55c2d5722680 2024-09-11T23:55:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.970+0000 7f559a7fc640 1 -- 172.21.15.62:0/4231609266 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f559c034040 con 0x55c2d5722680 2024-09-11T23:55:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.972+0000 7f55bb9ca740 10 client.25588 fetch_fsmap learned FSMap version 27 2024-09-11T23:55:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.972+0000 7f55bb9ca740 1 -- 172.21.15.62:0/4231609266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2d57952d0 con 0x55c2d5722680 2024-09-11T23:55:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.973+0000 7f559a7fc640 1 -- 172.21.15.62:0/4231609266 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c2d5684d60 con 0x55c2d5722680 2024-09-11T23:55:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.973+0000 7f559a7fc640 1 -- 172.21.15.62:0/4231609266 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f559c042480 con 0x55c2d5722680 2024-09-11T23:55:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.974+0000 7f55bb9ca740 10 client.25588 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:55:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.974+0000 7f55bb9ca740 10 client.25588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.974+0000 7f55bb9ca740 20 client.25588 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.974+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c2d5795a00 0x55c2d5797e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.974+0000 7f55bb9ca740 4 client.25588 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:55:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.974+0000 7f55bb9ca740 1 -- 172.21.15.62:0/4231609266 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2d5601210 con 0x55c2d5795a00 2024-09-11T23:55:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.975+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c2d5795a00 0x55c2d5797e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.975+0000 7f55b2ffd640 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c2d5795a00 0x55c2d5797e20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f345f7131fe93b server_cookie=17b30a11f7e8792a in_seq=0 out_seq=0 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f559a7fc640 10 client.25588 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f559a7fc640 1 -- 172.21.15.62:0/4231609266 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c2d5601210 con 0x55c2d5795a00 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f559a7fc640 10 client.25588 handle_command_reply: tid=0 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 client.25588 shutdown 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 -- 172.21.15.62:0/4231609266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 msgr2=0x55c2d572a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d572a2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f559c0118d0 tx=0x7f559c00e6a0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 -- 172.21.15.62:0/4231609266 shutdown_connections 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c2d5795a00 0x55c2d5797e20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2d572a880 0x55c2d579e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2d5722680 0x55c2d572a2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 --2- 172.21.15.62:0/4231609266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d57174a0 0x55c2d57a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 -- 172.21.15.62:0/4231609266 >> 172.21.15.62:0/4231609266 conn(0x55c2d5713a30 msgr2=0x55c2d5720b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f559a7fc640 0 client.25588 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 -- 172.21.15.62:0/4231609266 shutdown_connections 2024-09-11T23:55:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.976+0000 7f55bb9ca740 1 -- 172.21.15.62:0/4231609266 wait complete. 2024-09-11T23:55:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55bb9ca740 20 client.25588 trim_cache size 0 max 16384 2024-09-11T23:55:14.978 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:55:14.978 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:55:14.978 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:55:14.978 INFO:teuthology.orchestra.run.smithi062.stdout: "1a2b97c3-57e3-4e35-9f51-81ffb230fbfd": { 2024-09-11T23:55:14.979 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:55:14.979 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "1a2b97c3-57e3-4e35-9f51-81ffb230fbfd", 2024-09-11T23:55:14.979 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:55:14.979 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:55:14.979 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:55:14.979 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:55:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55b95f1640 1 -- 172.21.15.62:0/3376872826 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55a00a2e40 msgr2=0x7f55a00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55b95f1640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55a00a2e40 0x7f55a00a5300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f55b4100220 tx=0x7f559c005f90 comp rx=0 tx=0).stop 2024-09-11T23:55:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55b95f1640 1 -- 172.21.15.62:0/3376872826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b4104000 msgr2=0x7f55b419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55b95f1640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b4104000 0x7f55b419a430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009850 tx=0x7f55a4030650 comp rx=0 tx=0).stop 2024-09-11T23:55:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55b95f1640 1 -- 172.21.15.62:0/3376872826 shutdown_connections 2024-09-11T23:55:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55b95f1640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55a00a2e40 0x7f55a00a5300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55b95f1640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4109130 0x7f55b419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55b95f1640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55b41049b0 0x7f55b419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55b95f1640 1 --2- 172.21.15.62:0/3376872826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b4104000 0x7f55b419a430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.977+0000 7f55b95f1640 1 -- 172.21.15.62:0/3376872826 >> 172.21.15.62:0/3376872826 conn(0x7f55b40ffa20 msgr2=0x7f55b4100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.978+0000 7f55b95f1640 1 -- 172.21.15.62:0/3376872826 shutdown_connections 2024-09-11T23:55:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:14.978+0000 7f55b95f1640 1 -- 172.21.15.62:0/3376872826 wait complete. 2024-09-11T23:55:15.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:15 smithi062 ceph-mon[30958]: pgmap v97: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 594 op/s 2024-09-11T23:55:15.537 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '63cb6b6e', total seconds elapsed: 30.2 2024-09-11T23:55:15.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:15 smithi120 ceph-mon[35182]: pgmap v97: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 594 op/s 2024-09-11T23:55:15.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:15 smithi083 ceph-mon[34805]: pgmap v97: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 594 op/s 2024-09-11T23:55:16.531 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:16 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:55:16.532 INFO:tasks.cephfs.filesystem:scrub status for tag:1a2b97c3-57e3-4e35-9f51-81ffb230fbfd - {'path': '/', 'tag': '1a2b97c3-57e3-4e35-9f51-81ffb230fbfd', 'options': 'recursive,force'} 2024-09-11T23:55:16.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:16 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:55:16.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:16 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:55:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 155.0 s, 2642.2 tps, lat 219.854 ms stddev 2179.912 2024-09-11T23:55:17.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:17 smithi120 ceph-mon[35182]: pgmap v98: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 594 op/s 2024-09-11T23:55:17.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:55:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:55:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:55:17.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:17 smithi083 ceph-mon[34805]: pgmap v98: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 594 op/s 2024-09-11T23:55:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:55:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:55:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:55:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:17 smithi062 ceph-mon[30958]: pgmap v98: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 594 op/s 2024-09-11T23:55:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:55:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:55:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:55:19.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:19 smithi120 ceph-mon[35182]: pgmap v99: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.20k op/s 2024-09-11T23:55:19.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:19 smithi083 ceph-mon[34805]: pgmap v99: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.20k op/s 2024-09-11T23:55:19.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:19 smithi062 ceph-mon[30958]: pgmap v99: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 1.20k op/s 2024-09-11T23:55:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:21 smithi120 ceph-mon[35182]: pgmap v100: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.19k op/s 2024-09-11T23:55:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:21 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:55:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:21 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:55:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:21 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:55:21.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:21 smithi083 ceph-mon[34805]: pgmap v100: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.19k op/s 2024-09-11T23:55:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:21 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:55:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:21 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:55:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:21 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:55:21.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:21 smithi062 ceph-mon[30958]: pgmap v100: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.19k op/s 2024-09-11T23:55:21.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:21 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:55:21.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:21 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:55:21.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:21 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:55:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 160.0 s, 3194.6 tps, lat 31.289 ms stddev 31.895 2024-09-11T23:55:22.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:22 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:55:22.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:22 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:55:22.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:22 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:55:23.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:23 smithi120 ceph-mon[35182]: pgmap v101: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 1.41k op/s 2024-09-11T23:55:23.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:23 smithi083 ceph-mon[34805]: pgmap v101: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 1.41k op/s 2024-09-11T23:55:23.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:23 smithi062 ceph-mon[30958]: pgmap v101: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 1.41k op/s 2024-09-11T23:55:25.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:25 smithi120 ceph-mon[35182]: pgmap v102: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 18 MiB/s wr, 1.00k op/s 2024-09-11T23:55:25.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:25 smithi083 ceph-mon[34805]: pgmap v102: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 18 MiB/s wr, 1.00k op/s 2024-09-11T23:55:25.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:25 smithi062 ceph-mon[30958]: pgmap v102: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 18 MiB/s wr, 1.00k op/s 2024-09-11T23:55:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 165.0 s, 2919.4 tps, lat 34.123 ms stddev 35.035 2024-09-11T23:55:27.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:27 smithi120 ceph-mon[35182]: pgmap v103: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 907 op/s 2024-09-11T23:55:27.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:27 smithi083 ceph-mon[34805]: pgmap v103: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 907 op/s 2024-09-11T23:55:27.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:27 smithi062 ceph-mon[30958]: pgmap v103: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 907 op/s 2024-09-11T23:55:29.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:29 smithi120 ceph-mon[35182]: pgmap v104: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 1.21k op/s 2024-09-11T23:55:29.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:29 smithi083 ceph-mon[34805]: pgmap v104: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 1.21k op/s 2024-09-11T23:55:29.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:29 smithi062 ceph-mon[30958]: pgmap v104: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 1.21k op/s 2024-09-11T23:55:31.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:31 smithi120 ceph-mon[35182]: pgmap v105: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-09-11T23:55:31.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:31 smithi083 ceph-mon[34805]: pgmap v105: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-09-11T23:55:31.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:31 smithi062 ceph-mon[30958]: pgmap v105: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-09-11T23:55:32.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 170.0 s, 3147.2 tps, lat 31.892 ms stddev 31.748 2024-09-11T23:55:33.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:33 smithi062 ceph-mon[30958]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 874 op/s 2024-09-11T23:55:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:33 smithi120 ceph-mon[35182]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 874 op/s 2024-09-11T23:55:33.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:33 smithi083 ceph-mon[34805]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 874 op/s 2024-09-11T23:55:35.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:35 smithi120 ceph-mon[35182]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.1 MiB/s wr, 681 op/s 2024-09-11T23:55:35.713 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:35 smithi083 ceph-mon[34805]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.1 MiB/s wr, 681 op/s 2024-09-11T23:55:35.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:35 smithi062 ceph-mon[30958]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.1 MiB/s wr, 681 op/s 2024-09-11T23:55:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 175.0 s, 3020.3 tps, lat 32.939 ms stddev 34.626 2024-09-11T23:55:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:37 smithi120 ceph-mon[35182]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.1 MiB/s wr, 596 op/s 2024-09-11T23:55:37.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:37 smithi083 ceph-mon[34805]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.1 MiB/s wr, 596 op/s 2024-09-11T23:55:37.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:37 smithi062 ceph-mon[30958]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.1 MiB/s wr, 596 op/s 2024-09-11T23:55:39.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:39 smithi120 ceph-mon[35182]: pgmap v109: 97 pgs: 97 active+clean; 2.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 886 op/s 2024-09-11T23:55:39.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:39 smithi083 ceph-mon[34805]: pgmap v109: 97 pgs: 97 active+clean; 2.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 886 op/s 2024-09-11T23:55:39.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:39 smithi062 ceph-mon[30958]: pgmap v109: 97 pgs: 97 active+clean; 2.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 886 op/s 2024-09-11T23:55:41.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:41 smithi120 ceph-mon[35182]: pgmap v110: 97 pgs: 97 active+clean; 2.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.0 MiB/s wr, 586 op/s 2024-09-11T23:55:41.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:41 smithi083 ceph-mon[34805]: pgmap v110: 97 pgs: 97 active+clean; 2.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.0 MiB/s wr, 586 op/s 2024-09-11T23:55:41.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:41 smithi062 ceph-mon[30958]: pgmap v110: 97 pgs: 97 active+clean; 2.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.0 MiB/s wr, 586 op/s 2024-09-11T23:55:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 180.0 s, 3236.6 tps, lat 31.085 ms stddev 32.356 2024-09-11T23:55:43.577 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:43 smithi062 ceph-mon[30958]: pgmap v111: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 10 MiB/s wr, 902 op/s 2024-09-11T23:55:43.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:43 smithi120 ceph-mon[35182]: pgmap v111: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 10 MiB/s wr, 902 op/s 2024-09-11T23:55:43.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:43 smithi083 ceph-mon[34805]: pgmap v111: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 10 MiB/s wr, 902 op/s 2024-09-11T23:55:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:45 smithi120 ceph-mon[35182]: pgmap v112: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.4 MiB/s wr, 633 op/s 2024-09-11T23:55:45.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:45 smithi083 ceph-mon[34805]: pgmap v112: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.4 MiB/s wr, 633 op/s 2024-09-11T23:55:45.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:45 smithi062 ceph-mon[30958]: pgmap v112: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.4 MiB/s wr, 633 op/s 2024-09-11T23:55:46.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:55:46.937 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:47.377 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:47 smithi062 ceph-mon[30958]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-09-11T23:55:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 185.0 s, 3341.6 tps, lat 29.825 ms stddev 30.678 2024-09-11T23:55:47.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.501+0000 7fe523916640 1 -- 172.21.15.62:0/3213377374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51c1049a0 msgr2=0x7fe51c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:47.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.501+0000 7fe523916640 1 --2- 172.21.15.62:0/3213377374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51c1049a0 0x7fe51c1089f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe514009ca0 tx=0x7fe51402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.502+0000 7fe523916640 1 -- 172.21.15.62:0/3213377374 shutdown_connections 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.502+0000 7fe523916640 1 --2- 172.21.15.62:0/3213377374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe51c109120 0x7fe51c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.502+0000 7fe523916640 1 --2- 172.21.15.62:0/3213377374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51c1049a0 0x7fe51c1089f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.502+0000 7fe523916640 1 --2- 172.21.15.62:0/3213377374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c103ff0 0x7fe51c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.502+0000 7fe523916640 1 -- 172.21.15.62:0/3213377374 >> 172.21.15.62:0/3213377374 conn(0x7fe51c0ffa70 msgr2=0x7fe51c101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.503+0000 7fe523916640 1 -- 172.21.15.62:0/3213377374 shutdown_connections 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.503+0000 7fe523916640 1 -- 172.21.15.62:0/3213377374 wait complete. 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.504+0000 7fe523916640 1 Processor -- start 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.504+0000 7fe523916640 1 -- start start 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.504+0000 7fe523916640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe51c103ff0 0x7fe51c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.504+0000 7fe523916640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c1049a0 0x7fe51c076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.504+0000 7fe523916640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51c109120 0x7fe51c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.504+0000 7fe523916640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1141c0 con 0x7fe51c109120 2024-09-11T23:55:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.504+0000 7fe523916640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c114040 con 0x7fe51c1049a0 2024-09-11T23:55:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.504+0000 7fe523916640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c114340 con 0x7fe51c103ff0 2024-09-11T23:55:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe5216b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe51c103ff0 0x7fe51c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe5216b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe51c103ff0 0x7fe51c076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:43762/0 (socket says 172.21.15.62:43762) 2024-09-11T23:55:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe5216b2640 1 -- 172.21.15.62:0/1034803344 learned_addr learned my addr 172.21.15.62:0/1034803344 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe521eb3640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51c109120 0x7fe51c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe520eb1640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c1049a0 0x7fe51c076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe521eb3640 1 -- 172.21.15.62:0/1034803344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe51c103ff0 msgr2=0x7fe51c076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe521eb3640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe51c103ff0 0x7fe51c076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe521eb3640 1 -- 172.21.15.62:0/1034803344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c1049a0 msgr2=0x7fe51c076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe521eb3640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c1049a0 0x7fe51c076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe521eb3640 1 -- 172.21.15.62:0/1034803344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c072bd0 con 0x7fe51c109120 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe521eb3640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51c109120 0x7fe51c077180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe51800e970 tx=0x7fe51800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.505+0000 7fe5127fc640 1 -- 172.21.15.62:0/1034803344 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51800ccc0 con 0x7fe51c109120 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.506+0000 7fe5127fc640 1 -- 172.21.15.62:0/1034803344 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe518005020 con 0x7fe51c109120 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.506+0000 7fe523916640 1 -- 172.21.15.62:0/1034803344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c072ec0 con 0x7fe51c109120 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.506+0000 7fe5127fc640 1 -- 172.21.15.62:0/1034803344 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518004a00 con 0x7fe51c109120 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.506+0000 7fe523916640 1 -- 172.21.15.62:0/1034803344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c0733d0 con 0x7fe51c109120 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.510+0000 7fe525cef740 1 Processor -- start 2024-09-11T23:55:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.511+0000 7fe5127fc640 1 -- 172.21.15.62:0/1034803344 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe5180051d0 con 0x7fe51c109120 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.512+0000 7fe525cef740 1 -- start start 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.512+0000 7fe525cef740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac3f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.512+0000 7fe525cef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 0x55a41ac4a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.512+0000 7fe525cef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 0x55a41ac4ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe525cef740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a41aba52e0 con 0x55a41ac3f4a0 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe525cef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a41ab4cce0 con 0x55a41ac3fdc0 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe525cef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a41aba12a0 con 0x55a41ac4a680 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe520eb1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac3f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe520eb1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac3f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38188/0 (socket says 172.21.15.62:38188) 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe520eb1640 1 -- 172.21.15.62:0/2980757394 learned_addr learned my addr 172.21.15.62:0/2980757394 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe520eb1640 1 --2- 172.21.15.62:0/2980757394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 0x55a41ac4ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe5216b2640 1 --2- 172.21.15.62:0/2980757394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 0x55a41ac4a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe520eb1640 1 -- 172.21.15.62:0/2980757394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 msgr2=0x55a41ac4ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe520eb1640 1 --2- 172.21.15.62:0/2980757394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 0x55a41ac4ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe520eb1640 1 -- 172.21.15.62:0/2980757394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 msgr2=0x55a41ac4a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe520eb1640 1 --2- 172.21.15.62:0/2980757394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 0x55a41ac4a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.513+0000 7fe520eb1640 1 -- 172.21.15.62:0/2980757394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a41ab31440 con 0x55a41ac3f4a0 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.514+0000 7fe520eb1640 1 --2- 172.21.15.62:0/2980757394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac3f880 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe51402f540 tx=0x7fe51402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fb73533eb41945 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.514+0000 7fe5127fc640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5080a2d70 0x7fe5080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.514+0000 7fe5077fe640 1 -- 172.21.15.62:0/2980757394 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514031450 con 0x55a41ac3f4a0 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.514+0000 7fe5216b2640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5080a2d70 0x7fe5080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.514+0000 7fe5127fc640 1 -- 172.21.15.62:0/1034803344 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe5180cf300 con 0x7fe51c109120 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.514+0000 7fe5077fe640 1 -- 172.21.15.62:0/2980757394 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe514036050 con 0x55a41ac3f4a0 2024-09-11T23:55:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.514+0000 7fe5077fe640 1 -- 172.21.15.62:0/2980757394 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51402fec0 con 0x55a41ac3f4a0 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.515+0000 7fe525cef740 1 -- 172.21.15.62:0/2980757394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 msgr2=0x55a41ac3f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.515+0000 7fe525cef740 1 --2- 172.21.15.62:0/2980757394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac3f880 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe51402f540 tx=0x7fe51402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.515+0000 7fe5216b2640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5080a2d70 0x7fe5080a5230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55a41ac3c890 tx=0x7fe50c0091c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.517+0000 7fe525cef740 1 -- 172.21.15.62:0/2980757394 shutdown_connections 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.517+0000 7fe525cef740 1 --2- 172.21.15.62:0/2980757394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 0x55a41ac4ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.517+0000 7fe525cef740 1 --2- 172.21.15.62:0/2980757394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 0x55a41ac4a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.517+0000 7fe525cef740 1 --2- 172.21.15.62:0/2980757394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac3f880 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.517+0000 7fe525cef740 1 -- 172.21.15.62:0/2980757394 >> 172.21.15.62:0/2980757394 conn(0x55a41ac3ba30 msgr2=0x55a41ac3be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.517+0000 7fe525cef740 1 -- 172.21.15.62:0/2980757394 shutdown_connections 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.518+0000 7fe525cef740 1 -- 172.21.15.62:0/2980757394 wait complete. 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.518+0000 7fe525cef740 1 -- start start 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.518+0000 7fe525cef740 1 Processor -- start 2024-09-11T23:55:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.519+0000 7fe525cef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3f4a0 0x55a41ac49fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.519+0000 7fe525cef740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3fdc0 0x55a41ac47290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.519+0000 7fe525cef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 0x55a41ac477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.519+0000 7fe525cef740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a41ac4d0d0 con 0x55a41ac3fdc0 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.519+0000 7fe525cef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a41ab60050 con 0x55a41ac3f4a0 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.519+0000 7fe525cef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a41ac4d250 con 0x55a41ac4a680 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.519+0000 7fe525cef740 10 client.0 _ll_register_callbacks cb 0x55a41aba0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.519+0000 7fe521eb3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 0x55a41ac477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.520+0000 7fe521eb3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 0x55a41ac477f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:43780/0 (socket says 172.21.15.62:43780) 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.520+0000 7fe521eb3640 1 -- 172.21.15.62:0/2268121363 learned_addr learned my addr 172.21.15.62:0/2268121363 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.520+0000 7fe5216b2640 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3f4a0 0x55a41ac49fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.520+0000 7fe520eb1640 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3fdc0 0x55a41ac47290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.520+0000 7fe5216b2640 1 -- 172.21.15.62:0/2268121363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 msgr2=0x55a41ac477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.520+0000 7fe5216b2640 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 0x55a41ac477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.520+0000 7fe5216b2640 1 -- 172.21.15.62:0/2268121363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3fdc0 msgr2=0x55a41ac47290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.520+0000 7fe5216b2640 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3fdc0 0x55a41ac47290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.520+0000 7fe5216b2640 1 -- 172.21.15.62:0/2268121363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a41a9163c0 con 0x55a41ac3f4a0 2024-09-11T23:55:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.520+0000 7fe520eb1640 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3fdc0 0x55a41ac47290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.521+0000 7fe5216b2640 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3f4a0 0x55a41ac49fe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe50c014970 tx=0x7fe50c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8cf24cdabb97c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.521+0000 7fe504ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:55:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.521+0000 7fe504ff9640 1 -- 172.21.15.62:0/2268121363 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c025070 con 0x55a41ac3f4a0 2024-09-11T23:55:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.521+0000 7fe525cef740 1 -- 172.21.15.62:0/2268121363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe514036050 con 0x55a41ac3f4a0 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.521+0000 7fe504ff9640 1 -- 172.21.15.62:0/2268121363 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe50c021600 con 0x55a41ac3f4a0 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.521+0000 7fe504ff9640 1 -- 172.21.15.62:0/2268121363 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe50c031050 con 0x55a41ac3f4a0 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.521+0000 7fe525cef740 10 client.25655 fetch_fsmap learned FSMap version 28 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.521+0000 7fe525cef740 1 -- 172.21.15.62:0/2268121363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a41ac58b50 con 0x55a41ac3f4a0 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.523+0000 7fe504ff9640 1 -- 172.21.15.62:0/2268121363 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe514036050 con 0x55a41ac3f4a0 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.523+0000 7fe504ff9640 1 -- 172.21.15.62:0/2268121363 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fe50c020070 con 0x55a41ac3f4a0 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.523+0000 7fe525cef740 10 client.25655 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.523+0000 7fe525cef740 10 client.25655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.523+0000 7fe525cef740 20 client.25655 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.523+0000 7fe525cef740 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a41ac59f60 0x55a41ac5c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.523+0000 7fe525cef740 4 client.25655 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.523+0000 7fe525cef740 1 -- 172.21.15.62:0/2268121363 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a41abb2850 con 0x55a41ac59f60 2024-09-11T23:55:47.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.523+0000 7fe520eb1640 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a41ac59f60 0x55a41ac5c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.524+0000 7fe504ff9640 10 client.25655 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.524+0000 7fe520eb1640 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a41ac59f60 0x55a41ac5c3d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634024228309c786 server_cookie=a94928bd5f017a1d in_seq=0 out_seq=0 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe504ff9640 1 -- 172.21.15.62:0/2268121363 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a41abb2850 con 0x55a41ac59f60 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe504ff9640 10 client.25655 handle_command_reply: tid=0 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe525cef740 1 client.25655 shutdown 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe525cef740 1 -- 172.21.15.62:0/2268121363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3f4a0 msgr2=0x55a41ac49fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe525cef740 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3f4a0 0x55a41ac49fe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe50c014970 tx=0x7fe50c0046c0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe525cef740 1 -- 172.21.15.62:0/2268121363 shutdown_connections 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe525cef740 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a41ac59f60 0x55a41ac5c3d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe525cef740 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac4a680 0x55a41ac477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe525cef740 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3fdc0 0x55a41ac47290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe525cef740 1 --2- 172.21.15.62:0/2268121363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3f4a0 0x55a41ac49fe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.525+0000 7fe525cef740 1 -- 172.21.15.62:0/2268121363 >> 172.21.15.62:0/2268121363 conn(0x55a41ac3ba30 msgr2=0x55a41ac3ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:47.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.526+0000 7fe504ff9640 0 client.25655 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:47.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.526+0000 7fe525cef740 1 -- 172.21.15.62:0/2268121363 shutdown_connections 2024-09-11T23:55:47.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.526+0000 7fe525cef740 1 -- 172.21.15.62:0/2268121363 wait complete. 2024-09-11T23:55:47.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.526+0000 7fe525cef740 20 client.25655 trim_cache size 0 max 16384 2024-09-11T23:55:47.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.552+0000 7fe525cef740 1 Processor -- start 2024-09-11T23:55:47.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe525cef740 1 -- start start 2024-09-11T23:55:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe525cef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3f4a0 0x55a41ac56330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe525cef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 0x55a41ac61ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe525cef740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac4a680 0x55a41ac621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe525cef740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1befb0 con 0x55a41ac4a680 2024-09-11T23:55:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe525cef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1bc880 con 0x55a41ac3fdc0 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe525cef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ba6d0 con 0x55a41ac3f4a0 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe5216b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3f4a0 0x55a41ac56330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe5216b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3f4a0 0x55a41ac56330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:43796/0 (socket says 172.21.15.62:43796) 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe5216b2640 1 -- 172.21.15.62:0/1623158918 learned_addr learned my addr 172.21.15.62:0/1623158918 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe520eb1640 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 0x55a41ac61ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe5216b2640 1 -- 172.21.15.62:0/1623158918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 msgr2=0x55a41ac61ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe5216b2640 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 0x55a41ac61ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe5216b2640 1 -- 172.21.15.62:0/1623158918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac4a680 msgr2=0x55a41ac621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.553+0000 7fe521eb3640 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac4a680 0x55a41ac621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.554+0000 7fe5216b2640 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac4a680 0x55a41ac621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.554+0000 7fe5216b2640 1 -- 172.21.15.62:0/1623158918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a41ac3fbf0 con 0x55a41ac3f4a0 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.554+0000 7fe520eb1640 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 0x55a41ac61ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:55:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.554+0000 7fe5216b2640 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3f4a0 0x55a41ac56330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe50c004810 tx=0x7fe50c011850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc0149d30129a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:47.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.554+0000 7fe521eb3640 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac4a680 0x55a41ac621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:55:47.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.554+0000 7fe506ffd640 1 -- 172.21.15.62:0/1623158918 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c03a070 con 0x55a41ac3f4a0 2024-09-11T23:55:47.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.554+0000 7fe506ffd640 1 -- 172.21.15.62:0/1623158918 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe50c01b070 con 0x55a41ac3f4a0 2024-09-11T23:55:47.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.554+0000 7fe506ffd640 1 -- 172.21.15.62:0/1623158918 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c025070 con 0x55a41ac3f4a0 2024-09-11T23:55:47.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.555+0000 7fe525cef740 1 -- 172.21.15.62:0/1623158918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3f4a0 msgr2=0x55a41ac56330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:47.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.555+0000 7fe525cef740 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3f4a0 0x55a41ac56330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe50c004810 tx=0x7fe50c011850 comp rx=0 tx=0).stop 2024-09-11T23:55:47.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.555+0000 7fe525cef740 1 -- 172.21.15.62:0/1623158918 shutdown_connections 2024-09-11T23:55:47.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.555+0000 7fe525cef740 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac4a680 0x55a41ac621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.555+0000 7fe525cef740 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac3fdc0 0x55a41ac61ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.555+0000 7fe525cef740 1 --2- 172.21.15.62:0/1623158918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3f4a0 0x55a41ac56330 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.555+0000 7fe525cef740 1 -- 172.21.15.62:0/1623158918 >> 172.21.15.62:0/1623158918 conn(0x55a41ac3ba30 msgr2=0x55a41ac42900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.556+0000 7fe525cef740 1 -- 172.21.15.62:0/1623158918 shutdown_connections 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.556+0000 7fe525cef740 1 -- 172.21.15.62:0/1623158918 wait complete. 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.556+0000 7fe525cef740 1 -- start start 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe525cef740 1 Processor -- start 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe525cef740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac52c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe525cef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3fdc0 0x55a41ac4df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe525cef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac4a680 0x55a41ac4e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe525cef740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1bad40 con 0x55a41ac3f4a0 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe525cef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1bc1b0 con 0x55a41ac4a680 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe525cef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1bb410 con 0x55a41ac3fdc0 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe525cef740 10 client.0 _ll_register_callbacks cb 0x7fe50c009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe5216b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac52c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe520eb1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3fdc0 0x55a41ac4df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe520eb1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3fdc0 0x55a41ac4df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:43800/0 (socket says 172.21.15.62:43800) 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe5216b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac52c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38208/0 (socket says 172.21.15.62:38208) 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.557+0000 7fe520eb1640 1 -- 172.21.15.62:0/3346319057 learned_addr learned my addr 172.21.15.62:0/3346319057 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe521eb3640 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac4a680 0x55a41ac4e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe520eb1640 1 -- 172.21.15.62:0/3346319057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac4a680 msgr2=0x55a41ac4e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe520eb1640 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac4a680 0x55a41ac4e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe520eb1640 1 -- 172.21.15.62:0/3346319057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 msgr2=0x55a41ac52c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe520eb1640 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac52c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe520eb1640 1 -- 172.21.15.62:0/3346319057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a41ac4eca0 con 0x55a41ac3fdc0 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe520eb1640 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3fdc0 0x55a41ac4df60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe51402a900 tx=0x7fe51402f3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c162da78fc6268d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe504ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:55:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe504ff9640 1 -- 172.21.15.62:0/3346319057 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514004dc0 con 0x55a41ac3fdc0 2024-09-11T23:55:47.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe525cef740 1 -- 172.21.15.62:0/3346319057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe50c01b070 con 0x55a41ac3fdc0 2024-09-11T23:55:47.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.558+0000 7fe504ff9640 1 -- 172.21.15.62:0/3346319057 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe51402f610 con 0x55a41ac3fdc0 2024-09-11T23:55:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.560+0000 7fe504ff9640 1 -- 172.21.15.62:0/3346319057 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe514009b20 con 0x55a41ac3fdc0 2024-09-11T23:55:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.560+0000 7fe525cef740 10 client.25606 fetch_fsmap learned FSMap version 28 2024-09-11T23:55:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.560+0000 7fe525cef740 1 -- 172.21.15.62:0/3346319057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a41acc4c20 con 0x55a41ac3fdc0 2024-09-11T23:55:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.561+0000 7fe504ff9640 1 -- 172.21.15.62:0/3346319057 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe50c01b070 con 0x55a41ac3fdc0 2024-09-11T23:55:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.562+0000 7fe504ff9640 1 -- 172.21.15.62:0/3346319057 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fe514031540 con 0x55a41ac3fdc0 2024-09-11T23:55:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.562+0000 7fe525cef740 10 client.25606 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:55:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.562+0000 7fe525cef740 10 client.25606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.562+0000 7fe525cef740 20 client.25606 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.562+0000 7fe525cef740 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a41acc53b0 0x55a41acc77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.562+0000 7fe525cef740 4 client.25606 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:55:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.562+0000 7fe525cef740 1 -- 172.21.15.62:0/3346319057 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a41abacd60 con 0x55a41acc53b0 2024-09-11T23:55:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.562+0000 7fe5216b2640 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a41acc53b0 0x55a41acc77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.563+0000 7fe5216b2640 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a41acc53b0 0x55a41acc77d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e1dfa13cbc0c66 server_cookie=d8a1d7c27034546c in_seq=0 out_seq=0 2024-09-11T23:55:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.563+0000 7fe504ff9640 10 client.25606 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.563+0000 7fe504ff9640 1 -- 172.21.15.62:0/3346319057 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a41abacd60 con 0x55a41acc53b0 2024-09-11T23:55:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.563+0000 7fe504ff9640 10 client.25606 handle_command_reply: tid=0 2024-09-11T23:55:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.563+0000 7fe525cef740 1 client.25606 shutdown 2024-09-11T23:55:47.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.564+0000 7fe525cef740 1 -- 172.21.15.62:0/3346319057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3fdc0 msgr2=0x55a41ac4df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:47.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.564+0000 7fe525cef740 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3fdc0 0x55a41ac4df60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe51402a900 tx=0x7fe51402f3f0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.564+0000 7fe525cef740 1 -- 172.21.15.62:0/3346319057 shutdown_connections 2024-09-11T23:55:47.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.564+0000 7fe525cef740 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a41acc53b0 0x55a41acc77d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.564+0000 7fe525cef740 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a41ac4a680 0x55a41ac4e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.564+0000 7fe525cef740 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a41ac3fdc0 0x55a41ac4df60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.564+0000 7fe525cef740 1 --2- 172.21.15.62:0/3346319057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a41ac3f4a0 0x55a41ac52c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.564+0000 7fe525cef740 1 -- 172.21.15.62:0/3346319057 >> 172.21.15.62:0/3346319057 conn(0x55a41ac3ba30 msgr2=0x55a41ac48af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:47.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.564+0000 7fe504ff9640 0 client.25606 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:47.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.564+0000 7fe525cef740 1 -- 172.21.15.62:0/3346319057 shutdown_connections 2024-09-11T23:55:47.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.565+0000 7fe525cef740 1 -- 172.21.15.62:0/3346319057 wait complete. 2024-09-11T23:55:47.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.565+0000 7fe525cef740 20 client.25606 trim_cache size 0 max 16384 2024-09-11T23:55:47.565 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:55:47.565 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-11T23:55:47.565 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-11T23:55:47.565 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:55:47.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.566+0000 7fe523916640 1 -- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5080a2d70 msgr2=0x7fe5080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:47.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.566+0000 7fe523916640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5080a2d70 0x7fe5080a5230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55a41ac3c890 tx=0x7fe50c0091c0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.566+0000 7fe523916640 1 -- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51c109120 msgr2=0x7fe51c077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:47.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.566+0000 7fe523916640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51c109120 0x7fe51c077180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe51800e970 tx=0x7fe51800ee30 comp rx=0 tx=0).stop 2024-09-11T23:55:47.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.566+0000 7fe523916640 1 -- 172.21.15.62:0/1034803344 shutdown_connections 2024-09-11T23:55:47.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.566+0000 7fe523916640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5080a2d70 0x7fe5080a5230 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.566+0000 7fe523916640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe51c109120 0x7fe51c077180 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.566+0000 7fe523916640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c1049a0 0x7fe51c076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.566+0000 7fe523916640 1 --2- 172.21.15.62:0/1034803344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe51c103ff0 0x7fe51c076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:47.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.566+0000 7fe523916640 1 -- 172.21.15.62:0/1034803344 >> 172.21.15.62:0/1034803344 conn(0x7fe51c0ffa70 msgr2=0x7fe51c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:47.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.567+0000 7fe523916640 1 -- 172.21.15.62:0/1034803344 shutdown_connections 2024-09-11T23:55:47.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:47.567+0000 7fe523916640 1 -- 172.21.15.62:0/1034803344 wait complete. 2024-09-11T23:55:47.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:47 smithi120 ceph-mon[35182]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-09-11T23:55:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:47 smithi083 ceph-mon[34805]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-09-11T23:55:48.392 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:55:48.393 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:55:48.808 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.347+0000 7fae12fe6640 1 -- 172.21.15.62:0/2032263254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c101df0 msgr2=0x7fae0c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.347+0000 7fae12fe6640 1 --2- 172.21.15.62:0/2032263254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c101df0 0x7fae0c1021d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae00009900 tx=0x7fae0002a3f0 comp rx=0 tx=0).stop 2024-09-11T23:55:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.347+0000 7fae12fe6640 1 -- 172.21.15.62:0/2032263254 shutdown_connections 2024-09-11T23:55:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.347+0000 7fae12fe6640 1 --2- 172.21.15.62:0/2032263254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c106f20 0x7fae0c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.347+0000 7fae12fe6640 1 --2- 172.21.15.62:0/2032263254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae0c1027a0 0x7fae0c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.347+0000 7fae12fe6640 1 --2- 172.21.15.62:0/2032263254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c101df0 0x7fae0c1021d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.347+0000 7fae12fe6640 1 -- 172.21.15.62:0/2032263254 >> 172.21.15.62:0/2032263254 conn(0x7fae0c0fd8f0 msgr2=0x7fae0c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.348+0000 7fae12fe6640 1 -- 172.21.15.62:0/2032263254 shutdown_connections 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.348+0000 7fae12fe6640 1 -- 172.21.15.62:0/2032263254 wait complete. 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.348+0000 7fae12fe6640 1 Processor -- start 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.348+0000 7fae12fe6640 1 -- start start 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae12fe6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae0c101df0 0x7fae0c075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae12fe6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1027a0 0x7fae0c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae12fe6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c106f20 0x7fae0c06cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae12fe6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c111fd0 con 0x7fae0c101df0 2024-09-11T23:55:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae12fe6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c111e50 con 0x7fae0c106f20 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae12fe6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c112150 con 0x7fae0c1027a0 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae10d82640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae0c101df0 0x7fae0c075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae10d82640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae0c101df0 0x7fae0c075ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38238/0 (socket says 172.21.15.62:38238) 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae10d82640 1 -- 172.21.15.62:0/623123984 learned_addr learned my addr 172.21.15.62:0/623123984 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae0bfff640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1027a0 0x7fae0c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae11583640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c106f20 0x7fae0c06cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae10d82640 1 -- 172.21.15.62:0/623123984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1027a0 msgr2=0x7fae0c076010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae10d82640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1027a0 0x7fae0c076010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae10d82640 1 -- 172.21.15.62:0/623123984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c106f20 msgr2=0x7fae0c06cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae10d82640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c106f20 0x7fae0c06cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae10d82640 1 -- 172.21.15.62:0/623123984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c06d430 con 0x7fae0c101df0 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae10d82640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae0c101df0 0x7fae0c075ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fae00004770 tx=0x7fae00003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.349+0000 7fae09ffb640 1 -- 172.21.15.62:0/623123984 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0003f070 con 0x7fae0c101df0 2024-09-11T23:55:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.350+0000 7fae12fe6640 1 -- 172.21.15.62:0/623123984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c06d6c0 con 0x7fae0c101df0 2024-09-11T23:55:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.350+0000 7fae09ffb640 1 -- 172.21.15.62:0/623123984 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae00031ca0 con 0x7fae0c101df0 2024-09-11T23:55:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.350+0000 7fae09ffb640 1 -- 172.21.15.62:0/623123984 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00035030 con 0x7fae0c101df0 2024-09-11T23:55:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.350+0000 7fae12fe6640 1 -- 172.21.15.62:0/623123984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c0716f0 con 0x7fae0c101df0 2024-09-11T23:55:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.352+0000 7fae12fe6640 1 -- 172.21.15.62:0/623123984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0c1032d0 con 0x7fae0c101df0 2024-09-11T23:55:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.355+0000 7fae09ffb640 1 -- 172.21.15.62:0/623123984 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fae00042070 con 0x7fae0c101df0 2024-09-11T23:55:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.356+0000 7fae09ffb640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadfc0a2df0 0x7fadfc0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.357+0000 7fae0bfff640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadfc0a2df0 0x7fadfc0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:49.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.357+0000 7fae09ffb640 1 -- 172.21.15.62:0/623123984 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fae000ef6e0 con 0x7fae0c101df0 2024-09-11T23:55:49.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.357+0000 7fae0bfff640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadfc0a2df0 0x7fadfc0a52b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae0c06e250 tx=0x7fadf8005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.364+0000 7fae09ffb640 1 -- 172.21.15.62:0/623123984 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae000bbc80 con 0x7fae0c101df0 2024-09-11T23:55:49.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:49 smithi120 ceph-mon[35182]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 13 MiB/s wr, 988 op/s 2024-09-11T23:55:49.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:49 smithi083 ceph-mon[34805]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 13 MiB/s wr, 988 op/s 2024-09-11T23:55:49.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.673+0000 7fae12fe6640 1 -- 172.21.15.62:0/623123984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae0c0721d0 con 0x7fae0c101df0 2024-09-11T23:55:49.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:49 smithi062 ceph-mon[30958]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 13 MiB/s wr, 988 op/s 2024-09-11T23:55:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.674+0000 7fae09ffb640 1 -- 172.21.15.62:0/623123984 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11910 (secure 0 0 0) 0x7fae000bff80 con 0x7fae0c101df0 2024-09-11T23:55:49.673 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:55:49.673 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"btime":"2024-09-11T23:55:21:270579+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:55:20.698541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:55:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 -- 172.21.15.62:0/623123984 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadfc0a2df0 msgr2=0x7fadfc0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadfc0a2df0 0x7fadfc0a52b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae0c06e250 tx=0x7fadf8005f90 comp rx=0 tx=0).stop 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 -- 172.21.15.62:0/623123984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae0c101df0 msgr2=0x7fae0c075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae0c101df0 0x7fae0c075ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fae00004770 tx=0x7fae00003c10 comp rx=0 tx=0).stop 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 -- 172.21.15.62:0/623123984 shutdown_connections 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadfc0a2df0 0x7fadfc0a52b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c106f20 0x7fae0c06cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1027a0 0x7fae0c076010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 --2- 172.21.15.62:0/623123984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae0c101df0 0x7fae0c075ad0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 -- 172.21.15.62:0/623123984 >> 172.21.15.62:0/623123984 conn(0x7fae0c0fd8f0 msgr2=0x7fae0c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 -- 172.21.15.62:0/623123984 shutdown_connections 2024-09-11T23:55:49.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:49.678+0000 7fae12fe6640 1 -- 172.21.15.62:0/623123984 wait complete. 2024-09-11T23:55:50.370 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:55:50.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:50 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/623123984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:55:50.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:50 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/623123984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:55:50.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:50 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/623123984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:55:50.725 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:51.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.261+0000 7fd88198a640 1 -- 172.21.15.62:0/3772027720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd87c102d50 msgr2=0x7fd87c103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.261+0000 7fd88198a640 1 --2- 172.21.15.62:0/3772027720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd87c102d50 0x7fd87c103130 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd870009900 tx=0x7fd87002a480 comp rx=0 tx=0).stop 2024-09-11T23:55:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.261+0000 7fd88198a640 1 -- 172.21.15.62:0/3772027720 shutdown_connections 2024-09-11T23:55:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.261+0000 7fd88198a640 1 --2- 172.21.15.62:0/3772027720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c10c180 0x7fd87c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.261+0000 7fd88198a640 1 --2- 172.21.15.62:0/3772027720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd87c103670 0x7fd87c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.261+0000 7fd88198a640 1 --2- 172.21.15.62:0/3772027720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd87c102d50 0x7fd87c103130 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.261+0000 7fd88198a640 1 -- 172.21.15.62:0/3772027720 >> 172.21.15.62:0/3772027720 conn(0x7fd87c0fc7d0 msgr2=0x7fd87c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.261+0000 7fd88198a640 1 -- 172.21.15.62:0/3772027720 shutdown_connections 2024-09-11T23:55:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.261+0000 7fd88198a640 1 -- 172.21.15.62:0/3772027720 wait complete. 2024-09-11T23:55:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.262+0000 7fd88198a640 1 Processor -- start 2024-09-11T23:55:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.262+0000 7fd88198a640 1 -- start start 2024-09-11T23:55:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd88198a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd87c102d50 0x7fd87c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd88198a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c103670 0x7fd87c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd87affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd87c102d50 0x7fd87c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd88198a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd87c10c180 0x7fd87c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd88198a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c111f80 con 0x7fd87c102d50 2024-09-11T23:55:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd88198a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c111e00 con 0x7fd87c103670 2024-09-11T23:55:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd88198a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c112100 con 0x7fd87c10c180 2024-09-11T23:55:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd87affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd87c102d50 0x7fd87c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48428/0 (socket says 172.21.15.62:48428) 2024-09-11T23:55:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd87affd640 1 -- 172.21.15.62:0/2356117491 learned_addr learned my addr 172.21.15.62:0/2356117491 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c103670 0x7fd87c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd87a7fc640 1 -- 172.21.15.62:0/2356117491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd87c10c180 msgr2=0x7fd87c19ef00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:55:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd87c10c180 0x7fd87c19ef00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd87a7fc640 1 -- 172.21.15.62:0/2356117491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd87c102d50 msgr2=0x7fd87c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd87c102d50 0x7fd87c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd87a7fc640 1 -- 172.21.15.62:0/2356117491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c19f5e0 con 0x7fd87c103670 2024-09-11T23:55:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.263+0000 7fd87affd640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd87c102d50 0x7fd87c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.264+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c103670 0x7fd87c19ab70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd87400ea50 tx=0x7fd87400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.264+0000 7fd880988640 1 -- 172.21.15.62:0/2356117491 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87400ce20 con 0x7fd87c103670 2024-09-11T23:55:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.264+0000 7fd880988640 1 -- 172.21.15.62:0/2356117491 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd874010910 con 0x7fd87c103670 2024-09-11T23:55:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.264+0000 7fd88198a640 1 -- 172.21.15.62:0/2356117491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c19f8d0 con 0x7fd87c103670 2024-09-11T23:55:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.264+0000 7fd880988640 1 -- 172.21.15.62:0/2356117491 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874017ac0 con 0x7fd87c103670 2024-09-11T23:55:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.264+0000 7fd88198a640 1 -- 172.21.15.62:0/2356117491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c1a71b0 con 0x7fd87c103670 2024-09-11T23:55:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.268+0000 7fd880988640 1 -- 172.21.15.62:0/2356117491 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd874010ac0 con 0x7fd87c103670 2024-09-11T23:55:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.269+0000 7fd880988640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd8640a2d70 0x7fd8640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.269+0000 7fd880988640 1 -- 172.21.15.62:0/2356117491 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fd8740cfcf0 con 0x7fd87c103670 2024-09-11T23:55:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.269+0000 7fd87affd640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd8640a2d70 0x7fd8640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.269+0000 7fd883d63740 1 Processor -- start 2024-09-11T23:55:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.269+0000 7fd87affd640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd8640a2d70 0x7fd8640a5230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd870009230 tx=0x7fd870009560 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd883d63740 1 -- start start 2024-09-11T23:55:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd883d63740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036a0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd883d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd883d63740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd883d63740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556d036062f0 con 0x556d036a1b40 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd883d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556d035adcf0 con 0x556d036a05d0 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd87affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd87affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48444/0 (socket says 172.21.15.62:48444) 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd87affd640 1 -- 172.21.15.62:0/1040801818 learned_addr learned my addr 172.21.15.62:0/1040801818 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/1040801818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd883d63740 1 -- 172.21.15.62:0/1040801818 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556d036022b0 con 0x556d036ab370 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd87b7fe640 1 --2- 172.21.15.62:0/1040801818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd87affd640 1 -- 172.21.15.62:0/1040801818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 msgr2=0x556d036ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd87affd640 1 --2- 172.21.15.62:0/1040801818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd87affd640 1 -- 172.21.15.62:0/1040801818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 msgr2=0x556d036a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd87affd640 1 --2- 172.21.15.62:0/1040801818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.270+0000 7fd87affd640 1 -- 172.21.15.62:0/1040801818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d035923a0 con 0x556d036a1b40 2024-09-11T23:55:51.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.271+0000 7fd87affd640 1 --2- 172.21.15.62:0/1040801818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036a0090 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd870031820 tx=0x7fd87003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea520a3cb6f7ad7a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:51.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.271+0000 7fd861ffb640 1 -- 172.21.15.62:0/1040801818 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870035050 con 0x556d036a1b40 2024-09-11T23:55:51.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.271+0000 7fd861ffb640 1 -- 172.21.15.62:0/1040801818 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87004c070 con 0x556d036a1b40 2024-09-11T23:55:51.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.271+0000 7fd861ffb640 1 -- 172.21.15.62:0/1040801818 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870004c70 con 0x556d036a1b40 2024-09-11T23:55:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.272+0000 7fd883d63740 1 -- 172.21.15.62:0/1040801818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 msgr2=0x556d036a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.272+0000 7fd883d63740 1 --2- 172.21.15.62:0/1040801818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036a0090 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd870031820 tx=0x7fd87003c000 comp rx=0 tx=0).stop 2024-09-11T23:55:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.272+0000 7fd883d63740 1 -- 172.21.15.62:0/1040801818 shutdown_connections 2024-09-11T23:55:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.272+0000 7fd883d63740 1 --2- 172.21.15.62:0/1040801818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.272+0000 7fd883d63740 1 --2- 172.21.15.62:0/1040801818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.272+0000 7fd883d63740 1 --2- 172.21.15.62:0/1040801818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036a0090 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.272+0000 7fd883d63740 1 -- 172.21.15.62:0/1040801818 >> 172.21.15.62:0/1040801818 conn(0x556d03669310 msgr2=0x556d0369cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.272+0000 7fd883d63740 1 -- 172.21.15.62:0/1040801818 shutdown_connections 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.272+0000 7fd883d63740 1 -- 172.21.15.62:0/1040801818 wait complete. 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.273+0000 7fd883d63740 1 -- start start 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.273+0000 7fd883d63740 1 Processor -- start 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.273+0000 7fd883d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.273+0000 7fd883d63740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.273+0000 7fd883d63740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036a85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd883d63740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556d036ae010 con 0x556d036a1b40 2024-09-11T23:55:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd883d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556d035c1060 con 0x556d036a05d0 2024-09-11T23:55:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:43958/0 (socket says 172.21.15.62:43958) 2024-09-11T23:55:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87affd640 1 -- 172.21.15.62:0/3263865245 learned_addr learned my addr 172.21.15.62:0/3263865245 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd883d63740 1 -- 172.21.15.62:0/3263865245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556d036ae190 con 0x556d036ab370 2024-09-11T23:55:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87b7fe640 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036a85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd883d63740 10 client.0 _ll_register_callbacks cb 0x556d033148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87affd640 1 -- 172.21.15.62:0/3263865245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 msgr2=0x556d036a85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87affd640 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036a85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87affd640 1 -- 172.21.15.62:0/3263865245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 msgr2=0x556d036a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87affd640 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87affd640 1 -- 172.21.15.62:0/3263865245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d033773c0 con 0x556d036a05d0 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87b7fe640 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036a85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd87affd640 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036aad00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd870009430 tx=0x7fd870003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17dee771f6de9717 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.274+0000 7fd85affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd85affd640 1 -- 172.21.15.62:0/3263865245 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870035030 con 0x556d036a05d0 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd85affd640 1 -- 172.21.15.62:0/3263865245 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fd87005a670 con 0x556d036a05d0 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd883d63740 1 -- 172.21.15.62:0/3263865245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd87004c070 con 0x556d036a05d0 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd85affd640 1 -- 172.21.15.62:0/3263865245 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd87005a8b0 con 0x556d036a05d0 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd883d63740 10 client.25676 fetch_fsmap learned FSMap version 28 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd883d63740 1 -- 172.21.15.62:0/3263865245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556d036c2250 con 0x556d036a05d0 2024-09-11T23:55:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd85affd640 1 -- 172.21.15.62:0/3263865245 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87004c070 con 0x556d036a05d0 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd85affd640 1 -- 172.21.15.62:0/3263865245 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fd870047870 con 0x556d036a05d0 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd883d63740 10 client.25676 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd883d63740 10 client.25676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd883d63740 20 client.25676 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd883d63740 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x556d036c37a0 0x556d036c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd883d63740 4 client.25676 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.275+0000 7fd883d63740 1 -- 172.21.15.62:0/3263865245 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d03613860 con 0x556d036c37a0 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.276+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x556d036c37a0 0x556d036c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.276+0000 7fd85affd640 10 client.25676 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.276+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x556d036c37a0 0x556d036c5bd0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a31447c0849fed5 server_cookie=3479b1e5f136d79c in_seq=0 out_seq=0 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.276+0000 7fd85affd640 1 -- 172.21.15.62:0/3263865245 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d03613860 con 0x556d036c37a0 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.276+0000 7fd85affd640 10 client.25676 handle_command_reply: tid=0 2024-09-11T23:55:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 client.25676 shutdown 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 -- 172.21.15.62:0/3263865245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 msgr2=0x556d036aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036aad00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd870009430 tx=0x7fd870003e80 comp rx=0 tx=0).stop 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 -- 172.21.15.62:0/3263865245 shutdown_connections 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x556d036c37a0 0x556d036c5bd0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036a85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 --2- 172.21.15.62:0/3263865245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036aad00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 -- 172.21.15.62:0/3263865245 >> 172.21.15.62:0/3263865245 conn(0x556d03669310 msgr2=0x556d036a2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd85affd640 0 client.25676 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 -- 172.21.15.62:0/3263865245 shutdown_connections 2024-09-11T23:55:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 1 -- 172.21.15.62:0/3263865245 wait complete. 2024-09-11T23:55:51.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.277+0000 7fd883d63740 20 client.25676 trim_cache size 0 max 16384 2024-09-11T23:55:51.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.296+0000 7fd883d63740 1 Processor -- start 2024-09-11T23:55:51.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd883d63740 1 -- start start 2024-09-11T23:55:51.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd883d63740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a05d0 0x556d0369bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd883d63740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036a1b40 0x556d0369c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd883d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036ab370 0x556d036a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd883d63740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1adf30 con 0x556d036a05d0 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd883d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1ab800 con 0x556d036ab370 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd883d63740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1a9690 con 0x556d036a1b40 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036a1b40 0x556d0369c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036ab370 0x556d036a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036ab370 0x556d036a3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:43966/0 (socket says 172.21.15.62:43966) 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a05d0 0x556d0369bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036a1b40 0x556d0369c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52766/0 (socket says 172.21.15.62:52766) 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87b7fe640 1 -- 172.21.15.62:0/1632428608 learned_addr learned my addr 172.21.15.62:0/1632428608 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87affd640 1 -- 172.21.15.62:0/1632428608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036a1b40 msgr2=0x556d0369c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87affd640 1 --2- 172.21.15.62:0/1632428608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036a1b40 0x556d0369c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87affd640 1 -- 172.21.15.62:0/1632428608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036ab370 msgr2=0x556d036a3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87affd640 1 --2- 172.21.15.62:0/1632428608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036ab370 0x556d036a3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.297+0000 7fd87affd640 1 -- 172.21.15.62:0/1632428608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d036a6550 con 0x556d036a05d0 2024-09-11T23:55:51.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.298+0000 7fd87affd640 1 --2- 172.21.15.62:0/1632428608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a05d0 0x556d0369bc10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd87003e040 tx=0x7fd8700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed338fd3f3b02b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:51.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.298+0000 7fd8617fa640 1 -- 172.21.15.62:0/1632428608 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870067070 con 0x556d036a05d0 2024-09-11T23:55:51.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.298+0000 7fd8617fa640 1 -- 172.21.15.62:0/1632428608 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87004c040 con 0x556d036a05d0 2024-09-11T23:55:51.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.298+0000 7fd8617fa640 1 -- 172.21.15.62:0/1632428608 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870035070 con 0x556d036a05d0 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.299+0000 7fd883d63740 1 -- 172.21.15.62:0/1632428608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a05d0 msgr2=0x556d0369bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.299+0000 7fd883d63740 1 --2- 172.21.15.62:0/1632428608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a05d0 0x556d0369bc10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd87003e040 tx=0x7fd8700046c0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.299+0000 7fd883d63740 1 -- 172.21.15.62:0/1632428608 shutdown_connections 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.299+0000 7fd883d63740 1 --2- 172.21.15.62:0/1632428608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036ab370 0x556d036a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.299+0000 7fd883d63740 1 --2- 172.21.15.62:0/1632428608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036a1b40 0x556d0369c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.299+0000 7fd883d63740 1 --2- 172.21.15.62:0/1632428608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a05d0 0x556d0369bc10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.299+0000 7fd883d63740 1 -- 172.21.15.62:0/1632428608 >> 172.21.15.62:0/1632428608 conn(0x556d03669310 msgr2=0x556d0369b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.299+0000 7fd883d63740 1 -- 172.21.15.62:0/1632428608 shutdown_connections 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.299+0000 7fd883d63740 1 -- 172.21.15.62:0/1632428608 wait complete. 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.300+0000 7fd883d63740 1 -- start start 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.301+0000 7fd883d63740 1 Processor -- start 2024-09-11T23:55:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.301+0000 7fd883d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.301+0000 7fd883d63740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.301+0000 7fd883d63740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.301+0000 7fd883d63740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1a9ce0 con 0x556d036a1b40 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.301+0000 7fd883d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1ab130 con 0x556d036a05d0 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.301+0000 7fd883d63740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1aa390 con 0x556d036ab370 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.301+0000 7fd883d63740 10 client.0 _ll_register_callbacks cb 0x7fd874005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:43980/0 (socket says 172.21.15.62:43980) 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87affd640 1 -- 172.21.15.62:0/3631863973 learned_addr learned my addr 172.21.15.62:0/3631863973 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87b7fe640 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87affd640 1 -- 172.21.15.62:0/3631863973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 msgr2=0x556d036c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87affd640 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87affd640 1 -- 172.21.15.62:0/3631863973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 msgr2=0x556d036c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87affd640 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87affd640 1 -- 172.21.15.62:0/3631863973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d0372c250 con 0x556d036a05d0 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd87affd640 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036cbab0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd870005970 tx=0x7fd870003fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d613cb0402c497f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd85affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd85affd640 1 -- 172.21.15.62:0/3631863973 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870067070 con 0x556d036a05d0 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd85affd640 1 -- 172.21.15.62:0/3631863973 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fd870047670 con 0x556d036a05d0 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd883d63740 1 -- 172.21.15.62:0/3631863973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd87004c040 con 0x556d036a05d0 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd85affd640 1 -- 172.21.15.62:0/3631863973 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd870035040 con 0x556d036a05d0 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.302+0000 7fd883d63740 10 client.25682 fetch_fsmap learned FSMap version 28 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.303+0000 7fd883d63740 1 -- 172.21.15.62:0/3631863973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556d0372d390 con 0x556d036a05d0 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.303+0000 7fd85affd640 1 -- 172.21.15.62:0/3631863973 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87004c040 con 0x556d036a05d0 2024-09-11T23:55:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.303+0000 7fd85affd640 1 -- 172.21.15.62:0/3631863973 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fd870063400 con 0x556d036a05d0 2024-09-11T23:55:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.304+0000 7fd883d63740 10 client.25682 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:55:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.304+0000 7fd883d63740 10 client.25682 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:55:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.304+0000 7fd883d63740 20 client.25682 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.304+0000 7fd883d63740 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x556d0372d770 0x556d0372fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.304+0000 7fd883d63740 4 client.25682 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:55:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.304+0000 7fd883d63740 1 -- 172.21.15.62:0/3631863973 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d0360dd70 con 0x556d0372d770 2024-09-11T23:55:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.304+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x556d0372d770 0x556d0372fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.304+0000 7fd87a7fc640 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x556d0372d770 0x556d0372fb60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d865b49498f6cdf server_cookie=f276930dc38a76e4 in_seq=0 out_seq=0 2024-09-11T23:55:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.304+0000 7fd85affd640 10 client.25682 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.305+0000 7fd85affd640 1 -- 172.21.15.62:0/3631863973 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d0360dd70 con 0x556d0372d770 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.305+0000 7fd85affd640 10 client.25682 handle_command_reply: tid=0 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.305+0000 7fd883d63740 1 client.25682 shutdown 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.305+0000 7fd883d63740 1 -- 172.21.15.62:0/3631863973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 msgr2=0x556d036cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.305+0000 7fd883d63740 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036cbab0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd870005970 tx=0x7fd870003fb0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.305+0000 7fd883d63740 1 -- 172.21.15.62:0/3631863973 shutdown_connections 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.305+0000 7fd883d63740 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x556d0372d770 0x556d0372fb60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.306+0000 7fd883d63740 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d036ab370 0x556d036c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.306+0000 7fd85affd640 0 client.25682 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.306+0000 7fd883d63740 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d036a1b40 0x556d036c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.306+0000 7fd883d63740 1 --2- 172.21.15.62:0/3631863973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d036a05d0 0x556d036cbab0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.306+0000 7fd883d63740 1 -- 172.21.15.62:0/3631863973 >> 172.21.15.62:0/3631863973 conn(0x556d03669310 msgr2=0x556d036a7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:51.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.306+0000 7fd883d63740 1 -- 172.21.15.62:0/3631863973 shutdown_connections 2024-09-11T23:55:51.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.306+0000 7fd883d63740 1 -- 172.21.15.62:0/3631863973 wait complete. 2024-09-11T23:55:51.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.306+0000 7fd883d63740 20 client.25682 trim_cache size 0 max 16384 2024-09-11T23:55:51.309 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:55:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 -- 172.21.15.62:0/2356117491 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd8640a2d70 msgr2=0x7fd8640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd8640a2d70 0x7fd8640a5230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd870009230 tx=0x7fd870009560 comp rx=0 tx=0).stop 2024-09-11T23:55:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 -- 172.21.15.62:0/2356117491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c103670 msgr2=0x7fd87c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c103670 0x7fd87c19ab70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd87400ea50 tx=0x7fd87400ef10 comp rx=0 tx=0).stop 2024-09-11T23:55:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 -- 172.21.15.62:0/2356117491 shutdown_connections 2024-09-11T23:55:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd8640a2d70 0x7fd8640a5230 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd87c10c180 0x7fd87c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c103670 0x7fd87c19ab70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 --2- 172.21.15.62:0/2356117491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd87c102d50 0x7fd87c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:51.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 -- 172.21.15.62:0/2356117491 >> 172.21.15.62:0/2356117491 conn(0x7fd87c0fc7d0 msgr2=0x7fd87c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:51.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 -- 172.21.15.62:0/2356117491 shutdown_connections 2024-09-11T23:55:51.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:51.307+0000 7fd88198a640 1 -- 172.21.15.62:0/2356117491 wait complete. 2024-09-11T23:55:51.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:51 smithi062 ceph-mon[30958]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.2 MiB/s wr, 698 op/s 2024-09-11T23:55:51.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:51 smithi120 ceph-mon[35182]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.2 MiB/s wr, 698 op/s 2024-09-11T23:55:51.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:51 smithi083 ceph-mon[34805]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.2 MiB/s wr, 698 op/s 2024-09-11T23:55:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 190.0 s, 2952.8 tps, lat 33.875 ms stddev 35.256 2024-09-11T23:55:52.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:52 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:55:52.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:52 smithi120 ceph-mon[35182]: pgmap v116: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 999 op/s 2024-09-11T23:55:52.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:52 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:55:52.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:52 smithi083 ceph-mon[34805]: pgmap v116: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 999 op/s 2024-09-11T23:55:52.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:52 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:55:52.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:52 smithi062 ceph-mon[30958]: pgmap v116: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 999 op/s 2024-09-11T23:55:52.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:55:53.135 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:53.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.657+0000 7f693d094640 1 -- 172.21.15.62:0/1713756234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69381027a0 msgr2=0x7f69381067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.657+0000 7f693d094640 1 --2- 172.21.15.62:0/1713756234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69381027a0 0x7f69381067f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f692c009860 tx=0x7f692c02e770 comp rx=0 tx=0).stop 2024-09-11T23:55:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.657+0000 7f693d094640 1 -- 172.21.15.62:0/1713756234 shutdown_connections 2024-09-11T23:55:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.657+0000 7f693d094640 1 --2- 172.21.15.62:0/1713756234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6938106f20 0x7f693810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.657+0000 7f693d094640 1 --2- 172.21.15.62:0/1713756234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69381027a0 0x7f69381067f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.657+0000 7f693d094640 1 --2- 172.21.15.62:0/1713756234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6938101df0 0x7f69381021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.657+0000 7f693d094640 1 -- 172.21.15.62:0/1713756234 >> 172.21.15.62:0/1713756234 conn(0x7f69380fd8f0 msgr2=0x7f69380ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.657+0000 7f693d094640 1 -- 172.21.15.62:0/1713756234 shutdown_connections 2024-09-11T23:55:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.657+0000 7f693d094640 1 -- 172.21.15.62:0/1713756234 wait complete. 2024-09-11T23:55:53.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.658+0000 7f693d094640 1 Processor -- start 2024-09-11T23:55:53.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.658+0000 7f693d094640 1 -- start start 2024-09-11T23:55:53.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f693d094640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6938101df0 0x7f69381962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f693d094640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69381027a0 0x7f6938196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f693d094640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6938106f20 0x7f693819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f693d094640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f693810fdb0 con 0x7f6938106f20 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f693659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69381027a0 0x7f6938196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f693659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69381027a0 0x7f6938196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:43998/0 (socket says 172.21.15.62:43998) 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f693659c640 1 -- 172.21.15.62:0/1716405385 learned_addr learned my addr 172.21.15.62:0/1716405385 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f693d094640 1 -- 172.21.15.62:0/1716405385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f693810fc30 con 0x7f69381027a0 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f693d094640 1 -- 172.21.15.62:0/1716405385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f693810ff30 con 0x7f6938101df0 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f6936d9d640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6938101df0 0x7f69381962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.659+0000 7f693759e640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6938106f20 0x7f693819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.660+0000 7f693659c640 1 -- 172.21.15.62:0/1716405385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6938101df0 msgr2=0x7f69381962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.660+0000 7f693659c640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6938101df0 0x7f69381962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.660+0000 7f693659c640 1 -- 172.21.15.62:0/1716405385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6938106f20 msgr2=0x7f693819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.660+0000 7f693659c640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6938106f20 0x7f693819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.660+0000 7f693659c640 1 -- 172.21.15.62:0/1716405385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693819b290 con 0x7f69381027a0 2024-09-11T23:55:53.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.660+0000 7f693659c640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69381027a0 0x7f6938196820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f692c035000 tx=0x7f692c0043a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:53.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.660+0000 7f691ffff640 1 -- 172.21.15.62:0/1716405385 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c045020 con 0x7f69381027a0 2024-09-11T23:55:53.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.660+0000 7f691ffff640 1 -- 172.21.15.62:0/1716405385 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c0358e0 con 0x7f69381027a0 2024-09-11T23:55:53.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.660+0000 7f691ffff640 1 -- 172.21.15.62:0/1716405385 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c03a040 con 0x7f69381027a0 2024-09-11T23:55:53.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.660+0000 7f693d094640 1 -- 172.21.15.62:0/1716405385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693819b520 con 0x7f69381027a0 2024-09-11T23:55:53.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.661+0000 7f693d094640 1 -- 172.21.15.62:0/1716405385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693819b8f0 con 0x7f69381027a0 2024-09-11T23:55:53.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.663+0000 7f691ffff640 1 -- 172.21.15.62:0/1716405385 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f692c048020 con 0x7f69381027a0 2024-09-11T23:55:53.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.664+0000 7f693f46d740 1 Processor -- start 2024-09-11T23:55:53.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.664+0000 7f691ffff640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69200a2e40 0x7f69200a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f693f46d740 1 -- start start 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f691ffff640 1 -- 172.21.15.62:0/1716405385 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f692c0f3d60 con 0x7f69381027a0 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f693f46d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fd28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f693f46d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fd3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f693f46d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 0x55cb22fdff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f693f46d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cb22f302f0 con 0x55cb22fd4350 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f693f46d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cb22ed7cf0 con 0x55cb22fd2de0 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f693f46d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cb22f2c2b0 con 0x55cb22fddb80 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f693659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fd3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f693659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fd3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:44014/0 (socket says 172.21.15.62:44014) 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f693659c640 1 -- 172.21.15.62:0/833417204 learned_addr learned my addr 172.21.15.62:0/833417204 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.665+0000 7f6936d9d640 1 --2- 172.21.15.62:0/833417204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fd28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f693759e640 1 --2- 172.21.15.62:0/833417204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 0x55cb22fdff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f693659c640 1 -- 172.21.15.62:0/833417204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 msgr2=0x55cb22fdff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f693659c640 1 --2- 172.21.15.62:0/833417204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 0x55cb22fdff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f693659c640 1 -- 172.21.15.62:0/833417204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 msgr2=0x55cb22fd28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f693659c640 1 --2- 172.21.15.62:0/833417204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fd28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f693659c640 1 -- 172.21.15.62:0/833417204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb22ebc3a0 con 0x55cb22fd2de0 2024-09-11T23:55:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f693759e640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69200a2e40 0x7f69200a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f693659c640 1 --2- 172.21.15.62:0/833417204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fd3220 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f692c009980 tx=0x7f692c0c0ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395c1342327956dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f691d7fa640 1 -- 172.21.15.62:0/833417204 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c003d50 con 0x55cb22fd2de0 2024-09-11T23:55:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f691d7fa640 1 -- 172.21.15.62:0/833417204 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c0f1700 con 0x55cb22fd2de0 2024-09-11T23:55:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f691d7fa640 1 -- 172.21.15.62:0/833417204 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0f1bd0 con 0x55cb22fd2de0 2024-09-11T23:55:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.666+0000 7f693759e640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69200a2e40 0x7f69200a5300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55cb22fd3880 tx=0x7f6930004ab0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:53.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.667+0000 7f693f46d740 1 -- 172.21.15.62:0/833417204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 msgr2=0x55cb22fd3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:53.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.667+0000 7f693f46d740 1 --2- 172.21.15.62:0/833417204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fd3220 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f692c009980 tx=0x7f692c0c0ca0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.667+0000 7f693f46d740 1 -- 172.21.15.62:0/833417204 shutdown_connections 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.667+0000 7f693f46d740 1 --2- 172.21.15.62:0/833417204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 0x55cb22fdff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.667+0000 7f693f46d740 1 --2- 172.21.15.62:0/833417204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fd3220 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.667+0000 7f693f46d740 1 --2- 172.21.15.62:0/833417204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fd28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.667+0000 7f693f46d740 1 -- 172.21.15.62:0/833417204 >> 172.21.15.62:0/833417204 conn(0x55cb22f93310 msgr2=0x55cb22fcf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.667+0000 7f693f46d740 1 -- 172.21.15.62:0/833417204 shutdown_connections 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.668+0000 7f693f46d740 1 -- 172.21.15.62:0/833417204 wait complete. 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.668+0000 7f693f46d740 1 -- start start 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.668+0000 7f693f46d740 1 Processor -- start 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.668+0000 7f693f46d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd2de0 0x55cb22fdd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.668+0000 7f693f46d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fda810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.668+0000 7f693f46d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 0x55cb22fdade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.668+0000 7f693f46d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cb22ff1650 con 0x55cb22fd4350 2024-09-11T23:55:53.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.668+0000 7f693659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fda810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.668+0000 7f693659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fda810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48500/0 (socket says 172.21.15.62:48500) 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.668+0000 7f693659c640 1 -- 172.21.15.62:0/945092729 learned_addr learned my addr 172.21.15.62:0/945092729 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f6936d9d640 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd2de0 0x55cb22fdd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f693f46d740 1 -- 172.21.15.62:0/945092729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cb22eeb060 con 0x55cb22fddb80 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f693f46d740 1 -- 172.21.15.62:0/945092729 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cb22ff17d0 con 0x55cb22fd2de0 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f693759e640 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 0x55cb22fdade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f693f46d740 10 client.0 _ll_register_callbacks cb 0x55cb22c3e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f6936d9d640 1 -- 172.21.15.62:0/945092729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 msgr2=0x55cb22fdade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f6936d9d640 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 0x55cb22fdade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f6936d9d640 1 -- 172.21.15.62:0/945092729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 msgr2=0x55cb22fda810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f6936d9d640 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fda810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f6936d9d640 1 -- 172.21.15.62:0/945092729 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb22ca13c0 con 0x55cb22fd2de0 2024-09-11T23:55:53.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f693659c640 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fda810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f6936d9d640 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd2de0 0x55cb22fdd560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6928009e30 tx=0x7f6928011630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e1adfcf7451e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.669+0000 7f691aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.670+0000 7f691aff5640 1 -- 172.21.15.62:0/945092729 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928019070 con 0x55cb22fd2de0 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.670+0000 7f691aff5640 1 -- 172.21.15.62:0/945092729 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6928015660 con 0x55cb22fd2de0 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.670+0000 7f693f46d740 1 -- 172.21.15.62:0/945092729 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f692c0f1700 con 0x55cb22fd2de0 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f691aff5640 1 -- 172.21.15.62:0/945092729 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6928023050 con 0x55cb22fd2de0 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f693f46d740 10 client.25636 fetch_fsmap learned FSMap version 29 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f693f46d740 1 -- 172.21.15.62:0/945092729 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb22ffd0b0 con 0x55cb22fd2de0 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f691aff5640 1 -- 172.21.15.62:0/945092729 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6928015d80 con 0x55cb22fd2de0 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f691aff5640 1 -- 172.21.15.62:0/945092729 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6928014cc0 con 0x55cb22fd2de0 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f693f46d740 10 client.25636 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f693f46d740 10 client.25636 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:55:53.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f693f46d740 20 client.25636 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:53.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f693f46d740 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cb22ffe570 0x55cb230009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f693f46d740 4 client.25636 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:53.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.671+0000 7f693f46d740 1 -- 172.21.15.62:0/945092729 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f692c0f1700 con 0x55cb22ffe570 2024-09-11T23:55:53.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.672+0000 7f693659c640 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cb22ffe570 0x55cb230009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.673+0000 7f693659c640 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cb22ffe570 0x55cb230009c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e59f31bcd67fae1 server_cookie=3492acc0c0cd6c04 in_seq=0 out_seq=0 2024-09-11T23:55:53.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.673+0000 7f691aff5640 10 client.25636 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:55:53.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.674+0000 7f691aff5640 1 -- 172.21.15.62:0/945092729 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f692c0cfe80 con 0x55cb22ffe570 2024-09-11T23:55:53.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.674+0000 7f691aff5640 10 client.25636 handle_command_reply: tid=0 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.674+0000 7f693f46d740 1 client.25636 shutdown 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.675+0000 7f693f46d740 1 -- 172.21.15.62:0/945092729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd2de0 msgr2=0x55cb22fdd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.675+0000 7f693f46d740 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd2de0 0x55cb22fdd560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6928009e30 tx=0x7f6928011630 comp rx=0 tx=0).stop 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.675+0000 7f693f46d740 1 -- 172.21.15.62:0/945092729 shutdown_connections 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.675+0000 7f693f46d740 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cb22ffe570 0x55cb230009c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.675+0000 7f693f46d740 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 0x55cb22fdade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.675+0000 7f693f46d740 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fda810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.675+0000 7f691aff5640 0 client.25636 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.675+0000 7f693f46d740 1 --2- 172.21.15.62:0/945092729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd2de0 0x55cb22fdd560 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.675+0000 7f693f46d740 1 -- 172.21.15.62:0/945092729 >> 172.21.15.62:0/945092729 conn(0x55cb22f93310 msgr2=0x55cb22fd51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.676+0000 7f693f46d740 1 -- 172.21.15.62:0/945092729 shutdown_connections 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.676+0000 7f693f46d740 1 -- 172.21.15.62:0/945092729 wait complete. 2024-09-11T23:55:53.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.676+0000 7f693f46d740 20 client.25636 trim_cache size 0 max 16384 2024-09-11T23:55:53.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693f46d740 1 Processor -- start 2024-09-11T23:55:53.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693f46d740 1 -- start start 2024-09-11T23:55:53.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693f46d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fc5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693f46d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fd5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693f46d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 0x55cb22fd5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fd5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693f46d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6924011050 con 0x55cb22fd4350 2024-09-11T23:55:53.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693f46d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f692400e920 con 0x55cb22fd2de0 2024-09-11T23:55:53.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f6936d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fc5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693f46d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f692400c770 con 0x55cb22fddb80 2024-09-11T23:55:53.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fd5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48506/0 (socket says 172.21.15.62:48506) 2024-09-11T23:55:53.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.695+0000 7f693659c640 1 -- 172.21.15.62:0/905709202 learned_addr learned my addr 172.21.15.62:0/905709202 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:53.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f693759e640 1 --2- 172.21.15.62:0/905709202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 0x55cb22fd5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f693659c640 1 -- 172.21.15.62:0/905709202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 msgr2=0x55cb22fd5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:53.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f693659c640 1 --2- 172.21.15.62:0/905709202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 0x55cb22fd5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f693659c640 1 -- 172.21.15.62:0/905709202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 msgr2=0x55cb22fc5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:53.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f693659c640 1 --2- 172.21.15.62:0/905709202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fc5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f693659c640 1 -- 172.21.15.62:0/905709202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb22fd0170 con 0x55cb22fd4350 2024-09-11T23:55:53.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f693759e640 1 --2- 172.21.15.62:0/905709202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 0x55cb22fd5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:55:53.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f6936d9d640 1 --2- 172.21.15.62:0/905709202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fc5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:55:53.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f693659c640 1 --2- 172.21.15.62:0/905709202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fd5930 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f692c0c1040 tx=0x7f692c0c90b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e47562a61f14f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:53.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f691cff9640 1 -- 172.21.15.62:0/905709202 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0d92b0 con 0x55cb22fd4350 2024-09-11T23:55:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.696+0000 7f691cff9640 1 -- 172.21.15.62:0/905709202 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c0f1700 con 0x55cb22fd4350 2024-09-11T23:55:53.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.697+0000 7f693f46d740 1 -- 172.21.15.62:0/905709202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 msgr2=0x55cb22fd5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:53.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.697+0000 7f693f46d740 1 --2- 172.21.15.62:0/905709202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fd5930 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f692c0c1040 tx=0x7f692c0c90b0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.698+0000 7f693f46d740 1 -- 172.21.15.62:0/905709202 shutdown_connections 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.698+0000 7f693f46d740 1 --2- 172.21.15.62:0/905709202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fddb80 0x55cb22fd5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.698+0000 7f693f46d740 1 --2- 172.21.15.62:0/905709202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd4350 0x55cb22fd5930 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.698+0000 7f693f46d740 1 --2- 172.21.15.62:0/905709202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fd2de0 0x55cb22fc5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.698+0000 7f693f46d740 1 -- 172.21.15.62:0/905709202 >> 172.21.15.62:0/905709202 conn(0x55cb22f93310 msgr2=0x55cb22fc54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.698+0000 7f693f46d740 1 -- 172.21.15.62:0/905709202 shutdown_connections 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.698+0000 7f693f46d740 1 -- 172.21.15.62:0/905709202 wait complete. 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.698+0000 7f693f46d740 1 -- start start 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.698+0000 7f693f46d740 1 Processor -- start 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f693f46d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd2de0 0x55cb22ff7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f693f46d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd4350 0x55cb22fd7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f693f46d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 0x55cb22fd7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f693f46d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f692400cde0 con 0x55cb22fd2de0 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f693f46d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f692400e250 con 0x55cb22fddb80 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f693f46d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f692400d4b0 con 0x55cb22fd4350 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f6936d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd2de0 0x55cb22ff7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f693f46d740 10 client.0 _ll_register_callbacks cb 0x7f6928009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f6936d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd2de0 0x55cb22ff7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48522/0 (socket says 172.21.15.62:48522) 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f6936d9d640 1 -- 172.21.15.62:0/1891945537 learned_addr learned my addr 172.21.15.62:0/1891945537 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f693759e640 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 0x55cb22fd7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f693659c640 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd4350 0x55cb22fd7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f6936d9d640 1 -- 172.21.15.62:0/1891945537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd4350 msgr2=0x55cb22fd7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f6936d9d640 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd4350 0x55cb22fd7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f6936d9d640 1 -- 172.21.15.62:0/1891945537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 msgr2=0x55cb22fd7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f6936d9d640 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 0x55cb22fd7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f6936d9d640 1 -- 172.21.15.62:0/1891945537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb22fd8250 con 0x55cb22fd2de0 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.699+0000 7f693759e640 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 0x55cb22fd7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f6936d9d640 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd2de0 0x55cb22ff7130 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6928006fd0 tx=0x7f6928004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7402f69f0d44f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f691aff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f691aff5640 1 -- 172.21.15.62:0/1891945537 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928019050 con 0x55cb22fd2de0 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f691aff5640 1 -- 172.21.15.62:0/1891945537 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6928003890 con 0x55cb22fd2de0 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f693f46d740 1 -- 172.21.15.62:0/1891945537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f692c0f1700 con 0x55cb22fd2de0 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f691aff5640 1 -- 172.21.15.62:0/1891945537 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6928003a90 con 0x55cb22fd2de0 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f693f46d740 10 client.16029 fetch_fsmap learned FSMap version 29 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f693f46d740 1 -- 172.21.15.62:0/1891945537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb230691a0 con 0x55cb22fd2de0 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f691aff5640 1 -- 172.21.15.62:0/1891945537 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c0f1700 con 0x55cb22fd2de0 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f691aff5640 1 -- 172.21.15.62:0/1891945537 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6928015940 con 0x55cb22fd2de0 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f693f46d740 10 client.16029 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:55:53.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f693f46d740 10 client.16029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:55:53.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f693f46d740 20 client.16029 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:53.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f693f46d740 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cb230698f0 0x55cb2306bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:53.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f693f46d740 4 client.16029 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:55:53.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.700+0000 7f693f46d740 1 -- 172.21.15.62:0/1891945537 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb22f3d860 con 0x55cb230698f0 2024-09-11T23:55:53.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.701+0000 7f693659c640 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cb230698f0 0x55cb2306bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:53.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.701+0000 7f693659c640 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cb230698f0 0x55cb2306bd10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92580f9cdd743c0a server_cookie=2ce3ab4d7a79a34 in_seq=0 out_seq=0 2024-09-11T23:55:53.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.701+0000 7f691aff5640 10 client.16029 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:55:53.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.701+0000 7f691aff5640 1 -- 172.21.15.62:0/1891945537 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb22f3d860 con 0x55cb230698f0 2024-09-11T23:55:53.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.701+0000 7f691aff5640 10 client.16029 handle_command_reply: tid=0 2024-09-11T23:55:53.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.702+0000 7f693f46d740 1 client.16029 shutdown 2024-09-11T23:55:53.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.702+0000 7f693f46d740 1 -- 172.21.15.62:0/1891945537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd2de0 msgr2=0x55cb22ff7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:53.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.702+0000 7f693f46d740 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd2de0 0x55cb22ff7130 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6928006fd0 tx=0x7f6928004270 comp rx=0 tx=0).stop 2024-09-11T23:55:53.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.704+0000 7f693f46d740 1 -- 172.21.15.62:0/1891945537 shutdown_connections 2024-09-11T23:55:53.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.704+0000 7f693f46d740 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cb230698f0 0x55cb2306bd10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.704+0000 7f691aff5640 0 client.16029 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:55:53.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.704+0000 7f693f46d740 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb22fddb80 0x55cb22fd7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.704+0000 7f693f46d740 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb22fd4350 0x55cb22fd7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.704+0000 7f693f46d740 1 --2- 172.21.15.62:0/1891945537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb22fd2de0 0x55cb22ff7130 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.704+0000 7f693f46d740 1 -- 172.21.15.62:0/1891945537 >> 172.21.15.62:0/1891945537 conn(0x55cb22f93310 msgr2=0x55cb22fda6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:53.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.704+0000 7f693f46d740 1 -- 172.21.15.62:0/1891945537 shutdown_connections 2024-09-11T23:55:53.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.704+0000 7f693f46d740 1 -- 172.21.15.62:0/1891945537 wait complete. 2024-09-11T23:55:53.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.704+0000 7f693f46d740 20 client.16029 trim_cache size 0 max 16384 2024-09-11T23:55:53.706 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:55:53.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 -- 172.21.15.62:0/1716405385 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69200a2e40 msgr2=0x7f69200a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:53.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69200a2e40 0x7f69200a5300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55cb22fd3880 tx=0x7f6930004ab0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 -- 172.21.15.62:0/1716405385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69381027a0 msgr2=0x7f6938196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:53.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69381027a0 0x7f6938196820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f692c035000 tx=0x7f692c0043a0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 -- 172.21.15.62:0/1716405385 shutdown_connections 2024-09-11T23:55:53.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69200a2e40 0x7f69200a5300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6938106f20 0x7f693819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69381027a0 0x7f6938196820 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 --2- 172.21.15.62:0/1716405385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6938101df0 0x7f69381962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:53.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 -- 172.21.15.62:0/1716405385 >> 172.21.15.62:0/1716405385 conn(0x7f69380fd8f0 msgr2=0x7f69380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:53.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 -- 172.21.15.62:0/1716405385 shutdown_connections 2024-09-11T23:55:53.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:53.705+0000 7f693d094640 1 -- 172.21.15.62:0/1716405385 wait complete. 2024-09-11T23:55:54.610 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:55:54.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:54 smithi120 ceph-mon[35182]: pgmap v117: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.2 MiB/s wr, 683 op/s 2024-09-11T23:55:54.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:54 smithi083 ceph-mon[34805]: pgmap v117: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.2 MiB/s wr, 683 op/s 2024-09-11T23:55:54.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:54 smithi062 ceph-mon[30958]: pgmap v117: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.2 MiB/s wr, 683 op/s 2024-09-11T23:55:55.031 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.576+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1963149196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c501082a0 msgr2=0x7f1c5010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.576+0000 7f1c54eb1640 1 --2- 172.21.15.62:0/1963149196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c501082a0 0x7f1c5010ec40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009cd0 tx=0x7f1c4802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.576+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1963149196 shutdown_connections 2024-09-11T23:55:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.576+0000 7f1c54eb1640 1 --2- 172.21.15.62:0/1963149196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c501082a0 0x7f1c5010ec40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.576+0000 7f1c54eb1640 1 --2- 172.21.15.62:0/1963149196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c501036e0 0x7f1c50107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.576+0000 7f1c54eb1640 1 --2- 172.21.15.62:0/1963149196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c50102d30 0x7f1c50103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.576+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1963149196 >> 172.21.15.62:0/1963149196 conn(0x7f1c500fc7d0 msgr2=0x7f1c500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.576+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1963149196 shutdown_connections 2024-09-11T23:55:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.576+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1963149196 wait complete. 2024-09-11T23:55:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.577+0000 7f1c54eb1640 1 Processor -- start 2024-09-11T23:55:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.577+0000 7f1c54eb1640 1 -- start start 2024-09-11T23:55:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c54eb1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c50102d30 0x7f1c50100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c54eb1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c501036e0 0x7f1c50100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c501036e0 0x7f1c50100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c54eb1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c501082a0 0x7f1c50101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c50102d30 0x7f1c50100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c50102d30 0x7f1c50100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:44066/0 (socket says 172.21.15.62:44066) 2024-09-11T23:55:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4e59c640 1 -- 172.21.15.62:0/1060179333 learned_addr learned my addr 172.21.15.62:0/1060179333 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50112080 con 0x7f1c501036e0 2024-09-11T23:55:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50111f00 con 0x7f1c50102d30 2024-09-11T23:55:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50112200 con 0x7f1c501082a0 2024-09-11T23:55:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4ed9d640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c501082a0 0x7f1c50101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c501036e0 0x7f1c50100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48528/0 (socket says 172.21.15.62:48528) 2024-09-11T23:55:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4ed9d640 1 -- 172.21.15.62:0/1060179333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c50102d30 msgr2=0x7f1c50100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4ed9d640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c50102d30 0x7f1c50100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4ed9d640 1 -- 172.21.15.62:0/1060179333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c501036e0 msgr2=0x7f1c50100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4ed9d640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c501036e0 0x7f1c50100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.578+0000 7f1c4ed9d640 1 -- 172.21.15.62:0/1060179333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c501a6ac0 con 0x7f1c501082a0 2024-09-11T23:55:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.579+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c501036e0 0x7f1c50100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.579+0000 7f1c4ed9d640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c501082a0 0x7f1c50101410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1c4802ae00 tx=0x7f1c48031df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.579+0000 7f1c3f7fe640 1 -- 172.21.15.62:0/1060179333 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48003d80 con 0x7f1c501082a0 2024-09-11T23:55:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.579+0000 7f1c3f7fe640 1 -- 172.21.15.62:0/1060179333 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c48004520 con 0x7f1c501082a0 2024-09-11T23:55:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.579+0000 7f1c3f7fe640 1 -- 172.21.15.62:0/1060179333 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48030d00 con 0x7f1c501082a0 2024-09-11T23:55:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.580+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5006bc10 con 0x7f1c501082a0 2024-09-11T23:55:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.580+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5006c100 con 0x7f1c501082a0 2024-09-11T23:55:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.582+0000 7f1c3f7fe640 1 -- 172.21.15.62:0/1060179333 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1c48046480 con 0x7f1c501082a0 2024-09-11T23:55:55.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.583+0000 7f1c3f7fe640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1c380b3d80 0x7f1c380b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.583+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1c380b3d80 0x7f1c380b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.583+0000 7f1c3f7fe640 1 -- 172.21.15.62:0/1060179333 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f1c4802e030 con 0x7f1c501082a0 2024-09-11T23:55:55.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.584+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1c380b3d80 0x7f1c380b6240 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009c90 tx=0x7f1c40005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:55.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.584+0000 7f1c5728a740 1 Processor -- start 2024-09-11T23:55:55.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.585+0000 7f1c5728a740 1 -- start start 2024-09-11T23:55:55.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.585+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d27b40 0x55a8e2d26090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.585+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 0x55a8e2d26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.585+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d39b80 0x55a8e2d3bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.585+0000 7f1c5728a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2c8c2f0 con 0x55a8e2d27b40 2024-09-11T23:55:55.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.585+0000 7f1c5728a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2c33cf0 con 0x55a8e2d265d0 2024-09-11T23:55:55.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.585+0000 7f1c5728a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2c882b0 con 0x55a8e2d39b80 2024-09-11T23:55:55.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.585+0000 7f1c4e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d27b40 0x55a8e2d26090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.585+0000 7f1c4e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d27b40 0x55a8e2d26090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48544/0 (socket says 172.21.15.62:48544) 2024-09-11T23:55:55.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.585+0000 7f1c4e59c640 1 -- 172.21.15.62:0/1012849747 learned_addr learned my addr 172.21.15.62:0/1012849747 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:55.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 0x55a8e2d26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c4ed9d640 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d39b80 0x55a8e2d3bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c4e59c640 1 -- 172.21.15.62:0/1012849747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d39b80 msgr2=0x55a8e2d3bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:55.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d39b80 0x55a8e2d3bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c4e59c640 1 -- 172.21.15.62:0/1012849747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 msgr2=0x55a8e2d26a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 0x55a8e2d26a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c4e59c640 1 -- 172.21.15.62:0/1012849747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e2c183a0 con 0x55a8e2d27b40 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c4ed9d640 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d39b80 0x55a8e2d3bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 0x55a8e2d26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d27b40 0x55a8e2d26090 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009fc0 tx=0x7f1c40014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32efc3896a93af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c3cff9640 1 -- 172.21.15.62:0/1012849747 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4002b070 con 0x55a8e2d27b40 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c3cff9640 1 -- 172.21.15.62:0/1012849747 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c40026b00 con 0x55a8e2d27b40 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.586+0000 7f1c3cff9640 1 -- 172.21.15.62:0/1012849747 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4000eb50 con 0x55a8e2d27b40 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.587+0000 7f1c5728a740 1 -- 172.21.15.62:0/1012849747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d27b40 msgr2=0x55a8e2d26090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.588+0000 7f1c5728a740 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d27b40 0x55a8e2d26090 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009fc0 tx=0x7f1c40014000 comp rx=0 tx=0).stop 2024-09-11T23:55:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.588+0000 7f1c5728a740 1 -- 172.21.15.62:0/1012849747 shutdown_connections 2024-09-11T23:55:55.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.588+0000 7f1c5728a740 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d39b80 0x55a8e2d3bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.588+0000 7f1c5728a740 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 0x55a8e2d26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.588+0000 7f1c5728a740 1 --2- 172.21.15.62:0/1012849747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d27b40 0x55a8e2d26090 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.588+0000 7f1c5728a740 1 -- 172.21.15.62:0/1012849747 >> 172.21.15.62:0/1012849747 conn(0x55a8e2cef310 msgr2=0x55a8e2d22c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.589+0000 7f1c5728a740 1 -- 172.21.15.62:0/1012849747 shutdown_connections 2024-09-11T23:55:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.589+0000 7f1c5728a740 1 -- 172.21.15.62:0/1012849747 wait complete. 2024-09-11T23:55:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.590+0000 7f1c5728a740 1 -- start start 2024-09-11T23:55:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.590+0000 7f1c5728a740 1 Processor -- start 2024-09-11T23:55:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.590+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d265d0 0x55a8e2d33530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.590+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d27b40 0x55a8e2d33a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.590+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 0x55a8e2d39240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c5728a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2d296e0 con 0x55a8e2d39b80 2024-09-11T23:55:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c5728a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2c47060 con 0x55a8e2d27b40 2024-09-11T23:55:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d27b40 0x55a8e2d33a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d27b40 0x55a8e2d33a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:44072/0 (socket says 172.21.15.62:44072) 2024-09-11T23:55:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/2325752833 learned_addr learned my addr 172.21.15.62:0/2325752833 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4ed9d640 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 0x55a8e2d39240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d265d0 0x55a8e2d33530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c5728a740 1 -- 172.21.15.62:0/2325752833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a8e2d29860 con 0x55a8e2d265d0 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c5728a740 10 client.0 _ll_register_callbacks cb 0x55a8e299a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/2325752833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d265d0 msgr2=0x55a8e2d33530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d265d0 0x55a8e2d33530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/2325752833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 msgr2=0x55a8e2d39240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 0x55a8e2d39240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/2325752833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e29fd3c0 con 0x55a8e2d27b40 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c5728a740 1 -- 172.21.15.62:0/2325752833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c40026b00 con 0x55a8e2d27b40 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d27b40 0x55a8e2d33a70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1c44013790 tx=0x7f1c44013c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c7315f4fd820cd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.591+0000 7f1c35ffb640 10 client.25712 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c35ffb640 1 -- 172.21.15.62:0/2325752833 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44020060 con 0x55a8e2d27b40 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c35ffb640 1 -- 172.21.15.62:0/2325752833 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f1c44011dd0 con 0x55a8e2d27b40 2024-09-11T23:55:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c35ffb640 1 -- 172.21.15.62:0/2325752833 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c4400f450 con 0x55a8e2d27b40 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c5728a740 10 client.25712 fetch_fsmap learned FSMap version 29 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c5728a740 1 -- 172.21.15.62:0/2325752833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55a8e2d50b20 con 0x55a8e2d27b40 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c35ffb640 1 -- 172.21.15.62:0/2325752833 <== mon.2 v2:172.21.15.83:3300/0 4 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f1c4401a030 con 0x55a8e2d27b40 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c5728a740 10 client.25712 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c5728a740 10 client.25712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c5728a740 20 client.25712 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c5728a740 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a8e2d51e70 0x55a8e2d54260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.592+0000 7f1c5728a740 4 client.25712 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.593+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a8e2d51e70 0x55a8e2d54260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.593+0000 7f1c5728a740 1 -- 172.21.15.62:0/2325752833 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8e2c99860 con 0x55a8e2d51e70 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.593+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a8e2d51e70 0x55a8e2d54260 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9130055f0aa38aee server_cookie=5176afdac84d4077 in_seq=0 out_seq=0 2024-09-11T23:55:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.593+0000 7f1c35ffb640 10 client.25712 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:55.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.594+0000 7f1c35ffb640 1 -- 172.21.15.62:0/2325752833 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8e2c99860 con 0x55a8e2d51e70 2024-09-11T23:55:55.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.594+0000 7f1c35ffb640 10 client.25712 handle_command_reply: tid=0 2024-09-11T23:55:55.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.594+0000 7f1c5728a740 1 client.25712 shutdown 2024-09-11T23:55:55.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.594+0000 7f1c5728a740 1 -- 172.21.15.62:0/2325752833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d27b40 msgr2=0x55a8e2d33a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:55.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c5728a740 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d27b40 0x55a8e2d33a70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1c44013790 tx=0x7f1c44013c50 comp rx=0 tx=0).stop 2024-09-11T23:55:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c5728a740 1 -- 172.21.15.62:0/2325752833 shutdown_connections 2024-09-11T23:55:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c5728a740 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a8e2d51e70 0x55a8e2d54260 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c5728a740 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 0x55a8e2d39240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c5728a740 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d27b40 0x55a8e2d33a70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c5728a740 1 --2- 172.21.15.62:0/2325752833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d265d0 0x55a8e2d33530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c5728a740 1 -- 172.21.15.62:0/2325752833 >> 172.21.15.62:0/2325752833 conn(0x55a8e2cef310 msgr2=0x55a8e2d28930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c35ffb640 0 client.25712 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c5728a740 1 -- 172.21.15.62:0/2325752833 shutdown_connections 2024-09-11T23:55:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c5728a740 1 -- 172.21.15.62:0/2325752833 wait complete. 2024-09-11T23:55:55.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.595+0000 7f1c5728a740 20 client.25712 trim_cache size 0 max 16384 2024-09-11T23:55:55.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.616+0000 7f1c5728a740 1 Processor -- start 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.616+0000 7f1c5728a740 1 -- start start 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.616+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d265d0 0x55a8e2d390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.616+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d59a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.616+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d39b80 0x55a8e2d59f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.616+0000 7f1c5728a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50072da0 con 0x55a8e2d265d0 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.616+0000 7f1c5728a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50070670 con 0x55a8e2d39b80 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.616+0000 7f1c5728a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5006e4e0 con 0x55a8e2d27b40 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.616+0000 7f1c4dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d59a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c4dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d59a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52898/0 (socket says 172.21.15.62:52898) 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c4ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d39b80 0x55a8e2d59f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/2558392576 learned_addr learned my addr 172.21.15.62:0/2558392576 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/2558392576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d265d0 0x55a8e2d390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/2558392576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d39b80 msgr2=0x55a8e2d59f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:55.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/2558392576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d39b80 0x55a8e2d59f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/2558392576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d265d0 msgr2=0x55a8e2d390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:55.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/2558392576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d265d0 0x55a8e2d390a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/2558392576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e2d33de0 con 0x55a8e2d27b40 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/2558392576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d59a40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1c44013760 tx=0x7f1c44016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5324c3d3d8d7732a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c37fff640 1 -- 172.21.15.62:0/2558392576 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44020070 con 0x55a8e2d27b40 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c37fff640 1 -- 172.21.15.62:0/2558392576 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c40026b00 con 0x55a8e2d27b40 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.617+0000 7f1c37fff640 1 -- 172.21.15.62:0/2558392576 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44003830 con 0x55a8e2d27b40 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.618+0000 7f1c5728a740 1 -- 172.21.15.62:0/2558392576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 msgr2=0x55a8e2d59a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.618+0000 7f1c5728a740 1 --2- 172.21.15.62:0/2558392576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d59a40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1c44013760 tx=0x7f1c44016000 comp rx=0 tx=0).stop 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.618+0000 7f1c5728a740 1 -- 172.21.15.62:0/2558392576 shutdown_connections 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.618+0000 7f1c5728a740 1 --2- 172.21.15.62:0/2558392576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d39b80 0x55a8e2d59f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.618+0000 7f1c5728a740 1 --2- 172.21.15.62:0/2558392576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d59a40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.618+0000 7f1c5728a740 1 --2- 172.21.15.62:0/2558392576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d265d0 0x55a8e2d390a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.618+0000 7f1c5728a740 1 -- 172.21.15.62:0/2558392576 >> 172.21.15.62:0/2558392576 conn(0x55a8e2cef310 msgr2=0x55a8e2d2d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:55.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.618+0000 7f1c5728a740 1 -- 172.21.15.62:0/2558392576 shutdown_connections 2024-09-11T23:55:55.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.618+0000 7f1c5728a740 1 -- 172.21.15.62:0/2558392576 wait complete. 2024-09-11T23:55:55.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.619+0000 7f1c5728a740 1 -- start start 2024-09-11T23:55:55.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.619+0000 7f1c5728a740 1 Processor -- start 2024-09-11T23:55:55.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.619+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 0x55a8e2d61770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.619+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d61cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.620+0000 7f1c5728a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 0x55a8e2d621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.620+0000 7f1c5728a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5006eb30 con 0x55a8e2d39b80 2024-09-11T23:55:55.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.620+0000 7f1c5728a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5006ffa0 con 0x55a8e2d265d0 2024-09-11T23:55:55.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.620+0000 7f1c5728a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5006f200 con 0x55a8e2d27b40 2024-09-11T23:55:55.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.620+0000 7f1c5728a740 10 client.0 _ll_register_callbacks cb 0x7f1c44009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:55:55.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.620+0000 7f1c4ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 0x55a8e2d621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.620+0000 7f1c4ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 0x55a8e2d621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48574/0 (socket says 172.21.15.62:48574) 2024-09-11T23:55:55.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.620+0000 7f1c4ed9d640 1 -- 172.21.15.62:0/1288867887 learned_addr learned my addr 172.21.15.62:0/1288867887 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:55:55.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.620+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 0x55a8e2d61770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.621+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d61cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.622+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/1288867887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 msgr2=0x55a8e2d61770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.622+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 0x55a8e2d61770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.622+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/1288867887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 msgr2=0x55a8e2d621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:55:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.622+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 0x55a8e2d621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.622+0000 7f1c4dd9b640 1 -- 172.21.15.62:0/1288867887 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e2d23960 con 0x55a8e2d27b40 2024-09-11T23:55:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.622+0000 7f1c4dd9b640 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d61cb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55a8e2d1e6c0 tx=0x7f1c4400f5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ecf754f110519c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:55:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.622+0000 7f1c35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:55:55.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.624+0000 7f1c35ffb640 1 -- 172.21.15.62:0/1288867887 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44020040 con 0x55a8e2d27b40 2024-09-11T23:55:55.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.624+0000 7f1c35ffb640 1 -- 172.21.15.62:0/1288867887 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f1c4402e630 con 0x55a8e2d27b40 2024-09-11T23:55:55.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.624+0000 7f1c5728a740 1 -- 172.21.15.62:0/1288867887 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c40026b00 con 0x55a8e2d27b40 2024-09-11T23:55:55.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c35ffb640 1 -- 172.21.15.62:0/1288867887 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c4400fdd0 con 0x55a8e2d27b40 2024-09-11T23:55:55.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c5728a740 10 client.25651 fetch_fsmap learned FSMap version 29 2024-09-11T23:55:55.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c5728a740 1 -- 172.21.15.62:0/1288867887 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a8e2dbcc40 con 0x55a8e2d27b40 2024-09-11T23:55:55.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c35ffb640 1 -- 172.21.15.62:0/1288867887 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c40026b00 con 0x55a8e2d27b40 2024-09-11T23:55:55.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c35ffb640 1 -- 172.21.15.62:0/1288867887 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f1c4401a040 con 0x55a8e2d27b40 2024-09-11T23:55:55.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c5728a740 10 client.25651 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:55:55.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c5728a740 10 client.25651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:55:55.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c5728a740 20 client.25651 populate_metadata read hostname 'smithi062' 2024-09-11T23:55:55.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c5728a740 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a8e2dbd1f0 0x55a8e2dbf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:55:55.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c5728a740 4 client.25651 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:55:55.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.625+0000 7f1c5728a740 1 -- 172.21.15.62:0/1288867887 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8e2c93d70 con 0x55a8e2dbd1f0 2024-09-11T23:55:55.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.626+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a8e2dbd1f0 0x55a8e2dbf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:55:55.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c4e59c640 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a8e2dbd1f0 0x55a8e2dbf5e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58335d87c7b70262 server_cookie=e04c77e577788f67 in_seq=0 out_seq=0 2024-09-11T23:55:55.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c35ffb640 10 client.25651 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:55.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c35ffb640 1 -- 172.21.15.62:0/1288867887 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8e2c93d70 con 0x55a8e2dbd1f0 2024-09-11T23:55:55.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c35ffb640 10 client.25651 handle_command_reply: tid=0 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c5728a740 1 client.25651 shutdown 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c5728a740 1 -- 172.21.15.62:0/1288867887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 msgr2=0x55a8e2d61cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c5728a740 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d61cb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55a8e2d1e6c0 tx=0x7f1c4400f5f0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c5728a740 1 -- 172.21.15.62:0/1288867887 shutdown_connections 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c5728a740 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a8e2dbd1f0 0x55a8e2dbf5e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c35ffb640 0 client.25651 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c5728a740 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8e2d39b80 0x55a8e2d621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c5728a740 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e2d27b40 0x55a8e2d61cb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c5728a740 1 --2- 172.21.15.62:0/1288867887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e2d265d0 0x55a8e2d61770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.628+0000 7f1c5728a740 1 -- 172.21.15.62:0/1288867887 >> 172.21.15.62:0/1288867887 conn(0x55a8e2cef310 msgr2=0x55a8e2d214e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.629+0000 7f1c5728a740 1 -- 172.21.15.62:0/1288867887 shutdown_connections 2024-09-11T23:55:55.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.629+0000 7f1c5728a740 1 -- 172.21.15.62:0/1288867887 wait complete. 2024-09-11T23:55:55.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.629+0000 7f1c5728a740 20 client.25651 trim_cache size 0 max 16384 2024-09-11T23:55:55.629 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:55:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1c380b3d80 msgr2=0x7f1c380b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1c380b3d80 0x7f1c380b6240 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009c90 tx=0x7f1c40005f90 comp rx=0 tx=0).stop 2024-09-11T23:55:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c501082a0 msgr2=0x7f1c50101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:55:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c501082a0 0x7f1c50101410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1c4802ae00 tx=0x7f1c48031df0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 shutdown_connections 2024-09-11T23:55:55.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1c380b3d80 0x7f1c380b6240 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009c90 tx=0x7f1c40005f90 comp rx=0 tx=0).stop 2024-09-11T23:55:55.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c501082a0 0x7f1c50101410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c501036e0 0x7f1c50100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 --2- 172.21.15.62:0/1060179333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c50102d30 0x7f1c50100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:55:55.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 >> 172.21.15.62:0/1060179333 conn(0x7f1c500fc7d0 msgr2=0x7f1c50106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:55:55.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 shutdown_connections 2024-09-11T23:55:55.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:55:55.631+0000 7f1c54eb1640 1 -- 172.21.15.62:0/1060179333 wait complete. 2024-09-11T23:55:56.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:56 smithi120 ceph-mon[35182]: pgmap v118: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.2 MiB/s wr, 684 op/s 2024-09-11T23:55:56.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:56 smithi083 ceph-mon[34805]: pgmap v118: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.2 MiB/s wr, 684 op/s 2024-09-11T23:55:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:56 smithi062 ceph-mon[30958]: pgmap v118: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.2 MiB/s wr, 684 op/s 2024-09-11T23:55:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 195.0 s, 2755.8 tps, lat 36.227 ms stddev 65.841 2024-09-11T23:55:57.553 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:55:57.553 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:55:57.953 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:55:58.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:55:58 smithi120 ceph-mon[35182]: pgmap v119: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 MiB/s wr, 1.70k op/s 2024-09-11T23:55:58.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:55:58 smithi083 ceph-mon[34805]: pgmap v119: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 MiB/s wr, 1.70k op/s 2024-09-11T23:55:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:55:58 smithi062 ceph-mon[30958]: pgmap v119: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 MiB/s wr, 1.70k op/s 2024-09-11T23:56:00.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.322+0000 7f2b84f5c640 1 -- 172.21.15.62:0/740602843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8010eb10 msgr2=0x7f2b8010eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:00.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.322+0000 7f2b84f5c640 1 --2- 172.21.15.62:0/740602843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8010eb10 0x7f2b8010eef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009860 tx=0x7f2b7002a360 comp rx=0 tx=0).stop 2024-09-11T23:56:00.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.322+0000 7f2b84f5c640 1 -- 172.21.15.62:0/740602843 shutdown_connections 2024-09-11T23:56:00.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.322+0000 7f2b84f5c640 1 --2- 172.21.15.62:0/740602843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b80109b90 0x7f2b801158a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.322+0000 7f2b84f5c640 1 --2- 172.21.15.62:0/740602843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b801091f0 0x7f2b80109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.322+0000 7f2b84f5c640 1 --2- 172.21.15.62:0/740602843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8010eb10 0x7f2b8010eef0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.322+0000 7f2b84f5c640 1 -- 172.21.15.62:0/740602843 >> 172.21.15.62:0/740602843 conn(0x7f2b8006e2d0 msgr2=0x7f2b8006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:00.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.322+0000 7f2b84f5c640 1 -- 172.21.15.62:0/740602843 shutdown_connections 2024-09-11T23:56:00.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.322+0000 7f2b84f5c640 1 -- 172.21.15.62:0/740602843 wait complete. 2024-09-11T23:56:00.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.323+0000 7f2b84f5c640 1 Processor -- start 2024-09-11T23:56:00.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.323+0000 7f2b84f5c640 1 -- start start 2024-09-11T23:56:00.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.323+0000 7f2b84f5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b801091f0 0x7f2b80114be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.323+0000 7f2b84f5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b80109b90 0x7f2b80115120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.323+0000 7f2b84f5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b8010eb10 0x7f2b801abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.323+0000 7f2b84f5c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8011a7d0 con 0x7f2b801091f0 2024-09-11T23:56:00.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.323+0000 7f2b84f5c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8011a650 con 0x7f2b80109b90 2024-09-11T23:56:00.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.323+0000 7f2b84f5c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8011a950 con 0x7f2b8010eb10 2024-09-11T23:56:00.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.323+0000 7f2b7e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b801091f0 0x7f2b80114be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.323+0000 7f2b7dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b80109b90 0x7f2b80115120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.324+0000 7f2b7dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b80109b90 0x7f2b80115120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:44108/0 (socket says 172.21.15.62:44108) 2024-09-11T23:56:00.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.324+0000 7f2b7dd9b640 1 -- 172.21.15.62:0/6710263 learned_addr learned my addr 172.21.15.62:0/6710263 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:00.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.324+0000 7f2b7e59c640 1 -- 172.21.15.62:0/6710263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b8010eb10 msgr2=0x7f2b801abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.324+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b8010eb10 0x7f2b801abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.324+0000 7f2b7e59c640 1 -- 172.21.15.62:0/6710263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b80109b90 msgr2=0x7f2b80115120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.324+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b80109b90 0x7f2b80115120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.324+0000 7f2b7e59c640 1 -- 172.21.15.62:0/6710263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b801ac210 con 0x7f2b801091f0 2024-09-11T23:56:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.325+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b801091f0 0x7f2b80114be0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2b7002a880 tx=0x7f2b70032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.325+0000 7f2b6f7fe640 1 -- 172.21.15.62:0/6710263 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70037040 con 0x7f2b801091f0 2024-09-11T23:56:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.325+0000 7f2b6f7fe640 1 -- 172.21.15.62:0/6710263 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b70033b60 con 0x7f2b801091f0 2024-09-11T23:56:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.325+0000 7f2b84f5c640 1 -- 172.21.15.62:0/6710263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b801ac4a0 con 0x7f2b801091f0 2024-09-11T23:56:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.325+0000 7f2b84f5c640 1 -- 172.21.15.62:0/6710263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b801b3de0 con 0x7f2b801091f0 2024-09-11T23:56:00.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.326+0000 7f2b6f7fe640 1 -- 172.21.15.62:0/6710263 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70003850 con 0x7f2b801091f0 2024-09-11T23:56:00.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.330+0000 7f2b87335740 1 Processor -- start 2024-09-11T23:56:00.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.330+0000 7f2b87335740 1 -- start start 2024-09-11T23:56:00.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b6f7fe640 1 -- 172.21.15.62:0/6710263 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2b70042030 con 0x7f2b801091f0 2024-09-11T23:56:00.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b87335740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b4cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b87335740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b57800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b87335740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b5a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b87335740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b7d5ab2280 con 0x55b7d5b4cb60 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b87335740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b7d5a59b10 con 0x55b7d5b4d480 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b87335740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b7d5aae240 con 0x55b7d5b57d40 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b7ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b4cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b7ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b4cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48588/0 (socket says 172.21.15.62:48588) 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b7e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b57800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b7ed9d640 1 -- 172.21.15.62:0/2307602588 learned_addr learned my addr 172.21.15.62:0/2307602588 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b7dd9b640 1 --2- 172.21.15.62:0/2307602588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b5a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.331+0000 7f2b7e59c640 1 -- 172.21.15.62:0/2307602588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 msgr2=0x55b7d5b5a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.332+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/2307602588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b5a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.332+0000 7f2b7e59c640 1 -- 172.21.15.62:0/2307602588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 msgr2=0x55b7d5b4cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:00.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.332+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/2307602588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b4cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.332+0000 7f2b7e59c640 1 -- 172.21.15.62:0/2307602588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d5a661f0 con 0x55b7d5b4d480 2024-09-11T23:56:00.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.332+0000 7f2b7ed9d640 1 --2- 172.21.15.62:0/2307602588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b4cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:56:00.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.332+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/2307602588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b57800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2b700ebc40 tx=0x7f2b700f44d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2d1b0237de9574 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:00.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.332+0000 7f2b6f7fe640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b680a2d70 0x7f2b680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.332+0000 7f2b6cff9640 1 -- 172.21.15.62:0/2307602588 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b700eea60 con 0x55b7d5b4d480 2024-09-11T23:56:00.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.332+0000 7f2b7ed9d640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b680a2d70 0x7f2b680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.333+0000 7f2b6cff9640 1 -- 172.21.15.62:0/2307602588 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b700fb030 con 0x55b7d5b4d480 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.333+0000 7f2b6cff9640 1 -- 172.21.15.62:0/2307602588 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b700ba5e0 con 0x55b7d5b4d480 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.334+0000 7f2b6f7fe640 1 -- 172.21.15.62:0/6710263 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f2b700ee3b0 con 0x7f2b801091f0 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.334+0000 7f2b87335740 1 -- 172.21.15.62:0/2307602588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 msgr2=0x55b7d5b57800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.334+0000 7f2b7ed9d640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b680a2d70 0x7f2b680a5230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800e7c0 tx=0x7f2b78010040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.334+0000 7f2b87335740 1 --2- 172.21.15.62:0/2307602588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b57800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2b700ebc40 tx=0x7f2b700f44d0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.334+0000 7f2b87335740 1 -- 172.21.15.62:0/2307602588 shutdown_connections 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.334+0000 7f2b87335740 1 --2- 172.21.15.62:0/2307602588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b5a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.334+0000 7f2b87335740 1 --2- 172.21.15.62:0/2307602588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b57800 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.334+0000 7f2b87335740 1 --2- 172.21.15.62:0/2307602588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b4cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.334+0000 7f2b87335740 1 -- 172.21.15.62:0/2307602588 >> 172.21.15.62:0/2307602588 conn(0x55b7d5b490f0 msgr2=0x55b7d5b494d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 1 -- 172.21.15.62:0/2307602588 shutdown_connections 2024-09-11T23:56:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 1 -- 172.21.15.62:0/2307602588 wait complete. 2024-09-11T23:56:00.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 1 -- start start 2024-09-11T23:56:00.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 1 Processor -- start 2024-09-11T23:56:00.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b54b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b4d480 0x55b7d5b550c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b57d40 0x55b7d5b55600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b7d5a59b10 con 0x55b7d5b4cb60 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b7d5a6cf10 con 0x55b7d5b57d40 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b7d5b5a6d0 con 0x55b7d5b4d480 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.335+0000 7f2b87335740 10 client.0 _ll_register_callbacks cb 0x55b7d5a35a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b4d480 0x55b7d5b550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b4d480 0x55b7d5b550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52948/0 (socket says 172.21.15.62:52948) 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7dd9b640 1 -- 172.21.15.62:0/3730214641 learned_addr learned my addr 172.21.15.62:0/3730214641 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b54b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7ed9d640 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b57d40 0x55b7d5b55600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7dd9b640 1 -- 172.21.15.62:0/3730214641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b57d40 msgr2=0x55b7d5b55600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7dd9b640 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b57d40 0x55b7d5b55600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7dd9b640 1 -- 172.21.15.62:0/3730214641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 msgr2=0x55b7d5b54b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7dd9b640 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b54b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7dd9b640 1 -- 172.21.15.62:0/3730214641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d58236a0 con 0x55b7d5b4d480 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b7dd9b640 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b4d480 0x55b7d5b550c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400df30 tx=0x7f2b7400f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c376ce774c01ec62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b65ffb640 1 -- 172.21.15.62:0/3730214641 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74019020 con 0x55b7d5b4d480 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b87335740 1 -- 172.21.15.62:0/3730214641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b700fb030 con 0x55b7d5b4d480 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.336+0000 7f2b65ffb640 1 -- 172.21.15.62:0/3730214641 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f2b74014670 con 0x55b7d5b4d480 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b65ffb640 1 -- 172.21.15.62:0/3730214641 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b74024050 con 0x55b7d5b4d480 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b87335740 10 client.25657 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b87335740 1 -- 172.21.15.62:0/3730214641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b7d5b660e0 con 0x55b7d5b4d480 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b65ffb640 1 -- 172.21.15.62:0/3730214641 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b700fb030 con 0x55b7d5b4d480 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b65ffb640 1 -- 172.21.15.62:0/3730214641 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2b74015a80 con 0x55b7d5b4d480 2024-09-11T23:56:00.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b87335740 10 client.25657 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:00.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b87335740 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:00.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b87335740 20 client.25657 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:00.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b87335740 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b7d5b675a0 0x55b7d5b69a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b87335740 4 client.25657 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:00.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b87335740 1 -- 172.21.15.62:0/3730214641 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7d5abf7f0 con 0x55b7d5b675a0 2024-09-11T23:56:00.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.337+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b7d5b675a0 0x55b7d5b69a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.338+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b7d5b675a0 0x55b7d5b69a10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bf781db9d1d2ac server_cookie=c5efc51ae92f2654 in_seq=0 out_seq=0 2024-09-11T23:56:00.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.338+0000 7f2b65ffb640 10 client.25657 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:00.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.339+0000 7f2b65ffb640 1 -- 172.21.15.62:0/3730214641 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7d5abf7f0 con 0x55b7d5b675a0 2024-09-11T23:56:00.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.339+0000 7f2b65ffb640 10 client.25657 handle_command_reply: tid=0 2024-09-11T23:56:00.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.339+0000 7f2b87335740 1 client.25657 shutdown 2024-09-11T23:56:00.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.339+0000 7f2b87335740 1 -- 172.21.15.62:0/3730214641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b4d480 msgr2=0x55b7d5b550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.339+0000 7f2b87335740 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b4d480 0x55b7d5b550c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400df30 tx=0x7f2b7400f670 comp rx=0 tx=0).stop 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.341+0000 7f2b87335740 1 -- 172.21.15.62:0/3730214641 shutdown_connections 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.341+0000 7f2b87335740 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b7d5b675a0 0x55b7d5b69a10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.341+0000 7f2b87335740 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b57d40 0x55b7d5b55600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.341+0000 7f2b87335740 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b4d480 0x55b7d5b550c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.341+0000 7f2b87335740 1 --2- 172.21.15.62:0/3730214641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b54b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.341+0000 7f2b87335740 1 -- 172.21.15.62:0/3730214641 >> 172.21.15.62:0/3730214641 conn(0x55b7d5b490f0 msgr2=0x55b7d5b4ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.341+0000 7f2b65ffb640 0 client.25657 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.341+0000 7f2b87335740 1 -- 172.21.15.62:0/3730214641 shutdown_connections 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.341+0000 7f2b87335740 1 -- 172.21.15.62:0/3730214641 wait complete. 2024-09-11T23:56:00.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.341+0000 7f2b87335740 20 client.25657 trim_cache size 0 max 16384 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.378+0000 7f2b87335740 1 Processor -- start 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.378+0000 7f2b87335740 1 -- start start 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.378+0000 7f2b87335740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.378+0000 7f2b87335740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b6f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.378+0000 7f2b87335740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b6f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.379+0000 7f2b7dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b6f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.379+0000 7f2b7dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b6f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:44142/0 (socket says 172.21.15.62:44142) 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.379+0000 7f2b7dd9b640 1 -- 172.21.15.62:0/2963952530 learned_addr learned my addr 172.21.15.62:0/2963952530 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.379+0000 7f2b87335740 1 -- 172.21.15.62:0/2963952530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b801bef00 con 0x55b7d5b4cb60 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.379+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/2963952530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.379+0000 7f2b7ed9d640 1 --2- 172.21.15.62:0/2963952530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b6f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.379+0000 7f2b87335740 1 -- 172.21.15.62:0/2963952530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b801bc7d0 con 0x55b7d5b4d480 2024-09-11T23:56:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.379+0000 7f2b87335740 1 -- 172.21.15.62:0/2963952530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2b801ba620 con 0x55b7d5b57d40 2024-09-11T23:56:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.380+0000 7f2b7dd9b640 1 -- 172.21.15.62:0/2963952530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 msgr2=0x55b7d5b6f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.380+0000 7f2b7dd9b640 1 --2- 172.21.15.62:0/2963952530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b6f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.380+0000 7f2b7dd9b640 1 -- 172.21.15.62:0/2963952530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 msgr2=0x55b7d5b569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.380+0000 7f2b7dd9b640 1 --2- 172.21.15.62:0/2963952530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.381+0000 7f2b7dd9b640 1 -- 172.21.15.62:0/2963952530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d5b69d80 con 0x55b7d5b4d480 2024-09-11T23:56:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.381+0000 7f2b7dd9b640 1 --2- 172.21.15.62:0/2963952530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b6f320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2b74006a50 tx=0x7f2b7400d440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=725b5db14f5ca75b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.381+0000 7f2b67fff640 1 -- 172.21.15.62:0/2963952530 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7402c070 con 0x55b7d5b4d480 2024-09-11T23:56:00.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.381+0000 7f2b67fff640 1 -- 172.21.15.62:0/2963952530 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b74004520 con 0x55b7d5b4d480 2024-09-11T23:56:00.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.381+0000 7f2b67fff640 1 -- 172.21.15.62:0/2963952530 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74019050 con 0x55b7d5b4d480 2024-09-11T23:56:00.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.382+0000 7f2b87335740 1 -- 172.21.15.62:0/2963952530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 msgr2=0x55b7d5b6f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:00.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.382+0000 7f2b87335740 1 --2- 172.21.15.62:0/2963952530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b6f320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2b74006a50 tx=0x7f2b7400d440 comp rx=0 tx=0).stop 2024-09-11T23:56:00.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.385+0000 7f2b87335740 1 -- 172.21.15.62:0/2963952530 shutdown_connections 2024-09-11T23:56:00.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.385+0000 7f2b87335740 1 --2- 172.21.15.62:0/2963952530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b6f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.385+0000 7f2b87335740 1 --2- 172.21.15.62:0/2963952530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4d480 0x55b7d5b6f320 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.385+0000 7f2b87335740 1 --2- 172.21.15.62:0/2963952530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.385+0000 7f2b87335740 1 -- 172.21.15.62:0/2963952530 >> 172.21.15.62:0/2963952530 conn(0x55b7d5b490f0 msgr2=0x55b7d5b562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:00.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.385+0000 7f2b87335740 1 -- 172.21.15.62:0/2963952530 shutdown_connections 2024-09-11T23:56:00.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.385+0000 7f2b87335740 1 -- 172.21.15.62:0/2963952530 wait complete. 2024-09-11T23:56:00.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.386+0000 7f2b87335740 1 -- start start 2024-09-11T23:56:00.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.386+0000 7f2b87335740 1 Processor -- start 2024-09-11T23:56:00.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b87335740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b60100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b87335740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4d480 0x55b7d5b5b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b87335740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b5ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b87335740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b801bac90 con 0x55b7d5b4d480 2024-09-11T23:56:00.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b87335740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b801bc100 con 0x55b7d5b4cb60 2024-09-11T23:56:00.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b87335740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2b801bb360 con 0x55b7d5b57d40 2024-09-11T23:56:00.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b7e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b7e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b60100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:44150/0 (socket says 172.21.15.62:44150) 2024-09-11T23:56:00.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b7e59c640 1 -- 172.21.15.62:0/2340274731 learned_addr learned my addr 172.21.15.62:0/2340274731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:00.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b7dd9b640 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4d480 0x55b7d5b5b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b87335740 10 client.0 _ll_register_callbacks cb 0x7f2b7400d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:00.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b7e59c640 1 -- 172.21.15.62:0/2340274731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 msgr2=0x55b7d5b5ba90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b5ba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b7e59c640 1 -- 172.21.15.62:0/2340274731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4d480 msgr2=0x55b7d5b5b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4d480 0x55b7d5b5b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.387+0000 7f2b7e59c640 1 -- 172.21.15.62:0/2340274731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d5b44830 con 0x55b7d5b4cb60 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.388+0000 7f2b7e59c640 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b60100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2b700f4360 tx=0x7f2b7002a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36525b4fd8891025 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.388+0000 7f2b65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.388+0000 7f2b65ffb640 1 -- 172.21.15.62:0/2340274731 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b700b9060 con 0x55b7d5b4cb60 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.388+0000 7f2b65ffb640 1 -- 172.21.15.62:0/2340274731 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f2b700c1320 con 0x55b7d5b4cb60 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.388+0000 7f2b87335740 1 -- 172.21.15.62:0/2340274731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b74004520 con 0x55b7d5b4cb60 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.388+0000 7f2b65ffb640 1 -- 172.21.15.62:0/2340274731 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b700b9250 con 0x55b7d5b4cb60 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.388+0000 7f2b87335740 10 client.25736 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.388+0000 7f2b87335740 1 -- 172.21.15.62:0/2340274731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b7d5bd2260 con 0x55b7d5b4cb60 2024-09-11T23:56:00.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.388+0000 7f2b65ffb640 1 -- 172.21.15.62:0/2340274731 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b74004520 con 0x55b7d5b4cb60 2024-09-11T23:56:00.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.389+0000 7f2b65ffb640 1 -- 172.21.15.62:0/2340274731 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2b700ee070 con 0x55b7d5b4cb60 2024-09-11T23:56:00.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.389+0000 7f2b87335740 10 client.25736 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:00.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.389+0000 7f2b87335740 10 client.25736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:00.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.389+0000 7f2b87335740 20 client.25736 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:00.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.389+0000 7f2b87335740 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b7d5bd2740 0x55b7d5bd4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:00.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.389+0000 7f2b87335740 4 client.25736 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:56:00.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.389+0000 7f2b87335740 1 -- 172.21.15.62:0/2340274731 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b7d5ab9d00 con 0x55b7d5bd2740 2024-09-11T23:56:00.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.389+0000 7f2b7dd9b640 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b7d5bd2740 0x55b7d5bd4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:00.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.391+0000 7f2b7dd9b640 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b7d5bd2740 0x55b7d5bd4b30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec2361bc5d4594c server_cookie=201f740de17b18a6 in_seq=0 out_seq=0 2024-09-11T23:56:00.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.391+0000 7f2b65ffb640 10 client.25736 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:00.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.392+0000 7f2b65ffb640 1 -- 172.21.15.62:0/2340274731 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b7d5ab9d00 con 0x55b7d5bd2740 2024-09-11T23:56:00.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.392+0000 7f2b65ffb640 10 client.25736 handle_command_reply: tid=0 2024-09-11T23:56:00.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.392+0000 7f2b87335740 1 client.25736 shutdown 2024-09-11T23:56:00.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.392+0000 7f2b87335740 1 -- 172.21.15.62:0/2340274731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4cb60 msgr2=0x55b7d5b60100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:00.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.392+0000 7f2b87335740 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b60100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2b700f4360 tx=0x7f2b7002a360 comp rx=0 tx=0).stop 2024-09-11T23:56:00.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.393+0000 7f2b87335740 1 -- 172.21.15.62:0/2340274731 shutdown_connections 2024-09-11T23:56:00.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.393+0000 7f2b87335740 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b7d5bd2740 0x55b7d5bd4b30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.393+0000 7f2b65ffb640 0 client.25736 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:00.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.393+0000 7f2b87335740 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7d5b57d40 0x55b7d5b5ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.393+0000 7f2b87335740 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d5b4d480 0x55b7d5b5b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.393+0000 7f2b87335740 1 --2- 172.21.15.62:0/2340274731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d5b4cb60 0x55b7d5b60100 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.393+0000 7f2b87335740 1 -- 172.21.15.62:0/2340274731 >> 172.21.15.62:0/2340274731 conn(0x55b7d5b490f0 msgr2=0x55b7d5b48270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:00.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.393+0000 7f2b87335740 1 -- 172.21.15.62:0/2340274731 shutdown_connections 2024-09-11T23:56:00.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.393+0000 7f2b87335740 1 -- 172.21.15.62:0/2340274731 wait complete. 2024-09-11T23:56:00.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.393+0000 7f2b87335740 20 client.25736 trim_cache size 0 max 16384 2024-09-11T23:56:00.394 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:56:00.394 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:56:00.394 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "9b74483c-65e6-4c1c-9d91-ed0133003974", 2024-09-11T23:56:00.395 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:56:00.395 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:56:00.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 -- 172.21.15.62:0/6710263 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b680a2d70 msgr2=0x7f2b680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:00.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b680a2d70 0x7f2b680a5230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800e7c0 tx=0x7f2b78010040 comp rx=0 tx=0).stop 2024-09-11T23:56:00.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 -- 172.21.15.62:0/6710263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b801091f0 msgr2=0x7f2b80114be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:00.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b801091f0 0x7f2b80114be0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2b7002a880 tx=0x7f2b70032740 comp rx=0 tx=0).stop 2024-09-11T23:56:00.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 -- 172.21.15.62:0/6710263 shutdown_connections 2024-09-11T23:56:00.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2b680a2d70 0x7f2b680a5230 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b8010eb10 0x7f2b801abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b80109b90 0x7f2b80115120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 --2- 172.21.15.62:0/6710263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b801091f0 0x7f2b80114be0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:00.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 -- 172.21.15.62:0/6710263 >> 172.21.15.62:0/6710263 conn(0x7f2b8006e2d0 msgr2=0x7f2b80072450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:00.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 -- 172.21.15.62:0/6710263 shutdown_connections 2024-09-11T23:56:00.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:00.394+0000 7f2b84f5c640 1 -- 172.21.15.62:0/6710263 wait complete. 2024-09-11T23:56:00.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:00 smithi062 ceph-mon[30958]: pgmap v120: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 MiB/s wr, 1.32k op/s 2024-09-11T23:56:00.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:00 smithi120 ceph-mon[35182]: pgmap v120: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 MiB/s wr, 1.32k op/s 2024-09-11T23:56:00.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:00 smithi083 ceph-mon[34805]: pgmap v120: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 MiB/s wr, 1.32k op/s 2024-09-11T23:56:01.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:56:01.410 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:56:01.410 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T23:56:01.410 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:56:01.735 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:01.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:01 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:56:01.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:01 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:01 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:56:01.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:01 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:56:01.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:01 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:01 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:56:01.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:01 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:01 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:01 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:56:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:01 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:01 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:56:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:01 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:56:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:01 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:01 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:56:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:01 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:01 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:01 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:56:01.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:01 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:01 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:56:01.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:01 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:56:01.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:01 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:01 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:56:01.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:01 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:01.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:01 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:02.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.310+0000 7f5d690b5640 1 -- 172.21.15.62:0/3538730667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d64100a90 msgr2=0x7f5d64100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:02.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.310+0000 7f5d690b5640 1 --2- 172.21.15.62:0/3538730667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d64100a90 0x7f5d64100ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009930 tx=0x7f5d5802a440 comp rx=0 tx=0).stop 2024-09-11T23:56:02.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.310+0000 7f5d690b5640 1 -- 172.21.15.62:0/3538730667 shutdown_connections 2024-09-11T23:56:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.310+0000 7f5d690b5640 1 --2- 172.21.15.62:0/3538730667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d64101430 0x7f5d6410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.310+0000 7f5d690b5640 1 --2- 172.21.15.62:0/3538730667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d64100a90 0x7f5d64100ef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.310+0000 7f5d690b5640 1 --2- 172.21.15.62:0/3538730667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d64106a90 0x7f5d64106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.310+0000 7f5d690b5640 1 -- 172.21.15.62:0/3538730667 >> 172.21.15.62:0/3538730667 conn(0x7f5d640fc7d0 msgr2=0x7f5d640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.310+0000 7f5d690b5640 1 -- 172.21.15.62:0/3538730667 shutdown_connections 2024-09-11T23:56:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.311+0000 7f5d690b5640 1 -- 172.21.15.62:0/3538730667 wait complete. 2024-09-11T23:56:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.312+0000 7f5d690b5640 1 Processor -- start 2024-09-11T23:56:02.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.312+0000 7f5d690b5640 1 -- start start 2024-09-11T23:56:02.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.313+0000 7f5d690b5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d64100a90 0x7f5d64197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.313+0000 7f5d690b5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d64101430 0x7f5d641978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.314+0000 7f5d690b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d64106a90 0x7f5d6419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.314+0000 7f5d6259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d64101430 0x7f5d641978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.314+0000 7f5d6259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d64101430 0x7f5d641978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52146/0 (socket says 172.21.15.62:52146) 2024-09-11T23:56:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.314+0000 7f5d6259c640 1 -- 172.21.15.62:0/1048662003 learned_addr learned my addr 172.21.15.62:0/1048662003 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.314+0000 7f5d6359e640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d64106a90 0x7f5d6419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.314+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5d64110ea0 con 0x7f5d64100a90 2024-09-11T23:56:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.314+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d64110d20 con 0x7f5d64106a90 2024-09-11T23:56:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.314+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d64111020 con 0x7f5d64101430 2024-09-11T23:56:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.315+0000 7f5d62d9d640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d64100a90 0x7f5d64197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.315+0000 7f5d6359e640 1 -- 172.21.15.62:0/1048662003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d64101430 msgr2=0x7f5d641978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:02.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.315+0000 7f5d6359e640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d64101430 0x7f5d641978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.315+0000 7f5d6359e640 1 -- 172.21.15.62:0/1048662003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d64100a90 msgr2=0x7f5d64197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:02.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.315+0000 7f5d6359e640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d64100a90 0x7f5d64197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.315+0000 7f5d6359e640 1 -- 172.21.15.62:0/1048662003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6419c310 con 0x7f5d64106a90 2024-09-11T23:56:02.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.317+0000 7f5d6359e640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d64106a90 0x7f5d6419bc30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00e940 tx=0x7f5d5c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:02.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.317+0000 7f5d4bfff640 1 -- 172.21.15.62:0/1048662003 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c015d90 con 0x7f5d64106a90 2024-09-11T23:56:02.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.317+0000 7f5d4bfff640 1 -- 172.21.15.62:0/1048662003 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5c00c850 con 0x7f5d64106a90 2024-09-11T23:56:02.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.317+0000 7f5d4bfff640 1 -- 172.21.15.62:0/1048662003 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c005550 con 0x7f5d64106a90 2024-09-11T23:56:02.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.318+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6419c600 con 0x7f5d64106a90 2024-09-11T23:56:02.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.318+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6406a790 con 0x7f5d64106a90 2024-09-11T23:56:02.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.323+0000 7f5d6b48e740 1 Processor -- start 2024-09-11T23:56:02.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.323+0000 7f5d4bfff640 1 -- 172.21.15.62:0/1048662003 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5d5c0056f0 con 0x7f5d64106a90 2024-09-11T23:56:02.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.324+0000 7f5d6b48e740 1 -- start start 2024-09-11T23:56:02.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.324+0000 7f5d4bfff640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5d4c0a2ec0 0x7f5d4c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40faab880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 0x55b40fab6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d6b48e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b40fa112e0 con 0x55b40faab4a0 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d6b48e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b40f9b8ce0 con 0x55b40faabdc0 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d6b48e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b40fa0d2a0 con 0x55b40fab6680 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d4bfff640 1 -- 172.21.15.62:0/1048662003 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f5d5c0d38c0 con 0x7f5d64106a90 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d62d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40faab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d6259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 0x55b40fab6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d62d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40faab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43632/0 (socket says 172.21.15.62:43632) 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d62d9d640 1 -- 172.21.15.62:0/669327234 learned_addr learned my addr 172.21.15.62:0/669327234 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:02.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.325+0000 7f5d6259c640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5d4c0a2ec0 0x7f5d4c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.326+0000 7f5d6359e640 1 --2- 172.21.15.62:0/669327234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.326+0000 7f5d6259c640 1 -- 172.21.15.62:0/669327234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 msgr2=0x55b40fab8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.326+0000 7f5d6259c640 1 --2- 172.21.15.62:0/669327234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.326+0000 7f5d6259c640 1 -- 172.21.15.62:0/669327234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 msgr2=0x55b40faab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.326+0000 7f5d6259c640 1 --2- 172.21.15.62:0/669327234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40faab880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.326+0000 7f5d6259c640 1 -- 172.21.15.62:0/669327234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b40f99d440 con 0x55b40faabdc0 2024-09-11T23:56:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.327+0000 7f5d6259c640 1 --2- 172.21.15.62:0/669327234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 0x55b40fab6140 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d640fed60 tx=0x7f5d580039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb6eeaf99050f9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.327+0000 7f5d6259c640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5d4c0a2ec0 0x7f5d4c0a5380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d58000c00 tx=0x7f5d58040000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.328+0000 7f5d497fa640 1 -- 172.21.15.62:0/669327234 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5804c070 con 0x55b40faabdc0 2024-09-11T23:56:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.328+0000 7f5d497fa640 1 -- 172.21.15.62:0/669327234 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5803d450 con 0x55b40faabdc0 2024-09-11T23:56:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.328+0000 7f5d497fa640 1 -- 172.21.15.62:0/669327234 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58035040 con 0x55b40faabdc0 2024-09-11T23:56:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.329+0000 7f5d6b48e740 1 -- 172.21.15.62:0/669327234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 msgr2=0x55b40fab6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.329+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/669327234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 0x55b40fab6140 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d640fed60 tx=0x7f5d580039c0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.329+0000 7f5d6b48e740 1 -- 172.21.15.62:0/669327234 shutdown_connections 2024-09-11T23:56:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.329+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/669327234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.329+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/669327234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 0x55b40fab6140 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.329+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/669327234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40faab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.329+0000 7f5d6b48e740 1 -- 172.21.15.62:0/669327234 >> 172.21.15.62:0/669327234 conn(0x55b40faa7a30 msgr2=0x55b40faa7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:02.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.329+0000 7f5d6b48e740 1 -- 172.21.15.62:0/669327234 shutdown_connections 2024-09-11T23:56:02.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.330+0000 7f5d6b48e740 1 -- 172.21.15.62:0/669327234 wait complete. 2024-09-11T23:56:02.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.330+0000 7f5d6b48e740 1 -- start start 2024-09-11T23:56:02.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.330+0000 7f5d6b48e740 1 Processor -- start 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.330+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faab4a0 0x55b40fab6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.330+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faabdc0 0x55b40fab32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.330+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.330+0000 7f5d6b48e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b40fab92a0 con 0x55b40faabdc0 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.330+0000 7f5d6b48e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b40f9cc050 con 0x55b40faab4a0 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.330+0000 7f5d6b48e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b40fab9420 con 0x55b40fab6680 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6b48e740 10 client.0 _ll_register_callbacks cb 0x55b40fa0c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52176/0 (socket says 172.21.15.62:52176) 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6359e640 1 -- 172.21.15.62:0/1694943896 learned_addr learned my addr 172.21.15.62:0/1694943896 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d62d9d640 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faab4a0 0x55b40fab6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6359e640 1 -- 172.21.15.62:0/1694943896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faab4a0 msgr2=0x55b40fab6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6259c640 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faabdc0 0x55b40fab32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6359e640 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faab4a0 0x55b40fab6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6359e640 1 -- 172.21.15.62:0/1694943896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faabdc0 msgr2=0x55b40fab32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6359e640 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faabdc0 0x55b40fab32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6359e640 1 -- 172.21.15.62:0/1694943896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40f7823c0 con 0x55b40fab6680 2024-09-11T23:56:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d62d9d640 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faab4a0 0x55b40fab6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:56:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6259c640 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faabdc0 0x55b40fab32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:56:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d6359e640 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab3830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c09c780 tx=0x7f5d5c0a05f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0b04d698a96c38c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.331+0000 7f5d46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:56:02.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.332+0000 7f5d46ff5640 1 -- 172.21.15.62:0/1694943896 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c023040 con 0x55b40fab6680 2024-09-11T23:56:02.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.332+0000 7f5d46ff5640 1 -- 172.21.15.62:0/1694943896 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f5d5c00ca00 con 0x55b40fab6680 2024-09-11T23:56:02.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.332+0000 7f5d6b48e740 1 -- 172.21.15.62:0/1694943896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5803d450 con 0x55b40fab6680 2024-09-11T23:56:02.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.336+0000 7f5d46ff5640 1 -- 172.21.15.62:0/1694943896 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d5c0d1260 con 0x55b40fab6680 2024-09-11T23:56:02.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.336+0000 7f5d6b48e740 10 client.25672 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:02.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.336+0000 7f5d6b48e740 1 -- 172.21.15.62:0/1694943896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b40fac4ce0 con 0x55b40fab6680 2024-09-11T23:56:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.336+0000 7f5d46ff5640 1 -- 172.21.15.62:0/1694943896 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5803d450 con 0x55b40fab6680 2024-09-11T23:56:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.337+0000 7f5d46ff5640 1 -- 172.21.15.62:0/1694943896 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f5d5c00f040 con 0x55b40fab6680 2024-09-11T23:56:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.338+0000 7f5d6b48e740 10 client.25672 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.338+0000 7f5d6b48e740 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.338+0000 7f5d6b48e740 20 client.25672 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.338+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b40fac61a0 0x55b40fac8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.338+0000 7f5d6b48e740 4 client.25672 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:02.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.338+0000 7f5d6b48e740 1 -- 172.21.15.62:0/1694943896 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b40fa1e850 con 0x55b40fac61a0 2024-09-11T23:56:02.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.338+0000 7f5d62d9d640 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b40fac61a0 0x55b40fac8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.339+0000 7f5d62d9d640 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b40fac61a0 0x55b40fac8590 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=859bebff68d8094d server_cookie=5b464dab5c129a4c in_seq=0 out_seq=0 2024-09-11T23:56:02.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.339+0000 7f5d46ff5640 10 client.25672 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:02.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.340+0000 7f5d46ff5640 1 -- 172.21.15.62:0/1694943896 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b40fa1e850 con 0x55b40fac61a0 2024-09-11T23:56:02.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.340+0000 7f5d46ff5640 10 client.25672 handle_command_reply: tid=0 2024-09-11T23:56:02.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.340+0000 7f5d6b48e740 1 client.25672 shutdown 2024-09-11T23:56:02.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.341+0000 7f5d6b48e740 1 -- 172.21.15.62:0/1694943896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 msgr2=0x55b40fab3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.341+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab3830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c09c780 tx=0x7f5d5c0a05f0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.341+0000 7f5d6b48e740 1 -- 172.21.15.62:0/1694943896 shutdown_connections 2024-09-11T23:56:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.341+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b40fac61a0 0x55b40fac8590 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.342+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab3830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.342+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faabdc0 0x55b40fab32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.342+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/1694943896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faab4a0 0x55b40fab6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.342+0000 7f5d6b48e740 1 -- 172.21.15.62:0/1694943896 >> 172.21.15.62:0/1694943896 conn(0x55b40faa7a30 msgr2=0x55b40faad760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.342+0000 7f5d46ff5640 0 client.25672 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.342+0000 7f5d6b48e740 1 -- 172.21.15.62:0/1694943896 shutdown_connections 2024-09-11T23:56:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.342+0000 7f5d6b48e740 1 -- 172.21.15.62:0/1694943896 wait complete. 2024-09-11T23:56:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.342+0000 7f5d6b48e740 20 client.25672 trim_cache size 0 max 16384 2024-09-11T23:56:02.353 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:02.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d6b48e740 1 Processor -- start 2024-09-11T23:56:02.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d6b48e740 1 -- start start 2024-09-11T23:56:02.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40facde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 0x55b40face3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d6b48e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641adf80 con 0x55b40faab4a0 2024-09-11T23:56:02.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d6b48e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641ab850 con 0x55b40faabdc0 2024-09-11T23:56:02.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d6b48e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641a96a0 con 0x55b40fab6680 2024-09-11T23:56:02.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d62d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40facde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d62d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40facde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43648/0 (socket says 172.21.15.62:43648) 2024-09-11T23:56:02.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d62d9d640 1 -- 172.21.15.62:0/2375015891 learned_addr learned my addr 172.21.15.62:0/2375015891 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:02.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.374+0000 7f5d6259c640 1 --2- 172.21.15.62:0/2375015891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 0x55b40face3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.375+0000 7f5d6359e640 1 --2- 172.21.15.62:0/2375015891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.375+0000 7f5d6259c640 1 -- 172.21.15.62:0/2375015891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 msgr2=0x55b40fab0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:02.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.375+0000 7f5d6259c640 1 --2- 172.21.15.62:0/2375015891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.375+0000 7f5d6259c640 1 -- 172.21.15.62:0/2375015891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 msgr2=0x55b40facde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:02.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.375+0000 7f5d6259c640 1 --2- 172.21.15.62:0/2375015891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40facde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.375+0000 7f5d6259c640 1 -- 172.21.15.62:0/2375015891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b40fac8900 con 0x55b40faabdc0 2024-09-11T23:56:02.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.375+0000 7f5d6259c640 1 --2- 172.21.15.62:0/2375015891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 0x55b40face3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55b40fab48f0 tx=0x7f5d58009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6746782269da13a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:02.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.376+0000 7f5d48ff9640 1 -- 172.21.15.62:0/2375015891 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58035040 con 0x55b40faabdc0 2024-09-11T23:56:02.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.376+0000 7f5d48ff9640 1 -- 172.21.15.62:0/2375015891 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5803d450 con 0x55b40faabdc0 2024-09-11T23:56:02.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.376+0000 7f5d48ff9640 1 -- 172.21.15.62:0/2375015891 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5804c040 con 0x55b40faabdc0 2024-09-11T23:56:02.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.378+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2375015891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 msgr2=0x55b40face3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:02.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.378+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/2375015891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 0x55b40face3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55b40fab48f0 tx=0x7f5d58009780 comp rx=0 tx=0).stop 2024-09-11T23:56:02.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.378+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2375015891 shutdown_connections 2024-09-11T23:56:02.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.378+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/2375015891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40fab6680 0x55b40fab0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.378+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/2375015891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40faabdc0 0x55b40face3c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.378+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/2375015891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40facde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.378+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2375015891 >> 172.21.15.62:0/2375015891 conn(0x55b40faa7a30 msgr2=0x55b40fab3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:02.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.378+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2375015891 shutdown_connections 2024-09-11T23:56:02.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.379+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2375015891 wait complete. 2024-09-11T23:56:02.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.379+0000 7f5d6b48e740 1 -- start start 2024-09-11T23:56:02.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.380+0000 7f5d6b48e740 1 Processor -- start 2024-09-11T23:56:02.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.380+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40fabcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.380+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40faabdc0 0x55b40fabd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.381+0000 7f5d6b48e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40fab6680 0x55b40fb29ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.381+0000 7f5d6b48e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641a9d10 con 0x55b40faab4a0 2024-09-11T23:56:02.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.381+0000 7f5d6259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40faabdc0 0x55b40fabd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.381+0000 7f5d6259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40faabdc0 0x55b40fabd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52182/0 (socket says 172.21.15.62:52182) 2024-09-11T23:56:02.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.381+0000 7f5d6259c640 1 -- 172.21.15.62:0/2203679588 learned_addr learned my addr 172.21.15.62:0/2203679588 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:02.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.381+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2203679588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641ab180 con 0x55b40fab6680 2024-09-11T23:56:02.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.381+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2203679588 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641aa3e0 con 0x55b40faabdc0 2024-09-11T23:56:02.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.381+0000 7f5d6b48e740 10 client.0 _ll_register_callbacks cb 0x7f5d5c09c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:02.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.381+0000 7f5d6359e640 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40fab6680 0x55b40fb29ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.382+0000 7f5d62d9d640 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40fabcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.383+0000 7f5d62d9d640 1 -- 172.21.15.62:0/2203679588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40faabdc0 msgr2=0x55b40fabd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:02.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.383+0000 7f5d62d9d640 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40faabdc0 0x55b40fabd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.383+0000 7f5d62d9d640 1 -- 172.21.15.62:0/2203679588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40fab6680 msgr2=0x55b40fb29ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:02.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.383+0000 7f5d62d9d640 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40fab6680 0x55b40fb29ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.383+0000 7f5d62d9d640 1 -- 172.21.15.62:0/2203679588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40fb2a4e0 con 0x55b40faab4a0 2024-09-11T23:56:02.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.383+0000 7f5d62d9d640 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40fabcd00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5d540045d0 tx=0x7f5d5400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9045540cae0f10a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:02.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.383+0000 7f5d46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:02.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.383+0000 7f5d46ff5640 1 -- 172.21.15.62:0/2203679588 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54005020 con 0x55b40faab4a0 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.383+0000 7f5d46ff5640 1 -- 172.21.15.62:0/2203679588 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f5d540051c0 con 0x55b40faab4a0 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.383+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2203679588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5803d450 con 0x55b40faab4a0 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d46ff5640 1 -- 172.21.15.62:0/2203679588 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d540053c0 con 0x55b40faab4a0 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d6b48e740 10 client.16080 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2203679588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b40fb31900 con 0x55b40faab4a0 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d46ff5640 1 -- 172.21.15.62:0/2203679588 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5803d450 con 0x55b40faab4a0 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d46ff5640 1 -- 172.21.15.62:0/2203679588 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f5d5401d620 con 0x55b40faab4a0 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d6b48e740 10 client.16080 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d6b48e740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d6b48e740 20 client.16080 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b40fb320b0 0x55b40fb344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d6b48e740 4 client.16080 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:56:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.384+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2203679588 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b40fa18d60 con 0x55b40fb320b0 2024-09-11T23:56:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.385+0000 7f5d6259c640 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b40fb320b0 0x55b40fb344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.385+0000 7f5d6259c640 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b40fb320b0 0x55b40fb344d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3b0bb97c1177563 server_cookie=f6c69e10f850332e in_seq=0 out_seq=0 2024-09-11T23:56:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.385+0000 7f5d46ff5640 10 client.16080 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.385+0000 7f5d46ff5640 1 -- 172.21.15.62:0/2203679588 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b40fa18d60 con 0x55b40fb320b0 2024-09-11T23:56:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.386+0000 7f5d46ff5640 10 client.16080 handle_command_reply: tid=0 2024-09-11T23:56:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.386+0000 7f5d6b48e740 1 client.16080 shutdown 2024-09-11T23:56:02.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2203679588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 msgr2=0x55b40fabcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40fabcd00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5d540045d0 tx=0x7f5d5400be10 comp rx=0 tx=0).stop 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2203679588 shutdown_connections 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b40fb320b0 0x55b40fb344d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d46ff5640 0 client.16080 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b40fab6680 0x55b40fb29ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b40faabdc0 0x55b40fabd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 1 --2- 172.21.15.62:0/2203679588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b40faab4a0 0x55b40fabcd00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2203679588 >> 172.21.15.62:0/2203679588 conn(0x55b40faa7a30 msgr2=0x55b40faa6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2203679588 shutdown_connections 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 1 -- 172.21.15.62:0/2203679588 wait complete. 2024-09-11T23:56:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.389+0000 7f5d6b48e740 20 client.16080 trim_cache size 0 max 16384 2024-09-11T23:56:02.391 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:56:02.391 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:56:02.391 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:56:02.391 INFO:teuthology.orchestra.run.smithi062.stdout: "9b74483c-65e6-4c1c-9d91-ed0133003974": { 2024-09-11T23:56:02.391 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:56:02.391 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "9b74483c-65e6-4c1c-9d91-ed0133003974", 2024-09-11T23:56:02.391 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:56:02.391 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:02.391 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:02.391 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:56:02.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.390+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5d4c0a2ec0 msgr2=0x7f5d4c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:02.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.390+0000 7f5d690b5640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5d4c0a2ec0 0x7f5d4c0a5380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d58000c00 tx=0x7f5d58040000 comp rx=0 tx=0).stop 2024-09-11T23:56:02.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.390+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d64106a90 msgr2=0x7f5d6419bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:02.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.390+0000 7f5d690b5640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d64106a90 0x7f5d6419bc30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00e940 tx=0x7f5d5c00ee00 comp rx=0 tx=0).stop 2024-09-11T23:56:02.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.391+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 shutdown_connections 2024-09-11T23:56:02.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.391+0000 7f5d690b5640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5d4c0a2ec0 0x7f5d4c0a5380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.391+0000 7f5d690b5640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d64106a90 0x7f5d6419bc30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.391+0000 7f5d690b5640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d64101430 0x7f5d641978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.391+0000 7f5d690b5640 1 --2- 172.21.15.62:0/1048662003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d64100a90 0x7f5d64197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:02.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.391+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 >> 172.21.15.62:0/1048662003 conn(0x7f5d640fc7d0 msgr2=0x7f5d640fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:02.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.392+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 shutdown_connections 2024-09-11T23:56:02.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:02.392+0000 7f5d690b5640 1 -- 172.21.15.62:0/1048662003 wait complete. 2024-09-11T23:56:02.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 200.0 s, 2296.4 tps, lat 43.703 ms stddev 66.123 2024-09-11T23:56:02.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:02 smithi062 ceph-mon[30958]: pgmap v121: 97 pgs: 97 active+clean; 2.4 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 1.51k op/s 2024-09-11T23:56:02.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:02 smithi120 ceph-mon[35182]: pgmap v121: 97 pgs: 97 active+clean; 2.4 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 1.51k op/s 2024-09-11T23:56:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:02 smithi083 ceph-mon[34805]: pgmap v121: 97 pgs: 97 active+clean; 2.4 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 1.51k op/s 2024-09-11T23:56:03.591 INFO:tasks.cephfs.filesystem:scrub status for tag:9b74483c-65e6-4c1c-9d91-ed0133003974 - {'path': '/', 'tag': '9b74483c-65e6-4c1c-9d91-ed0133003974', 'options': 'recursive,force'} 2024-09-11T23:56:03.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.776+0000 7f1614dad640 1 -- 172.21.15.62:0/4045445473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f161010c2f0 msgr2=0x7f161010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.776+0000 7f1614dad640 1 --2- 172.21.15.62:0/4045445473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f161010c2f0 0x7f161010e790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1608009ca0 tx=0x7f160802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:56:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.776+0000 7f1614dad640 1 -- 172.21.15.62:0/4045445473 shutdown_connections 2024-09-11T23:56:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.776+0000 7f1614dad640 1 --2- 172.21.15.62:0/4045445473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f161010c2f0 0x7f161010e790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.776+0000 7f1614dad640 1 --2- 172.21.15.62:0/4045445473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16101037a0 0x7f161010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.776+0000 7f1614dad640 1 --2- 172.21.15.62:0/4045445473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1610102df0 0x7f16101031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.776+0000 7f1614dad640 1 -- 172.21.15.62:0/4045445473 >> 172.21.15.62:0/4045445473 conn(0x7f16100fc7d0 msgr2=0x7f16100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.776+0000 7f1614dad640 1 -- 172.21.15.62:0/4045445473 shutdown_connections 2024-09-11T23:56:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.776+0000 7f1614dad640 1 -- 172.21.15.62:0/4045445473 wait complete. 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.777+0000 7f1614dad640 1 Processor -- start 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.777+0000 7f1614dad640 1 -- start start 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f1614dad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1610102df0 0x7f1610198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f1614dad640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16101037a0 0x7f1610198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f1614dad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161010c2f0 0x7f161019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f1614dad640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f16101121a0 con 0x7f16101037a0 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f1614dad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1610112020 con 0x7f161010c2f0 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f1614dad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1610112320 con 0x7f1610102df0 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f160e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1610102df0 0x7f1610198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f160e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1610102df0 0x7f1610198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52210/0 (socket says 172.21.15.62:52210) 2024-09-11T23:56:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f160e59c640 1 -- 172.21.15.62:0/3681911741 learned_addr learned my addr 172.21.15.62:0/3681911741 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f160ed9d640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161010c2f0 0x7f161019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f160dd9b640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16101037a0 0x7f1610198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f160e59c640 1 -- 172.21.15.62:0/3681911741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161010c2f0 msgr2=0x7f161019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f160e59c640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161010c2f0 0x7f161019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f160e59c640 1 -- 172.21.15.62:0/3681911741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16101037a0 msgr2=0x7f1610198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f160e59c640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16101037a0 0x7f1610198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.778+0000 7f160e59c640 1 -- 172.21.15.62:0/3681911741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161019d5d0 con 0x7f1610102df0 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.779+0000 7f160e59c640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1610102df0 0x7f1610198620 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f160000ea60 tx=0x7f160000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.779+0000 7f15ff7fe640 1 -- 172.21.15.62:0/3681911741 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160000cde0 con 0x7f1610102df0 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.779+0000 7f15ff7fe640 1 -- 172.21.15.62:0/3681911741 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16000035f0 con 0x7f1610102df0 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.779+0000 7f15ff7fe640 1 -- 172.21.15.62:0/3681911741 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600010670 con 0x7f1610102df0 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.779+0000 7f1614dad640 1 -- 172.21.15.62:0/3681911741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161019d8c0 con 0x7f1610102df0 2024-09-11T23:56:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.779+0000 7f1614dad640 1 -- 172.21.15.62:0/3681911741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1610100980 con 0x7f1610102df0 2024-09-11T23:56:03.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.780+0000 7f1614dad640 1 -- 172.21.15.62:0/3681911741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161006a790 con 0x7f1610102df0 2024-09-11T23:56:03.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.782+0000 7f15ff7fe640 1 -- 172.21.15.62:0/3681911741 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1600004020 con 0x7f1610102df0 2024-09-11T23:56:03.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.783+0000 7f15ff7fe640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15f80a2cf0 0x7f15f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:03.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.783+0000 7f15ff7fe640 1 -- 172.21.15.62:0/3681911741 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f16000cf8b0 con 0x7f1610102df0 2024-09-11T23:56:03.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.783+0000 7f160dd9b640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15f80a2cf0 0x7f15f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:03.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.784+0000 7f160dd9b640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15f80a2cf0 0x7f15f80a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f161010d610 tx=0x7f1604006ec0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:03.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:03.788+0000 7f15ff7fe640 1 -- 172.21.15.62:0/3681911741 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f160009c500 con 0x7f1610102df0 2024-09-11T23:56:04.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.138+0000 7f1614dad640 1 -- 172.21.15.62:0/3681911741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16101999b0 con 0x7f1610102df0 2024-09-11T23:56:04.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.139+0000 7f15ff7fe640 1 -- 172.21.15.62:0/3681911741 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7f16101999b0 con 0x7f1610102df0 2024-09-11T23:56:04.139 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:56:04.139 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"btime":"2024-09-11T23:55:51:325581+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:55:50.698407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:56:04.139 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-09-11T23:56:04.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.141+0000 7f1614dad640 1 -- 172.21.15.62:0/3681911741 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15f80a2cf0 msgr2=0x7f15f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:04.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.141+0000 7f1614dad640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15f80a2cf0 0x7f15f80a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f161010d610 tx=0x7f1604006ec0 comp rx=0 tx=0).stop 2024-09-11T23:56:04.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.141+0000 7f1614dad640 1 -- 172.21.15.62:0/3681911741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1610102df0 msgr2=0x7f1610198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:04.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.141+0000 7f1614dad640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1610102df0 0x7f1610198620 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f160000ea60 tx=0x7f160000ef20 comp rx=0 tx=0).stop 2024-09-11T23:56:04.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.142+0000 7f1614dad640 1 -- 172.21.15.62:0/3681911741 shutdown_connections 2024-09-11T23:56:04.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.142+0000 7f1614dad640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15f80a2cf0 0x7f15f80a51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:04.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.142+0000 7f1614dad640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161010c2f0 0x7f161019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:04.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.142+0000 7f1614dad640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16101037a0 0x7f1610198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:04.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.142+0000 7f1614dad640 1 --2- 172.21.15.62:0/3681911741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1610102df0 0x7f1610198620 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:04.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.142+0000 7f1614dad640 1 -- 172.21.15.62:0/3681911741 >> 172.21.15.62:0/3681911741 conn(0x7f16100fc7d0 msgr2=0x7f16100fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:04.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.142+0000 7f1614dad640 1 -- 172.21.15.62:0/3681911741 shutdown_connections 2024-09-11T23:56:04.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:04.142+0000 7f1614dad640 1 -- 172.21.15.62:0/3681911741 wait complete. 2024-09-11T23:56:04.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:04 smithi120 ceph-mon[35182]: pgmap v122: 97 pgs: 97 active+clean; 2.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.95k op/s 2024-09-11T23:56:04.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:04 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3681911741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:04.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:04 smithi083 ceph-mon[34805]: pgmap v122: 97 pgs: 97 active+clean; 2.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.95k op/s 2024-09-11T23:56:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:04 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3681911741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:04.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:04 smithi062 ceph-mon[30958]: pgmap v122: 97 pgs: 97 active+clean; 2.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.95k op/s 2024-09-11T23:56:04.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:04 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3681911741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:06.231 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T23:56:06.231 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T23:56:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:06.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:06 smithi062 ceph-mon[30958]: pgmap v123: 97 pgs: 97 active+clean; 2.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.95k op/s 2024-09-11T23:56:06.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:06 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:56:06.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:06 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:56:06.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:06 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:56:06.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:06 smithi120 ceph-mon[35182]: pgmap v123: 97 pgs: 97 active+clean; 2.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.95k op/s 2024-09-11T23:56:06.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:06 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:56:06.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:06 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:56:06.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:06 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:56:06.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:06 smithi083 ceph-mon[34805]: pgmap v123: 97 pgs: 97 active+clean; 2.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.95k op/s 2024-09-11T23:56:06.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:06 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:56:06.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:06 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:56:06.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:06 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:56:07.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.151+0000 7fbc42e5d640 1 -- 172.21.15.62:0/2192400729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc3c106f10 msgr2=0x7fbc3c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:07.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.151+0000 7fbc42e5d640 1 --2- 172.21.15.62:0/2192400729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc3c106f10 0x7fbc3c10ea90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009ca0 tx=0x7fbc3402a8d0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.151+0000 7fbc42e5d640 1 -- 172.21.15.62:0/2192400729 shutdown_connections 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.151+0000 7fbc42e5d640 1 --2- 172.21.15.62:0/2192400729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc3c106f10 0x7fbc3c10ea90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.151+0000 7fbc42e5d640 1 --2- 172.21.15.62:0/2192400729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc3c102790 0x7fbc3c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.151+0000 7fbc42e5d640 1 --2- 172.21.15.62:0/2192400729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3c101de0 0x7fbc3c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.151+0000 7fbc42e5d640 1 -- 172.21.15.62:0/2192400729 >> 172.21.15.62:0/2192400729 conn(0x7fbc3c0fd880 msgr2=0x7fbc3c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.151+0000 7fbc42e5d640 1 -- 172.21.15.62:0/2192400729 shutdown_connections 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.151+0000 7fbc42e5d640 1 -- 172.21.15.62:0/2192400729 wait complete. 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.152+0000 7fbc42e5d640 1 Processor -- start 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.152+0000 7fbc42e5d640 1 -- start start 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.152+0000 7fbc42e5d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc3c101de0 0x7fbc3c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.152+0000 7fbc42e5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3c102790 0x7fbc3c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.152+0000 7fbc42e5d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc3c106f10 0x7fbc3c10c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.152+0000 7fbc42e5d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c111f70 con 0x7fbc3c101de0 2024-09-11T23:56:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.152+0000 7fbc42e5d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c111df0 con 0x7fbc3c102790 2024-09-11T23:56:07.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc42e5d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1120f0 con 0x7fbc3c106f10 2024-09-11T23:56:07.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc413fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc3c106f10 0x7fbc3c10c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3c102790 0x7fbc3c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3c102790 0x7fbc3c19f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57834/0 (socket says 172.21.15.62:57834) 2024-09-11T23:56:07.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1895349528 learned_addr learned my addr 172.21.15.62:0/1895349528 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:07.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc413fa640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc3c106f10 0x7fbc3c10c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52226/0 (socket says 172.21.15.62:52226) 2024-09-11T23:56:07.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc40bf9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc3c101de0 0x7fbc3c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1895349528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc3c106f10 msgr2=0x7fbc3c10c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc3c106f10 0x7fbc3c10c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1895349528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc3c101de0 msgr2=0x7fbc3c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc3c101de0 0x7fbc3c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1895349528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c10c9c0 con 0x7fbc3c102790 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc413fa640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc3c106f10 0x7fbc3c10c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.153+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3c102790 0x7fbc3c19f5d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800d950 tx=0x7fbc2800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.154+0000 7fbc39ffb640 1 -- 172.21.15.62:0/1895349528 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28017030 con 0x7fbc3c102790 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.154+0000 7fbc39ffb640 1 -- 172.21.15.62:0/1895349528 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc28004020 con 0x7fbc3c102790 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.154+0000 7fbc39ffb640 1 -- 172.21.15.62:0/1895349528 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc280047a0 con 0x7fbc3c102790 2024-09-11T23:56:07.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.154+0000 7fbc42e5d640 1 -- 172.21.15.62:0/1895349528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c10ccb0 con 0x7fbc3c102790 2024-09-11T23:56:07.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.154+0000 7fbc42e5d640 1 -- 172.21.15.62:0/1895349528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c19ad10 con 0x7fbc3c102790 2024-09-11T23:56:07.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.159+0000 7fbc39ffb640 1 -- 172.21.15.62:0/1895349528 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbc280041d0 con 0x7fbc3c102790 2024-09-11T23:56:07.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.159+0000 7fbc45236740 1 Processor -- start 2024-09-11T23:56:07.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.159+0000 7fbc45236740 1 -- start start 2024-09-11T23:56:07.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.159+0000 7fbc45236740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230275c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.159+0000 7fbc45236740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x5642302804f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.159+0000 7fbc45236740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 0x564230282e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.159+0000 7fbc45236740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5642301da160 con 0x564230275850 2024-09-11T23:56:07.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.159+0000 7fbc45236740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564230181720 con 0x564230280a30 2024-09-11T23:56:07.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.159+0000 7fbc45236740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642301d6120 con 0x564230276170 2024-09-11T23:56:07.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.159+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 0x564230282e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.160+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 0x564230282e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57836/0 (socket says 172.21.15.62:57836) 2024-09-11T23:56:07.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.160+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1731740597 learned_addr learned my addr 172.21.15.62:0/1731740597 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:07.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.160+0000 7fbc413fa640 1 --2- 172.21.15.62:0/1731740597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230275c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.160+0000 7fbc40bf9640 1 --2- 172.21.15.62:0/1731740597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x5642302804f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.160+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1731740597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 msgr2=0x5642302804f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:07.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.160+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/1731740597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x5642302804f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.160+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1731740597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 msgr2=0x564230275c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:07.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.160+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/1731740597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230275c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.160+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1731740597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56422ff4bba0 con 0x564230280a30 2024-09-11T23:56:07.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.160+0000 7fbc413fa640 1 --2- 172.21.15.62:0/1731740597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230275c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:07.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.161+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/1731740597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 0x564230282e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbc280cc2b0 tx=0x7fbc280d7710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f05fabac708ef00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:07.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.161+0000 7fbc26ffd640 1 -- 172.21.15.62:0/1731740597 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc280d9620 con 0x564230280a30 2024-09-11T23:56:07.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.161+0000 7fbc39ffb640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc2c0a2e40 0x7fbc2c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.161+0000 7fbc413fa640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc2c0a2e40 0x7fbc2c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.161+0000 7fbc39ffb640 1 -- 172.21.15.62:0/1895349528 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fbc280cec50 con 0x7fbc3c102790 2024-09-11T23:56:07.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.162+0000 7fbc26ffd640 1 -- 172.21.15.62:0/1731740597 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2809b840 con 0x564230280a30 2024-09-11T23:56:07.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.162+0000 7fbc26ffd640 1 -- 172.21.15.62:0/1731740597 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc280973e0 con 0x564230280a30 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.163+0000 7fbc45236740 1 -- 172.21.15.62:0/1731740597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 msgr2=0x564230282e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.163+0000 7fbc413fa640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc2c0a2e40 0x7fbc2c0a5300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbc340308d0 tx=0x7fbc34005ef0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.163+0000 7fbc45236740 1 --2- 172.21.15.62:0/1731740597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 0x564230282e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbc280cc2b0 tx=0x7fbc280d7710 comp rx=0 tx=0).stop 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.163+0000 7fbc45236740 1 -- 172.21.15.62:0/1731740597 shutdown_connections 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.163+0000 7fbc45236740 1 --2- 172.21.15.62:0/1731740597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 0x564230282e20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.163+0000 7fbc45236740 1 --2- 172.21.15.62:0/1731740597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x5642302804f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.163+0000 7fbc45236740 1 --2- 172.21.15.62:0/1731740597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230275c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.163+0000 7fbc45236740 1 -- 172.21.15.62:0/1731740597 >> 172.21.15.62:0/1731740597 conn(0x564230271de0 msgr2=0x5642302721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.163+0000 7fbc45236740 1 -- 172.21.15.62:0/1731740597 shutdown_connections 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.163+0000 7fbc45236740 1 -- 172.21.15.62:0/1731740597 wait complete. 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.164+0000 7fbc45236740 1 -- start start 2024-09-11T23:56:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.164+0000 7fbc45236740 1 Processor -- start 2024-09-11T23:56:07.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.164+0000 7fbc45236740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230275850 0x56423027d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.165+0000 7fbc45236740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230276170 0x56423027de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.165+0000 7fbc45236740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230280a30 0x56423027e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.165+0000 7fbc45236740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564230283600 con 0x564230280a30 2024-09-11T23:56:07.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.165+0000 7fbc45236740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564230162900 con 0x564230276170 2024-09-11T23:56:07.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.165+0000 7fbc45236740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564230283780 con 0x564230275850 2024-09-11T23:56:07.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc45236740 10 client.0 _ll_register_callbacks cb 0x564230160780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230276170 0x56423027de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc413fa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230280a30 0x56423027e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230276170 0x56423027de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57838/0 (socket says 172.21.15.62:57838) 2024-09-11T23:56:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc3bfff640 1 -- 172.21.15.62:0/135808672 learned_addr learned my addr 172.21.15.62:0/135808672 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc40bf9640 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230275850 0x56423027d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc3bfff640 1 -- 172.21.15.62:0/135808672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230275850 msgr2=0x56423027d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230275850 0x56423027d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc3bfff640 1 -- 172.21.15.62:0/135808672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230280a30 msgr2=0x56423027e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230280a30 0x56423027e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.166+0000 7fbc3bfff640 1 -- 172.21.15.62:0/135808672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642301a2c50 con 0x564230276170 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.167+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230276170 0x56423027de40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005fd0 tx=0x7fbc28004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5264a9b6d3f30f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.167+0000 7fbc1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.167+0000 7fbc1bfff640 1 -- 172.21.15.62:0/135808672 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2809aaa0 con 0x564230276170 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.167+0000 7fbc1bfff640 1 -- 172.21.15.62:0/135808672 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fbc2809f170 con 0x564230276170 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.167+0000 7fbc45236740 1 -- 172.21.15.62:0/135808672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2809b840 con 0x564230276170 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.167+0000 7fbc1bfff640 1 -- 172.21.15.62:0/135808672 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc2809ac80 con 0x564230276170 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.167+0000 7fbc45236740 10 client.25766 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.167+0000 7fbc45236740 1 -- 172.21.15.62:0/135808672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56423028f2c0 con 0x564230276170 2024-09-11T23:56:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.167+0000 7fbc1bfff640 1 -- 172.21.15.62:0/135808672 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2809b840 con 0x564230276170 2024-09-11T23:56:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.168+0000 7fbc1bfff640 1 -- 172.21.15.62:0/135808672 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fbc280977b0 con 0x564230276170 2024-09-11T23:56:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.168+0000 7fbc45236740 10 client.25766 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.168+0000 7fbc45236740 10 client.25766 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:56:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.168+0000 7fbc45236740 20 client.25766 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.168+0000 7fbc45236740 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564230290610 0x564230292a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.168+0000 7fbc45236740 4 client.25766 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.168+0000 7fbc45236740 1 -- 172.21.15.62:0/135808672 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642301e7890 con 0x564230290610 2024-09-11T23:56:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.168+0000 7fbc40bf9640 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564230290610 0x564230292a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.168+0000 7fbc40bf9640 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564230290610 0x564230292a00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4af4c2b2edaaca server_cookie=d1fb2d783ae47e8 in_seq=0 out_seq=0 2024-09-11T23:56:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.169+0000 7fbc1bfff640 10 client.25766 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.169+0000 7fbc1bfff640 1 -- 172.21.15.62:0/135808672 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642301e7890 con 0x564230290610 2024-09-11T23:56:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.169+0000 7fbc1bfff640 10 client.25766 handle_command_reply: tid=0 2024-09-11T23:56:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.169+0000 7fbc45236740 1 client.25766 shutdown 2024-09-11T23:56:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.169+0000 7fbc45236740 1 -- 172.21.15.62:0/135808672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230276170 msgr2=0x56423027de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.169+0000 7fbc45236740 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230276170 0x56423027de40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005fd0 tx=0x7fbc28004540 comp rx=0 tx=0).stop 2024-09-11T23:56:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.170+0000 7fbc45236740 1 -- 172.21.15.62:0/135808672 shutdown_connections 2024-09-11T23:56:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.170+0000 7fbc45236740 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564230290610 0x564230292a00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.170+0000 7fbc45236740 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230280a30 0x56423027e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.170+0000 7fbc45236740 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230276170 0x56423027de40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.170+0000 7fbc45236740 1 --2- 172.21.15.62:0/135808672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230275850 0x56423027d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.170+0000 7fbc45236740 1 -- 172.21.15.62:0/135808672 >> 172.21.15.62:0/135808672 conn(0x564230271de0 msgr2=0x564230277b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.170+0000 7fbc1bfff640 0 client.25766 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.170+0000 7fbc45236740 1 -- 172.21.15.62:0/135808672 shutdown_connections 2024-09-11T23:56:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.170+0000 7fbc45236740 1 -- 172.21.15.62:0/135808672 wait complete. 2024-09-11T23:56:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.170+0000 7fbc45236740 20 client.25766 trim_cache size 0 max 16384 2024-09-11T23:56:07.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.195+0000 7fbc45236740 1 Processor -- start 2024-09-11T23:56:07.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.196+0000 7fbc45236740 1 -- start start 2024-09-11T23:56:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc45236740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230270d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc45236740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x56423027e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc45236740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 0x564230295640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc45236740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1beeb0 con 0x564230275850 2024-09-11T23:56:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc45236740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1bc780 con 0x564230280a30 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc45236740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1ba5d0 con 0x564230276170 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc40bf9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230270d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x56423027e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc40bf9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230270d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43736/0 (socket says 172.21.15.62:43736) 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc40bf9640 1 -- 172.21.15.62:0/1305314159 learned_addr learned my addr 172.21.15.62:0/1305314159 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc413fa640 1 --2- 172.21.15.62:0/1305314159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 0x564230295640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1305314159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 msgr2=0x564230295640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/1305314159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 0x564230295640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1305314159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 msgr2=0x564230270d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/1305314159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230270d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc3bfff640 1 -- 172.21.15.62:0/1305314159 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564230271c10 con 0x564230276170 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.197+0000 7fbc40bf9640 1 --2- 172.21.15.62:0/1305314159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230270d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:56:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.198+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/1305314159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x56423027e1d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009fd0 tx=0x7fbc280d7710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f8c91318ceff710 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.198+0000 7fbc267fc640 1 -- 172.21.15.62:0/1305314159 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2809af10 con 0x564230276170 2024-09-11T23:56:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.198+0000 7fbc267fc640 1 -- 172.21.15.62:0/1305314159 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2809b840 con 0x564230276170 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.198+0000 7fbc267fc640 1 -- 172.21.15.62:0/1305314159 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc280d24d0 con 0x564230276170 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 -- 172.21.15.62:0/1305314159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 msgr2=0x56423027e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 --2- 172.21.15.62:0/1305314159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x56423027e1d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009fd0 tx=0x7fbc280d7710 comp rx=0 tx=0).stop 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 -- 172.21.15.62:0/1305314159 shutdown_connections 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 --2- 172.21.15.62:0/1305314159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230280a30 0x564230295640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 --2- 172.21.15.62:0/1305314159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x56423027e1d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 --2- 172.21.15.62:0/1305314159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230275850 0x564230270d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 -- 172.21.15.62:0/1305314159 >> 172.21.15.62:0/1305314159 conn(0x564230271de0 msgr2=0x56423028c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 -- 172.21.15.62:0/1305314159 shutdown_connections 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 -- 172.21.15.62:0/1305314159 wait complete. 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 -- start start 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 Processor -- start 2024-09-11T23:56:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230275850 0x564230295ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.199+0000 7fbc45236740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x564230292ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc45236740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230280a30 0x564230293530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc45236740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1bac40 con 0x564230280a30 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc45236740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1bc0b0 con 0x564230275850 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc45236740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1bb310 con 0x564230276170 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc40bf9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230275850 0x564230295ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x564230292ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x564230292ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52258/0 (socket says 172.21.15.62:52258) 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc40bf9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230275850 0x564230295ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57854/0 (socket says 172.21.15.62:57854) 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc3bfff640 1 -- 172.21.15.62:0/211439782 learned_addr learned my addr 172.21.15.62:0/211439782 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc45236740 10 client.0 _ll_register_callbacks cb 0x7fbc3000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc413fa640 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230280a30 0x564230293530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc3bfff640 1 -- 172.21.15.62:0/211439782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230275850 msgr2=0x564230295ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230275850 0x564230295ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc3bfff640 1 -- 172.21.15.62:0/211439782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230280a30 msgr2=0x564230293530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230280a30 0x564230293530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc3bfff640 1 -- 172.21.15.62:0/211439782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642302786c0 con 0x564230276170 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc3bfff640 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x564230292ff0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbc2809b0d0 tx=0x7fbc280ceed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf45011cf9bcc49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc1bfff640 1 -- 172.21.15.62:0/211439782 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2809af10 con 0x564230276170 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc1bfff640 1 -- 172.21.15.62:0/211439782 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fbc280977b0 con 0x564230276170 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.200+0000 7fbc45236740 1 -- 172.21.15.62:0/211439782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2809b840 con 0x564230276170 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc1bfff640 1 -- 172.21.15.62:0/211439782 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc280a3e00 con 0x564230276170 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc45236740 10 client.25702 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc45236740 1 -- 172.21.15.62:0/211439782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56423026d3f0 con 0x564230276170 2024-09-11T23:56:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc1bfff640 1 -- 172.21.15.62:0/211439782 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc280a8050 con 0x564230276170 2024-09-11T23:56:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc1bfff640 1 -- 172.21.15.62:0/211439782 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fbc280cc5f0 con 0x564230276170 2024-09-11T23:56:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc45236740 10 client.25702 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc45236740 10 client.25702 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:56:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc45236740 20 client.25702 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc45236740 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5642302fadd0 0x5642302fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc45236740 4 client.25702 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T23:56:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc45236740 1 -- 172.21.15.62:0/211439782 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x7fbc2809b840 con 0x5642302fadd0 2024-09-11T23:56:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.201+0000 7fbc40bf9640 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5642302fadd0 0x5642302fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.202+0000 7fbc40bf9640 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5642302fadd0 0x5642302fd1c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a76195425198a3f5 server_cookie=e5a5443df3b7808 in_seq=0 out_seq=0 2024-09-11T23:56:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.202+0000 7fbc1bfff640 10 client.25702 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 205.0 s, 3339.1 tps, lat 28.602 ms stddev 29.909 2024-09-11T23:56:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc1bfff640 1 -- 172.21.15.62:0/211439782 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fbc2809b840 con 0x5642302fadd0 2024-09-11T23:56:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc1bfff640 10 client.25702 handle_command_reply: tid=0 2024-09-11T23:56:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 client.25702 shutdown 2024-09-11T23:56:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 -- 172.21.15.62:0/211439782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 msgr2=0x564230292ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x564230292ff0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbc2809b0d0 tx=0x7fbc280ceed0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 -- 172.21.15.62:0/211439782 shutdown_connections 2024-09-11T23:56:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5642302fadd0 0x5642302fd1c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564230280a30 0x564230293530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564230276170 0x564230292ff0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc1bfff640 0 client.25702 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:07.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 --2- 172.21.15.62:0/211439782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564230275850 0x564230295ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 -- 172.21.15.62:0/211439782 >> 172.21.15.62:0/211439782 conn(0x564230271de0 msgr2=0x56423027c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:07.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 -- 172.21.15.62:0/211439782 shutdown_connections 2024-09-11T23:56:07.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 1 -- 172.21.15.62:0/211439782 wait complete. 2024-09-11T23:56:07.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.481+0000 7fbc45236740 20 client.25702 trim_cache size 0 max 16384 2024-09-11T23:56:07.485 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:56:07.485 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 29, 2024-09-11T23:56:07.485 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 14, 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "e5125de7": { 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 14, 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 140.2, 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:56:07.486 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:56:07.487 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:56:07.487 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:56:07.487 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:56:07.487 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:56:07.487 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:07.487 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:07.487 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:07.487 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:07.487 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:07.487 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:56:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 -- 172.21.15.62:0/1895349528 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc2c0a2e40 msgr2=0x7fbc2c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc2c0a2e40 0x7fbc2c0a5300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbc340308d0 tx=0x7fbc34005ef0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 -- 172.21.15.62:0/1895349528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3c102790 msgr2=0x7fbc3c19f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3c102790 0x7fbc3c19f5d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800d950 tx=0x7fbc2800de10 comp rx=0 tx=0).stop 2024-09-11T23:56:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 -- 172.21.15.62:0/1895349528 shutdown_connections 2024-09-11T23:56:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc2c0a2e40 0x7fbc2c0a5300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc3c106f10 0x7fbc3c10c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3c102790 0x7fbc3c19f5d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 --2- 172.21.15.62:0/1895349528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc3c101de0 0x7fbc3c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 -- 172.21.15.62:0/1895349528 >> 172.21.15.62:0/1895349528 conn(0x7fbc3c0fd880 msgr2=0x7fbc3c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 -- 172.21.15.62:0/1895349528 shutdown_connections 2024-09-11T23:56:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:07.482+0000 7fbc42e5d640 1 -- 172.21.15.62:0/1895349528 wait complete. 2024-09-11T23:56:08.278 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e5125de7', quiesce took 6.9 seconds. Will release after: 13.249999999999998 2024-09-11T23:56:08.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:08 smithi120 ceph-mon[35182]: pgmap v124: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 48 MiB/s wr, 2.26k op/s 2024-09-11T23:56:08.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:08 smithi083 ceph-mon[34805]: pgmap v124: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 48 MiB/s wr, 2.26k op/s 2024-09-11T23:56:08.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:08 smithi062 ceph-mon[30958]: pgmap v124: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 48 MiB/s wr, 2.26k op/s 2024-09-11T23:56:10.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:10 smithi120 ceph-mon[35182]: pgmap v125: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-09-11T23:56:10.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:10 smithi083 ceph-mon[34805]: pgmap v125: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-09-11T23:56:10.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:10 smithi062 ceph-mon[30958]: pgmap v125: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-09-11T23:56:12.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 210.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:56:12.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:12 smithi120 ceph-mon[35182]: pgmap v126: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.27k op/s 2024-09-11T23:56:12.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:12 smithi083 ceph-mon[34805]: pgmap v126: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.27k op/s 2024-09-11T23:56:12.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:12 smithi062 ceph-mon[30958]: pgmap v126: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.27k op/s 2024-09-11T23:56:14.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:14 smithi120 ceph-mon[35182]: pgmap v127: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-11T23:56:14.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:14 smithi083 ceph-mon[34805]: pgmap v127: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-11T23:56:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:14 smithi062 ceph-mon[30958]: pgmap v127: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-11T23:56:16.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:16 smithi120 ceph-mon[35182]: pgmap v128: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.8 MiB/s wr, 376 op/s 2024-09-11T23:56:16.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:16 smithi083 ceph-mon[34805]: pgmap v128: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.8 MiB/s wr, 376 op/s 2024-09-11T23:56:16.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:16 smithi062 ceph-mon[30958]: pgmap v128: 97 pgs: 97 active+clean; 2.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.8 MiB/s wr, 376 op/s 2024-09-11T23:56:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 215.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:56:17.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:56:17.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:56:17.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:56:17.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:56:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:56:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:56:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:56:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:56:17.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:56:17.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:56:17.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:56:17.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:56:19.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:19 smithi120 ceph-mon[35182]: pgmap v129: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.3 MiB/s wr, 532 op/s 2024-09-11T23:56:19.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:19 smithi083 ceph-mon[34805]: pgmap v129: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.3 MiB/s wr, 532 op/s 2024-09-11T23:56:19.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:19 smithi062 ceph-mon[30958]: pgmap v129: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.3 MiB/s wr, 532 op/s 2024-09-11T23:56:21.529 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:56:21.558 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:21 smithi062 ceph-mon[30958]: pgmap v130: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.2 MiB/s wr, 216 op/s 2024-09-11T23:56:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:21 smithi120 ceph-mon[35182]: pgmap v130: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.2 MiB/s wr, 216 op/s 2024-09-11T23:56:21.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:21 smithi083 ceph-mon[34805]: pgmap v130: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.2 MiB/s wr, 216 op/s 2024-09-11T23:56:21.801 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.296+0000 7f49b3bf5640 1 -- 172.21.15.62:0/2934715371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49ac1027a0 msgr2=0x7f49ac1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.296+0000 7f49b3bf5640 1 --2- 172.21.15.62:0/2934715371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49ac1027a0 0x7f49ac1067f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f499c009900 tx=0x7f499c02a410 comp rx=0 tx=0).stop 2024-09-11T23:56:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.296+0000 7f49b3bf5640 1 -- 172.21.15.62:0/2934715371 shutdown_connections 2024-09-11T23:56:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.296+0000 7f49b3bf5640 1 --2- 172.21.15.62:0/2934715371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49ac106f20 0x7f49ac10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.296+0000 7f49b3bf5640 1 --2- 172.21.15.62:0/2934715371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49ac1027a0 0x7f49ac1067f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.296+0000 7f49b3bf5640 1 --2- 172.21.15.62:0/2934715371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49ac101df0 0x7f49ac1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.296+0000 7f49b3bf5640 1 -- 172.21.15.62:0/2934715371 >> 172.21.15.62:0/2934715371 conn(0x7f49ac0fd8f0 msgr2=0x7f49ac0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.297+0000 7f49b3bf5640 1 -- 172.21.15.62:0/2934715371 shutdown_connections 2024-09-11T23:56:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.297+0000 7f49b3bf5640 1 -- 172.21.15.62:0/2934715371 wait complete. 2024-09-11T23:56:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.297+0000 7f49b3bf5640 1 Processor -- start 2024-09-11T23:56:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.297+0000 7f49b3bf5640 1 -- start start 2024-09-11T23:56:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b3bf5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49ac101df0 0x7f49ac198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b3bf5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49ac1027a0 0x7f49ac1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b3bf5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49ac106f20 0x7f49ac19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b3bf5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac111f80 con 0x7f49ac106f20 2024-09-11T23:56:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b3bf5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac111e00 con 0x7f49ac101df0 2024-09-11T23:56:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b3bf5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac112100 con 0x7f49ac1027a0 2024-09-11T23:56:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b2192640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49ac106f20 0x7f49ac19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b2192640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49ac106f20 0x7f49ac19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35828/0 (socket says 172.21.15.62:35828) 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b2192640 1 -- 172.21.15.62:0/308191679 learned_addr learned my addr 172.21.15.62:0/308191679 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b1991640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49ac101df0 0x7f49ac198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b1190640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49ac1027a0 0x7f49ac1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b2192640 1 -- 172.21.15.62:0/308191679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49ac1027a0 msgr2=0x7f49ac1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b2192640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49ac1027a0 0x7f49ac1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b2192640 1 -- 172.21.15.62:0/308191679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49ac101df0 msgr2=0x7f49ac198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b2192640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49ac101df0 0x7f49ac198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b2192640 1 -- 172.21.15.62:0/308191679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac19d410 con 0x7f49ac106f20 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.298+0000 7f49b1190640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49ac1027a0 0x7f49ac1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.299+0000 7f49b2192640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49ac106f20 0x7f49ac19cd30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f49a400eb10 tx=0x7f49a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.299+0000 7f49a2ffd640 1 -- 172.21.15.62:0/308191679 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4004b20 con 0x7f49ac106f20 2024-09-11T23:56:22.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.299+0000 7f49a2ffd640 1 -- 172.21.15.62:0/308191679 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a4004cc0 con 0x7f49ac106f20 2024-09-11T23:56:22.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.299+0000 7f49a2ffd640 1 -- 172.21.15.62:0/308191679 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4010760 con 0x7f49ac106f20 2024-09-11T23:56:22.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.299+0000 7f49b3bf5640 1 -- 172.21.15.62:0/308191679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ac19d700 con 0x7f49ac106f20 2024-09-11T23:56:22.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.299+0000 7f49b3bf5640 1 -- 172.21.15.62:0/308191679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ac1a4fe0 con 0x7f49ac106f20 2024-09-11T23:56:22.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.300+0000 7f49b3bf5640 1 -- 172.21.15.62:0/308191679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ac1021d0 con 0x7f49ac106f20 2024-09-11T23:56:22.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.301+0000 7f49a2ffd640 1 -- 172.21.15.62:0/308191679 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f49a4005020 con 0x7f49ac106f20 2024-09-11T23:56:22.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.302+0000 7f49a2ffd640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f49980a2cf0 0x7f49980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:22.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.302+0000 7f49b1991640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f49980a2cf0 0x7f49980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:22.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.302+0000 7f49a2ffd640 1 -- 172.21.15.62:0/308191679 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f49a40cf9c0 con 0x7f49ac106f20 2024-09-11T23:56:22.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.302+0000 7f49b1991640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f49980a2cf0 0x7f49980a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49a800bbe0 tx=0x7f49a800b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:22.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.310+0000 7f49a2ffd640 1 -- 172.21.15.62:0/308191679 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49a40d1e20 con 0x7f49ac106f20 2024-09-11T23:56:22.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 220.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:56:22.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.525+0000 7f49b3bf5640 1 -- 172.21.15.62:0/308191679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49ac109bd0 con 0x7f49ac106f20 2024-09-11T23:56:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.526+0000 7f49a2ffd640 1 -- 172.21.15.62:0/308191679 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7f49a409c610 con 0x7f49ac106f20 2024-09-11T23:56:22.526 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:56:22.526 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"btime":"2024-09-11T23:55:51:325581+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:55:50.698407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:56:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-09-11T23:56:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.528+0000 7f49b3bf5640 1 -- 172.21.15.62:0/308191679 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f49980a2cf0 msgr2=0x7f49980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.528+0000 7f49b3bf5640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f49980a2cf0 0x7f49980a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49a800bbe0 tx=0x7f49a800b390 comp rx=0 tx=0).stop 2024-09-11T23:56:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.528+0000 7f49b3bf5640 1 -- 172.21.15.62:0/308191679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49ac106f20 msgr2=0x7f49ac19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.528+0000 7f49b3bf5640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49ac106f20 0x7f49ac19cd30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f49a400eb10 tx=0x7f49a400c2d0 comp rx=0 tx=0).stop 2024-09-11T23:56:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.528+0000 7f49b3bf5640 1 -- 172.21.15.62:0/308191679 shutdown_connections 2024-09-11T23:56:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.528+0000 7f49b3bf5640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f49980a2cf0 0x7f49980a51b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.528+0000 7f49b3bf5640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49ac106f20 0x7f49ac19cd30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.528+0000 7f49b3bf5640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49ac1027a0 0x7f49ac1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.528+0000 7f49b3bf5640 1 --2- 172.21.15.62:0/308191679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49ac101df0 0x7f49ac198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.528+0000 7f49b3bf5640 1 -- 172.21.15.62:0/308191679 >> 172.21.15.62:0/308191679 conn(0x7f49ac0fd8f0 msgr2=0x7f49ac0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.529+0000 7f49b3bf5640 1 -- 172.21.15.62:0/308191679 shutdown_connections 2024-09-11T23:56:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:22.529+0000 7f49b3bf5640 1 -- 172.21.15.62:0/308191679 wait complete. 2024-09-11T23:56:23.293 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id e5125de7 --release --await' 2024-09-11T23:56:23.293 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id e5125de7 --release --await 2024-09-11T23:56:23.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:23 smithi062 ceph-mon[30958]: pgmap v131: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-09-11T23:56:23.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:23 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/308191679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:23.597 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:23.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:23 smithi120 ceph-mon[35182]: pgmap v131: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-09-11T23:56:23.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:23 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/308191679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:23 smithi083 ceph-mon[34805]: pgmap v131: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-09-11T23:56:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:23 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/308191679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 -- 172.21.15.62:0/1422302100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f764010c1a0 msgr2=0x7f764010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 --2- 172.21.15.62:0/1422302100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f764010c1a0 0x7f764010e610 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7638009ca0 tx=0x7f763802a950 comp rx=0 tx=0).stop 2024-09-11T23:56:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 -- 172.21.15.62:0/1422302100 shutdown_connections 2024-09-11T23:56:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 --2- 172.21.15.62:0/1422302100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f764010c1a0 0x7f764010e610 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 --2- 172.21.15.62:0/1422302100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76401036e0 0x7f764010bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 --2- 172.21.15.62:0/1422302100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7640102dc0 0x7f76401031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 -- 172.21.15.62:0/1422302100 >> 172.21.15.62:0/1422302100 conn(0x7f76400fc7a0 msgr2=0x7f76400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 -- 172.21.15.62:0/1422302100 shutdown_connections 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 -- 172.21.15.62:0/1422302100 wait complete. 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 Processor -- start 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.248+0000 7f7646e35640 1 -- start start 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f7646e35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7640102dc0 0x7f7640198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f7646e35640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76401036e0 0x7f7640198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f7646e35640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f764010c1a0 0x7f764019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f7646e35640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7640111f70 con 0x7f764010c1a0 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f7646e35640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7640111df0 con 0x7f7640102dc0 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f7646e35640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76401120f0 con 0x7f76401036e0 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f76453d2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f764010c1a0 0x7f764019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f76453d2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f764010c1a0 0x7f764019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35844/0 (socket says 172.21.15.62:35844) 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f76453d2640 1 -- 172.21.15.62:0/563406084 learned_addr learned my addr 172.21.15.62:0/563406084 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f7637fff640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76401036e0 0x7f7640198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f7644bd1640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7640102dc0 0x7f7640198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f76453d2640 1 -- 172.21.15.62:0/563406084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76401036e0 msgr2=0x7f7640198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f76453d2640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76401036e0 0x7f7640198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f76453d2640 1 -- 172.21.15.62:0/563406084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7640102dc0 msgr2=0x7f7640198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f76453d2640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7640102dc0 0x7f7640198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f76453d2640 1 -- 172.21.15.62:0/563406084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764019d2e0 con 0x7f764010c1a0 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f7644bd1640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7640102dc0 0x7f7640198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f76453d2640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f764010c1a0 0x7f764019cc00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f763802ae70 tx=0x7f763802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.249+0000 7f7635ffb640 1 -- 172.21.15.62:0/563406084 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638034040 con 0x7f764010c1a0 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.250+0000 7f7635ffb640 1 -- 172.21.15.62:0/563406084 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7638030920 con 0x7f764010c1a0 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.250+0000 7f7646e35640 1 -- 172.21.15.62:0/563406084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764019d570 con 0x7f764010c1a0 2024-09-11T23:56:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.250+0000 7f7635ffb640 1 -- 172.21.15.62:0/563406084 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638045480 con 0x7f764010c1a0 2024-09-11T23:56:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.250+0000 7f7646e35640 1 -- 172.21.15.62:0/563406084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76401a4eb0 con 0x7f764010c1a0 2024-09-11T23:56:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f764920e740 1 Processor -- start 2024-09-11T23:56:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f7635ffb640 1 -- 172.21.15.62:0/563406084 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7638037020 con 0x7f764010c1a0 2024-09-11T23:56:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f764920e740 1 -- start start 2024-09-11T23:56:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f764920e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 0x556a53ce3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f764920e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cedd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f764920e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53cee240 0x556a53cf0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f764920e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556a53c48620 con 0x556a53ce3980 2024-09-11T23:56:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f764920e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a53befe50 con 0x556a53cee240 2024-09-11T23:56:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f764920e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556a53c445e0 con 0x556a53ce3060 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f7644bd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cedd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f7644bd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cedd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35846/0 (socket says 172.21.15.62:35846) 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.253+0000 7f7644bd1640 1 -- 172.21.15.62:0/2409500236 learned_addr learned my addr 172.21.15.62:0/2409500236 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7637fff640 1 --2- 172.21.15.62:0/2409500236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 0x556a53ce3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7635ffb640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76280a2cf0 0x7f76280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7637fff640 1 --2- 172.21.15.62:0/2409500236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53cee240 0x556a53cf0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7644bd1640 1 -- 172.21.15.62:0/2409500236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 msgr2=0x556a53ce3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7644bd1640 1 --2- 172.21.15.62:0/2409500236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 0x556a53ce3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7644bd1640 1 -- 172.21.15.62:0/2409500236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53cee240 msgr2=0x556a53cf0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7644bd1640 1 --2- 172.21.15.62:0/2409500236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53cee240 0x556a53cf0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7644bd1640 1 -- 172.21.15.62:0/2409500236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a53be0d40 con 0x556a53ce3980 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7637fff640 1 --2- 172.21.15.62:0/2409500236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 0x556a53ce3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7635ffb640 1 -- 172.21.15.62:0/563406084 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f76380f0390 con 0x7f764010c1a0 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f76453d2640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76280a2cf0 0x7f76280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7644bd1640 1 --2- 172.21.15.62:0/2409500236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cedd00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f763c009c50 tx=0x7f763c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c47157c0c8c0f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7626ffd640 1 -- 172.21.15.62:0/2409500236 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c016070 con 0x556a53ce3980 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7626ffd640 1 -- 172.21.15.62:0/2409500236 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c011950 con 0x556a53ce3980 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.254+0000 7f7626ffd640 1 -- 172.21.15.62:0/2409500236 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c0108d0 con 0x556a53ce3980 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.255+0000 7f764920e740 1 -- 172.21.15.62:0/2409500236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 msgr2=0x556a53cedd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.255+0000 7f76453d2640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76280a2cf0 0x7f76280a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7638009430 tx=0x7f763802a950 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.255+0000 7f764920e740 1 --2- 172.21.15.62:0/2409500236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cedd00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f763c009c50 tx=0x7f763c00e6c0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.255+0000 7f764920e740 1 -- 172.21.15.62:0/2409500236 shutdown_connections 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.255+0000 7f764920e740 1 --2- 172.21.15.62:0/2409500236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53cee240 0x556a53cf0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.255+0000 7f764920e740 1 --2- 172.21.15.62:0/2409500236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cedd00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.255+0000 7f764920e740 1 --2- 172.21.15.62:0/2409500236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 0x556a53ce3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.255+0000 7f764920e740 1 -- 172.21.15.62:0/2409500236 >> 172.21.15.62:0/2409500236 conn(0x556a53cdf5f0 msgr2=0x556a53cdf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.255+0000 7f764920e740 1 -- 172.21.15.62:0/2409500236 shutdown_connections 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.255+0000 7f764920e740 1 -- 172.21.15.62:0/2409500236 wait complete. 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f764920e740 1 -- start start 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f764920e740 1 Processor -- start 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f764920e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3060 0x556a53ceb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f764920e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3980 0x556a53ceb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f764920e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53cee240 0x556a53cebc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f764920e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556a53cf0e10 con 0x556a53ce3060 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f764920e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a53bd0f10 con 0x556a53ce3980 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f764920e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556a53cf0f90 con 0x556a53cee240 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f764920e740 10 client.0 _ll_register_callbacks cb 0x556a53bf4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f7637fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3060 0x556a53ceb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f7637fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3060 0x556a53ceb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35850/0 (socket says 172.21.15.62:35850) 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f7637fff640 1 -- 172.21.15.62:0/1327481342 learned_addr learned my addr 172.21.15.62:0/1327481342 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f7644bd1640 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3980 0x556a53ceb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f7637fff640 1 -- 172.21.15.62:0/1327481342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53cee240 msgr2=0x556a53cebc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f7637fff640 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53cee240 0x556a53cebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f7637fff640 1 -- 172.21.15.62:0/1327481342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3980 msgr2=0x556a53ceb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f7637fff640 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3980 0x556a53ceb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.256+0000 7f7637fff640 1 -- 172.21.15.62:0/1327481342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a53957580 con 0x556a53ce3060 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f7637fff640 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3060 0x556a53ceb180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7630013b00 tx=0x7f7630010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337a5c5dfea38546 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f761ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f761ffff640 1 -- 172.21.15.62:0/1327481342 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630010e10 con 0x556a53ce3060 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f761ffff640 1 -- 172.21.15.62:0/1327481342 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f7630004030 con 0x556a53ce3060 2024-09-11T23:56:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f764920e740 1 -- 172.21.15.62:0/1327481342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c011950 con 0x556a53ce3060 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f761ffff640 1 -- 172.21.15.62:0/1327481342 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7630004230 con 0x556a53ce3060 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f764920e740 10 client.16122 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f764920e740 1 -- 172.21.15.62:0/1327481342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a53a21960 con 0x556a53ce3060 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f761ffff640 1 -- 172.21.15.62:0/1327481342 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c011950 con 0x556a53ce3060 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f761ffff640 1 -- 172.21.15.62:0/1327481342 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7630025570 con 0x556a53ce3060 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f764920e740 10 client.16122 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f764920e740 10 client.16122 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.257+0000 7f764920e740 20 client.16122 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.258+0000 7f764920e740 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556a53cfddb0 0x556a53d001a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.258+0000 7f764920e740 4 client.16122 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.258+0000 7f764920e740 1 -- 172.21.15.62:0/1327481342 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a53c55b90 con 0x556a53cfddb0 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.258+0000 7f7644bd1640 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556a53cfddb0 0x556a53d001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.258+0000 7f7644bd1640 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556a53cfddb0 0x556a53d001a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7d9a24cc31eac9 server_cookie=b7f874455153f67d in_seq=0 out_seq=0 2024-09-11T23:56:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.258+0000 7f761ffff640 10 client.16122 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f761ffff640 1 -- 172.21.15.62:0/1327481342 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a53c55b90 con 0x556a53cfddb0 2024-09-11T23:56:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f761ffff640 10 client.16122 handle_command_reply: tid=0 2024-09-11T23:56:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 client.16122 shutdown 2024-09-11T23:56:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 -- 172.21.15.62:0/1327481342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3060 msgr2=0x556a53ceb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3060 0x556a53ceb180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7630013b00 tx=0x7f7630010430 comp rx=0 tx=0).stop 2024-09-11T23:56:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 -- 172.21.15.62:0/1327481342 shutdown_connections 2024-09-11T23:56:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556a53cfddb0 0x556a53d001a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53cee240 0x556a53cebc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3980 0x556a53ceb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 --2- 172.21.15.62:0/1327481342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3060 0x556a53ceb180 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 -- 172.21.15.62:0/1327481342 >> 172.21.15.62:0/1327481342 conn(0x556a53cdf5f0 msgr2=0x556a53ce52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f761ffff640 0 client.16122 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 -- 172.21.15.62:0/1327481342 shutdown_connections 2024-09-11T23:56:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 1 -- 172.21.15.62:0/1327481342 wait complete. 2024-09-11T23:56:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.259+0000 7f764920e740 20 client.16122 trim_cache size 0 max 16384 2024-09-11T23:56:24.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.276+0000 7f764920e740 1 Processor -- start 2024-09-11T23:56:24.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.276+0000 7f764920e740 1 -- start start 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f764920e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3060 0x556a53ce5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f764920e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cde640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f764920e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53cee240 0x556a53d05a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f764920e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76401affb0 con 0x556a53ce3980 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f764920e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76401ad880 con 0x556a53ce3060 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f764920e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76401ab6d0 con 0x556a53cee240 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7644bd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cde640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7644bd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cde640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35852/0 (socket says 172.21.15.62:35852) 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7644bd1640 1 -- 172.21.15.62:0/1756956351 learned_addr learned my addr 172.21.15.62:0/1756956351 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7637fff640 1 --2- 172.21.15.62:0/1756956351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3060 0x556a53ce5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7644bd1640 1 -- 172.21.15.62:0/1756956351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53cee240 msgr2=0x556a53d05a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7644bd1640 1 --2- 172.21.15.62:0/1756956351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53cee240 0x556a53d05a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7644bd1640 1 -- 172.21.15.62:0/1756956351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3060 msgr2=0x556a53ce5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:24.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7644bd1640 1 --2- 172.21.15.62:0/1756956351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3060 0x556a53ce5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7644bd1640 1 -- 172.21.15.62:0/1756956351 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a53ce2e90 con 0x556a53ce3980 2024-09-11T23:56:24.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7637fff640 1 --2- 172.21.15.62:0/1756956351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3060 0x556a53ce5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:56:24.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.277+0000 7f7644bd1640 1 --2- 172.21.15.62:0/1756956351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cde640 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f763c000c00 tx=0x7f763c0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd5d56ae2083068 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:24.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f76267fc640 1 -- 172.21.15.62:0/1756956351 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c016040 con 0x556a53ce3980 2024-09-11T23:56:24.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f76267fc640 1 -- 172.21.15.62:0/1756956351 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c0053d0 con 0x556a53ce3980 2024-09-11T23:56:24.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f76267fc640 1 -- 172.21.15.62:0/1756956351 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c029430 con 0x556a53ce3980 2024-09-11T23:56:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f764920e740 1 -- 172.21.15.62:0/1756956351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 msgr2=0x556a53cde640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f764920e740 1 --2- 172.21.15.62:0/1756956351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cde640 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f763c000c00 tx=0x7f763c0103d0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f764920e740 1 -- 172.21.15.62:0/1756956351 shutdown_connections 2024-09-11T23:56:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f764920e740 1 --2- 172.21.15.62:0/1756956351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53cee240 0x556a53d05a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f764920e740 1 --2- 172.21.15.62:0/1756956351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cde640 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f764920e740 1 --2- 172.21.15.62:0/1756956351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53ce3060 0x556a53ce5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f764920e740 1 -- 172.21.15.62:0/1756956351 >> 172.21.15.62:0/1756956351 conn(0x556a53cdf5f0 msgr2=0x556a53cddf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f764920e740 1 -- 172.21.15.62:0/1756956351 shutdown_connections 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.278+0000 7f764920e740 1 -- 172.21.15.62:0/1756956351 wait complete. 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f764920e740 1 -- start start 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f764920e740 1 Processor -- start 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f764920e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 0x556a53cf6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f764920e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cf2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f764920e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53cee240 0x556a53cf30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f764920e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76401abd40 con 0x556a53ce3980 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f764920e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76401ad1b0 con 0x556a53cee240 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f764920e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76401ac410 con 0x556a53ce3060 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f764920e740 10 client.0 _ll_register_callbacks cb 0x7f763000ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f7644bd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cf2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f7644bd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cf2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35862/0 (socket says 172.21.15.62:35862) 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f7644bd1640 1 -- 172.21.15.62:0/2365432581 learned_addr learned my addr 172.21.15.62:0/2365432581 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f7637fff640 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53cee240 0x556a53cf30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.279+0000 7f7637fff640 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 0x556a53cf6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f7644bd1640 1 -- 172.21.15.62:0/2365432581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 msgr2=0x556a53cf6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f7644bd1640 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 0x556a53cf6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f7644bd1640 1 -- 172.21.15.62:0/2365432581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53cee240 msgr2=0x556a53cf30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f7644bd1640 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53cee240 0x556a53cf30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f7644bd1640 1 -- 172.21.15.62:0/2365432581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a53cf3910 con 0x556a53ce3980 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f7637fff640 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 0x556a53cf6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f7644bd1640 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cf2ba0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f763c010440 tx=0x7f763c009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69414cd5d7332a6c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f761ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f761ffff640 1 -- 172.21.15.62:0/2365432581 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c035070 con 0x556a53ce3980 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f761ffff640 1 -- 172.21.15.62:0/2365432581 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f763c004620 con 0x556a53ce3980 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f764920e740 1 -- 172.21.15.62:0/2365432581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c0053d0 con 0x556a53ce3980 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f761ffff640 1 -- 172.21.15.62:0/2365432581 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f763c041050 con 0x556a53ce3980 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f764920e740 10 client.16128 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f764920e740 1 -- 172.21.15.62:0/2365432581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a53d698f0 con 0x556a53ce3980 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.280+0000 7f761ffff640 1 -- 172.21.15.62:0/2365432581 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c0053d0 con 0x556a53ce3980 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.281+0000 7f761ffff640 1 -- 172.21.15.62:0/2365432581 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f763c00f040 con 0x556a53ce3980 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.281+0000 7f764920e740 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.281+0000 7f764920e740 10 client.16128 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.281+0000 7f764920e740 20 client.16128 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.281+0000 7f764920e740 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556a53d69dc0 0x556a53d6c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.281+0000 7f764920e740 4 client.16128 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e5125de7", "release": true, "await": true}] 2024-09-11T23:56:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.281+0000 7f764920e740 1 -- 172.21.15.62:0/2365432581 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e5125de7", "release": true, "await": true}) -- 0x556a53c500a0 con 0x556a53d69dc0 2024-09-11T23:56:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.281+0000 7f7637fff640 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556a53d69dc0 0x556a53d6c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.281+0000 7f7637fff640 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556a53d69dc0 0x556a53d6c1e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c09621a280dcdf server_cookie=9b48c924a1fa7e86 in_seq=0 out_seq=0 2024-09-11T23:56:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.281+0000 7f761ffff640 10 client.16128 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:24.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.327+0000 7f761ffff640 1 -- 172.21.15.62:0/2365432581 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556a53c500a0 con 0x556a53d69dc0 2024-09-11T23:56:24.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.327+0000 7f761ffff640 10 client.16128 handle_command_reply: tid=0 2024-09-11T23:56:24.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.327+0000 7f764920e740 1 client.16128 shutdown 2024-09-11T23:56:24.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.328+0000 7f764920e740 1 -- 172.21.15.62:0/2365432581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 msgr2=0x556a53cf2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:24.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.328+0000 7f764920e740 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cf2ba0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f763c010440 tx=0x7f763c009760 comp rx=0 tx=0).stop 2024-09-11T23:56:24.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.328+0000 7f764920e740 1 -- 172.21.15.62:0/2365432581 shutdown_connections 2024-09-11T23:56:24.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.328+0000 7f764920e740 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556a53d69dc0 0x556a53d6c1e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.328+0000 7f764920e740 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a53cee240 0x556a53cf30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.328+0000 7f764920e740 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a53ce3980 0x556a53cf2ba0 secure :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f763c010440 tx=0x7f763c009760 comp rx=0 tx=0).stop 2024-09-11T23:56:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.328+0000 7f764920e740 1 --2- 172.21.15.62:0/2365432581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556a53ce3060 0x556a53cf6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.328+0000 7f764920e740 1 -- 172.21.15.62:0/2365432581 >> 172.21.15.62:0/2365432581 conn(0x556a53cdf5f0 msgr2=0x556a53ceb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.328+0000 7f761ffff640 0 client.16128 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.328+0000 7f764920e740 1 -- 172.21.15.62:0/2365432581 shutdown_connections 2024-09-11T23:56:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.329+0000 7f764920e740 1 -- 172.21.15.62:0/2365432581 wait complete. 2024-09-11T23:56:24.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.329+0000 7f764920e740 20 client.16128 trim_cache size 0 max 16384 2024-09-11T23:56:24.330 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:56:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 29, 2024-09-11T23:56:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:56:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 16, 2024-09-11T23:56:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:56:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "e5125de7": { 2024-09-11T23:56:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 16, 2024-09-11T23:56:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 140.2, 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:56:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:24.332 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:56:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 -- 172.21.15.62:0/563406084 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76280a2cf0 msgr2=0x7f76280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76280a2cf0 0x7f76280a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7638009430 tx=0x7f763802a950 comp rx=0 tx=0).stop 2024-09-11T23:56:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 -- 172.21.15.62:0/563406084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f764010c1a0 msgr2=0x7f764019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f764010c1a0 0x7f764019cc00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f763802ae70 tx=0x7f763802f5b0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 -- 172.21.15.62:0/563406084 shutdown_connections 2024-09-11T23:56:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76280a2cf0 0x7f76280a51b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f764010c1a0 0x7f764019cc00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76401036e0 0x7f7640198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 --2- 172.21.15.62:0/563406084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7640102dc0 0x7f7640198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 -- 172.21.15.62:0/563406084 >> 172.21.15.62:0/563406084 conn(0x7f76400fc7a0 msgr2=0x7f76400fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 -- 172.21.15.62:0/563406084 shutdown_connections 2024-09-11T23:56:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:24.330+0000 7f7646e35640 1 -- 172.21.15.62:0/563406084 wait complete. 2024-09-11T23:56:25.213 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e5125de7', total seconds elapsed: 23.8 2024-09-11T23:56:25.214 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T23:56:25.214 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:56:25.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:25 smithi062 ceph-mon[30958]: pgmap v132: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-09-11T23:56:25.597 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:25.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:25 smithi120 ceph-mon[35182]: pgmap v132: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-09-11T23:56:25.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:25 smithi083 ceph-mon[34805]: pgmap v132: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-09-11T23:56:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.198+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/2424487373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2dc100a80 msgr2=0x7fa2dc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.198+0000 7fa2e1dc1640 1 --2- 172.21.15.62:0/2424487373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2dc100a80 0x7fa2dc100ee0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009930 tx=0x7fa2d002a420 comp rx=0 tx=0).stop 2024-09-11T23:56:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.198+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/2424487373 shutdown_connections 2024-09-11T23:56:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.198+0000 7fa2e1dc1640 1 --2- 172.21.15.62:0/2424487373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2dc101420 0x7fa2dc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.198+0000 7fa2e1dc1640 1 --2- 172.21.15.62:0/2424487373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2dc100a80 0x7fa2dc100ee0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.198+0000 7fa2e1dc1640 1 --2- 172.21.15.62:0/2424487373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2dc106a80 0x7fa2dc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.198+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/2424487373 >> 172.21.15.62:0/2424487373 conn(0x7fa2dc0fc780 msgr2=0x7fa2dc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.198+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/2424487373 shutdown_connections 2024-09-11T23:56:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.198+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/2424487373 wait complete. 2024-09-11T23:56:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.199+0000 7fa2e1dc1640 1 Processor -- start 2024-09-11T23:56:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.201+0000 7fa2e1dc1640 1 -- start start 2024-09-11T23:56:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.201+0000 7fa2e1dc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2dc100a80 0x7fa2dc0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.201+0000 7fa2e1dc1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2dc101420 0x7fa2dc06a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.201+0000 7fa2e1dc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2dc106a80 0x7fa2dc06acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2e1dc1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc110e80 con 0x7fa2dc101420 2024-09-11T23:56:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2e1dc1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc110d00 con 0x7fa2dc100a80 2024-09-11T23:56:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2e1dc1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc111000 con 0x7fa2dc106a80 2024-09-11T23:56:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2daffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2dc101420 0x7fa2dc06a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2daffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2dc101420 0x7fa2dc06a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35880/0 (socket says 172.21.15.62:35880) 2024-09-11T23:56:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2daffd640 1 -- 172.21.15.62:0/739201352 learned_addr learned my addr 172.21.15.62:0/739201352 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2dbfff640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2dc106a80 0x7fa2dc06acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2db7fe640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2dc100a80 0x7fa2dc0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2daffd640 1 -- 172.21.15.62:0/739201352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2dc106a80 msgr2=0x7fa2dc06acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2daffd640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2dc106a80 0x7fa2dc06acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2daffd640 1 -- 172.21.15.62:0/739201352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2dc100a80 msgr2=0x7fa2dc0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2daffd640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2dc100a80 0x7fa2dc0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.202+0000 7fa2daffd640 1 -- 172.21.15.62:0/739201352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc06b210 con 0x7fa2dc101420 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.203+0000 7fa2daffd640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2dc101420 0x7fa2dc06a790 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009230 tx=0x7fa2d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.203+0000 7fa2d8ff9640 1 -- 172.21.15.62:0/739201352 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d003f030 con 0x7fa2dc101420 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.203+0000 7fa2d8ff9640 1 -- 172.21.15.62:0/739201352 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d0003ad0 con 0x7fa2dc101420 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.203+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/739201352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc06b4a0 con 0x7fa2dc101420 2024-09-11T23:56:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.203+0000 7fa2d8ff9640 1 -- 172.21.15.62:0/739201352 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0035070 con 0x7fa2dc101420 2024-09-11T23:56:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.203+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/739201352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc1b3d20 con 0x7fa2dc101420 2024-09-11T23:56:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.204+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/739201352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc1021c0 con 0x7fa2dc101420 2024-09-11T23:56:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.206+0000 7fa2d8ff9640 1 -- 172.21.15.62:0/739201352 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa2d0042030 con 0x7fa2dc101420 2024-09-11T23:56:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.206+0000 7fa2d8ff9640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa2c40a2d70 0x7fa2c40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.207+0000 7fa2d8ff9640 1 -- 172.21.15.62:0/739201352 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fa2d00ef5f0 con 0x7fa2dc101420 2024-09-11T23:56:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.207+0000 7fa2db7fe640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa2c40a2d70 0x7fa2c40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.207+0000 7fa2db7fe640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa2c40a2d70 0x7fa2c40a5230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0095f0 tx=0x7fa2cc005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:26.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.213+0000 7fa2d8ff9640 1 -- 172.21.15.62:0/739201352 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2d00bbbf0 con 0x7fa2dc101420 2024-09-11T23:56:26.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.562+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/739201352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2dc109010 con 0x7fa2dc101420 2024-09-11T23:56:26.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.563+0000 7fa2d8ff9640 1 -- 172.21.15.62:0/739201352 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7fa2d00bfef0 con 0x7fa2dc101420 2024-09-11T23:56:26.563 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:56:26.564 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-09-11T23:56:26.564 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"btime":"2024-09-11T23:55:51:325581+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:55:50.698407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:56:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/739201352 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa2c40a2d70 msgr2=0x7fa2c40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa2c40a2d70 0x7fa2c40a5230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0095f0 tx=0x7fa2cc005f90 comp rx=0 tx=0).stop 2024-09-11T23:56:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/739201352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2dc101420 msgr2=0x7fa2dc06a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2dc101420 0x7fa2dc06a790 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009230 tx=0x7fa2d0004270 comp rx=0 tx=0).stop 2024-09-11T23:56:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/739201352 shutdown_connections 2024-09-11T23:56:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa2c40a2d70 0x7fa2c40a5230 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2dc106a80 0x7fa2dc06acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2dc101420 0x7fa2dc06a790 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 --2- 172.21.15.62:0/739201352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2dc100a80 0x7fa2dc0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:26.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/739201352 >> 172.21.15.62:0/739201352 conn(0x7fa2dc0fc780 msgr2=0x7fa2dc0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:26.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.567+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/739201352 shutdown_connections 2024-09-11T23:56:26.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:26.568+0000 7fa2e1dc1640 1 -- 172.21.15.62:0/739201352 wait complete. 2024-09-11T23:56:27.349 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T23:56:27.349 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T23:56:27.446 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 225.0 s, 2276.5 tps, lat 177.886 ms stddev 1595.640 2024-09-11T23:56:27.623 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:27 smithi120 ceph-mon[35182]: pgmap v133: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-09-11T23:56:27.623 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:27 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/739201352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:27.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:27 smithi083 ceph-mon[34805]: pgmap v133: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-09-11T23:56:27.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:27 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/739201352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:27.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:27 smithi062 ceph-mon[30958]: pgmap v133: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-09-11T23:56:27.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:27 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/739201352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.273+0000 7f5b13fba640 1 -- 172.21.15.62:0/3741105483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b0c106f20 msgr2=0x7f5b0c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:28.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.273+0000 7f5b13fba640 1 --2- 172.21.15.62:0/3741105483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b0c106f20 0x7f5b0c10eaa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009be0 tx=0x7f5b0402a6c0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.274+0000 7f5b13fba640 1 -- 172.21.15.62:0/3741105483 shutdown_connections 2024-09-11T23:56:28.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.274+0000 7f5b13fba640 1 --2- 172.21.15.62:0/3741105483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b0c106f20 0x7f5b0c10eaa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.274+0000 7f5b13fba640 1 --2- 172.21.15.62:0/3741105483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b0c1027a0 0x7f5b0c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.274+0000 7f5b13fba640 1 --2- 172.21.15.62:0/3741105483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c101df0 0x7f5b0c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.274+0000 7f5b13fba640 1 -- 172.21.15.62:0/3741105483 >> 172.21.15.62:0/3741105483 conn(0x7f5b0c0fd8f0 msgr2=0x7f5b0c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:28.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.275+0000 7f5b13fba640 1 -- 172.21.15.62:0/3741105483 shutdown_connections 2024-09-11T23:56:28.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.275+0000 7f5b13fba640 1 -- 172.21.15.62:0/3741105483 wait complete. 2024-09-11T23:56:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.275+0000 7f5b13fba640 1 Processor -- start 2024-09-11T23:56:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b13fba640 1 -- start start 2024-09-11T23:56:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b13fba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b0c101df0 0x7f5b0c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b13fba640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c1027a0 0x7f5b0c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b13fba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b0c106f20 0x7f5b0c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b13fba640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c111f80 con 0x7f5b0c1027a0 2024-09-11T23:56:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b13fba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c111e00 con 0x7f5b0c101df0 2024-09-11T23:56:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b13fba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c112100 con 0x7f5b0c106f20 2024-09-11T23:56:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b11d56640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b0c101df0 0x7f5b0c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b11d56640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b0c101df0 0x7f5b0c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37472/0 (socket says 172.21.15.62:37472) 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b11d56640 1 -- 172.21.15.62:0/397261697 learned_addr learned my addr 172.21.15.62:0/397261697 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b12557640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b0c106f20 0x7f5b0c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.276+0000 7f5b11555640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c1027a0 0x7f5b0c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.277+0000 7f5b11d56640 1 -- 172.21.15.62:0/397261697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b0c106f20 msgr2=0x7f5b0c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.277+0000 7f5b11d56640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b0c106f20 0x7f5b0c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.277+0000 7f5b11d56640 1 -- 172.21.15.62:0/397261697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c1027a0 msgr2=0x7f5b0c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.277+0000 7f5b11d56640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c1027a0 0x7f5b0c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.277+0000 7f5b11d56640 1 -- 172.21.15.62:0/397261697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c1a68a0 con 0x7f5b0c101df0 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.277+0000 7f5b11d56640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b0c101df0 0x7f5b0c0744f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800e9c0 tx=0x7f5b0800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.277+0000 7f5b02ffd640 1 -- 172.21.15.62:0/397261697 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800ccc0 con 0x7f5b0c101df0 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.277+0000 7f5b02ffd640 1 -- 172.21.15.62:0/397261697 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b08005500 con 0x7f5b0c101df0 2024-09-11T23:56:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.277+0000 7f5b02ffd640 1 -- 172.21.15.62:0/397261697 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08010430 con 0x7f5b0c101df0 2024-09-11T23:56:28.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.279+0000 7f5b13fba640 1 -- 172.21.15.62:0/397261697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0c1a6b90 con 0x7f5b0c101df0 2024-09-11T23:56:28.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.279+0000 7f5b13fba640 1 -- 172.21.15.62:0/397261697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0c1a7050 con 0x7f5b0c101df0 2024-09-11T23:56:28.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.281+0000 7f5b02ffd640 1 -- 172.21.15.62:0/397261697 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5b080046c0 con 0x7f5b0c101df0 2024-09-11T23:56:28.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.282+0000 7f5b02ffd640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5af80a2d70 0x7f5af80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.282+0000 7f5b02ffd640 1 -- 172.21.15.62:0/397261697 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f5b080cf710 con 0x7f5b0c101df0 2024-09-11T23:56:28.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.282+0000 7f5b11555640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5af80a2d70 0x7f5af80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.282+0000 7f5b16393740 1 Processor -- start 2024-09-11T23:56:28.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b16393740 1 -- start start 2024-09-11T23:56:28.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b11555640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5af80a2d70 0x7f5af80a5230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c075f80 tx=0x7f5afc00b410 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:28.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b16393740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 0x5650b7882c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b16393740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 0x5650b788d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b16393740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 0x5650b788fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b16393740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650b77e7160 con 0x5650b7883170 2024-09-11T23:56:28.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b16393740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5650b778e720 con 0x5650b7882850 2024-09-11T23:56:28.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b16393740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650b77e3120 con 0x5650b788da30 2024-09-11T23:56:28.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b12557640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 0x5650b788fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b12557640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 0x5650b788fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:59014/0 (socket says 172.21.15.62:59014) 2024-09-11T23:56:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b12557640 1 -- 172.21.15.62:0/1160558228 learned_addr learned my addr 172.21.15.62:0/1160558228 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b11d56640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 0x5650b7882c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b11555640 1 --2- 172.21.15.62:0/1160558228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 0x5650b788d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b12557640 1 -- 172.21.15.62:0/1160558228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 msgr2=0x5650b7882c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.283+0000 7f5b12557640 1 --2- 172.21.15.62:0/1160558228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 0x5650b7882c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.284+0000 7f5b12557640 1 -- 172.21.15.62:0/1160558228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 msgr2=0x5650b788d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.284+0000 7f5b12557640 1 --2- 172.21.15.62:0/1160558228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 0x5650b788d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.284+0000 7f5b12557640 1 -- 172.21.15.62:0/1160558228 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b7558ba0 con 0x5650b788da30 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.284+0000 7f5b12557640 1 --2- 172.21.15.62:0/1160558228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 0x5650b788fe20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009be0 tx=0x7f5b040314d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9aee8d54db7abef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.284+0000 7f5af7fff640 1 -- 172.21.15.62:0/1160558228 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04041030 con 0x5650b788da30 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.284+0000 7f5af7fff640 1 -- 172.21.15.62:0/1160558228 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b040303a0 con 0x5650b788da30 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.284+0000 7f5af7fff640 1 -- 172.21.15.62:0/1160558228 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04036040 con 0x5650b788da30 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.285+0000 7f5b16393740 1 -- 172.21.15.62:0/1160558228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 msgr2=0x5650b788fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.285+0000 7f5b16393740 1 --2- 172.21.15.62:0/1160558228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 0x5650b788fe20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009be0 tx=0x7f5b040314d0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.285+0000 7f5b16393740 1 -- 172.21.15.62:0/1160558228 shutdown_connections 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.285+0000 7f5b16393740 1 --2- 172.21.15.62:0/1160558228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 0x5650b788fe20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.285+0000 7f5b16393740 1 --2- 172.21.15.62:0/1160558228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 0x5650b788d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.285+0000 7f5b16393740 1 --2- 172.21.15.62:0/1160558228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 0x5650b7882c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.285+0000 7f5b16393740 1 -- 172.21.15.62:0/1160558228 >> 172.21.15.62:0/1160558228 conn(0x5650b787ede0 msgr2=0x5650b787f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.285+0000 7f5b16393740 1 -- 172.21.15.62:0/1160558228 shutdown_connections 2024-09-11T23:56:28.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.285+0000 7f5b16393740 1 -- 172.21.15.62:0/1160558228 wait complete. 2024-09-11T23:56:28.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.286+0000 7f5b16393740 1 -- start start 2024-09-11T23:56:28.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.286+0000 7f5b16393740 1 Processor -- start 2024-09-11T23:56:28.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b16393740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7882850 0x5650b788a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b16393740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7883170 0x5650b788ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b16393740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b788da30 0x5650b788b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b16393740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650b7890600 con 0x5650b788da30 2024-09-11T23:56:28.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b11d56640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7882850 0x5650b788a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b11d56640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7882850 0x5650b788a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:59026/0 (socket says 172.21.15.62:59026) 2024-09-11T23:56:28.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b11d56640 1 -- 172.21.15.62:0/462916566 learned_addr learned my addr 172.21.15.62:0/462916566 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b16393740 1 -- 172.21.15.62:0/462916566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5650b776f900 con 0x5650b7883170 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b16393740 1 -- 172.21.15.62:0/462916566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650b7890780 con 0x5650b7882850 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b11555640 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7883170 0x5650b788ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b12557640 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b788da30 0x5650b788b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b16393740 10 client.0 _ll_register_callbacks cb 0x5650b776d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b11d56640 1 -- 172.21.15.62:0/462916566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7883170 msgr2=0x5650b788ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b11d56640 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7883170 0x5650b788ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b11d56640 1 -- 172.21.15.62:0/462916566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b788da30 msgr2=0x5650b788b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b11d56640 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b788da30 0x5650b788b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b11d56640 1 -- 172.21.15.62:0/462916566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b77afc50 con 0x5650b7882850 2024-09-11T23:56:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.287+0000 7f5b16393740 1 -- 172.21.15.62:0/462916566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b040303a0 con 0x5650b7882850 2024-09-11T23:56:28.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.288+0000 7f5b11d56640 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7882850 0x5650b788a900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c6a0 tx=0x7f5b0809c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a9af4345bc972b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:28.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.288+0000 7f5af57fa640 10 client.25738 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:56:28.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.288+0000 7f5af57fa640 1 -- 172.21.15.62:0/462916566 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b080b4e60 con 0x5650b7882850 2024-09-11T23:56:28.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.288+0000 7f5af57fa640 1 -- 172.21.15.62:0/462916566 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f5b0809b0b0 con 0x5650b7882850 2024-09-11T23:56:28.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.288+0000 7f5af57fa640 1 -- 172.21.15.62:0/462916566 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b0809b310 con 0x5650b7882850 2024-09-11T23:56:28.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.288+0000 7f5b16393740 10 client.25738 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.288+0000 7f5b16393740 1 -- 172.21.15.62:0/462916566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5650b789c2e0 con 0x5650b7882850 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.289+0000 7f5af57fa640 1 -- 172.21.15.62:0/462916566 <== mon.1 v2:172.21.15.120:3300/0 4 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f5b080a4b20 con 0x5650b7882850 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.289+0000 7f5b16393740 10 client.25738 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.290+0000 7f5b16393740 10 client.25738 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.290+0000 7f5b16393740 20 client.25738 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.290+0000 7f5b16393740 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5650b789d690 0x5650b789fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.290+0000 7f5b16393740 4 client.25738 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.290+0000 7f5b16393740 1 -- 172.21.15.62:0/462916566 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b77f4890 con 0x5650b789d690 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.291+0000 7f5b11555640 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5650b789d690 0x5650b789fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.291+0000 7f5b11555640 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5650b789d690 0x5650b789fa80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d658b3675b439b server_cookie=f922a3ecbde97b32 in_seq=0 out_seq=0 2024-09-11T23:56:28.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.291+0000 7f5af57fa640 10 client.25738 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5af57fa640 1 -- 172.21.15.62:0/462916566 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650b77f4890 con 0x5650b789d690 2024-09-11T23:56:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5af57fa640 10 client.25738 handle_command_reply: tid=0 2024-09-11T23:56:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 client.25738 shutdown 2024-09-11T23:56:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 -- 172.21.15.62:0/462916566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7882850 msgr2=0x5650b788a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7882850 0x5650b788a900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c6a0 tx=0x7f5b0809c420 comp rx=0 tx=0).stop 2024-09-11T23:56:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 -- 172.21.15.62:0/462916566 shutdown_connections 2024-09-11T23:56:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5650b789d690 0x5650b789fa80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5af57fa640 0 client.25738 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b788da30 0x5650b788b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7883170 0x5650b788ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 --2- 172.21.15.62:0/462916566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7882850 0x5650b788a900 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 -- 172.21.15.62:0/462916566 >> 172.21.15.62:0/462916566 conn(0x5650b787ede0 msgr2=0x5650b7884b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:28.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 -- 172.21.15.62:0/462916566 shutdown_connections 2024-09-11T23:56:28.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.292+0000 7f5b16393740 1 -- 172.21.15.62:0/462916566 wait complete. 2024-09-11T23:56:28.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.293+0000 7f5b16393740 20 client.25738 trim_cache size 0 max 16384 2024-09-11T23:56:28.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b16393740 1 Processor -- start 2024-09-11T23:56:28.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b16393740 1 -- start start 2024-09-11T23:56:28.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b16393740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 0x5650b7884960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b16393740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 0x5650b788b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b16393740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 0x5650b78a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b16393740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1adf00 con 0x5650b7883170 2024-09-11T23:56:28.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b16393740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1ab7d0 con 0x5650b7882850 2024-09-11T23:56:28.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b16393740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1a9620 con 0x5650b788da30 2024-09-11T23:56:28.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b11d56640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 0x5650b7884960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b11d56640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 0x5650b7884960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37506/0 (socket says 172.21.15.62:37506) 2024-09-11T23:56:28.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b11555640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 0x5650b788b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b11d56640 1 -- 172.21.15.62:0/2628137204 learned_addr learned my addr 172.21.15.62:0/2628137204 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:28.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.315+0000 7f5b12557640 1 --2- 172.21.15.62:0/2628137204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 0x5650b78a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.316+0000 7f5b11555640 1 -- 172.21.15.62:0/2628137204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 msgr2=0x5650b78a52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:28.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.316+0000 7f5b11555640 1 --2- 172.21.15.62:0/2628137204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 0x5650b78a52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.316+0000 7f5b11555640 1 -- 172.21.15.62:0/2628137204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 msgr2=0x5650b7884960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:28.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.316+0000 7f5b11555640 1 --2- 172.21.15.62:0/2628137204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 0x5650b7884960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.316+0000 7f5b11555640 1 -- 172.21.15.62:0/2628137204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b787ec10 con 0x5650b7883170 2024-09-11T23:56:28.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.316+0000 7f5b11555640 1 --2- 172.21.15.62:0/2628137204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 0x5650b788b1d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5650b788bf40 tx=0x7f5afc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf1b3dd36eb789e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:28.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.317+0000 7f5af77fe640 1 -- 172.21.15.62:0/2628137204 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc02e070 con 0x5650b7883170 2024-09-11T23:56:28.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.317+0000 7f5af77fe640 1 -- 172.21.15.62:0/2628137204 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5afc0211d0 con 0x5650b7883170 2024-09-11T23:56:28.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.317+0000 7f5af77fe640 1 -- 172.21.15.62:0/2628137204 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc011680 con 0x5650b7883170 2024-09-11T23:56:28.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.317+0000 7f5b16393740 1 -- 172.21.15.62:0/2628137204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 msgr2=0x5650b788b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:28.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.317+0000 7f5b16393740 1 --2- 172.21.15.62:0/2628137204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 0x5650b788b1d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5650b788bf40 tx=0x7f5afc013000 comp rx=0 tx=0).stop 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.317+0000 7f5b16393740 1 -- 172.21.15.62:0/2628137204 shutdown_connections 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.317+0000 7f5b16393740 1 --2- 172.21.15.62:0/2628137204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b788da30 0x5650b78a52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.317+0000 7f5b16393740 1 --2- 172.21.15.62:0/2628137204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7883170 0x5650b788b1d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.317+0000 7f5b16393740 1 --2- 172.21.15.62:0/2628137204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b7882850 0x5650b7884960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.317+0000 7f5b16393740 1 -- 172.21.15.62:0/2628137204 >> 172.21.15.62:0/2628137204 conn(0x5650b787ede0 msgr2=0x5650b787dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 1 -- 172.21.15.62:0/2628137204 shutdown_connections 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 1 -- 172.21.15.62:0/2628137204 wait complete. 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 1 -- start start 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 1 Processor -- start 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7882850 0x5650b7891f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7883170 0x5650b78924a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b788da30 0x5650b79072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1a9c90 con 0x5650b7882850 2024-09-11T23:56:28.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1ab100 con 0x5650b788da30 2024-09-11T23:56:28.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1aa360 con 0x5650b7883170 2024-09-11T23:56:28.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.318+0000 7f5b16393740 10 client.0 _ll_register_callbacks cb 0x7f5b08097f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:28.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b11d56640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7882850 0x5650b7891f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b12557640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b788da30 0x5650b79072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b11d56640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7882850 0x5650b7891f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35922/0 (socket says 172.21.15.62:35922) 2024-09-11T23:56:28.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b12557640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b788da30 0x5650b79072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37516/0 (socket says 172.21.15.62:37516) 2024-09-11T23:56:28.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b11d56640 1 -- 172.21.15.62:0/2398569371 learned_addr learned my addr 172.21.15.62:0/2398569371 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:28.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b11555640 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7883170 0x5650b78924a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b12557640 1 -- 172.21.15.62:0/2398569371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7883170 msgr2=0x5650b78924a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:28.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b12557640 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7883170 0x5650b78924a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b12557640 1 -- 172.21.15.62:0/2398569371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7882850 msgr2=0x5650b7891f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:28.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b12557640 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7882850 0x5650b7891f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b12557640 1 -- 172.21.15.62:0/2398569371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b787a3f0 con 0x5650b788da30 2024-09-11T23:56:28.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b11d56640 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7882850 0x5650b7891f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:56:28.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b11555640 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7883170 0x5650b78924a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:56:28.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b12557640 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b788da30 0x5650b79072b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b04004af0 tx=0x7f5b04003b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c0058efc59ef0f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:28.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5af57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5af57fa640 1 -- 172.21.15.62:0/2398569371 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04041040 con 0x5650b788da30 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5af57fa640 1 -- 172.21.15.62:0/2398569371 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f5b04030690 con 0x5650b788da30 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.319+0000 7f5b16393740 1 -- 172.21.15.62:0/2398569371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5afc0211d0 con 0x5650b788da30 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.320+0000 7f5af57fa640 1 -- 172.21.15.62:0/2398569371 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b04036040 con 0x5650b788da30 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.320+0000 7f5b16393740 10 client.25811 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.320+0000 7f5b16393740 1 -- 172.21.15.62:0/2398569371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650b7908990 con 0x5650b788da30 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.320+0000 7f5af57fa640 1 -- 172.21.15.62:0/2398569371 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5afc0211d0 con 0x5650b788da30 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.320+0000 7f5af57fa640 1 -- 172.21.15.62:0/2398569371 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f5b0403d460 con 0x5650b788da30 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.321+0000 7f5b16393740 10 client.25811 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.321+0000 7f5b16393740 10 client.25811 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.321+0000 7f5b16393740 20 client.25811 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.321+0000 7f5b16393740 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5650b789d690 0x5650b790abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:28.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.321+0000 7f5b11d56640 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5650b789d690 0x5650b790abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:28.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.321+0000 7f5b16393740 4 client.25811 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T23:56:28.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.321+0000 7f5b16393740 1 -- 172.21.15.62:0/2398569371 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x5650b77eebe0 con 0x5650b789d690 2024-09-11T23:56:28.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.322+0000 7f5b11d56640 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5650b789d690 0x5650b790abd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a4e40d565d3eb8 server_cookie=56ef462b9213ead8 in_seq=0 out_seq=0 2024-09-11T23:56:28.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.322+0000 7f5af57fa640 10 client.25811 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:28.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.557+0000 7f5af57fa640 1 -- 172.21.15.62:0/2398569371 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5650b77eebe0 con 0x5650b789d690 2024-09-11T23:56:28.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5af57fa640 10 client.25811 handle_command_reply: tid=0 2024-09-11T23:56:28.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 client.25811 shutdown 2024-09-11T23:56:28.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 -- 172.21.15.62:0/2398569371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b788da30 msgr2=0x5650b79072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:28.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b788da30 0x5650b79072b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b04004af0 tx=0x7f5b04003b10 comp rx=0 tx=0).stop 2024-09-11T23:56:28.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 -- 172.21.15.62:0/2398569371 shutdown_connections 2024-09-11T23:56:28.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5650b789d690 0x5650b790abd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650b788da30 0x5650b79072b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650b7883170 0x5650b78924a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 --2- 172.21.15.62:0/2398569371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650b7882850 0x5650b7891f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5af57fa640 0 client.25811 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:28.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 -- 172.21.15.62:0/2398569371 >> 172.21.15.62:0/2398569371 conn(0x5650b787ede0 msgr2=0x5650b787dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:28.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 -- 172.21.15.62:0/2398569371 shutdown_connections 2024-09-11T23:56:28.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 1 -- 172.21.15.62:0/2398569371 wait complete. 2024-09-11T23:56:28.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.558+0000 7f5b16393740 20 client.25811 trim_cache size 0 max 16384 2024-09-11T23:56:28.559 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 29, 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 18, 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "a13053d1": { 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 18, 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:56:28.560 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 140.2, 2024-09-11T23:56:28.561 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:56:28.561 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:56:28.561 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:56:28.561 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:56:28.561 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:56:28.561 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:56:28.561 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:28.561 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:28.565 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:28.565 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:28.565 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:28.565 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:56:28.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 -- 172.21.15.62:0/397261697 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5af80a2d70 msgr2=0x7f5af80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5af80a2d70 0x7f5af80a5230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c075f80 tx=0x7f5afc00b410 comp rx=0 tx=0).stop 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 -- 172.21.15.62:0/397261697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b0c101df0 msgr2=0x7f5b0c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b0c101df0 0x7f5b0c0744f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800e9c0 tx=0x7f5b0800ee80 comp rx=0 tx=0).stop 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 -- 172.21.15.62:0/397261697 shutdown_connections 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5af80a2d70 0x7f5af80a5230 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b0c106f20 0x7f5b0c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c1027a0 0x7f5b0c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 --2- 172.21.15.62:0/397261697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b0c101df0 0x7f5b0c0744f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 -- 172.21.15.62:0/397261697 >> 172.21.15.62:0/397261697 conn(0x7f5b0c0fd8f0 msgr2=0x7f5b0c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 -- 172.21.15.62:0/397261697 shutdown_connections 2024-09-11T23:56:28.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:28.559+0000 7f5b13fba640 1 -- 172.21.15.62:0/397261697 wait complete. 2024-09-11T23:56:29.263 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a13053d1', quiesce took 4.0 seconds. Will release after: 16.15 2024-09-11T23:56:29.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:29 smithi120 ceph-mon[35182]: pgmap v134: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-09-11T23:56:29.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:29 smithi083 ceph-mon[34805]: pgmap v134: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-09-11T23:56:29.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:29 smithi062 ceph-mon[30958]: pgmap v134: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-09-11T23:56:31.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:31 smithi062 ceph-mon[30958]: pgmap v135: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-09-11T23:56:31.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:31 smithi120 ceph-mon[35182]: pgmap v135: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-09-11T23:56:31.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:31 smithi083 ceph-mon[34805]: pgmap v135: 97 pgs: 97 active+clean; 2.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-09-11T23:56:32.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 230.0 s, 595.9 tps, lat 29.950 ms stddev 31.095 2024-09-11T23:56:33.592 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:56:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:33 smithi120 ceph-mon[35182]: pgmap v136: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 201 op/s 2024-09-11T23:56:33.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:33 smithi062 ceph-mon[30958]: pgmap v136: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 201 op/s 2024-09-11T23:56:33.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:33 smithi083 ceph-mon[34805]: pgmap v136: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 201 op/s 2024-09-11T23:56:33.866 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.338+0000 7f6052cdd640 1 -- 172.21.15.62:0/656729560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c100a70 msgr2=0x7f604c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.338+0000 7f6052cdd640 1 --2- 172.21.15.62:0/656729560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c100a70 0x7f604c100ed0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6038009930 tx=0x7f603802a420 comp rx=0 tx=0).stop 2024-09-11T23:56:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.338+0000 7f6052cdd640 1 -- 172.21.15.62:0/656729560 shutdown_connections 2024-09-11T23:56:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.338+0000 7f6052cdd640 1 --2- 172.21.15.62:0/656729560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604c101410 0x7f604c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.338+0000 7f6052cdd640 1 --2- 172.21.15.62:0/656729560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c100a70 0x7f604c100ed0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.338+0000 7f6052cdd640 1 --2- 172.21.15.62:0/656729560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c106a70 0x7f604c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.338+0000 7f6052cdd640 1 -- 172.21.15.62:0/656729560 >> 172.21.15.62:0/656729560 conn(0x7f604c0fc790 msgr2=0x7f604c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.338+0000 7f6052cdd640 1 -- 172.21.15.62:0/656729560 shutdown_connections 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.338+0000 7f6052cdd640 1 -- 172.21.15.62:0/656729560 wait complete. 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.338+0000 7f6052cdd640 1 Processor -- start 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6052cdd640 1 -- start start 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6052cdd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c100a70 0x7f604c0fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6052cdd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604c101410 0x7f604c0fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6052cdd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c106a70 0x7f604c0fd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6052cdd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f604c110e70 con 0x7f604c100a70 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6052cdd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f604c110cf0 con 0x7f604c101410 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6052cdd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f604c110ff0 con 0x7f604c106a70 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6050a79640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c100a70 0x7f604c0fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6050a79640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c100a70 0x7f604c0fcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59824/0 (socket says 172.21.15.62:59824) 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6050a79640 1 -- 172.21.15.62:0/4248483965 learned_addr learned my addr 172.21.15.62:0/4248483965 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f604bfff640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604c101410 0x7f604c0fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f605127a640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c106a70 0x7f604c0fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6050a79640 1 -- 172.21.15.62:0/4248483965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c106a70 msgr2=0x7f604c0fd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6050a79640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c106a70 0x7f604c0fd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6050a79640 1 -- 172.21.15.62:0/4248483965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604c101410 msgr2=0x7f604c0fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6050a79640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604c101410 0x7f604c0fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6050a79640 1 -- 172.21.15.62:0/4248483965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c1a6930 con 0x7f604c100a70 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f604bfff640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604c101410 0x7f604c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.339+0000 7f6050a79640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c100a70 0x7f604c0fcec0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f604000c970 tx=0x7f604000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.340+0000 7f6049ffb640 1 -- 172.21.15.62:0/4248483965 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040007bb0 con 0x7f604c100a70 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.340+0000 7f6049ffb640 1 -- 172.21.15.62:0/4248483965 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6040007d50 con 0x7f604c100a70 2024-09-11T23:56:34.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.340+0000 7f6052cdd640 1 -- 172.21.15.62:0/4248483965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c1a6c20 con 0x7f604c100a70 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.340+0000 7f6049ffb640 1 -- 172.21.15.62:0/4248483965 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604000f450 con 0x7f604c100a70 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.340+0000 7f6052cdd640 1 -- 172.21.15.62:0/4248483965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c1a70e0 con 0x7f604c100a70 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.342+0000 7f6049ffb640 1 -- 172.21.15.62:0/4248483965 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6040005020 con 0x7f604c100a70 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f60550b6740 1 Processor -- start 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f60550b6740 1 -- start start 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f60550b6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c8194a0 0x56539c819880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f60550b6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c819dc0 0x56539c824140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f60550b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c824680 0x56539c826a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f60550b6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56539c77f2e0 con 0x56539c819dc0 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f60550b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56539c726ce0 con 0x56539c824680 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f60550b6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56539c77b2a0 con 0x56539c8194a0 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f6049ffb640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f603c0a2cf0 0x7f603c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f6050a79640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c819dc0 0x56539c824140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f6050a79640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c819dc0 0x56539c824140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59840/0 (socket says 172.21.15.62:59840) 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f6050a79640 1 -- 172.21.15.62:0/1819139178 learned_addr learned my addr 172.21.15.62:0/1819139178 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.343+0000 7f605127a640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f603c0a2cf0 0x7f603c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f605127a640 1 --2- 172.21.15.62:0/1819139178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c8194a0 0x56539c819880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f604bfff640 1 --2- 172.21.15.62:0/1819139178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c824680 0x56539c826a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f6049ffb640 1 -- 172.21.15.62:0/4248483965 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f60400cf2c0 con 0x7f604c100a70 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f6050a79640 1 -- 172.21.15.62:0/1819139178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c8194a0 msgr2=0x56539c819880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f6050a79640 1 --2- 172.21.15.62:0/1819139178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c8194a0 0x56539c819880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f6050a79640 1 -- 172.21.15.62:0/1819139178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c824680 msgr2=0x56539c826a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f6050a79640 1 --2- 172.21.15.62:0/1819139178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c824680 0x56539c826a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f6050a79640 1 -- 172.21.15.62:0/1819139178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56539c70b440 con 0x56539c819dc0 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f605127a640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f603c0a2cf0 0x7f603c0a51b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6044005c40 tx=0x7f604400ac90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f6050a79640 1 --2- 172.21.15.62:0/1819139178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c819dc0 0x56539c824140 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6040010a50 tx=0x7f604009b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5bbfd7a5b46ef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f6036ffd640 1 -- 172.21.15.62:0/1819139178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604001d070 con 0x56539c819dc0 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f6036ffd640 1 -- 172.21.15.62:0/1819139178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f604009beb0 con 0x56539c819dc0 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.344+0000 7f6036ffd640 1 -- 172.21.15.62:0/1819139178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040086070 con 0x56539c819dc0 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 -- 172.21.15.62:0/1819139178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c819dc0 msgr2=0x56539c824140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 --2- 172.21.15.62:0/1819139178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c819dc0 0x56539c824140 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6040010a50 tx=0x7f604009b630 comp rx=0 tx=0).stop 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 -- 172.21.15.62:0/1819139178 shutdown_connections 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 --2- 172.21.15.62:0/1819139178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c824680 0x56539c826a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 --2- 172.21.15.62:0/1819139178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c819dc0 0x56539c824140 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 --2- 172.21.15.62:0/1819139178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c8194a0 0x56539c819880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 -- 172.21.15.62:0/1819139178 >> 172.21.15.62:0/1819139178 conn(0x56539c815a30 msgr2=0x56539c815e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 -- 172.21.15.62:0/1819139178 shutdown_connections 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 -- 172.21.15.62:0/1819139178 wait complete. 2024-09-11T23:56:34.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 -- start start 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.345+0000 7f60550b6740 1 Processor -- start 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f60550b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c8194a0 0x56539c823ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f60550b6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c819dc0 0x56539c8212a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f60550b6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c824680 0x56539c821800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f60550b6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56539c827250 con 0x56539c824680 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f60550b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56539c73a050 con 0x56539c8194a0 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f60550b6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56539c8273d0 con 0x56539c819dc0 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f60550b6740 10 client.0 _ll_register_callbacks cb 0x56539c77a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f605127a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c824680 0x56539c821800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f605127a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c824680 0x56539c821800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59844/0 (socket says 172.21.15.62:59844) 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f605127a640 1 -- 172.21.15.62:0/3721594857 learned_addr learned my addr 172.21.15.62:0/3721594857 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f6050a79640 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c8194a0 0x56539c823ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f604bfff640 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c819dc0 0x56539c8212a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f605127a640 1 -- 172.21.15.62:0/3721594857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c819dc0 msgr2=0x56539c8212a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f605127a640 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c819dc0 0x56539c8212a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f605127a640 1 -- 172.21.15.62:0/3721594857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c8194a0 msgr2=0x56539c823ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f605127a640 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c8194a0 0x56539c823ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.346+0000 7f605127a640 1 -- 172.21.15.62:0/3721594857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56539c4f03c0 con 0x56539c824680 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.347+0000 7f605127a640 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c824680 0x56539c821800 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f604400af50 tx=0x7f6044005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc75ecded24f63f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.347+0000 7f602bfff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.347+0000 7f602bfff640 1 -- 172.21.15.62:0/3721594857 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044026070 con 0x56539c824680 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.347+0000 7f602bfff640 1 -- 172.21.15.62:0/3721594857 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6044004d20 con 0x56539c824680 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.347+0000 7f60550b6740 1 -- 172.21.15.62:0/3721594857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604009beb0 con 0x56539c824680 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.347+0000 7f602bfff640 1 -- 172.21.15.62:0/3721594857 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6044031050 con 0x56539c824680 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.347+0000 7f60550b6740 10 client.16164 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.347+0000 7f60550b6740 1 -- 172.21.15.62:0/3721594857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56539c832c20 con 0x56539c824680 2024-09-11T23:56:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.347+0000 7f602bfff640 1 -- 172.21.15.62:0/3721594857 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f604009beb0 con 0x56539c824680 2024-09-11T23:56:34.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.347+0000 7f602bfff640 1 -- 172.21.15.62:0/3721594857 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f60440205a0 con 0x56539c824680 2024-09-11T23:56:34.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.348+0000 7f60550b6740 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:34.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.348+0000 7f60550b6740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:34.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.348+0000 7f60550b6740 20 client.16164 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:34.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.348+0000 7f60550b6740 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56539c834030 0x56539c836490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.348+0000 7f60550b6740 4 client.16164 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:34.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.348+0000 7f60550b6740 1 -- 172.21.15.62:0/3721594857 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56539c78c850 con 0x56539c834030 2024-09-11T23:56:34.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.348+0000 7f6050a79640 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56539c834030 0x56539c836490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.348+0000 7f6050a79640 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56539c834030 0x56539c836490 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47df58d0112d7ad server_cookie=b02fc2a9de6e050e in_seq=0 out_seq=0 2024-09-11T23:56:34.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.348+0000 7f602bfff640 10 client.16164 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:34.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f602bfff640 1 -- 172.21.15.62:0/3721594857 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56539c78c850 con 0x56539c834030 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f602bfff640 10 client.16164 handle_command_reply: tid=0 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f60550b6740 1 client.16164 shutdown 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f60550b6740 1 -- 172.21.15.62:0/3721594857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c824680 msgr2=0x56539c821800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f60550b6740 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c824680 0x56539c821800 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f604400af50 tx=0x7f6044005bd0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f60550b6740 1 -- 172.21.15.62:0/3721594857 shutdown_connections 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f60550b6740 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56539c834030 0x56539c836490 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f60550b6740 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c824680 0x56539c821800 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f60550b6740 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c819dc0 0x56539c8212a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f60550b6740 1 --2- 172.21.15.62:0/3721594857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c8194a0 0x56539c823ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f602bfff640 0 client.16164 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.349+0000 7f60550b6740 1 -- 172.21.15.62:0/3721594857 >> 172.21.15.62:0/3721594857 conn(0x56539c815a30 msgr2=0x56539c81b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.350+0000 7f60550b6740 1 -- 172.21.15.62:0/3721594857 shutdown_connections 2024-09-11T23:56:34.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.350+0000 7f60550b6740 1 -- 172.21.15.62:0/3721594857 wait complete. 2024-09-11T23:56:34.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.350+0000 7f60550b6740 20 client.16164 trim_cache size 0 max 16384 2024-09-11T23:56:34.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f60550b6740 1 Processor -- start 2024-09-11T23:56:34.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f60550b6740 1 -- start start 2024-09-11T23:56:34.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f60550b6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c823210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f60550b6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c819dc0 0x56539c83bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f60550b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c824680 0x56539c83c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f60550b6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b22a0 con 0x56539c8194a0 2024-09-11T23:56:34.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f60550b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1afb70 con 0x56539c824680 2024-09-11T23:56:34.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f60550b6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1ad9c0 con 0x56539c819dc0 2024-09-11T23:56:34.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f6050a79640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c823210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f6050a79640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c823210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59860/0 (socket says 172.21.15.62:59860) 2024-09-11T23:56:34.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f6050a79640 1 -- 172.21.15.62:0/2105483865 learned_addr learned my addr 172.21.15.62:0/2105483865 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:34.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f6050a79640 1 -- 172.21.15.62:0/2105483865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c819dc0 msgr2=0x56539c83bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:34.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f6050a79640 1 --2- 172.21.15.62:0/2105483865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c819dc0 0x56539c83bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f6050a79640 1 -- 172.21.15.62:0/2105483865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c824680 msgr2=0x56539c83c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:34.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f6050a79640 1 --2- 172.21.15.62:0/2105483865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c824680 0x56539c83c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f6050a79640 1 -- 172.21.15.62:0/2105483865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56539c836b10 con 0x56539c8194a0 2024-09-11T23:56:34.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.367+0000 7f6050a79640 1 --2- 172.21.15.62:0/2105483865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c823210 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f604009b4b0 tx=0x7f6040097a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6efb008a8609881 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:34.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60367fc640 1 -- 172.21.15.62:0/2105483865 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60400b4070 con 0x56539c8194a0 2024-09-11T23:56:34.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60367fc640 1 -- 172.21.15.62:0/2105483865 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6040099850 con 0x56539c8194a0 2024-09-11T23:56:34.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60367fc640 1 -- 172.21.15.62:0/2105483865 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604001d040 con 0x56539c8194a0 2024-09-11T23:56:34.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60550b6740 1 -- 172.21.15.62:0/2105483865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 msgr2=0x56539c823210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60550b6740 1 --2- 172.21.15.62:0/2105483865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c823210 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f604009b4b0 tx=0x7f6040097a50 comp rx=0 tx=0).stop 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60550b6740 1 -- 172.21.15.62:0/2105483865 shutdown_connections 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60550b6740 1 --2- 172.21.15.62:0/2105483865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c824680 0x56539c83c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60550b6740 1 --2- 172.21.15.62:0/2105483865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c819dc0 0x56539c83bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60550b6740 1 --2- 172.21.15.62:0/2105483865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c823210 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60550b6740 1 -- 172.21.15.62:0/2105483865 >> 172.21.15.62:0/2105483865 conn(0x56539c815a30 msgr2=0x56539c822b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60550b6740 1 -- 172.21.15.62:0/2105483865 shutdown_connections 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.368+0000 7f60550b6740 1 -- 172.21.15.62:0/2105483865 wait complete. 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f60550b6740 1 -- start start 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f60550b6740 1 Processor -- start 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f60550b6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c82ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f60550b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c819dc0 0x56539c82a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f60550b6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c824680 0x56539c82a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f60550b6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1ae030 con 0x56539c8194a0 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f60550b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1af4a0 con 0x56539c819dc0 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f60550b6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1ae700 con 0x56539c824680 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f60550b6740 10 client.0 _ll_register_callbacks cb 0x7f6044010cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f6050a79640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c82ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f6050a79640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c82ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59866/0 (socket says 172.21.15.62:59866) 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f6050a79640 1 -- 172.21.15.62:0/1829726239 learned_addr learned my addr 172.21.15.62:0/1829726239 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f605127a640 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c824680 0x56539c82a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.369+0000 7f604bfff640 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c819dc0 0x56539c82a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f6050a79640 1 -- 172.21.15.62:0/1829726239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c824680 msgr2=0x56539c82a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f6050a79640 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c824680 0x56539c82a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f6050a79640 1 -- 172.21.15.62:0/1829726239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c819dc0 msgr2=0x56539c82a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f6050a79640 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c819dc0 0x56539c82a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f6050a79640 1 -- 172.21.15.62:0/1829726239 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56539c82aee0 con 0x56539c8194a0 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f6050a79640 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c82ce00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6040086010 tx=0x7f60400a44b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0d1f4ffcd252c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f602bfff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f602bfff640 1 -- 172.21.15.62:0/1829726239 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040066060 con 0x56539c8194a0 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f602bfff640 1 -- 172.21.15.62:0/1829726239 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f60400ba170 con 0x56539c8194a0 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f60550b6740 1 -- 172.21.15.62:0/1829726239 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6040099850 con 0x56539c8194a0 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f602bfff640 1 -- 172.21.15.62:0/1829726239 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60400b4050 con 0x56539c8194a0 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f60550b6740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f60550b6740 1 -- 172.21.15.62:0/1829726239 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56539c89ee30 con 0x56539c8194a0 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f602bfff640 1 -- 172.21.15.62:0/1829726239 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6040099850 con 0x56539c8194a0 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.370+0000 7f602bfff640 1 -- 172.21.15.62:0/1829726239 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f60400ccc60 con 0x56539c8194a0 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f60550b6740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f60550b6740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:34.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f60550b6740 20 client.16170 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f60550b6740 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56539c89f110 0x56539c8a1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f60550b6740 4 client.16170 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f60550b6740 1 -- 172.21.15.62:0/1829726239 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56539c786d60 con 0x56539c89f110 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f604bfff640 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56539c89f110 0x56539c8a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f604bfff640 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56539c89f110 0x56539c8a1500 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230b3bece0e9ff1b server_cookie=9ecb91af92679b4e in_seq=0 out_seq=0 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f602bfff640 10 client.16170 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f602bfff640 1 -- 172.21.15.62:0/1829726239 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56539c786d60 con 0x56539c89f110 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f602bfff640 10 client.16170 handle_command_reply: tid=0 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.371+0000 7f60550b6740 1 client.16170 shutdown 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 1 -- 172.21.15.62:0/1829726239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 msgr2=0x56539c82ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c82ce00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6040086010 tx=0x7f60400a44b0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 1 -- 172.21.15.62:0/1829726239 shutdown_connections 2024-09-11T23:56:34.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56539c89f110 0x56539c8a1500 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56539c824680 0x56539c82a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56539c819dc0 0x56539c82a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 1 --2- 172.21.15.62:0/1829726239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56539c8194a0 0x56539c82ce00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 1 -- 172.21.15.62:0/1829726239 >> 172.21.15.62:0/1829726239 conn(0x56539c815a30 msgr2=0x56539c823320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:34.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f602bfff640 0 client.16170 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:34.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 1 -- 172.21.15.62:0/1829726239 shutdown_connections 2024-09-11T23:56:34.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 1 -- 172.21.15.62:0/1829726239 wait complete. 2024-09-11T23:56:34.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f60550b6740 20 client.16170 trim_cache size 0 max 16384 2024-09-11T23:56:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:56:34.378 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-11T23:56:34.378 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-11T23:56:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:56:34.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f6052cdd640 1 -- 172.21.15.62:0/4248483965 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f603c0a2cf0 msgr2=0x7f603c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:34.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f6052cdd640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f603c0a2cf0 0x7f603c0a51b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6044005c40 tx=0x7f604400ac90 comp rx=0 tx=0).stop 2024-09-11T23:56:34.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f6052cdd640 1 -- 172.21.15.62:0/4248483965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c100a70 msgr2=0x7f604c0fcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:34.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.372+0000 7f6052cdd640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c100a70 0x7f604c0fcec0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f604000c970 tx=0x7f604000ce30 comp rx=0 tx=0).stop 2024-09-11T23:56:34.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.373+0000 7f6052cdd640 1 -- 172.21.15.62:0/4248483965 shutdown_connections 2024-09-11T23:56:34.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.373+0000 7f6052cdd640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f603c0a2cf0 0x7f603c0a51b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.373+0000 7f6052cdd640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c106a70 0x7f604c0fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.373+0000 7f6052cdd640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604c101410 0x7f604c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.373+0000 7f6052cdd640 1 --2- 172.21.15.62:0/4248483965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c100a70 0x7f604c0fcec0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:34.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.373+0000 7f6052cdd640 1 -- 172.21.15.62:0/4248483965 >> 172.21.15.62:0/4248483965 conn(0x7f604c0fc790 msgr2=0x7f604c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:34.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.373+0000 7f6052cdd640 1 -- 172.21.15.62:0/4248483965 shutdown_connections 2024-09-11T23:56:34.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:34.373+0000 7f6052cdd640 1 -- 172.21.15.62:0/4248483965 wait complete. 2024-09-11T23:56:35.164 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:56:35.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:56:35.461 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:35.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:35 smithi062 ceph-mon[30958]: pgmap v137: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.9 MiB/s wr, 272 op/s 2024-09-11T23:56:35.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:35 smithi120 ceph-mon[35182]: pgmap v137: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.9 MiB/s wr, 272 op/s 2024-09-11T23:56:35.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:35 smithi083 ceph-mon[34805]: pgmap v137: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.9 MiB/s wr, 272 op/s 2024-09-11T23:56:36.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.085+0000 7f293639f640 1 -- 172.21.15.62:0/1680085938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930101de0 msgr2=0x7f29301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:36.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.085+0000 7f293639f640 1 --2- 172.21.15.62:0/1680085938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930101de0 0x7f29301021c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2920009900 tx=0x7f292002a410 comp rx=0 tx=0).stop 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.085+0000 7f293639f640 1 -- 172.21.15.62:0/1680085938 shutdown_connections 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.085+0000 7f293639f640 1 --2- 172.21.15.62:0/1680085938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2930106f10 0x7f293010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.085+0000 7f293639f640 1 --2- 172.21.15.62:0/1680085938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2930102790 0x7f29301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.085+0000 7f293639f640 1 --2- 172.21.15.62:0/1680085938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930101de0 0x7f29301021c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.085+0000 7f293639f640 1 -- 172.21.15.62:0/1680085938 >> 172.21.15.62:0/1680085938 conn(0x7f29300fd8c0 msgr2=0x7f29300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.085+0000 7f293639f640 1 -- 172.21.15.62:0/1680085938 shutdown_connections 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.085+0000 7f293639f640 1 -- 172.21.15.62:0/1680085938 wait complete. 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.086+0000 7f293639f640 1 Processor -- start 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.086+0000 7f293639f640 1 -- start start 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.086+0000 7f293639f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930101de0 0x7f29301983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.086+0000 7f293639f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2930102790 0x7f2930198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.086+0000 7f293639f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2930106f10 0x7f293019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:36.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.086+0000 7f293639f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2930111f70 con 0x7f2930101de0 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.086+0000 7f293639f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2930111df0 con 0x7f2930106f10 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.086+0000 7f293639f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29301120f0 con 0x7f2930102790 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930101de0 0x7f29301983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930101de0 0x7f29301983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59888/0 (socket says 172.21.15.62:59888) 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292ffff640 1 -- 172.21.15.62:0/18012838 learned_addr learned my addr 172.21.15.62:0/18012838 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292f7fe640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2930102790 0x7f2930198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f293493c640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2930106f10 0x7f293019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292ffff640 1 -- 172.21.15.62:0/18012838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2930102790 msgr2=0x7f2930198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292ffff640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2930102790 0x7f2930198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292ffff640 1 -- 172.21.15.62:0/18012838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2930106f10 msgr2=0x7f293019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292ffff640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2930106f10 0x7f293019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292ffff640 1 -- 172.21.15.62:0/18012838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293019d380 con 0x7f2930101de0 2024-09-11T23:56:36.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f293493c640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2930106f10 0x7f293019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292ffff640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930101de0 0x7f29301983d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2920005ab0 tx=0x7f29200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292d7fa640 1 -- 172.21.15.62:0/18012838 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292003f030 con 0x7f2930101de0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292d7fa640 1 -- 172.21.15.62:0/18012838 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2920003ce0 con 0x7f2930101de0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f293639f640 1 -- 172.21.15.62:0/18012838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293019d610 con 0x7f2930101de0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f292d7fa640 1 -- 172.21.15.62:0/18012838 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920035070 con 0x7f2930101de0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.087+0000 7f293639f640 1 -- 172.21.15.62:0/18012838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29301a4f50 con 0x7f2930101de0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.089+0000 7f293639f640 1 -- 172.21.15.62:0/18012838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29301021c0 con 0x7f2930101de0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.091+0000 7f292d7fa640 1 -- 172.21.15.62:0/18012838 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2920042030 con 0x7f2930101de0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.091+0000 7f292d7fa640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f29180a2d70 0x7f29180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.091+0000 7f292f7fe640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f29180a2d70 0x7f29180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.091+0000 7f292d7fa640 1 -- 172.21.15.62:0/18012838 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f29200eff60 con 0x7f2930101de0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.092+0000 7f292f7fe640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f29180a2d70 0x7f29180a5230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f29300fdce0 tx=0x7f292400a4c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.095+0000 7f292d7fa640 1 -- 172.21.15.62:0/18012838 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29200bcbb0 con 0x7f2930101de0 2024-09-11T23:56:36.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.305+0000 7f293639f640 1 -- 172.21.15.62:0/18012838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2930199730 con 0x7f2930101de0 2024-09-11T23:56:36.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.306+0000 7f292d7fa640 1 -- 172.21.15.62:0/18012838 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7f29200bc4b0 con 0x7f2930101de0 2024-09-11T23:56:36.305 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:56:36.305 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"btime":"2024-09-11T23:55:51:325581+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:55:50.698407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:56:36.305 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-09-11T23:56:36.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 -- 172.21.15.62:0/18012838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f29180a2d70 msgr2=0x7f29180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f29180a2d70 0x7f29180a5230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f29300fdce0 tx=0x7f292400a4c0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 -- 172.21.15.62:0/18012838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930101de0 msgr2=0x7f29301983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930101de0 0x7f29301983d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2920005ab0 tx=0x7f29200042f0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 -- 172.21.15.62:0/18012838 shutdown_connections 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f29180a2d70 0x7f29180a5230 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2930106f10 0x7f293019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2930102790 0x7f2930198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 --2- 172.21.15.62:0/18012838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2930101de0 0x7f29301983d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 -- 172.21.15.62:0/18012838 >> 172.21.15.62:0/18012838 conn(0x7f29300fd8c0 msgr2=0x7f29300febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 -- 172.21.15.62:0/18012838 shutdown_connections 2024-09-11T23:56:36.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:36.308+0000 7f293639f640 1 -- 172.21.15.62:0/18012838 wait complete. 2024-09-11T23:56:37.104 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:56:37.365 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:37.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 235.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:56:37.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:37 smithi062 ceph-mon[30958]: pgmap v138: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-09-11T23:56:37.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:37 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/18012838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:37 smithi120 ceph-mon[35182]: pgmap v138: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-09-11T23:56:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:37 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/18012838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:37.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:37 smithi083 ceph-mon[34805]: pgmap v138: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-09-11T23:56:37.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:37 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/18012838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:37.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.844+0000 7fa9cf444640 1 -- 172.21.15.62:0/582654253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c8103740 msgr2=0x7fa9c8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.844+0000 7fa9cf444640 1 --2- 172.21.15.62:0/582654253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c8103740 0x7fa9c8107b80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009930 tx=0x7fa9bc02a420 comp rx=0 tx=0).stop 2024-09-11T23:56:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.844+0000 7fa9cf444640 1 -- 172.21.15.62:0/582654253 shutdown_connections 2024-09-11T23:56:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.844+0000 7fa9cf444640 1 --2- 172.21.15.62:0/582654253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c8108290 0x7fa9c810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.844+0000 7fa9cf444640 1 --2- 172.21.15.62:0/582654253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c8103740 0x7fa9c8107b80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.844+0000 7fa9cf444640 1 --2- 172.21.15.62:0/582654253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c8102d90 0x7fa9c8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.844+0000 7fa9cf444640 1 -- 172.21.15.62:0/582654253 >> 172.21.15.62:0/582654253 conn(0x7fa9c80fc7d0 msgr2=0x7fa9c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.844+0000 7fa9cf444640 1 -- 172.21.15.62:0/582654253 shutdown_connections 2024-09-11T23:56:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.844+0000 7fa9cf444640 1 -- 172.21.15.62:0/582654253 wait complete. 2024-09-11T23:56:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.844+0000 7fa9cf444640 1 Processor -- start 2024-09-11T23:56:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cf444640 1 -- start start 2024-09-11T23:56:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cf444640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c8102d90 0x7fa9c81a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cf444640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c8103740 0x7fa9c81a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cf444640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c8108290 0x7fa9c819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cf444640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c8112050 con 0x7fa9c8108290 2024-09-11T23:56:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cf444640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c8111ed0 con 0x7fa9c8102d90 2024-09-11T23:56:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cf444640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c81121d0 con 0x7fa9c8103740 2024-09-11T23:56:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd9e1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c8108290 0x7fa9c819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd9e1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c8108290 0x7fa9c819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59906/0 (socket says 172.21.15.62:59906) 2024-09-11T23:56:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd9e1640 1 -- 172.21.15.62:0/1442521972 learned_addr learned my addr 172.21.15.62:0/1442521972 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c8102d90 0x7fa9c81a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd9e1640 1 -- 172.21.15.62:0/1442521972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c8103740 msgr2=0x7fa9c81a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c8103740 0x7fa9c81a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd9e1640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c8103740 0x7fa9c81a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd9e1640 1 -- 172.21.15.62:0/1442521972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c8102d90 msgr2=0x7fa9c81a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd9e1640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c8102d90 0x7fa9c81a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd9e1640 1 -- 172.21.15.62:0/1442521972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c819aeb0 con 0x7fa9c8108290 2024-09-11T23:56:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c8102d90 0x7fa9c81a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:56:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.845+0000 7fa9cd9e1640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c8108290 0x7fa9c819a640 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800cad0 tx=0x7fa9b800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.846+0000 7fa9c67fc640 1 -- 172.21.15.62:0/1442521972 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b800ae20 con 0x7fa9c8108290 2024-09-11T23:56:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.846+0000 7fa9cf444640 1 -- 172.21.15.62:0/1442521972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c819b1a0 con 0x7fa9c8108290 2024-09-11T23:56:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.846+0000 7fa9c67fc640 1 -- 172.21.15.62:0/1442521972 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b8005020 con 0x7fa9c8108290 2024-09-11T23:56:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.846+0000 7fa9c67fc640 1 -- 172.21.15.62:0/1442521972 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b800f6f0 con 0x7fa9c8108290 2024-09-11T23:56:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.846+0000 7fa9cf444640 1 -- 172.21.15.62:0/1442521972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c81a74f0 con 0x7fa9c8108290 2024-09-11T23:56:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.848+0000 7fa9c67fc640 1 -- 172.21.15.62:0/1442521972 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa9b80051d0 con 0x7fa9c8108290 2024-09-11T23:56:37.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9d181d740 1 Processor -- start 2024-09-11T23:56:37.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9d181d740 1 -- start start 2024-09-11T23:56:37.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44fb40 0x55670f44e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f44ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f459370 0x55670f45b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9d181d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55670f3b42f0 con 0x55670f44e5d0 2024-09-11T23:56:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9d181d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55670f35bcf0 con 0x55670f44fb40 2024-09-11T23:56:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9d181d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55670f3b02b0 con 0x55670f459370 2024-09-11T23:56:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9cd1e0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f44ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9cd1e0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f44ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59918/0 (socket says 172.21.15.62:59918) 2024-09-11T23:56:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9cd1e0640 1 -- 172.21.15.62:0/4293992285 learned_addr learned my addr 172.21.15.62:0/4293992285 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/4293992285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44fb40 0x55670f44e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/4293992285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f459370 0x55670f45b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.849+0000 7fa9c67fc640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9b00a2cf0 0x7fa9b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9cd1e0640 1 -- 172.21.15.62:0/4293992285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f459370 msgr2=0x55670f45b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/4293992285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f459370 0x55670f45b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9cd1e0640 1 -- 172.21.15.62:0/4293992285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44fb40 msgr2=0x55670f44e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/4293992285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44fb40 0x55670f44e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9cd9e1640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9b00a2cf0 0x7fa9b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9cd1e0640 1 -- 172.21.15.62:0/4293992285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55670f3403a0 con 0x55670f44e5d0 2024-09-11T23:56:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/4293992285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f459370 0x55670f45b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9c67fc640 1 -- 172.21.15.62:0/1442521972 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fa9b80cf500 con 0x7fa9c8108290 2024-09-11T23:56:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/4293992285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f44ea10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009c50 tx=0x7fa9c000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d7ab8cdb212119 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9af7fe640 1 -- 172.21.15.62:0/4293992285 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0016070 con 0x55670f44e5d0 2024-09-11T23:56:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9cd9e1640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9b00a2cf0 0x7fa9b00a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80cd280 tx=0x7fa9b8084f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9af7fe640 1 -- 172.21.15.62:0/4293992285 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c0005500 con 0x55670f44e5d0 2024-09-11T23:56:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.850+0000 7fa9af7fe640 1 -- 172.21.15.62:0/4293992285 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00116d0 con 0x55670f44e5d0 2024-09-11T23:56:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 -- 172.21.15.62:0/4293992285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 msgr2=0x55670f44ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 --2- 172.21.15.62:0/4293992285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f44ea10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009c50 tx=0x7fa9c000e6c0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 -- 172.21.15.62:0/4293992285 shutdown_connections 2024-09-11T23:56:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 --2- 172.21.15.62:0/4293992285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f459370 0x55670f45b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 --2- 172.21.15.62:0/4293992285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f44ea10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 --2- 172.21.15.62:0/4293992285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44fb40 0x55670f44e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 -- 172.21.15.62:0/4293992285 >> 172.21.15.62:0/4293992285 conn(0x55670f417310 msgr2=0x55670f44ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 -- 172.21.15.62:0/4293992285 shutdown_connections 2024-09-11T23:56:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 -- 172.21.15.62:0/4293992285 wait complete. 2024-09-11T23:56:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 -- start start 2024-09-11T23:56:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.851+0000 7fa9d181d740 1 Processor -- start 2024-09-11T23:56:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f458c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44fb40 0x55670f455f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f459370 0x55670f456480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9d181d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55670f45c070 con 0x55670f44e5d0 2024-09-11T23:56:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9d181d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55670f36f060 con 0x55670f459370 2024-09-11T23:56:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9d181d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55670f45c1f0 con 0x55670f44fb40 2024-09-11T23:56:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9cc9df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f458c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9cc9df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f458c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59932/0 (socket says 172.21.15.62:59932) 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9cc9df640 1 -- 172.21.15.62:0/3047810453 learned_addr learned my addr 172.21.15.62:0/3047810453 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9d181d740 10 client.0 _ll_register_callbacks cb 0x55670f0c28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44fb40 0x55670f455f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9cc9df640 1 -- 172.21.15.62:0/3047810453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44fb40 msgr2=0x55670f455f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44fb40 0x55670f455f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9cc9df640 1 -- 172.21.15.62:0/3047810453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f459370 msgr2=0x55670f456480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f459370 0x55670f456480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.852+0000 7fa9cc9df640 1 -- 172.21.15.62:0/3047810453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55670f1253c0 con 0x55670f44e5d0 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f458c90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009930 tx=0x7fa9bc031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd555a78082ebed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9acff9640 1 -- 172.21.15.62:0/3047810453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc044070 con 0x55670f44e5d0 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9acff9640 1 -- 172.21.15.62:0/3047810453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fa9bc004450 con 0x55670f44e5d0 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9d181d740 1 -- 172.21.15.62:0/3047810453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c0005500 con 0x55670f44e5d0 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9acff9640 1 -- 172.21.15.62:0/3047810453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9bc050050 con 0x55670f44e5d0 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9d181d740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9d181d740 1 -- 172.21.15.62:0/3047810453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55670f467a90 con 0x55670f44e5d0 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9acff9640 1 -- 172.21.15.62:0/3047810453 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c0005500 con 0x55670f44e5d0 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9acff9640 1 -- 172.21.15.62:0/3047810453 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fa9bc03a040 con 0x55670f44e5d0 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9d181d740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9d181d740 10 client.16188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9d181d740 20 client.16188 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9d181d740 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55670f468ea0 0x55670f46b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9d181d740 4 client.16188 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.853+0000 7fa9d181d740 1 -- 172.21.15.62:0/3047810453 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55670f3c1860 con 0x55670f468ea0 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.854+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55670f468ea0 0x55670f46b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.854+0000 7fa9acff9640 10 client.16188 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.854+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55670f468ea0 0x55670f46b320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90ade4c2ce570e5 server_cookie=db7920c16cc9ee0c in_seq=0 out_seq=0 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9acff9640 1 -- 172.21.15.62:0/3047810453 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55670f3c1860 con 0x55670f468ea0 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9acff9640 10 client.16188 handle_command_reply: tid=0 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 client.16188 shutdown 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 -- 172.21.15.62:0/3047810453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 msgr2=0x55670f458c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f458c90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009930 tx=0x7fa9bc031920 comp rx=0 tx=0).stop 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 -- 172.21.15.62:0/3047810453 shutdown_connections 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55670f468ea0 0x55670f46b320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f459370 0x55670f456480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44fb40 0x55670f455f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 --2- 172.21.15.62:0/3047810453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f44e5d0 0x55670f458c90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9acff9640 0 client.16188 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 -- 172.21.15.62:0/3047810453 >> 172.21.15.62:0/3047810453 conn(0x55670f417310 msgr2=0x55670f450930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 -- 172.21.15.62:0/3047810453 shutdown_connections 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 1 -- 172.21.15.62:0/3047810453 wait complete. 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.855+0000 7fa9d181d740 20 client.16188 trim_cache size 0 max 16384 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.868+0000 7fa9d181d740 1 Processor -- start 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.868+0000 7fa9d181d740 1 -- start start 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.868+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44e5d0 0x55670f44ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.868+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44fb40 0x55670f465370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.868+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f451120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9d181d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c81b2470 con 0x55670f459370 2024-09-11T23:56:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9d181d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c81afd40 con 0x55670f44e5d0 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9d181d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c81adb90 con 0x55670f44fb40 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cc9df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f451120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cc9df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f451120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59936/0 (socket says 172.21.15.62:59936) 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cc9df640 1 -- 172.21.15.62:0/2231703501 learned_addr learned my addr 172.21.15.62:0/2231703501 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/2231703501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44fb40 0x55670f465370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/2231703501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44e5d0 0x55670f44ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cc9df640 1 -- 172.21.15.62:0/2231703501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44fb40 msgr2=0x55670f465370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/2231703501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44fb40 0x55670f465370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cc9df640 1 -- 172.21.15.62:0/2231703501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44e5d0 msgr2=0x55670f44ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/2231703501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44e5d0 0x55670f44ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cc9df640 1 -- 172.21.15.62:0/2231703501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55670f454e80 con 0x55670f459370 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/2231703501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f451120 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc03da10 tx=0x7fa9bc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e70e1a0504bf0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9aeffd640 1 -- 172.21.15.62:0/2231703501 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc044040 con 0x55670f459370 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9aeffd640 1 -- 172.21.15.62:0/2231703501 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9bc03de70 con 0x55670f459370 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.869+0000 7fa9aeffd640 1 -- 172.21.15.62:0/2231703501 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc035040 con 0x55670f459370 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 -- 172.21.15.62:0/2231703501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 msgr2=0x55670f451120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 --2- 172.21.15.62:0/2231703501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f451120 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc03da10 tx=0x7fa9bc031400 comp rx=0 tx=0).stop 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 -- 172.21.15.62:0/2231703501 shutdown_connections 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 --2- 172.21.15.62:0/2231703501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f451120 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 --2- 172.21.15.62:0/2231703501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44fb40 0x55670f465370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 --2- 172.21.15.62:0/2231703501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44e5d0 0x55670f44ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 -- 172.21.15.62:0/2231703501 >> 172.21.15.62:0/2231703501 conn(0x55670f417310 msgr2=0x55670f44a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 -- 172.21.15.62:0/2231703501 shutdown_connections 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 -- 172.21.15.62:0/2231703501 wait complete. 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 -- start start 2024-09-11T23:56:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.870+0000 7fa9d181d740 1 Processor -- start 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44e5d0 0x55670f461800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44fb40 0x55670f461d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9d181d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f452930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9d181d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c81ae200 con 0x55670f459370 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9d181d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c81af670 con 0x55670f44fb40 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9d181d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c81ae8d0 con 0x55670f44e5d0 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9d181d740 10 client.0 _ll_register_callbacks cb 0x7fa9bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9cc9df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f452930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9cc9df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f452930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59948/0 (socket says 172.21.15.62:59948) 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9cc9df640 1 -- 172.21.15.62:0/2874222497 learned_addr learned my addr 172.21.15.62:0/2874222497 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44fb40 0x55670f461d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44e5d0 0x55670f461800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9cc9df640 1 -- 172.21.15.62:0/2874222497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44e5d0 msgr2=0x55670f461800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44e5d0 0x55670f461800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9cc9df640 1 -- 172.21.15.62:0/2874222497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44fb40 msgr2=0x55670f461d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44fb40 0x55670f461d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.871+0000 7fa9cc9df640 1 -- 172.21.15.62:0/2874222497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55670f453130 con 0x55670f459370 2024-09-11T23:56:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9cc9df640 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f452930 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc003870 tx=0x7fa9bc003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1308add0f1748697 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9acff9640 1 -- 172.21.15.62:0/2874222497 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc035070 con 0x55670f459370 2024-09-11T23:56:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9acff9640 1 -- 172.21.15.62:0/2874222497 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fa9bc004250 con 0x55670f459370 2024-09-11T23:56:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9d181d740 1 -- 172.21.15.62:0/2874222497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9bc03de70 con 0x55670f459370 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9acff9640 1 -- 172.21.15.62:0/2874222497 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9bc044040 con 0x55670f459370 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9d181d740 10 client.16194 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9d181d740 1 -- 172.21.15.62:0/2874222497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55670f4d3780 con 0x55670f459370 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9acff9640 1 -- 172.21.15.62:0/2874222497 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9bc03de70 con 0x55670f459370 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9acff9640 1 -- 172.21.15.62:0/2874222497 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fa9bc049440 con 0x55670f459370 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9d181d740 10 client.16194 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9d181d740 10 client.16194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9d181d740 20 client.16194 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9d181d740 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55670f4d3d90 0x55670f4d6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9d181d740 4 client.16194 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.872+0000 7fa9d181d740 1 -- 172.21.15.62:0/2874222497 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55670f3bbd70 con 0x55670f4d3d90 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55670f4d3d90 0x55670f4d6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9cd1e0640 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55670f4d3d90 0x55670f4d6180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5716754eaa86b5c4 server_cookie=339577abe57253ee in_seq=0 out_seq=0 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9acff9640 10 client.16194 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9acff9640 1 -- 172.21.15.62:0/2874222497 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55670f3bbd70 con 0x55670f4d3d90 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9acff9640 10 client.16194 handle_command_reply: tid=0 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9d181d740 1 client.16194 shutdown 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9d181d740 1 -- 172.21.15.62:0/2874222497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 msgr2=0x55670f452930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9d181d740 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f452930 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc003870 tx=0x7fa9bc003770 comp rx=0 tx=0).stop 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9d181d740 1 -- 172.21.15.62:0/2874222497 shutdown_connections 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9d181d740 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55670f4d3d90 0x55670f4d6180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9d181d740 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55670f459370 0x55670f452930 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.873+0000 7fa9d181d740 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55670f44fb40 0x55670f461d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9acff9640 0 client.16194 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9d181d740 1 --2- 172.21.15.62:0/2874222497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55670f44e5d0 0x55670f461800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9d181d740 1 -- 172.21.15.62:0/2874222497 >> 172.21.15.62:0/2874222497 conn(0x55670f417310 msgr2=0x55670f4494e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9d181d740 1 -- 172.21.15.62:0/2874222497 shutdown_connections 2024-09-11T23:56:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9d181d740 1 -- 172.21.15.62:0/2874222497 wait complete. 2024-09-11T23:56:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9d181d740 20 client.16194 trim_cache size 0 max 16384 2024-09-11T23:56:37.901 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:56:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9cf444640 1 -- 172.21.15.62:0/1442521972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9b00a2cf0 msgr2=0x7fa9b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9cf444640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9b00a2cf0 0x7fa9b00a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80cd280 tx=0x7fa9b8084f90 comp rx=0 tx=0).stop 2024-09-11T23:56:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9cf444640 1 -- 172.21.15.62:0/1442521972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c8108290 msgr2=0x7fa9c819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9cf444640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c8108290 0x7fa9c819a640 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800cad0 tx=0x7fa9b800cf90 comp rx=0 tx=0).stop 2024-09-11T23:56:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9cf444640 1 -- 172.21.15.62:0/1442521972 shutdown_connections 2024-09-11T23:56:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9cf444640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9b00a2cf0 0x7fa9b00a51b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9cf444640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c8108290 0x7fa9c819a640 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9cf444640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c8103740 0x7fa9c81a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9cf444640 1 --2- 172.21.15.62:0/1442521972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c8102d90 0x7fa9c81a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.874+0000 7fa9cf444640 1 -- 172.21.15.62:0/1442521972 >> 172.21.15.62:0/1442521972 conn(0x7fa9c80fc7d0 msgr2=0x7fa9c8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.875+0000 7fa9cf444640 1 -- 172.21.15.62:0/1442521972 shutdown_connections 2024-09-11T23:56:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:37.875+0000 7fa9cf444640 1 -- 172.21.15.62:0/1442521972 wait complete. 2024-09-11T23:56:38.719 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:56:38.975 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:39.377 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:39 smithi062 ceph-mon[30958]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-09-11T23:56:39.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.378+0000 7f6bcb734640 1 -- 172.21.15.62:0/1106013326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bc4101de0 msgr2=0x7f6bc41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:39.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.378+0000 7f6bcb734640 1 --2- 172.21.15.62:0/1106013326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bc4101de0 0x7f6bc41021c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009900 tx=0x7f6bc002a410 comp rx=0 tx=0).stop 2024-09-11T23:56:39.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.378+0000 7f6bcb734640 1 -- 172.21.15.62:0/1106013326 shutdown_connections 2024-09-11T23:56:39.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.378+0000 7f6bcb734640 1 --2- 172.21.15.62:0/1106013326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bc4106f10 0x7f6bc410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.378+0000 7f6bcb734640 1 --2- 172.21.15.62:0/1106013326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bc4102790 0x7f6bc41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.378+0000 7f6bcb734640 1 --2- 172.21.15.62:0/1106013326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bc4101de0 0x7f6bc41021c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.378+0000 7f6bcb734640 1 -- 172.21.15.62:0/1106013326 >> 172.21.15.62:0/1106013326 conn(0x7f6bc40fd8c0 msgr2=0x7f6bc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.378+0000 7f6bcb734640 1 -- 172.21.15.62:0/1106013326 shutdown_connections 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.378+0000 7f6bcb734640 1 -- 172.21.15.62:0/1106013326 wait complete. 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.379+0000 7f6bcb734640 1 Processor -- start 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.379+0000 7f6bcb734640 1 -- start start 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bcb734640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bc4101de0 0x7f6bc4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bcb734640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bc4102790 0x7f6bc4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bcb734640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bcb734640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4111f30 con 0x7f6bc4102790 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bcb734640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4111db0 con 0x7f6bc4106f10 2024-09-11T23:56:39.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bcb734640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41120b0 con 0x7f6bc4101de0 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc8ccf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bc4102790 0x7f6bc4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc8ccf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bc4102790 0x7f6bc4198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59966/0 (socket says 172.21.15.62:59966) 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc8ccf640 1 -- 172.21.15.62:0/3744744240 learned_addr learned my addr 172.21.15.62:0/3744744240 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc94d0640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bc4101de0 0x7f6bc4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc8ccf640 1 -- 172.21.15.62:0/3744744240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bc4101de0 msgr2=0x7f6bc4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc8ccf640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bc4101de0 0x7f6bc4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc8ccf640 1 -- 172.21.15.62:0/3744744240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bc4106f10 msgr2=0x7f6bc419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc8ccf640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc8ccf640 1 -- 172.21.15.62:0/3744744240 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc419d4c0 con 0x7f6bc4102790 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bc8ccf640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bc4102790 0x7f6bc4198a50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400d950 tx=0x7f6bb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.380+0000 7f6bba7fc640 1 -- 172.21.15.62:0/3744744240 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4017030 con 0x7f6bc4102790 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.381+0000 7f6bcb734640 1 -- 172.21.15.62:0/3744744240 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc419d7b0 con 0x7f6bc4102790 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.381+0000 7f6bba7fc640 1 -- 172.21.15.62:0/3744744240 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb4004020 con 0x7f6bc4102790 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.381+0000 7f6bba7fc640 1 -- 172.21.15.62:0/3744744240 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40047a0 con 0x7f6bc4102790 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.381+0000 7f6bcb734640 1 -- 172.21.15.62:0/3744744240 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41a5090 con 0x7f6bc4102790 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.383+0000 7f6bba7fc640 1 -- 172.21.15.62:0/3744744240 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6bb40041d0 con 0x7f6bc4102790 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.383+0000 7f6bcdb0d740 1 Processor -- start 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.383+0000 7f6bcdb0d740 1 -- start start 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.383+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 0x5603c65df4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.383+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65df9e0 0x5603c65dfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.383+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.383+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5603c65412f0 con 0x5603c65ea780 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.383+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603c64e8cf0 con 0x5603c65e0f50 2024-09-11T23:56:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.383+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5603c653d2b0 con 0x5603c65df9e0 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc8ccf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc94d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65df9e0 0x5603c65dfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc8ccf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65ecb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59978/0 (socket says 172.21.15.62:59978) 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc8ccf640 1 -- 172.21.15.62:0/121469119 learned_addr learned my addr 172.21.15.62:0/121469119 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/121469119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 0x5603c65df4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc8ccf640 1 -- 172.21.15.62:0/121469119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65df9e0 msgr2=0x5603c65dfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc8ccf640 1 --2- 172.21.15.62:0/121469119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65df9e0 0x5603c65dfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc8ccf640 1 -- 172.21.15.62:0/121469119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 msgr2=0x5603c65df4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc8ccf640 1 --2- 172.21.15.62:0/121469119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 0x5603c65df4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc8ccf640 1 -- 172.21.15.62:0/121469119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c64cd3a0 con 0x5603c65ea780 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/121469119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 0x5603c65df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bba7fc640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6bb00a2cf0 0x7f6bb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc8ccf640 1 --2- 172.21.15.62:0/121469119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65ecb70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40cc5b0 tx=0x7f6bb40d74b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2a5664827940d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:39.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6bb00a2cf0 0x7f6bb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6bba7fc640 1 -- 172.21.15.62:0/3744744240 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6bb40ceec0 con 0x7f6bc4102790 2024-09-11T23:56:39.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6baf7fe640 1 -- 172.21.15.62:0/121469119 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40976b0 con 0x5603c65ea780 2024-09-11T23:56:39.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6baf7fe640 1 -- 172.21.15.62:0/121469119 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb409bb10 con 0x5603c65ea780 2024-09-11T23:56:39.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.384+0000 7f6baf7fe640 1 -- 172.21.15.62:0/121469119 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40c52f0 con 0x5603c65ea780 2024-09-11T23:56:39.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.385+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/121469119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 msgr2=0x5603c65ecb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:39.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.385+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6bb00a2cf0 0x7f6bb00a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc005fd0 tx=0x7f6bbc009530 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:39.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.385+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/121469119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65ecb70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40cc5b0 tx=0x7f6bb40d74b0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.385+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/121469119 shutdown_connections 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.385+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/121469119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65ecb70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.385+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/121469119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65df9e0 0x5603c65dfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.385+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/121469119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 0x5603c65df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.385+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/121469119 >> 172.21.15.62:0/121469119 conn(0x5603c65a4310 msgr2=0x5603c65dc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.385+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/121469119 shutdown_connections 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.385+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/121469119 wait complete. 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.386+0000 7f6bcdb0d740 1 -- start start 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.386+0000 7f6bcdb0d740 1 Processor -- start 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.386+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 0x5603c65ea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.386+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 0x5603c65e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.386+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.386+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5603c65ed480 con 0x5603c65ea780 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.386+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603c64fc060 con 0x5603c65df9e0 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.386+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5603c65ed600 con 0x5603c65e0f50 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.386+0000 7f6bcdb0d740 10 client.0 _ll_register_callbacks cb 0x5603c624f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc9cd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc94d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 0x5603c65ea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc9cd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59992/0 (socket says 172.21.15.62:59992) 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc9cd1640 1 -- 172.21.15.62:0/2620504126 learned_addr learned my addr 172.21.15.62:0/2620504126 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc8ccf640 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 0x5603c65e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc9cd1640 1 -- 172.21.15.62:0/2620504126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 msgr2=0x5603c65e74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 0x5603c65e74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc9cd1640 1 -- 172.21.15.62:0/2620504126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 msgr2=0x5603c65ea1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 0x5603c65ea1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc9cd1640 1 -- 172.21.15.62:0/2620504126 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c62b23c0 con 0x5603c65ea780 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc8ccf640 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 0x5603c65e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e7be0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009d80 tx=0x7f6bbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156a91c539668b36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bacff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bacff9640 1 -- 172.21.15.62:0/2620504126 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc022070 con 0x5603c65ea780 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bacff9640 1 -- 172.21.15.62:0/2620504126 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6bbc00fd60 con 0x5603c65ea780 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/2620504126 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb409bb10 con 0x5603c65ea780 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bacff9640 1 -- 172.21.15.62:0/2620504126 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6bbc012e50 con 0x5603c65ea780 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bcdb0d740 10 client.16206 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.387+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/2620504126 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5603c65f9010 con 0x5603c65ea780 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.388+0000 7f6bacff9640 1 -- 172.21.15.62:0/2620504126 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb409bb10 con 0x5603c65ea780 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.388+0000 7f6bacff9640 1 -- 172.21.15.62:0/2620504126 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6bbc030750 con 0x5603c65ea780 2024-09-11T23:56:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.388+0000 7f6bcdb0d740 10 client.16206 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.388+0000 7f6bcdb0d740 10 client.16206 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.388+0000 7f6bcdb0d740 20 client.16206 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.388+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5603c65fa420 0x5603c65fc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.388+0000 7f6bcdb0d740 4 client.16206 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.388+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/2620504126 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603c654e860 con 0x5603c65fa420 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.388+0000 7f6bc94d0640 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5603c65fa420 0x5603c65fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bc94d0640 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5603c65fa420 0x5603c65fc810 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9075bd3701de369f server_cookie=9c3121624a4c806 in_seq=0 out_seq=0 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bacff9640 10 client.16206 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bacff9640 1 -- 172.21.15.62:0/2620504126 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603c654e860 con 0x5603c65fa420 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bacff9640 10 client.16206 handle_command_reply: tid=0 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bcdb0d740 1 client.16206 shutdown 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/2620504126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 msgr2=0x5603c65e7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e7be0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009d80 tx=0x7f6bbc004a40 comp rx=0 tx=0).stop 2024-09-11T23:56:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/2620504126 shutdown_connections 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5603c65fa420 0x5603c65fc810 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e7be0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 0x5603c65e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.389+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/2620504126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 0x5603c65ea1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.390+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/2620504126 >> 172.21.15.62:0/2620504126 conn(0x5603c65a4310 msgr2=0x5603c65e1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.390+0000 7f6bacff9640 0 client.16206 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.390+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/2620504126 shutdown_connections 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.390+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/2620504126 wait complete. 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.390+0000 7f6bcdb0d740 20 client.16206 trim_cache size 0 max 16384 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bcdb0d740 1 Processor -- start 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bcdb0d740 1 -- start start 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65df9e0 0x5603c65f6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 0x5603c65e2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65ea780 0x5603c65e2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41adfe0 con 0x5603c65df9e0 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41ab8b0 con 0x5603c65e0f50 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41a9700 con 0x5603c65ea780 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bc94d0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65df9e0 0x5603c65f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bc94d0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65df9e0 0x5603c65f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60002/0 (socket says 172.21.15.62:60002) 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bc94d0640 1 -- 172.21.15.62:0/847829491 learned_addr learned my addr 172.21.15.62:0/847829491 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/847829491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65ea780 0x5603c65e2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bc8ccf640 1 --2- 172.21.15.62:0/847829491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 0x5603c65e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bc94d0640 1 -- 172.21.15.62:0/847829491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65ea780 msgr2=0x5603c65e2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bc94d0640 1 --2- 172.21.15.62:0/847829491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65ea780 0x5603c65e2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.403+0000 7f6bc94d0640 1 -- 172.21.15.62:0/847829491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 msgr2=0x5603c65e2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:39.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6bc94d0640 1 --2- 172.21.15.62:0/847829491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 0x5603c65e2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6bc94d0640 1 -- 172.21.15.62:0/847829491 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c65f6c60 con 0x5603c65df9e0 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6bc94d0640 1 --2- 172.21.15.62:0/847829491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65df9e0 0x5603c65f6720 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009900 tx=0x7f6bc00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5420248543fc3bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6baeffd640 1 -- 172.21.15.62:0/847829491 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc004a070 con 0x5603c65df9e0 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6baeffd640 1 -- 172.21.15.62:0/847829491 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc0005520 con 0x5603c65df9e0 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6baeffd640 1 -- 172.21.15.62:0/847829491 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0035040 con 0x5603c65df9e0 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/847829491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65df9e0 msgr2=0x5603c65f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/847829491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65df9e0 0x5603c65f6720 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009900 tx=0x7f6bc00058b0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/847829491 shutdown_connections 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/847829491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65ea780 0x5603c65e2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/847829491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65e0f50 0x5603c65e2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.404+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/847829491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65df9e0 0x5603c65f6720 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/847829491 >> 172.21.15.62:0/847829491 conn(0x5603c65a4310 msgr2=0x5603c65e65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:39.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/847829491 shutdown_connections 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/847829491 wait complete. 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 -- start start 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 Processor -- start 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 0x5603c65e4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 0x5603c65e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41a9d70 con 0x5603c65ea780 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41ab1e0 con 0x5603c65df9e0 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.405+0000 7f6bcdb0d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41aa440 con 0x5603c65e0f50 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bcdb0d740 10 client.0 _ll_register_callbacks cb 0x7f6bbc009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc9cd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc8ccf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 0x5603c65e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc9cd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60016/0 (socket says 172.21.15.62:60016) 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc9cd1640 1 -- 172.21.15.62:0/1886101119 learned_addr learned my addr 172.21.15.62:0/1886101119 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc94d0640 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 0x5603c65e4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc9cd1640 1 -- 172.21.15.62:0/1886101119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 msgr2=0x5603c65e4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 0x5603c65e4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc9cd1640 1 -- 172.21.15.62:0/1886101119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 msgr2=0x5603c65e4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 0x5603c65e4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc9cd1640 1 -- 172.21.15.62:0/1886101119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c6664130 con 0x5603c65ea780 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc8ccf640 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 0x5603c65e4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc94d0640 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 0x5603c65e4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bc9cd1640 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e4fc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc011c40 tx=0x7f6bbc012ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30eb13146ee51ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bacff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bacff9640 1 -- 172.21.15.62:0/1886101119 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc03f060 con 0x5603c65ea780 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bacff9640 1 -- 172.21.15.62:0/1886101119 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f6bbc00fd60 con 0x5603c65ea780 2024-09-11T23:56:39.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.406+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/1886101119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bc0005520 con 0x5603c65ea780 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bacff9640 1 -- 172.21.15.62:0/1886101119 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6bbc036040 con 0x5603c65ea780 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bcdb0d740 10 client.16212 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/1886101119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5603c6665410 con 0x5603c65ea780 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bacff9640 1 -- 172.21.15.62:0/1886101119 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc0005520 con 0x5603c65ea780 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bacff9640 1 -- 172.21.15.62:0/1886101119 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f6bbc039070 con 0x5603c65ea780 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bcdb0d740 10 client.16212 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bcdb0d740 10 client.16212 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bcdb0d740 20 client.16212 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5603c6665b60 0x5603c6667f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bcdb0d740 4 client.16212 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/1886101119 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603c6548d70 con 0x5603c6665b60 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.407+0000 7f6bc94d0640 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5603c6665b60 0x5603c6667f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:39.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bc94d0640 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5603c6665b60 0x5603c6667f80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=143753cf27e17d39 server_cookie=fbf742ae5c9c6688 in_seq=0 out_seq=0 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bacff9640 10 client.16212 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bacff9640 1 -- 172.21.15.62:0/1886101119 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603c6548d70 con 0x5603c6665b60 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bacff9640 10 client.16212 handle_command_reply: tid=0 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bcdb0d740 1 client.16212 shutdown 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/1886101119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 msgr2=0x5603c65e4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e4fc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc011c40 tx=0x7f6bbc012ee0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/1886101119 shutdown_connections 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5603c6665b60 0x5603c6667f80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603c65ea780 0x5603c65e4fc0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5603c65e0f50 0x5603c65e4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bcdb0d740 1 --2- 172.21.15.62:0/1886101119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603c65df9e0 0x5603c65e4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/1886101119 >> 172.21.15.62:0/1886101119 conn(0x5603c65a4310 msgr2=0x5603c65e8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:39.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bacff9640 0 client.16212 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:56:39.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.408+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/1886101119 shutdown_connections 2024-09-11T23:56:39.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcdb0d740 1 -- 172.21.15.62:0/1886101119 wait complete. 2024-09-11T23:56:39.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcdb0d740 20 client.16212 trim_cache size 0 max 16384 2024-09-11T23:56:39.425 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 -- 172.21.15.62:0/3744744240 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6bb00a2cf0 msgr2=0x7f6bb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6bb00a2cf0 0x7f6bb00a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc005fd0 tx=0x7f6bbc009530 comp rx=0 tx=0).stop 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 -- 172.21.15.62:0/3744744240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bc4102790 msgr2=0x7f6bc4198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bc4102790 0x7f6bc4198a50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400d950 tx=0x7f6bb400de10 comp rx=0 tx=0).stop 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 -- 172.21.15.62:0/3744744240 shutdown_connections 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6bb00a2cf0 0x7f6bb00a51b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bc4102790 0x7f6bc4198a50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 --2- 172.21.15.62:0/3744744240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bc4101de0 0x7f6bc4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 -- 172.21.15.62:0/3744744240 >> 172.21.15.62:0/3744744240 conn(0x7f6bc40fd8c0 msgr2=0x7f6bc40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 -- 172.21.15.62:0/3744744240 shutdown_connections 2024-09-11T23:56:39.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:39.409+0000 7f6bcb734640 1 -- 172.21.15.62:0/3744744240 wait complete. 2024-09-11T23:56:39.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:39 smithi120 ceph-mon[35182]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-09-11T23:56:39.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:39 smithi083 ceph-mon[34805]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-09-11T23:56:40.179 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:56:40.436 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.923+0000 7f2035fa8640 1 -- 172.21.15.62:0/3676120400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030106a80 msgr2=0x7f2030106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.923+0000 7f2035fa8640 1 --2- 172.21.15.62:0/3676120400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030106a80 0x7f2030106e60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2020009930 tx=0x7f202002a420 comp rx=0 tx=0).stop 2024-09-11T23:56:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.923+0000 7f2035fa8640 1 -- 172.21.15.62:0/3676120400 shutdown_connections 2024-09-11T23:56:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.923+0000 7f2035fa8640 1 --2- 172.21.15.62:0/3676120400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2030101420 0x7f203010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.923+0000 7f2035fa8640 1 --2- 172.21.15.62:0/3676120400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2030100a80 0x7f2030100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.923+0000 7f2035fa8640 1 --2- 172.21.15.62:0/3676120400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030106a80 0x7f2030106e60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.923+0000 7f2035fa8640 1 -- 172.21.15.62:0/3676120400 >> 172.21.15.62:0/3676120400 conn(0x7f20300fc7a0 msgr2=0x7f20300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.923+0000 7f2035fa8640 1 -- 172.21.15.62:0/3676120400 shutdown_connections 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.923+0000 7f2035fa8640 1 -- 172.21.15.62:0/3676120400 wait complete. 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f2035fa8640 1 Processor -- start 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f2035fa8640 1 -- start start 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f2035fa8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2030100a80 0x7f20301992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f2035fa8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030101420 0x7f2030199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f2035fa8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2030106a80 0x7f20301a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f2035fa8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2030110e80 con 0x7f2030101420 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f2035fa8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2030110d00 con 0x7f2030100a80 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f2035fa8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2030111000 con 0x7f2030106a80 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f202effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030101420 0x7f2030199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f202effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030101420 0x7f2030199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60028/0 (socket says 172.21.15.62:60028) 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.924+0000 7f202effd640 1 -- 172.21.15.62:0/2009302943 learned_addr learned my addr 172.21.15.62:0/2009302943 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202f7fe640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2030100a80 0x7f20301992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202ffff640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2030106a80 0x7f20301a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202effd640 1 -- 172.21.15.62:0/2009302943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2030106a80 msgr2=0x7f20301a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202effd640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2030106a80 0x7f20301a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202effd640 1 -- 172.21.15.62:0/2009302943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2030100a80 msgr2=0x7f20301992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202effd640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2030100a80 0x7f20301992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202effd640 1 -- 172.21.15.62:0/2009302943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20301a4890 con 0x7f2030101420 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202ffff640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2030106a80 0x7f20301a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202effd640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030101420 0x7f2030199810 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f201800e9f0 tx=0x7f201800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202cff9640 1 -- 172.21.15.62:0/2009302943 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201800cd80 con 0x7f2030101420 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202cff9640 1 -- 172.21.15.62:0/2009302943 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2018010910 con 0x7f2030101420 2024-09-11T23:56:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f2035fa8640 1 -- 172.21.15.62:0/2009302943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20301a4a60 con 0x7f2030101420 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f202cff9640 1 -- 172.21.15.62:0/2009302943 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180188f0 con 0x7f2030101420 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.925+0000 7f2035fa8640 1 -- 172.21.15.62:0/2009302943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20301a4f70 con 0x7f2030101420 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f202cff9640 1 -- 172.21.15.62:0/2009302943 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2018010ac0 con 0x7f2030101420 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f2038381740 1 Processor -- start 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f2038381740 1 -- start start 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f2038381740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a38b40 0x55b5e2a37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f2038381740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a375d0 0x55b5e2a37a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f2038381740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a42370 0x55b5e2a44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f2038381740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b5e299d2f0 con 0x55b5e2a38b40 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f2038381740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2944cf0 con 0x55b5e2a42370 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f2038381740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b5e29992b0 con 0x55b5e2a375d0 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f202ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a38b40 0x55b5e2a37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f202ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a38b40 0x55b5e2a37090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60044/0 (socket says 172.21.15.62:60044) 2024-09-11T23:56:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.928+0000 7f202ffff640 1 -- 172.21.15.62:0/415006151 learned_addr learned my addr 172.21.15.62:0/415006151 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202f7fe640 1 --2- 172.21.15.62:0/415006151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a375d0 0x55b5e2a37a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202cff9640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f201c0a2cf0 0x7f201c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202effd640 1 --2- 172.21.15.62:0/415006151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a42370 0x55b5e2a44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202ffff640 1 -- 172.21.15.62:0/415006151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a375d0 msgr2=0x55b5e2a37a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202ffff640 1 --2- 172.21.15.62:0/415006151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a375d0 0x55b5e2a37a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202ffff640 1 -- 172.21.15.62:0/415006151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a42370 msgr2=0x55b5e2a44760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202ffff640 1 --2- 172.21.15.62:0/415006151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a42370 0x55b5e2a44760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202ffff640 1 -- 172.21.15.62:0/415006151 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e29293a0 con 0x55b5e2a38b40 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202ffff640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f201c0a2cf0 0x7f201c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202cff9640 1 -- 172.21.15.62:0/2009302943 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f20180cf860 con 0x7f2030101420 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202ffff640 1 --2- 172.21.15.62:0/415006151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a38b40 0x55b5e2a37090 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2028010420 tx=0x7f20280108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3b679f43361776 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f2025ffb640 1 -- 172.21.15.62:0/415006151 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028023070 con 0x55b5e2a38b40 2024-09-11T23:56:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f2025ffb640 1 -- 172.21.15.62:0/415006151 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f202801c470 con 0x55b5e2a38b40 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f2025ffb640 1 -- 172.21.15.62:0/415006151 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202801d6a0 con 0x55b5e2a38b40 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.929+0000 7f202ffff640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f201c0a2cf0 0x7f201c0a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2028020000 tx=0x7f202801a000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.930+0000 7f2038381740 1 -- 172.21.15.62:0/415006151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a38b40 msgr2=0x55b5e2a37090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.930+0000 7f2038381740 1 --2- 172.21.15.62:0/415006151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a38b40 0x55b5e2a37090 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2028010420 tx=0x7f20280108e0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.930+0000 7f2038381740 1 -- 172.21.15.62:0/415006151 shutdown_connections 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.930+0000 7f2038381740 1 --2- 172.21.15.62:0/415006151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a42370 0x55b5e2a44760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.930+0000 7f2038381740 1 --2- 172.21.15.62:0/415006151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a375d0 0x55b5e2a37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.930+0000 7f2038381740 1 --2- 172.21.15.62:0/415006151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a38b40 0x55b5e2a37090 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.930+0000 7f2038381740 1 -- 172.21.15.62:0/415006151 >> 172.21.15.62:0/415006151 conn(0x55b5e2a00310 msgr2=0x55b5e2a33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.930+0000 7f2038381740 1 -- 172.21.15.62:0/415006151 shutdown_connections 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.930+0000 7f2038381740 1 -- 172.21.15.62:0/415006151 wait complete. 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.930+0000 7f2038381740 1 -- start start 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f2038381740 1 Processor -- start 2024-09-11T23:56:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f2038381740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a41e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f2038381740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a38b40 0x55b5e2a3f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f2038381740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a42370 0x55b5e2a3f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f2038381740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2a45070 con 0x55b5e2a375d0 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f2038381740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2958060 con 0x55b5e2a38b40 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f2038381740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2a451f0 con 0x55b5e2a42370 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f2038381740 10 client.0 _ll_register_callbacks cb 0x55b5e26ab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f202f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a41e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f202f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a41e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60060/0 (socket says 172.21.15.62:60060) 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f202f7fe640 1 -- 172.21.15.62:0/2965082485 learned_addr learned my addr 172.21.15.62:0/2965082485 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f202ffff640 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a42370 0x55b5e2a3f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f202effd640 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a38b40 0x55b5e2a3f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f202f7fe640 1 -- 172.21.15.62:0/2965082485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a42370 msgr2=0x55b5e2a3f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f202f7fe640 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a42370 0x55b5e2a3f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.931+0000 7f202f7fe640 1 -- 172.21.15.62:0/2965082485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a38b40 msgr2=0x55b5e2a3f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f202f7fe640 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a38b40 0x55b5e2a3f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f202f7fe640 1 -- 172.21.15.62:0/2965082485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e270e3c0 con 0x55b5e2a375d0 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f202f7fe640 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a41e00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2020000c00 tx=0x7f2020031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34b5c8724253d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f200effd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f200effd640 1 -- 172.21.15.62:0/2965082485 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020035040 con 0x55b5e2a375d0 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f200effd640 1 -- 172.21.15.62:0/2965082485 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f2020004240 con 0x55b5e2a375d0 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f2038381740 1 -- 172.21.15.62:0/2965082485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f202801c470 con 0x55b5e2a375d0 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f200effd640 1 -- 172.21.15.62:0/2965082485 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2020004440 con 0x55b5e2a375d0 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f2038381740 10 client.16224 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f2038381740 1 -- 172.21.15.62:0/2965082485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5e2a50bc0 con 0x55b5e2a375d0 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.932+0000 7f200effd640 1 -- 172.21.15.62:0/2965082485 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f202801c470 con 0x55b5e2a375d0 2024-09-11T23:56:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.933+0000 7f200effd640 1 -- 172.21.15.62:0/2965082485 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2020004cc0 con 0x55b5e2a375d0 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.933+0000 7f2038381740 10 client.16224 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.933+0000 7f2038381740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.933+0000 7f2038381740 20 client.16224 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.933+0000 7f2038381740 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b5e2a52080 0x55b5e2a54490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.933+0000 7f2038381740 4 client.16224 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.933+0000 7f2038381740 1 -- 172.21.15.62:0/2965082485 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5e29aa860 con 0x55b5e2a52080 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.933+0000 7f202effd640 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b5e2a52080 0x55b5e2a54490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.933+0000 7f200effd640 10 client.16224 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.933+0000 7f202effd640 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b5e2a52080 0x55b5e2a54490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6645cc094a522 server_cookie=4e7ca0a2763aed94 in_seq=0 out_seq=0 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f200effd640 1 -- 172.21.15.62:0/2965082485 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5e29aa860 con 0x55b5e2a52080 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f200effd640 10 client.16224 handle_command_reply: tid=0 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 client.16224 shutdown 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 -- 172.21.15.62:0/2965082485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 msgr2=0x55b5e2a41e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a41e00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2020000c00 tx=0x7f2020031920 comp rx=0 tx=0).stop 2024-09-11T23:56:40.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 -- 172.21.15.62:0/2965082485 shutdown_connections 2024-09-11T23:56:40.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b5e2a52080 0x55b5e2a54490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a42370 0x55b5e2a3f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a38b40 0x55b5e2a3f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f200effd640 0 client.16224 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:40.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 --2- 172.21.15.62:0/2965082485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a41e00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 -- 172.21.15.62:0/2965082485 >> 172.21.15.62:0/2965082485 conn(0x55b5e2a00310 msgr2=0x55b5e2a39960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:40.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 -- 172.21.15.62:0/2965082485 shutdown_connections 2024-09-11T23:56:40.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 1 -- 172.21.15.62:0/2965082485 wait complete. 2024-09-11T23:56:40.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.934+0000 7f2038381740 20 client.16224 trim_cache size 0 max 16384 2024-09-11T23:56:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.947+0000 7f2038381740 1 Processor -- start 2024-09-11T23:56:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.947+0000 7f2038381740 1 -- start start 2024-09-11T23:56:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.947+0000 7f2038381740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.947+0000 7f2038381740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a38b40 0x55b5e2a3a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f2038381740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a42370 0x55b5e2a59d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f2038381740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f20301b0130 con 0x55b5e2a375d0 2024-09-11T23:56:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f2038381740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20301ada00 con 0x55b5e2a42370 2024-09-11T23:56:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f2038381740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20301ab850 con 0x55b5e2a38b40 2024-09-11T23:56:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a3a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60066/0 (socket says 172.21.15.62:60066) 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202f7fe640 1 -- 172.21.15.62:0/266368552 learned_addr learned my addr 172.21.15.62:0/266368552 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202effd640 1 --2- 172.21.15.62:0/266368552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a38b40 0x55b5e2a3a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202ffff640 1 --2- 172.21.15.62:0/266368552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a42370 0x55b5e2a59d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202f7fe640 1 -- 172.21.15.62:0/266368552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a38b40 msgr2=0x55b5e2a3a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202f7fe640 1 --2- 172.21.15.62:0/266368552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a38b40 0x55b5e2a3a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202f7fe640 1 -- 172.21.15.62:0/266368552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a42370 msgr2=0x55b5e2a59d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202f7fe640 1 --2- 172.21.15.62:0/266368552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a42370 0x55b5e2a59d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202f7fe640 1 -- 172.21.15.62:0/266368552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e2a42170 con 0x55b5e2a375d0 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f202f7fe640 1 --2- 172.21.15.62:0/266368552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a3a120 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2020006010 tx=0x7f2020003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4bd5cc90551b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f20257fa640 1 -- 172.21.15.62:0/266368552 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020035030 con 0x55b5e2a375d0 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f20257fa640 1 -- 172.21.15.62:0/266368552 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20200054e0 con 0x55b5e2a375d0 2024-09-11T23:56:40.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.948+0000 7f20257fa640 1 -- 172.21.15.62:0/266368552 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020053040 con 0x55b5e2a375d0 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.949+0000 7f2038381740 1 -- 172.21.15.62:0/266368552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 msgr2=0x55b5e2a3a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.949+0000 7f2038381740 1 --2- 172.21.15.62:0/266368552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a3a120 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2020006010 tx=0x7f2020003b20 comp rx=0 tx=0).stop 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.949+0000 7f2038381740 1 -- 172.21.15.62:0/266368552 shutdown_connections 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.949+0000 7f2038381740 1 --2- 172.21.15.62:0/266368552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a42370 0x55b5e2a59d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.949+0000 7f2038381740 1 --2- 172.21.15.62:0/266368552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a38b40 0x55b5e2a3a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.949+0000 7f2038381740 1 --2- 172.21.15.62:0/266368552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a375d0 0x55b5e2a3a120 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.949+0000 7f2038381740 1 -- 172.21.15.62:0/266368552 >> 172.21.15.62:0/266368552 conn(0x55b5e2a00310 msgr2=0x55b5e2a32d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.949+0000 7f2038381740 1 -- 172.21.15.62:0/266368552 shutdown_connections 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.949+0000 7f2038381740 1 -- 172.21.15.62:0/266368552 wait complete. 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f2038381740 1 -- start start 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f2038381740 1 Processor -- start 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f2038381740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a375d0 0x55b5e2a4a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f2038381740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a38b40 0x55b5e2a4aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f2038381740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a42370 0x55b5e2a3b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f2038381740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f20301abec0 con 0x55b5e2a42370 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f2038381740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20301ad330 con 0x55b5e2a38b40 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f2038381740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20301ac590 con 0x55b5e2a375d0 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f2038381740 10 client.0 _ll_register_callbacks cb 0x7f2020009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f202ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a42370 0x55b5e2a3b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f202ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a42370 0x55b5e2a3b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60068/0 (socket says 172.21.15.62:60068) 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f202ffff640 1 -- 172.21.15.62:0/3819423905 learned_addr learned my addr 172.21.15.62:0/3819423905 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f202f7fe640 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a375d0 0x55b5e2a4a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.950+0000 7f202effd640 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a38b40 0x55b5e2a4aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f202ffff640 1 -- 172.21.15.62:0/3819423905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a375d0 msgr2=0x55b5e2a4a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f202ffff640 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a375d0 0x55b5e2a4a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f202ffff640 1 -- 172.21.15.62:0/3819423905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a38b40 msgr2=0x55b5e2a4aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f202ffff640 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a38b40 0x55b5e2a4aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f202ffff640 1 -- 172.21.15.62:0/3819423905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e2a3c120 con 0x55b5e2a42370 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f202ffff640 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a42370 0x55b5e2a3b940 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f20280103f0 tx=0x7f202801ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cf018d59a39f91 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f200effd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f200effd640 1 -- 172.21.15.62:0/3819423905 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028023040 con 0x55b5e2a42370 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f200effd640 1 -- 172.21.15.62:0/3819423905 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f202801db70 con 0x55b5e2a42370 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f2038381740 1 -- 172.21.15.62:0/3819423905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20200054e0 con 0x55b5e2a42370 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f200effd640 1 -- 172.21.15.62:0/3819423905 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2028009bb0 con 0x55b5e2a42370 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f2038381740 10 client.16230 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f2038381740 1 -- 172.21.15.62:0/3819423905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5e2abc7b0 con 0x55b5e2a42370 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.951+0000 7f200effd640 1 -- 172.21.15.62:0/3819423905 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20200054e0 con 0x55b5e2a42370 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.952+0000 7f200effd640 1 -- 172.21.15.62:0/3819423905 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f202800f040 con 0x55b5e2a42370 2024-09-11T23:56:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.952+0000 7f2038381740 10 client.16230 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.952+0000 7f2038381740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.952+0000 7f2038381740 20 client.16230 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.952+0000 7f2038381740 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b5e2abcd90 0x55b5e2abf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.952+0000 7f2038381740 4 client.16230 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:56:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.952+0000 7f2038381740 1 -- 172.21.15.62:0/3819423905 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5e29a4d70 con 0x55b5e2abcd90 2024-09-11T23:56:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.952+0000 7f202f7fe640 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b5e2abcd90 0x55b5e2abf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:40.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f200effd640 10 client.16230 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:40.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f202f7fe640 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b5e2abcd90 0x55b5e2abf1b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f1190c1785bcff server_cookie=1c2a89c7e8cd8a5b in_seq=0 out_seq=0 2024-09-11T23:56:40.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f200effd640 1 -- 172.21.15.62:0/3819423905 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5e29a4d70 con 0x55b5e2abcd90 2024-09-11T23:56:40.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f200effd640 10 client.16230 handle_command_reply: tid=0 2024-09-11T23:56:40.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 client.16230 shutdown 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 -- 172.21.15.62:0/3819423905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a42370 msgr2=0x55b5e2a3b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a42370 0x55b5e2a3b940 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f20280103f0 tx=0x7f202801ce50 comp rx=0 tx=0).stop 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 -- 172.21.15.62:0/3819423905 shutdown_connections 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b5e2abcd90 0x55b5e2abf1b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5e2a42370 0x55b5e2a3b940 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f200effd640 0 client.16230 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5e2a38b40 0x55b5e2a4aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 --2- 172.21.15.62:0/3819423905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b5e2a375d0 0x55b5e2a4a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 -- 172.21.15.62:0/3819423905 >> 172.21.15.62:0/3819423905 conn(0x55b5e2a00310 msgr2=0x55b5e2a324e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 -- 172.21.15.62:0/3819423905 shutdown_connections 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 1 -- 172.21.15.62:0/3819423905 wait complete. 2024-09-11T23:56:40.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.953+0000 7f2038381740 20 client.16230 trim_cache size 0 max 16384 2024-09-11T23:56:40.959 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:56:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 -- 172.21.15.62:0/2009302943 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f201c0a2cf0 msgr2=0x7f201c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f201c0a2cf0 0x7f201c0a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2028020000 tx=0x7f202801a000 comp rx=0 tx=0).stop 2024-09-11T23:56:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 -- 172.21.15.62:0/2009302943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030101420 msgr2=0x7f2030199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030101420 0x7f2030199810 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f201800e9f0 tx=0x7f201800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 -- 172.21.15.62:0/2009302943 shutdown_connections 2024-09-11T23:56:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f201c0a2cf0 0x7f201c0a51b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2030106a80 0x7f20301a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030101420 0x7f2030199810 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 --2- 172.21.15.62:0/2009302943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2030100a80 0x7f20301992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 -- 172.21.15.62:0/2009302943 >> 172.21.15.62:0/2009302943 conn(0x7f20300fc7a0 msgr2=0x7f20300fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 -- 172.21.15.62:0/2009302943 shutdown_connections 2024-09-11T23:56:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:40.954+0000 7f2035fa8640 1 -- 172.21.15.62:0/2009302943 wait complete. 2024-09-11T23:56:41.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:41 smithi062 ceph-mon[30958]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.5 MiB/s wr, 126 op/s 2024-09-11T23:56:41.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:41 smithi120 ceph-mon[35182]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.5 MiB/s wr, 126 op/s 2024-09-11T23:56:41.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:41 smithi083 ceph-mon[34805]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.5 MiB/s wr, 126 op/s 2024-09-11T23:56:42.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 240.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:56:42.772 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:56:42.773 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:56:43.048 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:43.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:43 smithi062 ceph-mon[30958]: pgmap v141: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.5 MiB/s wr, 129 op/s 2024-09-11T23:56:43.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.528+0000 7fc21a6a3640 1 -- 172.21.15.62:0/4214696191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc214103740 msgr2=0x7fc214107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.528+0000 7fc21a6a3640 1 --2- 172.21.15.62:0/4214696191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc214103740 0x7fc214107b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009930 tx=0x7fc1fc02a420 comp rx=0 tx=0).stop 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.528+0000 7fc21a6a3640 1 -- 172.21.15.62:0/4214696191 shutdown_connections 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.528+0000 7fc21a6a3640 1 --2- 172.21.15.62:0/4214696191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc214108290 0x7fc21410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.528+0000 7fc21a6a3640 1 --2- 172.21.15.62:0/4214696191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc214103740 0x7fc214107b80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.528+0000 7fc21a6a3640 1 --2- 172.21.15.62:0/4214696191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc214102d90 0x7fc214103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.528+0000 7fc21a6a3640 1 -- 172.21.15.62:0/4214696191 >> 172.21.15.62:0/4214696191 conn(0x7fc2140fc7d0 msgr2=0x7fc2140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.528+0000 7fc21a6a3640 1 -- 172.21.15.62:0/4214696191 shutdown_connections 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.528+0000 7fc21a6a3640 1 -- 172.21.15.62:0/4214696191 wait complete. 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc21a6a3640 1 Processor -- start 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc21a6a3640 1 -- start start 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc21a6a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc214102d90 0x7fc2141009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc21a6a3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc214103740 0x7fc214100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc21a6a3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc214108290 0x7fc214101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc21a6a3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc214112090 con 0x7fc214108290 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc21a6a3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc214111f10 con 0x7fc214102d90 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc21a6a3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc214112210 con 0x7fc214103740 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc218c40640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc214108290 0x7fc214101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc218c40640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc214108290 0x7fc214101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53794/0 (socket says 172.21.15.62:53794) 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc218c40640 1 -- 172.21.15.62:0/665487412 learned_addr learned my addr 172.21.15.62:0/665487412 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.529+0000 7fc213fff640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc214102d90 0x7fc2141009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc20bfff640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc214103740 0x7fc214100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc218c40640 1 -- 172.21.15.62:0/665487412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc214103740 msgr2=0x7fc214100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc218c40640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc214103740 0x7fc214100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc218c40640 1 -- 172.21.15.62:0/665487412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc214102d90 msgr2=0x7fc2141009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc218c40640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc214102d90 0x7fc2141009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc218c40640 1 -- 172.21.15.62:0/665487412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2141a6a90 con 0x7fc214108290 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc218c40640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc214108290 0x7fc214101470 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00eb10 tx=0x7fc20c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc211ffb640 1 -- 172.21.15.62:0/665487412 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20c004b20 con 0x7fc214108290 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc211ffb640 1 -- 172.21.15.62:0/665487412 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc20c004cc0 con 0x7fc214108290 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc21a6a3640 1 -- 172.21.15.62:0/665487412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2141a6d80 con 0x7fc214108290 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc211ffb640 1 -- 172.21.15.62:0/665487412 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20c010760 con 0x7fc214108290 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.530+0000 7fc21a6a3640 1 -- 172.21.15.62:0/665487412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2141a7240 con 0x7fc214108290 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc211ffb640 1 -- 172.21.15.62:0/665487412 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc20c005020 con 0x7fc214108290 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc21ca7c740 1 Processor -- start 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc21ca7c740 1 -- start start 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5562214feb60 0x5562214fef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214ff480 0x556221509800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x55622150c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556221464280 con 0x556221509d40 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55622140bb10 con 0x5562214ff480 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556221460240 con 0x5562214feb60 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x55622150c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc213fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214ff480 0x556221509800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x55622150c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53808/0 (socket says 172.21.15.62:53808) 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc20bfff640 1 -- 172.21.15.62:0/1082474296 learned_addr learned my addr 172.21.15.62:0/1082474296 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc213fff640 1 --2- 172.21.15.62:0/1082474296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214ff480 0x556221509800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:56934/0 (socket says 172.21.15.62:56934) 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc211ffb640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2000a2cf0 0x7fc2000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.533+0000 7fc20bfff640 1 --2- 172.21.15.62:0/1082474296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5562214feb60 0x5562214fef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc218c40640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2000a2cf0 0x7fc2000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc213fff640 1 -- 172.21.15.62:0/1082474296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5562214feb60 msgr2=0x5562214fef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc213fff640 1 --2- 172.21.15.62:0/1082474296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5562214feb60 0x5562214fef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc213fff640 1 -- 172.21.15.62:0/1082474296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 msgr2=0x55622150c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc213fff640 1 --2- 172.21.15.62:0/1082474296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x55622150c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc213fff640 1 -- 172.21.15.62:0/1082474296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562214181f0 con 0x5562214ff480 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc211ffb640 1 -- 172.21.15.62:0/665487412 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc20c0cf9c0 con 0x7fc214108290 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc20bfff640 1 --2- 172.21.15.62:0/1082474296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x55622150c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc213fff640 1 --2- 172.21.15.62:0/1082474296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214ff480 0x556221509800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc204004630 tx=0x7fc20400bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2659a1aef3dd585 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc218c40640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2000a2cf0 0x7fc2000a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc20c09bc10 tx=0x7fc20c004a40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc20a7fc640 1 -- 172.21.15.62:0/1082474296 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204003bd0 con 0x5562214ff480 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc20a7fc640 1 -- 172.21.15.62:0/1082474296 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc204015070 con 0x5562214ff480 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.534+0000 7fc20a7fc640 1 -- 172.21.15.62:0/1082474296 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204004d30 con 0x5562214ff480 2024-09-11T23:56:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 -- 172.21.15.62:0/1082474296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214ff480 msgr2=0x556221509800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/1082474296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214ff480 0x556221509800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc204004630 tx=0x7fc20400bb20 comp rx=0 tx=0).stop 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 -- 172.21.15.62:0/1082474296 shutdown_connections 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/1082474296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x55622150c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/1082474296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214ff480 0x556221509800 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/1082474296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5562214feb60 0x5562214fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 -- 172.21.15.62:0/1082474296 >> 172.21.15.62:0/1082474296 conn(0x5562214fb0f0 msgr2=0x5562214fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 -- 172.21.15.62:0/1082474296 shutdown_connections 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 -- 172.21.15.62:0/1082474296 wait complete. 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 -- start start 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.535+0000 7fc21ca7c740 1 Processor -- start 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x556221509800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x556221506b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 0x5562215072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55622150c910 con 0x5562214ff480 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55622141ef10 con 0x5562214feb60 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55622150ca90 con 0x556221509d40 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc21ca7c740 10 client.0 _ll_register_callbacks cb 0x5562213e7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc213fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x556221506b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc213fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x556221506b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53816/0 (socket says 172.21.15.62:53816) 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc213fff640 1 -- 172.21.15.62:0/3571887721 learned_addr learned my addr 172.21.15.62:0/3571887721 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc20bfff640 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x556221509800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc20bfff640 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 0x5562215072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc213fff640 1 -- 172.21.15.62:0/3571887721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 msgr2=0x5562215072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc213fff640 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 0x5562215072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc213fff640 1 -- 172.21.15.62:0/3571887721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 msgr2=0x556221509800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc213fff640 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x556221509800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc213fff640 1 -- 172.21.15.62:0/3571887721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562211d56a0 con 0x5562214ff480 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc20bfff640 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 0x5562215072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc213fff640 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x556221506b50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc204006010 tx=0x7fc20400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f2026e1134bbf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.536+0000 7fc1f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc1f37fe640 1 -- 172.21.15.62:0/3571887721 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204028060 con 0x5562214ff480 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc1f37fe640 1 -- 172.21.15.62:0/3571887721 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc204011640 con 0x5562214ff480 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc21ca7c740 1 -- 172.21.15.62:0/3571887721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc204015070 con 0x5562214ff480 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc1f37fe640 1 -- 172.21.15.62:0/3571887721 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc204003e60 con 0x5562214ff480 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc21ca7c740 10 client.16242 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc21ca7c740 1 -- 172.21.15.62:0/3571887721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5562215184a0 con 0x5562214ff480 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc1f37fe640 1 -- 172.21.15.62:0/3571887721 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc204015070 con 0x5562214ff480 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc1f37fe640 1 -- 172.21.15.62:0/3571887721 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc204022070 con 0x5562214ff480 2024-09-11T23:56:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc21ca7c740 10 client.16242 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc21ca7c740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc21ca7c740 20 client.16242 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556221519960 0x55622151bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc21ca7c740 4 client.16242 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.537+0000 7fc21ca7c740 1 -- 172.21.15.62:0/3571887721 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562214717f0 con 0x556221519960 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.538+0000 7fc20bfff640 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556221519960 0x55622151bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.538+0000 7fc1f37fe640 10 client.16242 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.538+0000 7fc20bfff640 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556221519960 0x55622151bd50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388dc298638f06e4 server_cookie=b83d531f2345efc8 in_seq=0 out_seq=0 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc1f37fe640 1 -- 172.21.15.62:0/3571887721 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562214717f0 con 0x556221519960 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc1f37fe640 10 client.16242 handle_command_reply: tid=0 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 client.16242 shutdown 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 -- 172.21.15.62:0/3571887721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 msgr2=0x556221506b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x556221506b50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc204006010 tx=0x7fc20400bfa0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 -- 172.21.15.62:0/3571887721 shutdown_connections 2024-09-11T23:56:43.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556221519960 0x55622151bd50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 0x5562215072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x556221506b50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/3571887721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x556221509800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 -- 172.21.15.62:0/3571887721 >> 172.21.15.62:0/3571887721 conn(0x5562214fb0f0 msgr2=0x5562214fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:43.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc1f37fe640 0 client.16242 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:43.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 -- 172.21.15.62:0/3571887721 shutdown_connections 2024-09-11T23:56:43.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 1 -- 172.21.15.62:0/3571887721 wait complete. 2024-09-11T23:56:43.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.539+0000 7fc21ca7c740 20 client.16242 trim_cache size 0 max 16384 2024-09-11T23:56:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.556+0000 7fc21ca7c740 1 Processor -- start 2024-09-11T23:56:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc21ca7c740 1 -- start start 2024-09-11T23:56:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x5562215015c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5562214ff480 0x5562215089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x556221503db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc2141b2370 con 0x556221509d40 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2141afc40 con 0x5562214feb60 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2141ada90 con 0x5562214ff480 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x556221503db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x556221503db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53820/0 (socket says 172.21.15.62:53820) 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc20bfff640 1 -- 172.21.15.62:0/854880133 learned_addr learned my addr 172.21.15.62:0/854880133 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc20bfff640 1 --2- 172.21.15.62:0/854880133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x5562215015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc213fff640 1 --2- 172.21.15.62:0/854880133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5562214ff480 0x5562215089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc20bfff640 1 -- 172.21.15.62:0/854880133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5562214ff480 msgr2=0x5562215089b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc20bfff640 1 --2- 172.21.15.62:0/854880133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5562214ff480 0x5562215089b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc20bfff640 1 -- 172.21.15.62:0/854880133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 msgr2=0x5562215015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc20bfff640 1 --2- 172.21.15.62:0/854880133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x5562215015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc20bfff640 1 -- 172.21.15.62:0/854880133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556221505b00 con 0x556221509d40 2024-09-11T23:56:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.557+0000 7fc20bfff640 1 --2- 172.21.15.62:0/854880133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x556221503db0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc02a940 tx=0x7fc1fc02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f34d68daf32b92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc209ffb640 1 -- 172.21.15.62:0/854880133 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc046040 con 0x556221509d40 2024-09-11T23:56:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc209ffb640 1 -- 172.21.15.62:0/854880133 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1fc003cd0 con 0x556221509d40 2024-09-11T23:56:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc209ffb640 1 -- 172.21.15.62:0/854880133 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc035040 con 0x556221509d40 2024-09-11T23:56:43.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc21ca7c740 1 -- 172.21.15.62:0/854880133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 msgr2=0x556221503db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:43.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/854880133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x556221503db0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc02a940 tx=0x7fc1fc02a420 comp rx=0 tx=0).stop 2024-09-11T23:56:43.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc21ca7c740 1 -- 172.21.15.62:0/854880133 shutdown_connections 2024-09-11T23:56:43.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/854880133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556221509d40 0x556221503db0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/854880133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5562214ff480 0x5562215089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/854880133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x5562215015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc21ca7c740 1 -- 172.21.15.62:0/854880133 >> 172.21.15.62:0/854880133 conn(0x5562214fb0f0 msgr2=0x5562215082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc21ca7c740 1 -- 172.21.15.62:0/854880133 shutdown_connections 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.558+0000 7fc21ca7c740 1 -- 172.21.15.62:0/854880133 wait complete. 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc21ca7c740 1 -- start start 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc21ca7c740 1 Processor -- start 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x55622151cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x55622151d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc21ca7c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 0x55622157d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc2141ae100 con 0x5562214ff480 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2141af570 con 0x5562214feb60 2024-09-11T23:56:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc21ca7c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2141ae7d0 con 0x556221509d40 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc21ca7c740 10 client.0 _ll_register_callbacks cb 0x7fc20c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc213fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x55622151d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc213fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x55622151d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53830/0 (socket says 172.21.15.62:53830) 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc213fff640 1 -- 172.21.15.62:0/2242747321 learned_addr learned my addr 172.21.15.62:0/2242747321 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.559+0000 7fc20bfff640 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x55622151cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc20bfff640 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 0x55622157d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc213fff640 1 -- 172.21.15.62:0/2242747321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 msgr2=0x55622157d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc213fff640 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 0x55622157d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc213fff640 1 -- 172.21.15.62:0/2242747321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 msgr2=0x55622151cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc213fff640 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x55622151cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc213fff640 1 -- 172.21.15.62:0/2242747321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562214f6830 con 0x5562214ff480 2024-09-11T23:56:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc213fff640 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x55622151d510 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc204001eb0 tx=0x7fc204004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5df27ae03d2ee2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc1f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc1f37fe640 1 -- 172.21.15.62:0/2242747321 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204019600 con 0x5562214ff480 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc1f37fe640 1 -- 172.21.15.62:0/2242747321 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc2040197a0 con 0x5562214ff480 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc21ca7c740 1 -- 172.21.15.62:0/2242747321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1fc003cd0 con 0x5562214ff480 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc1f37fe640 1 -- 172.21.15.62:0/2242747321 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2040199a0 con 0x5562214ff480 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc21ca7c740 10 client.16248 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.560+0000 7fc21ca7c740 1 -- 172.21.15.62:0/2242747321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556221507650 con 0x5562214ff480 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc1f37fe640 1 -- 172.21.15.62:0/2242747321 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1fc003cd0 con 0x5562214ff480 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc1f37fe640 1 -- 172.21.15.62:0/2242747321 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc204022040 con 0x5562214ff480 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc21ca7c740 10 client.16248 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc21ca7c740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc21ca7c740 20 client.16248 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556221584fc0 0x5562215873e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:43.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc21ca7c740 4 client.16248 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:56:43.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc21ca7c740 1 -- 172.21.15.62:0/2242747321 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55622146bd00 con 0x556221584fc0 2024-09-11T23:56:43.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc20bfff640 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556221584fc0 0x5562215873e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:43.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc20bfff640 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556221584fc0 0x5562215873e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8ca2e84206e054 server_cookie=e96e31e35d6597d7 in_seq=0 out_seq=0 2024-09-11T23:56:43.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.561+0000 7fc1f37fe640 10 client.16248 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.562+0000 7fc1f37fe640 1 -- 172.21.15.62:0/2242747321 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55622146bd00 con 0x556221584fc0 2024-09-11T23:56:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.562+0000 7fc1f37fe640 10 client.16248 handle_command_reply: tid=0 2024-09-11T23:56:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.562+0000 7fc21ca7c740 1 client.16248 shutdown 2024-09-11T23:56:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 1 -- 172.21.15.62:0/2242747321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 msgr2=0x55622151d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x55622151d510 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc204001eb0 tx=0x7fc204004310 comp rx=0 tx=0).stop 2024-09-11T23:56:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 1 -- 172.21.15.62:0/2242747321 shutdown_connections 2024-09-11T23:56:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556221584fc0 0x5562215873e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556221509d40 0x55622157d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562214ff480 0x55622151d510 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 1 --2- 172.21.15.62:0/2242747321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562214feb60 0x55622151cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 1 -- 172.21.15.62:0/2242747321 >> 172.21.15.62:0/2242747321 conn(0x5562214fb0f0 msgr2=0x556221515560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc1f37fe640 0 client.16248 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 1 -- 172.21.15.62:0/2242747321 shutdown_connections 2024-09-11T23:56:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 1 -- 172.21.15.62:0/2242747321 wait complete. 2024-09-11T23:56:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21ca7c740 20 client.16248 trim_cache size 0 max 16384 2024-09-11T23:56:43.566 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:56:43.566 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:56:43.566 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "4ccd88f1-b3da-44a5-8ab3-dd5bc4d7c8ca", 2024-09-11T23:56:43.567 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:56:43.567 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:56:43.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21a6a3640 1 -- 172.21.15.62:0/665487412 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2000a2cf0 msgr2=0x7fc2000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:43.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21a6a3640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2000a2cf0 0x7fc2000a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc20c09bc10 tx=0x7fc20c004a40 comp rx=0 tx=0).stop 2024-09-11T23:56:43.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21a6a3640 1 -- 172.21.15.62:0/665487412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc214108290 msgr2=0x7fc214101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:43.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.563+0000 7fc21a6a3640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc214108290 0x7fc214101470 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00eb10 tx=0x7fc20c00c2d0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.564+0000 7fc21a6a3640 1 -- 172.21.15.62:0/665487412 shutdown_connections 2024-09-11T23:56:43.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.564+0000 7fc21a6a3640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2000a2cf0 0x7fc2000a51b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.564+0000 7fc21a6a3640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc214108290 0x7fc214101470 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.564+0000 7fc21a6a3640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc214103740 0x7fc214100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.564+0000 7fc21a6a3640 1 --2- 172.21.15.62:0/665487412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc214102d90 0x7fc2141009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:43.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.564+0000 7fc21a6a3640 1 -- 172.21.15.62:0/665487412 >> 172.21.15.62:0/665487412 conn(0x7fc2140fc7d0 msgr2=0x7fc214105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:43.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.564+0000 7fc21a6a3640 1 -- 172.21.15.62:0/665487412 shutdown_connections 2024-09-11T23:56:43.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:43.564+0000 7fc21a6a3640 1 -- 172.21.15.62:0/665487412 wait complete. 2024-09-11T23:56:43.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:43 smithi120 ceph-mon[35182]: pgmap v141: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.5 MiB/s wr, 129 op/s 2024-09-11T23:56:43.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:43 smithi083 ceph-mon[34805]: pgmap v141: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.5 MiB/s wr, 129 op/s 2024-09-11T23:56:44.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:44 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:56:44.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:44 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:44 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:56:44.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:44 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:56:44.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:44 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:44 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:56:44.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:44 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:44 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:44 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:44 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:44 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:44 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:44 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:44 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:44 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:44 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:44 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:56:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:44 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:44 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:56:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:44 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:56:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:44 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:44 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:56:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:44 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:44 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:56:44.783 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 -- 172.21.15.62:0/2212580516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec0102ef0 msgr2=0x7fdec010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 --2- 172.21.15.62:0/2212580516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec0102ef0 0x7fdec010da40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009ca0 tx=0x7fdeb802a950 comp rx=0 tx=0).stop 2024-09-11T23:56:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 -- 172.21.15.62:0/2212580516 shutdown_connections 2024-09-11T23:56:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 --2- 172.21.15.62:0/2212580516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec0102ef0 0x7fdec010da40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 --2- 172.21.15.62:0/2212580516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdec0102570 0x7fdec01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 --2- 172.21.15.62:0/2212580516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec0100e20 0x7fdec0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 -- 172.21.15.62:0/2212580516 >> 172.21.15.62:0/2212580516 conn(0x7fdec00fc7d0 msgr2=0x7fdec00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 -- 172.21.15.62:0/2212580516 shutdown_connections 2024-09-11T23:56:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 -- 172.21.15.62:0/2212580516 wait complete. 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 Processor -- start 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.271+0000 7fdec810d640 1 -- start start 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec810d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdec0100e20 0x7fdec01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec810d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec0102570 0x7fdec0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec810d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec0102ef0 0x7fdec019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec810d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdec01120a0 con 0x7fdec0102570 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec810d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdec0111f20 con 0x7fdec0102ef0 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec810d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdec0112220 con 0x7fdec0100e20 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec56a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec0102570 0x7fdec0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec56a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec0102570 0x7fdec0198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53842/0 (socket says 172.21.15.62:53842) 2024-09-11T23:56:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec56a8640 1 -- 172.21.15.62:0/779481445 learned_addr learned my addr 172.21.15.62:0/779481445 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdec0100e20 0x7fdec01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec66aa640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec0102ef0 0x7fdec019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec56a8640 1 -- 172.21.15.62:0/779481445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdec0100e20 msgr2=0x7fdec01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec56a8640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdec0100e20 0x7fdec01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec56a8640 1 -- 172.21.15.62:0/779481445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec0102ef0 msgr2=0x7fdec019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec56a8640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec0102ef0 0x7fdec019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec56a8640 1 -- 172.21.15.62:0/779481445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec019d370 con 0x7fdec0102570 2024-09-11T23:56:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdec56a8640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec0102570 0x7fdec0198900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400c920 tx=0x7fdeb400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.272+0000 7fdeb2ffd640 1 -- 172.21.15.62:0/779481445 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4007bb0 con 0x7fdec0102570 2024-09-11T23:56:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.273+0000 7fdeb2ffd640 1 -- 172.21.15.62:0/779481445 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb4007d50 con 0x7fdec0102570 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.273+0000 7fdec810d640 1 -- 172.21.15.62:0/779481445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec019d660 con 0x7fdec0102570 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.273+0000 7fdeb2ffd640 1 -- 172.21.15.62:0/779481445 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40187d0 con 0x7fdec0102570 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.273+0000 7fdec810d640 1 -- 172.21.15.62:0/779481445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec01a4f40 con 0x7fdec0102570 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.275+0000 7fdeca4e6740 1 Processor -- start 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.275+0000 7fdeca4e6740 1 -- start start 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.275+0000 7fdeb2ffd640 1 -- 172.21.15.62:0/779481445 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdeb400f450 con 0x7fdec0102570 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee00557880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00557dc0 0x55ee00562140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee00564a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ee004bd2e0 con 0x55ee00562680 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee00464ce0 con 0x55ee005574a0 2024-09-11T23:56:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ee004b92a0 con 0x55ee00557dc0 2024-09-11T23:56:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec56a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee00564a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec56a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee00564a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53854/0 (socket says 172.21.15.62:53854) 2024-09-11T23:56:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec56a8640 1 -- 172.21.15.62:0/806530795 learned_addr learned my addr 172.21.15.62:0/806530795 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec66aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee00557880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/806530795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00557dc0 0x55ee00562140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec66aa640 1 -- 172.21.15.62:0/806530795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00557dc0 msgr2=0x55ee00562140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec66aa640 1 --2- 172.21.15.62:0/806530795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00557dc0 0x55ee00562140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec66aa640 1 -- 172.21.15.62:0/806530795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 msgr2=0x55ee00564a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec66aa640 1 --2- 172.21.15.62:0/806530795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee00564a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec66aa640 1 -- 172.21.15.62:0/806530795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee00449440 con 0x55ee005574a0 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdeb2ffd640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdeac0a2cf0 0x7fdeac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.276+0000 7fdec66aa640 1 --2- 172.21.15.62:0/806530795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee00557880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8005cc0 tx=0x7fdeb802ff40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d62c8d90b8b956 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.277+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdeac0a2cf0 0x7fdeac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.277+0000 7fdeb2ffd640 1 -- 172.21.15.62:0/779481445 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fdeb40cf4c0 con 0x7fdec0102570 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.277+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdeac0a2cf0 0x7fdeac0a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55ee00554890 tx=0x7fdebc00a2d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.278+0000 7fdeabfff640 1 -- 172.21.15.62:0/806530795 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8048070 con 0x55ee005574a0 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.278+0000 7fdeabfff640 1 -- 172.21.15.62:0/806530795 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb802faf0 con 0x55ee005574a0 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.278+0000 7fdeabfff640 1 -- 172.21.15.62:0/806530795 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8034040 con 0x55ee005574a0 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 -- 172.21.15.62:0/806530795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 msgr2=0x55ee00557880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/806530795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee00557880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8005cc0 tx=0x7fdeb802ff40 comp rx=0 tx=0).stop 2024-09-11T23:56:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 -- 172.21.15.62:0/806530795 shutdown_connections 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/806530795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee00564a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/806530795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00557dc0 0x55ee00562140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/806530795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee00557880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 -- 172.21.15.62:0/806530795 >> 172.21.15.62:0/806530795 conn(0x55ee00553a30 msgr2=0x55ee00553e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 -- 172.21.15.62:0/806530795 shutdown_connections 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 -- 172.21.15.62:0/806530795 wait complete. 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 -- start start 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.279+0000 7fdeca4e6740 1 Processor -- start 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee005574a0 0x55ee00562030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee00557dc0 0x55ee0055f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee0055f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ee00565250 con 0x55ee00562680 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee00478050 con 0x55ee00557dc0 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ee005653d0 con 0x55ee005574a0 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdeca4e6740 10 client.0 _ll_register_callbacks cb 0x55ee004b8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec66aa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee0055f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec66aa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee0055f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53856/0 (socket says 172.21.15.62:53856) 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec66aa640 1 -- 172.21.15.62:0/940951500 learned_addr learned my addr 172.21.15.62:0/940951500 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee005574a0 0x55ee00562030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec56a8640 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee00557dc0 0x55ee0055f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec66aa640 1 -- 172.21.15.62:0/940951500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee005574a0 msgr2=0x55ee00562030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec66aa640 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee005574a0 0x55ee00562030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec66aa640 1 -- 172.21.15.62:0/940951500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee00557dc0 msgr2=0x55ee0055f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec66aa640 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee00557dc0 0x55ee0055f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec66aa640 1 -- 172.21.15.62:0/940951500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0022e3c0 con 0x55ee00562680 2024-09-11T23:56:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee005574a0 0x55ee00562030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:56:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdec66aa640 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee0055f880 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8006010 tx=0x7fdeb8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9269178f19715e0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdea97fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdea97fa640 1 -- 172.21.15.62:0/940951500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8051060 con 0x55ee00562680 2024-09-11T23:56:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdeca4e6740 1 -- 172.21.15.62:0/940951500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb802faf0 con 0x55ee00562680 2024-09-11T23:56:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.280+0000 7fdea97fa640 1 -- 172.21.15.62:0/940951500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fdeb80042c0 con 0x55ee00562680 2024-09-11T23:56:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdea97fa640 1 -- 172.21.15.62:0/940951500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdeb8048070 con 0x55ee00562680 2024-09-11T23:56:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdeca4e6740 10 client.16260 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdeca4e6740 1 -- 172.21.15.62:0/940951500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee00570c90 con 0x55ee00562680 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdea97fa640 1 -- 172.21.15.62:0/940951500 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb802faf0 con 0x55ee00562680 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdea97fa640 1 -- 172.21.15.62:0/940951500 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fdeb80307e0 con 0x55ee00562680 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdeca4e6740 10 client.16260 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdeca4e6740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdeca4e6740 20 client.16260 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ee00571fe0 0x55ee005743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdeca4e6740 4 client.16260 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdeca4e6740 1 -- 172.21.15.62:0/940951500 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee004ca850 con 0x55ee00571fe0 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.281+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ee00571fe0 0x55ee005743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.282+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ee00571fe0 0x55ee005743d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b197bf19fce3c1 server_cookie=a5e176af52ea92ed in_seq=0 out_seq=0 2024-09-11T23:56:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.282+0000 7fdea97fa640 10 client.16260 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.282+0000 7fdea97fa640 1 -- 172.21.15.62:0/940951500 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee004ca850 con 0x55ee00571fe0 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.282+0000 7fdea97fa640 10 client.16260 handle_command_reply: tid=0 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.282+0000 7fdeca4e6740 1 client.16260 shutdown 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 1 -- 172.21.15.62:0/940951500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 msgr2=0x55ee0055f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee0055f880 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8006010 tx=0x7fdeb8003c20 comp rx=0 tx=0).stop 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 1 -- 172.21.15.62:0/940951500 shutdown_connections 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ee00571fe0 0x55ee005743d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00562680 0x55ee0055f880 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee00557dc0 0x55ee0055f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/940951500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee005574a0 0x55ee00562030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 1 -- 172.21.15.62:0/940951500 >> 172.21.15.62:0/940951500 conn(0x55ee00553a30 msgr2=0x55ee00552b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:45.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdea97fa640 0 client.16260 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:45.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 1 -- 172.21.15.62:0/940951500 shutdown_connections 2024-09-11T23:56:45.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 1 -- 172.21.15.62:0/940951500 wait complete. 2024-09-11T23:56:45.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.283+0000 7fdeca4e6740 20 client.16260 trim_cache size 0 max 16384 2024-09-11T23:56:45.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.300+0000 7fdeca4e6740 1 Processor -- start 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.300+0000 7fdeca4e6740 1 -- start start 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee00553e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee0055c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00562680 0x55ee0055ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdec01afe60 con 0x55ee00557dc0 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdec01ad730 con 0x55ee005574a0 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdec01ab580 con 0x55ee00562680 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec56a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee0055c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec56a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee0055c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53858/0 (socket says 172.21.15.62:53858) 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec56a8640 1 -- 172.21.15.62:0/3279104263 learned_addr learned my addr 172.21.15.62:0/3279104263 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/3279104263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee00553e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec66aa640 1 --2- 172.21.15.62:0/3279104263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00562680 0x55ee0055ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec56a8640 1 -- 172.21.15.62:0/3279104263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00562680 msgr2=0x55ee0055ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec56a8640 1 --2- 172.21.15.62:0/3279104263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00562680 0x55ee0055ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec56a8640 1 -- 172.21.15.62:0/3279104263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 msgr2=0x55ee00553e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec56a8640 1 --2- 172.21.15.62:0/3279104263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee00553e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec56a8640 1 -- 172.21.15.62:0/3279104263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee005623a0 con 0x55ee00557dc0 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.301+0000 7fdec56a8640 1 --2- 172.21.15.62:0/3279104263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee0055c540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4000c00 tx=0x7fdeb40d2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3812b1edc6de1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeab7fe640 1 -- 172.21.15.62:0/3279104263 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb409b370 con 0x55ee00557dc0 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeab7fe640 1 -- 172.21.15.62:0/3279104263 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb409c110 con 0x55ee00557dc0 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeab7fe640 1 -- 172.21.15.62:0/3279104263 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40ccec0 con 0x55ee00557dc0 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3279104263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 msgr2=0x55ee0055c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/3279104263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee0055c540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4000c00 tx=0x7fdeb40d2f90 comp rx=0 tx=0).stop 2024-09-11T23:56:45.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3279104263 shutdown_connections 2024-09-11T23:56:45.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/3279104263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00562680 0x55ee0055ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/3279104263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee0055c540 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/3279104263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee00553e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3279104263 >> 172.21.15.62:0/3279104263 conn(0x55ee00553a30 msgr2=0x55ee0055dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:45.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3279104263 shutdown_connections 2024-09-11T23:56:45.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.302+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3279104263 wait complete. 2024-09-11T23:56:45.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdeca4e6740 1 -- start start 2024-09-11T23:56:45.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdeca4e6740 1 Processor -- start 2024-09-11T23:56:45.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee005761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee005766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdeca4e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00562680 0x55ee005ded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdec01abbf0 con 0x55ee00557dc0 2024-09-11T23:56:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdec01ad060 con 0x55ee005574a0 2024-09-11T23:56:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdeca4e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdec01ac2c0 con 0x55ee00562680 2024-09-11T23:56:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdeca4e6740 10 client.0 _ll_register_callbacks cb 0x7fdeb8046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdec56a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee005766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdec56a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee005766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53866/0 (socket says 172.21.15.62:53866) 2024-09-11T23:56:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.303+0000 7fdec56a8640 1 -- 172.21.15.62:0/3959022326 learned_addr learned my addr 172.21.15.62:0/3959022326 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdec66aa640 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00562680 0x55ee005ded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee005761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdec56a8640 1 -- 172.21.15.62:0/3959022326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00562680 msgr2=0x55ee005ded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdec56a8640 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00562680 0x55ee005ded30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdec56a8640 1 -- 172.21.15.62:0/3959022326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 msgr2=0x55ee005761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdec56a8640 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee005761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdec56a8640 1 -- 172.21.15.62:0/3959022326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee00576fc0 con 0x55ee00557dc0 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdec56a8640 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee005766e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40ccbe0 tx=0x7fdeb409b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97b12d118383574 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdea97fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdea97fa640 1 -- 172.21.15.62:0/3959022326 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40d2d00 con 0x55ee00557dc0 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdea97fa640 1 -- 172.21.15.62:0/3959022326 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fdeb4098080 con 0x55ee00557dc0 2024-09-11T23:56:45.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3959022326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb409c110 con 0x55ee00557dc0 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdea97fa640 1 -- 172.21.15.62:0/3959022326 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdeb409ba10 con 0x55ee00557dc0 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdeca4e6740 10 client.16266 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.304+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3959022326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee005e0390 con 0x55ee00557dc0 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdea97fa640 1 -- 172.21.15.62:0/3959022326 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb409c110 con 0x55ee00557dc0 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdea97fa640 1 -- 172.21.15.62:0/3959022326 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fdeb40cf040 con 0x55ee00557dc0 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdeca4e6740 10 client.16266 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdeca4e6740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdeca4e6740 20 client.16266 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ee005e0980 0x55ee005e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdeca4e6740 4 client.16266 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:56:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3959022326 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee004c4d60 con 0x55ee005e0980 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ee005e0980 0x55ee005e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdec5ea9640 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ee005e0980 0x55ee005e2da0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78742579c0c20711 server_cookie=44fb71139b8cf77e in_seq=0 out_seq=0 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.305+0000 7fdea97fa640 10 client.16266 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdea97fa640 1 -- 172.21.15.62:0/3959022326 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ee004c4d60 con 0x55ee005e0980 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdea97fa640 10 client.16266 handle_command_reply: tid=0 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 client.16266 shutdown 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3959022326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 msgr2=0x55ee005766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee005766e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40ccbe0 tx=0x7fdeb409b3e0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3959022326 shutdown_connections 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ee005e0980 0x55ee005e2da0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ee00562680 0x55ee005ded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee00557dc0 0x55ee005766e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdea97fa640 0 client.16266 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:45.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 --2- 172.21.15.62:0/3959022326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee005574a0 0x55ee005761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3959022326 >> 172.21.15.62:0/3959022326 conn(0x55ee00553a30 msgr2=0x55ee00552350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:45.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3959022326 shutdown_connections 2024-09-11T23:56:45.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 1 -- 172.21.15.62:0/3959022326 wait complete. 2024-09-11T23:56:45.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.306+0000 7fdeca4e6740 20 client.16266 trim_cache size 0 max 16384 2024-09-11T23:56:45.312 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:56:45.312 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:56:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:56:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: "4ccd88f1-b3da-44a5-8ab3-dd5bc4d7c8ca": { 2024-09-11T23:56:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:56:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "4ccd88f1-b3da-44a5-8ab3-dd5bc4d7c8ca", 2024-09-11T23:56:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:56:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:45.313 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:45.313 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:56:45.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 -- 172.21.15.62:0/779481445 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdeac0a2cf0 msgr2=0x7fdeac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:45.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdeac0a2cf0 0x7fdeac0a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55ee00554890 tx=0x7fdebc00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 -- 172.21.15.62:0/779481445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec0102570 msgr2=0x7fdec0198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:45.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec0102570 0x7fdec0198900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400c920 tx=0x7fdeb400cd20 comp rx=0 tx=0).stop 2024-09-11T23:56:45.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 -- 172.21.15.62:0/779481445 shutdown_connections 2024-09-11T23:56:45.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdeac0a2cf0 0x7fdeac0a51b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec0102ef0 0x7fdec019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdec0102570 0x7fdec0198900 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 --2- 172.21.15.62:0/779481445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdec0100e20 0x7fdec01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:45.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 -- 172.21.15.62:0/779481445 >> 172.21.15.62:0/779481445 conn(0x7fdec00fc7d0 msgr2=0x7fdec0105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:45.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 -- 172.21.15.62:0/779481445 shutdown_connections 2024-09-11T23:56:45.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:45.307+0000 7fdec810d640 1 -- 172.21.15.62:0/779481445 wait complete. 2024-09-11T23:56:45.414 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:56:45.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:45 smithi062 ceph-mon[30958]: pgmap v142: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-09-11T23:56:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:45 smithi120 ceph-mon[35182]: pgmap v142: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-09-11T23:56:45.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:45 smithi083 ceph-mon[34805]: pgmap v142: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-09-11T23:56:46.102 INFO:tasks.cephfs.filesystem:scrub status for tag:4ccd88f1-b3da-44a5-8ab3-dd5bc4d7c8ca - {'path': '/', 'tag': '4ccd88f1-b3da-44a5-8ab3-dd5bc4d7c8ca', 'options': 'recursive,force'} 2024-09-11T23:56:46.205 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:46.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.671+0000 7fcc6685b640 1 -- 172.21.15.62:0/3803811849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc601027a0 msgr2=0x7fcc601067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:46.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.671+0000 7fcc6685b640 1 --2- 172.21.15.62:0/3803811849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc601027a0 0x7fcc601067f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009900 tx=0x7fcc4802a410 comp rx=0 tx=0).stop 2024-09-11T23:56:46.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.671+0000 7fcc6685b640 1 -- 172.21.15.62:0/3803811849 shutdown_connections 2024-09-11T23:56:46.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.671+0000 7fcc6685b640 1 --2- 172.21.15.62:0/3803811849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc60106f20 0x7fcc6010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:46.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.671+0000 7fcc6685b640 1 --2- 172.21.15.62:0/3803811849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc601027a0 0x7fcc601067f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:46.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.671+0000 7fcc6685b640 1 --2- 172.21.15.62:0/3803811849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc60101df0 0x7fcc601021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:46.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.671+0000 7fcc6685b640 1 -- 172.21.15.62:0/3803811849 >> 172.21.15.62:0/3803811849 conn(0x7fcc600fd8b0 msgr2=0x7fcc600ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:46.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.671+0000 7fcc6685b640 1 -- 172.21.15.62:0/3803811849 shutdown_connections 2024-09-11T23:56:46.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.671+0000 7fcc6685b640 1 -- 172.21.15.62:0/3803811849 wait complete. 2024-09-11T23:56:46.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc6685b640 1 Processor -- start 2024-09-11T23:56:46.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc6685b640 1 -- start start 2024-09-11T23:56:46.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc6685b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc60101df0 0x7fcc60198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:46.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc6685b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc601027a0 0x7fcc60198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:46.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc6685b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc60106f20 0x7fcc6019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:46.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc6685b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcc60111f80 con 0x7fcc60106f20 2024-09-11T23:56:46.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc6685b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc60111e00 con 0x7fcc601027a0 2024-09-11T23:56:46.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc6685b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcc60112100 con 0x7fcc60101df0 2024-09-11T23:56:46.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc64df8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc60106f20 0x7fcc6019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:46.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc64df8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc60106f20 0x7fcc6019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53876/0 (socket says 172.21.15.62:53876) 2024-09-11T23:56:46.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc64df8640 1 -- 172.21.15.62:0/3100306731 learned_addr learned my addr 172.21.15.62:0/3100306731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:46.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc57fff640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc601027a0 0x7fcc60198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:46.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.672+0000 7fcc5ffff640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc60101df0 0x7fcc60198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:46.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc64df8640 1 -- 172.21.15.62:0/3100306731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc60101df0 msgr2=0x7fcc60198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:46.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc64df8640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc60101df0 0x7fcc60198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:46.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc64df8640 1 -- 172.21.15.62:0/3100306731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc601027a0 msgr2=0x7fcc60198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:46.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc64df8640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc601027a0 0x7fcc60198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:46.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc64df8640 1 -- 172.21.15.62:0/3100306731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6019d3f0 con 0x7fcc60106f20 2024-09-11T23:56:46.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc64df8640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc60106f20 0x7fcc6019cd10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800cad0 tx=0x7fcc5800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:46.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc5dffb640 1 -- 172.21.15.62:0/3100306731 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800ae20 con 0x7fcc60106f20 2024-09-11T23:56:46.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc5dffb640 1 -- 172.21.15.62:0/3100306731 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc58005020 con 0x7fcc60106f20 2024-09-11T23:56:46.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc6685b640 1 -- 172.21.15.62:0/3100306731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6019d6e0 con 0x7fcc60106f20 2024-09-11T23:56:46.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc5dffb640 1 -- 172.21.15.62:0/3100306731 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800f6f0 con 0x7fcc60106f20 2024-09-11T23:56:46.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.673+0000 7fcc6685b640 1 -- 172.21.15.62:0/3100306731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc601a4fc0 con 0x7fcc60106f20 2024-09-11T23:56:46.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.674+0000 7fcc6685b640 1 -- 172.21.15.62:0/3100306731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc601021d0 con 0x7fcc60106f20 2024-09-11T23:56:46.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.675+0000 7fcc5dffb640 1 -- 172.21.15.62:0/3100306731 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcc580051d0 con 0x7fcc60106f20 2024-09-11T23:56:46.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.676+0000 7fcc5dffb640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcc4c0a2cf0 0x7fcc4c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:46.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.676+0000 7fcc5ffff640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcc4c0a2cf0 0x7fcc4c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:46.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.676+0000 7fcc5dffb640 1 -- 172.21.15.62:0/3100306731 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fcc580cf700 con 0x7fcc60106f20 2024-09-11T23:56:46.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.677+0000 7fcc5ffff640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcc4c0a2cf0 0x7fcc4c0a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000bbe0 tx=0x7fcc5000b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:46.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.681+0000 7fcc5dffb640 1 -- 172.21.15.62:0/3100306731 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc58014030 con 0x7fcc60106f20 2024-09-11T23:56:46.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.893+0000 7fcc6685b640 1 -- 172.21.15.62:0/3100306731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc6002d7c0 con 0x7fcc60106f20 2024-09-11T23:56:46.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.894+0000 7fcc5dffb640 1 -- 172.21.15.62:0/3100306731 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7fcc5809c350 con 0x7fcc60106f20 2024-09-11T23:56:46.893 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:56:46.893 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"btime":"2024-09-11T23:55:51:325581+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:55:50.698407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:56:46.893 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-09-11T23:56:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 -- 172.21.15.62:0/3100306731 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcc4c0a2cf0 msgr2=0x7fcc4c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcc4c0a2cf0 0x7fcc4c0a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000bbe0 tx=0x7fcc5000b390 comp rx=0 tx=0).stop 2024-09-11T23:56:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 -- 172.21.15.62:0/3100306731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc60106f20 msgr2=0x7fcc6019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc60106f20 0x7fcc6019cd10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800cad0 tx=0x7fcc5800cf90 comp rx=0 tx=0).stop 2024-09-11T23:56:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 -- 172.21.15.62:0/3100306731 shutdown_connections 2024-09-11T23:56:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fcc4c0a2cf0 0x7fcc4c0a51b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc60106f20 0x7fcc6019cd10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc601027a0 0x7fcc60198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 --2- 172.21.15.62:0/3100306731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc60101df0 0x7fcc60198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:46.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 -- 172.21.15.62:0/3100306731 >> 172.21.15.62:0/3100306731 conn(0x7fcc600fd8b0 msgr2=0x7fcc600fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:46.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 -- 172.21.15.62:0/3100306731 shutdown_connections 2024-09-11T23:56:46.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:46.896+0000 7fcc6685b640 1 -- 172.21.15.62:0/3100306731 wait complete. 2024-09-11T23:56:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 245.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:56:47.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:47 smithi120 ceph-mon[35182]: pgmap v143: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-09-11T23:56:47.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:47 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3100306731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:47.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:47 smithi062 ceph-mon[30958]: pgmap v143: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-09-11T23:56:47.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:47 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3100306731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:47 smithi083 ceph-mon[34805]: pgmap v143: 97 pgs: 97 active+clean; 2.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-09-11T23:56:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:47 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3100306731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:56:47.696 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id a13053d1 --release --await' 2024-09-11T23:56:47.696 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id a13053d1 --release --await 2024-09-11T23:56:47.961 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:56:48.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.428+0000 7f97c67f2640 1 -- 172.21.15.62:0/1386755199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97c0100e20 msgr2=0x7f97c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:48.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.428+0000 7f97c67f2640 1 --2- 172.21.15.62:0/1386755199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97c0100e20 0x7f97c0101200 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009900 tx=0x7f97b002a480 comp rx=0 tx=0).stop 2024-09-11T23:56:48.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.428+0000 7f97c67f2640 1 -- 172.21.15.62:0/1386755199 shutdown_connections 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.428+0000 7f97c67f2640 1 --2- 172.21.15.62:0/1386755199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102ef0 0x7f97c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.428+0000 7f97c67f2640 1 --2- 172.21.15.62:0/1386755199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c0102570 0x7f97c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.428+0000 7f97c67f2640 1 --2- 172.21.15.62:0/1386755199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97c0100e20 0x7f97c0101200 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.428+0000 7f97c67f2640 1 -- 172.21.15.62:0/1386755199 >> 172.21.15.62:0/1386755199 conn(0x7f97c00fc7d0 msgr2=0x7f97c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.428+0000 7f97c67f2640 1 -- 172.21.15.62:0/1386755199 shutdown_connections 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.428+0000 7f97c67f2640 1 -- 172.21.15.62:0/1386755199 wait complete. 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.428+0000 7f97c67f2640 1 Processor -- start 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97c67f2640 1 -- start start 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97c67f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c0100e20 0x7f97c0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97c67f2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97c0102570 0x7f97c0199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97c67f2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102ef0 0x7f97c01a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97c67f2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0110f50 con 0x7f97c0102570 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97c67f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0110dd0 con 0x7f97c0100e20 2024-09-11T23:56:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97c67f2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01110d0 con 0x7f97c0102ef0 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97c0102570 0x7f97c0199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97c0102570 0x7f97c0199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53888/0 (socket says 172.21.15.62:53888) 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97bf7fe640 1 -- 172.21.15.62:0/837473957 learned_addr learned my addr 172.21.15.62:0/837473957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97bffff640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c0100e20 0x7f97c0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97bf7fe640 1 -- 172.21.15.62:0/837473957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102ef0 msgr2=0x7f97c01a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102ef0 0x7f97c01a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102ef0 0x7f97c01a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97bf7fe640 1 -- 172.21.15.62:0/837473957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c0100e20 msgr2=0x7f97c0199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c0100e20 0x7f97c0199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97bf7fe640 1 -- 172.21.15.62:0/837473957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c01a4810 con 0x7f97c0102570 2024-09-11T23:56:48.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.429+0000 7f97bffff640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c0100e20 0x7f97c0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:56:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.430+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97c0102570 0x7f97c0199790 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f97b400c920 tx=0x7f97b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.430+0000 7f97bd7fa640 1 -- 172.21.15.62:0/837473957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4007c70 con 0x7f97c0102570 2024-09-11T23:56:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.430+0000 7f97bd7fa640 1 -- 172.21.15.62:0/837473957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b4007e10 con 0x7f97c0102570 2024-09-11T23:56:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.430+0000 7f97c67f2640 1 -- 172.21.15.62:0/837473957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c01a49e0 con 0x7f97c0102570 2024-09-11T23:56:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.430+0000 7f97bd7fa640 1 -- 172.21.15.62:0/837473957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4018890 con 0x7f97c0102570 2024-09-11T23:56:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.430+0000 7f97c67f2640 1 -- 172.21.15.62:0/837473957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c01a4f20 con 0x7f97c0102570 2024-09-11T23:56:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.432+0000 7f97bd7fa640 1 -- 172.21.15.62:0/837473957 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f97b4018a70 con 0x7f97c0102570 2024-09-11T23:56:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.432+0000 7f97c8bcb740 1 Processor -- start 2024-09-11T23:56:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.432+0000 7f97c8bcb740 1 -- start start 2024-09-11T23:56:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.432+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a977351340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.432+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 0x55a977360110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a977362a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a9772b6620 con 0x55a977360650 2024-09-11T23:56:48.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a97725de50 con 0x55a977351880 2024-09-11T23:56:48.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a9772b25e0 con 0x55a977350f60 2024-09-11T23:56:48.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a977362a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c4d8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a977351340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c4d8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a977351340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:39028/0 (socket says 172.21.15.62:39028) 2024-09-11T23:56:48.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c4d8f640 1 -- 172.21.15.62:0/3962782367 learned_addr learned my addr 172.21.15.62:0/3962782367 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:48.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97bd7fa640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f97a80a2cf0 0x7f97a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97bffff640 1 --2- 172.21.15.62:0/3962782367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 0x55a977360110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f97a80a2cf0 0x7f97a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c4d8f640 1 -- 172.21.15.62:0/3962782367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 msgr2=0x55a977360110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/3962782367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 0x55a977360110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c4d8f640 1 -- 172.21.15.62:0/3962782367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 msgr2=0x55a977362a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97bd7fa640 1 -- 172.21.15.62:0/837473957 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f97b40cf620 con 0x7f97c0102570 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/3962782367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a977362a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c4d8f640 1 -- 172.21.15.62:0/3962782367 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a97724ed40 con 0x55a977350f60 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97bffff640 1 --2- 172.21.15.62:0/3962782367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 0x55a977360110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/3962782367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a977362a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.433+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/3962782367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a977351340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97b8013840 tx=0x7f97b8013d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95fd643f858dde40 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f97a80a2cf0 0x7f97a80a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f97b8004000 tx=0x7f97b801c000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97a67fc640 1 -- 172.21.15.62:0/3962782367 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8022070 con 0x55a977350f60 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97a67fc640 1 -- 172.21.15.62:0/3962782367 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b8003d40 con 0x55a977350f60 2024-09-11T23:56:48.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97a67fc640 1 -- 172.21.15.62:0/3962782367 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b801d410 con 0x55a977350f60 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3962782367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 msgr2=0x55a977351340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/3962782367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a977351340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97b8013840 tx=0x7f97b8013d00 comp rx=0 tx=0).stop 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3962782367 shutdown_connections 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/3962782367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a977362a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/3962782367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 0x55a977360110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/3962782367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a977351340 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3962782367 >> 172.21.15.62:0/3962782367 conn(0x55a97734d5f0 msgr2=0x55a97734d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3962782367 shutdown_connections 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.434+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3962782367 wait complete. 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.435+0000 7f97c8bcb740 1 -- start start 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.435+0000 7f97c8bcb740 1 Processor -- start 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.435+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a97735d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.435+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 0x55a97735daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.435+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a97735e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.435+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a977363220 con 0x55a977360650 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.435+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a97723ef10 con 0x55a977351880 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.435+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a9773633a0 con 0x55a977350f60 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c8bcb740 10 client.0 _ll_register_callbacks cb 0x55a977262db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c4d8f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a97735e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c4d8f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a97735e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53904/0 (socket says 172.21.15.62:53904) 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c4d8f640 1 -- 172.21.15.62:0/2529639 learned_addr learned my addr 172.21.15.62:0/2529639 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97bffff640 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a97735d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 0x55a97735daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c4d8f640 1 -- 172.21.15.62:0/2529639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 msgr2=0x55a97735d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a97735d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c4d8f640 1 -- 172.21.15.62:0/2529639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 msgr2=0x55a97735daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 0x55a97735daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c4d8f640 1 -- 172.21.15.62:0/2529639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a976fc5580 con 0x55a977360650 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a97735e030 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009570 tx=0x7f97b8004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c0d881448aeafa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f979f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f979f7fe640 1 -- 172.21.15.62:0/2529639 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8022030 con 0x55a977360650 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f979f7fe640 1 -- 172.21.15.62:0/2529639 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f97b80045f0 con 0x55a977360650 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c8bcb740 1 -- 172.21.15.62:0/2529639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b8003d40 con 0x55a977360650 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f979f7fe640 1 -- 172.21.15.62:0/2529639 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97b8039050 con 0x55a977360650 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c8bcb740 10 client.16284 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.436+0000 7f97c8bcb740 1 -- 172.21.15.62:0/2529639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a97708f960 con 0x55a977360650 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.437+0000 7f979f7fe640 1 -- 172.21.15.62:0/2529639 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b8003d40 con 0x55a977360650 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.437+0000 7f979f7fe640 1 -- 172.21.15.62:0/2529639 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f97b802a410 con 0x55a977360650 2024-09-11T23:56:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.437+0000 7f97c8bcb740 10 client.16284 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.437+0000 7f97c8bcb740 10 client.16284 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.437+0000 7f97c8bcb740 20 client.16284 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.437+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a977370220 0x55a977372610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.437+0000 7f97c8bcb740 4 client.16284 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.437+0000 7f97c8bcb740 1 -- 172.21.15.62:0/2529639 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9772c3b90 con 0x55a977370220 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.437+0000 7f97bffff640 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a977370220 0x55a977372610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.438+0000 7f97bffff640 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a977370220 0x55a977372610 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1834abc2731e8834 server_cookie=439b5eaa0c5ba716 in_seq=0 out_seq=0 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.438+0000 7f979f7fe640 10 client.16284 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.438+0000 7f979f7fe640 1 -- 172.21.15.62:0/2529639 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9772c3b90 con 0x55a977370220 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.438+0000 7f979f7fe640 10 client.16284 handle_command_reply: tid=0 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.438+0000 7f97c8bcb740 1 client.16284 shutdown 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 1 -- 172.21.15.62:0/2529639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 msgr2=0x55a97735e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a97735e030 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009570 tx=0x7f97b8004970 comp rx=0 tx=0).stop 2024-09-11T23:56:48.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 1 -- 172.21.15.62:0/2529639 shutdown_connections 2024-09-11T23:56:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a977370220 0x55a977372610 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977360650 0x55a97735e030 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977351880 0x55a97735daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/2529639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a97735d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 1 -- 172.21.15.62:0/2529639 >> 172.21.15.62:0/2529639 conn(0x55a97734d5f0 msgr2=0x55a977353320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f979f7fe640 0 client.16284 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 1 -- 172.21.15.62:0/2529639 shutdown_connections 2024-09-11T23:56:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 1 -- 172.21.15.62:0/2529639 wait complete. 2024-09-11T23:56:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.439+0000 7f97c8bcb740 20 client.16284 trim_cache size 0 max 16384 2024-09-11T23:56:48.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97c8bcb740 1 Processor -- start 2024-09-11T23:56:48.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97c8bcb740 1 -- start start 2024-09-11T23:56:48.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a9773531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977377c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 0x55a977378180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01adeb0 con 0x55a977351880 2024-09-11T23:56:48.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01ab780 con 0x55a977360650 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01a95d0 con 0x55a977350f60 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977377c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977377c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53912/0 (socket says 172.21.15.62:53912) 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.456+0000 7f97bf7fe640 1 -- 172.21.15.62:0/1865654199 learned_addr learned my addr 172.21.15.62:0/1865654199 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.457+0000 7f97bf7fe640 1 -- 172.21.15.62:0/1865654199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 msgr2=0x55a9773531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.457+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/1865654199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a9773531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.457+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/1865654199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 0x55a977378180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.457+0000 7f97bf7fe640 1 -- 172.21.15.62:0/1865654199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 msgr2=0x55a977378180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.457+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/1865654199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 0x55a977378180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.457+0000 7f97bf7fe640 1 -- 172.21.15.62:0/1865654199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a97734bbb0 con 0x55a977351880 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.457+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/1865654199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977377c40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f97b4098540 tx=0x7f97b40d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd3bb8d4e316477 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:48.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.457+0000 7f97a5ffb640 1 -- 172.21.15.62:0/1865654199 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b400de20 con 0x55a977351880 2024-09-11T23:56:48.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.457+0000 7f97a5ffb640 1 -- 172.21.15.62:0/1865654199 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b409ff20 con 0x55a977351880 2024-09-11T23:56:48.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.457+0000 7f97a5ffb640 1 -- 172.21.15.62:0/1865654199 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40981e0 con 0x55a977351880 2024-09-11T23:56:48.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 -- 172.21.15.62:0/1865654199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 msgr2=0x55a977377c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:48.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/1865654199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977377c40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f97b4098540 tx=0x7f97b40d1040 comp rx=0 tx=0).stop 2024-09-11T23:56:48.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 -- 172.21.15.62:0/1865654199 shutdown_connections 2024-09-11T23:56:48.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/1865654199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 0x55a977378180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/1865654199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977377c40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/1865654199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a9773531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 -- 172.21.15.62:0/1865654199 >> 172.21.15.62:0/1865654199 conn(0x55a97734d5f0 msgr2=0x55a977352ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 -- 172.21.15.62:0/1865654199 shutdown_connections 2024-09-11T23:56:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 -- 172.21.15.62:0/1865654199 wait complete. 2024-09-11T23:56:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 -- start start 2024-09-11T23:56:48.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.458+0000 7f97c8bcb740 1 Processor -- start 2024-09-11T23:56:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a977368630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977368b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97c8bcb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 0x55a977365600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01a9c40 con 0x55a977351880 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01ab0b0 con 0x55a977360650 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97c8bcb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01aa310 con 0x55a977350f60 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97c8bcb740 10 client.0 _ll_register_callbacks cb 0x7f97b8013610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977368b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977368b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53918/0 (socket says 172.21.15.62:53918) 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97bf7fe640 1 -- 172.21.15.62:0/3974292986 learned_addr learned my addr 172.21.15.62:0/3974292986 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97bffff640 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a977368630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 0x55a977365600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97bf7fe640 1 -- 172.21.15.62:0/3974292986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 msgr2=0x55a977368630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a977368630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97bf7fe640 1 -- 172.21.15.62:0/3974292986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 msgr2=0x55a977365600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 0x55a977365600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97bf7fe640 1 -- 172.21.15.62:0/3974292986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a977365b40 con 0x55a977351880 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97c4d8f640 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 0x55a977365600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97bf7fe640 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977368b70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f97b40cce50 tx=0x7f97b409bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4072b895d59f246b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f979f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f979f7fe640 1 -- 172.21.15.62:0/3974292986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40b9070 con 0x55a977351880 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f979f7fe640 1 -- 172.21.15.62:0/3974292986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f97b400fb30 con 0x55a977351880 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.459+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3974292986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b409ff20 con 0x55a977351880 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f979f7fe640 1 -- 172.21.15.62:0/3974292986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97b40a4350 con 0x55a977351880 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f97c8bcb740 10 client.16290 fetch_fsmap learned FSMap version 29 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3974292986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9773db370 con 0x55a977351880 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f979f7fe640 1 -- 172.21.15.62:0/3974292986 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b409ff20 con 0x55a977351880 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f979f7fe640 1 -- 172.21.15.62:0/3974292986 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f97b40d1c50 con 0x55a977351880 2024-09-11T23:56:48.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f97c8bcb740 10 client.16290 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:56:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f97c8bcb740 10 client.16290 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:56:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f97c8bcb740 20 client.16290 populate_metadata read hostname 'smithi062' 2024-09-11T23:56:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a9773db990 0x55a9773dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:56:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f97c8bcb740 4 client.16290 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a13053d1", "release": true, "await": true}] 2024-09-11T23:56:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3974292986 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a13053d1", "release": true, "await": true}) -- 0x55a9772be0a0 con 0x55a9773db990 2024-09-11T23:56:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.460+0000 7f97bffff640 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a9773db990 0x55a9773dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:56:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.461+0000 7f97bffff640 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a9773db990 0x55a9773dddb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22bbd919aa6574cc server_cookie=12585a3983b5d597 in_seq=0 out_seq=0 2024-09-11T23:56:48.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.461+0000 7f979f7fe640 10 client.16290 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:48.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f979f7fe640 1 -- 172.21.15.62:0/3974292986 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a9772be0a0 con 0x55a9773db990 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f979f7fe640 10 client.16290 handle_command_reply: tid=0 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f97c8bcb740 1 client.16290 shutdown 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3974292986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 msgr2=0x55a977368b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977368b70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f97b40cce50 tx=0x7f97b409bb70 comp rx=0 tx=0).stop 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3974292986 shutdown_connections 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a9773db990 0x55a9773dddb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a977360650 0x55a977365600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f979f7fe640 0 client.16290 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a977351880 0x55a977368b70 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f97c8bcb740 1 --2- 172.21.15.62:0/3974292986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a977350f60 0x55a977368630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.509+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3974292986 >> 172.21.15.62:0/3974292986 conn(0x55a97734d5f0 msgr2=0x55a977353300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.510+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3974292986 shutdown_connections 2024-09-11T23:56:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.510+0000 7f97c8bcb740 1 -- 172.21.15.62:0/3974292986 wait complete. 2024-09-11T23:56:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.510+0000 7f97c8bcb740 20 client.16290 trim_cache size 0 max 16384 2024-09-11T23:56:48.510 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:56:48.510 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 29, 2024-09-11T23:56:48.510 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 20, 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "a13053d1": { 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 20, 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 140.2, 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:56:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:56:48.512 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:56:48.512 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:56:48.512 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:56:48.512 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:48.512 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:48.512 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:48.512 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:48.512 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:56:48.512 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:56:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.512+0000 7f97c67f2640 1 -- 172.21.15.62:0/837473957 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f97a80a2cf0 msgr2=0x7f97a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.512+0000 7f97c67f2640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f97a80a2cf0 0x7f97a80a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f97b8004000 tx=0x7f97b801c000 comp rx=0 tx=0).stop 2024-09-11T23:56:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.512+0000 7f97c67f2640 1 -- 172.21.15.62:0/837473957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97c0102570 msgr2=0x7f97c0199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:56:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.512+0000 7f97c67f2640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97c0102570 0x7f97c0199790 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f97b400c920 tx=0x7f97b400cde0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.512+0000 7f97c67f2640 1 -- 172.21.15.62:0/837473957 shutdown_connections 2024-09-11T23:56:48.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.512+0000 7f97c67f2640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f97a80a2cf0 0x7f97a80a51b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.513+0000 7f97c67f2640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102ef0 0x7f97c01a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.513+0000 7f97c67f2640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97c0102570 0x7f97c0199790 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.513+0000 7f97c67f2640 1 --2- 172.21.15.62:0/837473957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c0100e20 0x7f97c0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:56:48.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.513+0000 7f97c67f2640 1 -- 172.21.15.62:0/837473957 >> 172.21.15.62:0/837473957 conn(0x7f97c00fc7d0 msgr2=0x7f97c0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:56:48.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.513+0000 7f97c67f2640 1 -- 172.21.15.62:0/837473957 shutdown_connections 2024-09-11T23:56:48.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:56:48.513+0000 7f97c67f2640 1 -- 172.21.15.62:0/837473957 wait complete. 2024-09-11T23:56:49.348 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a13053d1', total seconds elapsed: 24.1 2024-09-11T23:56:49.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:49 smithi120 ceph-mon[35182]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 715 KiB/s wr, 63 op/s 2024-09-11T23:56:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:49 smithi083 ceph-mon[34805]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 715 KiB/s wr, 63 op/s 2024-09-11T23:56:49.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:49 smithi062 ceph-mon[30958]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 715 KiB/s wr, 63 op/s 2024-09-11T23:56:51.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:51 smithi120 ceph-mon[35182]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 702 KiB/s wr, 60 op/s 2024-09-11T23:56:51.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:51 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:56:51.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:51 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:56:51.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:51 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:56:51.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:51 smithi083 ceph-mon[34805]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 702 KiB/s wr, 60 op/s 2024-09-11T23:56:51.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:51 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:56:51.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:51 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:56:51.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:51 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:56:51.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:51 smithi062 ceph-mon[30958]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 702 KiB/s wr, 60 op/s 2024-09-11T23:56:51.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:51 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:56:51.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:51 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:56:51.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:51 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:56:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 250.0 s, 2679.3 tps, lat 180.152 ms stddev 1734.935 2024-09-11T23:56:53.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:53 smithi120 ceph-mon[35182]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1024 KiB/s wr, 103 op/s 2024-09-11T23:56:53.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:53 smithi083 ceph-mon[34805]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1024 KiB/s wr, 103 op/s 2024-09-11T23:56:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:53 smithi062 ceph-mon[30958]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1024 KiB/s wr, 103 op/s 2024-09-11T23:56:55.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:55 smithi120 ceph-mon[35182]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.6 MiB/s wr, 259 op/s 2024-09-11T23:56:55.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:55 smithi083 ceph-mon[34805]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.6 MiB/s wr, 259 op/s 2024-09-11T23:56:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:55 smithi062 ceph-mon[30958]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.6 MiB/s wr, 259 op/s 2024-09-11T23:56:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 255.0 s, 3464.4 tps, lat 28.759 ms stddev 30.476 2024-09-11T23:56:57.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:57 smithi120 ceph-mon[35182]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-09-11T23:56:57.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:57 smithi083 ceph-mon[34805]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-09-11T23:56:57.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:57 smithi062 ceph-mon[30958]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-09-11T23:56:59.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:56:59 smithi120 ceph-mon[35182]: pgmap v149: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-09-11T23:56:59.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:56:59 smithi083 ceph-mon[34805]: pgmap v149: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-09-11T23:56:59.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:56:59 smithi062 ceph-mon[30958]: pgmap v149: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-09-11T23:57:01.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:01 smithi120 ceph-mon[35182]: pgmap v150: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.1 MiB/s wr, 586 op/s 2024-09-11T23:57:01.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:01 smithi083 ceph-mon[34805]: pgmap v150: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.1 MiB/s wr, 586 op/s 2024-09-11T23:57:01.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:01 smithi062 ceph-mon[30958]: pgmap v150: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.1 MiB/s wr, 586 op/s 2024-09-11T23:57:02.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 260.0 s, 2414.8 tps, lat 40.814 ms stddev 86.209 2024-09-11T23:57:03.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:03 smithi120 ceph-mon[35182]: pgmap v151: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-09-11T23:57:03.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:03 smithi083 ceph-mon[34805]: pgmap v151: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-09-11T23:57:03.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:03 smithi062 ceph-mon[30958]: pgmap v151: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-09-11T23:57:05.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:05 smithi120 ceph-mon[35182]: pgmap v152: 97 pgs: 97 active+clean; 2.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-09-11T23:57:05.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:05 smithi083 ceph-mon[34805]: pgmap v152: 97 pgs: 97 active+clean; 2.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-09-11T23:57:05.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:05 smithi062 ceph-mon[30958]: pgmap v152: 97 pgs: 97 active+clean; 2.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-09-11T23:57:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 265.0 s, 3007.7 tps, lat 33.725 ms stddev 40.642 2024-09-11T23:57:07.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:07 smithi120 ceph-mon[35182]: pgmap v153: 97 pgs: 97 active+clean; 2.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-11T23:57:07.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:07 smithi083 ceph-mon[34805]: pgmap v153: 97 pgs: 97 active+clean; 2.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-11T23:57:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:07 smithi062 ceph-mon[30958]: pgmap v153: 97 pgs: 97 active+clean; 2.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 1.18k op/s 2024-09-11T23:57:09.623 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:09 smithi120 ceph-mon[35182]: pgmap v154: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 2.05k op/s 2024-09-11T23:57:09.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:09 smithi083 ceph-mon[34805]: pgmap v154: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 2.05k op/s 2024-09-11T23:57:09.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:09 smithi062 ceph-mon[30958]: pgmap v154: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 2.05k op/s 2024-09-11T23:57:11.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:11 smithi120 ceph-mon[35182]: pgmap v155: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 1.67k op/s 2024-09-11T23:57:11.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:11 smithi083 ceph-mon[34805]: pgmap v155: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 1.67k op/s 2024-09-11T23:57:11.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:11 smithi062 ceph-mon[30958]: pgmap v155: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 1.67k op/s 2024-09-11T23:57:12.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 270.0 s, 3435.0 tps, lat 29.144 ms stddev 31.317 2024-09-11T23:57:13.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:13 smithi062 ceph-mon[30958]: pgmap v156: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2024-09-11T23:57:13.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:13 smithi120 ceph-mon[35182]: pgmap v156: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2024-09-11T23:57:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:13 smithi083 ceph-mon[34805]: pgmap v156: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2024-09-11T23:57:15.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:15 smithi120 ceph-mon[35182]: pgmap v157: 97 pgs: 97 active+clean; 2.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 1.97k op/s 2024-09-11T23:57:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:15 smithi083 ceph-mon[34805]: pgmap v157: 97 pgs: 97 active+clean; 2.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 1.97k op/s 2024-09-11T23:57:15.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:15 smithi062 ceph-mon[30958]: pgmap v157: 97 pgs: 97 active+clean; 2.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 1.97k op/s 2024-09-11T23:57:16.104 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:57:16.483 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:57:17.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.081+0000 7f0777d00640 1 -- 172.21.15.62:0/532704585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0770102ef0 msgr2=0x7f077010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:17.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.081+0000 7f0777d00640 1 --2- 172.21.15.62:0/532704585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0770102ef0 0x7f077010da40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f076c009930 tx=0x7f076c02a4d0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.081+0000 7f0777d00640 1 -- 172.21.15.62:0/532704585 shutdown_connections 2024-09-11T23:57:17.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.081+0000 7f0777d00640 1 --2- 172.21.15.62:0/532704585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0770102ef0 0x7f077010da40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.081+0000 7f0777d00640 1 --2- 172.21.15.62:0/532704585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0770102570 0x7f07701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.081+0000 7f0777d00640 1 --2- 172.21.15.62:0/532704585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0770100e20 0x7f0770101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.081+0000 7f0777d00640 1 -- 172.21.15.62:0/532704585 >> 172.21.15.62:0/532704585 conn(0x7f07700fc7d0 msgr2=0x7f07700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:17.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.081+0000 7f0777d00640 1 -- 172.21.15.62:0/532704585 shutdown_connections 2024-09-11T23:57:17.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.081+0000 7f0777d00640 1 -- 172.21.15.62:0/532704585 wait complete. 2024-09-11T23:57:17.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.082+0000 7f0777d00640 1 Processor -- start 2024-09-11T23:57:17.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.082+0000 7f0777d00640 1 -- start start 2024-09-11T23:57:17.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f0777d00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0770100e20 0x7f0770107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f0777d00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0770102570 0x7f0770107f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f0777d00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0770102ef0 0x7f07701084d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f0777d00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f077010ff00 con 0x7f0770102570 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f0777d00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f077010fd80 con 0x7f0770100e20 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f0777d00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0770110080 con 0x7f0770102ef0 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f077529b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0770102570 0x7f0770107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f077529b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0770102570 0x7f0770107f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59980/0 (socket says 172.21.15.62:59980) 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f077529b640 1 -- 172.21.15.62:0/3167010972 learned_addr learned my addr 172.21.15.62:0/3167010972 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f077629d640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0770102ef0 0x7f07701084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f0775a9c640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0770100e20 0x7f0770107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f077529b640 1 -- 172.21.15.62:0/3167010972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0770102ef0 msgr2=0x7f07701084d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f077529b640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0770102ef0 0x7f07701084d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f077529b640 1 -- 172.21.15.62:0/3167010972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0770100e20 msgr2=0x7f0770107a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f077529b640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0770100e20 0x7f0770107a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f077529b640 1 -- 172.21.15.62:0/3167010972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07701aae50 con 0x7f0770102570 2024-09-11T23:57:17.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.083+0000 7f077529b640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0770102570 0x7f0770107f90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f076000b4d0 tx=0x7f076000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:17.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.084+0000 7f0766ffd640 1 -- 172.21.15.62:0/3167010972 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760014070 con 0x7f0770102570 2024-09-11T23:57:17.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.084+0000 7f0777d00640 1 -- 172.21.15.62:0/3167010972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07701ab140 con 0x7f0770102570 2024-09-11T23:57:17.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.084+0000 7f0766ffd640 1 -- 172.21.15.62:0/3167010972 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0760010d50 con 0x7f0770102570 2024-09-11T23:57:17.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.084+0000 7f0777d00640 1 -- 172.21.15.62:0/3167010972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07701ab700 con 0x7f0770102570 2024-09-11T23:57:17.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.084+0000 7f0766ffd640 1 -- 172.21.15.62:0/3167010972 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076001e060 con 0x7f0770102570 2024-09-11T23:57:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.088+0000 7f0766ffd640 1 -- 172.21.15.62:0/3167010972 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0760017070 con 0x7f0770102570 2024-09-11T23:57:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.088+0000 7f077a0d9740 1 Processor -- start 2024-09-11T23:57:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.088+0000 7f077a0d9740 1 -- start start 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93225f4a0 0x55b93225f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b93226a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93226a680 0x55b93226ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f077a0d9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b9321c52e0 con 0x55b93226a680 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f077a0d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b93216cce0 con 0x55b93225fdc0 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f077a0d9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b9321c12a0 con 0x55b93225f4a0 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f077629d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93225f4a0 0x55b93225f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f0775a9c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b93226a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f0775a9c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b93226a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35906/0 (socket says 172.21.15.62:35906) 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f0775a9c640 1 -- 172.21.15.62:0/2884528486 learned_addr learned my addr 172.21.15.62:0/2884528486 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f077529b640 1 --2- 172.21.15.62:0/2884528486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93226a680 0x55b93226ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f0775a9c640 1 -- 172.21.15.62:0/2884528486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93225f4a0 msgr2=0x55b93225f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:17.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f0775a9c640 1 --2- 172.21.15.62:0/2884528486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93225f4a0 0x55b93225f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f0775a9c640 1 -- 172.21.15.62:0/2884528486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93226a680 msgr2=0x55b93226ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f0775a9c640 1 --2- 172.21.15.62:0/2884528486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93226a680 0x55b93226ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f0775a9c640 1 -- 172.21.15.62:0/2884528486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b932151440 con 0x55b93225fdc0 2024-09-11T23:57:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.089+0000 7f077529b640 1 --2- 172.21.15.62:0/2884528486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93226a680 0x55b93226ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:57:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.090+0000 7f0766ffd640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f075c0a2d70 0x7f075c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.090+0000 7f0766ffd640 1 -- 172.21.15.62:0/3167010972 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f07600cecf0 con 0x7f0770102570 2024-09-11T23:57:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.091+0000 7f077629d640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f075c0a2d70 0x7f075c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.092+0000 7f077629d640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f075c0a2d70 0x7f075c0a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f076c004770 tx=0x7f076c0046c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.092+0000 7f0775a9c640 1 --2- 172.21.15.62:0/2884528486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b93226a140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0768003a50 tx=0x7f076800b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88a73ddd15025099 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.092+0000 7f075bfff640 1 -- 172.21.15.62:0/2884528486 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768004710 con 0x55b93225fdc0 2024-09-11T23:57:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.092+0000 7f075bfff640 1 -- 172.21.15.62:0/2884528486 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0768005020 con 0x55b93225fdc0 2024-09-11T23:57:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.093+0000 7f075bfff640 1 -- 172.21.15.62:0/2884528486 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07680166d0 con 0x55b93225fdc0 2024-09-11T23:57:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.093+0000 7f077a0d9740 1 -- 172.21.15.62:0/2884528486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 msgr2=0x55b93226a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.093+0000 7f077a0d9740 1 --2- 172.21.15.62:0/2884528486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b93226a140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0768003a50 tx=0x7f076800b710 comp rx=0 tx=0).stop 2024-09-11T23:57:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.093+0000 7f077a0d9740 1 -- 172.21.15.62:0/2884528486 shutdown_connections 2024-09-11T23:57:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.093+0000 7f077a0d9740 1 --2- 172.21.15.62:0/2884528486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93226a680 0x55b93226ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.093+0000 7f077a0d9740 1 --2- 172.21.15.62:0/2884528486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b93226a140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.093+0000 7f077a0d9740 1 --2- 172.21.15.62:0/2884528486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93225f4a0 0x55b93225f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.093+0000 7f077a0d9740 1 -- 172.21.15.62:0/2884528486 >> 172.21.15.62:0/2884528486 conn(0x55b93225ba30 msgr2=0x55b93225be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.094+0000 7f077a0d9740 1 -- 172.21.15.62:0/2884528486 shutdown_connections 2024-09-11T23:57:17.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.094+0000 7f077a0d9740 1 -- 172.21.15.62:0/2884528486 wait complete. 2024-09-11T23:57:17.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.094+0000 7f077a0d9740 1 -- start start 2024-09-11T23:57:17.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.094+0000 7f077a0d9740 1 Processor -- start 2024-09-11T23:57:17.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.095+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 0x55b932269ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.095+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932267190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.095+0000 7f0775a9c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 0x55b932269ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.095+0000 7f0775a9c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 0x55b932269ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35910/0 (socket says 172.21.15.62:35910) 2024-09-11T23:57:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.095+0000 7f0775a9c640 1 -- 172.21.15.62:0/1526219977 learned_addr learned my addr 172.21.15.62:0/1526219977 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.095+0000 7f077a0d9740 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b9322676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.095+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b93226d2f0 con 0x55b93225fdc0 2024-09-11T23:57:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.095+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b932180050 con 0x55b93225f4a0 2024-09-11T23:57:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.095+0000 7f077629d640 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b9322676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.096+0000 7f077529b640 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932267190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.096+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b93226d470 con 0x55b93226a680 2024-09-11T23:57:17.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.096+0000 7f0775a9c640 1 -- 172.21.15.62:0/1526219977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 msgr2=0x55b9322676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.096+0000 7f0775a9c640 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b9322676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.096+0000 7f0775a9c640 1 -- 172.21.15.62:0/1526219977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 msgr2=0x55b932267190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.096+0000 7f0775a9c640 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932267190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.096+0000 7f0775a9c640 1 -- 172.21.15.62:0/1526219977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b931f363c0 con 0x55b93225f4a0 2024-09-11T23:57:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.097+0000 7f077529b640 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932267190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:57:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.097+0000 7f0775a9c640 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 0x55b932269ee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0768006010 tx=0x7f0768005400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b3acaf954f1ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.097+0000 7f07597fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:57:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.097+0000 7f07597fa640 1 -- 172.21.15.62:0/1526219977 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768004710 con 0x55b93225f4a0 2024-09-11T23:57:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.097+0000 7f07597fa640 1 -- 172.21.15.62:0/1526219977 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f0768016470 con 0x55b93225f4a0 2024-09-11T23:57:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.097+0000 7f077a0d9740 10 client.0 _ll_register_callbacks cb 0x55b9321c0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:17.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.097+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0768005020 con 0x55b93225f4a0 2024-09-11T23:57:17.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.097+0000 7f07597fa640 1 -- 172.21.15.62:0/1526219977 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f076800bd30 con 0x55b93225f4a0 2024-09-11T23:57:17.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.097+0000 7f077a0d9740 10 client.25919 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:17.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.098+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55b932278860 con 0x55b93225f4a0 2024-09-11T23:57:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.100+0000 7f07597fa640 1 -- 172.21.15.62:0/1526219977 <== mon.2 v2:172.21.15.83:3300/0 4 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f0768016670 con 0x55b93225f4a0 2024-09-11T23:57:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.100+0000 7f077a0d9740 10 client.25919 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.100+0000 7f077a0d9740 10 client.25919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:57:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.100+0000 7f077a0d9740 20 client.25919 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.100+0000 7f077a0d9740 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b932279c80 0x55b93227c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.100+0000 7f077a0d9740 4 client.25919 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:57:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.100+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9321d2850 con 0x55b932279c80 2024-09-11T23:57:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.100+0000 7f077529b640 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b932279c80 0x55b93227c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.101+0000 7f07597fa640 10 client.25919 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:17.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.101+0000 7f077529b640 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b932279c80 0x55b93227c070 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f52d59b716f5f4 server_cookie=76d609e8e0f2da7e in_seq=0 out_seq=0 2024-09-11T23:57:17.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.101+0000 7f07597fa640 1 -- 172.21.15.62:0/1526219977 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9321d2850 con 0x55b932279c80 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.101+0000 7f07597fa640 10 client.25919 handle_command_reply: tid=0 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.101+0000 7f077a0d9740 1 client.25919 shutdown 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 msgr2=0x55b932269ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 0x55b932269ee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0768006010 tx=0x7f0768005400 comp rx=0 tx=0).stop 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 shutdown_connections 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b932279c80 0x55b93227c070 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b9322676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932267190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 1 --2- 172.21.15.62:0/1526219977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 0x55b932269ee0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 >> 172.21.15.62:0/1526219977 conn(0x55b93225ba30 msgr2=0x55b932261710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f07597fa640 0 client.25919 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 shutdown_connections 2024-09-11T23:57:17.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 1 -- 172.21.15.62:0/1526219977 wait complete. 2024-09-11T23:57:17.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.102+0000 7f077a0d9740 20 client.25919 trim_cache size 0 max 16384 2024-09-11T23:57:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f077a0d9740 1 Processor -- start 2024-09-11T23:57:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f077a0d9740 1 -- start start 2024-09-11T23:57:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225f4a0 0x55b932281700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b932281c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b932264410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f0775a9c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225f4a0 0x55b932281700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f077a0d9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b6800 con 0x55b93225f4a0 2024-09-11T23:57:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f077a0d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b40d0 con 0x55b93225fdc0 2024-09-11T23:57:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f077a0d9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b1f20 con 0x55b93226a680 2024-09-11T23:57:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f0775a9c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225f4a0 0x55b932281700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60020/0 (socket says 172.21.15.62:60020) 2024-09-11T23:57:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.128+0000 7f0775a9c640 1 -- 172.21.15.62:0/2040905231 learned_addr learned my addr 172.21.15.62:0/2040905231 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.129+0000 7f077629d640 1 --2- 172.21.15.62:0/2040905231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b932264410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.129+0000 7f077529b640 1 --2- 172.21.15.62:0/2040905231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b932281c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.129+0000 7f0775a9c640 1 -- 172.21.15.62:0/2040905231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 msgr2=0x55b932264410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.129+0000 7f0775a9c640 1 --2- 172.21.15.62:0/2040905231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b932264410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.129+0000 7f0775a9c640 1 -- 172.21.15.62:0/2040905231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 msgr2=0x55b932281c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.129+0000 7f0775a9c640 1 --2- 172.21.15.62:0/2040905231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b932281c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.129+0000 7f0775a9c640 1 -- 172.21.15.62:0/2040905231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93225fbf0 con 0x55b93225f4a0 2024-09-11T23:57:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.129+0000 7f077529b640 1 --2- 172.21.15.62:0/2040905231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b932281c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:57:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.129+0000 7f0775a9c640 1 --2- 172.21.15.62:0/2040905231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225f4a0 0x55b932281700 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0768004eb0 tx=0x7f076801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5686014d96f5a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.130+0000 7f075b7fe640 1 -- 172.21.15.62:0/2040905231 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768016c20 con 0x55b93225f4a0 2024-09-11T23:57:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.130+0000 7f075b7fe640 1 -- 172.21.15.62:0/2040905231 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0768005020 con 0x55b93225f4a0 2024-09-11T23:57:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.130+0000 7f075b7fe640 1 -- 172.21.15.62:0/2040905231 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076803b450 con 0x55b93225f4a0 2024-09-11T23:57:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.131+0000 7f077a0d9740 1 -- 172.21.15.62:0/2040905231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225f4a0 msgr2=0x55b932281700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.131+0000 7f077a0d9740 1 --2- 172.21.15.62:0/2040905231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225f4a0 0x55b932281700 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0768004eb0 tx=0x7f076801e000 comp rx=0 tx=0).stop 2024-09-11T23:57:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.131+0000 7f077a0d9740 1 -- 172.21.15.62:0/2040905231 shutdown_connections 2024-09-11T23:57:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.131+0000 7f077a0d9740 1 --2- 172.21.15.62:0/2040905231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b932264410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.131+0000 7f077a0d9740 1 --2- 172.21.15.62:0/2040905231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225fdc0 0x55b932281c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.131+0000 7f077a0d9740 1 --2- 172.21.15.62:0/2040905231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225f4a0 0x55b932281700 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.131+0000 7f077a0d9740 1 -- 172.21.15.62:0/2040905231 >> 172.21.15.62:0/2040905231 conn(0x55b93225ba30 msgr2=0x55b932261710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.133+0000 7f077a0d9740 1 -- 172.21.15.62:0/2040905231 shutdown_connections 2024-09-11T23:57:17.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.134+0000 7f077a0d9740 1 -- 172.21.15.62:0/2040905231 wait complete. 2024-09-11T23:57:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.134+0000 7f077a0d9740 1 -- start start 2024-09-11T23:57:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.134+0000 7f077a0d9740 1 Processor -- start 2024-09-11T23:57:17.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 0x55b932272e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932263ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f077a0d9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b932264400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f077a0d9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b2590 con 0x55b93225fdc0 2024-09-11T23:57:17.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f077a0d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b3a00 con 0x55b93225f4a0 2024-09-11T23:57:17.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f077a0d9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b2c60 con 0x55b93226a680 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f077a0d9740 10 client.0 _ll_register_callbacks cb 0x7f076c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f077529b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932263ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f077529b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932263ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60022/0 (socket says 172.21.15.62:60022) 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f077529b640 1 -- 172.21.15.62:0/444846587 learned_addr learned my addr 172.21.15.62:0/444846587 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.135+0000 7f0775a9c640 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 0x55b932272e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f077629d640 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b932264400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f077529b640 1 -- 172.21.15.62:0/444846587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 msgr2=0x55b932264400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f077529b640 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b932264400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f077529b640 1 -- 172.21.15.62:0/444846587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 msgr2=0x55b932272e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f077529b640 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 0x55b932272e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f077529b640 1 -- 172.21.15.62:0/444846587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b932264c30 con 0x55b93225fdc0 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f077529b640 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932263ec0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f076000f910 tx=0x7f07600047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ecae7055b24cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:17.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f07597fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:57:17.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f07597fa640 1 -- 172.21.15.62:0/444846587 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07600a3f60 con 0x55b93225fdc0 2024-09-11T23:57:17.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f07597fa640 1 -- 172.21.15.62:0/444846587 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f07600b4290 con 0x55b93225fdc0 2024-09-11T23:57:17.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.136+0000 7f077a0d9740 1 -- 172.21.15.62:0/444846587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0768005020 con 0x55b93225fdc0 2024-09-11T23:57:17.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.137+0000 7f07597fa640 1 -- 172.21.15.62:0/444846587 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07600cc690 con 0x55b93225fdc0 2024-09-11T23:57:17.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.137+0000 7f077a0d9740 10 client.16299 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:17.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.137+0000 7f077a0d9740 1 -- 172.21.15.62:0/444846587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9322f5720 con 0x55b93225fdc0 2024-09-11T23:57:17.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.137+0000 7f07597fa640 1 -- 172.21.15.62:0/444846587 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0760097a10 con 0x55b93225fdc0 2024-09-11T23:57:17.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.139+0000 7f07597fa640 1 -- 172.21.15.62:0/444846587 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f07600ccb60 con 0x55b93225fdc0 2024-09-11T23:57:17.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.139+0000 7f077a0d9740 10 client.16299 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:17.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.139+0000 7f077a0d9740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:57:17.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.139+0000 7f077a0d9740 20 client.16299 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:17.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.139+0000 7f077a0d9740 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9322f5df0 0x55b9322f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:17.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.139+0000 7f077a0d9740 4 client.16299 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:57:17.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.139+0000 7f077a0d9740 1 -- 172.21.15.62:0/444846587 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0768005020 con 0x55b9322f5df0 2024-09-11T23:57:17.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.139+0000 7f0775a9c640 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9322f5df0 0x55b9322f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:17.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.142+0000 7f07597fa640 10 client.16299 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:17.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.142+0000 7f0775a9c640 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9322f5df0 0x55b9322f81e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218eca2fc4bd65f3 server_cookie=cb3c30858c3db80c in_seq=0 out_seq=0 2024-09-11T23:57:17.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.142+0000 7f07597fa640 1 -- 172.21.15.62:0/444846587 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f0768005020 con 0x55b9322f5df0 2024-09-11T23:57:17.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.142+0000 7f07597fa640 10 client.16299 handle_command_reply: tid=0 2024-09-11T23:57:17.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.142+0000 7f077a0d9740 1 client.16299 shutdown 2024-09-11T23:57:17.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.142+0000 7f077a0d9740 1 -- 172.21.15.62:0/444846587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 msgr2=0x55b932263ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:17.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.142+0000 7f077a0d9740 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932263ec0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f076000f910 tx=0x7f07600047d0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f077a0d9740 1 -- 172.21.15.62:0/444846587 shutdown_connections 2024-09-11T23:57:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f077a0d9740 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9322f5df0 0x55b9322f81e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f077a0d9740 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b93226a680 0x55b932264400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f077a0d9740 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b93225fdc0 0x55b932263ec0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f077a0d9740 1 --2- 172.21.15.62:0/444846587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b93225f4a0 0x55b932272e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f07597fa640 0 client.16299 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f077a0d9740 1 -- 172.21.15.62:0/444846587 >> 172.21.15.62:0/444846587 conn(0x55b93225ba30 msgr2=0x55b93225a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f077a0d9740 1 -- 172.21.15.62:0/444846587 shutdown_connections 2024-09-11T23:57:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f077a0d9740 1 -- 172.21.15.62:0/444846587 wait complete. 2024-09-11T23:57:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f077a0d9740 20 client.16299 trim_cache size 0 max 16384 2024-09-11T23:57:17.145 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:57:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-11T23:57:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-11T23:57:17.145 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:57:17.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f0777d00640 1 -- 172.21.15.62:0/3167010972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f075c0a2d70 msgr2=0x7f075c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:17.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f0777d00640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f075c0a2d70 0x7f075c0a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f076c004770 tx=0x7f076c0046c0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.143+0000 7f0777d00640 1 -- 172.21.15.62:0/3167010972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0770102570 msgr2=0x7f0770107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:17.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.144+0000 7f0777d00640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0770102570 0x7f0770107f90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f076000b4d0 tx=0x7f076000b990 comp rx=0 tx=0).stop 2024-09-11T23:57:17.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.144+0000 7f0777d00640 1 -- 172.21.15.62:0/3167010972 shutdown_connections 2024-09-11T23:57:17.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.144+0000 7f0777d00640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f075c0a2d70 0x7f075c0a5230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.144+0000 7f0777d00640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0770102ef0 0x7f07701084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.144+0000 7f0777d00640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0770102570 0x7f0770107f90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.144+0000 7f0777d00640 1 --2- 172.21.15.62:0/3167010972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0770100e20 0x7f0770107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:17.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.144+0000 7f0777d00640 1 -- 172.21.15.62:0/3167010972 >> 172.21.15.62:0/3167010972 conn(0x7f07700fc7d0 msgr2=0x7f07700fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:17.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.144+0000 7f0777d00640 1 -- 172.21.15.62:0/3167010972 shutdown_connections 2024-09-11T23:57:17.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:17.144+0000 7f0777d00640 1 -- 172.21.15.62:0/3167010972 wait complete. 2024-09-11T23:57:17.446 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 275.0 s, 3002.8 tps, lat 33.228 ms stddev 39.802 2024-09-11T23:57:17.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:17 smithi120 ceph-mon[35182]: pgmap v158: 97 pgs: 97 active+clean; 2.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-09-11T23:57:17.648 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:17 smithi062 ceph-mon[30958]: pgmap v158: 97 pgs: 97 active+clean; 2.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-09-11T23:57:17.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:17 smithi083 ceph-mon[34805]: pgmap v158: 97 pgs: 97 active+clean; 2.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-09-11T23:57:17.980 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:57:17.980 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:57:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:57:18.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:18 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:57:18.583 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:18 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:57:18.583 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:18 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:57:18.583 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:18 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:57:18.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:18 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:57:18.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:18 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:57:18.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:18 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:57:18.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:18 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:57:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:18 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:57:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:18 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:57:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:18 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:57:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:18 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:57:18.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.981+0000 7f8c49538640 1 -- 172.21.15.62:0/3499528807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c44109120 msgr2=0x7f8c44110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:18.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.981+0000 7f8c49538640 1 --2- 172.21.15.62:0/3499528807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c44109120 0x7f8c44110ca0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009be0 tx=0x7f8c3c02a6c0 comp rx=0 tx=0).stop 2024-09-11T23:57:18.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.981+0000 7f8c49538640 1 -- 172.21.15.62:0/3499528807 shutdown_connections 2024-09-11T23:57:18.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.981+0000 7f8c49538640 1 --2- 172.21.15.62:0/3499528807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c44109120 0x7f8c44110ca0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:18.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.981+0000 7f8c49538640 1 --2- 172.21.15.62:0/3499528807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c441049a0 0x7f8c441089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:18.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.981+0000 7f8c49538640 1 --2- 172.21.15.62:0/3499528807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c44103ff0 0x7f8c441043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:18.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.981+0000 7f8c49538640 1 -- 172.21.15.62:0/3499528807 >> 172.21.15.62:0/3499528807 conn(0x7f8c440ffab0 msgr2=0x7f8c44101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:18.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.981+0000 7f8c49538640 1 -- 172.21.15.62:0/3499528807 shutdown_connections 2024-09-11T23:57:18.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.981+0000 7f8c49538640 1 -- 172.21.15.62:0/3499528807 wait complete. 2024-09-11T23:57:18.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.982+0000 7f8c49538640 1 Processor -- start 2024-09-11T23:57:18.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.983+0000 7f8c49538640 1 -- start start 2024-09-11T23:57:18.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.983+0000 7f8c49538640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c44103ff0 0x7f8c44076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:18.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.983+0000 7f8c49538640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c441049a0 0x7f8c44076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:18.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.983+0000 7f8c49538640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c44109120 0x7f8c44077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:18.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.984+0000 7f8c49538640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8c441140e0 con 0x7f8c441049a0 2024-09-11T23:57:18.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.984+0000 7f8c49538640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c44113f60 con 0x7f8c44109120 2024-09-11T23:57:18.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.984+0000 7f8c49538640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c44114260 con 0x7f8c44103ff0 2024-09-11T23:57:18.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.984+0000 7f8c437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c44109120 0x7f8c44077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:18.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.984+0000 7f8c42ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c44103ff0 0x7f8c44076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:18.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.984+0000 7f8c437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c44109120 0x7f8c44077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35950/0 (socket says 172.21.15.62:35950) 2024-09-11T23:57:18.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.984+0000 7f8c437fe640 1 -- 172.21.15.62:0/2599321341 learned_addr learned my addr 172.21.15.62:0/2599321341 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.984+0000 7f8c427fc640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c441049a0 0x7f8c44076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c437fe640 1 -- 172.21.15.62:0/2599321341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c44103ff0 msgr2=0x7f8c44076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c437fe640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c44103ff0 0x7f8c44076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c437fe640 1 -- 172.21.15.62:0/2599321341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c441049a0 msgr2=0x7f8c44076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c437fe640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c441049a0 0x7f8c44076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c437fe640 1 -- 172.21.15.62:0/2599321341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c441aaca0 con 0x7f8c44109120 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c42ffd640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c44103ff0 0x7f8c44076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c437fe640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c44109120 0x7f8c44077180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0305b0 tx=0x7f8c3c0305e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c2bfff640 1 -- 172.21.15.62:0/2599321341 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c040020 con 0x7f8c44109120 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c2bfff640 1 -- 172.21.15.62:0/2599321341 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3c037040 con 0x7f8c44109120 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c49538640 1 -- 172.21.15.62:0/2599321341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c441aaf30 con 0x7f8c44109120 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c2bfff640 1 -- 172.21.15.62:0/2599321341 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c036030 con 0x7f8c44109120 2024-09-11T23:57:18.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.985+0000 7f8c49538640 1 -- 172.21.15.62:0/2599321341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c441ab3e0 con 0x7f8c44109120 2024-09-11T23:57:18.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.986+0000 7f8c49538640 1 -- 172.21.15.62:0/2599321341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c441054d0 con 0x7f8c44109120 2024-09-11T23:57:18.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.987+0000 7f8c2bfff640 1 -- 172.21.15.62:0/2599321341 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8c3c043020 con 0x7f8c44109120 2024-09-11T23:57:18.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.988+0000 7f8c2bfff640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8c2c0a2cf0 0x7f8c2c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:18.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.989+0000 7f8c2bfff640 1 -- 172.21.15.62:0/2599321341 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f8c3c0efbd0 con 0x7f8c44109120 2024-09-11T23:57:18.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.989+0000 7f8c42ffd640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8c2c0a2cf0 0x7f8c2c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:18.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.990+0000 7f8c42ffd640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8c2c0a2cf0 0x7f8c2c0a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8c340096c0 tx=0x7f8c34009260 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:18.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:18.992+0000 7f8c2bfff640 1 -- 172.21.15.62:0/2599321341 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c3c04a020 con 0x7f8c44109120 2024-09-11T23:57:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.323+0000 7f8c49538640 1 -- 172.21.15.62:0/2599321341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c44077c30 con 0x7f8c44109120 2024-09-11T23:57:19.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.324+0000 7f8c2bfff640 1 -- 172.21.15.62:0/2599321341 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7f8c3c0bc820 con 0x7f8c44109120 2024-09-11T23:57:19.325 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:57:19.325 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"btime":"2024-09-11T23:55:51:325581+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:55:50.698407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-11T23:57:19.325 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-09-11T23:57:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 -- 172.21.15.62:0/2599321341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8c2c0a2cf0 msgr2=0x7f8c2c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8c2c0a2cf0 0x7f8c2c0a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8c340096c0 tx=0x7f8c34009260 comp rx=0 tx=0).stop 2024-09-11T23:57:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 -- 172.21.15.62:0/2599321341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c44109120 msgr2=0x7f8c44077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c44109120 0x7f8c44077180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0305b0 tx=0x7f8c3c0305e0 comp rx=0 tx=0).stop 2024-09-11T23:57:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 -- 172.21.15.62:0/2599321341 shutdown_connections 2024-09-11T23:57:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8c2c0a2cf0 0x7f8c2c0a51b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c44109120 0x7f8c44077180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c441049a0 0x7f8c44076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 --2- 172.21.15.62:0/2599321341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c44103ff0 0x7f8c44076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 -- 172.21.15.62:0/2599321341 >> 172.21.15.62:0/2599321341 conn(0x7f8c440ffab0 msgr2=0x7f8c44100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 -- 172.21.15.62:0/2599321341 shutdown_connections 2024-09-11T23:57:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:19.328+0000 7f8c49538640 1 -- 172.21.15.62:0/2599321341 wait complete. 2024-09-11T23:57:19.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:19 smithi120 ceph-mon[35182]: pgmap v159: 97 pgs: 97 active+clean; 2.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 1.53k op/s 2024-09-11T23:57:19.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:19 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2599321341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:57:19.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:19 smithi083 ceph-mon[34805]: pgmap v159: 97 pgs: 97 active+clean; 2.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 1.53k op/s 2024-09-11T23:57:19.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:19 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2599321341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:57:19.712 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:19 smithi062 ceph-mon[30958]: pgmap v159: 97 pgs: 97 active+clean; 2.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 1.53k op/s 2024-09-11T23:57:19.712 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:19 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2599321341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:57:20.113 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:57:20.489 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:57:21.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.009+0000 7f10c0877640 1 -- 172.21.15.62:0/1355680193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10bc102ef0 msgr2=0x7f10bc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:21.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.009+0000 7f10c0877640 1 --2- 172.21.15.62:0/1355680193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10bc102ef0 0x7f10bc10da40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009ca0 tx=0x7f10b402a950 comp rx=0 tx=0).stop 2024-09-11T23:57:21.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.010+0000 7f10c0877640 1 -- 172.21.15.62:0/1355680193 shutdown_connections 2024-09-11T23:57:21.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.010+0000 7f10c0877640 1 --2- 172.21.15.62:0/1355680193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10bc102ef0 0x7f10bc10da40 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.010+0000 7f10c0877640 1 --2- 172.21.15.62:0/1355680193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc102570 0x7f10bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.010+0000 7f10c0877640 1 --2- 172.21.15.62:0/1355680193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10bc100e20 0x7f10bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.010+0000 7f10c0877640 1 -- 172.21.15.62:0/1355680193 >> 172.21.15.62:0/1355680193 conn(0x7f10bc0fc7d0 msgr2=0x7f10bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:21.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.011+0000 7f10c0877640 1 -- 172.21.15.62:0/1355680193 shutdown_connections 2024-09-11T23:57:21.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.011+0000 7f10c0877640 1 -- 172.21.15.62:0/1355680193 wait complete. 2024-09-11T23:57:21.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.011+0000 7f10c0877640 1 Processor -- start 2024-09-11T23:57:21.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10c0877640 1 -- start start 2024-09-11T23:57:21.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10c0877640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10bc100e20 0x7f10bc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10c0877640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10bc102570 0x7f10bc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10c0877640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc102ef0 0x7f10bc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10c0877640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc110f80 con 0x7f10bc102570 2024-09-11T23:57:21.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10c0877640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc110e00 con 0x7f10bc100e20 2024-09-11T23:57:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10c0877640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc111100 con 0x7f10bc102ef0 2024-09-11T23:57:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10bc102570 0x7f10bc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10bc102570 0x7f10bc1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60046/0 (socket says 172.21.15.62:60046) 2024-09-11T23:57:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10b9d9b640 1 -- 172.21.15.62:0/3154946851 learned_addr learned my addr 172.21.15.62:0/3154946851 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10bad9d640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc102ef0 0x7f10bc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10ba59c640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10bc100e20 0x7f10bc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10bad9d640 1 -- 172.21.15.62:0/3154946851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10bc100e20 msgr2=0x7f10bc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10bad9d640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10bc100e20 0x7f10bc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10bad9d640 1 -- 172.21.15.62:0/3154946851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10bc102570 msgr2=0x7f10bc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10bad9d640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10bc102570 0x7f10bc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10bad9d640 1 -- 172.21.15.62:0/3154946851 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc19d440 con 0x7f10bc102ef0 2024-09-11T23:57:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.012+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10bc102570 0x7f10bc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:57:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.013+0000 7f10bad9d640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc102ef0 0x7f10bc19cd60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f10b4005cc0 tx=0x7f10b402f420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.013+0000 7f10ab7fe640 1 -- 172.21.15.62:0/3154946851 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4034040 con 0x7f10bc102ef0 2024-09-11T23:57:21.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.013+0000 7f10ab7fe640 1 -- 172.21.15.62:0/3154946851 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b4030920 con 0x7f10bc102ef0 2024-09-11T23:57:21.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.013+0000 7f10c0877640 1 -- 172.21.15.62:0/3154946851 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc19d6d0 con 0x7f10bc102ef0 2024-09-11T23:57:21.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.013+0000 7f10ab7fe640 1 -- 172.21.15.62:0/3154946851 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4004600 con 0x7f10bc102ef0 2024-09-11T23:57:21.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.013+0000 7f10c0877640 1 -- 172.21.15.62:0/3154946851 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc19daf0 con 0x7f10bc102ef0 2024-09-11T23:57:21.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.017+0000 7f10ab7fe640 1 -- 172.21.15.62:0/3154946851 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f10b4037020 con 0x7f10bc102ef0 2024-09-11T23:57:21.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.018+0000 7f10c2c50740 1 Processor -- start 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10ab7fe640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10a40a2cf0 0x7f10a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10c2c50740 1 -- start start 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10a40a2cf0 0x7f10a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10ab7fe640 1 -- 172.21.15.62:0/3154946851 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f10b40eff00 con 0x7f10bc102ef0 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105dab40 0x5614105d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 0x5614105d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 0x5614105e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10c2c50740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56141053f2f0 con 0x5614105d95d0 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10c2c50740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5614104e6cf0 con 0x5614105dab40 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10c2c50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56141053b2b0 con 0x5614105e4370 2024-09-11T23:57:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 0x5614105e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 0x5614105e6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:49496/0 (socket says 172.21.15.62:49496) 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10bad9d640 1 -- 172.21.15.62:0/59794586 learned_addr learned my addr 172.21.15.62:0/59794586 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10ba59c640 1 --2- 172.21.15.62:0/59794586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105dab40 0x5614105d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/59794586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 0x5614105d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10bad9d640 1 -- 172.21.15.62:0/59794586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105dab40 msgr2=0x5614105d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10bad9d640 1 --2- 172.21.15.62:0/59794586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105dab40 0x5614105d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10bad9d640 1 -- 172.21.15.62:0/59794586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 msgr2=0x5614105d9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10bad9d640 1 --2- 172.21.15.62:0/59794586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 0x5614105d9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.020+0000 7f10bad9d640 1 -- 172.21.15.62:0/59794586 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614104cb3a0 con 0x5614105e4370 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.019+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10a40a2cf0 0x7f10a40a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0fec10 tx=0x7f10b0009ce0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:21.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.020+0000 7f10bad9d640 1 --2- 172.21.15.62:0/59794586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 0x5614105e6760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10b40bd590 tx=0x7f10b40bd250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f33c8e4e4c5719d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.021+0000 7f10a8ff9640 1 -- 172.21.15.62:0/59794586 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b40b9040 con 0x5614105e4370 2024-09-11T23:57:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.021+0000 7f10a8ff9640 1 -- 172.21.15.62:0/59794586 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b40c19b0 con 0x5614105e4370 2024-09-11T23:57:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.021+0000 7f10a8ff9640 1 -- 172.21.15.62:0/59794586 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b40bdac0 con 0x5614105e4370 2024-09-11T23:57:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 -- 172.21.15.62:0/59794586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 msgr2=0x5614105e6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 --2- 172.21.15.62:0/59794586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 0x5614105e6760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10b40bd590 tx=0x7f10b40bd250 comp rx=0 tx=0).stop 2024-09-11T23:57:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 -- 172.21.15.62:0/59794586 shutdown_connections 2024-09-11T23:57:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 --2- 172.21.15.62:0/59794586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 0x5614105e6760 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 --2- 172.21.15.62:0/59794586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 0x5614105d9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 --2- 172.21.15.62:0/59794586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105dab40 0x5614105d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 -- 172.21.15.62:0/59794586 >> 172.21.15.62:0/59794586 conn(0x5614105a2310 msgr2=0x5614105d5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 -- 172.21.15.62:0/59794586 shutdown_connections 2024-09-11T23:57:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 -- 172.21.15.62:0/59794586 wait complete. 2024-09-11T23:57:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 -- start start 2024-09-11T23:57:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.022+0000 7f10c2c50740 1 Processor -- start 2024-09-11T23:57:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105d95d0 0x5614105e3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105dab40 0x5614105e0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 0x5614105e1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10c2c50740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56141053b2b0 con 0x5614105dab40 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10c2c50740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5614104fa060 con 0x5614105d95d0 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10c2c50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5614105eb240 con 0x5614105e4370 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105dab40 0x5614105e0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105dab40 0x5614105e0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60064/0 (socket says 172.21.15.62:60064) 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10b9d9b640 1 -- 172.21.15.62:0/3526215421 learned_addr learned my addr 172.21.15.62:0/3526215421 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10c2c50740 10 client.0 _ll_register_callbacks cb 0x56141024d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10bad9d640 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 0x5614105e1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10ba59c640 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105d95d0 0x5614105e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10b9d9b640 1 -- 172.21.15.62:0/3526215421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 msgr2=0x5614105e1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 0x5614105e1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10b9d9b640 1 -- 172.21.15.62:0/3526215421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105d95d0 msgr2=0x5614105e3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105d95d0 0x5614105e3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10b9d9b640 1 -- 172.21.15.62:0/3526215421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614102b03c0 con 0x5614105dab40 2024-09-11T23:57:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105dab40 0x5614105e0f00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f10b000f820 tx=0x7f10b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1fd0ba09033aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.023+0000 7f10a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:57:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.024+0000 7f10a1ffb640 1 -- 172.21.15.62:0/3526215421 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b000fe60 con 0x5614105dab40 2024-09-11T23:57:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.024+0000 7f10a1ffb640 1 -- 172.21.15.62:0/3526215421 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f10b0010b40 con 0x5614105dab40 2024-09-11T23:57:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.024+0000 7f10c2c50740 1 -- 172.21.15.62:0/3526215421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b40c19b0 con 0x5614105dab40 2024-09-11T23:57:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.024+0000 7f10a1ffb640 1 -- 172.21.15.62:0/3526215421 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10b0010d80 con 0x5614105dab40 2024-09-11T23:57:21.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.024+0000 7f10c2c50740 10 client.16317 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:21.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.024+0000 7f10c2c50740 1 -- 172.21.15.62:0/3526215421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5614105f6a80 con 0x5614105dab40 2024-09-11T23:57:21.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.024+0000 7f10a1ffb640 1 -- 172.21.15.62:0/3526215421 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b40c19b0 con 0x5614105dab40 2024-09-11T23:57:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.025+0000 7f10a1ffb640 1 -- 172.21.15.62:0/3526215421 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f10b0025070 con 0x5614105dab40 2024-09-11T23:57:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.025+0000 7f10c2c50740 10 client.16317 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.025+0000 7f10c2c50740 10 client.16317 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:57:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.025+0000 7f10c2c50740 20 client.16317 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.025+0000 7f10c2c50740 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5614105f8000 0x5614105fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.025+0000 7f10c2c50740 4 client.16317 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:57:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.025+0000 7f10c2c50740 1 -- 172.21.15.62:0/3526215421 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56141054c860 con 0x5614105f8000 2024-09-11T23:57:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.025+0000 7f10ba59c640 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5614105f8000 0x5614105fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.026+0000 7f10ba59c640 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5614105f8000 0x5614105fa480 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60c33a68df540734 server_cookie=79bfccbcd737aac3 in_seq=0 out_seq=0 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.026+0000 7f10a1ffb640 10 client.16317 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10a1ffb640 1 -- 172.21.15.62:0/3526215421 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56141054c860 con 0x5614105f8000 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10a1ffb640 10 client.16317 handle_command_reply: tid=0 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 client.16317 shutdown 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 -- 172.21.15.62:0/3526215421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105dab40 msgr2=0x5614105e0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105dab40 0x5614105e0f00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f10b000f820 tx=0x7f10b0004730 comp rx=0 tx=0).stop 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 -- 172.21.15.62:0/3526215421 shutdown_connections 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5614105f8000 0x5614105fa480 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10a1ffb640 0 client.16317 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105e4370 0x5614105e1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105dab40 0x5614105e0f00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 --2- 172.21.15.62:0/3526215421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105d95d0 0x5614105e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 -- 172.21.15.62:0/3526215421 >> 172.21.15.62:0/3526215421 conn(0x5614105a2310 msgr2=0x5614105db930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:21.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 -- 172.21.15.62:0/3526215421 shutdown_connections 2024-09-11T23:57:21.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.027+0000 7f10c2c50740 1 -- 172.21.15.62:0/3526215421 wait complete. 2024-09-11T23:57:21.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.028+0000 7f10c2c50740 20 client.16317 trim_cache size 0 max 16384 2024-09-11T23:57:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10c2c50740 1 Processor -- start 2024-09-11T23:57:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10c2c50740 1 -- start start 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105a2310 0x5614105f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105d95d0 0x5614105dff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105dab40 0x5614105ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10c2c50740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f109c010f90 con 0x5614105a2310 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10c2c50740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f109c00e860 con 0x5614105dab40 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10c2c50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f109c00c6b0 con 0x5614105d95d0 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105d95d0 0x5614105dff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10ba59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105a2310 0x5614105f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10ba59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105a2310 0x5614105f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60076/0 (socket says 172.21.15.62:60076) 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105d95d0 0x5614105dff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:49514/0 (socket says 172.21.15.62:49514) 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10ba59c640 1 -- 172.21.15.62:0/3871030597 learned_addr learned my addr 172.21.15.62:0/3871030597 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.046+0000 7f10bad9d640 1 --2- 172.21.15.62:0/3871030597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105dab40 0x5614105ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.047+0000 7f10b9d9b640 1 -- 172.21.15.62:0/3871030597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105dab40 msgr2=0x5614105ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.047+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/3871030597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105dab40 0x5614105ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.047+0000 7f10b9d9b640 1 -- 172.21.15.62:0/3871030597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105a2310 msgr2=0x5614105f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.047+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/3871030597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105a2310 0x5614105f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.047+0000 7f10b9d9b640 1 -- 172.21.15.62:0/3871030597 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614105faa60 con 0x5614105d95d0 2024-09-11T23:57:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.047+0000 7f10ba59c640 1 --2- 172.21.15.62:0/3871030597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105a2310 0x5614105f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:57:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.047+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/3871030597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105d95d0 0x5614105dff00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f10b0013e70 tx=0x7f10b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab358351dfd5a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.047+0000 7f10a3fff640 1 -- 172.21.15.62:0/3871030597 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b000fe60 con 0x5614105d95d0 2024-09-11T23:57:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.047+0000 7f10a3fff640 1 -- 172.21.15.62:0/3871030597 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b0010b40 con 0x5614105d95d0 2024-09-11T23:57:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.047+0000 7f10a3fff640 1 -- 172.21.15.62:0/3871030597 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b002f430 con 0x5614105d95d0 2024-09-11T23:57:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.048+0000 7f10c2c50740 1 -- 172.21.15.62:0/3871030597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105d95d0 msgr2=0x5614105dff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.048+0000 7f10c2c50740 1 --2- 172.21.15.62:0/3871030597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105d95d0 0x5614105dff00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f10b0013e70 tx=0x7f10b00046c0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.048+0000 7f10c2c50740 1 -- 172.21.15.62:0/3871030597 shutdown_connections 2024-09-11T23:57:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.048+0000 7f10c2c50740 1 --2- 172.21.15.62:0/3871030597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105dab40 0x5614105ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.048+0000 7f10c2c50740 1 --2- 172.21.15.62:0/3871030597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105d95d0 0x5614105dff00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.048+0000 7f10c2c50740 1 --2- 172.21.15.62:0/3871030597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105a2310 0x5614105f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.048+0000 7f10c2c50740 1 -- 172.21.15.62:0/3871030597 >> 172.21.15.62:0/3871030597 conn(0x5614105e15a0 msgr2=0x5614105df7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.048+0000 7f10c2c50740 1 -- 172.21.15.62:0/3871030597 shutdown_connections 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.048+0000 7f10c2c50740 1 -- 172.21.15.62:0/3871030597 wait complete. 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.048+0000 7f10c2c50740 1 -- start start 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10c2c50740 1 Processor -- start 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105a2310 0x5614105ddca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 0x5614105de1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10c2c50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105dab40 0x5614105ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10c2c50740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f109c00cd20 con 0x5614105d95d0 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10c2c50740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f109c00e190 con 0x5614105a2310 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10c2c50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f109c00d3f0 con 0x5614105dab40 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10c2c50740 10 client.0 _ll_register_callbacks cb 0x7f10b000f730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 0x5614105de1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10b9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 0x5614105de1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60082/0 (socket says 172.21.15.62:60082) 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10b9d9b640 1 -- 172.21.15.62:0/4212946692 learned_addr learned my addr 172.21.15.62:0/4212946692 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10bad9d640 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105dab40 0x5614105ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10ba59c640 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105a2310 0x5614105ddca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10b9d9b640 1 -- 172.21.15.62:0/4212946692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105dab40 msgr2=0x5614105ef4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105dab40 0x5614105ef4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10b9d9b640 1 -- 172.21.15.62:0/4212946692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105a2310 msgr2=0x5614105ddca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105a2310 0x5614105ddca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.049+0000 7f10b9d9b640 1 -- 172.21.15.62:0/4212946692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614105efa90 con 0x5614105d95d0 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.050+0000 7f10bad9d640 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105dab40 0x5614105ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.050+0000 7f10b9d9b640 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 0x5614105de1e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f10b0004e10 tx=0x7f10b00049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ea7b0120bacd5b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.050+0000 7f10a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:57:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.050+0000 7f10a1ffb640 1 -- 172.21.15.62:0/4212946692 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b003d020 con 0x5614105d95d0 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.050+0000 7f10a1ffb640 1 -- 172.21.15.62:0/4212946692 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f10b0005190 con 0x5614105d95d0 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.050+0000 7f10c2c50740 1 -- 172.21.15.62:0/4212946692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b0010b40 con 0x5614105d95d0 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.051+0000 7f10a1ffb640 1 -- 172.21.15.62:0/4212946692 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10b0048050 con 0x5614105d95d0 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.051+0000 7f10c2c50740 10 client.16323 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.051+0000 7f10c2c50740 1 -- 172.21.15.62:0/4212946692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5614105f0d50 con 0x5614105d95d0 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.051+0000 7f10a1ffb640 1 -- 172.21.15.62:0/4212946692 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b0010b40 con 0x5614105d95d0 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.052+0000 7f10a1ffb640 1 -- 172.21.15.62:0/4212946692 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f10b0025040 con 0x5614105d95d0 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.052+0000 7f10c2c50740 10 client.16323 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.052+0000 7f10c2c50740 10 client.16323 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.052+0000 7f10c2c50740 20 client.16323 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.052+0000 7f10c2c50740 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5614105e4370 0x561410660e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.052+0000 7f10c2c50740 4 client.16323 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.052+0000 7f10c2c50740 1 -- 172.21.15.62:0/4212946692 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561410546d70 con 0x5614105e4370 2024-09-11T23:57:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10ba59c640 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5614105e4370 0x561410660e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10ba59c640 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5614105e4370 0x561410660e20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3e808c7dd93f99b server_cookie=1a084681c27943ef in_seq=0 out_seq=0 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10a1ffb640 10 client.16323 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10a1ffb640 1 -- 172.21.15.62:0/4212946692 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561410546d70 con 0x5614105e4370 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10a1ffb640 10 client.16323 handle_command_reply: tid=0 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10c2c50740 1 client.16323 shutdown 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10c2c50740 1 -- 172.21.15.62:0/4212946692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 msgr2=0x5614105de1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10c2c50740 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 0x5614105de1e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f10b0004e10 tx=0x7f10b00049e0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10c2c50740 1 -- 172.21.15.62:0/4212946692 shutdown_connections 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10c2c50740 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5614105e4370 0x561410660e20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10c2c50740 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614105dab40 0x5614105ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10c2c50740 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614105d95d0 0x5614105de1e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10a1ffb640 0 client.16323 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:57:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10c2c50740 1 --2- 172.21.15.62:0/4212946692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614105a2310 0x5614105ddca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.053+0000 7f10c2c50740 1 -- 172.21.15.62:0/4212946692 >> 172.21.15.62:0/4212946692 conn(0x5614105e15a0 msgr2=0x5614105e0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:21.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c2c50740 1 -- 172.21.15.62:0/4212946692 shutdown_connections 2024-09-11T23:57:21.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c2c50740 1 -- 172.21.15.62:0/4212946692 wait complete. 2024-09-11T23:57:21.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c2c50740 20 client.16323 trim_cache size 0 max 16384 2024-09-11T23:57:21.060 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:57:21.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 -- 172.21.15.62:0/3154946851 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10a40a2cf0 msgr2=0x7f10a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:21.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10a40a2cf0 0x7f10a40a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0fec10 tx=0x7f10b0009ce0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 -- 172.21.15.62:0/3154946851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc102ef0 msgr2=0x7f10bc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:21.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc102ef0 0x7f10bc19cd60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f10b4005cc0 tx=0x7f10b402f420 comp rx=0 tx=0).stop 2024-09-11T23:57:21.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 -- 172.21.15.62:0/3154946851 shutdown_connections 2024-09-11T23:57:21.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10a40a2cf0 0x7f10a40a51b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc102ef0 0x7f10bc19cd60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10bc102570 0x7f10bc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 --2- 172.21.15.62:0/3154946851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10bc100e20 0x7f10bc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:21.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 -- 172.21.15.62:0/3154946851 >> 172.21.15.62:0/3154946851 conn(0x7f10bc0fc7d0 msgr2=0x7f10bc103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:21.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 -- 172.21.15.62:0/3154946851 shutdown_connections 2024-09-11T23:57:21.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:21.054+0000 7f10c0877640 1 -- 172.21.15.62:0/3154946851 wait complete. 2024-09-11T23:57:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:21 smithi120 ceph-mon[35182]: pgmap v160: 97 pgs: 97 active+clean; 2.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 660 op/s 2024-09-11T23:57:21.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:21 smithi083 ceph-mon[34805]: pgmap v160: 97 pgs: 97 active+clean; 2.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 660 op/s 2024-09-11T23:57:21.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:21 smithi062 ceph-mon[30958]: pgmap v160: 97 pgs: 97 active+clean; 2.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 660 op/s 2024-09-11T23:57:22.087 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:57:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 280.0 s, 2956.6 tps, lat 33.875 ms stddev 37.611 2024-09-11T23:57:22.469 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:57:22.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:22 smithi083 ceph-mon[34805]: pgmap v161: 97 pgs: 97 active+clean; 2.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 669 op/s 2024-09-11T23:57:22.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:22 smithi062 ceph-mon[30958]: pgmap v161: 97 pgs: 97 active+clean; 2.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 669 op/s 2024-09-11T23:57:22.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:22 smithi120 ceph-mon[35182]: pgmap v161: 97 pgs: 97 active+clean; 2.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 669 op/s 2024-09-11T23:57:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.051+0000 7f46be348640 1 -- 172.21.15.62:0/984968065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46b81011e0 msgr2=0x7f46b8105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.051+0000 7f46be348640 1 --2- 172.21.15.62:0/984968065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46b81011e0 0x7f46b8105980 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009890 tx=0x7f46a002a360 comp rx=0 tx=0).stop 2024-09-11T23:57:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.051+0000 7f46be348640 1 -- 172.21.15.62:0/984968065 shutdown_connections 2024-09-11T23:57:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.051+0000 7f46be348640 1 --2- 172.21.15.62:0/984968065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b8105ec0 0x7f46b8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.051+0000 7f46be348640 1 --2- 172.21.15.62:0/984968065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46b81011e0 0x7f46b8105980 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.051+0000 7f46be348640 1 --2- 172.21.15.62:0/984968065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b81008c0 0x7f46b8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.051+0000 7f46be348640 1 -- 172.21.15.62:0/984968065 >> 172.21.15.62:0/984968065 conn(0x7f46b80fc7d0 msgr2=0x7f46b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.051+0000 7f46be348640 1 -- 172.21.15.62:0/984968065 shutdown_connections 2024-09-11T23:57:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.051+0000 7f46be348640 1 -- 172.21.15.62:0/984968065 wait complete. 2024-09-11T23:57:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46be348640 1 Processor -- start 2024-09-11T23:57:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46be348640 1 -- start start 2024-09-11T23:57:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46be348640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46b81008c0 0x7f46b81a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46be348640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b81011e0 0x7f46b81a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46be348640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b8105ec0 0x7f46b819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46be348640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8118630 con 0x7f46b8105ec0 2024-09-11T23:57:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46be348640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81184b0 con 0x7f46b81008c0 2024-09-11T23:57:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46be348640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81187b0 con 0x7f46b81011e0 2024-09-11T23:57:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46b81008c0 0x7f46b81a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46bc8e5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b8105ec0 0x7f46b819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46b81008c0 0x7f46b81a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:50560/0 (socket says 172.21.15.62:50560) 2024-09-11T23:57:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46bc8e5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b8105ec0 0x7f46b819e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41580/0 (socket says 172.21.15.62:41580) 2024-09-11T23:57:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46b7fff640 1 -- 172.21.15.62:0/3157914652 learned_addr learned my addr 172.21.15.62:0/3157914652 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.052+0000 7f46affff640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b81011e0 0x7f46b81a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.053+0000 7f46b7fff640 1 -- 172.21.15.62:0/3157914652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b81011e0 msgr2=0x7f46b81a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.053+0000 7f46b7fff640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b81011e0 0x7f46b81a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.053+0000 7f46b7fff640 1 -- 172.21.15.62:0/3157914652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b8105ec0 msgr2=0x7f46b819e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.053+0000 7f46b7fff640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b8105ec0 0x7f46b819e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.053+0000 7f46b7fff640 1 -- 172.21.15.62:0/3157914652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b819f110 con 0x7f46b81008c0 2024-09-11T23:57:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.054+0000 7f46b7fff640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46b81008c0 0x7f46b81a4820 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f46a800e9f0 tx=0x7f46a800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:23.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.054+0000 7f46b5ffb640 1 -- 172.21.15.62:0/3157914652 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a800cd80 con 0x7f46b81008c0 2024-09-11T23:57:23.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.055+0000 7f46b5ffb640 1 -- 172.21.15.62:0/3157914652 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a8005500 con 0x7f46b81008c0 2024-09-11T23:57:23.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.055+0000 7f46be348640 1 -- 172.21.15.62:0/3157914652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b819f400 con 0x7f46b81008c0 2024-09-11T23:57:23.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.056+0000 7f46b5ffb640 1 -- 172.21.15.62:0/3157914652 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8010690 con 0x7f46b81008c0 2024-09-11T23:57:23.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.056+0000 7f46be348640 1 -- 172.21.15.62:0/3157914652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b81ab6f0 con 0x7f46b81008c0 2024-09-11T23:57:23.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.058+0000 7f46b5ffb640 1 -- 172.21.15.62:0/3157914652 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f46a8004ef0 con 0x7f46b81008c0 2024-09-11T23:57:23.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.060+0000 7f46b5ffb640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f46a40a2e40 0x7f46a40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.060+0000 7f46b5ffb640 1 -- 172.21.15.62:0/3157914652 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f46a80cf6b0 con 0x7f46b81008c0 2024-09-11T23:57:23.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.060+0000 7f46affff640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f46a40a2e40 0x7f46a40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.061+0000 7f46c0721740 1 Processor -- start 2024-09-11T23:57:23.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46c0721740 1 -- start start 2024-09-11T23:57:23.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46c0721740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc5b40 0x56042edc4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46c0721740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edc4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46c0721740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edd1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46c0721740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56042ed2a2f0 con 0x56042edcf370 2024-09-11T23:57:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46bc8e5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edd1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edc4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46bc8e5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edd1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41596/0 (socket says 172.21.15.62:41596) 2024-09-11T23:57:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46bc8e5640 1 -- 172.21.15.62:0/2750021043 learned_addr learned my addr 172.21.15.62:0/2750021043 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46b7fff640 1 --2- 172.21.15.62:0/2750021043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc5b40 0x56042edc4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46c0721740 1 -- 172.21.15.62:0/2750021043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56042ecd1cf0 con 0x56042edc45d0 2024-09-11T23:57:23.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.062+0000 7f46c0721740 1 -- 172.21.15.62:0/2750021043 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56042ed262b0 con 0x56042edc5b40 2024-09-11T23:57:23.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.063+0000 7f46affff640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f46a40a2e40 0x7f46a40a5300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46a0035040 tx=0x7f46a00056d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:23.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.063+0000 7f46affff640 1 -- 172.21.15.62:0/2750021043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc5b40 msgr2=0x56042edc4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:23.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.063+0000 7f46affff640 1 --2- 172.21.15.62:0/2750021043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc5b40 0x56042edc4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.063+0000 7f46affff640 1 -- 172.21.15.62:0/2750021043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 msgr2=0x56042edd1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:23.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.063+0000 7f46affff640 1 --2- 172.21.15.62:0/2750021043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edd1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.063+0000 7f46affff640 1 -- 172.21.15.62:0/2750021043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56042ecb63a0 con 0x56042edc45d0 2024-09-11T23:57:23.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.063+0000 7f46affff640 1 --2- 172.21.15.62:0/2750021043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edc4a10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f46a00057c0 tx=0x7f46a003c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0baac433c35fba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:23.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.063+0000 7f46ae7fc640 1 -- 172.21.15.62:0/2750021043 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0037040 con 0x56042edc45d0 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.063+0000 7f46ae7fc640 1 -- 172.21.15.62:0/2750021043 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a0041030 con 0x56042edc45d0 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.063+0000 7f46ae7fc640 1 -- 172.21.15.62:0/2750021043 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a003d440 con 0x56042edc45d0 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.064+0000 7f46c0721740 1 -- 172.21.15.62:0/2750021043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 msgr2=0x56042edc4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.064+0000 7f46c0721740 1 --2- 172.21.15.62:0/2750021043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edc4a10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f46a00057c0 tx=0x7f46a003c000 comp rx=0 tx=0).stop 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.064+0000 7f46c0721740 1 -- 172.21.15.62:0/2750021043 shutdown_connections 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.064+0000 7f46c0721740 1 --2- 172.21.15.62:0/2750021043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edd1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.064+0000 7f46c0721740 1 --2- 172.21.15.62:0/2750021043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edc4a10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.064+0000 7f46c0721740 1 --2- 172.21.15.62:0/2750021043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc5b40 0x56042edc4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.064+0000 7f46c0721740 1 -- 172.21.15.62:0/2750021043 >> 172.21.15.62:0/2750021043 conn(0x56042ed8d310 msgr2=0x56042edc0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.064+0000 7f46c0721740 1 -- 172.21.15.62:0/2750021043 shutdown_connections 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.065+0000 7f46c0721740 1 -- 172.21.15.62:0/2750021043 wait complete. 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.065+0000 7f46c0721740 1 -- start start 2024-09-11T23:57:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46c0721740 1 Processor -- start 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46c0721740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edcc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46c0721740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc5b40 0x56042edcc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46c0721740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46c0721740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56042edd2010 con 0x56042edcf370 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46c0721740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56042ece5060 con 0x56042edc45d0 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46c0721740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56042edd2190 con 0x56042edc5b40 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46c0721740 10 client.0 _ll_register_callbacks cb 0x56042ea388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edcc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edcc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:50578/0 (socket says 172.21.15.62:50578) 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.066+0000 7f46b7fff640 1 -- 172.21.15.62:0/2804915390 learned_addr learned my addr 172.21.15.62:0/2804915390 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.067+0000 7f46b7fff640 1 -- 172.21.15.62:0/2804915390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc5b40 msgr2=0x56042edcc630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.067+0000 7f46b7fff640 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc5b40 0x56042edcc630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.067+0000 7f46b7fff640 1 -- 172.21.15.62:0/2804915390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 msgr2=0x56042edccb70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:57:23.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.067+0000 7f46b7fff640 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edccb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.067+0000 7f46b7fff640 1 -- 172.21.15.62:0/2804915390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56042ea9b3c0 con 0x56042edc45d0 2024-09-11T23:57:23.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.067+0000 7f46b7fff640 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edcc0f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f46a809b900 tx=0x7f46a809c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=585406af87b71e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.068+0000 7f46977fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:57:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.068+0000 7f46977fe640 1 -- 172.21.15.62:0/2804915390 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a80a4870 con 0x56042edc45d0 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.068+0000 7f46977fe640 1 -- 172.21.15.62:0/2804915390 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f46a80b4e00 con 0x56042edc45d0 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.068+0000 7f46c0721740 1 -- 172.21.15.62:0/2804915390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a0041030 con 0x56042edc45d0 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.069+0000 7f46977fe640 1 -- 172.21.15.62:0/2804915390 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f46a8098270 con 0x56042edc45d0 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.069+0000 7f46c0721740 10 client.25958 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.069+0000 7f46c0721740 1 -- 172.21.15.62:0/2804915390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56042edddbf0 con 0x56042edc45d0 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.069+0000 7f46977fe640 1 -- 172.21.15.62:0/2804915390 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a80046c0 con 0x56042edc45d0 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.069+0000 7f46977fe640 1 -- 172.21.15.62:0/2804915390 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f46a809b050 con 0x56042edc45d0 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.070+0000 7f46c0721740 10 client.25958 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.070+0000 7f46c0721740 10 client.25958 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.070+0000 7f46c0721740 20 client.25958 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.070+0000 7f46c0721740 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56042eddef40 0x56042ede1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.070+0000 7f46c0721740 4 client.25958 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:57:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.070+0000 7f46c0721740 1 -- 172.21.15.62:0/2804915390 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f46a0041030 con 0x56042eddef40 2024-09-11T23:57:23.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.070+0000 7f46affff640 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56042eddef40 0x56042ede1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.071+0000 7f46977fe640 10 client.25958 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:57:23.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.071+0000 7f46affff640 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56042eddef40 0x56042ede1390 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=422a6aead4fa31c0 server_cookie=a2777c581b2dad47 in_seq=0 out_seq=0 2024-09-11T23:57:23.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.072+0000 7f46977fe640 1 -- 172.21.15.62:0/2804915390 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f46a0041030 con 0x56042eddef40 2024-09-11T23:57:23.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.072+0000 7f46977fe640 10 client.25958 handle_command_reply: tid=0 2024-09-11T23:57:23.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.072+0000 7f46c0721740 1 client.25958 shutdown 2024-09-11T23:57:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.072+0000 7f46c0721740 1 -- 172.21.15.62:0/2804915390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 msgr2=0x56042edcc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.072+0000 7f46c0721740 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edcc0f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f46a809b900 tx=0x7f46a809c300 comp rx=0 tx=0).stop 2024-09-11T23:57:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.072+0000 7f46c0721740 1 -- 172.21.15.62:0/2804915390 shutdown_connections 2024-09-11T23:57:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.072+0000 7f46c0721740 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56042eddef40 0x56042ede1390 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.073+0000 7f46c0721740 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.073+0000 7f46977fe640 0 client.25958 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:57:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.073+0000 7f46c0721740 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc5b40 0x56042edcc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.073+0000 7f46c0721740 1 --2- 172.21.15.62:0/2804915390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edcc0f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.073+0000 7f46c0721740 1 -- 172.21.15.62:0/2804915390 >> 172.21.15.62:0/2804915390 conn(0x56042ed8d310 msgr2=0x56042edc69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.073+0000 7f46c0721740 1 -- 172.21.15.62:0/2804915390 shutdown_connections 2024-09-11T23:57:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.073+0000 7f46c0721740 1 -- 172.21.15.62:0/2804915390 wait complete. 2024-09-11T23:57:23.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.073+0000 7f46c0721740 20 client.25958 trim_cache size 0 max 16384 2024-09-11T23:57:23.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.095+0000 7f46c0721740 1 Processor -- start 2024-09-11T23:57:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46c0721740 1 -- start start 2024-09-11T23:57:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46c0721740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46c0721740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edc5b40 0x56042edc71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46c0721740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edcf370 0x56042edc76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46c0721740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b6660 con 0x56042edc5b40 2024-09-11T23:57:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46c0721740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b3f30 con 0x56042edc45d0 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46c0721740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b1d80 con 0x56042edcf370 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:50582/0 (socket says 172.21.15.62:50582) 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46b7fff640 1 -- 172.21.15.62:0/3653986437 learned_addr learned my addr 172.21.15.62:0/3653986437 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46bc8e5640 1 --2- 172.21.15.62:0/3653986437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edcf370 0x56042edc76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.096+0000 7f46affff640 1 --2- 172.21.15.62:0/3653986437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edc5b40 0x56042edc71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.097+0000 7f46b7fff640 1 -- 172.21.15.62:0/3653986437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edcf370 msgr2=0x56042edc76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.097+0000 7f46b7fff640 1 --2- 172.21.15.62:0/3653986437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edcf370 0x56042edc76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.097+0000 7f46b7fff640 1 -- 172.21.15.62:0/3653986437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edc5b40 msgr2=0x56042edc71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.097+0000 7f46b7fff640 1 --2- 172.21.15.62:0/3653986437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edc5b40 0x56042edc71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.097+0000 7f46b7fff640 1 -- 172.21.15.62:0/3653986437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56042edca640 con 0x56042edc45d0 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.097+0000 7f46b7fff640 1 --2- 172.21.15.62:0/3653986437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edca9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c6a0 tx=0x7f46a8004b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d87e0b0f14c9af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.097+0000 7f46adffb640 1 -- 172.21.15.62:0/3653986437 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a809b740 con 0x56042edc45d0 2024-09-11T23:57:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.097+0000 7f46adffb640 1 -- 172.21.15.62:0/3653986437 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a8097cc0 con 0x56042edc45d0 2024-09-11T23:57:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.097+0000 7f46adffb640 1 -- 172.21.15.62:0/3653986437 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8098410 con 0x56042edc45d0 2024-09-11T23:57:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.098+0000 7f46c0721740 1 -- 172.21.15.62:0/3653986437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 msgr2=0x56042edca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.098+0000 7f46c0721740 1 --2- 172.21.15.62:0/3653986437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edca9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c6a0 tx=0x7f46a8004b70 comp rx=0 tx=0).stop 2024-09-11T23:57:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.098+0000 7f46c0721740 1 -- 172.21.15.62:0/3653986437 shutdown_connections 2024-09-11T23:57:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.099+0000 7f46c0721740 1 --2- 172.21.15.62:0/3653986437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edcf370 0x56042edc76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.099+0000 7f46c0721740 1 --2- 172.21.15.62:0/3653986437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edc5b40 0x56042edc71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.099+0000 7f46c0721740 1 --2- 172.21.15.62:0/3653986437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc45d0 0x56042edca9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.099+0000 7f46c0721740 1 -- 172.21.15.62:0/3653986437 >> 172.21.15.62:0/3653986437 conn(0x56042ed8d310 msgr2=0x56042edbfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.099+0000 7f46c0721740 1 -- 172.21.15.62:0/3653986437 shutdown_connections 2024-09-11T23:57:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.099+0000 7f46c0721740 1 -- 172.21.15.62:0/3653986437 wait complete. 2024-09-11T23:57:23.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.100+0000 7f46c0721740 1 -- start start 2024-09-11T23:57:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.100+0000 7f46c0721740 1 Processor -- start 2024-09-11T23:57:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.101+0000 7f46c0721740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc45d0 0x56042edd7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.101+0000 7f46c0721740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc5b40 0x56042edd2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.101+0000 7f46c0721740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edd3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.101+0000 7f46c0721740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b23f0 con 0x56042edcf370 2024-09-11T23:57:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.101+0000 7f46c0721740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b3860 con 0x56042edc5b40 2024-09-11T23:57:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.101+0000 7f46c0721740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b2ac0 con 0x56042edc45d0 2024-09-11T23:57:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46c0721740 10 client.0 _ll_register_callbacks cb 0x7f46a809b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46bc8e5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edd3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46bc8e5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edd3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41604/0 (socket says 172.21.15.62:41604) 2024-09-11T23:57:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46bc8e5640 1 -- 172.21.15.62:0/2106303296 learned_addr learned my addr 172.21.15.62:0/2106303296 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46b7fff640 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc45d0 0x56042edd7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46affff640 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc5b40 0x56042edd2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46b7fff640 1 -- 172.21.15.62:0/2106303296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc5b40 msgr2=0x56042edd2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:23.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46b7fff640 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc5b40 0x56042edd2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46b7fff640 1 -- 172.21.15.62:0/2106303296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 msgr2=0x56042edd3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:23.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46b7fff640 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edd3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.102+0000 7f46b7fff640 1 -- 172.21.15.62:0/2106303296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56042edd3680 con 0x56042edc45d0 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.103+0000 7f46affff640 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc5b40 0x56042edd2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.103+0000 7f46b7fff640 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc45d0 0x56042edd7830 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f46a8098060 tx=0x7f46a80a4690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3452e16a894f6141 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.103+0000 7f46977fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.103+0000 7f46977fe640 1 -- 172.21.15.62:0/2106303296 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a80cd2c0 con 0x56042edc45d0 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.103+0000 7f46977fe640 1 -- 172.21.15.62:0/2106303296 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f46a80b9050 con 0x56042edc45d0 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.103+0000 7f46c0721740 1 -- 172.21.15.62:0/2106303296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a8097cc0 con 0x56042edc45d0 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46977fe640 1 -- 172.21.15.62:0/2106303296 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f46a8098410 con 0x56042edc45d0 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46c0721740 10 client.25909 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46c0721740 1 -- 172.21.15.62:0/2106303296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56042edd3680 con 0x56042edc45d0 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46977fe640 1 -- 172.21.15.62:0/2106303296 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a8097cc0 con 0x56042edc45d0 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46977fe640 1 -- 172.21.15.62:0/2106303296 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f46a80ad020 con 0x56042edc45d0 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46c0721740 10 client.25909 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46c0721740 10 client.25909 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:57:23.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46c0721740 20 client.25909 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:23.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46c0721740 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56042eddef40 0x56042ee4bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:23.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46c0721740 4 client.25909 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:57:23.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46c0721740 1 -- 172.21.15.62:0/2106303296 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56042ed37860 con 0x56042eddef40 2024-09-11T23:57:23.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.104+0000 7f46affff640 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56042eddef40 0x56042ee4bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:23.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.105+0000 7f46977fe640 10 client.25909 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:57:23.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.105+0000 7f46affff640 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56042eddef40 0x56042ee4bbb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=66d9caa52da8b470 server_cookie=5f5cf976a48969d1 in_seq=0 out_seq=0 2024-09-11T23:57:23.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.105+0000 7f46977fe640 1 -- 172.21.15.62:0/2106303296 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56042ed37860 con 0x56042eddef40 2024-09-11T23:57:23.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.105+0000 7f46977fe640 10 client.25909 handle_command_reply: tid=0 2024-09-11T23:57:23.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.105+0000 7f46c0721740 1 client.25909 shutdown 2024-09-11T23:57:23.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46c0721740 1 -- 172.21.15.62:0/2106303296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc45d0 msgr2=0x56042edd7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:23.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46c0721740 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc45d0 0x56042edd7830 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f46a8098060 tx=0x7f46a80a4690 comp rx=0 tx=0).stop 2024-09-11T23:57:23.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46c0721740 1 -- 172.21.15.62:0/2106303296 shutdown_connections 2024-09-11T23:57:23.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46c0721740 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56042eddef40 0x56042ee4bbb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46c0721740 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56042edcf370 0x56042edd3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46977fe640 0 client.25909 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:57:23.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46c0721740 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56042edc5b40 0x56042edd2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46c0721740 1 --2- 172.21.15.62:0/2106303296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56042edc45d0 0x56042edd7830 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46c0721740 1 -- 172.21.15.62:0/2106303296 >> 172.21.15.62:0/2106303296 conn(0x56042ed8d310 msgr2=0x56042edda2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:23.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46c0721740 1 -- 172.21.15.62:0/2106303296 shutdown_connections 2024-09-11T23:57:23.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.106+0000 7f46c0721740 1 -- 172.21.15.62:0/2106303296 wait complete. 2024-09-11T23:57:23.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.107+0000 7f46c0721740 20 client.25909 trim_cache size 0 max 16384 2024-09-11T23:57:23.110 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:57:23.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 -- 172.21.15.62:0/3157914652 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f46a40a2e40 msgr2=0x7f46a40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:23.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f46a40a2e40 0x7f46a40a5300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46a0035040 tx=0x7f46a00056d0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 -- 172.21.15.62:0/3157914652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46b81008c0 msgr2=0x7f46b81a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:23.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46b81008c0 0x7f46b81a4820 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f46a800e9f0 tx=0x7f46a800edf0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 -- 172.21.15.62:0/3157914652 shutdown_connections 2024-09-11T23:57:23.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f46a40a2e40 0x7f46a40a5300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b8105ec0 0x7f46b819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b81011e0 0x7f46b81a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 --2- 172.21.15.62:0/3157914652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46b81008c0 0x7f46b81a4820 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:23.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 -- 172.21.15.62:0/3157914652 >> 172.21.15.62:0/3157914652 conn(0x7f46b80fc7d0 msgr2=0x7f46b8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:23.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 -- 172.21.15.62:0/3157914652 shutdown_connections 2024-09-11T23:57:23.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:23.108+0000 7f46be348640 1 -- 172.21.15.62:0/3157914652 wait complete. 2024-09-11T23:57:23.995 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:57:24.412 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:57:24.869 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:24 smithi062 ceph-mon[30958]: pgmap v162: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 944 op/s 2024-09-11T23:57:24.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:24 smithi120 ceph-mon[35182]: pgmap v162: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 944 op/s 2024-09-11T23:57:24.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:24 smithi083 ceph-mon[34805]: pgmap v162: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 944 op/s 2024-09-11T23:57:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.996+0000 7f79594a5640 1 -- 172.21.15.62:0/1267529038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f795410a560 msgr2=0x7f795410a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.996+0000 7f79594a5640 1 --2- 172.21.15.62:0/1267529038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f795410a560 0x7f795410a940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7944009900 tx=0x7f794402a400 comp rx=0 tx=0).stop 2024-09-11T23:57:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.996+0000 7f79594a5640 1 -- 172.21.15.62:0/1267529038 shutdown_connections 2024-09-11T23:57:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.996+0000 7f79594a5640 1 --2- 172.21.15.62:0/1267529038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954113940 0x7f7954115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.996+0000 7f79594a5640 1 --2- 172.21.15.62:0/1267529038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795410ae80 0x7f7954113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.996+0000 7f79594a5640 1 --2- 172.21.15.62:0/1267529038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f795410a560 0x7f795410a940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.996+0000 7f79594a5640 1 -- 172.21.15.62:0/1267529038 >> 172.21.15.62:0/1267529038 conn(0x7f795406e140 msgr2=0x7f795406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.997+0000 7f79594a5640 1 -- 172.21.15.62:0/1267529038 shutdown_connections 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.997+0000 7f79594a5640 1 -- 172.21.15.62:0/1267529038 wait complete. 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.998+0000 7f79594a5640 1 Processor -- start 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.998+0000 7f79594a5640 1 -- start start 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.998+0000 7f79594a5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f795410a560 0x7f79541a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f79594a5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f795410ae80 0x7f79541a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f79594a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954113940 0x7f79541acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f79594a5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7954119780 con 0x7f795410a560 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f79594a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7954119600 con 0x7f7954113940 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f79594a5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7954119900 con 0x7f795410ae80 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f795410a560 0x7f79541a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f795410a560 0x7f79541a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41616/0 (socket says 172.21.15.62:41616) 2024-09-11T23:57:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f7952ffd640 1 -- 172.21.15.62:0/845990860 learned_addr learned my addr 172.21.15.62:0/845990860 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f79537fe640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954113940 0x7f79541acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f794bfff640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f795410ae80 0x7f79541a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f794bfff640 1 -- 172.21.15.62:0/845990860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954113940 msgr2=0x7f79541acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f794bfff640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954113940 0x7f79541acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f794bfff640 1 -- 172.21.15.62:0/845990860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f795410a560 msgr2=0x7f79541a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f794bfff640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f795410a560 0x7f79541a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:24.999+0000 7f794bfff640 1 -- 172.21.15.62:0/845990860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79541ad1b0 con 0x7f795410ae80 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.000+0000 7f794bfff640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f795410ae80 0x7f79541a2130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f793c00b3f0 tx=0x7f793c00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.000+0000 7f7950ff9640 1 -- 172.21.15.62:0/845990860 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c015020 con 0x7f795410ae80 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.000+0000 7f79594a5640 1 -- 172.21.15.62:0/845990860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79541ad380 con 0x7f795410ae80 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.000+0000 7f79594a5640 1 -- 172.21.15.62:0/845990860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795419fc70 con 0x7f795410ae80 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.000+0000 7f7950ff9640 1 -- 172.21.15.62:0/845990860 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793c011d10 con 0x7f795410ae80 2024-09-11T23:57:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.000+0000 7f7950ff9640 1 -- 172.21.15.62:0/845990860 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c00f2e0 con 0x7f795410ae80 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.005+0000 7f795b87e740 1 Processor -- start 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.005+0000 7f795b87e740 1 -- start start 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f7950ff9640 1 -- 172.21.15.62:0/845990860 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f793c018020 con 0x7f795410ae80 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f795b87e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c9585350 0x5573c95838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f795b87e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9584220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f795b87e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c958eb80 0x5573c9590f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f795b87e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5573c94e12f0 con 0x5573c9585350 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f795b87e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5573c9488cf0 con 0x5573c958eb80 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f795b87e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5573c94dd2b0 con 0x5573c9583de0 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9584220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9584220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40148/0 (socket says 172.21.15.62:40148) 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f7952ffd640 1 -- 172.21.15.62:0/2204128304 learned_addr learned my addr 172.21.15.62:0/2204128304 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f79537fe640 1 --2- 172.21.15.62:0/2204128304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c9585350 0x5573c95838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f7952ffd640 1 -- 172.21.15.62:0/2204128304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c958eb80 msgr2=0x5573c9590f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f7952ffd640 1 --2- 172.21.15.62:0/2204128304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c958eb80 0x5573c9590f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f7952ffd640 1 -- 172.21.15.62:0/2204128304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c9585350 msgr2=0x5573c95838a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f7952ffd640 1 --2- 172.21.15.62:0/2204128304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c9585350 0x5573c95838a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.006+0000 7f7952ffd640 1 -- 172.21.15.62:0/2204128304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c946d3a0 con 0x5573c9583de0 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.007+0000 7f7950ff9640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79340a2df0 0x7f79340a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.008+0000 7f7950ff9640 1 -- 172.21.15.62:0/845990860 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f793c0cea70 con 0x7f795410ae80 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.008+0000 7f7952ffd640 1 --2- 172.21.15.62:0/2204128304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9584220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7944004230 tx=0x7f7944004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94c00215c094407c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.008+0000 7f79537fe640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79340a2df0 0x7f79340a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.008+0000 7f79497fa640 1 -- 172.21.15.62:0/2204128304 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944040070 con 0x5573c9583de0 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.008+0000 7f79497fa640 1 -- 172.21.15.62:0/2204128304 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79440047a0 con 0x5573c9583de0 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.008+0000 7f79497fa640 1 -- 172.21.15.62:0/2204128304 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944035040 con 0x5573c9583de0 2024-09-11T23:57:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.009+0000 7f79537fe640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79340a2df0 0x7f79340a52b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f794c00a400 tx=0x7f794c00d040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:25.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.010+0000 7f795b87e740 1 -- 172.21.15.62:0/2204128304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 msgr2=0x5573c9584220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:25.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.010+0000 7f795b87e740 1 --2- 172.21.15.62:0/2204128304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9584220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7944004230 tx=0x7f7944004260 comp rx=0 tx=0).stop 2024-09-11T23:57:25.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.011+0000 7f795b87e740 1 -- 172.21.15.62:0/2204128304 shutdown_connections 2024-09-11T23:57:25.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.011+0000 7f795b87e740 1 --2- 172.21.15.62:0/2204128304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c958eb80 0x5573c9590f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.011+0000 7f795b87e740 1 --2- 172.21.15.62:0/2204128304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9584220 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.011+0000 7f795b87e740 1 --2- 172.21.15.62:0/2204128304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c9585350 0x5573c95838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.011+0000 7f795b87e740 1 -- 172.21.15.62:0/2204128304 >> 172.21.15.62:0/2204128304 conn(0x5573c9544310 msgr2=0x5573c9580470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.012+0000 7f795b87e740 1 -- 172.21.15.62:0/2204128304 shutdown_connections 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.012+0000 7f795b87e740 1 -- 172.21.15.62:0/2204128304 wait complete. 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.012+0000 7f795b87e740 1 -- start start 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.012+0000 7f795b87e740 1 Processor -- start 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f795b87e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c958e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f795b87e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c958b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f795b87e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c958eb80 0x5573c958c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f795b87e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5573c9591820 con 0x5573c958eb80 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f795b87e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5573c949c060 con 0x5573c9585350 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f795b87e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5573c95919a0 con 0x5573c9583de0 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f795b87e740 10 client.0 _ll_register_callbacks cb 0x5573c91ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c958e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f794bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c958b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c958e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40154/0 (socket says 172.21.15.62:40154) 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f7952ffd640 1 -- 172.21.15.62:0/1700595524 learned_addr learned my addr 172.21.15.62:0/1700595524 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.013+0000 7f79537fe640 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c958eb80 0x5573c958c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.014+0000 7f7952ffd640 1 -- 172.21.15.62:0/1700595524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 msgr2=0x5573c958b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.014+0000 7f7952ffd640 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c958b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.014+0000 7f7952ffd640 1 -- 172.21.15.62:0/1700595524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c958eb80 msgr2=0x5573c958c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.014+0000 7f7952ffd640 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c958eb80 0x5573c958c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.014+0000 7f7952ffd640 1 -- 172.21.15.62:0/1700595524 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c92523c0 con 0x5573c9583de0 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.014+0000 7f794bfff640 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c958b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.015+0000 7f7952ffd640 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c958e640 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7944006010 tx=0x7f7944004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed5fa750104812bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.015+0000 7f79427fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.015+0000 7f79427fc640 1 -- 172.21.15.62:0/1700595524 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794404d060 con 0x5573c9583de0 2024-09-11T23:57:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.015+0000 7f79427fc640 1 -- 172.21.15.62:0/1700595524 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f7944005430 con 0x5573c9583de0 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.015+0000 7f795b87e740 1 -- 172.21.15.62:0/1700595524 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79440047a0 con 0x5573c9583de0 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.015+0000 7f79427fc640 1 -- 172.21.15.62:0/1700595524 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7944040070 con 0x5573c9583de0 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.016+0000 7f795b87e740 10 client.25921 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.016+0000 7f795b87e740 1 -- 172.21.15.62:0/1700595524 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573c9591820 con 0x5573c9583de0 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.016+0000 7f79427fc640 1 -- 172.21.15.62:0/1700595524 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79440047a0 con 0x5573c9583de0 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.016+0000 7f79427fc640 1 -- 172.21.15.62:0/1700595524 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7944031640 con 0x5573c9583de0 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.017+0000 7f795b87e740 10 client.25921 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.017+0000 7f795b87e740 10 client.25921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.017+0000 7f795b87e740 20 client.25921 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.017+0000 7f795b87e740 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5573c957ae70 0x5573c957d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.017+0000 7f795b87e740 4 client.25921 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.017+0000 7f795b87e740 1 -- 172.21.15.62:0/1700595524 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573c94ee860 con 0x5573c957ae70 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.017+0000 7f794bfff640 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5573c957ae70 0x5573c957d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.017+0000 7f794bfff640 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5573c957ae70 0x5573c957d2a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e755a716b3468f6e server_cookie=73347a4b27c17918 in_seq=0 out_seq=0 2024-09-11T23:57:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.017+0000 7f79427fc640 10 client.25921 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f79427fc640 1 -- 172.21.15.62:0/1700595524 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573c94ee860 con 0x5573c957ae70 2024-09-11T23:57:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f79427fc640 10 client.25921 handle_command_reply: tid=0 2024-09-11T23:57:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 client.25921 shutdown 2024-09-11T23:57:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 -- 172.21.15.62:0/1700595524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 msgr2=0x5573c958e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c958e640 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7944006010 tx=0x7f7944004540 comp rx=0 tx=0).stop 2024-09-11T23:57:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 -- 172.21.15.62:0/1700595524 shutdown_connections 2024-09-11T23:57:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5573c957ae70 0x5573c957d2a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c958eb80 0x5573c958c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c958b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 --2- 172.21.15.62:0/1700595524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c958e640 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 -- 172.21.15.62:0/1700595524 >> 172.21.15.62:0/1700595524 conn(0x5573c9544310 msgr2=0x5573c9586170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f79427fc640 0 client.25921 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 -- 172.21.15.62:0/1700595524 shutdown_connections 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 1 -- 172.21.15.62:0/1700595524 wait complete. 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.018+0000 7f795b87e740 20 client.25921 trim_cache size 0 max 16384 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.044+0000 7f795b87e740 1 Processor -- start 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.044+0000 7f795b87e740 1 -- start start 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f795b87e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9587110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f795b87e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c9585350 0x5573c95774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f795b87e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c958eb80 0x5573c958b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9587110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9587110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40170/0 (socket says 172.21.15.62:40170) 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f7952ffd640 1 -- 172.21.15.62:0/1221135216 learned_addr learned my addr 172.21.15.62:0/1221135216 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f795b87e740 1 -- 172.21.15.62:0/1221135216 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b6890 con 0x5573c9585350 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f795b87e740 1 -- 172.21.15.62:0/1221135216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b4160 con 0x5573c958eb80 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f795b87e740 1 -- 172.21.15.62:0/1221135216 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b1fb0 con 0x5573c9583de0 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f794bfff640 1 --2- 172.21.15.62:0/1221135216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c9585350 0x5573c95774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.045+0000 7f79537fe640 1 --2- 172.21.15.62:0/1221135216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c958eb80 0x5573c958b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.046+0000 7f7952ffd640 1 -- 172.21.15.62:0/1221135216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c958eb80 msgr2=0x5573c958b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.046+0000 7f7952ffd640 1 --2- 172.21.15.62:0/1221135216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c958eb80 0x5573c958b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.046+0000 7f7952ffd640 1 -- 172.21.15.62:0/1221135216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c9585350 msgr2=0x5573c95774f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.046+0000 7f7952ffd640 1 --2- 172.21.15.62:0/1221135216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c9585350 0x5573c95774f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.046+0000 7f7952ffd640 1 -- 172.21.15.62:0/1221135216 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c958e970 con 0x5573c9583de0 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.046+0000 7f7952ffd640 1 --2- 172.21.15.62:0/1221135216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9587110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7944004150 tx=0x7f7944005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb6a8c6592a84974 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.047+0000 7f7948ff9640 1 -- 172.21.15.62:0/1221135216 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794404d070 con 0x5573c9583de0 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.047+0000 7f7948ff9640 1 -- 172.21.15.62:0/1221135216 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7944009520 con 0x5573c9583de0 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.047+0000 7f7948ff9640 1 -- 172.21.15.62:0/1221135216 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944035040 con 0x5573c9583de0 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.048+0000 7f795b87e740 1 -- 172.21.15.62:0/1221135216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 msgr2=0x5573c9587110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.048+0000 7f795b87e740 1 --2- 172.21.15.62:0/1221135216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9587110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7944004150 tx=0x7f7944005b50 comp rx=0 tx=0).stop 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.048+0000 7f795b87e740 1 -- 172.21.15.62:0/1221135216 shutdown_connections 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.048+0000 7f795b87e740 1 --2- 172.21.15.62:0/1221135216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c958eb80 0x5573c958b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.048+0000 7f795b87e740 1 --2- 172.21.15.62:0/1221135216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c9585350 0x5573c95774f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.048+0000 7f795b87e740 1 --2- 172.21.15.62:0/1221135216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9587110 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.048+0000 7f795b87e740 1 -- 172.21.15.62:0/1221135216 >> 172.21.15.62:0/1221135216 conn(0x5573c9544310 msgr2=0x5573c9585920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.049+0000 7f795b87e740 1 -- 172.21.15.62:0/1221135216 shutdown_connections 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.049+0000 7f795b87e740 1 -- 172.21.15.62:0/1221135216 wait complete. 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.049+0000 7f795b87e740 1 -- start start 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.050+0000 7f795b87e740 1 Processor -- start 2024-09-11T23:57:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.050+0000 7f795b87e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9589280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.050+0000 7f795b87e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c9596c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.050+0000 7f795b87e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c958eb80 0x5573c9597170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.050+0000 7f795b87e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b2620 con 0x5573c958eb80 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.050+0000 7f795b87e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b3a90 con 0x5573c9585350 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.050+0000 7f795b87e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79541b2cf0 con 0x5573c9583de0 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.050+0000 7f795b87e740 10 client.0 _ll_register_callbacks cb 0x7f794c009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f794bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c9596c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9589280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9589280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40180/0 (socket says 172.21.15.62:40180) 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f794bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c9596c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:50642/0 (socket says 172.21.15.62:50642) 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f7952ffd640 1 -- 172.21.15.62:0/1184025601 learned_addr learned my addr 172.21.15.62:0/1184025601 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f7952ffd640 1 -- 172.21.15.62:0/1184025601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 msgr2=0x5573c9596c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f7952ffd640 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c9596c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f7952ffd640 1 -- 172.21.15.62:0/1184025601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c958eb80 msgr2=0x5573c9597170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f7952ffd640 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c958eb80 0x5573c9597170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f7952ffd640 1 -- 172.21.15.62:0/1184025601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c95977b0 con 0x5573c9583de0 2024-09-11T23:57:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.051+0000 7f794bfff640 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c9596c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:57:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.052+0000 7f7952ffd640 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9589280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7944004d90 tx=0x7f7944004dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eea828f90d310 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.052+0000 7f79427fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:57:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.053+0000 7f79427fc640 1 -- 172.21.15.62:0/1184025601 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944060060 con 0x5573c9583de0 2024-09-11T23:57:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.053+0000 7f795b87e740 1 -- 172.21.15.62:0/1184025601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7944009520 con 0x5573c9583de0 2024-09-11T23:57:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.053+0000 7f79427fc640 1 -- 172.21.15.62:0/1184025601 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f7944003660 con 0x5573c9583de0 2024-09-11T23:57:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.053+0000 7f79427fc640 1 -- 172.21.15.62:0/1184025601 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7944035070 con 0x5573c9583de0 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.053+0000 7f795b87e740 10 client.25927 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.053+0000 7f795b87e740 1 -- 172.21.15.62:0/1184025601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573c9610cd0 con 0x5573c9583de0 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.054+0000 7f79427fc640 1 -- 172.21.15.62:0/1184025601 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7944009520 con 0x5573c9583de0 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.054+0000 7f79427fc640 1 -- 172.21.15.62:0/1184025601 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7944052690 con 0x5573c9583de0 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.054+0000 7f795b87e740 10 client.25927 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.054+0000 7f795b87e740 10 client.25927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.054+0000 7f795b87e740 20 client.25927 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.054+0000 7f795b87e740 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5573c9611040 0x5573c96134a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.054+0000 7f795b87e740 4 client.25927 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.054+0000 7f795b87e740 1 -- 172.21.15.62:0/1184025601 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573c94e8d70 con 0x5573c9611040 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.054+0000 7f794bfff640 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5573c9611040 0x5573c96134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.055+0000 7f79427fc640 10 client.25927 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.055+0000 7f794bfff640 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5573c9611040 0x5573c96134a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47337917ebd99892 server_cookie=4049a2134339db08 in_seq=0 out_seq=0 2024-09-11T23:57:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.055+0000 7f79427fc640 1 -- 172.21.15.62:0/1184025601 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573c94e8d70 con 0x5573c9611040 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.055+0000 7f79427fc640 10 client.25927 handle_command_reply: tid=0 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.055+0000 7f795b87e740 1 client.25927 shutdown 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f795b87e740 1 -- 172.21.15.62:0/1184025601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 msgr2=0x5573c9589280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f795b87e740 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9589280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7944004d90 tx=0x7f7944004dc0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f795b87e740 1 -- 172.21.15.62:0/1184025601 shutdown_connections 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f795b87e740 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5573c9611040 0x5573c96134a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f795b87e740 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c958eb80 0x5573c9597170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f795b87e740 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573c9585350 0x5573c9596c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f795b87e740 1 --2- 172.21.15.62:0/1184025601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c9583de0 0x5573c9589280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f795b87e740 1 -- 172.21.15.62:0/1184025601 >> 172.21.15.62:0/1184025601 conn(0x5573c9544310 msgr2=0x5573c9586140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f79427fc640 0 client.25927 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f795b87e740 1 -- 172.21.15.62:0/1184025601 shutdown_connections 2024-09-11T23:57:25.496 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:57:25.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.056+0000 7f795b87e740 1 -- 172.21.15.62:0/1184025601 wait complete. 2024-09-11T23:57:25.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.057+0000 7f795b87e740 20 client.25927 trim_cache size 0 max 16384 2024-09-11T23:57:25.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.058+0000 7f79594a5640 1 -- 172.21.15.62:0/845990860 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79340a2df0 msgr2=0x7f79340a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.058+0000 7f79594a5640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79340a2df0 0x7f79340a52b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f794c00a400 tx=0x7f794c00d040 comp rx=0 tx=0).stop 2024-09-11T23:57:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.058+0000 7f79594a5640 1 -- 172.21.15.62:0/845990860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f795410ae80 msgr2=0x7f79541a2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.058+0000 7f79594a5640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f795410ae80 0x7f79541a2130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f793c00b3f0 tx=0x7f793c00b8b0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.059+0000 7f79594a5640 1 -- 172.21.15.62:0/845990860 shutdown_connections 2024-09-11T23:57:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.059+0000 7f79594a5640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79340a2df0 0x7f79340a52b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.059+0000 7f79594a5640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7954113940 0x7f79541acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.059+0000 7f79594a5640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f795410ae80 0x7f79541a2130 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.059+0000 7f79594a5640 1 --2- 172.21.15.62:0/845990860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f795410a560 0x7f79541a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.059+0000 7f79594a5640 1 -- 172.21.15.62:0/845990860 >> 172.21.15.62:0/845990860 conn(0x7f795406e140 msgr2=0x7f79540709a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.059+0000 7f79594a5640 1 -- 172.21.15.62:0/845990860 shutdown_connections 2024-09-11T23:57:25.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:25.059+0000 7f79594a5640 1 -- 172.21.15.62:0/845990860 wait complete. 2024-09-11T23:57:26.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:26 smithi120 ceph-mon[35182]: pgmap v163: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 638 op/s 2024-09-11T23:57:26.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:26 smithi083 ceph-mon[34805]: pgmap v163: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 638 op/s 2024-09-11T23:57:26.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:26 smithi062 ceph-mon[30958]: pgmap v163: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 638 op/s 2024-09-11T23:57:27.301 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:57:27.302 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:57:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 285.0 s, 3009.2 tps, lat 33.261 ms stddev 39.778 2024-09-11T23:57:27.710 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:57:28.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.351+0000 7faa70198640 1 -- 172.21.15.62:0/250616190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa68076720 msgr2=0x7faa68076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:28.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.351+0000 7faa70198640 1 --2- 172.21.15.62:0/250616190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa68076720 0x7faa68076b00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faa60009be0 tx=0x7faa6002a710 comp rx=0 tx=0).stop 2024-09-11T23:57:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.352+0000 7faa70198640 1 -- 172.21.15.62:0/250616190 shutdown_connections 2024-09-11T23:57:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.352+0000 7faa70198640 1 --2- 172.21.15.62:0/250616190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa68112a30 0x7faa68114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.352+0000 7faa70198640 1 --2- 172.21.15.62:0/250616190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa68077040 0x7faa681124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.352+0000 7faa70198640 1 --2- 172.21.15.62:0/250616190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa68076720 0x7faa68076b00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.352+0000 7faa70198640 1 -- 172.21.15.62:0/250616190 >> 172.21.15.62:0/250616190 conn(0x7faa680ffa70 msgr2=0x7faa68101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.353+0000 7faa70198640 1 -- 172.21.15.62:0/250616190 shutdown_connections 2024-09-11T23:57:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.353+0000 7faa70198640 1 -- 172.21.15.62:0/250616190 wait complete. 2024-09-11T23:57:28.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.355+0000 7faa70198640 1 Processor -- start 2024-09-11T23:57:28.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.355+0000 7faa70198640 1 -- start start 2024-09-11T23:57:28.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa70198640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa68076720 0x7faa681a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa70198640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa68077040 0x7faa681a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa70198640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa68112a30 0x7faa6819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa6df34640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa68076720 0x7faa681a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa70198640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faa68118650 con 0x7faa68076720 2024-09-11T23:57:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa70198640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa681184d0 con 0x7faa68112a30 2024-09-11T23:57:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa70198640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faa681187d0 con 0x7faa68077040 2024-09-11T23:57:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa6df34640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa68076720 0x7faa681a47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41670/0 (socket says 172.21.15.62:41670) 2024-09-11T23:57:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa6df34640 1 -- 172.21.15.62:0/936707560 learned_addr learned my addr 172.21.15.62:0/936707560 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa6d733640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa68077040 0x7faa681a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.356+0000 7faa6e735640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa68112a30 0x7faa6819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.357+0000 7faa6d733640 1 -- 172.21.15.62:0/936707560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa68112a30 msgr2=0x7faa6819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:28.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.357+0000 7faa6d733640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa68112a30 0x7faa6819e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.357+0000 7faa6d733640 1 -- 172.21.15.62:0/936707560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa68076720 msgr2=0x7faa681a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.357+0000 7faa6d733640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa68076720 0x7faa681a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.357+0000 7faa6d733640 1 -- 172.21.15.62:0/936707560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6819f0b0 con 0x7faa68077040 2024-09-11T23:57:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.357+0000 7faa6d733640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa68077040 0x7faa681a4d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faa5800d6d0 tx=0x7faa5800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.357+0000 7faa5effd640 1 -- 172.21.15.62:0/936707560 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58017070 con 0x7faa68077040 2024-09-11T23:57:28.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.357+0000 7faa5effd640 1 -- 172.21.15.62:0/936707560 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa58010460 con 0x7faa68077040 2024-09-11T23:57:28.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.357+0000 7faa5effd640 1 -- 172.21.15.62:0/936707560 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5800f600 con 0x7faa68077040 2024-09-11T23:57:28.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.357+0000 7faa70198640 1 -- 172.21.15.62:0/936707560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6819f3a0 con 0x7faa68077040 2024-09-11T23:57:28.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.358+0000 7faa70198640 1 -- 172.21.15.62:0/936707560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa681ab690 con 0x7faa68077040 2024-09-11T23:57:28.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.361+0000 7faa5effd640 1 -- 172.21.15.62:0/936707560 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faa5801a070 con 0x7faa68077040 2024-09-11T23:57:28.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.362+0000 7faa5effd640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7faa540a2f00 0x7faa540a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.362+0000 7faa5effd640 1 -- 172.21.15.62:0/936707560 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7faa580cef50 con 0x7faa68077040 2024-09-11T23:57:28.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.363+0000 7faa72571740 1 Processor -- start 2024-09-11T23:57:28.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.363+0000 7faa6df34640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7faa540a2f00 0x7faa540a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.363+0000 7faa72571740 1 -- start start 2024-09-11T23:57:28.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.364+0000 7faa72571740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281edf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.364+0000 7faa72571740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.364+0000 7faa72571740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281fb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.364+0000 7faa6df34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.364+0000 7faa6df34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281edf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40198/0 (socket says 172.21.15.62:40198) 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.365+0000 7faa6df34640 1 -- 172.21.15.62:0/669921458 learned_addr learned my addr 172.21.15.62:0/669921458 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.365+0000 7faa6d733640 1 --2- 172.21.15.62:0/669921458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.365+0000 7faa6e735640 1 --2- 172.21.15.62:0/669921458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.365+0000 7faa72571740 1 -- 172.21.15.62:0/669921458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bb28153280 con 0x55bb281ee480 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.365+0000 7faa72571740 1 -- 172.21.15.62:0/669921458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bb280fab10 con 0x55bb281f8d40 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.365+0000 7faa72571740 1 -- 172.21.15.62:0/669921458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bb2814f240 con 0x55bb281edb60 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.366+0000 7faa6df34640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7faa540a2f00 0x7faa540a53c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa60004af0 tx=0x7faa60004a80 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.366+0000 7faa6df34640 1 -- 172.21.15.62:0/669921458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 msgr2=0x55bb281fb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.366+0000 7faa6df34640 1 --2- 172.21.15.62:0/669921458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281fb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.366+0000 7faa6df34640 1 -- 172.21.15.62:0/669921458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 msgr2=0x55bb281f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:28.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.366+0000 7faa6df34640 1 --2- 172.21.15.62:0/669921458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.366+0000 7faa6df34640 1 -- 172.21.15.62:0/669921458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb281071f0 con 0x55bb281edb60 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.366+0000 7faa6df34640 1 --2- 172.21.15.62:0/669921458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281edf40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faa60030820 tx=0x7faa6003d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae5b18c7042f49a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.366+0000 7faa53fff640 1 -- 172.21.15.62:0/669921458 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60036040 con 0x55bb281edb60 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.367+0000 7faa53fff640 1 -- 172.21.15.62:0/669921458 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa6004e070 con 0x55bb281edb60 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.367+0000 7faa53fff640 1 -- 172.21.15.62:0/669921458 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6003d6c0 con 0x55bb281edb60 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.368+0000 7faa72571740 1 -- 172.21.15.62:0/669921458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 msgr2=0x55bb281edf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.368+0000 7faa72571740 1 --2- 172.21.15.62:0/669921458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281edf40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faa60030820 tx=0x7faa6003d000 comp rx=0 tx=0).stop 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.368+0000 7faa72571740 1 -- 172.21.15.62:0/669921458 shutdown_connections 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.368+0000 7faa72571740 1 --2- 172.21.15.62:0/669921458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.368+0000 7faa72571740 1 --2- 172.21.15.62:0/669921458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.368+0000 7faa72571740 1 --2- 172.21.15.62:0/669921458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281edf40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.368+0000 7faa72571740 1 -- 172.21.15.62:0/669921458 >> 172.21.15.62:0/669921458 conn(0x55bb281ea0f0 msgr2=0x55bb281ea4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.368+0000 7faa72571740 1 -- 172.21.15.62:0/669921458 shutdown_connections 2024-09-11T23:57:28.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.368+0000 7faa72571740 1 -- 172.21.15.62:0/669921458 wait complete. 2024-09-11T23:57:28.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.369+0000 7faa72571740 1 -- start start 2024-09-11T23:57:28.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.369+0000 7faa72571740 1 Processor -- start 2024-09-11T23:57:28.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa72571740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa72571740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281ee480 0x55bb281f6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa72571740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281f8d40 0x55bb281f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa72571740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bb281fb910 con 0x55bb281f8d40 2024-09-11T23:57:28.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa72571740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bb2810df10 con 0x55bb281ee480 2024-09-11T23:57:28.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa72571740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bb281fba90 con 0x55bb281edb60 2024-09-11T23:57:28.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa72571740 10 client.0 _ll_register_callbacks cb 0x55bb280d6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa6df34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa6df34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40214/0 (socket says 172.21.15.62:40214) 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa6df34640 1 -- 172.21.15.62:0/3401237187 learned_addr learned my addr 172.21.15.62:0/3401237187 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa6e735640 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281f8d40 0x55bb281f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa6d733640 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281ee480 0x55bb281f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa6df34640 1 -- 172.21.15.62:0/3401237187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281ee480 msgr2=0x55bb281f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.370+0000 7faa6df34640 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281ee480 0x55bb281f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa6df34640 1 -- 172.21.15.62:0/3401237187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281f8d40 msgr2=0x55bb281f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa6df34640 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281f8d40 0x55bb281f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa6df34640 1 -- 172.21.15.62:0/3401237187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb27ec46a0 con 0x55bb281edb60 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa6df34640 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f5c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faa60004610 tx=0x7faa6002ae90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9ab318ee5dbdfb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:28.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa517fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa517fa640 1 -- 172.21.15.62:0/3401237187 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60003b70 con 0x55bb281edb60 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa517fa640 1 -- 172.21.15.62:0/3401237187 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7faa60004050 con 0x55bb281edb60 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa72571740 1 -- 172.21.15.62:0/3401237187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa6004e070 con 0x55bb281edb60 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa517fa640 1 -- 172.21.15.62:0/3401237187 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa60003d10 con 0x55bb281edb60 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa72571740 10 client.25936 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.371+0000 7faa72571740 1 -- 172.21.15.62:0/3401237187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb281fb910 con 0x55bb281edb60 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.372+0000 7faa517fa640 1 -- 172.21.15.62:0/3401237187 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa6004e070 con 0x55bb281edb60 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.372+0000 7faa517fa640 1 -- 172.21.15.62:0/3401237187 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7faa60048070 con 0x55bb281edb60 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.372+0000 7faa72571740 10 client.25936 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.372+0000 7faa72571740 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:57:28.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.372+0000 7faa72571740 20 client.25936 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:28.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.372+0000 7faa72571740 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bb28208690 0x55bb2820aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.372+0000 7faa72571740 4 client.25936 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:57:28.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.372+0000 7faa72571740 1 -- 172.21.15.62:0/3401237187 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb281607f0 con 0x55bb28208690 2024-09-11T23:57:28.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.372+0000 7faa6d733640 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bb28208690 0x55bb2820aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.373+0000 7faa6d733640 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bb28208690 0x55bb2820aa80 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cecd1e763de494b9 server_cookie=24e6f971dc69d875 in_seq=0 out_seq=0 2024-09-11T23:57:28.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.373+0000 7faa517fa640 10 client.25936 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:28.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.374+0000 7faa517fa640 1 -- 172.21.15.62:0/3401237187 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb281607f0 con 0x55bb28208690 2024-09-11T23:57:28.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.374+0000 7faa517fa640 10 client.25936 handle_command_reply: tid=0 2024-09-11T23:57:28.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.374+0000 7faa72571740 1 client.25936 shutdown 2024-09-11T23:57:28.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.374+0000 7faa72571740 1 -- 172.21.15.62:0/3401237187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 msgr2=0x55bb281f5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:28.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.374+0000 7faa72571740 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f5c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faa60004610 tx=0x7faa6002ae90 comp rx=0 tx=0).stop 2024-09-11T23:57:28.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.375+0000 7faa72571740 1 -- 172.21.15.62:0/3401237187 shutdown_connections 2024-09-11T23:57:28.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.375+0000 7faa72571740 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bb28208690 0x55bb2820aa80 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.375+0000 7faa72571740 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281f8d40 0x55bb281f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.375+0000 7faa72571740 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281ee480 0x55bb281f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.375+0000 7faa72571740 1 --2- 172.21.15.62:0/3401237187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f5c20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.375+0000 7faa517fa640 0 client.25936 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:28.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.375+0000 7faa72571740 1 -- 172.21.15.62:0/3401237187 >> 172.21.15.62:0/3401237187 conn(0x55bb281ea0f0 msgr2=0x55bb281efe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:28.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.375+0000 7faa72571740 1 -- 172.21.15.62:0/3401237187 shutdown_connections 2024-09-11T23:57:28.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.375+0000 7faa72571740 1 -- 172.21.15.62:0/3401237187 wait complete. 2024-09-11T23:57:28.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.375+0000 7faa72571740 20 client.25936 trim_cache size 0 max 16384 2024-09-11T23:57:28.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa72571740 1 Processor -- start 2024-09-11T23:57:28.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa72571740 1 -- start start 2024-09-11T23:57:28.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa72571740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa72571740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa72571740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa6df34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa72571740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faa681b6790 con 0x55bb281ee480 2024-09-11T23:57:28.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa72571740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa681b4060 con 0x55bb281f8d40 2024-09-11T23:57:28.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa72571740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faa681b1eb0 con 0x55bb281edb60 2024-09-11T23:57:28.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa6df34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40218/0 (socket says 172.21.15.62:40218) 2024-09-11T23:57:28.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa6df34640 1 -- 172.21.15.62:0/2219752566 learned_addr learned my addr 172.21.15.62:0/2219752566 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:28.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.409+0000 7faa6d733640 1 --2- 172.21.15.62:0/2219752566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.410+0000 7faa6df34640 1 -- 172.21.15.62:0/2219752566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 msgr2=0x55bb281f0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:57:28.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.410+0000 7faa6df34640 1 --2- 172.21.15.62:0/2219752566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281f0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.410+0000 7faa6df34640 1 -- 172.21.15.62:0/2219752566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 msgr2=0x55bb281efcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:28.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.410+0000 7faa6df34640 1 --2- 172.21.15.62:0/2219752566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281efcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.410+0000 7faa6df34640 1 -- 172.21.15.62:0/2219752566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb281eb1a0 con 0x55bb281edb60 2024-09-11T23:57:28.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.410+0000 7faa6df34640 1 --2- 172.21.15.62:0/2219752566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f64f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa60009920 tx=0x7faa60004780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b970330c0495d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:28.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.410+0000 7faa537fe640 1 -- 172.21.15.62:0/2219752566 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60003b70 con 0x55bb281edb60 2024-09-11T23:57:28.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.410+0000 7faa537fe640 1 -- 172.21.15.62:0/2219752566 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa6004e040 con 0x55bb281edb60 2024-09-11T23:57:28.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.410+0000 7faa537fe640 1 -- 172.21.15.62:0/2219752566 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60004110 con 0x55bb281edb60 2024-09-11T23:57:28.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.411+0000 7faa72571740 1 -- 172.21.15.62:0/2219752566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 msgr2=0x55bb281f64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:28.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.411+0000 7faa72571740 1 --2- 172.21.15.62:0/2219752566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f64f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa60009920 tx=0x7faa60004780 comp rx=0 tx=0).stop 2024-09-11T23:57:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.411+0000 7faa72571740 1 -- 172.21.15.62:0/2219752566 shutdown_connections 2024-09-11T23:57:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.411+0000 7faa72571740 1 --2- 172.21.15.62:0/2219752566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.411+0000 7faa72571740 1 --2- 172.21.15.62:0/2219752566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.411+0000 7faa72571740 1 --2- 172.21.15.62:0/2219752566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb281f64f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.411+0000 7faa72571740 1 -- 172.21.15.62:0/2219752566 >> 172.21.15.62:0/2219752566 conn(0x55bb281ea0f0 msgr2=0x55bb281ef5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.412+0000 7faa72571740 1 -- 172.21.15.62:0/2219752566 shutdown_connections 2024-09-11T23:57:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.412+0000 7faa72571740 1 -- 172.21.15.62:0/2219752566 wait complete. 2024-09-11T23:57:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.412+0000 7faa72571740 1 -- start start 2024-09-11T23:57:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.412+0000 7faa72571740 1 Processor -- start 2024-09-11T23:57:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.413+0000 7faa72571740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb28201140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.413+0000 7faa72571740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281f2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.413+0000 7faa72571740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.413+0000 7faa72571740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faa681b2520 con 0x55bb281ee480 2024-09-11T23:57:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.413+0000 7faa72571740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa681b3990 con 0x55bb281f8d40 2024-09-11T23:57:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.413+0000 7faa72571740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faa681b2bf0 con 0x55bb281edb60 2024-09-11T23:57:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.413+0000 7faa72571740 10 client.0 _ll_register_callbacks cb 0x7faa58097790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.413+0000 7faa6df34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb28201140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.414+0000 7faa6e735640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.414+0000 7faa6d733640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.414+0000 7faa6df34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb28201140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40230/0 (socket says 172.21.15.62:40230) 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.414+0000 7faa6df34640 1 -- 172.21.15.62:0/1152005799 learned_addr learned my addr 172.21.15.62:0/1152005799 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.414+0000 7faa6df34640 1 -- 172.21.15.62:0/1152005799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 msgr2=0x55bb281f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.414+0000 7faa6df34640 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.414+0000 7faa6df34640 1 -- 172.21.15.62:0/1152005799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 msgr2=0x55bb281f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.414+0000 7faa6df34640 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.414+0000 7faa6df34640 1 -- 172.21.15.62:0/1152005799 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb281f31e0 con 0x55bb281edb60 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.415+0000 7faa6df34640 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb28201140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa60030fc0 tx=0x7faa60003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3df43d13e3dba6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.415+0000 7faa517fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.415+0000 7faa517fa640 1 -- 172.21.15.62:0/1152005799 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60069020 con 0x55bb281edb60 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.415+0000 7faa517fa640 1 -- 172.21.15.62:0/1152005799 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7faa600040b0 con 0x55bb281edb60 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.415+0000 7faa72571740 1 -- 172.21.15.62:0/1152005799 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa6004e040 con 0x55bb281edb60 2024-09-11T23:57:28.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.416+0000 7faa517fa640 1 -- 172.21.15.62:0/1152005799 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa60041070 con 0x55bb281edb60 2024-09-11T23:57:28.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.416+0000 7faa72571740 10 client.25942 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:28.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.416+0000 7faa72571740 1 -- 172.21.15.62:0/1152005799 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb28272ec0 con 0x55bb281edb60 2024-09-11T23:57:28.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.416+0000 7faa517fa640 1 -- 172.21.15.62:0/1152005799 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa6004e040 con 0x55bb281edb60 2024-09-11T23:57:28.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.416+0000 7faa517fa640 1 -- 172.21.15.62:0/1152005799 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7faa60048040 con 0x55bb281edb60 2024-09-11T23:57:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.417+0000 7faa72571740 10 client.25942 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.417+0000 7faa72571740 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:57:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.417+0000 7faa72571740 20 client.25942 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.417+0000 7faa72571740 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bb28273350 0x55bb28275770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.417+0000 7faa72571740 4 client.25942 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:57:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.417+0000 7faa72571740 1 -- 172.21.15.62:0/1152005799 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bb2815ad00 con 0x55bb28273350 2024-09-11T23:57:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.417+0000 7faa6d733640 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bb28273350 0x55bb28275770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.418+0000 7faa6d733640 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bb28273350 0x55bb28275770 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=545e7fd63ebd14a5 server_cookie=141e0f3132770f19 in_seq=0 out_seq=0 2024-09-11T23:57:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.418+0000 7faa517fa640 10 client.25942 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.419+0000 7faa517fa640 1 -- 172.21.15.62:0/1152005799 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bb2815ad00 con 0x55bb28273350 2024-09-11T23:57:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.419+0000 7faa517fa640 10 client.25942 handle_command_reply: tid=0 2024-09-11T23:57:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.419+0000 7faa72571740 1 client.25942 shutdown 2024-09-11T23:57:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.421+0000 7faa72571740 1 -- 172.21.15.62:0/1152005799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 msgr2=0x55bb28201140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.421+0000 7faa72571740 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb28201140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa60030fc0 tx=0x7faa60003ba0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.421+0000 7faa72571740 1 -- 172.21.15.62:0/1152005799 shutdown_connections 2024-09-11T23:57:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.421+0000 7faa72571740 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bb28273350 0x55bb28275770 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.421+0000 7faa72571740 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb281f8d40 0x55bb281f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.421+0000 7faa72571740 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb281ee480 0x55bb281f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.421+0000 7faa72571740 1 --2- 172.21.15.62:0/1152005799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb281edb60 0x55bb28201140 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.421+0000 7faa517fa640 0 client.25942 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:28.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.421+0000 7faa72571740 1 -- 172.21.15.62:0/1152005799 >> 172.21.15.62:0/1152005799 conn(0x55bb281ea0f0 msgr2=0x55bb28204380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:28.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.422+0000 7faa72571740 1 -- 172.21.15.62:0/1152005799 shutdown_connections 2024-09-11T23:57:28.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.422+0000 7faa72571740 1 -- 172.21.15.62:0/1152005799 wait complete. 2024-09-11T23:57:28.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.422+0000 7faa72571740 20 client.25942 trim_cache size 0 max 16384 2024-09-11T23:57:28.424 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:57:28.424 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:57:28.425 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "d0d53114-7511-4050-8682-4fed6cb8c4d7", 2024-09-11T23:57:28.425 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:57:28.425 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:57:28.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 -- 172.21.15.62:0/936707560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7faa540a2f00 msgr2=0x7faa540a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7faa540a2f00 0x7faa540a53c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa60004af0 tx=0x7faa60004a80 comp rx=0 tx=0).stop 2024-09-11T23:57:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 -- 172.21.15.62:0/936707560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa68077040 msgr2=0x7faa681a4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa68077040 0x7faa681a4d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faa5800d6d0 tx=0x7faa5800db90 comp rx=0 tx=0).stop 2024-09-11T23:57:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 -- 172.21.15.62:0/936707560 shutdown_connections 2024-09-11T23:57:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7faa540a2f00 0x7faa540a53c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa68112a30 0x7faa6819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa68077040 0x7faa681a4d00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 --2- 172.21.15.62:0/936707560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa68076720 0x7faa681a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 -- 172.21.15.62:0/936707560 >> 172.21.15.62:0/936707560 conn(0x7faa680ffa70 msgr2=0x7faa68101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:28.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 -- 172.21.15.62:0/936707560 shutdown_connections 2024-09-11T23:57:28.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:28.423+0000 7faa70198640 1 -- 172.21.15.62:0/936707560 wait complete. 2024-09-11T23:57:28.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:28 smithi062 ceph-mon[30958]: pgmap v164: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 1.07k op/s 2024-09-11T23:57:28.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:28 smithi120 ceph-mon[35182]: pgmap v164: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 1.07k op/s 2024-09-11T23:57:28.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:28 smithi083 ceph-mon[34805]: pgmap v164: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 1.07k op/s 2024-09-11T23:57:29.269 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:57:29.694 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:57:29.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:29 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:57:29.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:29 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:29 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:57:29.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:29 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:57:29.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:29 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:29 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:57:29.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:29 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:29 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:29 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:57:29.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:29 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:29 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:57:29.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:29 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:57:29.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:29 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:29 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:57:29.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:29 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:29 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:29 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:57:29.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:29 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:29 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:57:29.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:29 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:57:29.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:29 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:29 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:57:29.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:29 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:29.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:29 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:57:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.517+0000 7f743292c640 1 -- 172.21.15.62:0/2273850864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c100090 msgr2=0x7f742c100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.517+0000 7f743292c640 1 --2- 172.21.15.62:0/2273850864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c100090 0x7f742c100510 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7414009900 tx=0x7f741402a410 comp rx=0 tx=0).stop 2024-09-11T23:57:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.517+0000 7f743292c640 1 -- 172.21.15.62:0/2273850864 shutdown_connections 2024-09-11T23:57:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.517+0000 7f743292c640 1 --2- 172.21.15.62:0/2273850864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f742c100a50 0x7f742c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.517+0000 7f743292c640 1 --2- 172.21.15.62:0/2273850864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c100090 0x7f742c100510 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.517+0000 7f743292c640 1 --2- 172.21.15.62:0/2273850864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f742c106b70 0x7f742c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.517+0000 7f743292c640 1 -- 172.21.15.62:0/2273850864 >> 172.21.15.62:0/2273850864 conn(0x7f742c077210 msgr2=0x7f742c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.517+0000 7f743292c640 1 -- 172.21.15.62:0/2273850864 shutdown_connections 2024-09-11T23:57:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.517+0000 7f743292c640 1 -- 172.21.15.62:0/2273850864 wait complete. 2024-09-11T23:57:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.518+0000 7f743292c640 1 Processor -- start 2024-09-11T23:57:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.518+0000 7f743292c640 1 -- start start 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.518+0000 7f743292c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f742c100090 0x7f742c1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.518+0000 7f743292c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f742c100a50 0x7f742c1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.518+0000 7f743292c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c106b70 0x7f742c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f743292c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f742c1185d0 con 0x7f742c106b70 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f742bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f742c100090 0x7f742c1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f7430ec9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c106b70 0x7f742c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f7423fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f742c100a50 0x7f742c1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f7423fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f742c100a50 0x7f742c1a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:50708/0 (socket says 172.21.15.62:50708) 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f7423fff640 1 -- 172.21.15.62:0/440156916 learned_addr learned my addr 172.21.15.62:0/440156916 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f742bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f742c100090 0x7f742c1a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40258/0 (socket says 172.21.15.62:40258) 2024-09-11T23:57:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f743292c640 1 -- 172.21.15.62:0/440156916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f742c118450 con 0x7f742c100a50 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f743292c640 1 -- 172.21.15.62:0/440156916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f742c118750 con 0x7f742c100090 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f742bfff640 1 -- 172.21.15.62:0/440156916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f742c100a50 msgr2=0x7f742c1a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f742bfff640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f742c100a50 0x7f742c1a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f742bfff640 1 -- 172.21.15.62:0/440156916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c106b70 msgr2=0x7f742c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f742bfff640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c106b70 0x7f742c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f742bfff640 1 -- 172.21.15.62:0/440156916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c19ef90 con 0x7f742c100090 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f7430ec9640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c106b70 0x7f742c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.519+0000 7f742bfff640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f742c100090 0x7f742c1a46a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f741c00d6d0 tx=0x7f741c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.520+0000 7f7429ffb640 1 -- 172.21.15.62:0/440156916 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c017070 con 0x7f742c100090 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.520+0000 7f7429ffb640 1 -- 172.21.15.62:0/440156916 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f741c009820 con 0x7f742c100090 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.520+0000 7f7429ffb640 1 -- 172.21.15.62:0/440156916 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c004cd0 con 0x7f742c100090 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.520+0000 7f743292c640 1 -- 172.21.15.62:0/440156916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742c19f280 con 0x7f742c100090 2024-09-11T23:57:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.520+0000 7f743292c640 1 -- 172.21.15.62:0/440156916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f742c1ab570 con 0x7f742c100090 2024-09-11T23:57:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.524+0000 7f7429ffb640 1 -- 172.21.15.62:0/440156916 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f741c0099d0 con 0x7f742c100090 2024-09-11T23:57:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.526+0000 7f7429ffb640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74180a2d70 0x7f74180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.526+0000 7f7423fff640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74180a2d70 0x7f74180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.526+0000 7f7429ffb640 1 -- 172.21.15.62:0/440156916 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f741c0ceec0 con 0x7f742c100090 2024-09-11T23:57:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.526+0000 7f7434d05740 1 Processor -- start 2024-09-11T23:57:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f7434d05740 1 -- start start 2024-09-11T23:57:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f7423fff640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74180a2d70 0x7f74180a5230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f742c100720 tx=0x7f74140046c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f7434d05740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3a9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f7434d05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a9dc0 0x55d30d3b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f7434d05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f7434d05740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d30d30f2e0 con 0x55d30d3a94a0 2024-09-11T23:57:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f7434d05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d30d2b6ce0 con 0x55d30d3a9dc0 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f7434d05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d30d30b2a0 con 0x55d30d3b4680 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f742bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f742bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3a9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41706/0 (socket says 172.21.15.62:41706) 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f742bfff640 1 -- 172.21.15.62:0/582681713 learned_addr learned my addr 172.21.15.62:0/582681713 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f7423fff640 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a9dc0 0x55d30d3b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f7430ec9640 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f742bfff640 1 -- 172.21.15.62:0/582681713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 msgr2=0x55d30d3b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f742bfff640 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f742bfff640 1 -- 172.21.15.62:0/582681713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a9dc0 msgr2=0x55d30d3b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f742bfff640 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a9dc0 0x55d30d3b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.527+0000 7f742bfff640 1 -- 172.21.15.62:0/582681713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30d29b440 con 0x55d30d3a94a0 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.528+0000 7f7423fff640 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a9dc0 0x55d30d3b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:57:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.528+0000 7f7430ec9640 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:57:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.528+0000 7f742bfff640 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3a9880 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f741c097920 tx=0x7f741c09b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1c0c19da502f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.528+0000 7f74227fc640 1 -- 172.21.15.62:0/582681713 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c020040 con 0x55d30d3a94a0 2024-09-11T23:57:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.528+0000 7f74227fc640 1 -- 172.21.15.62:0/582681713 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f741c009d40 con 0x55d30d3a94a0 2024-09-11T23:57:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.528+0000 7f74227fc640 1 -- 172.21.15.62:0/582681713 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c09ac40 con 0x55d30d3a94a0 2024-09-11T23:57:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.529+0000 7f7434d05740 1 -- 172.21.15.62:0/582681713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 msgr2=0x55d30d3a9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.529+0000 7f7434d05740 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3a9880 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f741c097920 tx=0x7f741c09b600 comp rx=0 tx=0).stop 2024-09-11T23:57:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.529+0000 7f7434d05740 1 -- 172.21.15.62:0/582681713 shutdown_connections 2024-09-11T23:57:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.529+0000 7f7434d05740 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.529+0000 7f7434d05740 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a9dc0 0x55d30d3b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.529+0000 7f7434d05740 1 --2- 172.21.15.62:0/582681713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3a9880 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.529+0000 7f7434d05740 1 -- 172.21.15.62:0/582681713 >> 172.21.15.62:0/582681713 conn(0x55d30d3a5a30 msgr2=0x55d30d3a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.529+0000 7f7434d05740 1 -- 172.21.15.62:0/582681713 shutdown_connections 2024-09-11T23:57:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.529+0000 7f7434d05740 1 -- 172.21.15.62:0/582681713 wait complete. 2024-09-11T23:57:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.530+0000 7f7434d05740 1 -- start start 2024-09-11T23:57:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.530+0000 7f7434d05740 1 Processor -- start 2024-09-11T23:57:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.531+0000 7f7434d05740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.531+0000 7f7434d05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a9dc0 0x55d30d3b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.531+0000 7f7434d05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.531+0000 7f7434d05740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d30d3b7250 con 0x55d30d3a94a0 2024-09-11T23:57:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.531+0000 7f7434d05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d30d2ca050 con 0x55d30d3a9dc0 2024-09-11T23:57:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.531+0000 7f7434d05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d30d3b73d0 con 0x55d30d3b4680 2024-09-11T23:57:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.531+0000 7f7434d05740 10 client.0 _ll_register_callbacks cb 0x55d30d30a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.531+0000 7f742bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.531+0000 7f742bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41718/0 (socket says 172.21.15.62:41718) 2024-09-11T23:57:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.531+0000 7f742bfff640 1 -- 172.21.15.62:0/4246237110 learned_addr learned my addr 172.21.15.62:0/4246237110 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.532+0000 7f7430ec9640 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.532+0000 7f742bfff640 1 -- 172.21.15.62:0/4246237110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 msgr2=0x55d30d3b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.532+0000 7f742bfff640 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.532+0000 7f742bfff640 1 -- 172.21.15.62:0/4246237110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a9dc0 msgr2=0x55d30d3b1250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.532+0000 7f742bfff640 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a9dc0 0x55d30d3b1250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.532+0000 7f742bfff640 1 -- 172.21.15.62:0/4246237110 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30d0803c0 con 0x55d30d3a94a0 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.532+0000 7f7430ec9640 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.533+0000 7f742bfff640 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3b3fa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f741c009610 tx=0x7f741c097760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a6ef06c6957709 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.533+0000 7f740b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.533+0000 7f740b7fe640 1 -- 172.21.15.62:0/4246237110 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c0b8020 con 0x55d30d3a94a0 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.533+0000 7f7434d05740 1 -- 172.21.15.62:0/4246237110 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f741c009d40 con 0x55d30d3a94a0 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.533+0000 7f740b7fe640 1 -- 172.21.15.62:0/4246237110 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f741c09a980 con 0x55d30d3a94a0 2024-09-11T23:57:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.533+0000 7f740b7fe640 1 -- 172.21.15.62:0/4246237110 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f741c020070 con 0x55d30d3a94a0 2024-09-11T23:57:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.533+0000 7f7434d05740 10 client.16380 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.533+0000 7f7434d05740 1 -- 172.21.15.62:0/4246237110 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d30d3b8ed0 con 0x55d30d3a94a0 2024-09-11T23:57:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.534+0000 7f740b7fe640 1 -- 172.21.15.62:0/4246237110 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f741c009d40 con 0x55d30d3a94a0 2024-09-11T23:57:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.534+0000 7f740b7fe640 1 -- 172.21.15.62:0/4246237110 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f741c005220 con 0x55d30d3a94a0 2024-09-11T23:57:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.534+0000 7f7434d05740 10 client.16380 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.534+0000 7f7434d05740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:57:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.534+0000 7f7434d05740 20 client.16380 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.534+0000 7f7434d05740 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d30d3ba380 0x55d30d3bc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.534+0000 7f7434d05740 4 client.16380 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:57:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.534+0000 7f7434d05740 1 -- 172.21.15.62:0/4246237110 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d30d31c850 con 0x55d30d3ba380 2024-09-11T23:57:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.534+0000 7f7423fff640 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d30d3ba380 0x55d30d3bc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.535+0000 7f7423fff640 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d30d3ba380 0x55d30d3bc7b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=841d7ac9c65cedce server_cookie=6ba1e2e873ea475b in_seq=0 out_seq=0 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.535+0000 7f740b7fe640 10 client.16380 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.535+0000 7f740b7fe640 1 -- 172.21.15.62:0/4246237110 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d30d31c850 con 0x55d30d3ba380 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.535+0000 7f740b7fe640 10 client.16380 handle_command_reply: tid=0 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.535+0000 7f7434d05740 1 client.16380 shutdown 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.536+0000 7f7434d05740 1 -- 172.21.15.62:0/4246237110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 msgr2=0x55d30d3b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.536+0000 7f7434d05740 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3b3fa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f741c009610 tx=0x7f741c097760 comp rx=0 tx=0).stop 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.536+0000 7f7434d05740 1 -- 172.21.15.62:0/4246237110 shutdown_connections 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.536+0000 7f7434d05740 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d30d3ba380 0x55d30d3bc7b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.536+0000 7f7434d05740 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.536+0000 7f7434d05740 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a9dc0 0x55d30d3b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.536+0000 7f7434d05740 1 --2- 172.21.15.62:0/4246237110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3b3fa0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.536+0000 7f7434d05740 1 -- 172.21.15.62:0/4246237110 >> 172.21.15.62:0/4246237110 conn(0x55d30d3a5a30 msgr2=0x55d30d3ab760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.536+0000 7f740b7fe640 0 client.16380 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.537+0000 7f7434d05740 1 -- 172.21.15.62:0/4246237110 shutdown_connections 2024-09-11T23:57:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.537+0000 7f7434d05740 1 -- 172.21.15.62:0/4246237110 wait complete. 2024-09-11T23:57:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.537+0000 7f7434d05740 20 client.16380 trim_cache size 0 max 16384 2024-09-11T23:57:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7434d05740 1 Processor -- start 2024-09-11T23:57:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7434d05740 1 -- start start 2024-09-11T23:57:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7434d05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a94a0 0x55d30d3b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7434d05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3a9dc0 0x55d30d3c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7434d05740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3b4680 0x55d30d3c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7434d05740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f742c1adf40 con 0x55d30d3b4680 2024-09-11T23:57:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7434d05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f742c1ab810 con 0x55d30d3a94a0 2024-09-11T23:57:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7434d05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f742c105820 con 0x55d30d3a9dc0 2024-09-11T23:57:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7423fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3a9dc0 0x55d30d3c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7423fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3a9dc0 0x55d30d3c2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40278/0 (socket says 172.21.15.62:40278) 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f7423fff640 1 -- 172.21.15.62:0/2853618134 learned_addr learned my addr 172.21.15.62:0/2853618134 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.563+0000 7f742bfff640 1 --2- 172.21.15.62:0/2853618134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a94a0 0x55d30d3b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7430ec9640 1 --2- 172.21.15.62:0/2853618134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3b4680 0x55d30d3c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7423fff640 1 -- 172.21.15.62:0/2853618134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a94a0 msgr2=0x55d30d3b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7423fff640 1 --2- 172.21.15.62:0/2853618134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a94a0 0x55d30d3b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7423fff640 1 -- 172.21.15.62:0/2853618134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3b4680 msgr2=0x55d30d3c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7423fff640 1 --2- 172.21.15.62:0/2853618134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3b4680 0x55d30d3c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7423fff640 1 -- 172.21.15.62:0/2853618134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30d3af7a0 con 0x55d30d3a9dc0 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7430ec9640 1 --2- 172.21.15.62:0/2853618134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3b4680 0x55d30d3c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7423fff640 1 --2- 172.21.15.62:0/2853618134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3a9dc0 0x55d30d3c2090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f74140096c0 tx=0x7f7414040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb560831ad12c1ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7421ffb640 1 -- 172.21.15.62:0/2853618134 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414035040 con 0x55d30d3a9dc0 2024-09-11T23:57:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7421ffb640 1 -- 172.21.15.62:0/2853618134 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7414057070 con 0x55d30d3a9dc0 2024-09-11T23:57:30.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.564+0000 7f7421ffb640 1 -- 172.21.15.62:0/2853618134 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414003c30 con 0x55d30d3a9dc0 2024-09-11T23:57:30.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.565+0000 7f7434d05740 1 -- 172.21.15.62:0/2853618134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3a9dc0 msgr2=0x55d30d3c2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:30.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.565+0000 7f7434d05740 1 --2- 172.21.15.62:0/2853618134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3a9dc0 0x55d30d3c2090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f74140096c0 tx=0x7f7414040000 comp rx=0 tx=0).stop 2024-09-11T23:57:30.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.565+0000 7f7434d05740 1 -- 172.21.15.62:0/2853618134 shutdown_connections 2024-09-11T23:57:30.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.565+0000 7f7434d05740 1 --2- 172.21.15.62:0/2853618134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3b4680 0x55d30d3c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.565+0000 7f7434d05740 1 --2- 172.21.15.62:0/2853618134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3a9dc0 0x55d30d3c2090 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f74140096c0 tx=0x7f7414040000 comp rx=0 tx=0).stop 2024-09-11T23:57:30.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.565+0000 7f7434d05740 1 --2- 172.21.15.62:0/2853618134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d3a94a0 0x55d30d3b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.565+0000 7f7434d05740 1 -- 172.21.15.62:0/2853618134 >> 172.21.15.62:0/2853618134 conn(0x55d30d3a5a30 msgr2=0x55d30d3a4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:30.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.566+0000 7f7434d05740 1 -- 172.21.15.62:0/2853618134 shutdown_connections 2024-09-11T23:57:30.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.566+0000 7f7434d05740 1 -- 172.21.15.62:0/2853618134 wait complete. 2024-09-11T23:57:30.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.566+0000 7f7434d05740 1 -- start start 2024-09-11T23:57:30.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.567+0000 7f7434d05740 1 Processor -- start 2024-09-11T23:57:30.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.567+0000 7f7434d05740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.567+0000 7f7434d05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.567+0000 7f7434d05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d4479c0 0x55d30d447e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.567+0000 7f7434d05740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f742c0729b0 con 0x55d30d3a94a0 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.567+0000 7f7434d05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f742c073e20 con 0x55d30d4479c0 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.567+0000 7f7434d05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f742c073080 con 0x55d30d3b4680 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.567+0000 7f7434d05740 10 client.0 _ll_register_callbacks cb 0x7f7414040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.567+0000 7f7430ec9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d4479c0 0x55d30d447e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.568+0000 7f7423fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.568+0000 7f7430ec9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d4479c0 0x55d30d447e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:50764/0 (socket says 172.21.15.62:50764) 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.568+0000 7f7430ec9640 1 -- 172.21.15.62:0/2209819890 learned_addr learned my addr 172.21.15.62:0/2209819890 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.568+0000 7f742bfff640 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.568+0000 7f7423fff640 1 -- 172.21.15.62:0/2209819890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d4479c0 msgr2=0x55d30d447e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:30.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.568+0000 7f7423fff640 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d4479c0 0x55d30d447e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.568+0000 7f7423fff640 1 -- 172.21.15.62:0/2209819890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 msgr2=0x55d30d3c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.568+0000 7f7423fff640 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3c1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.568+0000 7f7423fff640 1 -- 172.21.15.62:0/2209819890 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30d448430 con 0x55d30d3b4680 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.568+0000 7f7423fff640 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3c2360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55d30d3d1f90 tx=0x7f7414003750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e07d4030fc1117ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.569+0000 7f740b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.569+0000 7f740b7fe640 1 -- 172.21.15.62:0/2209819890 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414035040 con 0x55d30d3b4680 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.569+0000 7f740b7fe640 1 -- 172.21.15.62:0/2209819890 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f7414003ab0 con 0x55d30d3b4680 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.569+0000 7f7434d05740 1 -- 172.21.15.62:0/2209819890 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7414057070 con 0x55d30d3b4680 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.569+0000 7f740b7fe640 1 -- 172.21.15.62:0/2209819890 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7414003cb0 con 0x55d30d3b4680 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.569+0000 7f7434d05740 10 client.25957 fetch_fsmap learned FSMap version 29 2024-09-11T23:57:30.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.569+0000 7f7434d05740 1 -- 172.21.15.62:0/2209819890 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d30d4508f0 con 0x55d30d3b4680 2024-09-11T23:57:30.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.570+0000 7f740b7fe640 1 -- 172.21.15.62:0/2209819890 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7414057070 con 0x55d30d3b4680 2024-09-11T23:57:30.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.570+0000 7f740b7fe640 1 -- 172.21.15.62:0/2209819890 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7414064750 con 0x55d30d3b4680 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.571+0000 7f7434d05740 10 client.25957 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.571+0000 7f7434d05740 10 client.25957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.571+0000 7f7434d05740 20 client.25957 populate_metadata read hostname 'smithi062' 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.571+0000 7f7434d05740 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d30d450f90 0x55d30d4533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.571+0000 7f7434d05740 4 client.25957 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.571+0000 7f7434d05740 1 -- 172.21.15.62:0/2209819890 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d30d316d60 con 0x55d30d450f90 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.571+0000 7f742bfff640 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d30d450f90 0x55d30d4533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.571+0000 7f742bfff640 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d30d450f90 0x55d30d4533b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f2fdbe7815f763f server_cookie=e8001a271d6d4462 in_seq=0 out_seq=0 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.571+0000 7f740b7fe640 10 client.25957 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f740b7fe640 1 -- 172.21.15.62:0/2209819890 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d30d316d60 con 0x55d30d450f90 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f740b7fe640 10 client.25957 handle_command_reply: tid=0 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 client.25957 shutdown 2024-09-11T23:57:30.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 -- 172.21.15.62:0/2209819890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 msgr2=0x55d30d3c2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3c2360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55d30d3d1f90 tx=0x7f7414003750 comp rx=0 tx=0).stop 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 -- 172.21.15.62:0/2209819890 shutdown_connections 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d30d450f90 0x55d30d4533b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30d4479c0 0x55d30d447e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30d3b4680 0x55d30d3c2360 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 --2- 172.21.15.62:0/2209819890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d30d3a94a0 0x55d30d3c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 -- 172.21.15.62:0/2209819890 >> 172.21.15.62:0/2209819890 conn(0x55d30d3a5a30 msgr2=0x55d30d3a4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f740b7fe640 0 client.25957 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 -- 172.21.15.62:0/2209819890 shutdown_connections 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 1 -- 172.21.15.62:0/2209819890 wait complete. 2024-09-11T23:57:30.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.572+0000 7f7434d05740 20 client.25957 trim_cache size 0 max 16384 2024-09-11T23:57:30.576 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:57:30.576 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:57:30.576 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:57:30.576 INFO:teuthology.orchestra.run.smithi062.stdout: "d0d53114-7511-4050-8682-4fed6cb8c4d7": { 2024-09-11T23:57:30.576 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:57:30.576 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "d0d53114-7511-4050-8682-4fed6cb8c4d7", 2024-09-11T23:57:30.576 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:57:30.576 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:57:30.576 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:57:30.576 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:57:30.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 -- 172.21.15.62:0/440156916 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74180a2d70 msgr2=0x7f74180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:30.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74180a2d70 0x7f74180a5230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f742c100720 tx=0x7f74140046c0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 -- 172.21.15.62:0/440156916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f742c100090 msgr2=0x7f742c1a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:57:30.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f742c100090 0x7f742c1a46a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f741c00d6d0 tx=0x7f741c00db90 comp rx=0 tx=0).stop 2024-09-11T23:57:30.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 -- 172.21.15.62:0/440156916 shutdown_connections 2024-09-11T23:57:30.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74180a2d70 0x7f74180a5230 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c106b70 0x7f742c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f742c100a50 0x7f742c1a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 --2- 172.21.15.62:0/440156916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f742c100090 0x7f742c1a46a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:57:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 -- 172.21.15.62:0/440156916 >> 172.21.15.62:0/440156916 conn(0x7f742c077210 msgr2=0x7f742c1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:57:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 -- 172.21.15.62:0/440156916 shutdown_connections 2024-09-11T23:57:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:57:30.573+0000 7f743292c640 1 -- 172.21.15.62:0/440156916 wait complete. 2024-09-11T23:57:30.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:30 smithi062 ceph-mon[30958]: pgmap v165: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 727 op/s 2024-09-11T23:57:30.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:30 smithi120 ceph-mon[35182]: pgmap v165: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 727 op/s 2024-09-11T23:57:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:30 smithi083 ceph-mon[34805]: pgmap v165: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 727 op/s 2024-09-11T23:57:31.403 INFO:tasks.cephfs.filesystem:scrub status for tag:d0d53114-7511-4050-8682-4fed6cb8c4d7 - {'path': '/', 'tag': 'd0d53114-7511-4050-8682-4fed6cb8c4d7', 'options': 'recursive,force'} 2024-09-11T23:57:32.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 290.0 s, 3299.4 tps, lat 30.351 ms stddev 35.975 2024-09-11T23:57:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:32 smithi062 ceph-mon[30958]: pgmap v166: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 752 op/s 2024-09-11T23:57:32.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:32 smithi120 ceph-mon[35182]: pgmap v166: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 752 op/s 2024-09-11T23:57:32.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:32 smithi083 ceph-mon[34805]: pgmap v166: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 752 op/s 2024-09-11T23:57:33.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:33 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:57:33.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:33 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:57:34.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:33 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:57:34.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:34 smithi083 ceph-mon[34805]: pgmap v167: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 41 MiB/s wr, 1.13k op/s 2024-09-11T23:57:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:34 smithi062 ceph-mon[30958]: pgmap v167: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 41 MiB/s wr, 1.13k op/s 2024-09-11T23:57:35.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:34 smithi120 ceph-mon[35182]: pgmap v167: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 41 MiB/s wr, 1.13k op/s 2024-09-11T23:57:36.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:36 smithi083 ceph-mon[34805]: pgmap v168: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 MiB/s wr, 845 op/s 2024-09-11T23:57:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:36 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:57:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:36 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:57:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:36 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:57:36.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:36 smithi062 ceph-mon[30958]: pgmap v168: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 MiB/s wr, 845 op/s 2024-09-11T23:57:36.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:36 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:57:36.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:36 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:57:36.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:36 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:57:37.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:36 smithi120 ceph-mon[35182]: pgmap v168: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 MiB/s wr, 845 op/s 2024-09-11T23:57:37.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:36 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:57:37.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:36 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:57:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:36 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:57:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 295.0 s, 3843.8 tps, lat 26.033 ms stddev 31.089 2024-09-11T23:57:38.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:38 smithi083 ceph-mon[34805]: pgmap v169: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 55 MiB/s wr, 1.19k op/s 2024-09-11T23:57:38.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:38 smithi062 ceph-mon[30958]: pgmap v169: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 55 MiB/s wr, 1.19k op/s 2024-09-11T23:57:39.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:38 smithi120 ceph-mon[35182]: pgmap v169: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 55 MiB/s wr, 1.19k op/s 2024-09-11T23:57:40.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:40 smithi083 ceph-mon[34805]: pgmap v170: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 755 op/s 2024-09-11T23:57:40.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:40 smithi062 ceph-mon[30958]: pgmap v170: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 755 op/s 2024-09-11T23:57:41.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:40 smithi120 ceph-mon[35182]: pgmap v170: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 755 op/s 2024-09-11T23:57:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 300.0 s, 3994.8 tps, lat 24.361 ms stddev 30.066 2024-09-11T23:57:42.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:42 smithi083 ceph-mon[34805]: pgmap v171: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 MiB/s wr, 769 op/s 2024-09-11T23:57:42.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:42 smithi062 ceph-mon[30958]: pgmap v171: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 MiB/s wr, 769 op/s 2024-09-11T23:57:43.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:42 smithi120 ceph-mon[35182]: pgmap v171: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 MiB/s wr, 769 op/s 2024-09-11T23:57:44.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:44 smithi083 ceph-mon[34805]: pgmap v172: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 48 MiB/s wr, 1.08k op/s 2024-09-11T23:57:44.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:44 smithi062 ceph-mon[30958]: pgmap v172: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 48 MiB/s wr, 1.08k op/s 2024-09-11T23:57:45.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:44 smithi120 ceph-mon[35182]: pgmap v172: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 48 MiB/s wr, 1.08k op/s 2024-09-11T23:57:46.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:46 smithi083 ceph-mon[34805]: pgmap v173: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 686 op/s 2024-09-11T23:57:46.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:46 smithi062 ceph-mon[30958]: pgmap v173: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 686 op/s 2024-09-11T23:57:47.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:46 smithi120 ceph-mon[35182]: pgmap v173: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 686 op/s 2024-09-11T23:57:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 305.0 s, 3928.8 tps, lat 26.125 ms stddev 57.132 2024-09-11T23:57:48.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:48 smithi083 ceph-mon[34805]: pgmap v174: 97 pgs: 97 active+clean; 3.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 66 MiB/s wr, 1.88k op/s 2024-09-11T23:57:48.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:48 smithi062 ceph-mon[30958]: pgmap v174: 97 pgs: 97 active+clean; 3.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 66 MiB/s wr, 1.88k op/s 2024-09-11T23:57:49.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:48 smithi120 ceph-mon[35182]: pgmap v174: 97 pgs: 97 active+clean; 3.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 66 MiB/s wr, 1.88k op/s 2024-09-11T23:57:50.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:50 smithi083 ceph-mon[34805]: pgmap v175: 97 pgs: 97 active+clean; 3.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 49 MiB/s wr, 1.54k op/s 2024-09-11T23:57:50.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:50 smithi062 ceph-mon[30958]: pgmap v175: 97 pgs: 97 active+clean; 3.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 49 MiB/s wr, 1.54k op/s 2024-09-11T23:57:51.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:50 smithi120 ceph-mon[35182]: pgmap v175: 97 pgs: 97 active+clean; 3.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 49 MiB/s wr, 1.54k op/s 2024-09-11T23:57:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 310.0 s, 3043.0 tps, lat 32.777 ms stddev 36.392 2024-09-11T23:57:52.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:52 smithi062 ceph-mon[30958]: pgmap v176: 97 pgs: 97 active+clean; 3.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 49 MiB/s wr, 1.56k op/s 2024-09-11T23:57:53.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:52 smithi120 ceph-mon[35182]: pgmap v176: 97 pgs: 97 active+clean; 3.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 49 MiB/s wr, 1.56k op/s 2024-09-11T23:57:53.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:52 smithi083 ceph-mon[34805]: pgmap v176: 97 pgs: 97 active+clean; 3.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 49 MiB/s wr, 1.56k op/s 2024-09-11T23:57:54.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:54 smithi062 ceph-mon[30958]: pgmap v177: 97 pgs: 97 active+clean; 3.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 56 MiB/s wr, 2.06k op/s 2024-09-11T23:57:55.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:54 smithi120 ceph-mon[35182]: pgmap v177: 97 pgs: 97 active+clean; 3.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 56 MiB/s wr, 2.06k op/s 2024-09-11T23:57:55.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:54 smithi083 ceph-mon[34805]: pgmap v177: 97 pgs: 97 active+clean; 3.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 56 MiB/s wr, 2.06k op/s 2024-09-11T23:57:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:56 smithi062 ceph-mon[30958]: pgmap v178: 97 pgs: 97 active+clean; 3.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 43 MiB/s wr, 1.72k op/s 2024-09-11T23:57:57.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:56 smithi120 ceph-mon[35182]: pgmap v178: 97 pgs: 97 active+clean; 3.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 43 MiB/s wr, 1.72k op/s 2024-09-11T23:57:57.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:56 smithi083 ceph-mon[34805]: pgmap v178: 97 pgs: 97 active+clean; 3.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 43 MiB/s wr, 1.72k op/s 2024-09-11T23:57:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 315.0 s, 3429.7 tps, lat 26.917 ms stddev 31.861 2024-09-11T23:57:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:57:58 smithi062 ceph-mon[30958]: pgmap v179: 97 pgs: 97 active+clean; 3.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 48 MiB/s wr, 2.05k op/s 2024-09-11T23:57:59.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:57:58 smithi120 ceph-mon[35182]: pgmap v179: 97 pgs: 97 active+clean; 3.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 48 MiB/s wr, 2.05k op/s 2024-09-11T23:57:59.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:57:58 smithi083 ceph-mon[34805]: pgmap v179: 97 pgs: 97 active+clean; 3.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 48 MiB/s wr, 2.05k op/s 2024-09-11T23:58:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:00 smithi062 ceph-mon[30958]: pgmap v180: 97 pgs: 97 active+clean; 3.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 853 op/s 2024-09-11T23:58:01.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:00 smithi120 ceph-mon[35182]: pgmap v180: 97 pgs: 97 active+clean; 3.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 853 op/s 2024-09-11T23:58:01.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:00 smithi083 ceph-mon[34805]: pgmap v180: 97 pgs: 97 active+clean; 3.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 853 op/s 2024-09-11T23:58:01.404 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:58:01.666 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:02.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 320.0 s, 174.8 tps, lat 605.937 ms stddev 1617.154 2024-09-11T23:58:02.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.449+0000 7f37e1abd640 1 -- 172.21.15.62:0/2619464676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37dc109450 msgr2=0x7f37dc110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:02.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.449+0000 7f37e1abd640 1 --2- 172.21.15.62:0/2619464676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37dc109450 0x7f37dc110250 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009860 tx=0x7f37cc02a360 comp rx=0 tx=0).stop 2024-09-11T23:58:02.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.449+0000 7f37e1abd640 1 -- 172.21.15.62:0/2619464676 shutdown_connections 2024-09-11T23:58:02.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.449+0000 7f37e1abd640 1 --2- 172.21.15.62:0/2619464676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37dc11b140 0x7f37dc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.449+0000 7f37e1abd640 1 --2- 172.21.15.62:0/2619464676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37dc109830 0x7f37dc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.449+0000 7f37e1abd640 1 --2- 172.21.15.62:0/2619464676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37dc109450 0x7f37dc110250 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.449+0000 7f37e1abd640 1 -- 172.21.15.62:0/2619464676 >> 172.21.15.62:0/2619464676 conn(0x7f37dc06e140 msgr2=0x7f37dc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:02.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.452+0000 7f37e1abd640 1 -- 172.21.15.62:0/2619464676 shutdown_connections 2024-09-11T23:58:02.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.452+0000 7f37e1abd640 1 -- 172.21.15.62:0/2619464676 wait complete. 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.452+0000 7f37e1abd640 1 Processor -- start 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.453+0000 7f37e1abd640 1 -- start start 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.453+0000 7f37e1abd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37dc109450 0x7f37dc1186d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.453+0000 7f37e1abd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37dc109830 0x7f37dc111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.453+0000 7f37e1abd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37dc11b140 0x7f37dc111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.453+0000 7f37e1abd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc120df0 con 0x7f37dc109450 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.453+0000 7f37e1abd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc120c70 con 0x7f37dc11b140 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.453+0000 7f37e1abd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc120f70 con 0x7f37dc109830 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.454+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37dc11b140 0x7f37dc111ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.454+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37dc11b140 0x7f37dc111ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57420/0 (socket says 172.21.15.62:57420) 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.454+0000 7f37dbfff640 1 -- 172.21.15.62:0/2096540356 learned_addr learned my addr 172.21.15.62:0/2096540356 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:02.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.454+0000 7f37daffd640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37dc109830 0x7f37dc111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.455+0000 7f37db7fe640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37dc109450 0x7f37dc1186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.455+0000 7f37daffd640 1 -- 172.21.15.62:0/2096540356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37dc11b140 msgr2=0x7f37dc111ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:02.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.455+0000 7f37daffd640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37dc11b140 0x7f37dc111ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.455+0000 7f37daffd640 1 -- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37dc109450 msgr2=0x7f37dc1186d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:02.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.455+0000 7f37daffd640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37dc109450 0x7f37dc1186d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.455+0000 7f37daffd640 1 -- 172.21.15.62:0/2096540356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37dc112260 con 0x7f37dc109830 2024-09-11T23:58:02.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.455+0000 7f37db7fe640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37dc109450 0x7f37dc1186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:58:02.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.459+0000 7f37daffd640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37dc109830 0x7f37dc111760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f37d000d6d0 tx=0x7f37d000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:02.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.467+0000 7f37d8ff9640 1 -- 172.21.15.62:0/2096540356 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0017070 con 0x7f37dc109830 2024-09-11T23:58:02.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.467+0000 7f37d8ff9640 1 -- 172.21.15.62:0/2096540356 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d0010460 con 0x7f37dc109830 2024-09-11T23:58:02.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.467+0000 7f37e1abd640 1 -- 172.21.15.62:0/2096540356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37dc112550 con 0x7f37dc109830 2024-09-11T23:58:02.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.467+0000 7f37e1abd640 1 -- 172.21.15.62:0/2096540356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37dc116520 con 0x7f37dc109830 2024-09-11T23:58:02.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.468+0000 7f37d8ff9640 1 -- 172.21.15.62:0/2096540356 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d000f620 con 0x7f37dc109830 2024-09-11T23:58:02.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.471+0000 7f37e3e96740 1 Processor -- start 2024-09-11T23:58:02.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.476+0000 7f37e3e96740 1 -- start start 2024-09-11T23:58:02.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.476+0000 7f37d8ff9640 1 -- 172.21.15.62:0/2096540356 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f37d001a070 con 0x7f37dc109830 2024-09-11T23:58:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.476+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72af6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.477+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72af6dc0 0x559d72b01140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.477+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72b01680 0x559d72b03a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.477+0000 7f37e3e96740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559d72a5c2e0 con 0x559d72af6dc0 2024-09-11T23:58:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.477+0000 7f37e3e96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559d72a03ce0 con 0x559d72b01680 2024-09-11T23:58:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.477+0000 7f37e3e96740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559d72a582a0 con 0x559d72af64a0 2024-09-11T23:58:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.477+0000 7f37d8ff9640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f37c40a2f00 0x7f37c40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.478+0000 7f37d8ff9640 1 -- 172.21.15.62:0/2096540356 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f37d00cecb0 con 0x7f37dc109830 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.478+0000 7f37db7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72af6dc0 0x559d72b01140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.478+0000 7f37db7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72af6dc0 0x559d72b01140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46452/0 (socket says 172.21.15.62:46452) 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.478+0000 7f37db7fe640 1 -- 172.21.15.62:0/1777200515 learned_addr learned my addr 172.21.15.62:0/1777200515 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.479+0000 7f37db7fe640 1 -- 172.21.15.62:0/1777200515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 msgr2=0x559d72af6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.479+0000 7f37db7fe640 1 --2- 172.21.15.62:0/1777200515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72af6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.479+0000 7f37db7fe640 1 -- 172.21.15.62:0/1777200515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72b01680 msgr2=0x559d72b03a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.479+0000 7f37db7fe640 1 --2- 172.21.15.62:0/1777200515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72b01680 0x559d72b03a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.479+0000 7f37db7fe640 1 -- 172.21.15.62:0/1777200515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d729e8440 con 0x559d72af6dc0 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.479+0000 7f37dbfff640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f37c40a2f00 0x7f37c40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.479+0000 7f37db7fe640 1 --2- 172.21.15.62:0/1777200515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72af6dc0 0x559d72b01140 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009780 tx=0x7f37cc030470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417506614e18c4c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.480+0000 7f37c1ffb640 1 -- 172.21.15.62:0/1777200515 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc037070 con 0x559d72af6dc0 2024-09-11T23:58:02.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.480+0000 7f37c1ffb640 1 -- 172.21.15.62:0/1777200515 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37cc038040 con 0x559d72af6dc0 2024-09-11T23:58:02.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.480+0000 7f37c1ffb640 1 -- 172.21.15.62:0/1777200515 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc037070 con 0x559d72af6dc0 2024-09-11T23:58:02.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.480+0000 7f37dbfff640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f37c40a2f00 0x7f37c40a53c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f37d4009270 tx=0x7f37d4009be0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:02.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.481+0000 7f37e3e96740 1 -- 172.21.15.62:0/1777200515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72af6dc0 msgr2=0x559d72b01140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:02.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.481+0000 7f37e3e96740 1 --2- 172.21.15.62:0/1777200515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72af6dc0 0x559d72b01140 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009780 tx=0x7f37cc030470 comp rx=0 tx=0).stop 2024-09-11T23:58:02.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.482+0000 7f37e3e96740 1 -- 172.21.15.62:0/1777200515 shutdown_connections 2024-09-11T23:58:02.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.482+0000 7f37e3e96740 1 --2- 172.21.15.62:0/1777200515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72b01680 0x559d72b03a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.482+0000 7f37e3e96740 1 --2- 172.21.15.62:0/1777200515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72af6dc0 0x559d72b01140 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.482+0000 7f37e3e96740 1 --2- 172.21.15.62:0/1777200515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72af6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.482+0000 7f37e3e96740 1 -- 172.21.15.62:0/1777200515 >> 172.21.15.62:0/1777200515 conn(0x559d72af2a30 msgr2=0x559d72af2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:02.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.482+0000 7f37e3e96740 1 -- 172.21.15.62:0/1777200515 shutdown_connections 2024-09-11T23:58:02.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.482+0000 7f37e3e96740 1 -- 172.21.15.62:0/1777200515 wait complete. 2024-09-11T23:58:02.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.482+0000 7f37e3e96740 1 -- start start 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.482+0000 7f37e3e96740 1 Processor -- start 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.483+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72b00f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.483+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 0x559d72afe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.483+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72afe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.483+0000 7f37e3e96740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559d72b04250 con 0x559d72b01680 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.483+0000 7f37e3e96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559d72a17050 con 0x559d72af6dc0 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.483+0000 7f37e3e96740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559d72b043d0 con 0x559d72af64a0 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.483+0000 7f37e3e96740 10 client.0 _ll_register_callbacks cb 0x559d72a57160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.483+0000 7f37daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 0x559d72afe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.483+0000 7f37daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 0x559d72afe1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57436/0 (socket says 172.21.15.62:57436) 2024-09-11T23:58:02.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.483+0000 7f37daffd640 1 -- 172.21.15.62:0/3167405276 learned_addr learned my addr 172.21.15.62:0/3167405276 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.484+0000 7f37db7fe640 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72b00f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.484+0000 7f37daffd640 1 -- 172.21.15.62:0/3167405276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 msgr2=0x559d72b00f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.484+0000 7f37daffd640 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72b00f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.484+0000 7f37daffd640 1 -- 172.21.15.62:0/3167405276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 msgr2=0x559d72afe6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.484+0000 7f37daffd640 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72afe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.484+0000 7f37daffd640 1 -- 172.21.15.62:0/3167405276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d727cd3c0 con 0x559d72af6dc0 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.484+0000 7f37daffd640 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 0x559d72afe1b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f37d000d6a0 tx=0x7f37d0097ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3257e9c98c4158 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.484+0000 7f37baffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.485+0000 7f37baffd640 1 -- 172.21.15.62:0/3167405276 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0021070 con 0x559d72af6dc0 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.485+0000 7f37e3e96740 1 -- 172.21.15.62:0/3167405276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37cc038040 con 0x559d72af6dc0 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.485+0000 7f37baffd640 1 -- 172.21.15.62:0/3167405276 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f37d00cc650 con 0x559d72af6dc0 2024-09-11T23:58:02.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.485+0000 7f37baffd640 1 -- 172.21.15.62:0/3167405276 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37d00cc850 con 0x559d72af6dc0 2024-09-11T23:58:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.489+0000 7f37e3e96740 10 client.26015 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.489+0000 7f37e3e96740 1 -- 172.21.15.62:0/3167405276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d72b0fc50 con 0x559d72af6dc0 2024-09-11T23:58:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.489+0000 7f37baffd640 1 -- 172.21.15.62:0/3167405276 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37cc038040 con 0x559d72af6dc0 2024-09-11T23:58:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.489+0000 7f37baffd640 1 -- 172.21.15.62:0/3167405276 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f37d009a5f0 con 0x559d72af6dc0 2024-09-11T23:58:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.490+0000 7f37e3e96740 10 client.26015 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.490+0000 7f37e3e96740 10 client.26015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.490+0000 7f37e3e96740 20 client.26015 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.490+0000 7f37e3e96740 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559d72b115f0 0x559d72b13a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.490+0000 7f37e3e96740 4 client.26015 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.490+0000 7f37e3e96740 1 -- 172.21.15.62:0/3167405276 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d72a69850 con 0x559d72b115f0 2024-09-11T23:58:02.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.490+0000 7f37db7fe640 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559d72b115f0 0x559d72b13a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.490+0000 7f37db7fe640 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559d72b115f0 0x559d72b13a10 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feb7a061182a1819 server_cookie=72e1c4a3bce4817e in_seq=0 out_seq=0 2024-09-11T23:58:02.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.491+0000 7f37baffd640 10 client.26015 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:02.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.491+0000 7f37baffd640 1 -- 172.21.15.62:0/3167405276 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d72a69850 con 0x559d72b115f0 2024-09-11T23:58:02.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.491+0000 7f37baffd640 10 client.26015 handle_command_reply: tid=0 2024-09-11T23:58:02.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 client.26015 shutdown 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 -- 172.21.15.62:0/3167405276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 msgr2=0x559d72afe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 0x559d72afe1b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f37d000d6a0 tx=0x7f37d0097ac0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 -- 172.21.15.62:0/3167405276 shutdown_connections 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559d72b115f0 0x559d72b13a10 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72afe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 0x559d72afe1b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 --2- 172.21.15.62:0/3167405276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72b00f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 -- 172.21.15.62:0/3167405276 >> 172.21.15.62:0/3167405276 conn(0x559d72af2a30 msgr2=0x559d72af8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37baffd640 0 client.26015 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 -- 172.21.15.62:0/3167405276 shutdown_connections 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.492+0000 7f37e3e96740 1 -- 172.21.15.62:0/3167405276 wait complete. 2024-09-11T23:58:02.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.493+0000 7f37e3e96740 20 client.26015 trim_cache size 0 max 16384 2024-09-11T23:58:02.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.526+0000 7f37e3e96740 1 Processor -- start 2024-09-11T23:58:02.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.526+0000 7f37e3e96740 1 -- start start 2024-09-11T23:58:02.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.527+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72b0d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.527+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 0x559d72af9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.527+0000 7f37db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72b0d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.527+0000 7f37db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72b0d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33254/0 (socket says 172.21.15.62:33254) 2024-09-11T23:58:02.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.527+0000 7f37db7fe640 1 -- 172.21.15.62:0/2253538348 learned_addr learned my addr 172.21.15.62:0/2253538348 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:02.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.527+0000 7f37e3e96740 1 --2- 172.21.15.62:0/2253538348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72b00120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.528+0000 7f37e3e96740 1 -- 172.21.15.62:0/2253538348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f37c80110d0 con 0x559d72b01680 2024-09-11T23:58:02.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.528+0000 7f37daffd640 1 --2- 172.21.15.62:0/2253538348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 0x559d72af9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.528+0000 7f37e3e96740 1 -- 172.21.15.62:0/2253538348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37c800e970 con 0x559d72af6dc0 2024-09-11T23:58:02.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.528+0000 7f37e3e96740 1 -- 172.21.15.62:0/2253538348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37c800c7c0 con 0x559d72af64a0 2024-09-11T23:58:02.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.528+0000 7f37dbfff640 1 --2- 172.21.15.62:0/2253538348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72b00120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.529+0000 7f37dbfff640 1 -- 172.21.15.62:0/2253538348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 msgr2=0x559d72b0d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:02.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.529+0000 7f37dbfff640 1 --2- 172.21.15.62:0/2253538348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72b0d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.529+0000 7f37dbfff640 1 -- 172.21.15.62:0/2253538348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 msgr2=0x559d72af9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:02.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.529+0000 7f37dbfff640 1 --2- 172.21.15.62:0/2253538348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 0x559d72af9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.529+0000 7f37dbfff640 1 -- 172.21.15.62:0/2253538348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d72af6bf0 con 0x559d72b01680 2024-09-11T23:58:02.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.529+0000 7f37dbfff640 1 --2- 172.21.15.62:0/2253538348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72b00120 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x559d72b01650 tx=0x7f37d4014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2d5b7424ea735a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.530+0000 7f37c17fa640 1 -- 172.21.15.62:0/2253538348 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d401f620 con 0x559d72b01680 2024-09-11T23:58:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.530+0000 7f37c17fa640 1 -- 172.21.15.62:0/2253538348 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d401fca0 con 0x559d72b01680 2024-09-11T23:58:02.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.530+0000 7f37c17fa640 1 -- 172.21.15.62:0/2253538348 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d401ddc0 con 0x559d72b01680 2024-09-11T23:58:02.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.531+0000 7f37e3e96740 1 -- 172.21.15.62:0/2253538348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 msgr2=0x559d72b00120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:02.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.531+0000 7f37e3e96740 1 --2- 172.21.15.62:0/2253538348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72b00120 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x559d72b01650 tx=0x7f37d4014070 comp rx=0 tx=0).stop 2024-09-11T23:58:02.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.531+0000 7f37e3e96740 1 -- 172.21.15.62:0/2253538348 shutdown_connections 2024-09-11T23:58:02.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.532+0000 7f37e3e96740 1 --2- 172.21.15.62:0/2253538348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72b00120 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.532+0000 7f37e3e96740 1 --2- 172.21.15.62:0/2253538348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af6dc0 0x559d72af9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.532+0000 7f37e3e96740 1 --2- 172.21.15.62:0/2253538348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af64a0 0x559d72b0d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.532+0000 7f37e3e96740 1 -- 172.21.15.62:0/2253538348 >> 172.21.15.62:0/2253538348 conn(0x559d72af2a30 msgr2=0x559d72affa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:02.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.532+0000 7f37e3e96740 1 -- 172.21.15.62:0/2253538348 shutdown_connections 2024-09-11T23:58:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.532+0000 7f37e3e96740 1 -- 172.21.15.62:0/2253538348 wait complete. 2024-09-11T23:58:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.533+0000 7f37e3e96740 1 -- start start 2024-09-11T23:58:02.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.534+0000 7f37e3e96740 1 Processor -- start 2024-09-11T23:58:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.534+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af64a0 0x559d72b09850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.534+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af6dc0 0x559d72b09d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.534+0000 7f37e3e96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72b04cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.535+0000 7f37db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af64a0 0x559d72b09850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.535+0000 7f37db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af64a0 0x559d72b09850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57462/0 (socket says 172.21.15.62:57462) 2024-09-11T23:58:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.535+0000 7f37daffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af6dc0 0x559d72b09d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.535+0000 7f37daffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af6dc0 0x559d72b09d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33268/0 (socket says 172.21.15.62:33268) 2024-09-11T23:58:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.535+0000 7f37e3e96740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f37c800ce30 con 0x559d72b01680 2024-09-11T23:58:02.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.535+0000 7f37db7fe640 1 -- 172.21.15.62:0/542426338 learned_addr learned my addr 172.21.15.62:0/542426338 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:02.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.535+0000 7f37e3e96740 1 -- 172.21.15.62:0/542426338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37c800e2a0 con 0x559d72af64a0 2024-09-11T23:58:02.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.535+0000 7f37e3e96740 1 -- 172.21.15.62:0/542426338 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37c800d500 con 0x559d72af6dc0 2024-09-11T23:58:02.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.535+0000 7f37e3e96740 10 client.0 _ll_register_callbacks cb 0x7f37d0005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:02.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.536+0000 7f37dbfff640 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72b04cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.536+0000 7f37db7fe640 1 -- 172.21.15.62:0/542426338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af6dc0 msgr2=0x559d72b09d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:02.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.536+0000 7f37db7fe640 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af6dc0 0x559d72b09d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.536+0000 7f37db7fe640 1 -- 172.21.15.62:0/542426338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 msgr2=0x559d72b04cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:02.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.536+0000 7f37db7fe640 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72b04cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.536+0000 7f37db7fe640 1 -- 172.21.15.62:0/542426338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d72b054d0 con 0x559d72af64a0 2024-09-11T23:58:02.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.536+0000 7f37dbfff640 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72b04cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:58:02.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.536+0000 7f37daffd640 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af6dc0 0x559d72b09d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:02.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.536+0000 7f37db7fe640 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af64a0 0x559d72b09850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0095e0 tx=0x7f37cc004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65cda7c52a4dfe9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:02.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.536+0000 7f37baffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:58:02.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.537+0000 7f37baffd640 1 -- 172.21.15.62:0/542426338 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc037070 con 0x559d72af64a0 2024-09-11T23:58:02.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.537+0000 7f37baffd640 1 -- 172.21.15.62:0/542426338 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f37cc04c070 con 0x559d72af64a0 2024-09-11T23:58:02.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.538+0000 7f37e3e96740 1 -- 172.21.15.62:0/542426338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d401fca0 con 0x559d72af64a0 2024-09-11T23:58:02.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.538+0000 7f37baffd640 1 -- 172.21.15.62:0/542426338 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37cc005040 con 0x559d72af64a0 2024-09-11T23:58:02.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.544+0000 7f37e3e96740 10 client.26021 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:02.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.544+0000 7f37e3e96740 1 -- 172.21.15.62:0/542426338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d72b7be10 con 0x559d72af64a0 2024-09-11T23:58:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.546+0000 7f37baffd640 1 -- 172.21.15.62:0/542426338 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d401fca0 con 0x559d72af64a0 2024-09-11T23:58:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.546+0000 7f37baffd640 1 -- 172.21.15.62:0/542426338 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f37cc0046c0 con 0x559d72af64a0 2024-09-11T23:58:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.546+0000 7f37e3e96740 10 client.26021 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.546+0000 7f37e3e96740 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.546+0000 7f37e3e96740 20 client.26021 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.547+0000 7f37e3e96740 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559d72b7c0a0 0x559d72b7e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.547+0000 7f37e3e96740 4 client.26021 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:58:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.547+0000 7f37e3e96740 1 -- 172.21.15.62:0/542426338 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d72a63d60 con 0x559d72b7c0a0 2024-09-11T23:58:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.547+0000 7f37daffd640 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559d72b7c0a0 0x559d72b7e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.548+0000 7f37baffd640 10 client.26021 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.548+0000 7f37daffd640 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559d72b7c0a0 0x559d72b7e490 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750860851e0f84 server_cookie=eb533c821e09c719 in_seq=0 out_seq=0 2024-09-11T23:58:02.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.548+0000 7f37baffd640 1 -- 172.21.15.62:0/542426338 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d72a63d60 con 0x559d72b7c0a0 2024-09-11T23:58:02.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.548+0000 7f37baffd640 10 client.26021 handle_command_reply: tid=0 2024-09-11T23:58:02.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.548+0000 7f37e3e96740 1 client.26021 shutdown 2024-09-11T23:58:02.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.549+0000 7f37e3e96740 1 -- 172.21.15.62:0/542426338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af64a0 msgr2=0x559d72b09850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:02.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.549+0000 7f37e3e96740 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af64a0 0x559d72b09850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0095e0 tx=0x7f37cc004340 comp rx=0 tx=0).stop 2024-09-11T23:58:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.550+0000 7f37e3e96740 1 -- 172.21.15.62:0/542426338 shutdown_connections 2024-09-11T23:58:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.550+0000 7f37e3e96740 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559d72b7c0a0 0x559d72b7e490 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.550+0000 7f37e3e96740 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d72b01680 0x559d72b04cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.550+0000 7f37e3e96740 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559d72af6dc0 0x559d72b09d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.550+0000 7f37e3e96740 1 --2- 172.21.15.62:0/542426338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d72af64a0 0x559d72b09850 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.550+0000 7f37baffd640 0 client.26021 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:02.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.550+0000 7f37e3e96740 1 -- 172.21.15.62:0/542426338 >> 172.21.15.62:0/542426338 conn(0x559d72af2a30 msgr2=0x559d72af8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:02.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.550+0000 7f37e3e96740 1 -- 172.21.15.62:0/542426338 shutdown_connections 2024-09-11T23:58:02.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.550+0000 7f37e3e96740 1 -- 172.21.15.62:0/542426338 wait complete. 2024-09-11T23:58:02.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.550+0000 7f37e3e96740 20 client.26021 trim_cache size 0 max 16384 2024-09-11T23:58:02.551 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:58:02.551 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-11T23:58:02.551 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-11T23:58:02.551 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:58:02.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.555+0000 7f37e1abd640 1 -- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f37c40a2f00 msgr2=0x7f37c40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:02.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.555+0000 7f37e1abd640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f37c40a2f00 0x7f37c40a53c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f37d4009270 tx=0x7f37d4009be0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.555+0000 7f37e1abd640 1 -- 172.21.15.62:0/2096540356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37dc109830 msgr2=0x7f37dc111760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:02.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.555+0000 7f37e1abd640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37dc109830 0x7f37dc111760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f37d000d6d0 tx=0x7f37d000db90 comp rx=0 tx=0).stop 2024-09-11T23:58:02.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.556+0000 7f37e1abd640 1 -- 172.21.15.62:0/2096540356 shutdown_connections 2024-09-11T23:58:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.556+0000 7f37e1abd640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f37c40a2f00 0x7f37c40a53c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.556+0000 7f37e1abd640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37dc11b140 0x7f37dc111ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.556+0000 7f37e1abd640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37dc109830 0x7f37dc111760 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.556+0000 7f37e1abd640 1 --2- 172.21.15.62:0/2096540356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37dc109450 0x7f37dc1186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.556+0000 7f37e1abd640 1 -- 172.21.15.62:0/2096540356 >> 172.21.15.62:0/2096540356 conn(0x7f37dc06e140 msgr2=0x7f37dc11bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.557+0000 7f37e1abd640 1 -- 172.21.15.62:0/2096540356 shutdown_connections 2024-09-11T23:58:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:02.557+0000 7f37e1abd640 1 -- 172.21.15.62:0/2096540356 wait complete. 2024-09-11T23:58:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:02 smithi062 ceph-mon[30958]: pgmap v181: 97 pgs: 97 active+clean; 3.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 919 op/s 2024-09-11T23:58:03.129 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:02 smithi120 ceph-mon[35182]: pgmap v181: 97 pgs: 97 active+clean; 3.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 919 op/s 2024-09-11T23:58:03.129 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:02 smithi083 ceph-mon[34805]: pgmap v181: 97 pgs: 97 active+clean; 3.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 919 op/s 2024-09-11T23:58:03.748 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:58:03.748 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:58:04.154 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:04.192 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-11T23:58:04.192 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:58:04.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.723+0000 7fb38f8bb640 1 -- 172.21.15.62:0/1859600021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb388100ca0 msgr2=0x7fb388106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:04.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.723+0000 7fb38f8bb640 1 --2- 172.21.15.62:0/1859600021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb388100ca0 0x7fb388106840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb384009900 tx=0x7fb38402a410 comp rx=0 tx=0).stop 2024-09-11T23:58:04.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.723+0000 7fb38f8bb640 1 -- 172.21.15.62:0/1859600021 shutdown_connections 2024-09-11T23:58:04.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.723+0000 7fb38f8bb640 1 --2- 172.21.15.62:0/1859600021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb388106f50 0x7fb38810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:04.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.723+0000 7fb38f8bb640 1 --2- 172.21.15.62:0/1859600021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb388100ca0 0x7fb388106840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:04.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.723+0000 7fb38f8bb640 1 --2- 172.21.15.62:0/1859600021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3881002f0 0x7fb3881006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:04.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.723+0000 7fb38f8bb640 1 -- 172.21.15.62:0/1859600021 >> 172.21.15.62:0/1859600021 conn(0x7fb388077210 msgr2=0x7fb3880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:04.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.723+0000 7fb38f8bb640 1 -- 172.21.15.62:0/1859600021 shutdown_connections 2024-09-11T23:58:04.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.723+0000 7fb38f8bb640 1 -- 172.21.15.62:0/1859600021 wait complete. 2024-09-11T23:58:04.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.724+0000 7fb38f8bb640 1 Processor -- start 2024-09-11T23:58:04.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.724+0000 7fb38f8bb640 1 -- start start 2024-09-11T23:58:04.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.725+0000 7fb38f8bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3881002f0 0x7fb388116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:04.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.725+0000 7fb38f8bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb388100ca0 0x7fb3881167d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:04.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.725+0000 7fb38f8bb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388106f50 0x7fb38811bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:04.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.725+0000 7fb38d657640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3881002f0 0x7fb388116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:04.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.725+0000 7fb38ce56640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb388100ca0 0x7fb3881167d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:04.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.725+0000 7fb38d657640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3881002f0 0x7fb388116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57484/0 (socket says 172.21.15.62:57484) 2024-09-11T23:58:04.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.725+0000 7fb38d657640 1 -- 172.21.15.62:0/41537186 learned_addr learned my addr 172.21.15.62:0/41537186 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:04.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.725+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881031c0 con 0x7fb388106f50 2024-09-11T23:58:04.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.725+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb388103040 con 0x7fb3881002f0 2024-09-11T23:58:04.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.725+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb38810ff90 con 0x7fb388100ca0 2024-09-11T23:58:04.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.726+0000 7fb38de58640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388106f50 0x7fb38811bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:04.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.726+0000 7fb38d657640 1 -- 172.21.15.62:0/41537186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb388100ca0 msgr2=0x7fb3881167d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:04.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.726+0000 7fb38d657640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb388100ca0 0x7fb3881167d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:04.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.726+0000 7fb38d657640 1 -- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388106f50 msgr2=0x7fb38811bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:04.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.726+0000 7fb38d657640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388106f50 0x7fb38811bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:04.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.726+0000 7fb38d657640 1 -- 172.21.15.62:0/41537186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38811c870 con 0x7fb3881002f0 2024-09-11T23:58:04.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.726+0000 7fb38de58640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388106f50 0x7fb38811bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:04.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.726+0000 7fb38d657640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3881002f0 0x7fb388116290 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb37800e9f0 tx=0x7fb37800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:04.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.727+0000 7fb37e7fc640 1 -- 172.21.15.62:0/41537186 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37800cd80 con 0x7fb3881002f0 2024-09-11T23:58:04.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.727+0000 7fb37e7fc640 1 -- 172.21.15.62:0/41537186 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb378010910 con 0x7fb3881002f0 2024-09-11T23:58:04.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.727+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3881113b0 con 0x7fb3881002f0 2024-09-11T23:58:04.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.727+0000 7fb37e7fc640 1 -- 172.21.15.62:0/41537186 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378018990 con 0x7fb3881002f0 2024-09-11T23:58:04.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.727+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb388111870 con 0x7fb3881002f0 2024-09-11T23:58:04.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.728+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3881033b0 con 0x7fb3881002f0 2024-09-11T23:58:04.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.730+0000 7fb37e7fc640 1 -- 172.21.15.62:0/41537186 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb378018b30 con 0x7fb3881002f0 2024-09-11T23:58:04.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.731+0000 7fb37e7fc640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb3740a2e40 0x7fb3740a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:04.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.732+0000 7fb37e7fc640 1 -- 172.21.15.62:0/41537186 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fb3780cf800 con 0x7fb3881002f0 2024-09-11T23:58:04.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:04 smithi062 ceph-mon[30958]: pgmap v182: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 31 MiB/s wr, 2.14k op/s 2024-09-11T23:58:04.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.733+0000 7fb38ce56640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb3740a2e40 0x7fb3740a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:04.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.734+0000 7fb38ce56640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb3740a2e40 0x7fb3740a5300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb38402a980 tx=0x7fb38402a910 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:04.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:04.735+0000 7fb37e7fc640 1 -- 172.21.15.62:0/41537186 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb37809c3f0 con 0x7fb3881002f0 2024-09-11T23:58:04.792 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:05.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.044+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3881128e0 con 0x7fb3881002f0 2024-09-11T23:58:05.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.045+0000 7fb37e7fc640 1 -- 172.21.15.62:0/41537186 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11910 (secure 0 0 0) 0x7fb37809bcf0 con 0x7fb3881002f0 2024-09-11T23:58:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:58:05.045 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-09-11T23:57:32:617369+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:57:32.075842+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:58:05.045 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-09-11T23:58:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb3740a2e40 msgr2=0x7fb3740a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb3740a2e40 0x7fb3740a5300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb38402a980 tx=0x7fb38402a910 comp rx=0 tx=0).stop 2024-09-11T23:58:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3881002f0 msgr2=0x7fb388116290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3881002f0 0x7fb388116290 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb37800e9f0 tx=0x7fb37800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:58:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 shutdown_connections 2024-09-11T23:58:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb3740a2e40 0x7fb3740a5300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388106f50 0x7fb38811bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb388100ca0 0x7fb3881167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 --2- 172.21.15.62:0/41537186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3881002f0 0x7fb388116290 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 >> 172.21.15.62:0/41537186 conn(0x7fb388077210 msgr2=0x7fb388105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 shutdown_connections 2024-09-11T23:58:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:05.048+0000 7fb38f8bb640 1 -- 172.21.15.62:0/41537186 wait complete. 2024-09-11T23:58:05.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:04 smithi120 ceph-mon[35182]: pgmap v182: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 31 MiB/s wr, 2.14k op/s 2024-09-11T23:58:05.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:04 smithi083 ceph-mon[34805]: pgmap v182: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 31 MiB/s wr, 2.14k op/s 2024-09-11T23:58:05.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:05 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/41537186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:06.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:05 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/41537186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:06.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:05 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/41537186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:06.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.241+0000 7f4360a82640 1 -- 172.21.15.62:0/3830186922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c074eb0 msgr2=0x7f435c112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:06.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.241+0000 7f4360a82640 1 --2- 172.21.15.62:0/3830186922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c074eb0 0x7f435c112640 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f434c009860 tx=0x7f434c02a3d0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.241+0000 7f4360a82640 1 -- 172.21.15.62:0/3830186922 shutdown_connections 2024-09-11T23:58:06.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.241+0000 7f4360a82640 1 --2- 172.21.15.62:0/3830186922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c112b80 0x7f435c114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.241+0000 7f4360a82640 1 --2- 172.21.15.62:0/3830186922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c074eb0 0x7f435c112640 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.241+0000 7f4360a82640 1 --2- 172.21.15.62:0/3830186922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f435c074590 0x7f435c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.241+0000 7f4360a82640 1 -- 172.21.15.62:0/3830186922 >> 172.21.15.62:0/3830186922 conn(0x7f435c0fd8b0 msgr2=0x7f435c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:06.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.242+0000 7f4360a82640 1 -- 172.21.15.62:0/3830186922 shutdown_connections 2024-09-11T23:58:06.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.242+0000 7f4360a82640 1 -- 172.21.15.62:0/3830186922 wait complete. 2024-09-11T23:58:06.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f4360a82640 1 Processor -- start 2024-09-11T23:58:06.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f4360a82640 1 -- start start 2024-09-11T23:58:06.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f4360a82640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c074590 0x7f435c1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:06.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f4360a82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f435c074eb0 0x7f435c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:06.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f4360a82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c112b80 0x7f435c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:06.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f4360a82640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f435c118550 con 0x7f435c074590 2024-09-11T23:58:06.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f4360a82640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1183d0 con 0x7f435c112b80 2024-09-11T23:58:06.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f4360a82640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1186d0 con 0x7f435c074eb0 2024-09-11T23:58:06.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f435a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c074590 0x7f435c1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:06.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f435a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c074590 0x7f435c1a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46524/0 (socket says 172.21.15.62:46524) 2024-09-11T23:58:06.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f435a59c640 1 -- 172.21.15.62:0/3611818127 learned_addr learned my addr 172.21.15.62:0/3611818127 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:06.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f435ad9d640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c112b80 0x7f435c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:06.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.243+0000 7f4359d9b640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f435c074eb0 0x7f435c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:06.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f435a59c640 1 -- 172.21.15.62:0/3611818127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f435c074eb0 msgr2=0x7f435c1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:06.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f435a59c640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f435c074eb0 0x7f435c1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f435a59c640 1 -- 172.21.15.62:0/3611818127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c112b80 msgr2=0x7f435c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:06.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f435a59c640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c112b80 0x7f435c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f435a59c640 1 -- 172.21.15.62:0/3611818127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c19eff0 con 0x7f435c074590 2024-09-11T23:58:06.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f435ad9d640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c112b80 0x7f435c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:58:06.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f435a59c640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c074590 0x7f435c1a4700 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f435400eb90 tx=0x7f435400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:06.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f434b7fe640 1 -- 172.21.15.62:0/3611818127 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354004b20 con 0x7f435c074590 2024-09-11T23:58:06.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f4360a82640 1 -- 172.21.15.62:0/3611818127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c19f2e0 con 0x7f435c074590 2024-09-11T23:58:06.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f434b7fe640 1 -- 172.21.15.62:0/3611818127 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4354004cc0 con 0x7f435c074590 2024-09-11T23:58:06.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f4360a82640 1 -- 172.21.15.62:0/3611818127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c1ab5d0 con 0x7f435c074590 2024-09-11T23:58:06.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.244+0000 7f434b7fe640 1 -- 172.21.15.62:0/3611818127 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354004b20 con 0x7f435c074590 2024-09-11T23:58:06.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.246+0000 7f4360a82640 1 -- 172.21.15.62:0/3611818127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f435c075cd0 con 0x7f435c074590 2024-09-11T23:58:06.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.250+0000 7f434b7fe640 1 -- 172.21.15.62:0/3611818127 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4354005020 con 0x7f435c074590 2024-09-11T23:58:06.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.251+0000 7f434b7fe640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43440a30f0 0x7f43440a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:06.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.252+0000 7f4359d9b640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43440a30f0 0x7f43440a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:06.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.252+0000 7f434b7fe640 1 -- 172.21.15.62:0/3611818127 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f435400cca0 con 0x7f435c074590 2024-09-11T23:58:06.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.252+0000 7f4359d9b640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43440a30f0 0x7f43440a55b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f435c1a00b0 tx=0x7f434c0094d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:06.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.258+0000 7f434b7fe640 1 -- 172.21.15.62:0/3611818127 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43540541b0 con 0x7f435c074590 2024-09-11T23:58:06.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.609+0000 7f4360a82640 1 -- 172.21.15.62:0/3611818127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f435c104fc0 con 0x7f435c074590 2024-09-11T23:58:06.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.612+0000 7f434b7fe640 1 -- 172.21.15.62:0/3611818127 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11910 (secure 0 0 0) 0x7f435409c340 con 0x7f435c074590 2024-09-11T23:58:06.612 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:58:06.612 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-09-11T23:57:32:617369+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:57:32.075842+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:58:06.613 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-09-11T23:58:06.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 -- 172.21.15.62:0/3611818127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43440a30f0 msgr2=0x7f43440a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:06.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43440a30f0 0x7f43440a55b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f435c1a00b0 tx=0x7f434c0094d0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 -- 172.21.15.62:0/3611818127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c074590 msgr2=0x7f435c1a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:06.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c074590 0x7f435c1a4700 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f435400eb90 tx=0x7f435400ef90 comp rx=0 tx=0).stop 2024-09-11T23:58:06.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 -- 172.21.15.62:0/3611818127 shutdown_connections 2024-09-11T23:58:06.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43440a30f0 0x7f43440a55b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c112b80 0x7f435c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f435c074eb0 0x7f435c1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 --2- 172.21.15.62:0/3611818127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c074590 0x7f435c1a4700 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:06.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 -- 172.21.15.62:0/3611818127 >> 172.21.15.62:0/3611818127 conn(0x7f435c0fd8b0 msgr2=0x7f435c0ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:06.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.615+0000 7f4360a82640 1 -- 172.21.15.62:0/3611818127 shutdown_connections 2024-09-11T23:58:06.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:06.616+0000 7f4360a82640 1 -- 172.21.15.62:0/3611818127 wait complete. 2024-09-11T23:58:06.658 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:58:06.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:06 smithi062 ceph-mon[30958]: pgmap v183: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 24 MiB/s wr, 1.63k op/s 2024-09-11T23:58:06.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:06 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3611818127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:07.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:06 smithi120 ceph-mon[35182]: pgmap v183: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 24 MiB/s wr, 1.63k op/s 2024-09-11T23:58:07.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:06 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3611818127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:07.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:06 smithi083 ceph-mon[34805]: pgmap v183: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 24 MiB/s wr, 1.63k op/s 2024-09-11T23:58:07.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:06 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3611818127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:07.343 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-11T23:58:07.344 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-11T23:58:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 325.0 s, 2568.2 tps, lat 39.689 ms stddev 68.783 2024-09-11T23:58:07.503 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:08.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.065+0000 7f44c7bc6640 1 -- 172.21.15.62:0/1579531898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c0100a80 msgr2=0x7f44c0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.065+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/1579531898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c0100a80 0x7f44c0100ee0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009930 tx=0x7f44b002a420 comp rx=0 tx=0).stop 2024-09-11T23:58:08.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.065+0000 7f44c7bc6640 1 -- 172.21.15.62:0/1579531898 shutdown_connections 2024-09-11T23:58:08.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.065+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/1579531898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c0101420 0x7f44c010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.065+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/1579531898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c0100a80 0x7f44c0100ee0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.065+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/1579531898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c0106a80 0x7f44c0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.065+0000 7f44c7bc6640 1 -- 172.21.15.62:0/1579531898 >> 172.21.15.62:0/1579531898 conn(0x7f44c00fc7a0 msgr2=0x7f44c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:08.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.065+0000 7f44c7bc6640 1 -- 172.21.15.62:0/1579531898 shutdown_connections 2024-09-11T23:58:08.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.065+0000 7f44c7bc6640 1 -- 172.21.15.62:0/1579531898 wait complete. 2024-09-11T23:58:08.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.066+0000 7f44c7bc6640 1 Processor -- start 2024-09-11T23:58:08.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.066+0000 7f44c7bc6640 1 -- start start 2024-09-11T23:58:08.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.066+0000 7f44c7bc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c0100a80 0x7f44c019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.066+0000 7f44c7bc6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c0101420 0x7f44c019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.067+0000 7f44c5962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c0100a80 0x7f44c019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.067+0000 7f44c5962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c0100a80 0x7f44c019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33318/0 (socket says 172.21.15.62:33318) 2024-09-11T23:58:08.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.067+0000 7f44c5962640 1 -- 172.21.15.62:0/3963953341 learned_addr learned my addr 172.21.15.62:0/3963953341 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:08.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.067+0000 7f44c5161640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c0101420 0x7f44c019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.067+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c0106a80 0x7f44c019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.067+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0111f90 con 0x7f44c0101420 2024-09-11T23:58:08.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.067+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0111e10 con 0x7f44c0106a80 2024-09-11T23:58:08.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.067+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0112110 con 0x7f44c0100a80 2024-09-11T23:58:08.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.069+0000 7f44c6163640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c0106a80 0x7f44c019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.069+0000 7f44c5962640 1 -- 172.21.15.62:0/3963953341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c0106a80 msgr2=0x7f44c019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.069+0000 7f44c5962640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c0106a80 0x7f44c019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.069+0000 7f44c5962640 1 -- 172.21.15.62:0/3963953341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c0101420 msgr2=0x7f44c019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.069+0000 7f44c5962640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c0101420 0x7f44c019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.069+0000 7f44c5962640 1 -- 172.21.15.62:0/3963953341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c019f640 con 0x7f44c0100a80 2024-09-11T23:58:08.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.070+0000 7f44c5962640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c0100a80 0x7f44c019a690 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e9f0 tx=0x7f44bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.070+0000 7f44b6ffd640 1 -- 172.21.15.62:0/3963953341 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00cd80 con 0x7f44c0100a80 2024-09-11T23:58:08.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.070+0000 7f44b6ffd640 1 -- 172.21.15.62:0/3963953341 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44bc010910 con 0x7f44c0100a80 2024-09-11T23:58:08.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.070+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c019f930 con 0x7f44c0100a80 2024-09-11T23:58:08.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.071+0000 7f44b6ffd640 1 -- 172.21.15.62:0/3963953341 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc018990 con 0x7f44c0100a80 2024-09-11T23:58:08.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.071+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c019fd80 con 0x7f44c0100a80 2024-09-11T23:58:08.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.073+0000 7f44b6ffd640 1 -- 172.21.15.62:0/3963953341 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f44bc010ac0 con 0x7f44c0100a80 2024-09-11T23:58:08.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.074+0000 7f44b6ffd640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f44ac0a2e40 0x7f44ac0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.074+0000 7f44c9f9f740 1 Processor -- start 2024-09-11T23:58:08.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.074+0000 7f44b6ffd640 1 -- 172.21.15.62:0/3963953341 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f44bc0cf6a0 con 0x7f44c0100a80 2024-09-11T23:58:08.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.074+0000 7f44c5161640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f44ac0a2e40 0x7f44ac0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.074+0000 7f44c9f9f740 1 -- start start 2024-09-11T23:58:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceaf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceafa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580acebc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5580ace152f0 con 0x5580aceaf5d0 2024-09-11T23:58:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5580acdbccf0 con 0x5580aceba370 2024-09-11T23:58:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5580ace112b0 con 0x5580aceb0b40 2024-09-11T23:58:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceaf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5161640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f44ac0a2e40 0x7f44ac0a5300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f44b0006010 tx=0x7f44b0000bc0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceaf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33332/0 (socket says 172.21.15.62:33332) 2024-09-11T23:58:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5962640 1 -- 172.21.15.62:0/1758350226 learned_addr learned my addr 172.21.15.62:0/1758350226 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5161640 1 --2- 172.21.15.62:0/1758350226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceafa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c6163640 1 --2- 172.21.15.62:0/1758350226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580acebc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5962640 1 -- 172.21.15.62:0/1758350226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 msgr2=0x5580acebc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5962640 1 --2- 172.21.15.62:0/1758350226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580acebc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5962640 1 -- 172.21.15.62:0/1758350226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 msgr2=0x5580aceafa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5962640 1 --2- 172.21.15.62:0/1758350226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceafa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5962640 1 -- 172.21.15.62:0/1758350226 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580acda13a0 con 0x5580aceb0b40 2024-09-11T23:58:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.075+0000 7f44c5962640 1 --2- 172.21.15.62:0/1758350226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceaf090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0985c0 tx=0x7f44bc09b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d6223be56aa8903 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.076+0000 7f44abfff640 1 -- 172.21.15.62:0/1758350226 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0a4860 con 0x5580aceb0b40 2024-09-11T23:58:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.076+0000 7f44abfff640 1 -- 172.21.15.62:0/1758350226 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44bc0cd0a0 con 0x5580aceb0b40 2024-09-11T23:58:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.076+0000 7f44abfff640 1 -- 172.21.15.62:0/1758350226 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc097940 con 0x5580aceb0b40 2024-09-11T23:58:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.076+0000 7f44c9f9f740 1 -- 172.21.15.62:0/1758350226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 msgr2=0x5580aceaf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.076+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/1758350226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceaf090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0985c0 tx=0x7f44bc09b610 comp rx=0 tx=0).stop 2024-09-11T23:58:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.077+0000 7f44c9f9f740 1 -- 172.21.15.62:0/1758350226 shutdown_connections 2024-09-11T23:58:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.077+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/1758350226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580acebc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.077+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/1758350226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceafa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.077+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/1758350226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceaf090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.077+0000 7f44c9f9f740 1 -- 172.21.15.62:0/1758350226 >> 172.21.15.62:0/1758350226 conn(0x5580ace78310 msgr2=0x5580aceabc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.077+0000 7f44c9f9f740 1 -- 172.21.15.62:0/1758350226 shutdown_connections 2024-09-11T23:58:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.077+0000 7f44c9f9f740 1 -- 172.21.15.62:0/1758350226 wait complete. 2024-09-11T23:58:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.077+0000 7f44c9f9f740 1 -- start start 2024-09-11T23:58:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.077+0000 7f44c9f9f740 1 Processor -- start 2024-09-11T23:58:08.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceb9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceb6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580aceb75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5580acebd010 con 0x5580aceaf5d0 2024-09-11T23:58:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5580acdd0060 con 0x5580aceba370 2024-09-11T23:58:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5580acebd190 con 0x5580aceb0b40 2024-09-11T23:58:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5962640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceb9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c9f9f740 10 client.0 _ll_register_callbacks cb 0x5580acb238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5161640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceb6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5161640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceb6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33344/0 (socket says 172.21.15.62:33344) 2024-09-11T23:58:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5962640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceb9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46556/0 (socket says 172.21.15.62:46556) 2024-09-11T23:58:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5161640 1 -- 172.21.15.62:0/3729708698 learned_addr learned my addr 172.21.15.62:0/3729708698 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c6163640 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580aceb75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5161640 1 -- 172.21.15.62:0/3729708698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 msgr2=0x5580aceb75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5161640 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580aceb75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5161640 1 -- 172.21.15.62:0/3729708698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 msgr2=0x5580aceb9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5161640 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceb9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5161640 1 -- 172.21.15.62:0/3729708698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580acb863c0 con 0x5580aceb0b40 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.078+0000 7f44c5161640 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceb6fe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f44b0031550 tx=0x7f44b0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5be39fb60d2bab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44a97fa640 1 -- 172.21.15.62:0/3729708698 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0035040 con 0x5580aceb0b40 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44a97fa640 1 -- 172.21.15.62:0/3729708698 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f44b0004bb0 con 0x5580aceb0b40 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44c9f9f740 1 -- 172.21.15.62:0/3729708698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44bc0cd0a0 con 0x5580aceb0b40 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44a97fa640 1 -- 172.21.15.62:0/3729708698 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44b0004db0 con 0x5580aceb0b40 2024-09-11T23:58:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44c9f9f740 10 client.25996 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44c9f9f740 1 -- 172.21.15.62:0/3729708698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580acec8a70 con 0x5580aceb0b40 2024-09-11T23:58:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44a97fa640 1 -- 172.21.15.62:0/3729708698 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44bc0cd0a0 con 0x5580aceb0b40 2024-09-11T23:58:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44a97fa640 1 -- 172.21.15.62:0/3729708698 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f44b0032890 con 0x5580aceb0b40 2024-09-11T23:58:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44c9f9f740 10 client.25996 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44c9f9f740 10 client.25996 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:58:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.079+0000 7f44c9f9f740 20 client.25996 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.080+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5580acec9e80 0x5580acecc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.080+0000 7f44c9f9f740 4 client.25996 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.080+0000 7f44c9f9f740 1 -- 172.21.15.62:0/3729708698 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580ace22860 con 0x5580acec9e80 2024-09-11T23:58:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.080+0000 7f44c5962640 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5580acec9e80 0x5580acecc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.080+0000 7f44c5962640 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5580acec9e80 0x5580acecc2e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e43e0217b525c11 server_cookie=997419b2c4b86fb1 in_seq=0 out_seq=0 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.080+0000 7f44a97fa640 10 client.25996 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.081+0000 7f44a97fa640 1 -- 172.21.15.62:0/3729708698 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580ace22860 con 0x5580acec9e80 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.081+0000 7f44a97fa640 10 client.25996 handle_command_reply: tid=0 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.081+0000 7f44c9f9f740 1 client.25996 shutdown 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.081+0000 7f44c9f9f740 1 -- 172.21.15.62:0/3729708698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 msgr2=0x5580aceb6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.081+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceb6fe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f44b0031550 tx=0x7f44b0009260 comp rx=0 tx=0).stop 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.082+0000 7f44c9f9f740 1 -- 172.21.15.62:0/3729708698 shutdown_connections 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.082+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5580acec9e80 0x5580acecc2e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.082+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580aceb75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.082+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceb6fe0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.082+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/3729708698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceb9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.082+0000 7f44c9f9f740 1 -- 172.21.15.62:0/3729708698 >> 172.21.15.62:0/3729708698 conn(0x5580ace78310 msgr2=0x5580aceb1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:08.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.082+0000 7f44a97fa640 0 client.25996 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:58:08.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.082+0000 7f44c9f9f740 1 -- 172.21.15.62:0/3729708698 shutdown_connections 2024-09-11T23:58:08.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.082+0000 7f44c9f9f740 1 -- 172.21.15.62:0/3729708698 wait complete. 2024-09-11T23:58:08.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.082+0000 7f44c9f9f740 20 client.25996 trim_cache size 0 max 16384 2024-09-11T23:58:08.093 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:08.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.107+0000 7f44c9f9f740 1 Processor -- start 2024-09-11T23:58:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.107+0000 7f44c9f9f740 1 -- start start 2024-09-11T23:58:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.107+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceb68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.107+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceb2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.107+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580aceb2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.107+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01adf10 con 0x5580aceaf5d0 2024-09-11T23:58:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.107+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01ab7b0 con 0x5580aceba370 2024-09-11T23:58:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.107+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01a9660 con 0x5580aceb0b40 2024-09-11T23:58:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.108+0000 7f44c6163640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580aceb2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.108+0000 7f44c6163640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580aceb2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57544/0 (socket says 172.21.15.62:57544) 2024-09-11T23:58:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.108+0000 7f44c6163640 1 -- 172.21.15.62:0/764982478 learned_addr learned my addr 172.21.15.62:0/764982478 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.108+0000 7f44c5161640 1 --2- 172.21.15.62:0/764982478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceb2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.108+0000 7f44c6163640 1 -- 172.21.15.62:0/764982478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 msgr2=0x5580aceb2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.108+0000 7f44c6163640 1 --2- 172.21.15.62:0/764982478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceb2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.108+0000 7f44c6163640 1 -- 172.21.15.62:0/764982478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 msgr2=0x5580aceb68a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:58:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.108+0000 7f44c6163640 1 --2- 172.21.15.62:0/764982478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceb68a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.108+0000 7f44c6163640 1 -- 172.21.15.62:0/764982478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580aceb5f90 con 0x5580aceba370 2024-09-11T23:58:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.109+0000 7f44c6163640 1 --2- 172.21.15.62:0/764982478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580aceb2660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009fc0 tx=0x7f44b80348f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af0a5992e4f3139 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.109+0000 7f44ab7fe640 1 -- 172.21.15.62:0/764982478 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8003970 con 0x5580aceba370 2024-09-11T23:58:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.109+0000 7f44ab7fe640 1 -- 172.21.15.62:0/764982478 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b8004020 con 0x5580aceba370 2024-09-11T23:58:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.109+0000 7f44ab7fe640 1 -- 172.21.15.62:0/764982478 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8011940 con 0x5580aceba370 2024-09-11T23:58:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.110+0000 7f44c9f9f740 1 -- 172.21.15.62:0/764982478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 msgr2=0x5580aceb2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.110+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/764982478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580aceb2660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009fc0 tx=0x7f44b80348f0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.110+0000 7f44c9f9f740 1 -- 172.21.15.62:0/764982478 shutdown_connections 2024-09-11T23:58:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.110+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/764982478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceba370 0x5580aceb2660 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.110+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/764982478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceb0b40 0x5580aceb2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.110+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/764982478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceaf5d0 0x5580aceb68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.110+0000 7f44c9f9f740 1 -- 172.21.15.62:0/764982478 >> 172.21.15.62:0/764982478 conn(0x5580ace78310 msgr2=0x5580aceb9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.112+0000 7f44c9f9f740 1 -- 172.21.15.62:0/764982478 shutdown_connections 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.112+0000 7f44c9f9f740 1 -- 172.21.15.62:0/764982478 wait complete. 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.113+0000 7f44c9f9f740 1 -- start start 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.113+0000 7f44c9f9f740 1 Processor -- start 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.114+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceaf5d0 0x5580acebe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.114+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceb0b40 0x5580acebeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.114+0000 7f44c9f9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceba370 0x5580acf33b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.114+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01a9cb0 con 0x5580aceba370 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.114+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01ab0e0 con 0x5580aceb0b40 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.114+0000 7f44c9f9f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01aa340 con 0x5580aceaf5d0 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.114+0000 7f44c9f9f740 10 client.0 _ll_register_callbacks cb 0x7f44b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.114+0000 7f44c5161640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceb0b40 0x5580acebeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.114+0000 7f44c5161640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceb0b40 0x5580acebeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57548/0 (socket says 172.21.15.62:57548) 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.115+0000 7f44c5161640 1 -- 172.21.15.62:0/540165752 learned_addr learned my addr 172.21.15.62:0/540165752 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.115+0000 7f44c5962640 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceaf5d0 0x5580acebe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.115+0000 7f44c6163640 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceba370 0x5580acf33b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.115+0000 7f44c5161640 1 -- 172.21.15.62:0/540165752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceaf5d0 msgr2=0x5580acebe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.115+0000 7f44c5161640 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceaf5d0 0x5580acebe5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.115+0000 7f44c5161640 1 -- 172.21.15.62:0/540165752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceba370 msgr2=0x5580acf33b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.115+0000 7f44c5161640 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceba370 0x5580acf33b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.115+0000 7f44c5161640 1 -- 172.21.15.62:0/540165752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580aceba0a0 con 0x5580aceb0b40 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.115+0000 7f44c5161640 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceb0b40 0x5580acebeae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f44b003f040 tx=0x7f44b0032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc855f0ade337c75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.116+0000 7f44a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:58:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.116+0000 7f44a97fa640 1 -- 172.21.15.62:0/540165752 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0035040 con 0x5580aceb0b40 2024-09-11T23:58:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.116+0000 7f44a97fa640 1 -- 172.21.15.62:0/540165752 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f44b00329d0 con 0x5580aceb0b40 2024-09-11T23:58:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.116+0000 7f44c9f9f740 1 -- 172.21.15.62:0/540165752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b8004020 con 0x5580aceb0b40 2024-09-11T23:58:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.117+0000 7f44a97fa640 1 -- 172.21.15.62:0/540165752 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44b00042c0 con 0x5580aceb0b40 2024-09-11T23:58:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.117+0000 7f44c9f9f740 10 client.26051 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.117+0000 7f44c9f9f740 1 -- 172.21.15.62:0/540165752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580acf350a0 con 0x5580aceb0b40 2024-09-11T23:58:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.117+0000 7f44a97fa640 1 -- 172.21.15.62:0/540165752 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b0003ad0 con 0x5580aceb0b40 2024-09-11T23:58:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.117+0000 7f44a97fa640 1 -- 172.21.15.62:0/540165752 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f44b006c030 con 0x5580aceb0b40 2024-09-11T23:58:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.117+0000 7f44c9f9f740 10 client.26051 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.117+0000 7f44c9f9f740 10 client.26051 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:58:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.117+0000 7f44c9f9f740 20 client.26051 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.118+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5580acf357c0 0x5580acf37be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.118+0000 7f44c9f9f740 4 client.26051 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:58:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.118+0000 7f44c9f9f740 1 -- 172.21.15.62:0/540165752 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f44b8004020 con 0x5580acf357c0 2024-09-11T23:58:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.118+0000 7f44c5962640 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5580acf357c0 0x5580acf37be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.126+0000 7f44c5962640 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5580acf357c0 0x5580acf37be0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e16bf19ae5d99f85 server_cookie=e3064c4158db08cb in_seq=0 out_seq=0 2024-09-11T23:58:08.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.126+0000 7f44a97fa640 10 client.26051 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:58:08.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.133+0000 7f44a97fa640 1 -- 172.21.15.62:0/540165752 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f44b8004020 con 0x5580acf357c0 2024-09-11T23:58:08.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.133+0000 7f44a97fa640 10 client.26051 handle_command_reply: tid=0 2024-09-11T23:58:08.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.134+0000 7f44c9f9f740 1 client.26051 shutdown 2024-09-11T23:58:08.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.135+0000 7f44c9f9f740 1 -- 172.21.15.62:0/540165752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceb0b40 msgr2=0x5580acebeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.135+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceb0b40 0x5580acebeae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f44b003f040 tx=0x7f44b0032ed0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.136+0000 7f44c9f9f740 1 -- 172.21.15.62:0/540165752 shutdown_connections 2024-09-11T23:58:08.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.136+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5580acf357c0 0x5580acf37be0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.136+0000 7f44a97fa640 0 client.26051 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:58:08.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.136+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580aceba370 0x5580acf33b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.136+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580aceb0b40 0x5580acebeae0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.136+0000 7f44c9f9f740 1 --2- 172.21.15.62:0/540165752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580aceaf5d0 0x5580acebe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.136+0000 7f44c9f9f740 1 -- 172.21.15.62:0/540165752 >> 172.21.15.62:0/540165752 conn(0x5580ace78310 msgr2=0x5580aceaa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:08.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.136+0000 7f44c9f9f740 1 -- 172.21.15.62:0/540165752 shutdown_connections 2024-09-11T23:58:08.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.136+0000 7f44c9f9f740 1 -- 172.21.15.62:0/540165752 wait complete. 2024-09-11T23:58:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.136+0000 7f44c9f9f740 20 client.26051 trim_cache size 0 max 16384 2024-09-11T23:58:08.137 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:58:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.138+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f44ac0a2e40 msgr2=0x7f44ac0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.138+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f44ac0a2e40 0x7f44ac0a5300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f44b0006010 tx=0x7f44b0000bc0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.138+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c0100a80 msgr2=0x7f44c019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.138+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c0100a80 0x7f44c019a690 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e9f0 tx=0x7f44bc00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.139+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 shutdown_connections 2024-09-11T23:58:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.139+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f44ac0a2e40 0x7f44ac0a5300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.139+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44c0106a80 0x7f44c019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.139+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c0101420 0x7f44c019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.139+0000 7f44c7bc6640 1 --2- 172.21.15.62:0/3963953341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c0100a80 0x7f44c019a690 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.139+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 >> 172.21.15.62:0/3963953341 conn(0x7f44c00fc7a0 msgr2=0x7f44c00fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.143+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 shutdown_connections 2024-09-11T23:58:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.143+0000 7f44c7bc6640 1 -- 172.21.15.62:0/3963953341 wait complete. 2024-09-11T23:58:08.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.728+0000 7f76b1949640 1 -- 172.21.15.62:0/2978946816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac108290 msgr2=0x7f76ac10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.728+0000 7f76b1949640 1 --2- 172.21.15.62:0/2978946816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac108290 0x7f76ac10ec30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009cd0 tx=0x7f76a402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.728+0000 7f76b1949640 1 -- 172.21.15.62:0/2978946816 shutdown_connections 2024-09-11T23:58:08.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.728+0000 7f76b1949640 1 --2- 172.21.15.62:0/2978946816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac108290 0x7f76ac10ec30 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.728+0000 7f76b1949640 1 --2- 172.21.15.62:0/2978946816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76ac103740 0x7f76ac107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.728+0000 7f76b1949640 1 --2- 172.21.15.62:0/2978946816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76ac102d90 0x7f76ac103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.728+0000 7f76b1949640 1 -- 172.21.15.62:0/2978946816 >> 172.21.15.62:0/2978946816 conn(0x7f76ac0fc7d0 msgr2=0x7f76ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:08.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.729+0000 7f76b1949640 1 -- 172.21.15.62:0/2978946816 shutdown_connections 2024-09-11T23:58:08.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.729+0000 7f76b1949640 1 -- 172.21.15.62:0/2978946816 wait complete. 2024-09-11T23:58:08.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.730+0000 7f76b1949640 1 Processor -- start 2024-09-11T23:58:08.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.730+0000 7f76b1949640 1 -- start start 2024-09-11T23:58:08.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.731+0000 7f76b1949640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76ac102d90 0x7f76ac198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.731+0000 7f76b1949640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac103740 0x7f76ac198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.731+0000 7f76b1949640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76ac108290 0x7f76ac19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.731+0000 7f76b1949640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac112090 con 0x7f76ac103740 2024-09-11T23:58:08.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.731+0000 7f76b1949640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac111f10 con 0x7f76ac102d90 2024-09-11T23:58:08.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.731+0000 7f76b1949640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac112210 con 0x7f76ac108290 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.731+0000 7f76aa7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac103740 0x7f76ac198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.731+0000 7f76aa7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac103740 0x7f76ac198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46592/0 (socket says 172.21.15.62:46592) 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.731+0000 7f76aa7fc640 1 -- 172.21.15.62:0/2945206243 learned_addr learned my addr 172.21.15.62:0/2945206243 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.731+0000 7f76ab7fe640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76ac108290 0x7f76ac19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.732+0000 7f76aa7fc640 1 -- 172.21.15.62:0/2945206243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76ac108290 msgr2=0x7f76ac19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.732+0000 7f76aa7fc640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76ac108290 0x7f76ac19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.732+0000 7f76aa7fc640 1 -- 172.21.15.62:0/2945206243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76ac102d90 msgr2=0x7f76ac198550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.732+0000 7f76aa7fc640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76ac102d90 0x7f76ac198550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.732+0000 7f76aa7fc640 1 -- 172.21.15.62:0/2945206243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac19d500 con 0x7f76ac103740 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.732+0000 7f76ab7fe640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76ac108290 0x7f76ac19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:58:08.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.732+0000 7f76aa7fc640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac103740 0x7f76ac198a90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e9f0 tx=0x7f76a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.733+0000 7f76b0947640 1 -- 172.21.15.62:0/2945206243 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a000ccc0 con 0x7f76ac103740 2024-09-11T23:58:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.733+0000 7f76b0947640 1 -- 172.21.15.62:0/2945206243 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a0005500 con 0x7f76ac103740 2024-09-11T23:58:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.733+0000 7f76b0947640 1 -- 172.21.15.62:0/2945206243 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00186d0 con 0x7f76ac103740 2024-09-11T23:58:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.733+0000 7f76b1949640 1 -- 172.21.15.62:0/2945206243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac19d7f0 con 0x7f76ac103740 2024-09-11T23:58:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.734+0000 7f76b1949640 1 -- 172.21.15.62:0/2945206243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac100920 con 0x7f76ac103740 2024-09-11T23:58:08.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.737+0000 7f76b0947640 1 -- 172.21.15.62:0/2945206243 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f76a00046c0 con 0x7f76ac103740 2024-09-11T23:58:08.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.738+0000 7f76b3d22740 1 Processor -- start 2024-09-11T23:58:08.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76b3d22740 1 -- start start 2024-09-11T23:58:08.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76b0947640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76940a2e80 0x7f76940a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76b0947640 1 -- 172.21.15.62:0/2945206243 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f76a00cf7b0 con 0x7f76ac103740 2024-09-11T23:58:08.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f30080 0x559451f3a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 0x559451f3cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76b3d22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559451e952a0 con 0x559451f30080 2024-09-11T23:58:08.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76b3d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559451e3cb10 con 0x559451f3a940 2024-09-11T23:58:08.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76b3d22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559451e91260 con 0x559451f2f760 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33394/0 (socket says 172.21.15.62:33394) 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76aaffd640 1 -- 172.21.15.62:0/3057859655 learned_addr learned my addr 172.21.15.62:0/3057859655 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76ab7fe640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76940a2e80 0x7f76940a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.739+0000 7f76aa7fc640 1 --2- 172.21.15.62:0/3057859655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f30080 0x559451f3a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.740+0000 7f76ab7fe640 1 --2- 172.21.15.62:0/3057859655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 0x559451f3cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.740+0000 7f76aaffd640 1 -- 172.21.15.62:0/3057859655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 msgr2=0x559451f3cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.740+0000 7f76aaffd640 1 --2- 172.21.15.62:0/3057859655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 0x559451f3cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.740+0000 7f76aaffd640 1 -- 172.21.15.62:0/3057859655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f30080 msgr2=0x559451f3a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.740+0000 7f76aaffd640 1 --2- 172.21.15.62:0/3057859655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f30080 0x559451f3a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.740+0000 7f76aaffd640 1 -- 172.21.15.62:0/3057859655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559451c06b90 con 0x559451f2f760 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.740+0000 7f76ab7fe640 1 --2- 172.21.15.62:0/3057859655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 0x559451f3cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.740+0000 7f76aaffd640 1 --2- 172.21.15.62:0/3057859655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2fb40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f769c009560 tx=0x7f769c009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fac3d8dbb49916f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.741+0000 7f76ab7fe640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76940a2e80 0x7f76940a5340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f76a4005e30 tx=0x7f76a4005dc0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.741+0000 7f7691ffb640 1 -- 172.21.15.62:0/3057859655 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c004020 con 0x559451f2f760 2024-09-11T23:58:08.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.741+0000 7f7691ffb640 1 -- 172.21.15.62:0/3057859655 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f769c0041c0 con 0x559451f2f760 2024-09-11T23:58:08.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.741+0000 7f7691ffb640 1 -- 172.21.15.62:0/3057859655 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c01e880 con 0x559451f2f760 2024-09-11T23:58:08.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.742+0000 7f76b3d22740 1 -- 172.21.15.62:0/3057859655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 msgr2=0x559451f2fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.742+0000 7f76b3d22740 1 --2- 172.21.15.62:0/3057859655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2fb40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f769c009560 tx=0x7f769c009960 comp rx=0 tx=0).stop 2024-09-11T23:58:08.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.743+0000 7f76b3d22740 1 -- 172.21.15.62:0/3057859655 shutdown_connections 2024-09-11T23:58:08.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.743+0000 7f76b3d22740 1 --2- 172.21.15.62:0/3057859655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 0x559451f3cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.743+0000 7f76b3d22740 1 --2- 172.21.15.62:0/3057859655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f30080 0x559451f3a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.743+0000 7f76b3d22740 1 --2- 172.21.15.62:0/3057859655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2fb40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.743+0000 7f76b3d22740 1 -- 172.21.15.62:0/3057859655 >> 172.21.15.62:0/3057859655 conn(0x559451f2bcf0 msgr2=0x559451f2c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.743+0000 7f76b3d22740 1 -- 172.21.15.62:0/3057859655 shutdown_connections 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.743+0000 7f76b3d22740 1 -- 172.21.15.62:0/3057859655 wait complete. 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.743+0000 7f76b3d22740 1 -- start start 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76b3d22740 1 Processor -- start 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f37740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f30080 0x559451f37c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 0x559451f381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76b3d22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559451f3d560 con 0x559451f30080 2024-09-11T23:58:08.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76b3d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559451e1d6e0 con 0x559451f3a940 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76b3d22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559451f3d6e0 con 0x559451f2f760 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76b3d22740 10 client.0 _ll_register_callbacks cb 0x559451e41a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76aa7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f30080 0x559451f37c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f37740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 0x559451f381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 0x559451f381c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57586/0 (socket says 172.21.15.62:57586) 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f37740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33400/0 (socket says 172.21.15.62:33400) 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.744+0000 7f76ab7fe640 1 -- 172.21.15.62:0/1618736653 learned_addr learned my addr 172.21.15.62:0/1618736653 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.745+0000 7f76aaffd640 1 -- 172.21.15.62:0/1618736653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 msgr2=0x559451f381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.745+0000 7f76aaffd640 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 0x559451f381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.745+0000 7f76aaffd640 1 -- 172.21.15.62:0/1618736653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f30080 msgr2=0x559451f37c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.745+0000 7f76aaffd640 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f30080 0x559451f37c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.745+0000 7f76aaffd640 1 -- 172.21.15.62:0/1618736653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559451f46a20 con 0x559451f2f760 2024-09-11T23:58:08.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.745+0000 7f76aaffd640 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f37740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f769c003f40 tx=0x7f769c003bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a528cc8da12bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.745+0000 7f768affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:58:08.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.745+0000 7f768affd640 1 -- 172.21.15.62:0/1618736653 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c004020 con 0x559451f2f760 2024-09-11T23:58:08.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.745+0000 7f768affd640 1 -- 172.21.15.62:0/1618736653 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f769c005030 con 0x559451f2f760 2024-09-11T23:58:08.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.745+0000 7f76b3d22740 1 -- 172.21.15.62:0/1618736653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f769c0041c0 con 0x559451f2f760 2024-09-11T23:58:08.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.746+0000 7f768affd640 1 -- 172.21.15.62:0/1618736653 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f769c003d30 con 0x559451f2f760 2024-09-11T23:58:08.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.747+0000 7f76b3d22740 10 client.26014 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:08.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.747+0000 7f76b3d22740 1 -- 172.21.15.62:0/1618736653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559451f492c0 con 0x559451f2f760 2024-09-11T23:58:08.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.747+0000 7f768affd640 1 -- 172.21.15.62:0/1618736653 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f769c0041c0 con 0x559451f2f760 2024-09-11T23:58:08.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.747+0000 7f768affd640 1 -- 172.21.15.62:0/1618736653 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f769c030030 con 0x559451f2f760 2024-09-11T23:58:08.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.747+0000 7f76b3d22740 10 client.26014 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:08.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.747+0000 7f76b3d22740 10 client.26014 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:58:08.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.747+0000 7f76b3d22740 20 client.26014 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:08.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.747+0000 7f76b3d22740 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559451f4a780 0x559451f4cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.748+0000 7f76b3d22740 4 client.26014 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:08.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.748+0000 7f76b3d22740 1 -- 172.21.15.62:0/1618736653 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559451ea29d0 con 0x559451f4a780 2024-09-11T23:58:08.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.748+0000 7f76aa7fc640 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559451f4a780 0x559451f4cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.748+0000 7f76aa7fc640 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559451f4a780 0x559451f4cb70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a2d790f3618fe05 server_cookie=6ce5f506a6dcdf53 in_seq=0 out_seq=0 2024-09-11T23:58:08.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.749+0000 7f768affd640 10 client.26014 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:08.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.749+0000 7f768affd640 1 -- 172.21.15.62:0/1618736653 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559451ea29d0 con 0x559451f4a780 2024-09-11T23:58:08.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.749+0000 7f768affd640 10 client.26014 handle_command_reply: tid=0 2024-09-11T23:58:08.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.750+0000 7f76b3d22740 1 client.26014 shutdown 2024-09-11T23:58:08.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.752+0000 7f76b3d22740 1 -- 172.21.15.62:0/1618736653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 msgr2=0x559451f37740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.752+0000 7f76b3d22740 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f37740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f769c003f40 tx=0x7f769c003bc0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.753+0000 7f76b3d22740 1 -- 172.21.15.62:0/1618736653 shutdown_connections 2024-09-11T23:58:08.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.753+0000 7f76b3d22740 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559451f4a780 0x559451f4cb70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.753+0000 7f76b3d22740 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f3a940 0x559451f381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.753+0000 7f76b3d22740 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f30080 0x559451f37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.753+0000 7f76b3d22740 1 --2- 172.21.15.62:0/1618736653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f37740 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.753+0000 7f76b3d22740 1 -- 172.21.15.62:0/1618736653 >> 172.21.15.62:0/1618736653 conn(0x559451f2bcf0 msgr2=0x559451f31a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:08.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.753+0000 7f768affd640 0 client.26014 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:08.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.753+0000 7f76b3d22740 1 -- 172.21.15.62:0/1618736653 shutdown_connections 2024-09-11T23:58:08.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.755+0000 7f76b3d22740 1 -- 172.21.15.62:0/1618736653 wait complete. 2024-09-11T23:58:08.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.756+0000 7f76b3d22740 20 client.26014 trim_cache size 0 max 16384 2024-09-11T23:58:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.796+0000 7f76b3d22740 1 Processor -- start 2024-09-11T23:58:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.796+0000 7f76b3d22740 1 -- start start 2024-09-11T23:58:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f30080 0x559451f521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f3a940 0x559451f526e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76b3d22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac1ade70 con 0x559451f3a940 2024-09-11T23:58:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76b3d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac1ab710 con 0x559451f30080 2024-09-11T23:58:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76b3d22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac1a9560 con 0x559451f2f760 2024-09-11T23:58:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76aa7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f30080 0x559451f521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33408/0 (socket says 172.21.15.62:33408) 2024-09-11T23:58:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76aaffd640 1 -- 172.21.15.62:0/3137362144 learned_addr learned my addr 172.21.15.62:0/3137362144 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76ab7fe640 1 --2- 172.21.15.62:0/3137362144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f3a940 0x559451f526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76aa7fc640 1 -- 172.21.15.62:0/3137362144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 msgr2=0x559451f2ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76aa7fc640 1 --2- 172.21.15.62:0/3137362144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76aa7fc640 1 -- 172.21.15.62:0/3137362144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f3a940 msgr2=0x559451f526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76aa7fc640 1 --2- 172.21.15.62:0/3137362144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f3a940 0x559451f526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76aa7fc640 1 -- 172.21.15.62:0/3137362144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559451f2bb20 con 0x559451f30080 2024-09-11T23:58:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.797+0000 7f76aaffd640 1 --2- 172.21.15.62:0/3137362144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:58:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.798+0000 7f76aa7fc640 1 --2- 172.21.15.62:0/3137362144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f30080 0x559451f521a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76a00ccf30 tx=0x7f76a00cf040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f42faad6fa3f26e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.798+0000 7f76917fa640 1 -- 172.21.15.62:0/3137362144 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00a8070 con 0x559451f30080 2024-09-11T23:58:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.798+0000 7f76917fa640 1 -- 172.21.15.62:0/3137362144 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a009a000 con 0x559451f30080 2024-09-11T23:58:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.798+0000 7f76917fa640 1 -- 172.21.15.62:0/3137362144 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a009b150 con 0x559451f30080 2024-09-11T23:58:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.799+0000 7f76b3d22740 1 -- 172.21.15.62:0/3137362144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f30080 msgr2=0x559451f521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.799+0000 7f76b3d22740 1 --2- 172.21.15.62:0/3137362144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f30080 0x559451f521a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76a00ccf30 tx=0x7f76a00cf040 comp rx=0 tx=0).stop 2024-09-11T23:58:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.799+0000 7f76b3d22740 1 -- 172.21.15.62:0/3137362144 shutdown_connections 2024-09-11T23:58:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.799+0000 7f76b3d22740 1 --2- 172.21.15.62:0/3137362144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f3a940 0x559451f526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.799+0000 7f76b3d22740 1 --2- 172.21.15.62:0/3137362144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f30080 0x559451f521a0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76a00ccf30 tx=0x7f76a00cf040 comp rx=0 tx=0).stop 2024-09-11T23:58:08.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.799+0000 7f76b3d22740 1 --2- 172.21.15.62:0/3137362144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f2f760 0x559451f2ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.799+0000 7f76b3d22740 1 -- 172.21.15.62:0/3137362144 >> 172.21.15.62:0/3137362144 conn(0x559451f2bcf0 msgr2=0x559451f2a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:08.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.800+0000 7f76b3d22740 1 -- 172.21.15.62:0/3137362144 shutdown_connections 2024-09-11T23:58:08.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.800+0000 7f76b3d22740 1 -- 172.21.15.62:0/3137362144 wait complete. 2024-09-11T23:58:08.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.800+0000 7f76b3d22740 1 -- start start 2024-09-11T23:58:08.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.801+0000 7f76b3d22740 1 Processor -- start 2024-09-11T23:58:08.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.801+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f2f760 0x559451f43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.801+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f3a940 0x559451f34540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.801+0000 7f76b3d22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f34c10 0x559451f40d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.801+0000 7f76aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f3a940 0x559451f34540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.801+0000 7f76b3d22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac1a9bd0 con 0x559451f34c10 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.801+0000 7f76b3d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac1ab040 con 0x559451f2f760 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.801+0000 7f76b3d22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac1aa2a0 con 0x559451f3a940 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.801+0000 7f76b3d22740 10 client.0 _ll_register_callbacks cb 0x7f769c003a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.802+0000 7f76ab7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f34c10 0x559451f40d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.802+0000 7f76aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f3a940 0x559451f34540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33424/0 (socket says 172.21.15.62:33424) 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.802+0000 7f76aa7fc640 1 -- 172.21.15.62:0/2846576342 learned_addr learned my addr 172.21.15.62:0/2846576342 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.803+0000 7f76ab7fe640 1 -- 172.21.15.62:0/2846576342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f3a940 msgr2=0x559451f34540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.803+0000 7f76ab7fe640 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f3a940 0x559451f34540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.803+0000 7f76ab7fe640 1 -- 172.21.15.62:0/2846576342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f2f760 msgr2=0x559451f43270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.803+0000 7f76ab7fe640 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f2f760 0x559451f43270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.803+0000 7f76ab7fe640 1 -- 172.21.15.62:0/2846576342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559451f27260 con 0x559451f34c10 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.803+0000 7f76ab7fe640 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f34c10 0x559451f40d60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f76a402aed0 tx=0x7f76a40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443b0f3e71e0b027 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.803+0000 7f768affd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:08.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.804+0000 7f768affd640 1 -- 172.21.15.62:0/2846576342 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4045070 con 0x559451f34c10 2024-09-11T23:58:08.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.804+0000 7f76b3d22740 1 -- 172.21.15.62:0/2846576342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76a009a000 con 0x559451f34c10 2024-09-11T23:58:08.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.805+0000 7f768affd640 1 -- 172.21.15.62:0/2846576342 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f76a4030a70 con 0x559451f34c10 2024-09-11T23:58:08.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.805+0000 7f768affd640 1 -- 172.21.15.62:0/2846576342 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76a4030cb0 con 0x559451f34c10 2024-09-11T23:58:08.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.806+0000 7f76b3d22740 10 client.16434 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:08.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.806+0000 7f76b3d22740 1 -- 172.21.15.62:0/2846576342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559451fc6530 con 0x559451f34c10 2024-09-11T23:58:08.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.806+0000 7f768affd640 1 -- 172.21.15.62:0/2846576342 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a009a000 con 0x559451f34c10 2024-09-11T23:58:08.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.806+0000 7f768affd640 1 -- 172.21.15.62:0/2846576342 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f76a4054070 con 0x559451f34c10 2024-09-11T23:58:08.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.806+0000 7f76b3d22740 10 client.16434 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:08.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.807+0000 7f76b3d22740 10 client.16434 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:58:08.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.807+0000 7f76b3d22740 20 client.16434 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:08.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.807+0000 7f76b3d22740 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559451fc6ca0 0x559451fc90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:08.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.807+0000 7f76b3d22740 4 client.16434 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-11T23:58:08.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.807+0000 7f76b3d22740 1 -- 172.21.15.62:0/2846576342 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x559451e9cd20 con 0x559451fc6ca0 2024-09-11T23:58:08.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.807+0000 7f76aaffd640 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559451fc6ca0 0x559451fc90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:08.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.808+0000 7f76aaffd640 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559451fc6ca0 0x559451fc90e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d1a68596d0cb1f server_cookie=6b467c3713f1c32e in_seq=0 out_seq=0 2024-09-11T23:58:08.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:08.808+0000 7f768affd640 10 client.16434 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:08.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:08 smithi062 ceph-mon[30958]: pgmap v184: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 27 MiB/s wr, 1.90k op/s 2024-09-11T23:58:09.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f768affd640 1 -- 172.21.15.62:0/2846576342 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559451e9cd20 con 0x559451fc6ca0 2024-09-11T23:58:09.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f768affd640 10 client.16434 handle_command_reply: tid=0 2024-09-11T23:58:09.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 client.16434 shutdown 2024-09-11T23:58:09.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 -- 172.21.15.62:0/2846576342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f34c10 msgr2=0x559451f40d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:09.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f34c10 0x559451f40d60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f76a402aed0 tx=0x7f76a40095c0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 -- 172.21.15.62:0/2846576342 shutdown_connections 2024-09-11T23:58:09.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559451fc6ca0 0x559451fc90e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559451f34c10 0x559451f40d60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559451f3a940 0x559451f34540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 --2- 172.21.15.62:0/2846576342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559451f2f760 0x559451f43270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 -- 172.21.15.62:0/2846576342 >> 172.21.15.62:0/2846576342 conn(0x559451f2bcf0 msgr2=0x559451f37eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:09.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f768affd640 0 client.16434 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:09.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 -- 172.21.15.62:0/2846576342 shutdown_connections 2024-09-11T23:58:09.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 1 -- 172.21.15.62:0/2846576342 wait complete. 2024-09-11T23:58:09.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.099+0000 7f76b3d22740 20 client.16434 trim_cache size 0 max 16384 2024-09-11T23:58:09.100 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 22, 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "42b4ed57": { 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 22, 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:58:09.101 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 140.1, 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:09.102 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:58:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 -- 172.21.15.62:0/2945206243 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76940a2e80 msgr2=0x7f76940a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76940a2e80 0x7f76940a5340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f76a4005e30 tx=0x7f76a4005dc0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 -- 172.21.15.62:0/2945206243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac103740 msgr2=0x7f76ac198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac103740 0x7f76ac198a90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e9f0 tx=0x7f76a000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 -- 172.21.15.62:0/2945206243 shutdown_connections 2024-09-11T23:58:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76940a2e80 0x7f76940a5340 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76ac108290 0x7f76ac19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac103740 0x7f76ac198a90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 --2- 172.21.15.62:0/2945206243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76ac102d90 0x7f76ac198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 -- 172.21.15.62:0/2945206243 >> 172.21.15.62:0/2945206243 conn(0x7f76ac0fc7d0 msgr2=0x7f76ac106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 -- 172.21.15.62:0/2945206243 shutdown_connections 2024-09-11T23:58:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:09.100+0000 7f76b1949640 1 -- 172.21.15.62:0/2945206243 wait complete. 2024-09-11T23:58:09.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:08 smithi120 ceph-mon[35182]: pgmap v184: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 27 MiB/s wr, 1.90k op/s 2024-09-11T23:58:09.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:08 smithi083 ceph-mon[34805]: pgmap v184: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 27 MiB/s wr, 1.90k op/s 2024-09-11T23:58:10.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:58:10.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:10 smithi062 ceph-mon[30958]: pgmap v185: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 22 MiB/s wr, 1.57k op/s 2024-09-11T23:58:11.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:10 smithi120 ceph-mon[35182]: pgmap v185: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 22 MiB/s wr, 1.57k op/s 2024-09-11T23:58:11.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:10 smithi083 ceph-mon[34805]: pgmap v185: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 22 MiB/s wr, 1.57k op/s 2024-09-11T23:58:11.509 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '42b4ed57', quiesce took 7.3 seconds. Will release after: 12.8 2024-09-11T23:58:11.590 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:12.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.053+0000 7f3dca640640 1 -- 172.21.15.62:0/1399368992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dc4106a80 msgr2=0x7f3dc4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:12.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.053+0000 7f3dca640640 1 --2- 172.21.15.62:0/1399368992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dc4106a80 0x7f3dc4106e60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009930 tx=0x7f3db402a420 comp rx=0 tx=0).stop 2024-09-11T23:58:12.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.053+0000 7f3dca640640 1 -- 172.21.15.62:0/1399368992 shutdown_connections 2024-09-11T23:58:12.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.053+0000 7f3dca640640 1 --2- 172.21.15.62:0/1399368992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc4101420 0x7f3dc410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.053+0000 7f3dca640640 1 --2- 172.21.15.62:0/1399368992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4100a80 0x7f3dc4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.053+0000 7f3dca640640 1 --2- 172.21.15.62:0/1399368992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dc4106a80 0x7f3dc4106e60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.053+0000 7f3dca640640 1 -- 172.21.15.62:0/1399368992 >> 172.21.15.62:0/1399368992 conn(0x7f3dc40fc7a0 msgr2=0x7f3dc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:12.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.053+0000 7f3dca640640 1 -- 172.21.15.62:0/1399368992 shutdown_connections 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.053+0000 7f3dca640640 1 -- 172.21.15.62:0/1399368992 wait complete. 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dca640640 1 Processor -- start 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dca640640 1 -- start start 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dca640640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc4100a80 0x7f3dc4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dca640640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4101420 0x7f3dc4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dca640640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dc4106a80 0x7f3dc419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dca640640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc406aeb0 con 0x7f3dc4106a80 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dca640640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc406ad30 con 0x7f3dc4101420 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dca640640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc406b030 con 0x7f3dc4100a80 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dc8bdd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dc4106a80 0x7f3dc419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dc8bdd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dc4106a80 0x7f3dc419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57696/0 (socket says 172.21.15.62:57696) 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dc8bdd640 1 -- 172.21.15.62:0/837068127 learned_addr learned my addr 172.21.15.62:0/837068127 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4101420 0x7f3dc4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dc3fff640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc4100a80 0x7f3dc4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dc8bdd640 1 -- 172.21.15.62:0/837068127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc4100a80 msgr2=0x7f3dc4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dc8bdd640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc4100a80 0x7f3dc4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dc8bdd640 1 -- 172.21.15.62:0/837068127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4101420 msgr2=0x7f3dc4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dc8bdd640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4101420 0x7f3dc4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.054+0000 7f3dc8bdd640 1 -- 172.21.15.62:0/837068127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc419d3c0 con 0x7f3dc4106a80 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.055+0000 7f3dc8bdd640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dc4106a80 0x7f3dc419cce0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00eaa0 tx=0x7f3dbc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.055+0000 7f3dc1ffb640 1 -- 172.21.15.62:0/837068127 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00cdb0 con 0x7f3dc4106a80 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.055+0000 7f3dca640640 1 -- 172.21.15.62:0/837068127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc419d6b0 con 0x7f3dc4106a80 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.055+0000 7f3dc1ffb640 1 -- 172.21.15.62:0/837068127 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dbc005020 con 0x7f3dc4106a80 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.055+0000 7f3dc1ffb640 1 -- 172.21.15.62:0/837068127 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0116d0 con 0x7f3dc4106a80 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.055+0000 7f3dca640640 1 -- 172.21.15.62:0/837068127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc41a4f90 con 0x7f3dc4106a80 2024-09-11T23:58:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.057+0000 7f3dc1ffb640 1 -- 172.21.15.62:0/837068127 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3dbc0054d0 con 0x7f3dc4106a80 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dcca19740 1 Processor -- start 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dcca19740 1 -- start start 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae32b40 0x564fbae31090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae315d0 0x564fbae31a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae3c370 0x564fbae3e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dcca19740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564fbad972f0 con 0x564fbae32b40 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dcca19740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564fbad3ecf0 con 0x564fbae3c370 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dcca19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564fbad932b0 con 0x564fbae315d0 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dbbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae32b40 0x564fbae31090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dc1ffb640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3db00a2cf0 0x7f3db00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dbbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae32b40 0x564fbae31090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57710/0 (socket says 172.21.15.62:57710) 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1658684260 learned_addr learned my addr 172.21.15.62:0/1658684260 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dc3fff640 1 --2- 172.21.15.62:0/1658684260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae315d0 0x564fbae31a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1658684260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae3c370 0x564fbae3e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dc8bdd640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3db00a2cf0 0x7f3db00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dc1ffb640 1 -- 172.21.15.62:0/837068127 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f3dbc0cf570 con 0x7f3dc4106a80 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1658684260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae315d0 msgr2=0x564fbae31a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1658684260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae315d0 0x564fbae31a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1658684260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae3c370 msgr2=0x564fbae3e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1658684260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae3c370 0x564fbae3e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.058+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1658684260 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fbad233a0 con 0x564fbae32b40 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.059+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1658684260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae32b40 0x564fbae31090 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0058a0 tx=0x7f3dac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4392e595b59d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.059+0000 7f3dc8bdd640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3db00a2cf0 0x7f3db00a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0cd2f0 tx=0x7f3dbc015840 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.059+0000 7f3dba7fc640 1 -- 172.21.15.62:0/1658684260 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac004030 con 0x564fbae32b40 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.059+0000 7f3dba7fc640 1 -- 172.21.15.62:0/1658684260 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dac0041d0 con 0x564fbae32b40 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.059+0000 7f3dba7fc640 1 -- 172.21.15.62:0/1658684260 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac00f4e0 con 0x564fbae32b40 2024-09-11T23:58:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 -- 172.21.15.62:0/1658684260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae32b40 msgr2=0x564fbae31090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1658684260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae32b40 0x564fbae31090 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0058a0 tx=0x7f3dac00b840 comp rx=0 tx=0).stop 2024-09-11T23:58:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 -- 172.21.15.62:0/1658684260 shutdown_connections 2024-09-11T23:58:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1658684260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae3c370 0x564fbae3e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1658684260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae315d0 0x564fbae31a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1658684260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae32b40 0x564fbae31090 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 -- 172.21.15.62:0/1658684260 >> 172.21.15.62:0/1658684260 conn(0x564fbadfa310 msgr2=0x564fbae2dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 -- 172.21.15.62:0/1658684260 shutdown_connections 2024-09-11T23:58:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 -- 172.21.15.62:0/1658684260 wait complete. 2024-09-11T23:58:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 -- start start 2024-09-11T23:58:12.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.060+0000 7f3dcca19740 1 Processor -- start 2024-09-11T23:58:12.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 0x564fbae39030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae3c370 0x564fbae39640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dcca19740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564fbae3f070 con 0x564fbae315d0 2024-09-11T23:58:12.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dcca19740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564fbad52060 con 0x564fbae32b40 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dcca19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564fbae3f1f0 con 0x564fbae3c370 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dcca19740 10 client.0 _ll_register_callbacks cb 0x564fbaaa58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dbbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dbbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57714/0 (socket says 172.21.15.62:57714) 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dbbfff640 1 -- 172.21.15.62:0/2546743001 learned_addr learned my addr 172.21.15.62:0/2546743001 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dc3fff640 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 0x564fbae39030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae3c370 0x564fbae39640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dc3fff640 1 -- 172.21.15.62:0/2546743001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae3c370 msgr2=0x564fbae39640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dc3fff640 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae3c370 0x564fbae39640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dc3fff640 1 -- 172.21.15.62:0/2546743001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 msgr2=0x564fbae3bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dc3fff640 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.061+0000 7f3dc3fff640 1 -- 172.21.15.62:0/2546743001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fbab083c0 con 0x564fbae32b40 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3dc3fff640 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 0x564fbae39030 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3db4005940 tx=0x7f3db4031920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0079c7f274bf1a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3da37fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3da37fe640 1 -- 172.21.15.62:0/2546743001 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4035040 con 0x564fbae32b40 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3dcca19740 1 -- 172.21.15.62:0/2546743001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dac0041d0 con 0x564fbae32b40 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3da37fe640 1 -- 172.21.15.62:0/2546743001 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f3db4004240 con 0x564fbae32b40 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3da37fe640 1 -- 172.21.15.62:0/2546743001 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3db4004440 con 0x564fbae32b40 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3dcca19740 10 client.26072 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3dcca19740 1 -- 172.21.15.62:0/2546743001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564fbae4ab90 con 0x564fbae32b40 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3da37fe640 1 -- 172.21.15.62:0/2546743001 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dac0041d0 con 0x564fbae32b40 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3da37fe640 1 -- 172.21.15.62:0/2546743001 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f3db4004be0 con 0x564fbae32b40 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3dcca19740 10 client.26072 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3dcca19740 10 client.26072 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3dcca19740 20 client.26072 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3dcca19740 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564fbae4bfa0 0x564fbae4e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3dcca19740 4 client.26072 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.062+0000 7f3dcca19740 1 -- 172.21.15.62:0/2546743001 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fbada4860 con 0x564fbae4bfa0 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.063+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564fbae4bfa0 0x564fbae4e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.063+0000 7f3da37fe640 10 client.26072 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.063+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564fbae4bfa0 0x564fbae4e3b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c06bc773c63b17c0 server_cookie=42ad1bd55b49558f in_seq=0 out_seq=0 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3da37fe640 1 -- 172.21.15.62:0/2546743001 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fbada4860 con 0x564fbae4bfa0 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3da37fe640 10 client.26072 handle_command_reply: tid=0 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 client.26072 shutdown 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 -- 172.21.15.62:0/2546743001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 msgr2=0x564fbae39030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 0x564fbae39030 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3db4005940 tx=0x7f3db4031920 comp rx=0 tx=0).stop 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 -- 172.21.15.62:0/2546743001 shutdown_connections 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564fbae4bfa0 0x564fbae4e3b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae3c370 0x564fbae39640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 0x564fbae39030 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 --2- 172.21.15.62:0/2546743001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 -- 172.21.15.62:0/2546743001 >> 172.21.15.62:0/2546743001 conn(0x564fbadfa310 msgr2=0x564fbae33960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3da37fe640 0 client.26072 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:58:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 -- 172.21.15.62:0/2546743001 shutdown_connections 2024-09-11T23:58:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 1 -- 172.21.15.62:0/2546743001 wait complete. 2024-09-11T23:58:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.064+0000 7f3dcca19740 20 client.26072 trim_cache size 0 max 16384 2024-09-11T23:58:12.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.077+0000 7f3dcca19740 1 Processor -- start 2024-09-11T23:58:12.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dcca19740 1 -- start start 2024-09-11T23:58:12.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae315d0 0x564fbae48310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 0x564fbae388f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae3c370 0x564fbae34120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dcca19740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41b0100 con 0x564fbae3c370 2024-09-11T23:58:12.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dcca19740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41ad9a0 con 0x564fbae32b40 2024-09-11T23:58:12.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dcca19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41ab7f0 con 0x564fbae315d0 2024-09-11T23:58:12.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dbbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae3c370 0x564fbae34120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dc3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 0x564fbae388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dbbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae3c370 0x564fbae34120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57724/0 (socket says 172.21.15.62:57724) 2024-09-11T23:58:12.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1088542900 learned_addr learned my addr 172.21.15.62:0/1088542900 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:12.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1088542900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae315d0 0x564fbae48310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1088542900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae315d0 msgr2=0x564fbae48310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:12.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1088542900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae315d0 0x564fbae48310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1088542900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 msgr2=0x564fbae388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:12.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.078+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1088542900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 0x564fbae388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.079+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1088542900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fbae3ea90 con 0x564fbae3c370 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.079+0000 7f3dc3fff640 1 --2- 172.21.15.62:0/1088542900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 0x564fbae388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.079+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1088542900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae3c370 0x564fbae34120 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x564fbae3a450 tx=0x7f3dac00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baebd1b2b47c03ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.079+0000 7f3db9ffb640 1 -- 172.21.15.62:0/1088542900 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac023070 con 0x564fbae3c370 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.079+0000 7f3db9ffb640 1 -- 172.21.15.62:0/1088542900 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dac029040 con 0x564fbae3c370 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.079+0000 7f3db9ffb640 1 -- 172.21.15.62:0/1088542900 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac028040 con 0x564fbae3c370 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 -- 172.21.15.62:0/1088542900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae3c370 msgr2=0x564fbae34120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1088542900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae3c370 0x564fbae34120 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x564fbae3a450 tx=0x7f3dac00b390 comp rx=0 tx=0).stop 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 -- 172.21.15.62:0/1088542900 shutdown_connections 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1088542900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae3c370 0x564fbae34120 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1088542900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae32b40 0x564fbae388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1088542900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae315d0 0x564fbae48310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 -- 172.21.15.62:0/1088542900 >> 172.21.15.62:0/1088542900 conn(0x564fbadfa310 msgr2=0x564fbae2d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 -- 172.21.15.62:0/1088542900 shutdown_connections 2024-09-11T23:58:12.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 -- 172.21.15.62:0/1088542900 wait complete. 2024-09-11T23:58:12.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 -- start start 2024-09-11T23:58:12.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.080+0000 7f3dcca19740 1 Processor -- start 2024-09-11T23:58:12.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae32b40 0x564fbae40100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dcca19740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae3c370 0x564fbaeaec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dcca19740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41abe60 con 0x564fbae315d0 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dcca19740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41ad2d0 con 0x564fbae3c370 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dcca19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41ac530 con 0x564fbae32b40 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dcca19740 10 client.0 _ll_register_callbacks cb 0x7f3db4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dbbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dbbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57734/0 (socket says 172.21.15.62:57734) 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1883703803 learned_addr learned my addr 172.21.15.62:0/1883703803 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dc3fff640 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae32b40 0x564fbae40100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae3c370 0x564fbaeaec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1883703803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae32b40 msgr2=0x564fbae40100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae32b40 0x564fbae40100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1883703803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae3c370 msgr2=0x564fbaeaec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae3c370 0x564fbaeaec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dbbfff640 1 -- 172.21.15.62:0/1883703803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fbaeaf2b0 con 0x564fbae315d0 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3dbbfff640 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3fbc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00b620 tx=0x7f3dac00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f16befba086b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.081+0000 7f3da37fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3da37fe640 1 -- 172.21.15.62:0/1883703803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac028070 con 0x564fbae315d0 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3da37fe640 1 -- 172.21.15.62:0/1883703803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f3dac0035f0 con 0x564fbae315d0 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3dcca19740 1 -- 172.21.15.62:0/1883703803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dac029040 con 0x564fbae315d0 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3da37fe640 1 -- 172.21.15.62:0/1883703803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3dac023040 con 0x564fbae315d0 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3dcca19740 10 client.16452 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:12.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3dcca19740 1 -- 172.21.15.62:0/1883703803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564fbaeb6660 con 0x564fbae315d0 2024-09-11T23:58:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3da37fe640 1 -- 172.21.15.62:0/1883703803 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dac029040 con 0x564fbae315d0 2024-09-11T23:58:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3da37fe640 1 -- 172.21.15.62:0/1883703803 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f3dac02e040 con 0x564fbae315d0 2024-09-11T23:58:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3dcca19740 10 client.16452 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3dcca19740 10 client.16452 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:58:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3dcca19740 20 client.16452 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564fbaeb6ac0 0x564fbaeb8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3dcca19740 4 client.16452 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.082+0000 7f3dcca19740 1 -- 172.21.15.62:0/1883703803 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fbad9ed70 con 0x564fbaeb6ac0 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.083+0000 7f3dc3fff640 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564fbaeb6ac0 0x564fbaeb8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.083+0000 7f3da37fe640 10 client.16452 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.083+0000 7f3dc3fff640 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564fbaeb6ac0 0x564fbaeb8ee0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4ae887b8a4e7351 server_cookie=6e7bb0d13b3dc8fc in_seq=0 out_seq=0 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.083+0000 7f3da37fe640 1 -- 172.21.15.62:0/1883703803 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fbad9ed70 con 0x564fbaeb6ac0 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.083+0000 7f3da37fe640 10 client.16452 handle_command_reply: tid=0 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.083+0000 7f3dcca19740 1 client.16452 shutdown 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 1 -- 172.21.15.62:0/1883703803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 msgr2=0x564fbae3fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3fbc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00b620 tx=0x7f3dac00ba90 comp rx=0 tx=0).stop 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 1 -- 172.21.15.62:0/1883703803 shutdown_connections 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x564fbaeb6ac0 0x564fbaeb8ee0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fbae3c370 0x564fbaeaec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564fbae32b40 0x564fbae40100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 1 --2- 172.21.15.62:0/1883703803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fbae315d0 0x564fbae3fbc0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 1 -- 172.21.15.62:0/1883703803 >> 172.21.15.62:0/1883703803 conn(0x564fbadfa310 msgr2=0x564fbae3a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3da37fe640 0 client.16452 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:58:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 1 -- 172.21.15.62:0/1883703803 shutdown_connections 2024-09-11T23:58:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 1 -- 172.21.15.62:0/1883703803 wait complete. 2024-09-11T23:58:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dcca19740 20 client.16452 trim_cache size 0 max 16384 2024-09-11T23:58:12.089 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:58:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dca640640 1 -- 172.21.15.62:0/837068127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3db00a2cf0 msgr2=0x7f3db00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dca640640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3db00a2cf0 0x7f3db00a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0cd2f0 tx=0x7f3dbc015840 comp rx=0 tx=0).stop 2024-09-11T23:58:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dca640640 1 -- 172.21.15.62:0/837068127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dc4106a80 msgr2=0x7f3dc419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.084+0000 7f3dca640640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dc4106a80 0x7f3dc419cce0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00eaa0 tx=0x7f3dbc00ef60 comp rx=0 tx=0).stop 2024-09-11T23:58:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.085+0000 7f3dca640640 1 -- 172.21.15.62:0/837068127 shutdown_connections 2024-09-11T23:58:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.085+0000 7f3dca640640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3db00a2cf0 0x7f3db00a51b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.085+0000 7f3dca640640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dc4106a80 0x7f3dc419cce0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.085+0000 7f3dca640640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4101420 0x7f3dc4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.085+0000 7f3dca640640 1 --2- 172.21.15.62:0/837068127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc4100a80 0x7f3dc4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:12.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.085+0000 7f3dca640640 1 -- 172.21.15.62:0/837068127 >> 172.21.15.62:0/837068127 conn(0x7f3dc40fc7a0 msgr2=0x7f3dc40fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:12.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.085+0000 7f3dca640640 1 -- 172.21.15.62:0/837068127 shutdown_connections 2024-09-11T23:58:12.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:12.085+0000 7f3dca640640 1 -- 172.21.15.62:0/837068127 wait complete. 2024-09-11T23:58:12.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 330.0 s, 903.5 tps, lat 29.739 ms stddev 29.742 2024-09-11T23:58:12.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:58:12.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:12 smithi062 ceph-mon[30958]: pgmap v186: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2024-09-11T23:58:13.044 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:13.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:12 smithi120 ceph-mon[35182]: pgmap v186: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2024-09-11T23:58:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:12 smithi083 ceph-mon[34805]: pgmap v186: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2024-09-11T23:58:13.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.880+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/3429417759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a041027a0 msgr2=0x7f9a041067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:13.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.880+0000 7f9a0ae5b640 1 --2- 172.21.15.62:0/3429417759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a041027a0 0x7f9a041067f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009900 tx=0x7f99f002a410 comp rx=0 tx=0).stop 2024-09-11T23:58:13.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.880+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/3429417759 shutdown_connections 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.880+0000 7f9a0ae5b640 1 --2- 172.21.15.62:0/3429417759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a04106f20 0x7f9a0410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.880+0000 7f9a0ae5b640 1 --2- 172.21.15.62:0/3429417759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a041027a0 0x7f9a041067f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.880+0000 7f9a0ae5b640 1 --2- 172.21.15.62:0/3429417759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a04101df0 0x7f9a041021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.880+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/3429417759 >> 172.21.15.62:0/3429417759 conn(0x7f9a040fd8b0 msgr2=0x7f9a040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.880+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/3429417759 shutdown_connections 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.880+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/3429417759 wait complete. 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a0ae5b640 1 Processor -- start 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a0ae5b640 1 -- start start 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a0ae5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a04101df0 0x7f9a04198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a0ae5b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a041027a0 0x7f9a04198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a0ae5b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a04106f20 0x7f9a0419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a0ae5b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04111fa0 con 0x7f9a041027a0 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a0ae5b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04111e20 con 0x7f9a04106f20 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a0ae5b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04112120 con 0x7f9a04101df0 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a03fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a041027a0 0x7f9a04198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a03fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a041027a0 0x7f9a04198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57750/0 (socket says 172.21.15.62:57750) 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a03fff640 1 -- 172.21.15.62:0/374758641 learned_addr learned my addr 172.21.15.62:0/374758641 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.881+0000 7f9a093f8640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a04106f20 0x7f9a0419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a04101df0 0x7f9a04198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a03fff640 1 -- 172.21.15.62:0/374758641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a04101df0 msgr2=0x7f9a04198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a03fff640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a04101df0 0x7f9a04198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a03fff640 1 -- 172.21.15.62:0/374758641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a04106f20 msgr2=0x7f9a0419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a03fff640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a04106f20 0x7f9a0419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a03fff640 1 -- 172.21.15.62:0/374758641 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0419d400 con 0x7f9a041027a0 2024-09-11T23:58:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a04101df0 0x7f9a04198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a03fff640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a041027a0 0x7f9a04198990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f99f0005ab0 tx=0x7f99f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a01ffb640 1 -- 172.21.15.62:0/374758641 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f003f030 con 0x7f9a041027a0 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/374758641 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0419d690 con 0x7f9a041027a0 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a01ffb640 1 -- 172.21.15.62:0/374758641 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f0003ce0 con 0x7f9a041027a0 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a01ffb640 1 -- 172.21.15.62:0/374758641 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0035070 con 0x7f9a041027a0 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.882+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/374758641 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a041a4fd0 con 0x7f9a041027a0 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a01ffb640 1 -- 172.21.15.62:0/374758641 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f99f0042030 con 0x7f9a041027a0 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a0d234740 1 Processor -- start 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a0d234740 1 -- start start 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f8438b40 0x5623f8437090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f8437a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8442370 0x5623f8444760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a0d234740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5623f839d2f0 con 0x5623f84375d0 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a0d234740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623f8344cf0 con 0x5623f8438b40 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a0d234740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5623f83992b0 con 0x5623f8442370 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a08bf7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f8437a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a08bf7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f8437a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57762/0 (socket says 172.21.15.62:57762) 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a08bf7640 1 -- 172.21.15.62:0/1330357537 learned_addr learned my addr 172.21.15.62:0/1330357537 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a03fff640 1 --2- 172.21.15.62:0/1330357537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8442370 0x5623f8444760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a01ffb640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f99f40a2cf0 0x7f99f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.885+0000 7f9a093f8640 1 --2- 172.21.15.62:0/1330357537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f8438b40 0x5623f8437090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f9a093f8640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f99f40a2cf0 0x7f99f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f9a08bf7640 1 -- 172.21.15.62:0/1330357537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8442370 msgr2=0x5623f8444760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/1330357537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8442370 0x5623f8444760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f9a08bf7640 1 -- 172.21.15.62:0/1330357537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f8438b40 msgr2=0x5623f8437090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/1330357537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f8438b40 0x5623f8437090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f9a08bf7640 1 -- 172.21.15.62:0/1330357537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f83293a0 con 0x5623f84375d0 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f9a01ffb640 1 -- 172.21.15.62:0/374758641 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f99f00efcf0 con 0x7f9a041027a0 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/1330357537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f8437a10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f99f800cca0 tx=0x7f99f800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ea16ef956afd16 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f9a093f8640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f99f40a2cf0 0x7f99f40a51b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009fd0 tx=0x7f99fc00d040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f99eeffd640 1 -- 172.21.15.62:0/1330357537 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8005220 con 0x5623f84375d0 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f99eeffd640 1 -- 172.21.15.62:0/1330357537 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f8004c80 con 0x5623f84375d0 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.886+0000 7f99eeffd640 1 -- 172.21.15.62:0/1330357537 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8010800 con 0x5623f84375d0 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 -- 172.21.15.62:0/1330357537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 msgr2=0x5623f8437a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 --2- 172.21.15.62:0/1330357537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f8437a10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f99f800cca0 tx=0x7f99f800e720 comp rx=0 tx=0).stop 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 -- 172.21.15.62:0/1330357537 shutdown_connections 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 --2- 172.21.15.62:0/1330357537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8442370 0x5623f8444760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 --2- 172.21.15.62:0/1330357537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f8437a10 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 --2- 172.21.15.62:0/1330357537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f8438b40 0x5623f8437090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 -- 172.21.15.62:0/1330357537 >> 172.21.15.62:0/1330357537 conn(0x5623f8400310 msgr2=0x5623f8433c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 -- 172.21.15.62:0/1330357537 shutdown_connections 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 -- 172.21.15.62:0/1330357537 wait complete. 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 -- start start 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.887+0000 7f9a0d234740 1 Processor -- start 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 0x5623f8441cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 0x5623f843ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f843f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a0d234740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5623f8445070 con 0x5623f8442370 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a0d234740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623f8358060 con 0x5623f84375d0 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a0d234740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5623f84451f0 con 0x5623f8438b40 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a0d234740 10 client.0 _ll_register_callbacks cb 0x5623f80ab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a093f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f843f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a093f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f843f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57764/0 (socket says 172.21.15.62:57764) 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a093f8640 1 -- 172.21.15.62:0/2341269541 learned_addr learned my addr 172.21.15.62:0/2341269541 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 0x5623f8441cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a03fff640 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 0x5623f843ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a093f8640 1 -- 172.21.15.62:0/2341269541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 msgr2=0x5623f843ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a093f8640 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 0x5623f843ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a093f8640 1 -- 172.21.15.62:0/2341269541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 msgr2=0x5623f8441cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a093f8640 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 0x5623f8441cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.888+0000 7f9a093f8640 1 -- 172.21.15.62:0/2341269541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f810e3c0 con 0x5623f8442370 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f9a093f8640 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f843f4d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f99fc018c30 tx=0x7f99fc004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faab91c5413406da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f99e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f99e3fff640 1 -- 172.21.15.62:0/2341269541 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc014040 con 0x5623f8442370 2024-09-11T23:58:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f99e3fff640 1 -- 172.21.15.62:0/2341269541 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f99fc0225f0 con 0x5623f8442370 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f9a0d234740 1 -- 172.21.15.62:0/2341269541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f8004c80 con 0x5623f8442370 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f99e3fff640 1 -- 172.21.15.62:0/2341269541 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f99fc030050 con 0x5623f8442370 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f9a0d234740 10 client.16464 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f9a0d234740 1 -- 172.21.15.62:0/2341269541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5623f8450ae0 con 0x5623f8442370 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f99e3fff640 1 -- 172.21.15.62:0/2341269541 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f8004c80 con 0x5623f8442370 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f99e3fff640 1 -- 172.21.15.62:0/2341269541 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f99fc02b660 con 0x5623f8442370 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f9a0d234740 10 client.16464 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f9a0d234740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f9a0d234740 20 client.16464 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f9a0d234740 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5623f8451fa0 0x5623f8454410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f9a0d234740 4 client.16464 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.889+0000 7f9a0d234740 1 -- 172.21.15.62:0/2341269541 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623f83aa860 con 0x5623f8451fa0 2024-09-11T23:58:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.890+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5623f8451fa0 0x5623f8454410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.890+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5623f8451fa0 0x5623f8454410 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fa431300aa3202a server_cookie=a78c2bc9a8ef7dcc in_seq=0 out_seq=0 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.890+0000 7f99e3fff640 10 client.16464 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f99e3fff640 1 -- 172.21.15.62:0/2341269541 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623f83aa860 con 0x5623f8451fa0 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f99e3fff640 10 client.16464 handle_command_reply: tid=0 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 client.16464 shutdown 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 -- 172.21.15.62:0/2341269541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 msgr2=0x5623f843f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f843f4d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f99fc018c30 tx=0x7f99fc004b50 comp rx=0 tx=0).stop 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 -- 172.21.15.62:0/2341269541 shutdown_connections 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5623f8451fa0 0x5623f8454410 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f843f4d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 0x5623f843ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 --2- 172.21.15.62:0/2341269541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 0x5623f8441cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f99e3fff640 0 client.16464 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 -- 172.21.15.62:0/2341269541 >> 172.21.15.62:0/2341269541 conn(0x5623f8400310 msgr2=0x5623f8439960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 -- 172.21.15.62:0/2341269541 shutdown_connections 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 1 -- 172.21.15.62:0/2341269541 wait complete. 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.891+0000 7f9a0d234740 20 client.16464 trim_cache size 0 max 16384 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.904+0000 7f9a0d234740 1 Processor -- start 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.904+0000 7f9a0d234740 1 -- start start 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 0x5623f844e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 0x5623f8433700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f8433c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a0d234740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041b0180 con 0x5623f8442370 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a0d234740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041ada20 con 0x5623f84375d0 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a0d234740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041ab870 con 0x5623f8438b40 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a093f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f8433c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a093f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f8433c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57770/0 (socket says 172.21.15.62:57770) 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a08bf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 0x5623f844e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a093f8640 1 -- 172.21.15.62:0/2779376525 learned_addr learned my addr 172.21.15.62:0/2779376525 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a03fff640 1 --2- 172.21.15.62:0/2779376525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 0x5623f8433700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a093f8640 1 -- 172.21.15.62:0/2779376525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 msgr2=0x5623f8433700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a093f8640 1 --2- 172.21.15.62:0/2779376525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 0x5623f8433700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a093f8640 1 -- 172.21.15.62:0/2779376525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 msgr2=0x5623f844e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a093f8640 1 --2- 172.21.15.62:0/2779376525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 0x5623f844e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a093f8640 1 -- 172.21.15.62:0/2779376525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f84334f0 con 0x5623f8442370 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a03fff640 1 --2- 172.21.15.62:0/2779376525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 0x5623f8433700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/2779376525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 0x5623f844e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f9a093f8640 1 --2- 172.21.15.62:0/2779376525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f8433c40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00af80 tx=0x7f99fc009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5b8fb0d703131 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f99ee7fc640 1 -- 172.21.15.62:0/2779376525 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc038030 con 0x5623f8442370 2024-09-11T23:58:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.905+0000 7f99ee7fc640 1 -- 172.21.15.62:0/2779376525 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99fc027070 con 0x5623f8442370 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.906+0000 7f99ee7fc640 1 -- 172.21.15.62:0/2779376525 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc034b90 con 0x5623f8442370 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.906+0000 7f9a0d234740 1 -- 172.21.15.62:0/2779376525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 msgr2=0x5623f8433c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.906+0000 7f9a0d234740 1 --2- 172.21.15.62:0/2779376525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f8433c40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00af80 tx=0x7f99fc009f30 comp rx=0 tx=0).stop 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.906+0000 7f9a0d234740 1 -- 172.21.15.62:0/2779376525 shutdown_connections 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.906+0000 7f9a0d234740 1 --2- 172.21.15.62:0/2779376525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f8442370 0x5623f8433c40 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.906+0000 7f9a0d234740 1 --2- 172.21.15.62:0/2779376525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8438b40 0x5623f8433700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.906+0000 7f9a0d234740 1 --2- 172.21.15.62:0/2779376525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f84375d0 0x5623f844e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.906+0000 7f9a0d234740 1 -- 172.21.15.62:0/2779376525 >> 172.21.15.62:0/2779376525 conn(0x5623f8400310 msgr2=0x5623f843ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.906+0000 7f9a0d234740 1 -- 172.21.15.62:0/2779376525 shutdown_connections 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.906+0000 7f9a0d234740 1 -- 172.21.15.62:0/2779376525 wait complete. 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a0d234740 1 -- start start 2024-09-11T23:58:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a0d234740 1 Processor -- start 2024-09-11T23:58:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f844a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f8438b40 0x5623f844ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a0d234740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8442370 0x5623f843bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a0d234740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041abee0 con 0x5623f84375d0 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a0d234740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041ad350 con 0x5623f8438b40 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a0d234740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041ac5b0 con 0x5623f8442370 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a0d234740 10 client.0 _ll_register_callbacks cb 0x7f99fc009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a08bf7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f844a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a08bf7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f844a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57772/0 (socket says 172.21.15.62:57772) 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a08bf7640 1 -- 172.21.15.62:0/3450303916 learned_addr learned my addr 172.21.15.62:0/3450303916 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a03fff640 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f8438b40 0x5623f844ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.907+0000 7f9a093f8640 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8442370 0x5623f843bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f9a08bf7640 1 -- 172.21.15.62:0/3450303916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8442370 msgr2=0x5623f843bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8442370 0x5623f843bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f9a08bf7640 1 -- 172.21.15.62:0/3450303916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f8438b40 msgr2=0x5623f844ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f8438b40 0x5623f844ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f9a08bf7640 1 -- 172.21.15.62:0/3450303916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f843c3c0 con 0x5623f84375d0 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f9a08bf7640 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f844a810 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f99f800cca0 tx=0x7f99f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7189cfabd71cc86d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f99e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f99e3fff640 1 -- 172.21.15.62:0/3450303916 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f802b030 con 0x5623f84375d0 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f99e3fff640 1 -- 172.21.15.62:0/3450303916 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f99f80052d0 con 0x5623f84375d0 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f9a0d234740 1 -- 172.21.15.62:0/3450303916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99fc027070 con 0x5623f84375d0 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f99e3fff640 1 -- 172.21.15.62:0/3450303916 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f99f8037050 con 0x5623f84375d0 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f9a0d234740 10 client.16470 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f9a0d234740 1 -- 172.21.15.62:0/3450303916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5623f84bc9d0 con 0x5623f84375d0 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.908+0000 7f99e3fff640 1 -- 172.21.15.62:0/3450303916 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99fc027070 con 0x5623f84375d0 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f99e3fff640 1 -- 172.21.15.62:0/3450303916 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f99f8010660 con 0x5623f84375d0 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f9a0d234740 10 client.16470 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f9a0d234740 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:13.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f9a0d234740 20 client.16470 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f9a0d234740 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5623f84bcea0 0x5623f84bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f9a0d234740 4 client.16470 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f9a0d234740 1 -- 172.21.15.62:0/3450303916 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623f83a4d70 con 0x5623f84bcea0 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f9a03fff640 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5623f84bcea0 0x5623f84bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f99e3fff640 10 client.16470 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f9a03fff640 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5623f84bcea0 0x5623f84bf2c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8735bb931486c7ba server_cookie=11bf40b2ff64a099 in_seq=0 out_seq=0 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f99e3fff640 1 -- 172.21.15.62:0/3450303916 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623f83a4d70 con 0x5623f84bcea0 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f99e3fff640 10 client.16470 handle_command_reply: tid=0 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.909+0000 7f9a0d234740 1 client.16470 shutdown 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 1 -- 172.21.15.62:0/3450303916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 msgr2=0x5623f844a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f844a810 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f99f800cca0 tx=0x7f99f8000bc0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 1 -- 172.21.15.62:0/3450303916 shutdown_connections 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5623f84bcea0 0x5623f84bf2c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f8442370 0x5623f843bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623f8438b40 0x5623f844ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 1 --2- 172.21.15.62:0/3450303916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623f84375d0 0x5623f844a810 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f99e3fff640 0 client.16470 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 1 -- 172.21.15.62:0/3450303916 >> 172.21.15.62:0/3450303916 conn(0x5623f8400310 msgr2=0x5623f84336c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 1 -- 172.21.15.62:0/3450303916 shutdown_connections 2024-09-11T23:58:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 1 -- 172.21.15.62:0/3450303916 wait complete. 2024-09-11T23:58:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0d234740 20 client.16470 trim_cache size 0 max 16384 2024-09-11T23:58:13.925 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:58:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/374758641 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f99f40a2cf0 msgr2=0x7f99f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0ae5b640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f99f40a2cf0 0x7f99f40a51b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009fd0 tx=0x7f99fc00d040 comp rx=0 tx=0).stop 2024-09-11T23:58:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/374758641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a041027a0 msgr2=0x7f9a04198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.910+0000 7f9a0ae5b640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a041027a0 0x7f9a04198990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f99f0005ab0 tx=0x7f99f00042f0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.911+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/374758641 shutdown_connections 2024-09-11T23:58:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.911+0000 7f9a0ae5b640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f99f40a2cf0 0x7f99f40a51b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.911+0000 7f9a0ae5b640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a04106f20 0x7f9a0419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.911+0000 7f9a0ae5b640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a041027a0 0x7f9a04198990 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.911+0000 7f9a0ae5b640 1 --2- 172.21.15.62:0/374758641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a04101df0 0x7f9a04198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.911+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/374758641 >> 172.21.15.62:0/374758641 conn(0x7f9a040fd8b0 msgr2=0x7f9a040fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.911+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/374758641 shutdown_connections 2024-09-11T23:58:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:13.911+0000 7f9a0ae5b640 1 -- 172.21.15.62:0/374758641 wait complete. 2024-09-11T23:58:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:14 smithi062 ceph-mon[30958]: pgmap v187: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2024-09-11T23:58:15.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:14 smithi120 ceph-mon[35182]: pgmap v187: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2024-09-11T23:58:15.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:14 smithi083 ceph-mon[34805]: pgmap v187: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2024-09-11T23:58:15.759 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:58:15.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:58:16.030 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.501+0000 7efd932a0640 1 -- 172.21.15.62:0/579731431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd8c106f10 msgr2=0x7efd8c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.501+0000 7efd932a0640 1 --2- 172.21.15.62:0/579731431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd8c106f10 0x7efd8c10ea90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efd84009ca0 tx=0x7efd8402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.501+0000 7efd932a0640 1 -- 172.21.15.62:0/579731431 shutdown_connections 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.501+0000 7efd932a0640 1 --2- 172.21.15.62:0/579731431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd8c106f10 0x7efd8c10ea90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.501+0000 7efd932a0640 1 --2- 172.21.15.62:0/579731431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd8c102790 0x7efd8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.501+0000 7efd932a0640 1 --2- 172.21.15.62:0/579731431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd8c101de0 0x7efd8c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.501+0000 7efd932a0640 1 -- 172.21.15.62:0/579731431 >> 172.21.15.62:0/579731431 conn(0x7efd8c0fd8c0 msgr2=0x7efd8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.501+0000 7efd932a0640 1 -- 172.21.15.62:0/579731431 shutdown_connections 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.501+0000 7efd932a0640 1 -- 172.21.15.62:0/579731431 wait complete. 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd932a0640 1 Processor -- start 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd932a0640 1 -- start start 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd932a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd8c101de0 0x7efd8c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd932a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd8c102790 0x7efd8c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd932a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd8c106f10 0x7efd8c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd932a0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c112010 con 0x7efd8c101de0 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd932a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c111e90 con 0x7efd8c102790 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd932a0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c112190 con 0x7efd8c106f10 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd9103c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd8c101de0 0x7efd8c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd9103c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd8c101de0 0x7efd8c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57784/0 (socket says 172.21.15.62:57784) 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.502+0000 7efd9103c640 1 -- 172.21.15.62:0/373040980 learned_addr learned my addr 172.21.15.62:0/373040980 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd9183d640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd8c106f10 0x7efd8c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd9103c640 1 -- 172.21.15.62:0/373040980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd8c106f10 msgr2=0x7efd8c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd9103c640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd8c106f10 0x7efd8c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd9083b640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd8c102790 0x7efd8c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd9103c640 1 -- 172.21.15.62:0/373040980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd8c102790 msgr2=0x7efd8c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd9103c640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd8c102790 0x7efd8c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd9103c640 1 -- 172.21.15.62:0/373040980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c19d5a0 con 0x7efd8c101de0 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd9083b640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd8c102790 0x7efd8c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd9103c640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd8c101de0 0x7efd8c1985f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efd8000b4a0 tx=0x7efd8000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd7e7fc640 1 -- 172.21.15.62:0/373040980 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80014070 con 0x7efd8c101de0 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd7e7fc640 1 -- 172.21.15.62:0/373040980 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd80010af0 con 0x7efd8c101de0 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd932a0640 1 -- 172.21.15.62:0/373040980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c19d890 con 0x7efd8c101de0 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd7e7fc640 1 -- 172.21.15.62:0/373040980 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80017d10 con 0x7efd8c101de0 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.503+0000 7efd932a0640 1 -- 172.21.15.62:0/373040980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c1a5170 con 0x7efd8c101de0 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd95679740 1 Processor -- start 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd95679740 1 -- start start 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd7e7fc640 1 -- 172.21.15.62:0/373040980 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efd80017070 con 0x7efd8c101de0 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd95679740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345cb60 0x56289345cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd95679740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345d480 0x562893467800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd95679740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 0x56289346a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd95679740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5628933c2280 con 0x56289345d480 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd95679740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562893369b10 con 0x562893467d40 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd95679740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5628933be240 con 0x56289345cb60 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd9103c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345d480 0x562893467800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd9103c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345d480 0x562893467800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57796/0 (socket says 172.21.15.62:57796) 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd9103c640 1 -- 172.21.15.62:0/3611776915 learned_addr learned my addr 172.21.15.62:0/3611776915 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd9083b640 1 --2- 172.21.15.62:0/3611776915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 0x56289346a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.506+0000 7efd9183d640 1 --2- 172.21.15.62:0/3611776915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345cb60 0x56289345cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd9103c640 1 -- 172.21.15.62:0/3611776915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345cb60 msgr2=0x56289345cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd9103c640 1 --2- 172.21.15.62:0/3611776915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345cb60 0x56289345cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd9103c640 1 -- 172.21.15.62:0/3611776915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 msgr2=0x56289346a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd9103c640 1 --2- 172.21.15.62:0/3611776915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 0x56289346a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd9103c640 1 -- 172.21.15.62:0/3611776915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628933761f0 con 0x56289345d480 2024-09-11T23:58:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd7e7fc640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efd780a2d70 0x7efd780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd9103c640 1 --2- 172.21.15.62:0/3611776915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345d480 0x562893467800 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7efd800cbe00 tx=0x7efd800d6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f87cab4ba33912 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd7e7fc640 1 -- 172.21.15.62:0/373040980 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7efd800ce600 con 0x7efd8c101de0 2024-09-11T23:58:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd9183d640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efd780a2d70 0x7efd780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd777fe640 1 -- 172.21.15.62:0/3611776915 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8001e030 con 0x56289345d480 2024-09-11T23:58:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd777fe640 1 -- 172.21.15.62:0/3611776915 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd8009b360 con 0x56289345d480 2024-09-11T23:58:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.507+0000 7efd777fe640 1 -- 172.21.15.62:0/3611776915 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8009aa30 con 0x56289345d480 2024-09-11T23:58:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.508+0000 7efd95679740 1 -- 172.21.15.62:0/3611776915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345d480 msgr2=0x562893467800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.508+0000 7efd9183d640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efd780a2d70 0x7efd780a5230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efd84009430 tx=0x7efd84005ef0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.508+0000 7efd95679740 1 --2- 172.21.15.62:0/3611776915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345d480 0x562893467800 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7efd800cbe00 tx=0x7efd800d6690 comp rx=0 tx=0).stop 2024-09-11T23:58:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.508+0000 7efd95679740 1 -- 172.21.15.62:0/3611776915 shutdown_connections 2024-09-11T23:58:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.508+0000 7efd95679740 1 --2- 172.21.15.62:0/3611776915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 0x56289346a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.508+0000 7efd95679740 1 --2- 172.21.15.62:0/3611776915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345d480 0x562893467800 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.508+0000 7efd95679740 1 --2- 172.21.15.62:0/3611776915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345cb60 0x56289345cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.508+0000 7efd95679740 1 -- 172.21.15.62:0/3611776915 >> 172.21.15.62:0/3611776915 conn(0x5628934590f0 msgr2=0x5628934594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.508+0000 7efd95679740 1 -- 172.21.15.62:0/3611776915 shutdown_connections 2024-09-11T23:58:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.508+0000 7efd95679740 1 -- 172.21.15.62:0/3611776915 wait complete. 2024-09-11T23:58:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd95679740 1 -- start start 2024-09-11T23:58:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd95679740 1 Processor -- start 2024-09-11T23:58:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd95679740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345cb60 0x562893464c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd95679740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 0x562893465150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd95679740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562893467d40 0x562893465690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd95679740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56289346a910 con 0x562893467d40 2024-09-11T23:58:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd95679740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56289337cf10 con 0x56289345d480 2024-09-11T23:58:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd95679740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56289346aa90 con 0x56289345cb60 2024-09-11T23:58:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd95679740 10 client.0 _ll_register_callbacks cb 0x562893345a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd9183d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562893467d40 0x562893465690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd9183d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562893467d40 0x562893465690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57810/0 (socket says 172.21.15.62:57810) 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd9183d640 1 -- 172.21.15.62:0/1144043685 learned_addr learned my addr 172.21.15.62:0/1144043685 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.509+0000 7efd9103c640 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345cb60 0x562893464c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd9083b640 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 0x562893465150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd9183d640 1 -- 172.21.15.62:0/1144043685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345cb60 msgr2=0x562893464c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd9183d640 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345cb60 0x562893464c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd9183d640 1 -- 172.21.15.62:0/1144043685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 msgr2=0x562893465150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd9183d640 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 0x562893465150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd9183d640 1 -- 172.21.15.62:0/1144043685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628931336a0 con 0x562893467d40 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd9083b640 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 0x562893465150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd9183d640 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562893467d40 0x562893465690 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efd84003f00 tx=0x7efd8402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4135489e9d9162cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd74ff9640 1 -- 172.21.15.62:0/1144043685 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd840045c0 con 0x562893467d40 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd74ff9640 1 -- 172.21.15.62:0/1144043685 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7efd84004760 con 0x562893467d40 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd95679740 1 -- 172.21.15.62:0/1144043685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd8009b360 con 0x562893467d40 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd74ff9640 1 -- 172.21.15.62:0/1144043685 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd8402f610 con 0x562893467d40 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd95679740 10 client.16482 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.510+0000 7efd95679740 1 -- 172.21.15.62:0/1144043685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562893476510 con 0x562893467d40 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.511+0000 7efd74ff9640 1 -- 172.21.15.62:0/1144043685 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd8009b360 con 0x562893467d40 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.511+0000 7efd74ff9640 1 -- 172.21.15.62:0/1144043685 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7efd84030990 con 0x562893467d40 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.511+0000 7efd95679740 10 client.16482 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.511+0000 7efd95679740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.511+0000 7efd95679740 20 client.16482 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.511+0000 7efd95679740 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562893477a20 0x562893479e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.511+0000 7efd95679740 4 client.16482 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.511+0000 7efd95679740 1 -- 172.21.15.62:0/1144043685 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628933cf7f0 con 0x562893477a20 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.511+0000 7efd9103c640 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562893477a20 0x562893479e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd74ff9640 10 client.16482 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd9103c640 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562893477a20 0x562893479e60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc008ba7dfbae3b server_cookie=a84fda0e83ad2d87 in_seq=0 out_seq=0 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd74ff9640 1 -- 172.21.15.62:0/1144043685 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628933cf7f0 con 0x562893477a20 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd74ff9640 10 client.16482 handle_command_reply: tid=0 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd95679740 1 client.16482 shutdown 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd95679740 1 -- 172.21.15.62:0/1144043685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562893467d40 msgr2=0x562893465690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd95679740 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562893467d40 0x562893465690 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efd84003f00 tx=0x7efd8402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd95679740 1 -- 172.21.15.62:0/1144043685 shutdown_connections 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd95679740 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562893477a20 0x562893479e60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd95679740 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562893467d40 0x562893465690 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd95679740 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 0x562893465150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.512+0000 7efd95679740 1 --2- 172.21.15.62:0/1144043685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345cb60 0x562893464c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.513+0000 7efd74ff9640 0 client.16482 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:16.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.513+0000 7efd95679740 1 -- 172.21.15.62:0/1144043685 >> 172.21.15.62:0/1144043685 conn(0x5628934590f0 msgr2=0x56289345edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:16.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.513+0000 7efd95679740 1 -- 172.21.15.62:0/1144043685 shutdown_connections 2024-09-11T23:58:16.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.513+0000 7efd95679740 1 -- 172.21.15.62:0/1144043685 wait complete. 2024-09-11T23:58:16.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.513+0000 7efd95679740 20 client.16482 trim_cache size 0 max 16384 2024-09-11T23:58:16.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd95679740 1 Processor -- start 2024-09-11T23:58:16.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd95679740 1 -- start start 2024-09-11T23:58:16.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd95679740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289345ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd95679740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345d480 0x56289347f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd95679740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 0x56289347fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd95679740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1b0220 con 0x56289345cb60 2024-09-11T23:58:16.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd95679740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1adac0 con 0x562893467d40 2024-09-11T23:58:16.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd95679740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1ab910 con 0x56289345d480 2024-09-11T23:58:16.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd9103c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289345ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd9103c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289345ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57820/0 (socket says 172.21.15.62:57820) 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd9103c640 1 -- 172.21.15.62:0/1355370555 learned_addr learned my addr 172.21.15.62:0/1355370555 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.530+0000 7efd9183d640 1 --2- 172.21.15.62:0/1355370555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 0x56289347fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd9083b640 1 --2- 172.21.15.62:0/1355370555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345d480 0x56289347f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd9103c640 1 -- 172.21.15.62:0/1355370555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345d480 msgr2=0x56289347f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd9103c640 1 --2- 172.21.15.62:0/1355370555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345d480 0x56289347f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd9103c640 1 -- 172.21.15.62:0/1355370555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 msgr2=0x56289347fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd9103c640 1 --2- 172.21.15.62:0/1355370555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 0x56289347fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd9103c640 1 -- 172.21.15.62:0/1355370555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562893459d80 con 0x56289345cb60 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd9183d640 1 --2- 172.21.15.62:0/1355370555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 0x56289347fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd9103c640 1 --2- 172.21.15.62:0/1355370555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289345ecc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7efd800cbf40 tx=0x7efd800ce900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdeda82303bf5320 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd76ffd640 1 -- 172.21.15.62:0/1355370555 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8001e040 con 0x56289345cb60 2024-09-11T23:58:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd76ffd640 1 -- 172.21.15.62:0/1355370555 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd800d4320 con 0x56289345cb60 2024-09-11T23:58:16.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.531+0000 7efd76ffd640 1 -- 172.21.15.62:0/1355370555 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8009a5c0 con 0x56289345cb60 2024-09-11T23:58:16.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 -- 172.21.15.62:0/1355370555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 msgr2=0x56289345ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 --2- 172.21.15.62:0/1355370555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289345ecc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7efd800cbf40 tx=0x7efd800ce900 comp rx=0 tx=0).stop 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 -- 172.21.15.62:0/1355370555 shutdown_connections 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 --2- 172.21.15.62:0/1355370555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562893467d40 0x56289347fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 --2- 172.21.15.62:0/1355370555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56289345d480 0x56289347f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 --2- 172.21.15.62:0/1355370555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289345ecc0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 -- 172.21.15.62:0/1355370555 >> 172.21.15.62:0/1355370555 conn(0x5628934590f0 msgr2=0x56289345e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 -- 172.21.15.62:0/1355370555 shutdown_connections 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 -- 172.21.15.62:0/1355370555 wait complete. 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 -- start start 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.532+0000 7efd95679740 1 Processor -- start 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd95679740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289346c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd95679740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 0x56289346c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd95679740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562893467d40 0x5628934e1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd95679740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1abf80 con 0x56289345cb60 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd95679740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1ad3f0 con 0x56289345d480 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd95679740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1ac650 con 0x562893467d40 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd95679740 10 client.0 _ll_register_callbacks cb 0x7efd84009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9103c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289346c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9183d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562893467d40 0x5628934e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9083b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 0x56289346c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9103c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289346c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57836/0 (socket says 172.21.15.62:57836) 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9103c640 1 -- 172.21.15.62:0/2847480624 learned_addr learned my addr 172.21.15.62:0/2847480624 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9103c640 1 -- 172.21.15.62:0/2847480624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562893467d40 msgr2=0x5628934e1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9103c640 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562893467d40 0x5628934e1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9103c640 1 -- 172.21.15.62:0/2847480624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 msgr2=0x56289346c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9103c640 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 0x56289346c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9103c640 1 -- 172.21.15.62:0/2847480624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628934e1c40 con 0x56289345cb60 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd9103c640 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289346c0f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efd80009420 tx=0x7efd800d4550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5288bcacd4146c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.533+0000 7efd74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd74ff9640 1 -- 172.21.15.62:0/2847480624 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd800b8060 con 0x56289345cb60 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd74ff9640 1 -- 172.21.15.62:0/2847480624 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7efd80017360 con 0x56289345cb60 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd95679740 1 -- 172.21.15.62:0/2847480624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd800d4320 con 0x56289345cb60 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd74ff9640 1 -- 172.21.15.62:0/2847480624 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd800b3e60 con 0x56289345cb60 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd95679740 10 client.16488 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd95679740 1 -- 172.21.15.62:0/2847480624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5628934e2ee0 con 0x56289345cb60 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd74ff9640 1 -- 172.21.15.62:0/2847480624 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd800d4320 con 0x56289345cb60 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd74ff9640 1 -- 172.21.15.62:0/2847480624 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7efd800972d0 con 0x56289345cb60 2024-09-11T23:58:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd95679740 10 client.16488 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd95679740 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd95679740 20 client.16488 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd95679740 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5628934e2ee0 0x5628934e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd95679740 4 client.16488 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.534+0000 7efd95679740 1 -- 172.21.15.62:0/2847480624 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5628933c9d00 con 0x5628934e2ee0 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.535+0000 7efd9083b640 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5628934e2ee0 0x5628934e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.535+0000 7efd9083b640 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5628934e2ee0 0x5628934e52d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ec6e949af1b682 server_cookie=649b30ac7ec606ce in_seq=0 out_seq=0 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.535+0000 7efd74ff9640 10 client.16488 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd74ff9640 1 -- 172.21.15.62:0/2847480624 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5628933c9d00 con 0x5628934e2ee0 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd74ff9640 10 client.16488 handle_command_reply: tid=0 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 client.16488 shutdown 2024-09-11T23:58:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 -- 172.21.15.62:0/2847480624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 msgr2=0x56289346c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289346c0f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efd80009420 tx=0x7efd800d4550 comp rx=0 tx=0).stop 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 -- 172.21.15.62:0/2847480624 shutdown_connections 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5628934e2ee0 0x5628934e52d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562893467d40 0x5628934e1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56289345d480 0x56289346c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 --2- 172.21.15.62:0/2847480624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56289345cb60 0x56289346c0f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd74ff9640 0 client.16488 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 -- 172.21.15.62:0/2847480624 >> 172.21.15.62:0/2847480624 conn(0x5628934590f0 msgr2=0x562893458270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 -- 172.21.15.62:0/2847480624 shutdown_connections 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.536+0000 7efd95679740 1 -- 172.21.15.62:0/2847480624 wait complete. 2024-09-11T23:58:16.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd95679740 20 client.16488 trim_cache size 0 max 16384 2024-09-11T23:58:16.542 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:58:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:58:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "ddd3fd6d-1a64-48e9-a780-47064ddf289d", 2024-09-11T23:58:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:58:16.542 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:58:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 -- 172.21.15.62:0/373040980 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efd780a2d70 msgr2=0x7efd780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efd780a2d70 0x7efd780a5230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efd84009430 tx=0x7efd84005ef0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 -- 172.21.15.62:0/373040980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd8c101de0 msgr2=0x7efd8c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd8c101de0 0x7efd8c1985f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efd8000b4a0 tx=0x7efd8000b960 comp rx=0 tx=0).stop 2024-09-11T23:58:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 -- 172.21.15.62:0/373040980 shutdown_connections 2024-09-11T23:58:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efd780a2d70 0x7efd780a5230 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd8c106f10 0x7efd8c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd8c102790 0x7efd8c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 --2- 172.21.15.62:0/373040980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd8c101de0 0x7efd8c1985f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 -- 172.21.15.62:0/373040980 >> 172.21.15.62:0/373040980 conn(0x7efd8c0fd8c0 msgr2=0x7efd8c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 -- 172.21.15.62:0/373040980 shutdown_connections 2024-09-11T23:58:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:16.537+0000 7efd932a0640 1 -- 172.21.15.62:0/373040980 wait complete. 2024-09-11T23:58:16.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:16 smithi062 ceph-mon[30958]: pgmap v188: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.0 MiB/s wr, 406 op/s 2024-09-11T23:58:17.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:16 smithi120 ceph-mon[35182]: pgmap v188: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.0 MiB/s wr, 406 op/s 2024-09-11T23:58:17.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:16 smithi083 ceph-mon[34805]: pgmap v188: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.0 MiB/s wr, 406 op/s 2024-09-11T23:58:17.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:58:17.447 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 335.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:58:17.625 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:18.009 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:17 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:58:18.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:17 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:17 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:58:18.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:17 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:58:18.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:17 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:17 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:58:18.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:17 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:17 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.094+0000 7f926c74c640 1 -- 172.21.15.62:0/2961965726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9264106a80 msgr2=0x7f9264106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.094+0000 7f926c74c640 1 --2- 172.21.15.62:0/2961965726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9264106a80 0x7f9264106e60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9260009930 tx=0x7f926002a420 comp rx=0 tx=0).stop 2024-09-11T23:58:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.094+0000 7f926c74c640 1 -- 172.21.15.62:0/2961965726 shutdown_connections 2024-09-11T23:58:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.094+0000 7f926c74c640 1 --2- 172.21.15.62:0/2961965726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9264101420 0x7f926410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.094+0000 7f926c74c640 1 --2- 172.21.15.62:0/2961965726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264100a80 0x7f9264100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.094+0000 7f926c74c640 1 --2- 172.21.15.62:0/2961965726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9264106a80 0x7f9264106e60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.094+0000 7f926c74c640 1 -- 172.21.15.62:0/2961965726 >> 172.21.15.62:0/2961965726 conn(0x7f92640fc7a0 msgr2=0x7f92640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.094+0000 7f926c74c640 1 -- 172.21.15.62:0/2961965726 shutdown_connections 2024-09-11T23:58:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.094+0000 7f926c74c640 1 -- 172.21.15.62:0/2961965726 wait complete. 2024-09-11T23:58:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926c74c640 1 Processor -- start 2024-09-11T23:58:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926c74c640 1 -- start start 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926c74c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9264100a80 0x7f9264199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926c74c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264101420 0x7f92641997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926c74c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9264106a80 0x7f92641a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926c74c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9264110e80 con 0x7f9264100a80 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926c74c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9264110d00 con 0x7f9264106a80 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926c74c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9264111000 con 0x7f9264101420 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926a4e8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9264100a80 0x7f9264199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926a4e8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9264100a80 0x7f9264199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57846/0 (socket says 172.21.15.62:57846) 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926a4e8640 1 -- 172.21.15.62:0/2461877406 learned_addr learned my addr 172.21.15.62:0/2461877406 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f9269ce7640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264101420 0x7f92641997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926ace9640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9264106a80 0x7f92641a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926a4e8640 1 -- 172.21.15.62:0/2461877406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264101420 msgr2=0x7f92641997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926a4e8640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264101420 0x7f92641997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926a4e8640 1 -- 172.21.15.62:0/2461877406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9264106a80 msgr2=0x7f92641a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926a4e8640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9264106a80 0x7f92641a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926a4e8640 1 -- 172.21.15.62:0/2461877406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92641a4830 con 0x7f9264100a80 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.095+0000 7f926ace9640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9264106a80 0x7f92641a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.096+0000 7f926a4e8640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9264100a80 0x7f9264199270 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9260009a50 tx=0x7f9260004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.096+0000 7f92577fe640 1 -- 172.21.15.62:0/2461877406 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926003f030 con 0x7f9264100a80 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.096+0000 7f92577fe640 1 -- 172.21.15.62:0/2461877406 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9260003ce0 con 0x7f9264100a80 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.096+0000 7f926c74c640 1 -- 172.21.15.62:0/2461877406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92641a4a00 con 0x7f9264100a80 2024-09-11T23:58:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.096+0000 7f92577fe640 1 -- 172.21.15.62:0/2461877406 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260035070 con 0x7f9264100a80 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.096+0000 7f926c74c640 1 -- 172.21.15.62:0/2461877406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92641a4e50 con 0x7f9264100a80 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.098+0000 7f926eb25740 1 Processor -- start 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.098+0000 7f92577fe640 1 -- 172.21.15.62:0/2461877406 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9260042030 con 0x7f9264100a80 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.098+0000 7f926eb25740 1 -- start start 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f926eb25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8b4a0 0x55fae1a8b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f926eb25740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8bdc0 0x55fae1a96140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f926eb25740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a98a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f926eb25740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fae19f12e0 con 0x55fae1a96680 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f926eb25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fae1998ce0 con 0x55fae1a8b4a0 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f926eb25740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fae19ed2a0 con 0x55fae1a8bdc0 2024-09-11T23:58:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f9269ce7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a98a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f926a4e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8bdc0 0x55fae1a96140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f9269ce7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a98a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57850/0 (socket says 172.21.15.62:57850) 2024-09-11T23:58:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f9269ce7640 1 -- 172.21.15.62:0/2135062393 learned_addr learned my addr 172.21.15.62:0/2135062393 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f926ace9640 1 --2- 172.21.15.62:0/2135062393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8b4a0 0x55fae1a8b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f92577fe640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f92500a2cf0 0x7f92500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f9269ce7640 1 -- 172.21.15.62:0/2135062393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8bdc0 msgr2=0x55fae1a96140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f9269ce7640 1 --2- 172.21.15.62:0/2135062393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8bdc0 0x55fae1a96140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f9269ce7640 1 -- 172.21.15.62:0/2135062393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8b4a0 msgr2=0x55fae1a8b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f9269ce7640 1 --2- 172.21.15.62:0/2135062393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8b4a0 0x55fae1a8b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f9269ce7640 1 -- 172.21.15.62:0/2135062393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae197d440 con 0x55fae1a96680 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f926ace9640 1 --2- 172.21.15.62:0/2135062393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8b4a0 0x55fae1a8b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f926ace9640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f92500a2cf0 0x7f92500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.099+0000 7f92577fe640 1 -- 172.21.15.62:0/2461877406 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f92600efa60 con 0x7f9264100a80 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.100+0000 7f9269ce7640 1 --2- 172.21.15.62:0/2135062393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a98a70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f925800cba0 tx=0x7f925800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9890b4892638f449 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.100+0000 7f9254ff9640 1 -- 172.21.15.62:0/2135062393 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258010630 con 0x55fae1a96680 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.100+0000 7f9254ff9640 1 -- 172.21.15.62:0/2135062393 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9258010cb0 con 0x55fae1a96680 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.100+0000 7f9254ff9640 1 -- 172.21.15.62:0/2135062393 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925800fc80 con 0x55fae1a96680 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.100+0000 7f926eb25740 1 -- 172.21.15.62:0/2135062393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 msgr2=0x55fae1a98a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.100+0000 7f926eb25740 1 --2- 172.21.15.62:0/2135062393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a98a70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f925800cba0 tx=0x7f925800e720 comp rx=0 tx=0).stop 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.100+0000 7f926ace9640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f92500a2cf0 0x7f92500a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f925c005cc0 tx=0x7f925c004a40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:18.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.101+0000 7f926eb25740 1 -- 172.21.15.62:0/2135062393 shutdown_connections 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.101+0000 7f926eb25740 1 --2- 172.21.15.62:0/2135062393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a98a70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.101+0000 7f926eb25740 1 --2- 172.21.15.62:0/2135062393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8bdc0 0x55fae1a96140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.101+0000 7f926eb25740 1 --2- 172.21.15.62:0/2135062393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8b4a0 0x55fae1a8b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.101+0000 7f926eb25740 1 -- 172.21.15.62:0/2135062393 >> 172.21.15.62:0/2135062393 conn(0x55fae1a87a30 msgr2=0x55fae1a87e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.101+0000 7f926eb25740 1 -- 172.21.15.62:0/2135062393 shutdown_connections 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.101+0000 7f926eb25740 1 -- 172.21.15.62:0/2135062393 wait complete. 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.101+0000 7f926eb25740 1 -- start start 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.101+0000 7f926eb25740 1 Processor -- start 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926eb25740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8b4a0 0x55fae1a95f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926eb25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8bdc0 0x55fae1a93200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926eb25740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a93740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926eb25740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fae1a99250 con 0x55fae1a96680 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926eb25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fae19ac050 con 0x55fae1a8bdc0 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926eb25740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fae1a993d0 con 0x55fae1a8b4a0 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926eb25740 10 client.0 _ll_register_callbacks cb 0x55fae19ec160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926ace9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a93740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926ace9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a93740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57866/0 (socket says 172.21.15.62:57866) 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926ace9640 1 -- 172.21.15.62:0/1746356108 learned_addr learned my addr 172.21.15.62:0/1746356108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926a4e8640 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8b4a0 0x55fae1a95f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f9269ce7640 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8bdc0 0x55fae1a93200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926ace9640 1 -- 172.21.15.62:0/1746356108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8b4a0 msgr2=0x55fae1a95f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926ace9640 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8b4a0 0x55fae1a95f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926ace9640 1 -- 172.21.15.62:0/1746356108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8bdc0 msgr2=0x55fae1a93200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926ace9640 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8bdc0 0x55fae1a93200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926ace9640 1 -- 172.21.15.62:0/1746356108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae17623c0 con 0x55fae1a96680 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926ace9640 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a93740 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f925c0119d0 tx=0x7f925c0084c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56eb51e18b2828f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f924dffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f924dffb640 1 -- 172.21.15.62:0/1746356108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c01ce40 con 0x55fae1a96680 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f924dffb640 1 -- 172.21.15.62:0/1746356108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f925c01a450 con 0x55fae1a96680 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.102+0000 7f926eb25740 1 -- 172.21.15.62:0/1746356108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9258010cb0 con 0x55fae1a96680 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f924dffb640 1 -- 172.21.15.62:0/1746356108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f925c01a650 con 0x55fae1a96680 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f926eb25740 10 client.16500 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f926eb25740 1 -- 172.21.15.62:0/1746356108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fae1aa4c50 con 0x55fae1a96680 2024-09-11T23:58:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f924dffb640 1 -- 172.21.15.62:0/1746356108 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9258010cb0 con 0x55fae1a96680 2024-09-11T23:58:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f924dffb640 1 -- 172.21.15.62:0/1746356108 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f925c023070 con 0x55fae1a96680 2024-09-11T23:58:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f926eb25740 10 client.16500 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f926eb25740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f926eb25740 20 client.16500 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f926eb25740 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fae1aa6160 0x55fae1aa85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f926eb25740 4 client.16500 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f926eb25740 1 -- 172.21.15.62:0/1746356108 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fae19fe850 con 0x55fae1aa6160 2024-09-11T23:58:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.103+0000 7f926a4e8640 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fae1aa6160 0x55fae1aa85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.104+0000 7f926a4e8640 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fae1aa6160 0x55fae1aa85d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc84179396425b58 server_cookie=455f9a92f9bd3540 in_seq=0 out_seq=0 2024-09-11T23:58:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.104+0000 7f924dffb640 10 client.16500 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:18.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.104+0000 7f924dffb640 1 -- 172.21.15.62:0/1746356108 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fae19fe850 con 0x55fae1aa6160 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.104+0000 7f924dffb640 10 client.16500 handle_command_reply: tid=0 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 client.16500 shutdown 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 -- 172.21.15.62:0/1746356108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 msgr2=0x55fae1a93740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a93740 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f925c0119d0 tx=0x7f925c0084c0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 -- 172.21.15.62:0/1746356108 shutdown_connections 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fae1aa6160 0x55fae1aa85d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a96680 0x55fae1a93740 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8bdc0 0x55fae1a93200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 --2- 172.21.15.62:0/1746356108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8b4a0 0x55fae1a95f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 -- 172.21.15.62:0/1746356108 >> 172.21.15.62:0/1746356108 conn(0x55fae1a87a30 msgr2=0x55fae1a8d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f924dffb640 0 client.16500 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 -- 172.21.15.62:0/1746356108 shutdown_connections 2024-09-11T23:58:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 1 -- 172.21.15.62:0/1746356108 wait complete. 2024-09-11T23:58:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.105+0000 7f926eb25740 20 client.16500 trim_cache size 0 max 16384 2024-09-11T23:58:18.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:17 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:58:18.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:17 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:17 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:58:18.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:17 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:58:18.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:17 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:17 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:58:18.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:17 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.123 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:17 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926eb25740 1 Processor -- start 2024-09-11T23:58:18.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926eb25740 1 -- start start 2024-09-11T23:58:18.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926eb25740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926eb25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8bdc0 0x55fae1a95170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926eb25740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a96680 0x55fae1aadea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926eb25740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f92641b0010 con 0x55fae1a8b4a0 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926eb25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92641ad8b0 con 0x55fae1a8bdc0 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926eb25740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92641ab700 con 0x55fae1a96680 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926a4e8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926a4e8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a933f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57874/0 (socket says 172.21.15.62:57874) 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f926a4e8640 1 -- 172.21.15.62:0/303640452 learned_addr learned my addr 172.21.15.62:0/303640452 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.122+0000 7f9269ce7640 1 --2- 172.21.15.62:0/303640452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8bdc0 0x55fae1a95170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.123+0000 7f926ace9640 1 --2- 172.21.15.62:0/303640452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a96680 0x55fae1aadea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.123+0000 7f926a4e8640 1 -- 172.21.15.62:0/303640452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a96680 msgr2=0x55fae1aadea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.123+0000 7f926a4e8640 1 --2- 172.21.15.62:0/303640452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a96680 0x55fae1aadea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.123+0000 7f926a4e8640 1 -- 172.21.15.62:0/303640452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8bdc0 msgr2=0x55fae1a95170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.123+0000 7f926a4e8640 1 --2- 172.21.15.62:0/303640452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8bdc0 0x55fae1a95170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.123+0000 7f926a4e8640 1 -- 172.21.15.62:0/303640452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae1a921b0 con 0x55fae1a8b4a0 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.123+0000 7f926a4e8640 1 --2- 172.21.15.62:0/303640452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a933f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9260031550 tx=0x7f92600dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad5d3afad24ab3e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.123+0000 7f924ffff640 1 -- 172.21.15.62:0/303640452 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260049040 con 0x55fae1a8b4a0 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.123+0000 7f924ffff640 1 -- 172.21.15.62:0/303640452 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92600bc6b0 con 0x55fae1a8b4a0 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.123+0000 7f924ffff640 1 -- 172.21.15.62:0/303640452 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92600c4c20 con 0x55fae1a8b4a0 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 -- 172.21.15.62:0/303640452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 msgr2=0x55fae1a933f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 --2- 172.21.15.62:0/303640452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a933f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9260031550 tx=0x7f92600dc040 comp rx=0 tx=0).stop 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 -- 172.21.15.62:0/303640452 shutdown_connections 2024-09-11T23:58:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 --2- 172.21.15.62:0/303640452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a96680 0x55fae1aadea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 --2- 172.21.15.62:0/303640452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a8bdc0 0x55fae1a95170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 --2- 172.21.15.62:0/303640452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a933f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 -- 172.21.15.62:0/303640452 >> 172.21.15.62:0/303640452 conn(0x55fae1a87a30 msgr2=0x55fae1a94a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 -- 172.21.15.62:0/303640452 shutdown_connections 2024-09-11T23:58:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 -- 172.21.15.62:0/303640452 wait complete. 2024-09-11T23:58:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 -- start start 2024-09-11T23:58:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 Processor -- start 2024-09-11T23:58:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a9ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8bdc0 0x55fae1a9bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a96680 0x55fae1a9c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f92641abd70 con 0x55fae1a8b4a0 2024-09-11T23:58:18.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92641ad1e0 con 0x55fae1a96680 2024-09-11T23:58:18.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92641ac440 con 0x55fae1a8bdc0 2024-09-11T23:58:18.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.124+0000 7f926eb25740 10 client.0 _ll_register_callbacks cb 0x7f925c00be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:18.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926a4e8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a9ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926a4e8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a9ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57890/0 (socket says 172.21.15.62:57890) 2024-09-11T23:58:18.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f9269ce7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8bdc0 0x55fae1a9bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926a4e8640 1 -- 172.21.15.62:0/3115891408 learned_addr learned my addr 172.21.15.62:0/3115891408 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926ace9640 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a96680 0x55fae1a9c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926a4e8640 1 -- 172.21.15.62:0/3115891408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8bdc0 msgr2=0x55fae1a9bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926a4e8640 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8bdc0 0x55fae1a9bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926a4e8640 1 -- 172.21.15.62:0/3115891408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a96680 msgr2=0x55fae1a9c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926a4e8640 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a96680 0x55fae1a9c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926a4e8640 1 -- 172.21.15.62:0/3115891408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae1a9cad0 con 0x55fae1a8b4a0 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926a4e8640 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a9ebd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f92600dc010 tx=0x7f92600ed2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba479473f9ab11cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f924dffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f924dffb640 1 -- 172.21.15.62:0/3115891408 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260049070 con 0x55fae1a8b4a0 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f924dffb640 1 -- 172.21.15.62:0/3115891408 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f92600ed460 con 0x55fae1a8b4a0 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926eb25740 1 -- 172.21.15.62:0/3115891408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92600bc6b0 con 0x55fae1a8b4a0 2024-09-11T23:58:18.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f924dffb640 1 -- 172.21.15.62:0/3115891408 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f92600bb910 con 0x55fae1a8b4a0 2024-09-11T23:58:18.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926eb25740 10 client.16506 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:18.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.125+0000 7f926eb25740 1 -- 172.21.15.62:0/3115891408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fae1b10d00 con 0x55fae1a8b4a0 2024-09-11T23:58:18.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f924dffb640 1 -- 172.21.15.62:0/3115891408 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92600bc6b0 con 0x55fae1a8b4a0 2024-09-11T23:58:18.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f924dffb640 1 -- 172.21.15.62:0/3115891408 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f92600d8030 con 0x55fae1a8b4a0 2024-09-11T23:58:18.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f926eb25740 10 client.16506 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:18.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f926eb25740 10 client.16506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:18.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f926eb25740 20 client.16506 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:18.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f926eb25740 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fae1b11370 0x55fae1b13760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:18.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f926eb25740 4 client.16506 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:58:18.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f926eb25740 1 -- 172.21.15.62:0/3115891408 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fae1a93ad0 con 0x55fae1b11370 2024-09-11T23:58:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f9269ce7640 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fae1b11370 0x55fae1b13760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f924dffb640 10 client.16506 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.126+0000 7f9269ce7640 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fae1b11370 0x55fae1b13760 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcbf3f8eecd261ba server_cookie=1ed4649918e00226 in_seq=0 out_seq=0 2024-09-11T23:58:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f924dffb640 1 -- 172.21.15.62:0/3115891408 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fae1a93ad0 con 0x55fae1b11370 2024-09-11T23:58:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f924dffb640 10 client.16506 handle_command_reply: tid=0 2024-09-11T23:58:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 client.16506 shutdown 2024-09-11T23:58:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 -- 172.21.15.62:0/3115891408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 msgr2=0x55fae1a9ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a9ebd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f92600dc010 tx=0x7f92600ed2c0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 -- 172.21.15.62:0/3115891408 shutdown_connections 2024-09-11T23:58:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fae1b11370 0x55fae1b13760 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fae1a96680 0x55fae1a9c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fae1a8bdc0 0x55fae1a9bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 --2- 172.21.15.62:0/3115891408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fae1a8b4a0 0x55fae1a9ebd0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 -- 172.21.15.62:0/3115891408 >> 172.21.15.62:0/3115891408 conn(0x55fae1a87a30 msgr2=0x55fae1a95280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f924dffb640 0 client.16506 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 -- 172.21.15.62:0/3115891408 shutdown_connections 2024-09-11T23:58:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 1 -- 172.21.15.62:0/3115891408 wait complete. 2024-09-11T23:58:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926eb25740 20 client.16506 trim_cache size 0 max 16384 2024-09-11T23:58:18.138 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:58:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:58:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:58:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "ddd3fd6d-1a64-48e9-a780-47064ddf289d": { 2024-09-11T23:58:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:58:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "ddd3fd6d-1a64-48e9-a780-47064ddf289d", 2024-09-11T23:58:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:58:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:18.139 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:18.139 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:58:18.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926c74c640 1 -- 172.21.15.62:0/2461877406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f92500a2cf0 msgr2=0x7f92500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:18.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926c74c640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f92500a2cf0 0x7f92500a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f925c005cc0 tx=0x7f925c004a40 comp rx=0 tx=0).stop 2024-09-11T23:58:18.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926c74c640 1 -- 172.21.15.62:0/2461877406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9264100a80 msgr2=0x7f9264199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:18.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.127+0000 7f926c74c640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9264100a80 0x7f9264199270 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9260009a50 tx=0x7f9260004270 comp rx=0 tx=0).stop 2024-09-11T23:58:18.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.128+0000 7f926c74c640 1 -- 172.21.15.62:0/2461877406 shutdown_connections 2024-09-11T23:58:18.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.128+0000 7f926c74c640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f92500a2cf0 0x7f92500a51b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.128+0000 7f926c74c640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9264106a80 0x7f92641a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.128+0000 7f926c74c640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9264101420 0x7f92641997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.128+0000 7f926c74c640 1 --2- 172.21.15.62:0/2461877406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9264100a80 0x7f9264199270 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:18.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.128+0000 7f926c74c640 1 -- 172.21.15.62:0/2461877406 >> 172.21.15.62:0/2461877406 conn(0x7f92640fc7a0 msgr2=0x7f92640fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:18.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.128+0000 7f926c74c640 1 -- 172.21.15.62:0/2461877406 shutdown_connections 2024-09-11T23:58:18.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:18.128+0000 7f926c74c640 1 -- 172.21.15.62:0/2461877406 wait complete. 2024-09-11T23:58:18.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:17 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:58:18.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:17 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:17 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:58:18.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:17 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:58:18.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:17 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:17 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:58:18.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:17 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:18.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:17 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:58:19.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:18 smithi120 ceph-mon[35182]: pgmap v189: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.0 MiB/s wr, 410 op/s 2024-09-11T23:58:19.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:18 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:58:19.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:18 smithi083 ceph-mon[34805]: pgmap v189: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.0 MiB/s wr, 410 op/s 2024-09-11T23:58:19.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:18 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:58:19.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:18 smithi062 ceph-mon[30958]: pgmap v189: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.0 MiB/s wr, 410 op/s 2024-09-11T23:58:19.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:18 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:58:19.494 INFO:tasks.cephfs.filesystem:scrub status for tag:ddd3fd6d-1a64-48e9-a780-47064ddf289d - {'path': '/', 'tag': 'ddd3fd6d-1a64-48e9-a780-47064ddf289d', 'options': 'recursive,force'} 2024-09-11T23:58:20.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:19 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:58:20.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:19 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:58:20.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:19 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:58:20.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:19 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:58:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:19 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:58:20.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:19 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:58:20.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:19 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:58:20.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:19 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:58:20.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:19 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:58:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:21 smithi120 ceph-mon[35182]: pgmap v190: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-11T23:58:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:21 smithi083 ceph-mon[34805]: pgmap v190: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-11T23:58:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:21 smithi062 ceph-mon[30958]: pgmap v190: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-11T23:58:22.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 340.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:58:23.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:23 smithi120 ceph-mon[35182]: pgmap v191: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-11T23:58:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:23 smithi083 ceph-mon[34805]: pgmap v191: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-11T23:58:23.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:23 smithi062 ceph-mon[30958]: pgmap v191: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-11T23:58:24.310 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:58:24.581 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:25.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.045+0000 7f2097fb9640 1 -- 172.21.15.62:0/3788802668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2090102d40 msgr2=0x7f2090103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:25.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.045+0000 7f2097fb9640 1 --2- 172.21.15.62:0/3788802668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2090102d40 0x7f2090103120 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2084009900 tx=0x7f208402a480 comp rx=0 tx=0).stop 2024-09-11T23:58:25.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.045+0000 7f2097fb9640 1 -- 172.21.15.62:0/3788802668 shutdown_connections 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.045+0000 7f2097fb9640 1 --2- 172.21.15.62:0/3788802668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f209010c170 0x7f209010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.045+0000 7f2097fb9640 1 --2- 172.21.15.62:0/3788802668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090103660 0x7f209010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.045+0000 7f2097fb9640 1 --2- 172.21.15.62:0/3788802668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2090102d40 0x7f2090103120 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.045+0000 7f2097fb9640 1 -- 172.21.15.62:0/3788802668 >> 172.21.15.62:0/3788802668 conn(0x7f20900fc7a0 msgr2=0x7f20900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.045+0000 7f2097fb9640 1 -- 172.21.15.62:0/3788802668 shutdown_connections 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.045+0000 7f2097fb9640 1 -- 172.21.15.62:0/3788802668 wait complete. 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2097fb9640 1 Processor -- start 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2097fb9640 1 -- start start 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2097fb9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2090102d40 0x7f2090198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2097fb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2090103660 0x7f2090198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2097fb9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f209010c170 0x7f209019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:25.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2097fb9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2090111f70 con 0x7f2090102d40 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2097fb9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2090111df0 con 0x7f209010c170 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2097fb9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20901120f0 con 0x7f2090103660 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2095d55640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2090102d40 0x7f2090198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2095d55640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2090102d40 0x7f2090198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49346/0 (socket says 172.21.15.62:49346) 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2095d55640 1 -- 172.21.15.62:0/2954686179 learned_addr learned my addr 172.21.15.62:0/2954686179 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2096556640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f209010c170 0x7f209019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.046+0000 7f2095554640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2090103660 0x7f2090198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2095d55640 1 -- 172.21.15.62:0/2954686179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2090103660 msgr2=0x7f2090198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2095d55640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2090103660 0x7f2090198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2095d55640 1 -- 172.21.15.62:0/2954686179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f209010c170 msgr2=0x7f209019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2095d55640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f209010c170 0x7f209019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2095d55640 1 -- 172.21.15.62:0/2954686179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209019d3e0 con 0x7f2090102d40 2024-09-11T23:58:25.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2095554640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2090103660 0x7f2090198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2095d55640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2090102d40 0x7f2090198430 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2084006010 tx=0x7f20840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2082ffd640 1 -- 172.21.15.62:0/2954686179 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208403f030 con 0x7f2090102d40 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2097fb9640 1 -- 172.21.15.62:0/2954686179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209019d670 con 0x7f2090102d40 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2082ffd640 1 -- 172.21.15.62:0/2954686179 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2084003ce0 con 0x7f2090102d40 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2082ffd640 1 -- 172.21.15.62:0/2954686179 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084035070 con 0x7f2090102d40 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.047+0000 7f2097fb9640 1 -- 172.21.15.62:0/2954686179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a4fb0 con 0x7f2090102d40 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.048+0000 7f2097fb9640 1 -- 172.21.15.62:0/2954686179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209006b070 con 0x7f2090102d40 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.050+0000 7f2082ffd640 1 -- 172.21.15.62:0/2954686179 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2084042030 con 0x7f2090102d40 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.051+0000 7f2082ffd640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f207c0a2cf0 0x7f207c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.051+0000 7f2095554640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f207c0a2cf0 0x7f207c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.051+0000 7f2082ffd640 1 -- 172.21.15.62:0/2954686179 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f20840efd80 con 0x7f2090102d40 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.051+0000 7f2095554640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f207c0a2cf0 0x7f207c0a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f209010d5b0 tx=0x7f208c00b690 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:25.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.054+0000 7f2082ffd640 1 -- 172.21.15.62:0/2954686179 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20840bca90 con 0x7f2090102d40 2024-09-11T23:58:25.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.267+0000 7f2097fb9640 1 -- 172.21.15.62:0/2954686179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2090107210 con 0x7f2090102d40 2024-09-11T23:58:25.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.268+0000 7f2082ffd640 1 -- 172.21.15.62:0/2954686179 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11910 (secure 0 0 0) 0x7f20840bc390 con 0x7f2090102d40 2024-09-11T23:58:25.269 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:58:25.269 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-09-11T23:57:32:617369+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:57:32.075842+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:58:25.269 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-09-11T23:58:25.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 -- 172.21.15.62:0/2954686179 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f207c0a2cf0 msgr2=0x7f207c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:25.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f207c0a2cf0 0x7f207c0a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f209010d5b0 tx=0x7f208c00b690 comp rx=0 tx=0).stop 2024-09-11T23:58:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 -- 172.21.15.62:0/2954686179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2090102d40 msgr2=0x7f2090198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2090102d40 0x7f2090198430 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2084006010 tx=0x7f20840042f0 comp rx=0 tx=0).stop 2024-09-11T23:58:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 -- 172.21.15.62:0/2954686179 shutdown_connections 2024-09-11T23:58:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f207c0a2cf0 0x7f207c0a51b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f209010c170 0x7f209019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2090103660 0x7f2090198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 --2- 172.21.15.62:0/2954686179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2090102d40 0x7f2090198430 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 -- 172.21.15.62:0/2954686179 >> 172.21.15.62:0/2954686179 conn(0x7f20900fc7a0 msgr2=0x7f20900fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.270+0000 7f2097fb9640 1 -- 172.21.15.62:0/2954686179 shutdown_connections 2024-09-11T23:58:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:25.271+0000 7f2097fb9640 1 -- 172.21.15.62:0/2954686179 wait complete. 2024-09-11T23:58:25.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:25 smithi120 ceph-mon[35182]: pgmap v192: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-09-11T23:58:25.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:25 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2954686179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:25.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:25 smithi083 ceph-mon[34805]: pgmap v192: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-09-11T23:58:25.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:25 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2954686179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:25.712 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:25 smithi062 ceph-mon[30958]: pgmap v192: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-09-11T23:58:25.712 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:25 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2954686179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:25.905 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 42b4ed57 --release --await' 2024-09-11T23:58:25.906 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 42b4ed57 --release --await 2024-09-11T23:58:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:26.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:26 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:58:26.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:26 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:58:26.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:26 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:58:26.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:26 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:58:26.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:26 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:58:26.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:26 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.620+0000 7fc464649640 1 -- 172.21.15.62:0/3722156944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c1000d0 msgr2=0x7fc45c100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.620+0000 7fc464649640 1 --2- 172.21.15.62:0/3722156944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c1000d0 0x7fc45c100530 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc458009900 tx=0x7fc45802a410 comp rx=0 tx=0).stop 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.620+0000 7fc464649640 1 -- 172.21.15.62:0/3722156944 shutdown_connections 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.620+0000 7fc464649640 1 --2- 172.21.15.62:0/3722156944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc45c100a70 0x7fc45c10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.620+0000 7fc464649640 1 --2- 172.21.15.62:0/3722156944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c1000d0 0x7fc45c100530 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.620+0000 7fc464649640 1 --2- 172.21.15.62:0/3722156944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc45c106ba0 0x7fc45c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.620+0000 7fc464649640 1 -- 172.21.15.62:0/3722156944 >> 172.21.15.62:0/3722156944 conn(0x7fc45c077210 msgr2=0x7fc45c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.620+0000 7fc464649640 1 -- 172.21.15.62:0/3722156944 shutdown_connections 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.620+0000 7fc464649640 1 -- 172.21.15.62:0/3722156944 wait complete. 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc464649640 1 Processor -- start 2024-09-11T23:58:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc464649640 1 -- start start 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc464649640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc45c1000d0 0x7fc45c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc464649640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c100a70 0x7fc45c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc464649640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc45c106ba0 0x7fc45c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc464649640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c112090 con 0x7fc45c100a70 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc464649640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c111f10 con 0x7fc45c1000d0 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc464649640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c112210 con 0x7fc45c106ba0 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc461be4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c100a70 0x7fc45c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc461be4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c100a70 0x7fc45c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49366/0 (socket says 172.21.15.62:49366) 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc461be4640 1 -- 172.21.15.62:0/1331314687 learned_addr learned my addr 172.21.15.62:0/1331314687 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc4623e5640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc45c1000d0 0x7fc45c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.621+0000 7fc462be6640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc45c106ba0 0x7fc45c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc461be4640 1 -- 172.21.15.62:0/1331314687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc45c106ba0 msgr2=0x7fc45c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:26.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc461be4640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc45c106ba0 0x7fc45c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc461be4640 1 -- 172.21.15.62:0/1331314687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc45c1000d0 msgr2=0x7fc45c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:26.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc461be4640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc45c1000d0 0x7fc45c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc461be4640 1 -- 172.21.15.62:0/1331314687 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45c19d460 con 0x7fc45c100a70 2024-09-11T23:58:26.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc461be4640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c100a70 0x7fc45c1989f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc458006010 tx=0x7fc458003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:26.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc44f7fe640 1 -- 172.21.15.62:0/1331314687 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45803f070 con 0x7fc45c100a70 2024-09-11T23:58:26.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc44f7fe640 1 -- 172.21.15.62:0/1331314687 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc458031c30 con 0x7fc45c100a70 2024-09-11T23:58:26.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc464649640 1 -- 172.21.15.62:0/1331314687 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45c19d6f0 con 0x7fc45c100a70 2024-09-11T23:58:26.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc44f7fe640 1 -- 172.21.15.62:0/1331314687 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458035030 con 0x7fc45c100a70 2024-09-11T23:58:26.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.622+0000 7fc464649640 1 -- 172.21.15.62:0/1331314687 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45c1a5030 con 0x7fc45c100a70 2024-09-11T23:58:26.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.624+0000 7fc44f7fe640 1 -- 172.21.15.62:0/1331314687 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc458042070 con 0x7fc45c100a70 2024-09-11T23:58:26.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc466a22740 1 Processor -- start 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc466a22740 1 -- start start 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc466a22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f381163300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc466a22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381163840 0x55f381172110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc466a22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381172650 0x55f381174a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc466a22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f3810c8620 con 0x55f381172650 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc466a22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f38106fe50 con 0x55f381163840 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc466a22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f3810c45e0 con 0x55f381162f20 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc44f7fe640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc4480a2d70 0x7fc4480a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc461be4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381172650 0x55f381174a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc461be4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381172650 0x55f381174a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49376/0 (socket says 172.21.15.62:49376) 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc461be4640 1 -- 172.21.15.62:0/3420009140 learned_addr learned my addr 172.21.15.62:0/3420009140 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:26.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.625+0000 7fc4623e5640 1 --2- 172.21.15.62:0/3420009140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381163840 0x55f381172110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc462be6640 1 --2- 172.21.15.62:0/3420009140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f381163300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc462be6640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc4480a2d70 0x7fc4480a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc44f7fe640 1 -- 172.21.15.62:0/1331314687 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc4580efb50 con 0x7fc45c100a70 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc4623e5640 1 -- 172.21.15.62:0/3420009140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 msgr2=0x55f381163300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc4623e5640 1 --2- 172.21.15.62:0/3420009140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f381163300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc4623e5640 1 -- 172.21.15.62:0/3420009140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381172650 msgr2=0x55f381174a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc4623e5640 1 --2- 172.21.15.62:0/3420009140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381172650 0x55f381174a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc4623e5640 1 -- 172.21.15.62:0/3420009140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f381060d40 con 0x55f381163840 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc4623e5640 1 --2- 172.21.15.62:0/3420009140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381163840 0x55f381172110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc45000ab40 tx=0x7fc45000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=691e0c0ac2da8e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc462be6640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc4480a2d70 0x7fc4480a5230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc454009fd0 tx=0x7fc454009600 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc44cff9640 1 -- 172.21.15.62:0/3420009140 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450004020 con 0x55f381163840 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc44cff9640 1 -- 172.21.15.62:0/3420009140 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4500041c0 con 0x55f381163840 2024-09-11T23:58:26.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.626+0000 7fc44cff9640 1 -- 172.21.15.62:0/3420009140 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450018790 con 0x55f381163840 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.627+0000 7fc466a22740 1 -- 172.21.15.62:0/3420009140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381163840 msgr2=0x55f381172110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.627+0000 7fc466a22740 1 --2- 172.21.15.62:0/3420009140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381163840 0x55f381172110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc45000ab40 tx=0x7fc45000f3c0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.627+0000 7fc466a22740 1 -- 172.21.15.62:0/3420009140 shutdown_connections 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.627+0000 7fc466a22740 1 --2- 172.21.15.62:0/3420009140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381172650 0x55f381174a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.627+0000 7fc466a22740 1 --2- 172.21.15.62:0/3420009140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381163840 0x55f381172110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.627+0000 7fc466a22740 1 --2- 172.21.15.62:0/3420009140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f381163300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.627+0000 7fc466a22740 1 -- 172.21.15.62:0/3420009140 >> 172.21.15.62:0/3420009140 conn(0x55f38115f5f0 msgr2=0x55f38115f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.627+0000 7fc466a22740 1 -- 172.21.15.62:0/3420009140 shutdown_connections 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.627+0000 7fc466a22740 1 -- 172.21.15.62:0/3420009140 wait complete. 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc466a22740 1 -- start start 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc466a22740 1 Processor -- start 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc466a22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381162f20 0x55f38116f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc466a22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38116fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc466a22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381172650 0x55f38116ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc466a22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f381175220 con 0x55f381163840 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc466a22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f381050f10 con 0x55f381162f20 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc466a22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f3811753a0 con 0x55f381172650 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc466a22740 10 client.0 _ll_register_callbacks cb 0x55f381074db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc461be4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38116fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc461be4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38116fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49378/0 (socket says 172.21.15.62:49378) 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc461be4640 1 -- 172.21.15.62:0/4021898721 learned_addr learned my addr 172.21.15.62:0/4021898721 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc4623e5640 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381162f20 0x55f38116f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc462be6640 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381172650 0x55f38116ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc461be4640 1 -- 172.21.15.62:0/4021898721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381172650 msgr2=0x55f38116ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:26.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc461be4640 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381172650 0x55f38116ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc461be4640 1 -- 172.21.15.62:0/4021898721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381162f20 msgr2=0x55f38116f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc461be4640 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381162f20 0x55f38116f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc461be4640 1 -- 172.21.15.62:0/4021898721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f380dd7580 con 0x55f381163840 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.628+0000 7fc462be6640 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381172650 0x55f38116ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc461be4640 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38116fa50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc4580ed8d0 tx=0x7fc4580b84d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c649ed4f285ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc445ffb640 1 -- 172.21.15.62:0/4021898721 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458049050 con 0x55f381163840 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc445ffb640 1 -- 172.21.15.62:0/4021898721 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc4580b87c0 con 0x55f381163840 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc466a22740 1 -- 172.21.15.62:0/4021898721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4500041c0 con 0x55f381163840 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc445ffb640 1 -- 172.21.15.62:0/4021898721 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4580c4d10 con 0x55f381163840 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc466a22740 10 client.16524 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc466a22740 1 -- 172.21.15.62:0/4021898721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f380ea1960 con 0x55f381163840 2024-09-11T23:58:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc445ffb640 1 -- 172.21.15.62:0/4021898721 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4500041c0 con 0x55f381163840 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc445ffb640 1 -- 172.21.15.62:0/4021898721 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fc4580dace0 con 0x55f381163840 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc466a22740 10 client.16524 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc466a22740 10 client.16524 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc466a22740 20 client.16524 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc466a22740 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f381182170 0x55f381184590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc466a22740 4 client.16524 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.629+0000 7fc466a22740 1 -- 172.21.15.62:0/4021898721 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3810d5b90 con 0x55f381182170 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.630+0000 7fc4623e5640 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f381182170 0x55f381184590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.630+0000 7fc445ffb640 10 client.16524 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.630+0000 7fc4623e5640 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f381182170 0x55f381184590 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c739b1ef1cbe18ee server_cookie=970247425122072a in_seq=0 out_seq=0 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc445ffb640 1 -- 172.21.15.62:0/4021898721 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3810d5b90 con 0x55f381182170 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc445ffb640 10 client.16524 handle_command_reply: tid=0 2024-09-11T23:58:26.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 client.16524 shutdown 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 -- 172.21.15.62:0/4021898721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 msgr2=0x55f38116fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38116fa50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc4580ed8d0 tx=0x7fc4580b84d0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 -- 172.21.15.62:0/4021898721 shutdown_connections 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f381182170 0x55f381184590 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381172650 0x55f38116ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38116fa50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 --2- 172.21.15.62:0/4021898721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381162f20 0x55f38116f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 -- 172.21.15.62:0/4021898721 >> 172.21.15.62:0/4021898721 conn(0x55f38115f5f0 msgr2=0x55f381165350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc445ffb640 0 client.16524 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 -- 172.21.15.62:0/4021898721 shutdown_connections 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 1 -- 172.21.15.62:0/4021898721 wait complete. 2024-09-11T23:58:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.631+0000 7fc466a22740 20 client.16524 trim_cache size 0 max 16384 2024-09-11T23:58:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.648+0000 7fc466a22740 1 Processor -- start 2024-09-11T23:58:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.648+0000 7fc466a22740 1 -- start start 2024-09-11T23:58:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.648+0000 7fc466a22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f38115e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.648+0000 7fc466a22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f381189ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.648+0000 7fc466a22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 0x55f38118a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc466a22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1ade30 con 0x55f381163840 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc466a22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1ab6d0 con 0x55f381172650 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc466a22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1a9540 con 0x55f381162f20 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc461be4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f381189ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc461be4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f381189ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49394/0 (socket says 172.21.15.62:49394) 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc461be4640 1 -- 172.21.15.62:0/1872937468 learned_addr learned my addr 172.21.15.62:0/1872937468 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc4623e5640 1 --2- 172.21.15.62:0/1872937468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f38115e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc461be4640 1 -- 172.21.15.62:0/1872937468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 msgr2=0x55f38115e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc461be4640 1 --2- 172.21.15.62:0/1872937468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f38115e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc461be4640 1 -- 172.21.15.62:0/1872937468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 msgr2=0x55f38118a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc462be6640 1 --2- 172.21.15.62:0/1872937468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 0x55f38118a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc461be4640 1 --2- 172.21.15.62:0/1872937468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 0x55f38118a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc461be4640 1 -- 172.21.15.62:0/1872937468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38116e4c0 con 0x55f381163840 2024-09-11T23:58:26.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc462be6640 1 --2- 172.21.15.62:0/1872937468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 0x55f38118a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:58:26.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc461be4640 1 --2- 172.21.15.62:0/1872937468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f381189ea0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc4580ba400 tx=0x7fc4580dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96323a771ea63fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:26.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc447fff640 1 -- 172.21.15.62:0/1872937468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458049070 con 0x55f381163840 2024-09-11T23:58:26.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc447fff640 1 -- 172.21.15.62:0/1872937468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4580bc150 con 0x55f381163840 2024-09-11T23:58:26.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.649+0000 7fc447fff640 1 -- 172.21.15.62:0/1872937468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4580c4880 con 0x55f381163840 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 -- 172.21.15.62:0/1872937468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 msgr2=0x55f381189ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 --2- 172.21.15.62:0/1872937468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f381189ea0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc4580ba400 tx=0x7fc4580dc040 comp rx=0 tx=0).stop 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 -- 172.21.15.62:0/1872937468 shutdown_connections 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 --2- 172.21.15.62:0/1872937468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 0x55f38118a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 --2- 172.21.15.62:0/1872937468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f381189ea0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 --2- 172.21.15.62:0/1872937468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f38115e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 -- 172.21.15.62:0/1872937468 >> 172.21.15.62:0/1872937468 conn(0x55f38115f5f0 msgr2=0x55f38115df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 -- 172.21.15.62:0/1872937468 shutdown_connections 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 -- 172.21.15.62:0/1872937468 wait complete. 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 -- start start 2024-09-11T23:58:26.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.650+0000 7fc466a22740 1 Processor -- start 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc466a22740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f38117a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc466a22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38117ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc466a22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 0x55f381177620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc466a22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1a9b90 con 0x55f381163840 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc466a22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1ab000 con 0x55f381172650 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc466a22740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1aa260 con 0x55f381162f20 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc466a22740 10 client.0 _ll_register_callbacks cb 0x7fc458005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc461be4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38117ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc461be4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38117ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49398/0 (socket says 172.21.15.62:49398) 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc461be4640 1 -- 172.21.15.62:0/1487076104 learned_addr learned my addr 172.21.15.62:0/1487076104 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc4623e5640 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f38117a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc462be6640 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 0x55f381177620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc4623e5640 1 -- 172.21.15.62:0/1487076104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 msgr2=0x55f381177620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc4623e5640 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 0x55f381177620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc4623e5640 1 -- 172.21.15.62:0/1487076104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 msgr2=0x55f38117ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc4623e5640 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38117ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc4623e5640 1 -- 172.21.15.62:0/1487076104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f381177e50 con 0x55f381162f20 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc461be4640 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38117ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc4623e5640 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f38117a810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc450003e80 tx=0x7fc450005ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5411d91f6f7ad70 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.651+0000 7fc445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc445ffb640 1 -- 172.21.15.62:0/1487076104 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450004140 con 0x55f381162f20 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc445ffb640 1 -- 172.21.15.62:0/1487076104 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc4500042e0 con 0x55f381162f20 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc466a22740 1 -- 172.21.15.62:0/1487076104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4580bc150 con 0x55f381162f20 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc445ffb640 1 -- 172.21.15.62:0/1487076104 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc450005040 con 0x55f381162f20 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc466a22740 10 client.26098 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:26.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc466a22740 1 -- 172.21.15.62:0/1487076104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f3811ed970 con 0x55f381162f20 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc445ffb640 1 -- 172.21.15.62:0/1487076104 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4580bc150 con 0x55f381162f20 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc445ffb640 1 -- 172.21.15.62:0/1487076104 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fc450003b60 con 0x55f381162f20 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc466a22740 10 client.26098 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc466a22740 10 client.26098 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc466a22740 20 client.26098 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc466a22740 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f3811eddb0 0x55f3811f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc466a22740 4 client.26098 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "42b4ed57", "release": true, "await": true}] 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.652+0000 7fc466a22740 1 -- 172.21.15.62:0/1487076104 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "42b4ed57", "release": true, "await": true}) -- 0x55f3810d00a0 con 0x55f3811eddb0 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.653+0000 7fc461be4640 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f3811eddb0 0x55f3811f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.653+0000 7fc445ffb640 10 client.26098 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:26.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.653+0000 7fc461be4640 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f3811eddb0 0x55f3811f01a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a3ecaebb83c196 server_cookie=dcf518c0ca875ded in_seq=0 out_seq=0 2024-09-11T23:58:26.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:26 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:58:26.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:26 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:58:26.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:26 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:58:26.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.700+0000 7fc445ffb640 1 -- 172.21.15.62:0/1487076104 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f3810d00a0 con 0x55f3811eddb0 2024-09-11T23:58:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.700+0000 7fc445ffb640 10 client.26098 handle_command_reply: tid=0 2024-09-11T23:58:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.700+0000 7fc466a22740 1 client.26098 shutdown 2024-09-11T23:58:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 1 -- 172.21.15.62:0/1487076104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 msgr2=0x55f38117a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f38117a810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc450003e80 tx=0x7fc450005ac0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 1 -- 172.21.15.62:0/1487076104 shutdown_connections 2024-09-11T23:58:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f3811eddb0 0x55f3811f01a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f381172650 0x55f381177620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f381163840 0x55f38117ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 1 --2- 172.21.15.62:0/1487076104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f381162f20 0x55f38117a810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 1 -- 172.21.15.62:0/1487076104 >> 172.21.15.62:0/1487076104 conn(0x55f38115f5f0 msgr2=0x55f38115e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc445ffb640 0 client.26098 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 1 -- 172.21.15.62:0/1487076104 shutdown_connections 2024-09-11T23:58:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 1 -- 172.21.15.62:0/1487076104 wait complete. 2024-09-11T23:58:26.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.701+0000 7fc466a22740 20 client.26098 trim_cache size 0 max 16384 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 24, 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: "42b4ed57": { 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 24, 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:58:26.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 140.1, 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:26.704 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:58:26.705 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:58:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.702+0000 7fc464649640 1 -- 172.21.15.62:0/1331314687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc4480a2d70 msgr2=0x7fc4480a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.702+0000 7fc464649640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc4480a2d70 0x7fc4480a5230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc454009fd0 tx=0x7fc454009600 comp rx=0 tx=0).stop 2024-09-11T23:58:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.702+0000 7fc464649640 1 -- 172.21.15.62:0/1331314687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c100a70 msgr2=0x7fc45c1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.702+0000 7fc464649640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c100a70 0x7fc45c1989f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc458006010 tx=0x7fc458003c10 comp rx=0 tx=0).stop 2024-09-11T23:58:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.703+0000 7fc464649640 1 -- 172.21.15.62:0/1331314687 shutdown_connections 2024-09-11T23:58:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.703+0000 7fc464649640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc4480a2d70 0x7fc4480a5230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.703+0000 7fc464649640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc45c106ba0 0x7fc45c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.703+0000 7fc464649640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c100a70 0x7fc45c1989f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.703+0000 7fc464649640 1 --2- 172.21.15.62:0/1331314687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc45c1000d0 0x7fc45c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.703+0000 7fc464649640 1 -- 172.21.15.62:0/1331314687 >> 172.21.15.62:0/1331314687 conn(0x7fc45c077210 msgr2=0x7fc45c104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.703+0000 7fc464649640 1 -- 172.21.15.62:0/1331314687 shutdown_connections 2024-09-11T23:58:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:26.703+0000 7fc464649640 1 -- 172.21.15.62:0/1331314687 wait complete. 2024-09-11T23:58:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 345.0 s, 420.6 tps, lat 886.444 ms stddev 3806.764 2024-09-11T23:58:27.493 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '42b4ed57', total seconds elapsed: 23.3 2024-09-11T23:58:27.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:27 smithi120 ceph-mon[35182]: pgmap v193: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 806 KiB/s wr, 78 op/s 2024-09-11T23:58:27.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:27 smithi083 ceph-mon[34805]: pgmap v193: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 806 KiB/s wr, 78 op/s 2024-09-11T23:58:27.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:27 smithi062 ceph-mon[30958]: pgmap v193: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 806 KiB/s wr, 78 op/s 2024-09-11T23:58:29.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:29 smithi120 ceph-mon[35182]: pgmap v194: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-09-11T23:58:29.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:29 smithi083 ceph-mon[34805]: pgmap v194: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-09-11T23:58:29.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:29 smithi062 ceph-mon[30958]: pgmap v194: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-09-11T23:58:31.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:31 smithi120 ceph-mon[35182]: pgmap v195: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1018 KiB/s wr, 94 op/s 2024-09-11T23:58:31.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:31 smithi083 ceph-mon[34805]: pgmap v195: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1018 KiB/s wr, 94 op/s 2024-09-11T23:58:31.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:31 smithi062 ceph-mon[30958]: pgmap v195: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1018 KiB/s wr, 94 op/s 2024-09-11T23:58:32.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 350.0 s, 3308.8 tps, lat 30.419 ms stddev 33.227 2024-09-11T23:58:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:33 smithi120 ceph-mon[35182]: pgmap v196: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-09-11T23:58:33.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:33 smithi083 ceph-mon[34805]: pgmap v196: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-09-11T23:58:33.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:33 smithi062 ceph-mon[30958]: pgmap v196: 97 pgs: 97 active+clean; 3.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-09-11T23:58:35.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:35 smithi120 ceph-mon[35182]: pgmap v197: 97 pgs: 97 active+clean; 3.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 420 op/s 2024-09-11T23:58:35.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:35 smithi083 ceph-mon[34805]: pgmap v197: 97 pgs: 97 active+clean; 3.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 420 op/s 2024-09-11T23:58:35.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:35 smithi062 ceph-mon[30958]: pgmap v197: 97 pgs: 97 active+clean; 3.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 420 op/s 2024-09-11T23:58:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 355.0 s, 3382.6 tps, lat 29.517 ms stddev 32.140 2024-09-11T23:58:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:37 smithi120 ceph-mon[35182]: pgmap v198: 97 pgs: 97 active+clean; 3.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 346 op/s 2024-09-11T23:58:37.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:37 smithi083 ceph-mon[34805]: pgmap v198: 97 pgs: 97 active+clean; 3.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 346 op/s 2024-09-11T23:58:37.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:37 smithi062 ceph-mon[30958]: pgmap v198: 97 pgs: 97 active+clean; 3.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 346 op/s 2024-09-11T23:58:39.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:39 smithi120 ceph-mon[35182]: pgmap v199: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 672 op/s 2024-09-11T23:58:39.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:39 smithi083 ceph-mon[34805]: pgmap v199: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 672 op/s 2024-09-11T23:58:39.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:39 smithi062 ceph-mon[30958]: pgmap v199: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 672 op/s 2024-09-11T23:58:41.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:41 smithi120 ceph-mon[35182]: pgmap v200: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.6 MiB/s wr, 652 op/s 2024-09-11T23:58:41.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:41 smithi083 ceph-mon[34805]: pgmap v200: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.6 MiB/s wr, 652 op/s 2024-09-11T23:58:41.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:41 smithi062 ceph-mon[30958]: pgmap v200: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.6 MiB/s wr, 652 op/s 2024-09-11T23:58:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 360.0 s, 3905.6 tps, lat 25.530 ms stddev 28.495 2024-09-11T23:58:43.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:43 smithi120 ceph-mon[35182]: pgmap v201: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 686 op/s 2024-09-11T23:58:43.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:43 smithi083 ceph-mon[34805]: pgmap v201: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 686 op/s 2024-09-11T23:58:43.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:43 smithi062 ceph-mon[30958]: pgmap v201: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 686 op/s 2024-09-11T23:58:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:45 smithi120 ceph-mon[35182]: pgmap v202: 97 pgs: 97 active+clean; 3.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 961 op/s 2024-09-11T23:58:45.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:45 smithi083 ceph-mon[34805]: pgmap v202: 97 pgs: 97 active+clean; 3.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 961 op/s 2024-09-11T23:58:45.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:45 smithi062 ceph-mon[30958]: pgmap v202: 97 pgs: 97 active+clean; 3.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 961 op/s 2024-09-11T23:58:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 365.0 s, 2916.2 tps, lat 34.303 ms stddev 35.558 2024-09-11T23:58:47.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:47 smithi120 ceph-mon[35182]: pgmap v203: 97 pgs: 97 active+clean; 3.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.2 MiB/s wr, 647 op/s 2024-09-11T23:58:47.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:47 smithi083 ceph-mon[34805]: pgmap v203: 97 pgs: 97 active+clean; 3.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.2 MiB/s wr, 647 op/s 2024-09-11T23:58:47.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:47 smithi062 ceph-mon[30958]: pgmap v203: 97 pgs: 97 active+clean; 3.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.2 MiB/s wr, 647 op/s 2024-09-11T23:58:49.495 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:58:49.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:49 smithi120 ceph-mon[35182]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-11T23:58:49.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:49 smithi083 ceph-mon[34805]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-11T23:58:49.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:49 smithi062 ceph-mon[30958]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-11T23:58:49.893 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.459+0000 7f71509af640 1 -- 172.21.15.62:0/4119060536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c10eb10 msgr2=0x7f714c10eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.459+0000 7f71509af640 1 --2- 172.21.15.62:0/4119060536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c10eb10 0x7f714c10eef0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f713c009900 tx=0x7f713c02a3f0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.459+0000 7f71509af640 1 -- 172.21.15.62:0/4119060536 shutdown_connections 2024-09-11T23:58:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.459+0000 7f71509af640 1 --2- 172.21.15.62:0/4119060536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f714c109b90 0x7f714c1158a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.459+0000 7f71509af640 1 --2- 172.21.15.62:0/4119060536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f714c1091f0 0x7f714c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.459+0000 7f71509af640 1 --2- 172.21.15.62:0/4119060536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c10eb10 0x7f714c10eef0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.459+0000 7f71509af640 1 -- 172.21.15.62:0/4119060536 >> 172.21.15.62:0/4119060536 conn(0x7f714c06e2d0 msgr2=0x7f714c06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:50.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.460+0000 7f71509af640 1 -- 172.21.15.62:0/4119060536 shutdown_connections 2024-09-11T23:58:50.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.460+0000 7f71509af640 1 -- 172.21.15.62:0/4119060536 wait complete. 2024-09-11T23:58:50.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.461+0000 7f71509af640 1 Processor -- start 2024-09-11T23:58:50.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.461+0000 7f71509af640 1 -- start start 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f71509af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f714c1091f0 0x7f714c1a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f71509af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c109b90 0x7f714c1a8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f71509af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f714c10eb10 0x7f714c1accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f71509af640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f714c118e70 con 0x7f714c109b90 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f71509af640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f714c118cf0 con 0x7f714c1091f0 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f71509af640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f714c118ff0 con 0x7f714c10eb10 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f7149d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c109b90 0x7f714c1a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f7149d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c109b90 0x7f714c1a8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51346/0 (socket says 172.21.15.62:51346) 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f714ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f714c10eb10 0x7f714c1accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f7149d9b640 1 -- 172.21.15.62:0/3235614562 learned_addr learned my addr 172.21.15.62:0/3235614562 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f714a59c640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f714c1091f0 0x7f714c1a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f7149d9b640 1 -- 172.21.15.62:0/3235614562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f714c10eb10 msgr2=0x7f714c1accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:50.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f7149d9b640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f714c10eb10 0x7f714c1accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f7149d9b640 1 -- 172.21.15.62:0/3235614562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f714c1091f0 msgr2=0x7f714c1a7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:50.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f7149d9b640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f714c1091f0 0x7f714c1a7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.462+0000 7f7149d9b640 1 -- 172.21.15.62:0/3235614562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c1ad3f0 con 0x7f714c109b90 2024-09-11T23:58:50.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.463+0000 7f7149d9b640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c109b90 0x7f714c1a8310 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f714000d950 tx=0x7f714000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:50.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.463+0000 7f714ad9d640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f714c10eb10 0x7f714c1accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:58:50.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.463+0000 7f714a59c640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f714c1091f0 0x7f714c1a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:50.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.463+0000 7f713b7fe640 1 -- 172.21.15.62:0/3235614562 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140017030 con 0x7f714c109b90 2024-09-11T23:58:50.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.464+0000 7f713b7fe640 1 -- 172.21.15.62:0/3235614562 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7140004020 con 0x7f714c109b90 2024-09-11T23:58:50.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.464+0000 7f713b7fe640 1 -- 172.21.15.62:0/3235614562 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140003d60 con 0x7f714c109b90 2024-09-11T23:58:50.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.464+0000 7f71509af640 1 -- 172.21.15.62:0/3235614562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c1ad6e0 con 0x7f714c109b90 2024-09-11T23:58:50.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.464+0000 7f71509af640 1 -- 172.21.15.62:0/3235614562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c1b3d70 con 0x7f714c109b90 2024-09-11T23:58:50.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.468+0000 7f713b7fe640 1 -- 172.21.15.62:0/3235614562 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f714001a070 con 0x7f714c109b90 2024-09-11T23:58:50.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.469+0000 7f713b7fe640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71340a2d70 0x7f71340a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.469+0000 7f713b7fe640 1 -- 172.21.15.62:0/3235614562 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f71400ceea0 con 0x7f714c109b90 2024-09-11T23:58:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f714a59c640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71340a2d70 0x7f71340a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7152d88740 1 Processor -- start 2024-09-11T23:58:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7152d88740 1 -- start start 2024-09-11T23:58:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7152d88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 0x557539e7a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7152d88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7adc0 0x557539e85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7152d88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e85680 0x557539e87a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7152d88740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557539de02e0 con 0x557539e85680 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7152d88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557539d87ce0 con 0x557539e7a4a0 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7152d88740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557539ddc2a0 con 0x557539e7adc0 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7149d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7adc0 0x557539e85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7149d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7adc0 0x557539e85140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:37818/0 (socket says 172.21.15.62:37818) 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f7149d9b640 1 -- 172.21.15.62:0/346133247 learned_addr learned my addr 172.21.15.62:0/346133247 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.470+0000 7f714ad9d640 1 --2- 172.21.15.62:0/346133247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e85680 0x557539e87a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f714a59c640 1 --2- 172.21.15.62:0/346133247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 0x557539e7a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f714a59c640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71340a2d70 0x7f71340a5230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f713c000c00 tx=0x7f713c005a40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f7149d9b640 1 -- 172.21.15.62:0/346133247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 msgr2=0x557539e7a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f7149d9b640 1 --2- 172.21.15.62:0/346133247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 0x557539e7a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f7149d9b640 1 -- 172.21.15.62:0/346133247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e85680 msgr2=0x557539e87a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f7149d9b640 1 --2- 172.21.15.62:0/346133247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e85680 0x557539e87a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f7149d9b640 1 -- 172.21.15.62:0/346133247 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557539d6c440 con 0x557539e7adc0 2024-09-11T23:58:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f714ad9d640 1 --2- 172.21.15.62:0/346133247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e85680 0x557539e87a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f7149d9b640 1 --2- 172.21.15.62:0/346133247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7adc0 0x557539e85140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7140097850 tx=0x7f714009bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9ff3093b9a240b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f7138ff9640 1 -- 172.21.15.62:0/346133247 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71400b4070 con 0x557539e7adc0 2024-09-11T23:58:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f7138ff9640 1 -- 172.21.15.62:0/346133247 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71400a4060 con 0x557539e7adc0 2024-09-11T23:58:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.471+0000 7f7138ff9640 1 -- 172.21.15.62:0/346133247 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140014070 con 0x557539e7adc0 2024-09-11T23:58:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.472+0000 7f7152d88740 1 -- 172.21.15.62:0/346133247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7adc0 msgr2=0x557539e85140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.472+0000 7f7152d88740 1 --2- 172.21.15.62:0/346133247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7adc0 0x557539e85140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7140097850 tx=0x7f714009bbb0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.472+0000 7f7152d88740 1 -- 172.21.15.62:0/346133247 shutdown_connections 2024-09-11T23:58:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.472+0000 7f7152d88740 1 --2- 172.21.15.62:0/346133247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e85680 0x557539e87a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.472+0000 7f7152d88740 1 --2- 172.21.15.62:0/346133247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7adc0 0x557539e85140 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7140097850 tx=0x7f714009bbb0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.472+0000 7f7152d88740 1 --2- 172.21.15.62:0/346133247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 0x557539e7a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.472+0000 7f7152d88740 1 -- 172.21.15.62:0/346133247 >> 172.21.15.62:0/346133247 conn(0x557539e76a30 msgr2=0x557539e76e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.473+0000 7f7152d88740 1 -- 172.21.15.62:0/346133247 shutdown_connections 2024-09-11T23:58:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.473+0000 7f7152d88740 1 -- 172.21.15.62:0/346133247 wait complete. 2024-09-11T23:58:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.474+0000 7f7152d88740 1 -- start start 2024-09-11T23:58:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.474+0000 7f7152d88740 1 Processor -- start 2024-09-11T23:58:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f7152d88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 0x557539e85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f7152d88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 0x557539e823f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f7152d88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 0x557539e91720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f7152d88740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557539e88250 con 0x557539e82ae0 2024-09-11T23:58:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f7152d88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557539d9b050 con 0x557539e7a4a0 2024-09-11T23:58:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f7152d88740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557539e883d0 con 0x557539e85680 2024-09-11T23:58:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f7152d88740 10 client.0 _ll_register_callbacks cb 0x557539ddb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f7149d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 0x557539e823f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f7149d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 0x557539e823f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:37832/0 (socket says 172.21.15.62:37832) 2024-09-11T23:58:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f7149d9b640 1 -- 172.21.15.62:0/3911426980 learned_addr learned my addr 172.21.15.62:0/3911426980 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f714ad9d640 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 0x557539e91720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.475+0000 7f714a59c640 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 0x557539e85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7149d9b640 1 -- 172.21.15.62:0/3911426980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 msgr2=0x557539e85140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7149d9b640 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 0x557539e85140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7149d9b640 1 -- 172.21.15.62:0/3911426980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 msgr2=0x557539e91720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7149d9b640 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 0x557539e91720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7149d9b640 1 -- 172.21.15.62:0/3911426980 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557539b513c0 con 0x557539e85680 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f714a59c640 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 0x557539e85140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7149d9b640 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 0x557539e823f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f71400ccc20 tx=0x7f714009f920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b984e67e2260278e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7131ffb640 1 -- 172.21.15.62:0/3911426980 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140014070 con 0x557539e85680 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7131ffb640 1 -- 172.21.15.62:0/3911426980 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f71400b41e0 con 0x557539e85680 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7152d88740 1 -- 172.21.15.62:0/3911426980 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71400a4060 con 0x557539e85680 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7131ffb640 1 -- 172.21.15.62:0/3911426980 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f71400b4040 con 0x557539e85680 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7152d88740 10 client.26107 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.476+0000 7f7152d88740 1 -- 172.21.15.62:0/3911426980 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557539e88250 con 0x557539e85680 2024-09-11T23:58:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.477+0000 7f7131ffb640 1 -- 172.21.15.62:0/3911426980 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71400a4060 con 0x557539e85680 2024-09-11T23:58:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.477+0000 7f7131ffb640 1 -- 172.21.15.62:0/3911426980 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f7140024060 con 0x557539e85680 2024-09-11T23:58:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.477+0000 7f7152d88740 10 client.26107 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.477+0000 7f7152d88740 10 client.26107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.477+0000 7f7152d88740 20 client.26107 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.477+0000 7f7152d88740 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557539e95350 0x557539e97740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.477+0000 7f7152d88740 4 client.26107 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.477+0000 7f7152d88740 1 -- 172.21.15.62:0/3911426980 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557539ded850 con 0x557539e95350 2024-09-11T23:58:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.477+0000 7f714a59c640 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557539e95350 0x557539e97740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.478+0000 7f714a59c640 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557539e95350 0x557539e97740 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ba7726dee4fe0 server_cookie=34ca264eb40a3949 in_seq=0 out_seq=0 2024-09-11T23:58:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.478+0000 7f7131ffb640 10 client.26107 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7131ffb640 1 -- 172.21.15.62:0/3911426980 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557539ded850 con 0x557539e95350 2024-09-11T23:58:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7131ffb640 10 client.26107 handle_command_reply: tid=0 2024-09-11T23:58:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7152d88740 1 client.26107 shutdown 2024-09-11T23:58:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7152d88740 1 -- 172.21.15.62:0/3911426980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 msgr2=0x557539e823f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7152d88740 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 0x557539e823f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f71400ccc20 tx=0x7f714009f920 comp rx=0 tx=0).stop 2024-09-11T23:58:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7152d88740 1 -- 172.21.15.62:0/3911426980 shutdown_connections 2024-09-11T23:58:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7152d88740 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557539e95350 0x557539e97740 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7152d88740 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 0x557539e91720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7152d88740 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 0x557539e823f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7152d88740 1 --2- 172.21.15.62:0/3911426980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e7a4a0 0x557539e85140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7152d88740 1 -- 172.21.15.62:0/3911426980 >> 172.21.15.62:0/3911426980 conn(0x557539e76a30 msgr2=0x557539e7c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7131ffb640 0 client.26107 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.479+0000 7f7152d88740 1 -- 172.21.15.62:0/3911426980 shutdown_connections 2024-09-11T23:58:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.481+0000 7f7152d88740 1 -- 172.21.15.62:0/3911426980 wait complete. 2024-09-11T23:58:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.481+0000 7f7152d88740 20 client.26107 trim_cache size 0 max 16384 2024-09-11T23:58:50.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.510+0000 7f7152d88740 1 Processor -- start 2024-09-11T23:58:50.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f7152d88740 1 -- start start 2024-09-11T23:58:50.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f7152d88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e7a4a0 0x557539e7d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f7152d88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e82ae0 0x557539e91510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f7152d88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 0x557539e91a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f7152d88740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f712c010f20 con 0x557539e7a4a0 2024-09-11T23:58:50.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f7152d88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f712c00e7c0 con 0x557539e82ae0 2024-09-11T23:58:50.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f7152d88740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f712c00c610 con 0x557539e85680 2024-09-11T23:58:50.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f714ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 0x557539e91a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f7149d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e82ae0 0x557539e91510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f7149d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e82ae0 0x557539e91510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:39326/0 (socket says 172.21.15.62:39326) 2024-09-11T23:58:50.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f7149d9b640 1 -- 172.21.15.62:0/2477309746 learned_addr learned my addr 172.21.15.62:0/2477309746 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:50.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.511+0000 7f714a59c640 1 --2- 172.21.15.62:0/2477309746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e7a4a0 0x557539e7d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.512+0000 7f7149d9b640 1 -- 172.21.15.62:0/2477309746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 msgr2=0x557539e91a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:50.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.512+0000 7f7149d9b640 1 --2- 172.21.15.62:0/2477309746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 0x557539e91a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.512+0000 7f7149d9b640 1 -- 172.21.15.62:0/2477309746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e7a4a0 msgr2=0x557539e7d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:50.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.512+0000 7f7149d9b640 1 --2- 172.21.15.62:0/2477309746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e7a4a0 0x557539e7d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.512+0000 7f7149d9b640 1 -- 172.21.15.62:0/2477309746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557539e80960 con 0x557539e82ae0 2024-09-11T23:58:50.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.512+0000 7f7149d9b640 1 --2- 172.21.15.62:0/2477309746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e82ae0 0x557539e91510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7140009610 tx=0x7f71400cc6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a50d19176ae243 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:50.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.512+0000 7f7133fff640 1 -- 172.21.15.62:0/2477309746 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140014070 con 0x557539e82ae0 2024-09-11T23:58:50.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.513+0000 7f7133fff640 1 -- 172.21.15.62:0/2477309746 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71400a4060 con 0x557539e82ae0 2024-09-11T23:58:50.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.513+0000 7f7133fff640 1 -- 172.21.15.62:0/2477309746 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71400b41e0 con 0x557539e82ae0 2024-09-11T23:58:50.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.514+0000 7f7152d88740 1 -- 172.21.15.62:0/2477309746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e82ae0 msgr2=0x557539e91510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:50.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.514+0000 7f7152d88740 1 --2- 172.21.15.62:0/2477309746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e82ae0 0x557539e91510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7140009610 tx=0x7f71400cc6d0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.514+0000 7f7152d88740 1 -- 172.21.15.62:0/2477309746 shutdown_connections 2024-09-11T23:58:50.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.514+0000 7f7152d88740 1 --2- 172.21.15.62:0/2477309746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e85680 0x557539e91a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.514+0000 7f7152d88740 1 --2- 172.21.15.62:0/2477309746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e82ae0 0x557539e91510 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.514+0000 7f7152d88740 1 --2- 172.21.15.62:0/2477309746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e7a4a0 0x557539e7d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.514+0000 7f7152d88740 1 -- 172.21.15.62:0/2477309746 >> 172.21.15.62:0/2477309746 conn(0x557539e76a30 msgr2=0x557539e7d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:50.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.514+0000 7f7152d88740 1 -- 172.21.15.62:0/2477309746 shutdown_connections 2024-09-11T23:58:50.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.514+0000 7f7152d88740 1 -- 172.21.15.62:0/2477309746 wait complete. 2024-09-11T23:58:50.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7152d88740 1 -- start start 2024-09-11T23:58:50.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7152d88740 1 Processor -- start 2024-09-11T23:58:50.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7152d88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7a4a0 0x557539ef8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7152d88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 0x557539ef85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7152d88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e85680 0x557539e88c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7152d88740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f712c00cc80 con 0x557539e82ae0 2024-09-11T23:58:50.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7152d88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f712c00e0f0 con 0x557539e85680 2024-09-11T23:58:50.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7152d88740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f712c00d350 con 0x557539e7a4a0 2024-09-11T23:58:50.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7149d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 0x557539ef85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7149d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 0x557539ef85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51384/0 (socket says 172.21.15.62:51384) 2024-09-11T23:58:50.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7149d9b640 1 -- 172.21.15.62:0/1103782277 learned_addr learned my addr 172.21.15.62:0/1103782277 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:50.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f714a59c640 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7a4a0 0x557539ef8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.515+0000 7f7152d88740 10 client.0 _ll_register_callbacks cb 0x7f71400b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:50.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f714ad9d640 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e85680 0x557539e88c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7149d9b640 1 -- 172.21.15.62:0/1103782277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7a4a0 msgr2=0x557539ef8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:50.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7149d9b640 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7a4a0 0x557539ef8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7149d9b640 1 -- 172.21.15.62:0/1103782277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e85680 msgr2=0x557539e88c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:50.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7149d9b640 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e85680 0x557539e88c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7149d9b640 1 -- 172.21.15.62:0/1103782277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557539e89350 con 0x557539e82ae0 2024-09-11T23:58:50.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7149d9b640 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 0x557539ef85a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f71400ccc20 tx=0x7f7140000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fd10b64eb7543c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:50.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7131ffb640 1 -- 172.21.15.62:0/1103782277 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140014070 con 0x557539e82ae0 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7131ffb640 1 -- 172.21.15.62:0/1103782277 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f71400041d0 con 0x557539e82ae0 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7152d88740 1 -- 172.21.15.62:0/1103782277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71400a4060 con 0x557539e82ae0 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7131ffb640 1 -- 172.21.15.62:0/1103782277 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f71400b43f0 con 0x557539e82ae0 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7152d88740 10 client.16548 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.516+0000 7f7152d88740 1 -- 172.21.15.62:0/1103782277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557539e89350 con 0x557539e82ae0 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.517+0000 7f7131ffb640 1 -- 172.21.15.62:0/1103782277 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71400a4060 con 0x557539e82ae0 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.517+0000 7f7131ffb640 1 -- 172.21.15.62:0/1103782277 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f7140009c30 con 0x557539e82ae0 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.517+0000 7f7152d88740 10 client.16548 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.517+0000 7f7152d88740 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.517+0000 7f7152d88740 20 client.16548 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.517+0000 7f7152d88740 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557539effbb0 0x557539f01fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.517+0000 7f7152d88740 4 client.16548 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:58:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.517+0000 7f7152d88740 1 -- 172.21.15.62:0/1103782277 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557539de7d60 con 0x557539effbb0 2024-09-11T23:58:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.517+0000 7f714a59c640 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557539effbb0 0x557539f01fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.518+0000 7f714a59c640 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557539effbb0 0x557539f01fa0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cedcf5735d7c7291 server_cookie=5c2b96d572ca3f5b in_seq=0 out_seq=0 2024-09-11T23:58:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.518+0000 7f7131ffb640 10 client.16548 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.518+0000 7f7131ffb640 1 -- 172.21.15.62:0/1103782277 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557539de7d60 con 0x557539effbb0 2024-09-11T23:58:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.518+0000 7f7131ffb640 10 client.16548 handle_command_reply: tid=0 2024-09-11T23:58:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.519+0000 7f7152d88740 1 client.16548 shutdown 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.519+0000 7f7152d88740 1 -- 172.21.15.62:0/1103782277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 msgr2=0x557539ef85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.519+0000 7f7152d88740 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 0x557539ef85a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f71400ccc20 tx=0x7f7140000bc0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.519+0000 7f7152d88740 1 -- 172.21.15.62:0/1103782277 shutdown_connections 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.519+0000 7f7152d88740 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557539effbb0 0x557539f01fa0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.519+0000 7f7152d88740 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557539e85680 0x557539e88c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.519+0000 7f7152d88740 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557539e82ae0 0x557539ef85a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.519+0000 7f7152d88740 1 --2- 172.21.15.62:0/1103782277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557539e7a4a0 0x557539ef8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.520+0000 7f7152d88740 1 -- 172.21.15.62:0/1103782277 >> 172.21.15.62:0/1103782277 conn(0x557539e76a30 msgr2=0x557539e75b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.520+0000 7f7131ffb640 0 client.16548 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.520+0000 7f7152d88740 1 -- 172.21.15.62:0/1103782277 shutdown_connections 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.520+0000 7f7152d88740 1 -- 172.21.15.62:0/1103782277 wait complete. 2024-09-11T23:58:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.520+0000 7f7152d88740 20 client.16548 trim_cache size 0 max 16384 2024-09-11T23:58:50.525 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:58:50.525 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-11T23:58:50.525 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-11T23:58:50.525 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:58:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 -- 172.21.15.62:0/3235614562 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71340a2d70 msgr2=0x7f71340a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71340a2d70 0x7f71340a5230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f713c000c00 tx=0x7f713c005a40 comp rx=0 tx=0).stop 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 -- 172.21.15.62:0/3235614562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c109b90 msgr2=0x7f714c1a8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c109b90 0x7f714c1a8310 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f714000d950 tx=0x7f714000de10 comp rx=0 tx=0).stop 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 -- 172.21.15.62:0/3235614562 shutdown_connections 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71340a2d70 0x7f71340a5230 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f714c10eb10 0x7f714c1accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c109b90 0x7f714c1a8310 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 --2- 172.21.15.62:0/3235614562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f714c1091f0 0x7f714c1a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 -- 172.21.15.62:0/3235614562 >> 172.21.15.62:0/3235614562 conn(0x7f714c06e2d0 msgr2=0x7f714c119df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.521+0000 7f71509af640 1 -- 172.21.15.62:0/3235614562 shutdown_connections 2024-09-11T23:58:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:50.522+0000 7f71509af640 1 -- 172.21.15.62:0/3235614562 wait complete. 2024-09-11T23:58:50.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:50 smithi083 ceph-mon[34805]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.6 MiB/s wr, 618 op/s 2024-09-11T23:58:50.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:50 smithi062 ceph-mon[30958]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.6 MiB/s wr, 618 op/s 2024-09-11T23:58:50.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:50 smithi120 ceph-mon[35182]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.6 MiB/s wr, 618 op/s 2024-09-11T23:58:51.571 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:58:51.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:58:51.982 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 370.0 s, 2859.6 tps, lat 34.794 ms stddev 38.707 2024-09-11T23:58:52.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:52 smithi062 ceph-mon[30958]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.7 MiB/s wr, 627 op/s 2024-09-11T23:58:52.901 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:58:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-09-11T23:57:32:617369+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:57:32.075842+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:58:52.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.544+0000 7f5937098640 1 -- 172.21.15.62:0/1790476548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5930109b10 msgr2=0x7f5930114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:52.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.544+0000 7f5937098640 1 --2- 172.21.15.62:0/1790476548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5930109b10 0x7f5930114570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f591c009900 tx=0x7f591c02a480 comp rx=0 tx=0).stop 2024-09-11T23:58:52.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.544+0000 7f5937098640 1 -- 172.21.15.62:0/1790476548 shutdown_connections 2024-09-11T23:58:52.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.544+0000 7f5937098640 1 --2- 172.21.15.62:0/1790476548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5930114ab0 0x7f5930116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:52.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.544+0000 7f5937098640 1 --2- 172.21.15.62:0/1790476548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5930109b10 0x7f5930114570 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:52.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.544+0000 7f5937098640 1 --2- 172.21.15.62:0/1790476548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59301091f0 0x7f59301095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:52.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.544+0000 7f5937098640 1 -- 172.21.15.62:0/1790476548 >> 172.21.15.62:0/1790476548 conn(0x7f593006e140 msgr2=0x7f593006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.544+0000 7f5937098640 1 -- 172.21.15.62:0/1790476548 shutdown_connections 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.544+0000 7f5937098640 1 -- 172.21.15.62:0/1790476548 wait complete. 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5937098640 1 Processor -- start 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5937098640 1 -- start start 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5937098640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59301091f0 0x7f59301a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5937098640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5930109b10 0x7f59301a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5937098640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5930114ab0 0x7f59301a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5937098640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f593011a820 con 0x7f5930114ab0 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5937098640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f593011a6a0 con 0x7f5930109b10 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5937098640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f593011a9a0 con 0x7f59301091f0 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5935635640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5930114ab0 0x7f59301a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5935635640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5930114ab0 0x7f59301a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38814/0 (socket says 172.21.15.62:38814) 2024-09-11T23:58:52.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f5935635640 1 -- 172.21.15.62:0/3436212309 learned_addr learned my addr 172.21.15.62:0/3436212309 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.545+0000 7f592ffff640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5930109b10 0x7f59301a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f5934e34640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59301091f0 0x7f59301a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f5935635640 1 -- 172.21.15.62:0/3436212309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59301091f0 msgr2=0x7f59301a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f5935635640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59301091f0 0x7f59301a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f5935635640 1 -- 172.21.15.62:0/3436212309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5930109b10 msgr2=0x7f59301a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f5935635640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5930109b10 0x7f59301a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f5935635640 1 -- 172.21.15.62:0/3436212309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59301a5d10 con 0x7f5930114ab0 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f5935635640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5930114ab0 0x7f59301a5630 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f592800eb10 tx=0x7f592800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f592dffb640 1 -- 172.21.15.62:0/3436212309 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5928004b20 con 0x7f5930114ab0 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f5937098640 1 -- 172.21.15.62:0/3436212309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59301a6000 con 0x7f5930114ab0 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f592dffb640 1 -- 172.21.15.62:0/3436212309 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5928004cc0 con 0x7f5930114ab0 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f592dffb640 1 -- 172.21.15.62:0/3436212309 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5928010790 con 0x7f5930114ab0 2024-09-11T23:58:52.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.546+0000 7f5937098640 1 -- 172.21.15.62:0/3436212309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593010a300 con 0x7f5930114ab0 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.547+0000 7f5937098640 1 -- 172.21.15.62:0/3436212309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59301095d0 con 0x7f5930114ab0 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.550+0000 7f592dffb640 1 -- 172.21.15.62:0/3436212309 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5928010930 con 0x7f5930114ab0 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.551+0000 7f592dffb640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f59200a2d70 0x7f59200a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.551+0000 7f592dffb640 1 -- 172.21.15.62:0/3436212309 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f59280cfb40 con 0x7f5930114ab0 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.551+0000 7f5934e34640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f59200a2d70 0x7f59200a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.551+0000 7f5934e34640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f59200a2d70 0x7f59200a5230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5924009bd0 tx=0x7f5924005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.555+0000 7f592dffb640 1 -- 172.21.15.62:0/3436212309 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f592809c730 con 0x7f5930114ab0 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.893+0000 7f5937098640 1 -- 172.21.15.62:0/3436212309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59301a20c0 con 0x7f5930114ab0 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.894+0000 7f592dffb640 1 -- 172.21.15.62:0/3436212309 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11910 (secure 0 0 0) 0x7f59301a20c0 con 0x7f5930114ab0 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.899+0000 7f5937098640 1 -- 172.21.15.62:0/3436212309 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f59200a2d70 msgr2=0x7f59200a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:52.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.899+0000 7f5937098640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f59200a2d70 0x7f59200a5230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5924009bd0 tx=0x7f5924005f90 comp rx=0 tx=0).stop 2024-09-11T23:58:52.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.899+0000 7f5937098640 1 -- 172.21.15.62:0/3436212309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5930114ab0 msgr2=0x7f59301a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:52.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.899+0000 7f5937098640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5930114ab0 0x7f59301a5630 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f592800eb10 tx=0x7f592800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:58:52.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.900+0000 7f5937098640 1 -- 172.21.15.62:0/3436212309 shutdown_connections 2024-09-11T23:58:52.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.900+0000 7f5937098640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f59200a2d70 0x7f59200a5230 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5924009bd0 tx=0x7f5924005f90 comp rx=0 tx=0).stop 2024-09-11T23:58:52.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.900+0000 7f5937098640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5930114ab0 0x7f59301a5630 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:52.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.900+0000 7f5937098640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5930109b10 0x7f59301a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:52.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.900+0000 7f5937098640 1 --2- 172.21.15.62:0/3436212309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59301091f0 0x7f59301a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:52.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.900+0000 7f5937098640 1 -- 172.21.15.62:0/3436212309 >> 172.21.15.62:0/3436212309 conn(0x7f593006e140 msgr2=0x7f5930071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:52.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.900+0000 7f5937098640 1 -- 172.21.15.62:0/3436212309 shutdown_connections 2024-09-11T23:58:52.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:52.901+0000 7f5937098640 1 -- 172.21.15.62:0/3436212309 wait complete. 2024-09-11T23:58:52.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:52 smithi083 ceph-mon[34805]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.7 MiB/s wr, 627 op/s 2024-09-11T23:58:53.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:52 smithi120 ceph-mon[35182]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.7 MiB/s wr, 627 op/s 2024-09-11T23:58:53.770 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:53 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3436212309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:53.808 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:58:53.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:53 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3436212309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:54.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:53 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3436212309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:58:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.725+0000 7f88cf6a0640 1 -- 172.21.15.62:0/726199184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101410 msgr2=0x7f88c810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.725+0000 7f88cf6a0640 1 --2- 172.21.15.62:0/726199184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101410 0x7f88c810d940 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009cd0 tx=0x7f88c002a910 comp rx=0 tx=0).stop 2024-09-11T23:58:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.725+0000 7f88cf6a0640 1 -- 172.21.15.62:0/726199184 shutdown_connections 2024-09-11T23:58:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.725+0000 7f88cf6a0640 1 --2- 172.21.15.62:0/726199184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101410 0x7f88c810d940 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.725+0000 7f88cf6a0640 1 --2- 172.21.15.62:0/726199184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8100a70 0x7f88c8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.725+0000 7f88cf6a0640 1 --2- 172.21.15.62:0/726199184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88c8106a70 0x7f88c8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.725+0000 7f88cf6a0640 1 -- 172.21.15.62:0/726199184 >> 172.21.15.62:0/726199184 conn(0x7f88c80fc790 msgr2=0x7f88c80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.725+0000 7f88cf6a0640 1 -- 172.21.15.62:0/726199184 shutdown_connections 2024-09-11T23:58:54.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.725+0000 7f88cf6a0640 1 -- 172.21.15.62:0/726199184 wait complete. 2024-09-11T23:58:54.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cf6a0640 1 Processor -- start 2024-09-11T23:58:54.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cf6a0640 1 -- start start 2024-09-11T23:58:54.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cf6a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8100a70 0x7f88c8102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cf6a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88c8101410 0x7f88c8102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cf6a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8106a70 0x7f88c81030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cf6a0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8110eb0 con 0x7f88c8100a70 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cf6a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8110d30 con 0x7f88c8106a70 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cf6a0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8111030 con 0x7f88c8101410 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cd43c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8100a70 0x7f88c8102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cd43c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8100a70 0x7f88c8102640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38830/0 (socket says 172.21.15.62:38830) 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cd43c640 1 -- 172.21.15.62:0/1821145592 learned_addr learned my addr 172.21.15.62:0/1821145592 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88cdc3d640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8106a70 0x7f88c81030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.726+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88c8101410 0x7f88c8102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.727+0000 7f88cdc3d640 1 -- 172.21.15.62:0/1821145592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88c8101410 msgr2=0x7f88c8102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.727+0000 7f88cdc3d640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88c8101410 0x7f88c8102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.727+0000 7f88cdc3d640 1 -- 172.21.15.62:0/1821145592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8100a70 msgr2=0x7f88c8102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.727+0000 7f88cdc3d640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8100a70 0x7f88c8102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.727+0000 7f88cdc3d640 1 -- 172.21.15.62:0/1821145592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c81a6960 con 0x7f88c8106a70 2024-09-11T23:58:54.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.727+0000 7f88cdc3d640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8106a70 0x7f88c81030c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009ca0 tx=0x7f88c0031460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:54.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.727+0000 7f88be7fc640 1 -- 172.21.15.62:0/1821145592 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0031850 con 0x7f88c8106a70 2024-09-11T23:58:54.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.727+0000 7f88cf6a0640 1 -- 172.21.15.62:0/1821145592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c81a6bf0 con 0x7f88c8106a70 2024-09-11T23:58:54.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.727+0000 7f88be7fc640 1 -- 172.21.15.62:0/1821145592 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c0004520 con 0x7f88c8106a70 2024-09-11T23:58:54.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.727+0000 7f88be7fc640 1 -- 172.21.15.62:0/1821145592 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c002fd90 con 0x7f88c8106a70 2024-09-11T23:58:54.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.728+0000 7f88cf6a0640 1 -- 172.21.15.62:0/1821145592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c81a7180 con 0x7f88c8106a70 2024-09-11T23:58:54.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.731+0000 7f88be7fc640 1 -- 172.21.15.62:0/1821145592 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f88c00046d0 con 0x7f88c8106a70 2024-09-11T23:58:54.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.732+0000 7f88d1a79740 1 Processor -- start 2024-09-11T23:58:54.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.732+0000 7f88d1a79740 1 -- start start 2024-09-11T23:58:54.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.732+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.732+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a65d0 0x5565b03a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.732+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.732+0000 7f88d1a79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5565b030c2f0 con 0x5565b03a7b40 2024-09-11T23:58:54.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.732+0000 7f88d1a79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5565b02b3cf0 con 0x5565b03a65d0 2024-09-11T23:58:54.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.732+0000 7f88d1a79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565b03082b0 con 0x5565b03b1370 2024-09-11T23:58:54.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.732+0000 7f88be7fc640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88b40a2ef0 0x7f88b40a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88ccc3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88cdc3d640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88b40a2ef0 0x7f88b40a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88ccc3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52806/0 (socket says 172.21.15.62:52806) 2024-09-11T23:58:54.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88ccc3b640 1 -- 172.21.15.62:0/1249595663 learned_addr learned my addr 172.21.15.62:0/1249595663 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:54.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/1249595663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88be7fc640 1 -- 172.21.15.62:0/1821145592 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f88c002fb30 con 0x7f88c8106a70 2024-09-11T23:58:54.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88cd43c640 1 --2- 172.21.15.62:0/1249595663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a65d0 0x5565b03a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88ccc3b640 1 -- 172.21.15.62:0/1249595663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 msgr2=0x5565b03b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:54.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/1249595663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88ccc3b640 1 -- 172.21.15.62:0/1249595663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a65d0 msgr2=0x5565b03a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/1249595663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a65d0 0x5565b03a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.733+0000 7f88ccc3b640 1 -- 172.21.15.62:0/1249595663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b02983a0 con 0x5565b03a7b40 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.734+0000 7f88cdc3d640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88b40a2ef0 0x7f88b40a53b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f88c00ee030 tx=0x7f88c002f8c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.734+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/1249595663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03a6090 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009fd0 tx=0x7f88b80090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28009dcfe12ab5a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.735+0000 7f88b37fe640 1 -- 172.21.15.62:0/1249595663 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8005970 con 0x5565b03a7b40 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.735+0000 7f88b37fe640 1 -- 172.21.15.62:0/1249595663 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b80136e0 con 0x5565b03a7b40 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.735+0000 7f88b37fe640 1 -- 172.21.15.62:0/1249595663 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8006020 con 0x5565b03a7b40 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.736+0000 7f88d1a79740 1 -- 172.21.15.62:0/1249595663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 msgr2=0x5565b03a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.736+0000 7f88d1a79740 1 --2- 172.21.15.62:0/1249595663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03a6090 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009fd0 tx=0x7f88b80090e0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.736+0000 7f88d1a79740 1 -- 172.21.15.62:0/1249595663 shutdown_connections 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.736+0000 7f88d1a79740 1 --2- 172.21.15.62:0/1249595663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.736+0000 7f88d1a79740 1 --2- 172.21.15.62:0/1249595663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a65d0 0x5565b03a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.736+0000 7f88d1a79740 1 --2- 172.21.15.62:0/1249595663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03a6090 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.736+0000 7f88d1a79740 1 -- 172.21.15.62:0/1249595663 >> 172.21.15.62:0/1249595663 conn(0x5565b036f310 msgr2=0x5565b03a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:54.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.736+0000 7f88d1a79740 1 -- 172.21.15.62:0/1249595663 shutdown_connections 2024-09-11T23:58:54.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.736+0000 7f88d1a79740 1 -- 172.21.15.62:0/1249595663 wait complete. 2024-09-11T23:58:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.737+0000 7f88d1a79740 1 -- start start 2024-09-11T23:58:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.737+0000 7f88d1a79740 1 Processor -- start 2024-09-11T23:58:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.737+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.737+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 0x5565b03ae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.737+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.737+0000 7f88d1a79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5565b03b4010 con 0x5565b03a65d0 2024-09-11T23:58:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.737+0000 7f88d1a79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5565b02c7060 con 0x5565b03a7b40 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.737+0000 7f88d1a79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565b03b4190 con 0x5565b03b1370 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.738+0000 7f88d1a79740 10 client.0 _ll_register_callbacks cb 0x5565b001a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.738+0000 7f88cd43c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 0x5565b03ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.738+0000 7f88ccc3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.738+0000 7f88ccc3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52820/0 (socket says 172.21.15.62:52820) 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.738+0000 7f88cd43c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 0x5565b03ae010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:46580/0 (socket says 172.21.15.62:46580) 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.738+0000 7f88ccc3b640 1 -- 172.21.15.62:0/521779727 learned_addr learned my addr 172.21.15.62:0/521779727 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88cd43c640 1 -- 172.21.15.62:0/521779727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 msgr2=0x5565b03ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88cd43c640 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88cd43c640 1 -- 172.21.15.62:0/521779727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 msgr2=0x5565b03b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88cd43c640 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88cd43c640 1 -- 172.21.15.62:0/521779727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b007d3c0 con 0x5565b03a7b40 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88cd43c640 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 0x5565b03ae010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88c40058c0 tx=0x7f88c4012ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad3059934dd31dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88b0ff9640 1 -- 172.21.15.62:0/521779727 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4010d90 con 0x5565b03a7b40 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88d1a79740 1 -- 172.21.15.62:0/521779727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b80136e0 con 0x5565b03a7b40 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.739+0000 7f88b0ff9640 1 -- 172.21.15.62:0/521779727 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f88c4005020 con 0x5565b03a7b40 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88b0ff9640 1 -- 172.21.15.62:0/521779727 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88c4005220 con 0x5565b03a7b40 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88d1a79740 10 client.26171 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88d1a79740 1 -- 172.21.15.62:0/521779727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565b03d0ba0 con 0x5565b03a7b40 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88b0ff9640 1 -- 172.21.15.62:0/521779727 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b80136e0 con 0x5565b03a7b40 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88b0ff9640 1 -- 172.21.15.62:0/521779727 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f88c40234c0 con 0x5565b03a7b40 2024-09-11T23:58:54.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88d1a79740 10 client.26171 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88d1a79740 10 client.26171 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88d1a79740 20 client.26171 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88d1a79740 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5565b03d24d0 0x5565b03d4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88d1a79740 4 client.26171 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88d1a79740 1 -- 172.21.15.62:0/521779727 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565b0319860 con 0x5565b03d24d0 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.740+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5565b03d24d0 0x5565b03d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.741+0000 7f88b0ff9640 10 client.26171 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.741+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5565b03d24d0 0x5565b03d4910 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f59bd3f1bc9abac server_cookie=cd8604847d107f1b in_seq=0 out_seq=0 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.741+0000 7f88b0ff9640 1 -- 172.21.15.62:0/521779727 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565b0319860 con 0x5565b03d24d0 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.741+0000 7f88b0ff9640 10 client.26171 handle_command_reply: tid=0 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.741+0000 7f88d1a79740 1 client.26171 shutdown 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.741+0000 7f88d1a79740 1 -- 172.21.15.62:0/521779727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 msgr2=0x5565b03ae010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.741+0000 7f88d1a79740 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 0x5565b03ae010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88c40058c0 tx=0x7f88c4012ca0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.741+0000 7f88d1a79740 1 -- 172.21.15.62:0/521779727 shutdown_connections 2024-09-11T23:58:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.741+0000 7f88d1a79740 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5565b03d24d0 0x5565b03d4910 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.742+0000 7f88d1a79740 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.742+0000 7f88d1a79740 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 0x5565b03ae010 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.742+0000 7f88d1a79740 1 --2- 172.21.15.62:0/521779727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.742+0000 7f88d1a79740 1 -- 172.21.15.62:0/521779727 >> 172.21.15.62:0/521779727 conn(0x5565b036f310 msgr2=0x5565b03a8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.742+0000 7f88b0ff9640 0 client.26171 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:58:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.742+0000 7f88d1a79740 1 -- 172.21.15.62:0/521779727 shutdown_connections 2024-09-11T23:58:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.742+0000 7f88d1a79740 1 -- 172.21.15.62:0/521779727 wait complete. 2024-09-11T23:58:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.742+0000 7f88d1a79740 20 client.26171 trim_cache size 0 max 16384 2024-09-11T23:58:54.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88d1a79740 1 Processor -- start 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88d1a79740 1 -- start start 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a65d0 0x5565b03ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88d1a79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81adfd0 con 0x5565b03a7b40 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88d1a79740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81ab870 con 0x5565b03a65d0 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88d1a79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81a9700 con 0x5565b03b1370 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88cd43c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88cd43c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38854/0 (socket says 172.21.15.62:38854) 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88ccc3b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a65d0 0x5565b03ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.761+0000 7f88cd43c640 1 -- 172.21.15.62:0/431729142 learned_addr learned my addr 172.21.15.62:0/431729142 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.762+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/431729142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.762+0000 7f88cd43c640 1 -- 172.21.15.62:0/431729142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 msgr2=0x5565b03ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:54.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.762+0000 7f88cd43c640 1 --2- 172.21.15.62:0/431729142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.762+0000 7f88cd43c640 1 -- 172.21.15.62:0/431729142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a65d0 msgr2=0x5565b03ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:54.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.762+0000 7f88cd43c640 1 --2- 172.21.15.62:0/431729142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a65d0 0x5565b03ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.762+0000 7f88cd43c640 1 -- 172.21.15.62:0/431729142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b03b10d0 con 0x5565b03a7b40 2024-09-11T23:58:54.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.762+0000 7f88cd43c640 1 --2- 172.21.15.62:0/431729142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03ac8e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f88c40238e0 tx=0x7f88c4023d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89db3de10a6769 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:54.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.762+0000 7f88b2ffd640 1 -- 172.21.15.62:0/431729142 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4031070 con 0x5565b03a7b40 2024-09-11T23:58:54.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.762+0000 7f88b2ffd640 1 -- 172.21.15.62:0/431729142 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c4023630 con 0x5565b03a7b40 2024-09-11T23:58:54.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.762+0000 7f88b2ffd640 1 -- 172.21.15.62:0/431729142 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c402c460 con 0x5565b03a7b40 2024-09-11T23:58:54.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 -- 172.21.15.62:0/431729142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 msgr2=0x5565b03ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:54.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 --2- 172.21.15.62:0/431729142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03ac8e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f88c40238e0 tx=0x7f88c4023d90 comp rx=0 tx=0).stop 2024-09-11T23:58:54.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 -- 172.21.15.62:0/431729142 shutdown_connections 2024-09-11T23:58:54.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 --2- 172.21.15.62:0/431729142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b03ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 --2- 172.21.15.62:0/431729142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a7b40 0x5565b03ac8e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 --2- 172.21.15.62:0/431729142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a65d0 0x5565b03ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 -- 172.21.15.62:0/431729142 >> 172.21.15.62:0/431729142 conn(0x5565b036f310 msgr2=0x5565b03a92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 -- 172.21.15.62:0/431729142 shutdown_connections 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 -- 172.21.15.62:0/431729142 wait complete. 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 -- start start 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.763+0000 7f88d1a79740 1 Processor -- start 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.764+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88d1a79740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 0x5565b03b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88ccc3b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88ccc3b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38862/0 (socket says 172.21.15.62:38862) 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88ccc3b640 1 -- 172.21.15.62:0/327830553 learned_addr learned my addr 172.21.15.62:0/327830553 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88d1a79740 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b043b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81a9d50 con 0x5565b03a65d0 2024-09-11T23:58:54.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81ab1a0 con 0x5565b03a7b40 2024-09-11T23:58:54.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81aa400 con 0x5565b03b1370 2024-09-11T23:58:54.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88d1a79740 10 client.0 _ll_register_callbacks cb 0x7f88c400a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:54.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b043b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88ccc3b640 1 -- 172.21.15.62:0/327830553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 msgr2=0x5565b043b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:54.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b043b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88ccc3b640 1 -- 172.21.15.62:0/327830553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 msgr2=0x5565b03b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:54.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 0x5565b03b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.765+0000 7f88ccc3b640 1 -- 172.21.15.62:0/327830553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b043b720 con 0x5565b03a65d0 2024-09-11T23:58:54.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.766+0000 7f88ccc3b640 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b49b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009f70 tx=0x7f88b802ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67b8e83b9a82aa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:54.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.766+0000 7f88b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:58:54.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.774+0000 7f88b0ff9640 1 -- 172.21.15.62:0/327830553 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8004050 con 0x5565b03a65d0 2024-09-11T23:58:54.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.775+0000 7f88b0ff9640 1 -- 172.21.15.62:0/327830553 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f88b801d040 con 0x5565b03a65d0 2024-09-11T23:58:54.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.781+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c4023630 con 0x5565b03a65d0 2024-09-11T23:58:54.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.786+0000 7f88b0ff9640 1 -- 172.21.15.62:0/327830553 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88b80041f0 con 0x5565b03a65d0 2024-09-11T23:58:54.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.786+0000 7f88d1a79740 10 client.16569 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:54.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.786+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565b043c940 con 0x5565b03a65d0 2024-09-11T23:58:54.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.787+0000 7f88b0ff9640 1 -- 172.21.15.62:0/327830553 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c4023630 con 0x5565b03a65d0 2024-09-11T23:58:54.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.787+0000 7f88b0ff9640 1 -- 172.21.15.62:0/327830553 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f88b8003690 con 0x5565b03a65d0 2024-09-11T23:58:54.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.787+0000 7f88d1a79740 10 client.16569 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:54.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.787+0000 7f88d1a79740 10 client.16569 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:58:54.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.787+0000 7f88d1a79740 20 client.16569 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:54.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.787+0000 7f88d1a79740 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5565b043cc20 0x5565b043f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:54.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.788+0000 7f88d1a79740 4 client.16569 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:58:54.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.788+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565b0313d70 con 0x5565b043cc20 2024-09-11T23:58:54.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.788+0000 7f88cd43c640 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5565b043cc20 0x5565b043f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:54.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.791+0000 7f88cd43c640 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5565b043cc20 0x5565b043f010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=198e4914b2873af0 server_cookie=6b3c2915c86b96f4 in_seq=0 out_seq=0 2024-09-11T23:58:54.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.791+0000 7f88b0ff9640 10 client.16569 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:58:54.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.792+0000 7f88b0ff9640 1 -- 172.21.15.62:0/327830553 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565b0313d70 con 0x5565b043cc20 2024-09-11T23:58:54.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.792+0000 7f88b0ff9640 10 client.16569 handle_command_reply: tid=0 2024-09-11T23:58:54.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.792+0000 7f88d1a79740 1 client.16569 shutdown 2024-09-11T23:58:54.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 msgr2=0x5565b03b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:54.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88d1a79740 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b49b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009f70 tx=0x7f88b802ae80 comp rx=0 tx=0).stop 2024-09-11T23:58:54.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 shutdown_connections 2024-09-11T23:58:54.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88d1a79740 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5565b043cc20 0x5565b043f010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88d1a79740 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565b03b1370 0x5565b043b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88d1a79740 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565b03a7b40 0x5565b03b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88d1a79740 1 --2- 172.21.15.62:0/327830553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565b03a65d0 0x5565b03b49b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 >> 172.21.15.62:0/327830553 conn(0x5565b036f310 msgr2=0x5565b03ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88b0ff9640 0 client.16569 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:58:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 shutdown_connections 2024-09-11T23:58:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.793+0000 7f88d1a79740 1 -- 172.21.15.62:0/327830553 wait complete. 2024-09-11T23:58:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.794+0000 7f88d1a79740 20 client.16569 trim_cache size 0 max 16384 2024-09-11T23:58:54.793 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:58:54.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 -- 172.21.15.62:0/1821145592 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88b40a2ef0 msgr2=0x7f88b40a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88b40a2ef0 0x7f88b40a53b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f88c00ee030 tx=0x7f88c002f8c0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 -- 172.21.15.62:0/1821145592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8106a70 msgr2=0x7f88c81030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8106a70 0x7f88c81030c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009ca0 tx=0x7f88c0031460 comp rx=0 tx=0).stop 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 -- 172.21.15.62:0/1821145592 shutdown_connections 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88b40a2ef0 0x7f88b40a53b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8106a70 0x7f88c81030c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88c8101410 0x7f88c8102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 --2- 172.21.15.62:0/1821145592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8100a70 0x7f88c8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 -- 172.21.15.62:0/1821145592 >> 172.21.15.62:0/1821145592 conn(0x7f88c80fc790 msgr2=0x7f88c8104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 -- 172.21.15.62:0/1821145592 shutdown_connections 2024-09-11T23:58:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:54.795+0000 7f88cf6a0640 1 -- 172.21.15.62:0/1821145592 wait complete. 2024-09-11T23:58:54.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:54 smithi083 ceph-mon[34805]: pgmap v207: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 898 op/s 2024-09-11T23:58:54.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:54 smithi062 ceph-mon[30958]: pgmap v207: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 898 op/s 2024-09-11T23:58:55.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:54 smithi120 ceph-mon[35182]: pgmap v207: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 898 op/s 2024-09-11T23:58:55.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:58:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:56.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:56 smithi083 ceph-mon[34805]: pgmap v208: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.1 MiB/s wr, 610 op/s 2024-09-11T23:58:57.161 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:56 smithi120 ceph-mon[35182]: pgmap v208: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.1 MiB/s wr, 610 op/s 2024-09-11T23:58:57.161 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:56 smithi062 ceph-mon[30958]: pgmap v208: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.1 MiB/s wr, 610 op/s 2024-09-11T23:58:57.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.075+0000 7fc3df280640 1 -- 172.21.15.62:0/3665533760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d8102570 msgr2=0x7fc3d81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:57.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.075+0000 7fc3df280640 1 --2- 172.21.15.62:0/3665533760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d8102570 0x7fc3d81029b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009900 tx=0x7fc3c802a480 comp rx=0 tx=0).stop 2024-09-11T23:58:57.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.076+0000 7fc3df280640 1 -- 172.21.15.62:0/3665533760 shutdown_connections 2024-09-11T23:58:57.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.076+0000 7fc3df280640 1 --2- 172.21.15.62:0/3665533760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d8102ef0 0x7fc3d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.076+0000 7fc3df280640 1 --2- 172.21.15.62:0/3665533760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d8102570 0x7fc3d81029b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.076+0000 7fc3df280640 1 --2- 172.21.15.62:0/3665533760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3d8100e20 0x7fc3d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.076+0000 7fc3df280640 1 -- 172.21.15.62:0/3665533760 >> 172.21.15.62:0/3665533760 conn(0x7fc3d80fc7d0 msgr2=0x7fc3d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.076+0000 7fc3df280640 1 -- 172.21.15.62:0/3665533760 shutdown_connections 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.076+0000 7fc3df280640 1 -- 172.21.15.62:0/3665533760 wait complete. 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.077+0000 7fc3df280640 1 Processor -- start 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.077+0000 7fc3df280640 1 -- start start 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.077+0000 7fc3df280640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d8100e20 0x7fc3d8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.077+0000 7fc3df280640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3d8102570 0x7fc3d8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.077+0000 7fc3df280640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d8102ef0 0x7fc3d819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.077+0000 7fc3df280640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81120d0 con 0x7fc3d8100e20 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.077+0000 7fc3df280640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8111f50 con 0x7fc3d8102ef0 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.077+0000 7fc3df280640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8112250 con 0x7fc3d8102570 2024-09-11T23:58:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dc81b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3d8102570 0x7fc3d8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dc81b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3d8102570 0x7fc3d8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52860/0 (socket says 172.21.15.62:52860) 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dc81b640 1 -- 172.21.15.62:0/3022565565 learned_addr learned my addr 172.21.15.62:0/3022565565 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dd81d640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d8102ef0 0x7fc3d819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d8100e20 0x7fc3d8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dd81d640 1 -- 172.21.15.62:0/3022565565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3d8102570 msgr2=0x7fc3d8198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dd81d640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3d8102570 0x7fc3d8198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dd81d640 1 -- 172.21.15.62:0/3022565565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d8100e20 msgr2=0x7fc3d8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dd81d640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d8100e20 0x7fc3d8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dd81d640 1 -- 172.21.15.62:0/3022565565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d819d540 con 0x7fc3d8102ef0 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.078+0000 7fc3dc81b640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3d8102570 0x7fc3d8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.079+0000 7fc3dd81d640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d8102ef0 0x7fc3d819ce40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009fd0 tx=0x7fc3d000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.079+0000 7fc3ce7fc640 1 -- 172.21.15.62:0/3022565565 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000cd60 con 0x7fc3d8102ef0 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.079+0000 7fc3ce7fc640 1 -- 172.21.15.62:0/3022565565 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d0005020 con 0x7fc3d8102ef0 2024-09-11T23:58:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.079+0000 7fc3df280640 1 -- 172.21.15.62:0/3022565565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d819d830 con 0x7fc3d8102ef0 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.079+0000 7fc3ce7fc640 1 -- 172.21.15.62:0/3022565565 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0010690 con 0x7fc3d8102ef0 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.079+0000 7fc3df280640 1 -- 172.21.15.62:0/3022565565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d810e630 con 0x7fc3d8102ef0 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.084+0000 7fc3e1659740 1 Processor -- start 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.084+0000 7fc3ce7fc640 1 -- 172.21.15.62:0/3022565565 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc3d0010830 con 0x7fc3d8102ef0 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.084+0000 7fc3e1659740 1 -- start start 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 0x5600dbe8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe9a370 0x5600dbe9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3e1659740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5600dbdf52f0 con 0x5600dbe8f5d0 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3e1659740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5600dbd9ccf0 con 0x5600dbe90b40 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3e1659740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5600dbdf12b0 con 0x5600dbe9a370 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3dd01c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3dc81b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 0x5600dbe8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3dd01c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38896/0 (socket says 172.21.15.62:38896) 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3dc81b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 0x5600dbe8f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:46622/0 (socket says 172.21.15.62:46622) 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.085+0000 7fc3dd01c640 1 -- 172.21.15.62:0/2174827488 learned_addr learned my addr 172.21.15.62:0/2174827488 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.086+0000 7fc3dd01c640 1 -- 172.21.15.62:0/2174827488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe9a370 msgr2=0x5600dbe9c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.087+0000 7fc3dc81b640 1 --2- 172.21.15.62:0/2174827488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe9a370 0x5600dbe9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.087+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/2174827488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe9a370 0x5600dbe9c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.087+0000 7fc3dd01c640 1 -- 172.21.15.62:0/2174827488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 msgr2=0x5600dbe8f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.087+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/2174827488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 0x5600dbe8f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.087+0000 7fc3dd01c640 1 -- 172.21.15.62:0/2174827488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600dbd813a0 con 0x5600dbe8f5d0 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.087+0000 7fc3dc81b640 1 --2- 172.21.15.62:0/2174827488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe9a370 0x5600dbe9c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.087+0000 7fc3ce7fc640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc3c00a2d70 0x7fc3c00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.087+0000 7fc3dc81b640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc3c00a2d70 0x7fc3c00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.087+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/2174827488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8fa10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009a70 tx=0x7fc3d4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f17e1fc358cf0a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.087+0000 7fc3ce7fc640 1 -- 172.21.15.62:0/3022565565 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc3d00cf9b0 con 0x7fc3d8102ef0 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.088+0000 7fc3bf7fe640 1 -- 172.21.15.62:0/2174827488 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4017070 con 0x5600dbe8f5d0 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.088+0000 7fc3bf7fe640 1 -- 172.21.15.62:0/2174827488 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d4013470 con 0x5600dbe8f5d0 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.088+0000 7fc3bf7fe640 1 -- 172.21.15.62:0/2174827488 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40125d0 con 0x5600dbe8f5d0 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.089+0000 7fc3e1659740 1 -- 172.21.15.62:0/2174827488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 msgr2=0x5600dbe8fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.089+0000 7fc3e1659740 1 --2- 172.21.15.62:0/2174827488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8fa10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009a70 tx=0x7fc3d4009f30 comp rx=0 tx=0).stop 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.089+0000 7fc3dc81b640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc3c00a2d70 0x7fc3c00a5230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005770 tx=0x7fc3c8005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.089+0000 7fc3e1659740 1 -- 172.21.15.62:0/2174827488 shutdown_connections 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.089+0000 7fc3e1659740 1 --2- 172.21.15.62:0/2174827488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe9a370 0x5600dbe9c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.089+0000 7fc3e1659740 1 --2- 172.21.15.62:0/2174827488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8fa10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.089+0000 7fc3e1659740 1 --2- 172.21.15.62:0/2174827488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 0x5600dbe8f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.089+0000 7fc3e1659740 1 -- 172.21.15.62:0/2174827488 >> 172.21.15.62:0/2174827488 conn(0x5600dbe58310 msgr2=0x5600dbe8bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.090+0000 7fc3e1659740 1 -- 172.21.15.62:0/2174827488 shutdown_connections 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.091+0000 7fc3e1659740 1 -- 172.21.15.62:0/2174827488 wait complete. 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.091+0000 7fc3e1659740 1 -- start start 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.091+0000 7fc3e1659740 1 Processor -- start 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe99d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 0x5600dbe96fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe9a370 0x5600dbe97530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3e1659740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5600dbe9d070 con 0x5600dbe8f5d0 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3e1659740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5600dbdb0060 con 0x5600dbe9a370 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3e1659740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5600dbe9d1f0 con 0x5600dbe90b40 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3dc81b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 0x5600dbe96fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3dd81d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe9a370 0x5600dbe97530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3dd81d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe9a370 0x5600dbe97530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:46638/0 (socket says 172.21.15.62:46638) 2024-09-11T23:58:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3dd81d640 1 -- 172.21.15.62:0/1717999137 learned_addr learned my addr 172.21.15.62:0/1717999137 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3e1659740 10 client.0 _ll_register_callbacks cb 0x5600dbb038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.092+0000 7fc3dd01c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe99d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.093+0000 7fc3dd81d640 1 -- 172.21.15.62:0/1717999137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 msgr2=0x5600dbe96fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.093+0000 7fc3dd81d640 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 0x5600dbe96fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.093+0000 7fc3dd81d640 1 -- 172.21.15.62:0/1717999137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 msgr2=0x5600dbe99d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.093+0000 7fc3dd81d640 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe99d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.093+0000 7fc3dd81d640 1 -- 172.21.15.62:0/1717999137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600dbb663c0 con 0x5600dbe9a370 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.093+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe99d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.093+0000 7fc3dd81d640 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe9a370 0x5600dbe97530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0005fd0 tx=0x7fc3d009bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82dd519194b95a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.093+0000 7fc3bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.093+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1717999137 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00b4020 con 0x5600dbe9a370 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.094+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1717999137 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc3d009c160 con 0x5600dbe9a370 2024-09-11T23:58:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.094+0000 7fc3e1659740 1 -- 172.21.15.62:0/1717999137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d4013470 con 0x5600dbe9a370 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.094+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1717999137 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3d00a4b70 con 0x5600dbe9a370 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.094+0000 7fc3e1659740 10 client.26186 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.094+0000 7fc3e1659740 1 -- 172.21.15.62:0/1717999137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600dbea8a70 con 0x5600dbe9a370 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.095+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1717999137 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d4013470 con 0x5600dbe9a370 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.095+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1717999137 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fc3d00bacd0 con 0x5600dbe9a370 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.095+0000 7fc3e1659740 10 client.26186 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.095+0000 7fc3e1659740 10 client.26186 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.095+0000 7fc3e1659740 20 client.26186 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.095+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5600dbea9e80 0x5600dbeac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.095+0000 7fc3e1659740 4 client.26186 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.095+0000 7fc3e1659740 1 -- 172.21.15.62:0/1717999137 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600dbe02860 con 0x5600dbea9e80 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.095+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5600dbea9e80 0x5600dbeac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.096+0000 7fc3bcff9640 10 client.26186 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:58:57.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.096+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5600dbea9e80 0x5600dbeac2d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bdd6cc4fcabf6d5 server_cookie=8c52cbea6e4a4cc4 in_seq=0 out_seq=0 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.096+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1717999137 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600dbe02860 con 0x5600dbea9e80 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.096+0000 7fc3bcff9640 10 client.26186 handle_command_reply: tid=0 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.096+0000 7fc3e1659740 1 client.26186 shutdown 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.096+0000 7fc3e1659740 1 -- 172.21.15.62:0/1717999137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe9a370 msgr2=0x5600dbe97530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.096+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe9a370 0x5600dbe97530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0005fd0 tx=0x7fc3d009bf00 comp rx=0 tx=0).stop 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.097+0000 7fc3e1659740 1 -- 172.21.15.62:0/1717999137 shutdown_connections 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.097+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5600dbea9e80 0x5600dbeac2d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.097+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe9a370 0x5600dbe97530 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.097+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 0x5600dbe96fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.097+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1717999137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe8f5d0 0x5600dbe99d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.097+0000 7fc3e1659740 1 -- 172.21.15.62:0/1717999137 >> 172.21.15.62:0/1717999137 conn(0x5600dbe58310 msgr2=0x5600dbe91960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.097+0000 7fc3bcff9640 0 client.26186 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.097+0000 7fc3e1659740 1 -- 172.21.15.62:0/1717999137 shutdown_connections 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.097+0000 7fc3e1659740 1 -- 172.21.15.62:0/1717999137 wait complete. 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.097+0000 7fc3e1659740 20 client.26186 trim_cache size 0 max 16384 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.122+0000 7fc3e1659740 1 Processor -- start 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.122+0000 7fc3e1659740 1 -- start start 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 0x5600dbea6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe9a370 0x5600dbeb1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3dd01c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3dd01c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52894/0 (socket says 172.21.15.62:52894) 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3dc81b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 0x5600dbea6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3dc81b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 0x5600dbea6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:46654/0 (socket says 172.21.15.62:46654) 2024-09-11T23:58:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3dd01c640 1 -- 172.21.15.62:0/4101784175 learned_addr learned my addr 172.21.15.62:0/4101784175 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3e1659740 1 -- 172.21.15.62:0/4101784175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81adf80 con 0x5600dbe9a370 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3e1659740 1 -- 172.21.15.62:0/4101784175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81ab820 con 0x5600dbe90b40 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.123+0000 7fc3e1659740 1 -- 172.21.15.62:0/4101784175 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81a9670 con 0x5600dbe8f5d0 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.124+0000 7fc3dd81d640 1 --2- 172.21.15.62:0/4101784175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe9a370 0x5600dbeb1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.124+0000 7fc3dd01c640 1 -- 172.21.15.62:0/4101784175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 msgr2=0x5600dbea6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.124+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/4101784175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 0x5600dbea6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.124+0000 7fc3dd01c640 1 -- 172.21.15.62:0/4101784175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe9a370 msgr2=0x5600dbeb1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.124+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/4101784175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe9a370 0x5600dbeb1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.124+0000 7fc3dd01c640 1 -- 172.21.15.62:0/4101784175 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600dbe8f2d0 con 0x5600dbe8f5d0 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.125+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/4101784175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8be10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009a70 tx=0x7fc3d4009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ea1564269dad52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.125+0000 7fc3beffd640 1 -- 172.21.15.62:0/4101784175 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4017040 con 0x5600dbe8f5d0 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.125+0000 7fc3beffd640 1 -- 172.21.15.62:0/4101784175 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d4035070 con 0x5600dbe8f5d0 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.125+0000 7fc3beffd640 1 -- 172.21.15.62:0/4101784175 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d402fbb0 con 0x5600dbe8f5d0 2024-09-11T23:58:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.126+0000 7fc3e1659740 1 -- 172.21.15.62:0/4101784175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe8f5d0 msgr2=0x5600dbe8be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.126+0000 7fc3e1659740 1 --2- 172.21.15.62:0/4101784175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8be10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009a70 tx=0x7fc3d4009110 comp rx=0 tx=0).stop 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.126+0000 7fc3e1659740 1 -- 172.21.15.62:0/4101784175 shutdown_connections 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.127+0000 7fc3e1659740 1 --2- 172.21.15.62:0/4101784175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe9a370 0x5600dbeb1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.127+0000 7fc3e1659740 1 --2- 172.21.15.62:0/4101784175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe90b40 0x5600dbea6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.127+0000 7fc3e1659740 1 --2- 172.21.15.62:0/4101784175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe8f5d0 0x5600dbe8be10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.128+0000 7fc3e1659740 1 -- 172.21.15.62:0/4101784175 >> 172.21.15.62:0/4101784175 conn(0x5600dbe58310 msgr2=0x5600dbe8b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.130+0000 7fc3e1659740 1 -- 172.21.15.62:0/4101784175 shutdown_connections 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.130+0000 7fc3e1659740 1 -- 172.21.15.62:0/4101784175 wait complete. 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.131+0000 7fc3e1659740 1 -- start start 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.131+0000 7fc3e1659740 1 Processor -- start 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.132+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe8f5d0 0x5600dbe9e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.133+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 0x5600dbe9e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.133+0000 7fc3dd01c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe8f5d0 0x5600dbe9e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.133+0000 7fc3e1659740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe9a370 0x5600dbe9edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.133+0000 7fc3e1659740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81a9ce0 con 0x5600dbe9a370 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.133+0000 7fc3e1659740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81ab150 con 0x5600dbe8f5d0 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.133+0000 7fc3dd01c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe8f5d0 0x5600dbe9e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:46670/0 (socket says 172.21.15.62:46670) 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.133+0000 7fc3dd01c640 1 -- 172.21.15.62:0/1795567566 learned_addr learned my addr 172.21.15.62:0/1795567566 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.133+0000 7fc3e1659740 1 -- 172.21.15.62:0/1795567566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81aa3b0 con 0x5600dbe90b40 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.133+0000 7fc3e1659740 10 client.0 _ll_register_callbacks cb 0x7fc3d00981a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.133+0000 7fc3dc81b640 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 0x5600dbe9e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.134+0000 7fc3dc81b640 1 -- 172.21.15.62:0/1795567566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe8f5d0 msgr2=0x5600dbe9e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.134+0000 7fc3dc81b640 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe8f5d0 0x5600dbe9e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.134+0000 7fc3dc81b640 1 -- 172.21.15.62:0/1795567566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe9a370 msgr2=0x5600dbe9edb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.134+0000 7fc3dc81b640 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe9a370 0x5600dbe9edb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.134+0000 7fc3dc81b640 1 -- 172.21.15.62:0/1795567566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600dbf13cd0 con 0x5600dbe90b40 2024-09-11T23:58:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.134+0000 7fc3dc81b640 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 0x5600dbe9e870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009230 tx=0x7fc3c8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1e96c5a80100dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.135+0000 7fc3bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.135+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1795567566 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8057060 con 0x5600dbe90b40 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.135+0000 7fc3e1659740 1 -- 172.21.15.62:0/1795567566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d4035070 con 0x5600dbe90b40 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.135+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1795567566 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fc3c8031c50 con 0x5600dbe90b40 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.135+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1795567566 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3c8035040 con 0x5600dbe90b40 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.135+0000 7fc3e1659740 10 client.26146 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.136+0000 7fc3e1659740 1 -- 172.21.15.62:0/1795567566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600dbf14f90 con 0x5600dbe90b40 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.136+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1795567566 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c8031e50 con 0x5600dbe90b40 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.136+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1795567566 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fc3c8044040 con 0x5600dbe90b40 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.136+0000 7fc3e1659740 10 client.26146 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.136+0000 7fc3e1659740 10 client.26146 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.136+0000 7fc3e1659740 20 client.26146 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.136+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5600dbf15720 0x5600dbf17b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.136+0000 7fc3e1659740 4 client.26146 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.136+0000 7fc3e1659740 1 -- 172.21.15.62:0/1795567566 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc3d4035070 con 0x5600dbf15720 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.137+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5600dbf15720 0x5600dbf17b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3dd01c640 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5600dbf15720 0x5600dbf17b40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b6d5a2302bca6f6 server_cookie=45c398d741c5187 in_seq=0 out_seq=0 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3bcff9640 10 client.26146 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3bcff9640 1 -- 172.21.15.62:0/1795567566 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc3d4035070 con 0x5600dbf15720 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3bcff9640 10 client.26146 handle_command_reply: tid=0 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3e1659740 1 client.26146 shutdown 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3e1659740 1 -- 172.21.15.62:0/1795567566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 msgr2=0x5600dbe9e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 0x5600dbe9e870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009230 tx=0x7fc3c8009260 comp rx=0 tx=0).stop 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3e1659740 1 -- 172.21.15.62:0/1795567566 shutdown_connections 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5600dbf15720 0x5600dbf17b40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600dbe9a370 0x5600dbe9edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600dbe90b40 0x5600dbe9e870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3e1659740 1 --2- 172.21.15.62:0/1795567566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600dbe8f5d0 0x5600dbe9e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3e1659740 1 -- 172.21.15.62:0/1795567566 >> 172.21.15.62:0/1795567566 conn(0x5600dbe58310 msgr2=0x5600dbe8a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:57.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.138+0000 7fc3bcff9640 0 client.26146 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:58:57.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3e1659740 1 -- 172.21.15.62:0/1795567566 shutdown_connections 2024-09-11T23:58:57.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3e1659740 1 -- 172.21.15.62:0/1795567566 wait complete. 2024-09-11T23:58:57.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3e1659740 20 client.26146 trim_cache size 0 max 16384 2024-09-11T23:58:57.190 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:58:57.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3df280640 1 -- 172.21.15.62:0/3022565565 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc3c00a2d70 msgr2=0x7fc3c00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:57.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3df280640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc3c00a2d70 0x7fc3c00a5230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005770 tx=0x7fc3c8005a20 comp rx=0 tx=0).stop 2024-09-11T23:58:57.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3df280640 1 -- 172.21.15.62:0/3022565565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d8102ef0 msgr2=0x7fc3d819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:57.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3df280640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d8102ef0 0x7fc3d819ce40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009fd0 tx=0x7fc3d000ee50 comp rx=0 tx=0).stop 2024-09-11T23:58:57.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3df280640 1 -- 172.21.15.62:0/3022565565 shutdown_connections 2024-09-11T23:58:57.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3df280640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc3c00a2d70 0x7fc3c00a5230 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3df280640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d8102ef0 0x7fc3d819ce40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3df280640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3d8102570 0x7fc3d8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3df280640 1 --2- 172.21.15.62:0/3022565565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d8100e20 0x7fc3d8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:57.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.139+0000 7fc3df280640 1 -- 172.21.15.62:0/3022565565 >> 172.21.15.62:0/3022565565 conn(0x7fc3d80fc7d0 msgr2=0x7fc3d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:57.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.140+0000 7fc3df280640 1 -- 172.21.15.62:0/3022565565 shutdown_connections 2024-09-11T23:58:57.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:57.140+0000 7fc3df280640 1 -- 172.21.15.62:0/3022565565 wait complete. 2024-09-11T23:58:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 375.0 s, 2556.8 tps, lat 39.163 ms stddev 43.546 2024-09-11T23:58:58.230 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:58:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:58:58.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:58:58 smithi083 ceph-mon[34805]: pgmap v209: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 889 op/s 2024-09-11T23:58:58.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:58:58 smithi062 ceph-mon[30958]: pgmap v209: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 889 op/s 2024-09-11T23:58:59.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:58:58 smithi120 ceph-mon[35182]: pgmap v209: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 889 op/s 2024-09-11T23:58:59.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.256+0000 7ff2ed895640 1 -- 172.21.15.62:0/3955989365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e8109e00 msgr2=0x7ff2e810e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:59.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.256+0000 7ff2ed895640 1 --2- 172.21.15.62:0/3955989365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e8109e00 0x7ff2e810e350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000adb0 tx=0x7ff2e002b7a0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.256+0000 7ff2ed895640 1 -- 172.21.15.62:0/3955989365 shutdown_connections 2024-09-11T23:58:59.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.256+0000 7ff2ed895640 1 --2- 172.21.15.62:0/3955989365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2e810ea80 0x7ff2e81154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.256+0000 7ff2ed895640 1 --2- 172.21.15.62:0/3955989365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e8109e00 0x7ff2e810e350 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000adb0 tx=0x7ff2e002b7a0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.256+0000 7ff2ed895640 1 --2- 172.21.15.62:0/3955989365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e8109450 0x7ff2e8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.256+0000 7ff2ed895640 1 -- 172.21.15.62:0/3955989365 >> 172.21.15.62:0/3955989365 conn(0x7ff2e806e2d0 msgr2=0x7ff2e806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:59.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.257+0000 7ff2ed895640 1 -- 172.21.15.62:0/3955989365 shutdown_connections 2024-09-11T23:58:59.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.259+0000 7ff2ed895640 1 -- 172.21.15.62:0/3955989365 wait complete. 2024-09-11T23:58:59.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.259+0000 7ff2ed895640 1 Processor -- start 2024-09-11T23:58:59.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.260+0000 7ff2ed895640 1 -- start start 2024-09-11T23:58:59.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.261+0000 7ff2ed895640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e8109450 0x7ff2e819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.261+0000 7ff2ed895640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2e810ea80 0x7ff2e819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.261+0000 7ff2ed895640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e819f670 0x7ff2e81a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.261+0000 7ff2ed895640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e8118580 con 0x7ff2e810ea80 2024-09-11T23:58:59.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.261+0000 7ff2ed895640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e8118400 con 0x7ff2e819f670 2024-09-11T23:58:59.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.261+0000 7ff2ed895640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e8118700 con 0x7ff2e8109450 2024-09-11T23:58:59.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.263+0000 7ff2e67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2e810ea80 0x7ff2e819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.263+0000 7ff2e67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2e810ea80 0x7ff2e819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38934/0 (socket says 172.21.15.62:38934) 2024-09-11T23:58:59.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.263+0000 7ff2e67fc640 1 -- 172.21.15.62:0/445211950 learned_addr learned my addr 172.21.15.62:0/445211950 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:59.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.264+0000 7ff2e6ffd640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e8109450 0x7ff2e819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.264+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e819f670 0x7ff2e81a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.265+0000 7ff2e77fe640 1 -- 172.21.15.62:0/445211950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e8109450 msgr2=0x7ff2e819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:59.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.265+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e8109450 0x7ff2e819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.265+0000 7ff2e77fe640 1 -- 172.21.15.62:0/445211950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2e810ea80 msgr2=0x7ff2e819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:59.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.265+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2e810ea80 0x7ff2e819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.265+0000 7ff2e77fe640 1 -- 172.21.15.62:0/445211950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e81a3e80 con 0x7ff2e819f670 2024-09-11T23:58:59.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.267+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e819f670 0x7ff2e81a35c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00d680 tx=0x7ff2dc00db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:59.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.268+0000 7ff2ec893640 1 -- 172.21.15.62:0/445211950 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc017070 con 0x7ff2e819f670 2024-09-11T23:58:59.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.268+0000 7ff2ec893640 1 -- 172.21.15.62:0/445211950 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2dc009820 con 0x7ff2e819f670 2024-09-11T23:58:59.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.268+0000 7ff2ec893640 1 -- 172.21.15.62:0/445211950 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc004e00 con 0x7ff2e819f670 2024-09-11T23:58:59.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.268+0000 7ff2ed895640 1 -- 172.21.15.62:0/445211950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e81a4170 con 0x7ff2e819f670 2024-09-11T23:58:59.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.268+0000 7ff2ed895640 1 -- 172.21.15.62:0/445211950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e81abbc0 con 0x7ff2e819f670 2024-09-11T23:58:59.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.273+0000 7ff2efc6e740 1 Processor -- start 2024-09-11T23:58:59.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.273+0000 7ff2ec893640 1 -- 172.21.15.62:0/445211950 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff2dc0099d0 con 0x7ff2e819f670 2024-09-11T23:58:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.274+0000 7ff2ec893640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2d00a2f00 0x7ff2d00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.274+0000 7ff2ec893640 1 -- 172.21.15.62:0/445211950 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7ff2dc0ceb10 con 0x7ff2e819f670 2024-09-11T23:58:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.275+0000 7ff2e67fc640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2d00a2f00 0x7ff2d00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.275+0000 7ff2efc6e740 1 -- start start 2024-09-11T23:58:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.275+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362b9e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.275+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9e5d0 0x565362b9ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.275+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362ba9370 0x565362bab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.275+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565362b042f0 con 0x565362ba9370 2024-09-11T23:58:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.275+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565362aabcf0 con 0x565362b9e5d0 2024-09-11T23:58:59.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.275+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x565362b002b0 con 0x565362b9fb40 2024-09-11T23:58:59.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.276+0000 7ff2e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362b9e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.276+0000 7ff2e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362b9e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52930/0 (socket says 172.21.15.62:52930) 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.276+0000 7ff2e6ffd640 1 -- 172.21.15.62:0/881718537 learned_addr learned my addr 172.21.15.62:0/881718537 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.276+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/881718537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362ba9370 0x565362bab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.276+0000 7ff2e77fe640 1 -- 172.21.15.62:0/881718537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 msgr2=0x565362b9e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.276+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/881718537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362b9e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.276+0000 7ff2e77fe640 1 -- 172.21.15.62:0/881718537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9e5d0 msgr2=0x565362b9ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.276+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/881718537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9e5d0 0x565362b9ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.276+0000 7ff2e77fe640 1 -- 172.21.15.62:0/881718537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565362a903a0 con 0x565362ba9370 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.277+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/881718537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362ba9370 0x565362bab760 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0cc230 tx=0x7ff2dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a520849f5f466987 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.277+0000 7ff2cdffb640 1 -- 172.21.15.62:0/881718537 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc020040 con 0x565362ba9370 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.277+0000 7ff2cdffb640 1 -- 172.21.15.62:0/881718537 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2dc0a3c70 con 0x565362ba9370 2024-09-11T23:58:59.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.277+0000 7ff2cdffb640 1 -- 172.21.15.62:0/881718537 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc09a500 con 0x565362ba9370 2024-09-11T23:58:59.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.278+0000 7ff2efc6e740 1 -- 172.21.15.62:0/881718537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362ba9370 msgr2=0x565362bab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:59.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.278+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/881718537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362ba9370 0x565362bab760 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0cc230 tx=0x7ff2dc005840 comp rx=0 tx=0).stop 2024-09-11T23:58:59.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.278+0000 7ff2efc6e740 1 -- 172.21.15.62:0/881718537 shutdown_connections 2024-09-11T23:58:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.278+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/881718537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362ba9370 0x565362bab760 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.278+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/881718537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9e5d0 0x565362b9ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.278+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/881718537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362b9e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.278+0000 7ff2efc6e740 1 -- 172.21.15.62:0/881718537 >> 172.21.15.62:0/881718537 conn(0x565362b67310 msgr2=0x565362b9ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.279+0000 7ff2efc6e740 1 -- 172.21.15.62:0/881718537 shutdown_connections 2024-09-11T23:58:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.279+0000 7ff2efc6e740 1 -- 172.21.15.62:0/881718537 wait complete. 2024-09-11T23:58:59.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.279+0000 7ff2efc6e740 1 -- start start 2024-09-11T23:58:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.280+0000 7ff2efc6e740 1 Processor -- start 2024-09-11T23:58:59.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.281+0000 7ff2e67fc640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2d00a2f00 0x7ff2d00a53c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0000f80 tx=0x7ff2e000a950 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:59.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.283+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9e5d0 0x565362ba8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.283+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9fb40 0x565362ba6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362ba9370 0x565362ba65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565362bac010 con 0x565362ba9370 2024-09-11T23:58:59.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9e5d0 0x565362ba8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9e5d0 0x565362ba8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52938/0 (socket says 172.21.15.62:52938) 2024-09-11T23:58:59.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2e6ffd640 1 -- 172.21.15.62:0/1935180179 learned_addr learned my addr 172.21.15.62:0/1935180179 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:59.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565362abf060 con 0x565362b9fb40 2024-09-11T23:58:59.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2efc6e740 1 -- 172.21.15.62:0/1935180179 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x565362bac190 con 0x565362b9e5d0 2024-09-11T23:58:59.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2efc6e740 10 client.0 _ll_register_callbacks cb 0x5653628128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:59.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2e6ffd640 1 -- 172.21.15.62:0/1935180179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9fb40 msgr2=0x565362ba6000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:58:59.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2e6ffd640 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9fb40 0x565362ba6000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2e6ffd640 1 -- 172.21.15.62:0/1935180179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362ba9370 msgr2=0x565362ba65d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:58:59.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2e6ffd640 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362ba9370 0x565362ba65d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.284+0000 7ff2e6ffd640 1 -- 172.21.15.62:0/1935180179 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653628753c0 con 0x565362b9e5d0 2024-09-11T23:58:59.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.285+0000 7ff2e6ffd640 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9e5d0 0x565362ba8d50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80126b0 tx=0x7ff2d8012b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af06b17184bf2008 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:59.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.285+0000 7ff2c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:58:59.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.285+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/1935180179 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d800f920 con 0x565362b9e5d0 2024-09-11T23:58:59.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.285+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/1935180179 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7ff2d800fac0 con 0x565362b9e5d0 2024-09-11T23:58:59.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.286+0000 7ff2efc6e740 1 -- 172.21.15.62:0/1935180179 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2dc0a3c70 con 0x565362b9e5d0 2024-09-11T23:58:59.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.286+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/1935180179 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff2d800fcc0 con 0x565362b9e5d0 2024-09-11T23:58:59.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.286+0000 7ff2efc6e740 10 client.26158 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:59.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.286+0000 7ff2efc6e740 1 -- 172.21.15.62:0/1935180179 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565362bb7af0 con 0x565362b9e5d0 2024-09-11T23:58:59.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.287+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/1935180179 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2d800fe60 con 0x565362b9e5d0 2024-09-11T23:58:59.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.287+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/1935180179 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7ff2d8010550 con 0x565362b9e5d0 2024-09-11T23:58:59.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.287+0000 7ff2efc6e740 10 client.26158 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:59.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.287+0000 7ff2efc6e740 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:59.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.287+0000 7ff2efc6e740 20 client.26158 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:59.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.287+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565362bb8f00 0x565362bbb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.287+0000 7ff2efc6e740 4 client.26158 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:58:59.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.288+0000 7ff2efc6e740 1 -- 172.21.15.62:0/1935180179 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff2dc0a3c70 con 0x565362bb8f00 2024-09-11T23:58:59.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.290+0000 7ff2e67fc640 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565362bb8f00 0x565362bbb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.290+0000 7ff2c6ffd640 10 client.26158 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:59.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.291+0000 7ff2e67fc640 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565362bb8f00 0x565362bbb390 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87e8c774675c9c21 server_cookie=13fd1a954bc485df in_seq=0 out_seq=0 2024-09-11T23:58:59.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.292+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/1935180179 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff2dc0a3c70 con 0x565362bb8f00 2024-09-11T23:58:59.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.292+0000 7ff2c6ffd640 10 client.26158 handle_command_reply: tid=0 2024-09-11T23:58:59.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.292+0000 7ff2efc6e740 1 client.26158 shutdown 2024-09-11T23:58:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.292+0000 7ff2efc6e740 1 -- 172.21.15.62:0/1935180179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9e5d0 msgr2=0x565362ba8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.293+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9e5d0 0x565362ba8d50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80126b0 tx=0x7ff2d8012b70 comp rx=0 tx=0).stop 2024-09-11T23:58:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.293+0000 7ff2efc6e740 1 -- 172.21.15.62:0/1935180179 shutdown_connections 2024-09-11T23:58:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.293+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565362bb8f00 0x565362bbb390 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.293+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362ba9370 0x565362ba65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.293+0000 7ff2c6ffd640 0 client.26158 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.293+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9fb40 0x565362ba6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.293+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/1935180179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9e5d0 0x565362ba8d50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.293+0000 7ff2efc6e740 1 -- 172.21.15.62:0/1935180179 >> 172.21.15.62:0/1935180179 conn(0x565362b67310 msgr2=0x565362ba0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:59.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.293+0000 7ff2efc6e740 1 -- 172.21.15.62:0/1935180179 shutdown_connections 2024-09-11T23:58:59.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.296+0000 7ff2efc6e740 1 -- 172.21.15.62:0/1935180179 wait complete. 2024-09-11T23:58:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.296+0000 7ff2efc6e740 20 client.26158 trim_cache size 0 max 16384 2024-09-11T23:58:59.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.316+0000 7ff2efc6e740 1 Processor -- start 2024-09-11T23:58:59.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2efc6e740 1 -- start start 2024-09-11T23:58:59.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9e5d0 0x565362bb52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9fb40 0x565362ba58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362ba9370 0x565362bc0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e81b6e70 con 0x565362b9fb40 2024-09-11T23:58:59.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e81b4710 con 0x565362b9e5d0 2024-09-11T23:58:59.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e81156f0 con 0x565362ba9370 2024-09-11T23:58:59.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9e5d0 0x565362bb52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9e5d0 0x565362bb52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:46736/0 (socket says 172.21.15.62:46736) 2024-09-11T23:58:59.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2e6ffd640 1 -- 172.21.15.62:0/2464294248 learned_addr learned my addr 172.21.15.62:0/2464294248 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:59.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.317+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/2464294248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362ba9370 0x565362bc0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.318+0000 7ff2e77fe640 1 -- 172.21.15.62:0/2464294248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9e5d0 msgr2=0x565362bb52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:59.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.318+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/2464294248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9e5d0 0x565362bb52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.318+0000 7ff2e77fe640 1 -- 172.21.15.62:0/2464294248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9fb40 msgr2=0x565362ba58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:59.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.318+0000 7ff2e67fc640 1 --2- 172.21.15.62:0/2464294248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9fb40 0x565362ba58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.318+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/2464294248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9fb40 0x565362ba58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.318+0000 7ff2e77fe640 1 -- 172.21.15.62:0/2464294248 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565362ba90c0 con 0x565362ba9370 2024-09-11T23:58:59.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.318+0000 7ff2e67fc640 1 --2- 172.21.15.62:0/2464294248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9fb40 0x565362ba58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:58:59.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.318+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/2464294248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362ba9370 0x565362bc0c60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0cc260 tx=0x7ff2dc09afb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52d5a6e6c62bcf57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:59.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.318+0000 7ff2cd7fa640 1 -- 172.21.15.62:0/2464294248 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0a8070 con 0x565362ba9370 2024-09-11T23:58:59.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.319+0000 7ff2cd7fa640 1 -- 172.21.15.62:0/2464294248 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2dc0cc6b0 con 0x565362ba9370 2024-09-11T23:58:59.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.319+0000 7ff2cd7fa640 1 -- 172.21.15.62:0/2464294248 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc020040 con 0x565362ba9370 2024-09-11T23:58:59.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.320+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2464294248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362ba9370 msgr2=0x565362bc0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:59.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.321+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/2464294248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362ba9370 0x565362bc0c60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0cc260 tx=0x7ff2dc09afb0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.321+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2464294248 shutdown_connections 2024-09-11T23:58:59.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.321+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/2464294248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362ba9370 0x565362bc0c60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.321+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/2464294248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9fb40 0x565362ba58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.321+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/2464294248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362b9e5d0 0x565362bb52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.321+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2464294248 >> 172.21.15.62:0/2464294248 conn(0x565362b67310 msgr2=0x565362b99d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:59.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.321+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2464294248 shutdown_connections 2024-09-11T23:58:59.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.322+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2464294248 wait complete. 2024-09-11T23:58:59.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.323+0000 7ff2efc6e740 1 -- start start 2024-09-11T23:58:59.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.323+0000 7ff2efc6e740 1 Processor -- start 2024-09-11T23:58:59.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9e5d0 0x565362bb1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362ba2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2e6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9e5d0 0x565362bb1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2e6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9e5d0 0x565362bb1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38964/0 (socket says 172.21.15.62:38964) 2024-09-11T23:58:59.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2efc6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362ba9370 0x565362ba35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e81b2bf0 con 0x565362b9e5d0 2024-09-11T23:58:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e81b4040 con 0x565362ba9370 2024-09-11T23:58:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2efc6e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e81b32a0 con 0x565362b9fb40 2024-09-11T23:58:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2efc6e740 10 client.0 _ll_register_callbacks cb 0x7ff2d8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:58:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2e67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362ba2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2e67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362ba2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:52962/0 (socket says 172.21.15.62:52962) 2024-09-11T23:58:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.324+0000 7ff2e67fc640 1 -- 172.21.15.62:0/2049064183 learned_addr learned my addr 172.21.15.62:0/2049064183 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:58:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362ba9370 0x565362ba35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2e67fc640 1 -- 172.21.15.62:0/2049064183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362ba9370 msgr2=0x565362ba35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2e67fc640 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362ba9370 0x565362ba35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2e67fc640 1 -- 172.21.15.62:0/2049064183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9e5d0 msgr2=0x565362bb1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:58:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2e67fc640 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9e5d0 0x565362bb1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2e67fc640 1 -- 172.21.15.62:0/2049064183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565362c22c30 con 0x565362b9fb40 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2e77fe640 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362ba9370 0x565362ba35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2e67fc640 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362ba2e90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0003990 tx=0x7ff2e000a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2294b8251938d077 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/2049064183 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0043070 con 0x565362b9fb40 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.325+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/2049064183 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7ff2e0031d60 con 0x565362b9fb40 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2049064183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2dc0cc6b0 con 0x565362b9fb40 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/2049064183 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff2e0004b40 con 0x565362b9fb40 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2efc6e740 10 client.26164 fetch_fsmap learned FSMap version 30 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2049064183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565362c23d90 con 0x565362b9fb40 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/2049064183 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2dc0cc6b0 con 0x565362b9fb40 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/2049064183 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7ff2e0005020 con 0x565362b9fb40 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2efc6e740 10 client.26164 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:58:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2efc6e740 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:58:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2efc6e740 20 client.26164 populate_metadata read hostname 'smithi062' 2024-09-11T23:58:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565362bb8f00 0x565362c26580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:58:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2efc6e740 4 client.26164 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:58:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.326+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2049064183 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565362b11860 con 0x565362bb8f00 2024-09-11T23:58:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.327+0000 7ff2e6ffd640 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565362bb8f00 0x565362c26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:58:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.327+0000 7ff2c6ffd640 10 client.26164 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.327+0000 7ff2e6ffd640 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565362bb8f00 0x565362c26580 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e60f977a7b093708 server_cookie=fbcd3f02d09652b in_seq=0 out_seq=0 2024-09-11T23:58:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.327+0000 7ff2c6ffd640 1 -- 172.21.15.62:0/2049064183 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565362b11860 con 0x565362bb8f00 2024-09-11T23:58:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.327+0000 7ff2c6ffd640 10 client.26164 handle_command_reply: tid=0 2024-09-11T23:58:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.327+0000 7ff2efc6e740 1 client.26164 shutdown 2024-09-11T23:58:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.328+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2049064183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 msgr2=0x565362ba2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.328+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362ba2e90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0003990 tx=0x7ff2e000a6d0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.328+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2049064183 shutdown_connections 2024-09-11T23:58:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.328+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565362bb8f00 0x565362c26580 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.328+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565362ba9370 0x565362ba35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.328+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565362b9fb40 0x565362ba2e90 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0003990 tx=0x7ff2e000a6d0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.328+0000 7ff2efc6e740 1 --2- 172.21.15.62:0/2049064183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565362b9e5d0 0x565362bb1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.328+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2049064183 >> 172.21.15.62:0/2049064183 conn(0x565362b67310 msgr2=0x565362ba00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.328+0000 7ff2c6ffd640 0 client.26164 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:58:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.329+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2049064183 shutdown_connections 2024-09-11T23:58:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.329+0000 7ff2efc6e740 1 -- 172.21.15.62:0/2049064183 wait complete. 2024-09-11T23:58:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.329+0000 7ff2efc6e740 20 client.26164 trim_cache size 0 max 16384 2024-09-11T23:58:59.334 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:58:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 -- 172.21.15.62:0/445211950 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2d00a2f00 msgr2=0x7ff2d00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2d00a2f00 0x7ff2d00a53c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0000f80 tx=0x7ff2e000a950 comp rx=0 tx=0).stop 2024-09-11T23:58:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 -- 172.21.15.62:0/445211950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e819f670 msgr2=0x7ff2e81a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:58:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e819f670 0x7ff2e81a35c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00d680 tx=0x7ff2dc00db40 comp rx=0 tx=0).stop 2024-09-11T23:58:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 -- 172.21.15.62:0/445211950 shutdown_connections 2024-09-11T23:58:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2d00a2f00 0x7ff2d00a53c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e819f670 0x7ff2e81a35c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2e810ea80 0x7ff2e819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 --2- 172.21.15.62:0/445211950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2e8109450 0x7ff2e819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:58:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 -- 172.21.15.62:0/445211950 >> 172.21.15.62:0/445211950 conn(0x7ff2e806e2d0 msgr2=0x7ff2e8072770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:58:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 -- 172.21.15.62:0/445211950 shutdown_connections 2024-09-11T23:58:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:58:59.330+0000 7ff2ed895640 1 -- 172.21.15.62:0/445211950 wait complete. 2024-09-11T23:59:00.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:00 smithi083 ceph-mon[34805]: pgmap v210: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 593 op/s 2024-09-11T23:59:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:00 smithi062 ceph-mon[30958]: pgmap v210: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 593 op/s 2024-09-11T23:59:01.111 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:59:01.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:59:01.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:00 smithi120 ceph-mon[35182]: pgmap v210: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 593 op/s 2024-09-11T23:59:01.399 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.898+0000 7f79cc3ce640 1 -- 172.21.15.62:0/3260356025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c41082a0 msgr2=0x7f79c410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.898+0000 7f79cc3ce640 1 --2- 172.21.15.62:0/3260356025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c41082a0 0x7f79c410ec40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009cd0 tx=0x7f79bc02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.898+0000 7f79cc3ce640 1 -- 172.21.15.62:0/3260356025 shutdown_connections 2024-09-11T23:59:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.898+0000 7f79cc3ce640 1 --2- 172.21.15.62:0/3260356025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c41082a0 0x7f79c410ec40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.898+0000 7f79cc3ce640 1 --2- 172.21.15.62:0/3260356025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c41036e0 0x7f79c4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.898+0000 7f79cc3ce640 1 --2- 172.21.15.62:0/3260356025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c4102d30 0x7f79c4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.898+0000 7f79cc3ce640 1 -- 172.21.15.62:0/3260356025 >> 172.21.15.62:0/3260356025 conn(0x7f79c40fc7d0 msgr2=0x7f79c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.898+0000 7f79cc3ce640 1 -- 172.21.15.62:0/3260356025 shutdown_connections 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.898+0000 7f79cc3ce640 1 -- 172.21.15.62:0/3260356025 wait complete. 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79cc3ce640 1 Processor -- start 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79cc3ce640 1 -- start start 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79cc3ce640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4102d30 0x7f79c4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79cc3ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c41036e0 0x7f79c4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79cc3ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c41082a0 0x7f79c419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79cc3ce640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41120a0 con 0x7f79c4102d30 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79cc3ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4111f20 con 0x7f79c41036e0 2024-09-11T23:59:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79cc3ce640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4112220 con 0x7f79c41082a0 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79ca16a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4102d30 0x7f79c4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79ca16a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4102d30 0x7f79c4198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38070/0 (socket says 172.21.15.62:38070) 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79ca16a640 1 -- 172.21.15.62:0/969386048 learned_addr learned my addr 172.21.15.62:0/969386048 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79c9969640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c41036e0 0x7f79c4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79ca96b640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c41082a0 0x7f79c419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79ca16a640 1 -- 172.21.15.62:0/969386048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c41082a0 msgr2=0x7f79c419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79ca16a640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c41082a0 0x7f79c419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79ca16a640 1 -- 172.21.15.62:0/969386048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c41036e0 msgr2=0x7f79c4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.899+0000 7f79ca16a640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c41036e0 0x7f79c4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.900+0000 7f79ca16a640 1 -- 172.21.15.62:0/969386048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c419d520 con 0x7f79c4102d30 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.900+0000 7f79ca16a640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4102d30 0x7f79c4198570 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f79c000e9f0 tx=0x7f79c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.900+0000 7f79bb7fe640 1 -- 172.21.15.62:0/969386048 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c000cd80 con 0x7f79c4102d30 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.900+0000 7f79bb7fe640 1 -- 172.21.15.62:0/969386048 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c0005500 con 0x7f79c4102d30 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.900+0000 7f79cc3ce640 1 -- 172.21.15.62:0/969386048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c419d810 con 0x7f79c4102d30 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.900+0000 7f79bb7fe640 1 -- 172.21.15.62:0/969386048 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0010690 con 0x7f79c4102d30 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.900+0000 7f79cc3ce640 1 -- 172.21.15.62:0/969386048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c41a50f0 con 0x7f79c4102d30 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.902+0000 7f79bb7fe640 1 -- 172.21.15.62:0/969386048 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f79c00046c0 con 0x7f79c4102d30 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.903+0000 7f79ce7a7740 1 Processor -- start 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.903+0000 7f79ce7a7740 1 -- start start 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.903+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.903+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 0x55d84d0676f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.903+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.903+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d84cfc3280 con 0x55d84d06cda0 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.903+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d84cf6ab10 con 0x55d84d067d60 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.903+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d84cfbf240 con 0x55d84d06d6c0 2024-09-11T23:59:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79bb7fe640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79b00a2d70 0x7f79b00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79ca96b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79c9969640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79c9969640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37694/0 (socket says 172.21.15.62:37694) 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79c9969640 1 -- 172.21.15.62:0/100022864 learned_addr learned my addr 172.21.15.62:0/100022864 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79ca96b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38074/0 (socket says 172.21.15.62:38074) 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79bb7fe640 1 -- 172.21.15.62:0/969386048 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f79c00cf7a0 con 0x7f79c4102d30 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79ca16a640 1 --2- 172.21.15.62:0/100022864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 0x55d84d0676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79ca96b640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79b00a2d70 0x7f79b00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79c9969640 1 -- 172.21.15.62:0/100022864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 msgr2=0x55d84d0676f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79c9969640 1 --2- 172.21.15.62:0/100022864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 0x55d84d0676f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79c9969640 1 -- 172.21.15.62:0/100022864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 msgr2=0x55d84d06d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79c9969640 1 --2- 172.21.15.62:0/100022864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79c9969640 1 -- 172.21.15.62:0/100022864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84cf771f0 con 0x55d84d067d60 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79ca16a640 1 --2- 172.21.15.62:0/100022864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 0x55d84d0676f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.904+0000 7f79c9969640 1 --2- 172.21.15.62:0/100022864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0681e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f79b400bba0 tx=0x7f79b40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5dd7d3b691e6d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.905+0000 7f79ca96b640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79b00a2d70 0x7f79b00a5230 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f79bc004ab0 tx=0x7f79bc004a40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.905+0000 7f79b8dea640 1 -- 172.21.15.62:0/100022864 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b401c480 con 0x55d84d067d60 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.905+0000 7f79b8dea640 1 -- 172.21.15.62:0/100022864 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b401cb00 con 0x55d84d067d60 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.905+0000 7f79b8dea640 1 -- 172.21.15.62:0/100022864 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4011b80 con 0x55d84d067d60 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.905+0000 7f79ce7a7740 1 -- 172.21.15.62:0/100022864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 msgr2=0x55d84d0681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.905+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/100022864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0681e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f79b400bba0 tx=0x7f79b40107a0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.906+0000 7f79ce7a7740 1 -- 172.21.15.62:0/100022864 shutdown_connections 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.906+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/100022864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0681e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.906+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/100022864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 0x55d84d0676f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.906+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/100022864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.906+0000 7f79ce7a7740 1 -- 172.21.15.62:0/100022864 >> 172.21.15.62:0/100022864 conn(0x55d84d059ba0 msgr2=0x55d84d059f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.906+0000 7f79ce7a7740 1 -- 172.21.15.62:0/100022864 shutdown_connections 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.906+0000 7f79ce7a7740 1 -- 172.21.15.62:0/100022864 wait complete. 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.906+0000 7f79ce7a7740 1 -- start start 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.906+0000 7f79ce7a7740 1 Processor -- start 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d067d60 0x55d84d070c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06cda0 0x55d84d071160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d84d0740e0 con 0x55d84d067d60 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d84cf7df10 con 0x55d84d06d6c0 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d84d074260 con 0x55d84d06cda0 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ce7a7740 10 client.0 _ll_register_callbacks cb 0x55d84cf46a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca96b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca16a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d067d60 0x55d84d070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca16a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d067d60 0x55d84d070c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38080/0 (socket says 172.21.15.62:38080) 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca96b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d0716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37704/0 (socket says 172.21.15.62:37704) 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca16a640 1 -- 172.21.15.62:0/1339996884 learned_addr learned my addr 172.21.15.62:0/1339996884 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79c9969640 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06cda0 0x55d84d071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca96b640 1 -- 172.21.15.62:0/1339996884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06cda0 msgr2=0x55d84d071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca96b640 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06cda0 0x55d84d071160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca96b640 1 -- 172.21.15.62:0/1339996884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d067d60 msgr2=0x55d84d070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca96b640 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d067d60 0x55d84d070c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca96b640 1 -- 172.21.15.62:0/1339996884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d84cd346a0 con 0x55d84d06d6c0 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79ca16a640 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d067d60 0x55d84d070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.907+0000 7f79c9969640 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06cda0 0x55d84d071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:01.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79ca96b640 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d0716a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009ca0 tx=0x7f79bc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9dc7a078c30ef19 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79adffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:59:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79adffb640 1 -- 172.21.15.62:0/1339996884 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0305a0 con 0x55d84d06d6c0 2024-09-11T23:59:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79ce7a7740 1 -- 172.21.15.62:0/1339996884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b401cb00 con 0x55d84d06d6c0 2024-09-11T23:59:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79adffb640 1 -- 172.21.15.62:0/1339996884 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f79bc030740 con 0x55d84d06d6c0 2024-09-11T23:59:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79adffb640 1 -- 172.21.15.62:0/1339996884 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f79bc030940 con 0x55d84d06d6c0 2024-09-11T23:59:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79ce7a7740 10 client.26216 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79ce7a7740 1 -- 172.21.15.62:0/1339996884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d84d07fe20 con 0x55d84d06d6c0 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79adffb640 1 -- 172.21.15.62:0/1339996884 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b401cb00 con 0x55d84d06d6c0 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79adffb640 1 -- 172.21.15.62:0/1339996884 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f79bc048040 con 0x55d84d06d6c0 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.908+0000 7f79ce7a7740 10 client.26216 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.909+0000 7f79ce7a7740 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.909+0000 7f79ce7a7740 20 client.26216 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.909+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d84d081330 0x55d84d0837d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.909+0000 7f79ce7a7740 4 client.26216 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.909+0000 7f79ce7a7740 1 -- 172.21.15.62:0/1339996884 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d84cfd07f0 con 0x55d84d081330 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.909+0000 7f79ca16a640 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d84d081330 0x55d84d0837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.909+0000 7f79adffb640 10 client.26216 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.909+0000 7f79ca16a640 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d84d081330 0x55d84d0837d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88d03f381cc8ff9 server_cookie=7244f104cf44f7e7 in_seq=0 out_seq=0 2024-09-11T23:59:01.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79adffb640 1 -- 172.21.15.62:0/1339996884 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d84cfd07f0 con 0x55d84d081330 2024-09-11T23:59:01.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79adffb640 10 client.26216 handle_command_reply: tid=0 2024-09-11T23:59:01.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 client.26216 shutdown 2024-09-11T23:59:01.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 -- 172.21.15.62:0/1339996884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 msgr2=0x55d84d0716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:01.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d0716a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009ca0 tx=0x7f79bc0095c0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 -- 172.21.15.62:0/1339996884 shutdown_connections 2024-09-11T23:59:01.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d84d081330 0x55d84d0837d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d0716a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79adffb640 0 client.26216 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06cda0 0x55d84d071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/1339996884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d067d60 0x55d84d070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 -- 172.21.15.62:0/1339996884 >> 172.21.15.62:0/1339996884 conn(0x55d84d059ba0 msgr2=0x55d84d059270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 -- 172.21.15.62:0/1339996884 shutdown_connections 2024-09-11T23:59:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 1 -- 172.21.15.62:0/1339996884 wait complete. 2024-09-11T23:59:01.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.910+0000 7f79ce7a7740 20 client.26216 trim_cache size 0 max 16384 2024-09-11T23:59:01.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79ce7a7740 1 Processor -- start 2024-09-11T23:59:01.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79ce7a7740 1 -- start start 2024-09-11T23:59:01.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0676f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 0x55d84d058a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41adf10 con 0x55d84d06cda0 2024-09-11T23:59:01.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41ab7b0 con 0x55d84d067d60 2024-09-11T23:59:01.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41a9600 con 0x55d84d06d6c0 2024-09-11T23:59:01.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79c9969640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79c9969640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38090/0 (socket says 172.21.15.62:38090) 2024-09-11T23:59:01.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79c9969640 1 -- 172.21.15.62:0/3373382540 learned_addr learned my addr 172.21.15.62:0/3373382540 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:01.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79ca16a640 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.930+0000 7f79ca96b640 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 0x55d84d058a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79c9969640 1 -- 172.21.15.62:0/3373382540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 msgr2=0x55d84d058a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79c9969640 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 0x55d84d058a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79c9969640 1 -- 172.21.15.62:0/3373382540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 msgr2=0x55d84d0676f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79c9969640 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0676f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79c9969640 1 -- 172.21.15.62:0/3373382540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84d06c9e0 con 0x55d84d06cda0 2024-09-11T23:59:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79ca96b640 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 0x55d84d058a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79ca16a640 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0676f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79c9969640 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06c410 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f79b4005d00 tx=0x7f79b400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb011d35a048e737 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:01.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79affff640 1 -- 172.21.15.62:0/3373382540 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b400bd50 con 0x55d84d06cda0 2024-09-11T23:59:01.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79affff640 1 -- 172.21.15.62:0/3373382540 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b4005520 con 0x55d84d06cda0 2024-09-11T23:59:01.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.931+0000 7f79affff640 1 -- 172.21.15.62:0/3373382540 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b401c480 con 0x55d84d06cda0 2024-09-11T23:59:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3373382540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 msgr2=0x55d84d06c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06c410 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f79b4005d00 tx=0x7f79b400b8a0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3373382540 shutdown_connections 2024-09-11T23:59:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d06d6c0 0x55d84d058a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d06c410 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/3373382540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d067d60 0x55d84d0676f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3373382540 >> 172.21.15.62:0/3373382540 conn(0x55d84d059ba0 msgr2=0x55d84d06fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3373382540 shutdown_connections 2024-09-11T23:59:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3373382540 wait complete. 2024-09-11T23:59:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 -- start start 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.932+0000 7f79ce7a7740 1 Processor -- start 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d067d60 0x55d84d0798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d079e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ce7a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d088890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41a9c70 con 0x55d84d06cda0 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41ab0e0 con 0x55d84d06d6c0 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ce7a7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41aa340 con 0x55d84d067d60 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ce7a7740 10 client.0 _ll_register_callbacks cb 0x7f79bc009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79c9969640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d079e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ca16a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d067d60 0x55d84d0798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ca16a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d067d60 0x55d84d0798e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:57568/0 (socket says 172.21.15.62:57568) 2024-09-11T23:59:01.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79c9969640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d079e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38098/0 (socket says 172.21.15.62:38098) 2024-09-11T23:59:01.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ca16a640 1 -- 172.21.15.62:0/3773093549 learned_addr learned my addr 172.21.15.62:0/3773093549 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:01.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ca96b640 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d088890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79c9969640 1 -- 172.21.15.62:0/3773093549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d067d60 msgr2=0x55d84d0798e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:01.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79c9969640 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d067d60 0x55d84d0798e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79c9969640 1 -- 172.21.15.62:0/3773093549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 msgr2=0x55d84d088890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:01.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79c9969640 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d088890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79c9969640 1 -- 172.21.15.62:0/3773093549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d84d0890c0 con 0x55d84d06cda0 2024-09-11T23:59:01.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ca16a640 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d067d60 0x55d84d0798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:01.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.933+0000 7f79ca96b640 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d088890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:01.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79c9969640 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d079e20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f79b4011010 tx=0x7f79b400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41e30ed83c779b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:01.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79adffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:59:01.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79adffb640 1 -- 172.21.15.62:0/3773093549 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4004870 con 0x55d84d06cda0 2024-09-11T23:59:01.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79adffb640 1 -- 172.21.15.62:0/3773093549 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7f79b4004020 con 0x55d84d06cda0 2024-09-11T23:59:01.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3773093549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b4005520 con 0x55d84d06cda0 2024-09-11T23:59:01.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79adffb640 1 -- 172.21.15.62:0/3773093549 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f79b4004220 con 0x55d84d06cda0 2024-09-11T23:59:01.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79ce7a7740 10 client.16614 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:01.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3773093549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d84d0ebd80 con 0x55d84d06cda0 2024-09-11T23:59:01.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79adffb640 1 -- 172.21.15.62:0/3773093549 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b4005520 con 0x55d84d06cda0 2024-09-11T23:59:01.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79adffb640 1 -- 172.21.15.62:0/3773093549 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f79b4003800 con 0x55d84d06cda0 2024-09-11T23:59:01.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79ce7a7740 10 client.16614 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:01.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79ce7a7740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:01.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79ce7a7740 20 client.16614 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:01.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d84d0ec280 0x55d84d0ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:01.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79ce7a7740 4 client.16614 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:59:01.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.934+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3773093549 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d84cfcad00 con 0x55d84d0ec280 2024-09-11T23:59:01.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.935+0000 7f79ca16a640 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d84d0ec280 0x55d84d0ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:01.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.935+0000 7f79ca16a640 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d84d0ec280 0x55d84d0ee6a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d4b74cbf4c7c00 server_cookie=2c9f4490e620b5fc in_seq=0 out_seq=0 2024-09-11T23:59:01.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.935+0000 7f79adffb640 10 client.16614 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:01.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79adffb640 1 -- 172.21.15.62:0/3773093549 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d84cfcad00 con 0x55d84d0ec280 2024-09-11T23:59:01.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79adffb640 10 client.16614 handle_command_reply: tid=0 2024-09-11T23:59:01.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 client.16614 shutdown 2024-09-11T23:59:01.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3773093549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 msgr2=0x55d84d079e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:01.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d079e20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f79b4011010 tx=0x7f79b400bf80 comp rx=0 tx=0).stop 2024-09-11T23:59:01.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3773093549 shutdown_connections 2024-09-11T23:59:01.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d84d0ec280 0x55d84d0ee6a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d84d06d6c0 0x55d84d088890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d84d06cda0 0x55d84d079e20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 --2- 172.21.15.62:0/3773093549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d84d067d60 0x55d84d0798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3773093549 >> 172.21.15.62:0/3773093549 conn(0x55d84d059ba0 msgr2=0x55d84d058a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:01.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79adffb640 0 client.16614 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:01.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3773093549 shutdown_connections 2024-09-11T23:59:01.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 1 -- 172.21.15.62:0/3773093549 wait complete. 2024-09-11T23:59:01.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.936+0000 7f79ce7a7740 20 client.16614 trim_cache size 0 max 16384 2024-09-11T23:59:01.947 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:59:01.947 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:59:01.947 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "09432efc-21c2-4038-822e-54cdd5069ff6", 2024-09-11T23:59:01.947 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:59:01.947 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:59:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 -- 172.21.15.62:0/969386048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79b00a2d70 msgr2=0x7f79b00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79b00a2d70 0x7f79b00a5230 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f79bc004ab0 tx=0x7f79bc004a40 comp rx=0 tx=0).stop 2024-09-11T23:59:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 -- 172.21.15.62:0/969386048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4102d30 msgr2=0x7f79c4198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4102d30 0x7f79c4198570 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f79c000e9f0 tx=0x7f79c000edf0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 -- 172.21.15.62:0/969386048 shutdown_connections 2024-09-11T23:59:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79b00a2d70 0x7f79b00a5230 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79c41082a0 0x7f79c419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c41036e0 0x7f79c4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 --2- 172.21.15.62:0/969386048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4102d30 0x7f79c4198570 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 -- 172.21.15.62:0/969386048 >> 172.21.15.62:0/969386048 conn(0x7f79c40fc7d0 msgr2=0x7f79c4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 -- 172.21.15.62:0/969386048 shutdown_connections 2024-09-11T23:59:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:01.937+0000 7f79cc3ce640 1 -- 172.21.15.62:0/969386048 wait complete. 2024-09-11T23:59:02.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 380.0 s, 1986.4 tps, lat 45.184 ms stddev 83.329 2024-09-11T23:59:02.697 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:59:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:02 smithi083 ceph-mon[34805]: pgmap v211: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 605 op/s 2024-09-11T23:59:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:02 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:59:02.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:02 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:02.922 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:02 smithi120 ceph-mon[35182]: pgmap v211: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 605 op/s 2024-09-11T23:59:02.922 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:02 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:59:02.922 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:02 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:02.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:02 smithi062 ceph-mon[30958]: pgmap v211: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 605 op/s 2024-09-11T23:59:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:02 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:59:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:02 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:03.111 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:03.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:03 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:59:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:03 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:03 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:59:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:03 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:59:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:03 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:03.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:03 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:03.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:03 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:59:03.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:03 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:03.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:03 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:59:03.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:03 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:59:03.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:03 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:03.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:03 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:04.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:03 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:59:04.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:03 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:04.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:03 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:59:04.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:03 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:59:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:03 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:04.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:03 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:04.822 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-11T23:59:04.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:59:04.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:04 smithi083 ceph-mon[34805]: pgmap v212: 97 pgs: 97 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 984 op/s 2024-09-11T23:59:04.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:04 smithi062 ceph-mon[30958]: pgmap v212: 97 pgs: 97 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 984 op/s 2024-09-11T23:59:05.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:04 smithi120 ceph-mon[35182]: pgmap v212: 97 pgs: 97 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 984 op/s 2024-09-11T23:59:05.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.166+0000 7fe78ff3f640 1 -- 172.21.15.62:0/3808518254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78810d9b0 msgr2=0x7fe788115510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:05.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.166+0000 7fe78ff3f640 1 --2- 172.21.15.62:0/3808518254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78810d9b0 0x7fe788115510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe780009be0 tx=0x7fe78002a6c0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.166+0000 7fe78ff3f640 1 -- 172.21.15.62:0/3808518254 shutdown_connections 2024-09-11T23:59:05.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.166+0000 7fe78ff3f640 1 --2- 172.21.15.62:0/3808518254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78810d9b0 0x7fe788115510 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.166+0000 7fe78ff3f640 1 --2- 172.21.15.62:0/3808518254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe78810ef30 0x7fe78810d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.166+0000 7fe78ff3f640 1 --2- 172.21.15.62:0/3808518254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe78810e580 0x7fe78810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.166+0000 7fe78ff3f640 1 -- 172.21.15.62:0/3808518254 >> 172.21.15.62:0/3808518254 conn(0x7fe78806e140 msgr2=0x7fe78806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.166+0000 7fe78ff3f640 1 -- 172.21.15.62:0/3808518254 shutdown_connections 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.166+0000 7fe78ff3f640 1 -- 172.21.15.62:0/3808518254 wait complete. 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.167+0000 7fe78ff3f640 1 Processor -- start 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.167+0000 7fe78ff3f640 1 -- start start 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.167+0000 7fe78ff3f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe78810d9b0 0x7fe788109330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.167+0000 7fe78ff3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78810e580 0x7fe788109870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.167+0000 7fe78ff3f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe78810ef30 0x7fe788109db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.167+0000 7fe78ff3f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe788118510 con 0x7fe78810d9b0 2024-09-11T23:59:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.167+0000 7fe78dcdb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe78810d9b0 0x7fe788109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.167+0000 7fe78dcdb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe78810d9b0 0x7fe788109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38120/0 (socket says 172.21.15.62:38120) 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.167+0000 7fe78dcdb640 1 -- 172.21.15.62:0/189037704 learned_addr learned my addr 172.21.15.62:0/189037704 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.168+0000 7fe78ff3f640 1 -- 172.21.15.62:0/189037704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe788118390 con 0x7fe78810e580 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.168+0000 7fe78ff3f640 1 -- 172.21.15.62:0/189037704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe788118690 con 0x7fe78810ef30 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.168+0000 7fe78d4da640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78810e580 0x7fe788109870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.168+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe78810ef30 0x7fe788109db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.168+0000 7fe78e4dc640 1 -- 172.21.15.62:0/189037704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78810e580 msgr2=0x7fe788109870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.168+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78810e580 0x7fe788109870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.168+0000 7fe78e4dc640 1 -- 172.21.15.62:0/189037704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe78810d9b0 msgr2=0x7fe788109330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.168+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe78810d9b0 0x7fe788109330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.168+0000 7fe78e4dc640 1 -- 172.21.15.62:0/189037704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7881af130 con 0x7fe78810ef30 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.168+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe78810ef30 0x7fe788109db0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe780034040 tx=0x7fe7800305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.169+0000 7fe77effd640 1 -- 172.21.15.62:0/189037704 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780041030 con 0x7fe78810ef30 2024-09-11T23:59:05.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.169+0000 7fe77effd640 1 -- 172.21.15.62:0/189037704 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe780003e50 con 0x7fe78810ef30 2024-09-11T23:59:05.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.169+0000 7fe78ff3f640 1 -- 172.21.15.62:0/189037704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7881af420 con 0x7fe78810ef30 2024-09-11T23:59:05.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.169+0000 7fe77effd640 1 -- 172.21.15.62:0/189037704 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780036040 con 0x7fe78810ef30 2024-09-11T23:59:05.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.169+0000 7fe78ff3f640 1 -- 172.21.15.62:0/189037704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7881af8f0 con 0x7fe78810ef30 2024-09-11T23:59:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.173+0000 7fe792318740 1 Processor -- start 2024-09-11T23:59:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.173+0000 7fe77effd640 1 -- 172.21.15.62:0/189037704 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe780044070 con 0x7fe78810ef30 2024-09-11T23:59:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.174+0000 7fe77effd640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7740a2e40 0x7fe7740a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.174+0000 7fe77effd640 1 -- 172.21.15.62:0/189037704 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe7800efd50 con 0x7fe78810ef30 2024-09-11T23:59:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.174+0000 7fe78d4da640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7740a2e40 0x7fe7740a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.175+0000 7fe792318740 1 -- start start 2024-09-11T23:59:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.175+0000 7fe792318740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef858cb0 0x5612ef859090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.175+0000 7fe792318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef863950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.175+0000 7fe792318740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef863e90 0x5612ef866280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.175+0000 7fe792318740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5612ef7b62e0 con 0x5612ef858cb0 2024-09-11T23:59:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.175+0000 7fe792318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612ef75dce0 con 0x5612ef8595d0 2024-09-11T23:59:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.175+0000 7fe792318740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5612ef7b22a0 con 0x5612ef863e90 2024-09-11T23:59:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78d4da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef863950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78e4dc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef863e90 0x5612ef866280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78d4da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef863950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37756/0 (socket says 172.21.15.62:37756) 2024-09-11T23:59:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78e4dc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef863e90 0x5612ef866280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:57584/0 (socket says 172.21.15.62:57584) 2024-09-11T23:59:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78d4da640 1 -- 172.21.15.62:0/1961447509 learned_addr learned my addr 172.21.15.62:0/1961447509 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78dcdb640 1 --2- 172.21.15.62:0/1961447509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef858cb0 0x5612ef859090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78d4da640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7740a2e40 0x7fe7740a5300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe78810adc0 tx=0x7fe784009260 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78d4da640 1 -- 172.21.15.62:0/1961447509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef863e90 msgr2=0x5612ef866280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78d4da640 1 --2- 172.21.15.62:0/1961447509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef863e90 0x5612ef866280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78d4da640 1 -- 172.21.15.62:0/1961447509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef858cb0 msgr2=0x5612ef859090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78d4da640 1 --2- 172.21.15.62:0/1961447509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef858cb0 0x5612ef859090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78d4da640 1 -- 172.21.15.62:0/1961447509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612ef742440 con 0x5612ef8595d0 2024-09-11T23:59:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.176+0000 7fe78d4da640 1 --2- 172.21.15.62:0/1961447509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef863950 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe78400e610 tx=0x7fe784014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6d5d49ac59e9522 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.177+0000 7fe773fff640 1 -- 172.21.15.62:0/1961447509 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784005020 con 0x5612ef8595d0 2024-09-11T23:59:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.177+0000 7fe773fff640 1 -- 172.21.15.62:0/1961447509 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7840047a0 con 0x5612ef8595d0 2024-09-11T23:59:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.177+0000 7fe773fff640 1 -- 172.21.15.62:0/1961447509 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78401e870 con 0x5612ef8595d0 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.178+0000 7fe792318740 1 -- 172.21.15.62:0/1961447509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 msgr2=0x5612ef863950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.178+0000 7fe792318740 1 --2- 172.21.15.62:0/1961447509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef863950 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe78400e610 tx=0x7fe784014000 comp rx=0 tx=0).stop 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.178+0000 7fe792318740 1 -- 172.21.15.62:0/1961447509 shutdown_connections 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.178+0000 7fe792318740 1 --2- 172.21.15.62:0/1961447509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef863e90 0x5612ef866280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.178+0000 7fe792318740 1 --2- 172.21.15.62:0/1961447509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef863950 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.178+0000 7fe792318740 1 --2- 172.21.15.62:0/1961447509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef858cb0 0x5612ef859090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.178+0000 7fe792318740 1 -- 172.21.15.62:0/1961447509 >> 172.21.15.62:0/1961447509 conn(0x5612ef855240 msgr2=0x5612ef855620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.179+0000 7fe792318740 1 -- 172.21.15.62:0/1961447509 shutdown_connections 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.179+0000 7fe792318740 1 -- 172.21.15.62:0/1961447509 wait complete. 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.180+0000 7fe792318740 1 -- start start 2024-09-11T23:59:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.180+0000 7fe792318740 1 Processor -- start 2024-09-11T23:59:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.180+0000 7fe792318740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef858cb0 0x5612ef8637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.181+0000 7fe792318740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef8595d0 0x5612ef860a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.181+0000 7fe792318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef863e90 0x5612ef860ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.181+0000 7fe792318740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5612ef866a60 con 0x5612ef858cb0 2024-09-11T23:59:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.181+0000 7fe792318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612ef771050 con 0x5612ef863e90 2024-09-11T23:59:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.181+0000 7fe792318740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5612ef866be0 con 0x5612ef8595d0 2024-09-11T23:59:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.181+0000 7fe792318740 10 client.0 _ll_register_callbacks cb 0x5612ef7b1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78e4dc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef863e90 0x5612ef860ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78e4dc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef863e90 0x5612ef860ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37770/0 (socket says 172.21.15.62:37770) 2024-09-11T23:59:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78e4dc640 1 -- 172.21.15.62:0/3591688964 learned_addr learned my addr 172.21.15.62:0/3591688964 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78dcdb640 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef858cb0 0x5612ef8637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78d4da640 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef8595d0 0x5612ef860a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78e4dc640 1 -- 172.21.15.62:0/3591688964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef8595d0 msgr2=0x5612ef860a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef8595d0 0x5612ef860a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78e4dc640 1 -- 172.21.15.62:0/3591688964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef858cb0 msgr2=0x5612ef8637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef858cb0 0x5612ef8637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78e4dc640 1 -- 172.21.15.62:0/3591688964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ef5273c0 con 0x5612ef863e90 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef863e90 0x5612ef860ff0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe7800bbf40 tx=0x7fe7800bca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4addca80a2bd9e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.182+0000 7fe7717fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.183+0000 7fe7717fa640 1 -- 172.21.15.62:0/3591688964 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7800c05f0 con 0x5612ef863e90 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.183+0000 7fe7717fa640 1 -- 172.21.15.62:0/3591688964 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe7800319e0 con 0x5612ef863e90 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.183+0000 7fe792318740 1 -- 172.21.15.62:0/3591688964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7840047a0 con 0x5612ef863e90 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.183+0000 7fe7717fa640 1 -- 172.21.15.62:0/3591688964 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe7800da050 con 0x5612ef863e90 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.183+0000 7fe792318740 10 client.26234 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.183+0000 7fe792318740 1 -- 172.21.15.62:0/3591688964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5612ef866a60 con 0x5612ef863e90 2024-09-11T23:59:05.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.184+0000 7fe7717fa640 1 -- 172.21.15.62:0/3591688964 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7800c4eb0 con 0x5612ef863e90 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.184+0000 7fe7717fa640 1 -- 172.21.15.62:0/3591688964 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fe78004e060 con 0x5612ef863e90 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.184+0000 7fe792318740 10 client.26234 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.184+0000 7fe792318740 10 client.26234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.184+0000 7fe792318740 20 client.26234 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.184+0000 7fe792318740 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5612ef873840 0x5612ef875c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.184+0000 7fe792318740 4 client.26234 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.184+0000 7fe792318740 1 -- 172.21.15.62:0/3591688964 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe7840047a0 con 0x5612ef873840 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.185+0000 7fe78dcdb640 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5612ef873840 0x5612ef875c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.186+0000 7fe78dcdb640 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5612ef873840 0x5612ef875c80 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5520ad565426bc6c server_cookie=7b28b321ed6ff467 in_seq=0 out_seq=0 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.186+0000 7fe7717fa640 10 client.26234 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe7717fa640 1 -- 172.21.15.62:0/3591688964 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe7840047a0 con 0x5612ef873840 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe7717fa640 10 client.26234 handle_command_reply: tid=0 2024-09-11T23:59:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe792318740 1 client.26234 shutdown 2024-09-11T23:59:05.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe792318740 1 -- 172.21.15.62:0/3591688964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef863e90 msgr2=0x5612ef860ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:05.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe792318740 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef863e90 0x5612ef860ff0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe7800bbf40 tx=0x7fe7800bca00 comp rx=0 tx=0).stop 2024-09-11T23:59:05.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe792318740 1 -- 172.21.15.62:0/3591688964 shutdown_connections 2024-09-11T23:59:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe792318740 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5612ef873840 0x5612ef875c80 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe7717fa640 0 client.26234 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe792318740 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef863e90 0x5612ef860ff0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe792318740 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef8595d0 0x5612ef860a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe792318740 1 --2- 172.21.15.62:0/3591688964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef858cb0 0x5612ef8637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe792318740 1 -- 172.21.15.62:0/3591688964 >> 172.21.15.62:0/3591688964 conn(0x5612ef855240 msgr2=0x5612ef85af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.187+0000 7fe792318740 1 -- 172.21.15.62:0/3591688964 shutdown_connections 2024-09-11T23:59:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.188+0000 7fe792318740 1 -- 172.21.15.62:0/3591688964 wait complete. 2024-09-11T23:59:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.188+0000 7fe792318740 20 client.26234 trim_cache size 0 max 16384 2024-09-11T23:59:05.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe792318740 1 Processor -- start 2024-09-11T23:59:05.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe792318740 1 -- start start 2024-09-11T23:59:05.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe792318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef858cb0 0x5612ef862dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe792318740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef8595d0 0x5612ef86fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe792318740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 0x5612ef87b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe792318740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe7881b6780 con 0x5612ef863e90 2024-09-11T23:59:05.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe792318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7881b4020 con 0x5612ef858cb0 2024-09-11T23:59:05.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe792318740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7881b1e70 con 0x5612ef8595d0 2024-09-11T23:59:05.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe78e4dc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 0x5612ef87b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe78dcdb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef858cb0 0x5612ef862dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe78e4dc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 0x5612ef87b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38152/0 (socket says 172.21.15.62:38152) 2024-09-11T23:59:05.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe78dcdb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef858cb0 0x5612ef862dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37784/0 (socket says 172.21.15.62:37784) 2024-09-11T23:59:05.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe78e4dc640 1 -- 172.21.15.62:0/2816450320 learned_addr learned my addr 172.21.15.62:0/2816450320 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:05.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe78e4dc640 1 -- 172.21.15.62:0/2816450320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef8595d0 msgr2=0x5612ef86fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:05.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/2816450320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef8595d0 0x5612ef86fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe78e4dc640 1 -- 172.21.15.62:0/2816450320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef858cb0 msgr2=0x5612ef862dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:05.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.219+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/2816450320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef858cb0 0x5612ef862dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.220+0000 7fe78e4dc640 1 -- 172.21.15.62:0/2816450320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612ef876300 con 0x5612ef863e90 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.220+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/2816450320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 0x5612ef87b590 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe7800b8c10 tx=0x7fe7800ba750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f41432a8e55ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.220+0000 7fe7737fe640 1 -- 172.21.15.62:0/2816450320 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7800a0080 con 0x5612ef863e90 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.220+0000 7fe7737fe640 1 -- 172.21.15.62:0/2816450320 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7800c05f0 con 0x5612ef863e90 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.220+0000 7fe7737fe640 1 -- 172.21.15.62:0/2816450320 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7800d5060 con 0x5612ef863e90 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.221+0000 7fe792318740 1 -- 172.21.15.62:0/2816450320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 msgr2=0x5612ef87b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.221+0000 7fe792318740 1 --2- 172.21.15.62:0/2816450320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 0x5612ef87b590 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe7800b8c10 tx=0x7fe7800ba750 comp rx=0 tx=0).stop 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.222+0000 7fe792318740 1 -- 172.21.15.62:0/2816450320 shutdown_connections 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.222+0000 7fe792318740 1 --2- 172.21.15.62:0/2816450320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 0x5612ef87b590 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.222+0000 7fe792318740 1 --2- 172.21.15.62:0/2816450320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef8595d0 0x5612ef86fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.222+0000 7fe792318740 1 --2- 172.21.15.62:0/2816450320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef858cb0 0x5612ef862dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.222+0000 7fe792318740 1 -- 172.21.15.62:0/2816450320 >> 172.21.15.62:0/2816450320 conn(0x5612ef855240 msgr2=0x5612ef84bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.224+0000 7fe792318740 1 -- 172.21.15.62:0/2816450320 shutdown_connections 2024-09-11T23:59:05.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.224+0000 7fe792318740 1 -- 172.21.15.62:0/2816450320 wait complete. 2024-09-11T23:59:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.225+0000 7fe792318740 1 -- start start 2024-09-11T23:59:05.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.225+0000 7fe792318740 1 Processor -- start 2024-09-11T23:59:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.225+0000 7fe792318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef86c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.225+0000 7fe792318740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 0x5612ef869620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.225+0000 7fe792318740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef869b60 0x5612ef869fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.225+0000 7fe792318740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe7881b24e0 con 0x5612ef863e90 2024-09-11T23:59:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.225+0000 7fe792318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7881b3950 con 0x5612ef8595d0 2024-09-11T23:59:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.225+0000 7fe792318740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7881b2bb0 con 0x5612ef869b60 2024-09-11T23:59:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.225+0000 7fe792318740 10 client.0 _ll_register_callbacks cb 0x7fe7800b84e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.225+0000 7fe78dcdb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef86c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.226+0000 7fe78dcdb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef86c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37790/0 (socket says 172.21.15.62:37790) 2024-09-11T23:59:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.226+0000 7fe78dcdb640 1 -- 172.21.15.62:0/4104764521 learned_addr learned my addr 172.21.15.62:0/4104764521 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.226+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef869b60 0x5612ef869fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.226+0000 7fe78d4da640 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 0x5612ef869620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.226+0000 7fe78e4dc640 1 -- 172.21.15.62:0/4104764521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 msgr2=0x5612ef86c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.226+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef86c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.226+0000 7fe78e4dc640 1 -- 172.21.15.62:0/4104764521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 msgr2=0x5612ef869620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.226+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 0x5612ef869620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.226+0000 7fe78e4dc640 1 -- 172.21.15.62:0/4104764521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ef8543b0 con 0x5612ef869b60 2024-09-11T23:59:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.226+0000 7fe78e4dc640 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef869b60 0x5612ef869fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe780009be0 tx=0x7fe7800d5290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e22f1fd87eeeb83a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:05.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.227+0000 7fe7717fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:59:05.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.227+0000 7fe7717fa640 1 -- 172.21.15.62:0/4104764521 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7800bbba0 con 0x5612ef869b60 2024-09-11T23:59:05.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.227+0000 7fe7717fa640 1 -- 172.21.15.62:0/4104764521 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fe7800dc2c0 con 0x5612ef869b60 2024-09-11T23:59:05.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.227+0000 7fe792318740 1 -- 172.21.15.62:0/4104764521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7800c05f0 con 0x5612ef869b60 2024-09-11T23:59:05.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.228+0000 7fe7717fa640 1 -- 172.21.15.62:0/4104764521 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe7800edbc0 con 0x5612ef869b60 2024-09-11T23:59:05.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.228+0000 7fe792318740 10 client.26191 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:05.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.228+0000 7fe792318740 1 -- 172.21.15.62:0/4104764521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5612ef8543b0 con 0x5612ef869b60 2024-09-11T23:59:05.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.229+0000 7fe7717fa640 1 -- 172.21.15.62:0/4104764521 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7800edd60 con 0x5612ef869b60 2024-09-11T23:59:05.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.229+0000 7fe7717fa640 1 -- 172.21.15.62:0/4104764521 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fe7800bb690 con 0x5612ef869b60 2024-09-11T23:59:05.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.229+0000 7fe792318740 10 client.26191 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:05.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.229+0000 7fe792318740 10 client.26191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:05.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.229+0000 7fe792318740 20 client.26191 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:05.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.230+0000 7fe792318740 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5612ef8e8050 0x5612ef8ea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:05.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.230+0000 7fe792318740 4 client.26191 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.230+0000 7fe792318740 1 -- 172.21.15.62:0/4104764521 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe7800c05f0 con 0x5612ef8e8050 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.230+0000 7fe78dcdb640 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5612ef8e8050 0x5612ef8ea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.231+0000 7fe7717fa640 10 client.26191 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.231+0000 7fe78dcdb640 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5612ef8e8050 0x5612ef8ea440 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a91401fc9ccf831c server_cookie=1a0320cb1eaff9a9 in_seq=0 out_seq=0 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.231+0000 7fe7717fa640 1 -- 172.21.15.62:0/4104764521 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fe7800c05f0 con 0x5612ef8e8050 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.231+0000 7fe7717fa640 10 client.26191 handle_command_reply: tid=0 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.231+0000 7fe792318740 1 client.26191 shutdown 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.231+0000 7fe792318740 1 -- 172.21.15.62:0/4104764521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef869b60 msgr2=0x5612ef869fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.231+0000 7fe792318740 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef869b60 0x5612ef869fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe780009be0 tx=0x7fe7800d5290 comp rx=0 tx=0).stop 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.232+0000 7fe792318740 1 -- 172.21.15.62:0/4104764521 shutdown_connections 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.232+0000 7fe792318740 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5612ef8e8050 0x5612ef8ea440 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.232+0000 7fe792318740 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612ef869b60 0x5612ef869fe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.232+0000 7fe792318740 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612ef863e90 0x5612ef869620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.232+0000 7fe792318740 1 --2- 172.21.15.62:0/4104764521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612ef8595d0 0x5612ef86c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.232+0000 7fe792318740 1 -- 172.21.15.62:0/4104764521 >> 172.21.15.62:0/4104764521 conn(0x5612ef855240 msgr2=0x5612ef860350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:05.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.232+0000 7fe7717fa640 0 client.26191 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:05.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.232+0000 7fe792318740 1 -- 172.21.15.62:0/4104764521 shutdown_connections 2024-09-11T23:59:05.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.232+0000 7fe792318740 1 -- 172.21.15.62:0/4104764521 wait complete. 2024-09-11T23:59:05.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.232+0000 7fe792318740 20 client.26191 trim_cache size 0 max 16384 2024-09-11T23:59:05.234 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:59:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:59:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:59:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "09432efc-21c2-4038-822e-54cdd5069ff6": { 2024-09-11T23:59:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:59:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "09432efc-21c2-4038-822e-54cdd5069ff6", 2024-09-11T23:59:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:59:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:05.235 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:59:05.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.233+0000 7fe78ff3f640 1 -- 172.21.15.62:0/189037704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7740a2e40 msgr2=0x7fe7740a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.233+0000 7fe78ff3f640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7740a2e40 0x7fe7740a5300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe78810adc0 tx=0x7fe784009260 comp rx=0 tx=0).stop 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.233+0000 7fe78ff3f640 1 -- 172.21.15.62:0/189037704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe78810ef30 msgr2=0x7fe788109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.233+0000 7fe78ff3f640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe78810ef30 0x7fe788109db0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe780034040 tx=0x7fe7800305d0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.234+0000 7fe78ff3f640 1 -- 172.21.15.62:0/189037704 shutdown_connections 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.234+0000 7fe78ff3f640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7740a2e40 0x7fe7740a5300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.234+0000 7fe78ff3f640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe78810ef30 0x7fe788109db0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.234+0000 7fe78ff3f640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78810e580 0x7fe788109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.234+0000 7fe78ff3f640 1 --2- 172.21.15.62:0/189037704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe78810d9b0 0x7fe788109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.234+0000 7fe78ff3f640 1 -- 172.21.15.62:0/189037704 >> 172.21.15.62:0/189037704 conn(0x7fe78806e140 msgr2=0x7fe78810b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.235+0000 7fe78ff3f640 1 -- 172.21.15.62:0/189037704 shutdown_connections 2024-09-11T23:59:05.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:05.235+0000 7fe78ff3f640 1 -- 172.21.15.62:0/189037704 wait complete. 2024-09-11T23:59:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:06.061 INFO:tasks.cephfs.filesystem:scrub status for tag:09432efc-21c2-4038-822e-54cdd5069ff6 - {'path': '/', 'tag': '09432efc-21c2-4038-822e-54cdd5069ff6', 'options': 'recursive,force'} 2024-09-11T23:59:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.566+0000 7fec29f95640 1 -- 172.21.15.62:0/2429394608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec24100e20 msgr2=0x7fec24101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.566+0000 7fec29f95640 1 --2- 172.21.15.62:0/2429394608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec24100e20 0x7fec24101200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec14009930 tx=0x7fec1402a4d0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.567+0000 7fec29f95640 1 -- 172.21.15.62:0/2429394608 shutdown_connections 2024-09-11T23:59:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.567+0000 7fec29f95640 1 --2- 172.21.15.62:0/2429394608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec24102ef0 0x7fec2410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.567+0000 7fec29f95640 1 --2- 172.21.15.62:0/2429394608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec24102570 0x7fec241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.567+0000 7fec29f95640 1 --2- 172.21.15.62:0/2429394608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec24100e20 0x7fec24101200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.567+0000 7fec29f95640 1 -- 172.21.15.62:0/2429394608 >> 172.21.15.62:0/2429394608 conn(0x7fec240fc7d0 msgr2=0x7fec240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.567+0000 7fec29f95640 1 -- 172.21.15.62:0/2429394608 shutdown_connections 2024-09-11T23:59:06.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.567+0000 7fec29f95640 1 -- 172.21.15.62:0/2429394608 wait complete. 2024-09-11T23:59:06.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.568+0000 7fec29f95640 1 Processor -- start 2024-09-11T23:59:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.568+0000 7fec29f95640 1 -- start start 2024-09-11T23:59:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.568+0000 7fec29f95640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec24100e20 0x7fec2419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.569+0000 7fec29f95640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec24102570 0x7fec2419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.569+0000 7fec29f95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec24102ef0 0x7fec2410cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.569+0000 7fec29f95640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fec24112090 con 0x7fec24100e20 2024-09-11T23:59:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.569+0000 7fec29f95640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec24111f10 con 0x7fec24102570 2024-09-11T23:59:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.569+0000 7fec29f95640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec24112210 con 0x7fec24102ef0 2024-09-11T23:59:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.569+0000 7fec23fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec24102ef0 0x7fec2410cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.569+0000 7fec22ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec24102570 0x7fec2419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.569+0000 7fec22ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec24102570 0x7fec2419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37806/0 (socket says 172.21.15.62:37806) 2024-09-11T23:59:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.569+0000 7fec22ffd640 1 -- 172.21.15.62:0/2014859217 learned_addr learned my addr 172.21.15.62:0/2014859217 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.570+0000 7fec22ffd640 1 -- 172.21.15.62:0/2014859217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec24102ef0 msgr2=0x7fec2410cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.570+0000 7fec22ffd640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec24102ef0 0x7fec2410cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.570+0000 7fec22ffd640 1 -- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec24100e20 msgr2=0x7fec2419f180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:59:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.570+0000 7fec237fe640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec24100e20 0x7fec2419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.570+0000 7fec22ffd640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec24100e20 0x7fec2419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.570+0000 7fec22ffd640 1 -- 172.21.15.62:0/2014859217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2410d4c0 con 0x7fec24102570 2024-09-11T23:59:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.570+0000 7fec237fe640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec24100e20 0x7fec2419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.570+0000 7fec22ffd640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec24102570 0x7fec2419f6c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec1800e9f0 tx=0x7fec1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.571+0000 7fec20ff9640 1 -- 172.21.15.62:0/2014859217 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1800cd80 con 0x7fec24102570 2024-09-11T23:59:06.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.571+0000 7fec20ff9640 1 -- 172.21.15.62:0/2014859217 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec18010910 con 0x7fec24102570 2024-09-11T23:59:06.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.571+0000 7fec20ff9640 1 -- 172.21.15.62:0/2014859217 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180188f0 con 0x7fec24102570 2024-09-11T23:59:06.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.572+0000 7fec29f95640 1 -- 172.21.15.62:0/2014859217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2410d7b0 con 0x7fec24102570 2024-09-11T23:59:06.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.572+0000 7fec29f95640 1 -- 172.21.15.62:0/2014859217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2419ada0 con 0x7fec24102570 2024-09-11T23:59:06.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.575+0000 7fec20ff9640 1 -- 172.21.15.62:0/2014859217 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fec18010ac0 con 0x7fec24102570 2024-09-11T23:59:06.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.576+0000 7fec20ff9640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fec0c0a2df0 0x7fec0c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:06.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.576+0000 7fec29f95640 1 -- 172.21.15.62:0/2014859217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec241021b0 con 0x7fec24102570 2024-09-11T23:59:06.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.577+0000 7fec237fe640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fec0c0a2df0 0x7fec0c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:06.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.577+0000 7fec20ff9640 1 -- 172.21.15.62:0/2014859217 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8469+0+0 (secure 0 0 0) 0x7fec180cf980 con 0x7fec24102570 2024-09-11T23:59:06.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.578+0000 7fec237fe640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fec0c0a2df0 0x7fec0c0a52b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec1402a9f0 tx=0x7fec14005fd0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:06.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.582+0000 7fec20ff9640 1 -- 172.21.15.62:0/2014859217 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec180541b0 con 0x7fec24102570 2024-09-11T23:59:06.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:06 smithi083 ceph-mon[34805]: pgmap v213: 97 pgs: 97 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.9 MiB/s wr, 679 op/s 2024-09-11T23:59:06.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:06 smithi062 ceph-mon[30958]: pgmap v213: 97 pgs: 97 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.9 MiB/s wr, 679 op/s 2024-09-11T23:59:06.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.966+0000 7fec29f95640 1 -- 172.21.15.62:0/2014859217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec2410d980 con 0x7fec24102570 2024-09-11T23:59:06.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.967+0000 7fec20ff9640 1 -- 172.21.15.62:0/2014859217 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11910 (secure 0 0 0) 0x7fec1809c570 con 0x7fec24102570 2024-09-11T23:59:06.968 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:59:06.968 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-09-11T23:57:32:617369+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:57:32.075842+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:59:06.968 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-09-11T23:59:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 -- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fec0c0a2df0 msgr2=0x7fec0c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fec0c0a2df0 0x7fec0c0a52b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec1402a9f0 tx=0x7fec14005fd0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 -- 172.21.15.62:0/2014859217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec24102570 msgr2=0x7fec2419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec24102570 0x7fec2419f6c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec1800e9f0 tx=0x7fec1800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 -- 172.21.15.62:0/2014859217 shutdown_connections 2024-09-11T23:59:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fec0c0a2df0 0x7fec0c0a52b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec24102ef0 0x7fec2410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec24102570 0x7fec2419f6c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 --2- 172.21.15.62:0/2014859217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec24100e20 0x7fec2419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 -- 172.21.15.62:0/2014859217 >> 172.21.15.62:0/2014859217 conn(0x7fec240fc7d0 msgr2=0x7fec240fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 -- 172.21.15.62:0/2014859217 shutdown_connections 2024-09-11T23:59:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:06.977+0000 7fec29f95640 1 -- 172.21.15.62:0/2014859217 wait complete. 2024-09-11T23:59:07.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:06 smithi120 ceph-mon[35182]: pgmap v213: 97 pgs: 97 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.9 MiB/s wr, 679 op/s 2024-09-11T23:59:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 385.0 s, 2511.9 tps, lat 43.887 ms stddev 86.692 2024-09-11T23:59:07.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:07 smithi062 ceph-mon[30958]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:59:07.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:07 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2014859217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:08.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:07 smithi120 ceph-mon[35182]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:59:08.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:07 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2014859217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:08.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:07 smithi083 ceph-mon[34805]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:59:08.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:07 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2014859217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:08.460 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-11T23:59:08.461 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-11T23:59:08.821 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:08.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:08 smithi062 ceph-mon[30958]: pgmap v215: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 MiB/s wr, 1.50k op/s 2024-09-11T23:59:09.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:08 smithi120 ceph-mon[35182]: pgmap v215: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 MiB/s wr, 1.50k op/s 2024-09-11T23:59:09.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:08 smithi083 ceph-mon[34805]: pgmap v215: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 MiB/s wr, 1.50k op/s 2024-09-11T23:59:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.434+0000 7fe14e25d640 1 -- 172.21.15.62:0/1648192626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe148101de0 msgr2=0x7fe1481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.434+0000 7fe14e25d640 1 --2- 172.21.15.62:0/1648192626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe148101de0 0x7fe1481021c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe138009900 tx=0x7fe13802a410 comp rx=0 tx=0).stop 2024-09-11T23:59:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.435+0000 7fe14e25d640 1 -- 172.21.15.62:0/1648192626 shutdown_connections 2024-09-11T23:59:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.435+0000 7fe14e25d640 1 --2- 172.21.15.62:0/1648192626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe148106f10 0x7fe14810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.435+0000 7fe14e25d640 1 --2- 172.21.15.62:0/1648192626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe148102790 0x7fe1481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.435+0000 7fe14e25d640 1 --2- 172.21.15.62:0/1648192626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe148101de0 0x7fe1481021c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.435+0000 7fe14e25d640 1 -- 172.21.15.62:0/1648192626 >> 172.21.15.62:0/1648192626 conn(0x7fe1480fd880 msgr2=0x7fe1480ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.435+0000 7fe14e25d640 1 -- 172.21.15.62:0/1648192626 shutdown_connections 2024-09-11T23:59:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.436+0000 7fe14e25d640 1 -- 172.21.15.62:0/1648192626 wait complete. 2024-09-11T23:59:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.436+0000 7fe14e25d640 1 Processor -- start 2024-09-11T23:59:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe14e25d640 1 -- start start 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe14e25d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe148101de0 0x7fe1480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe14e25d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe148102790 0x7fe148074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe14e25d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe148106f10 0x7fe148074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe14e25d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe148111f90 con 0x7fe148101de0 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe14e25d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe148111e10 con 0x7fe148106f10 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe14e25d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe148112110 con 0x7fe148102790 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe1477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe148101de0 0x7fe1480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe147fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe148106f10 0x7fe148074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe147fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe148106f10 0x7fe148074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37826/0 (socket says 172.21.15.62:37826) 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe1477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe148101de0 0x7fe1480744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38182/0 (socket says 172.21.15.62:38182) 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe147fff640 1 -- 172.21.15.62:0/1352342168 learned_addr learned my addr 172.21.15.62:0/1352342168 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.437+0000 7fe146ffd640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe148102790 0x7fe148074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.438+0000 7fe147fff640 1 -- 172.21.15.62:0/1352342168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe148102790 msgr2=0x7fe148074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:09.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.438+0000 7fe147fff640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe148102790 0x7fe148074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.438+0000 7fe147fff640 1 -- 172.21.15.62:0/1352342168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe148101de0 msgr2=0x7fe1480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:09.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.439+0000 7fe147fff640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe148101de0 0x7fe1480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.439+0000 7fe147fff640 1 -- 172.21.15.62:0/1352342168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1481a6950 con 0x7fe148106f10 2024-09-11T23:59:09.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.439+0000 7fe1477fe640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe148101de0 0x7fe1480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:59:09.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.439+0000 7fe147fff640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe148106f10 0x7fe148074f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe14000eaa0 tx=0x7fe14000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:09.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.439+0000 7fe144ff9640 1 -- 172.21.15.62:0/1352342168 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14000cdb0 con 0x7fe148106f10 2024-09-11T23:59:09.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.439+0000 7fe144ff9640 1 -- 172.21.15.62:0/1352342168 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe140005020 con 0x7fe148106f10 2024-09-11T23:59:09.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.439+0000 7fe14e25d640 1 -- 172.21.15.62:0/1352342168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1481a6c40 con 0x7fe148106f10 2024-09-11T23:59:09.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.439+0000 7fe144ff9640 1 -- 172.21.15.62:0/1352342168 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400116b0 con 0x7fe148106f10 2024-09-11T23:59:09.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.440+0000 7fe14e25d640 1 -- 172.21.15.62:0/1352342168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1481a7180 con 0x7fe148106f10 2024-09-11T23:59:09.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.442+0000 7fe144ff9640 1 -- 172.21.15.62:0/1352342168 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe140005560 con 0x7fe148106f10 2024-09-11T23:59:09.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.442+0000 7fe150636740 1 Processor -- start 2024-09-11T23:59:09.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.442+0000 7fe150636740 1 -- start start 2024-09-11T23:59:09.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.443+0000 7fe150636740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91231680 0x55fc91231a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.443+0000 7fe150636740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91244c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.443+0000 7fe150636740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91245150 0x55fc91247540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.443+0000 7fe150636740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fc911972a0 con 0x55fc91231680 2024-09-11T23:59:09.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.443+0000 7fe150636740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fc9113eb10 con 0x55fc91231fa0 2024-09-11T23:59:09.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.443+0000 7fe150636740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc91193260 con 0x55fc91245150 2024-09-11T23:59:09.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.443+0000 7fe1477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91244c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.443+0000 7fe1477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91244c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37834/0 (socket says 172.21.15.62:37834) 2024-09-11T23:59:09.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.443+0000 7fe144ff9640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1300a2d70 0x7fe1300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.443+0000 7fe1477fe640 1 -- 172.21.15.62:0/2737461219 learned_addr learned my addr 172.21.15.62:0/2737461219 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:09.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.444+0000 7fe144ff9640 1 -- 172.21.15.62:0/1352342168 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7fe1400cf710 con 0x7fe148106f10 2024-09-11T23:59:09.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.444+0000 7fe146ffd640 1 --2- 172.21.15.62:0/2737461219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91231680 0x55fc91231a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.444+0000 7fe146ffd640 1 --2- 172.21.15.62:0/2737461219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91245150 0x55fc91247540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.444+0000 7fe1477fe640 1 -- 172.21.15.62:0/2737461219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91245150 msgr2=0x55fc91247540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:09.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.444+0000 7fe1477fe640 1 --2- 172.21.15.62:0/2737461219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91245150 0x55fc91247540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.444+0000 7fe1477fe640 1 -- 172.21.15.62:0/2737461219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91231680 msgr2=0x55fc91231a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:09.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.444+0000 7fe1477fe640 1 --2- 172.21.15.62:0/2737461219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91231680 0x55fc91231a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.444+0000 7fe1477fe640 1 -- 172.21.15.62:0/2737461219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc90f08b90 con 0x55fc91231fa0 2024-09-11T23:59:09.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.445+0000 7fe1477fe640 1 --2- 172.21.15.62:0/2737461219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91244c10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe138033000 tx=0x7fe138032d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ace948eb5d8900 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:09.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.445+0000 7fe12dffb640 1 -- 172.21.15.62:0/2737461219 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13803f070 con 0x55fc91231fa0 2024-09-11T23:59:09.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.445+0000 7fe12dffb640 1 -- 172.21.15.62:0/2737461219 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1380317b0 con 0x55fc91231fa0 2024-09-11T23:59:09.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.445+0000 7fe12dffb640 1 -- 172.21.15.62:0/2737461219 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138035070 con 0x55fc91231fa0 2024-09-11T23:59:09.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.445+0000 7fe147fff640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1300a2d70 0x7fe1300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.446+0000 7fe150636740 1 -- 172.21.15.62:0/2737461219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 msgr2=0x55fc91244c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:09.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.446+0000 7fe150636740 1 --2- 172.21.15.62:0/2737461219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91244c10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe138033000 tx=0x7fe138032d60 comp rx=0 tx=0).stop 2024-09-11T23:59:09.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.446+0000 7fe147fff640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1300a2d70 0x7fe1300a5230 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe140009270 tx=0x7fe140097eb0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:09.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.446+0000 7fe150636740 1 -- 172.21.15.62:0/2737461219 shutdown_connections 2024-09-11T23:59:09.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.446+0000 7fe150636740 1 --2- 172.21.15.62:0/2737461219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91245150 0x55fc91247540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.446+0000 7fe150636740 1 --2- 172.21.15.62:0/2737461219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91244c10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.446+0000 7fe150636740 1 --2- 172.21.15.62:0/2737461219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91231680 0x55fc91231a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.446+0000 7fe150636740 1 -- 172.21.15.62:0/2737461219 >> 172.21.15.62:0/2737461219 conn(0x55fc9122dcf0 msgr2=0x55fc9122e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.446+0000 7fe150636740 1 -- 172.21.15.62:0/2737461219 shutdown_connections 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.446+0000 7fe150636740 1 -- 172.21.15.62:0/2737461219 wait complete. 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.447+0000 7fe150636740 1 -- start start 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.447+0000 7fe150636740 1 Processor -- start 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.447+0000 7fe150636740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc91242130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.447+0000 7fe150636740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91242670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.447+0000 7fe150636740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 0x55fc91242bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc91242130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc91242130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:57670/0 (socket says 172.21.15.62:57670) 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe146ffd640 1 -- 172.21.15.62:0/144407264 learned_addr learned my addr 172.21.15.62:0/144407264 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fc91247c60 con 0x55fc91245150 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fc9111f6e0 con 0x55fc91231fa0 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc91247de0 con 0x55fc91231680 2024-09-11T23:59:09.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe1477fe640 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91242670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe146ffd640 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 0x55fc91242bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe150636740 10 client.0 _ll_register_callbacks cb 0x55fc91143a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe1477fe640 1 -- 172.21.15.62:0/144407264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 msgr2=0x55fc91242130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe1477fe640 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc91242130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe1477fe640 1 -- 172.21.15.62:0/144407264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 msgr2=0x55fc91242bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe1477fe640 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 0x55fc91242bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe1477fe640 1 -- 172.21.15.62:0/144407264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc912512e0 con 0x55fc91231fa0 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe146ffd640 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc91242130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1380317b0 con 0x55fc91231fa0 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe1477fe640 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91242670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe138009900 tx=0x7fe138005450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d992d57508679f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.448+0000 7fe126ffd640 10 client.26258 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.449+0000 7fe126ffd640 1 -- 172.21.15.62:0/144407264 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13803f030 con 0x55fc91231fa0 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.449+0000 7fe126ffd640 1 -- 172.21.15.62:0/144407264 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7fe1380055f0 con 0x55fc91231fa0 2024-09-11T23:59:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.449+0000 7fe126ffd640 1 -- 172.21.15.62:0/144407264 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe138035040 con 0x55fc91231fa0 2024-09-11T23:59:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.450+0000 7fe150636740 10 client.26258 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.450+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55fc91253ad0 con 0x55fc91231fa0 2024-09-11T23:59:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.450+0000 7fe126ffd640 1 -- 172.21.15.62:0/144407264 <== mon.2 v2:172.21.15.83:3300/0 4 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fe138003800 con 0x55fc91231fa0 2024-09-11T23:59:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.450+0000 7fe150636740 10 client.26258 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.450+0000 7fe150636740 10 client.26258 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:59:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.450+0000 7fe150636740 20 client.26258 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.450+0000 7fe150636740 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fc91254e20 0x55fc91257210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.450+0000 7fe150636740 4 client.26258 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.450+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc911a49d0 con 0x55fc91254e20 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.451+0000 7fe146ffd640 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fc91254e20 0x55fc91257210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.451+0000 7fe146ffd640 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fc91254e20 0x55fc91257210 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce800c673792f8d server_cookie=ac2b89190bc02a05 in_seq=0 out_seq=0 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.451+0000 7fe126ffd640 10 client.26258 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe126ffd640 1 -- 172.21.15.62:0/144407264 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc911a49d0 con 0x55fc91254e20 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe126ffd640 10 client.26258 handle_command_reply: tid=0 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 client.26258 shutdown 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 msgr2=0x55fc91242670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91242670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe138009900 tx=0x7fe138005450 comp rx=0 tx=0).stop 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 shutdown_connections 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fc91254e20 0x55fc91257210 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 0x55fc91242bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231fa0 0x55fc91242670 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 --2- 172.21.15.62:0/144407264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc91242130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 >> 172.21.15.62:0/144407264 conn(0x55fc9122dcf0 msgr2=0x55fc9122ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe126ffd640 0 client.26258 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 shutdown_connections 2024-09-11T23:59:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 1 -- 172.21.15.62:0/144407264 wait complete. 2024-09-11T23:59:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.452+0000 7fe150636740 20 client.26258 trim_cache size 0 max 16384 2024-09-11T23:59:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.486+0000 7fe150636740 1 Processor -- start 2024-09-11T23:59:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.487+0000 7fe150636740 1 -- start start 2024-09-11T23:59:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.487+0000 7fe150636740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231680 0x55fc91241070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.487+0000 7fe150636740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231fa0 0x55fc91259e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.487+0000 7fe150636740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 0x55fc9125a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.487+0000 7fe150636740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe1480727e0 con 0x55fc91245150 2024-09-11T23:59:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.487+0000 7fe150636740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe148070080 con 0x55fc91231680 2024-09-11T23:59:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.487+0000 7fe150636740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe14806def0 con 0x55fc91231fa0 2024-09-11T23:59:09.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.488+0000 7fe1477fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231fa0 0x55fc91259e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.488+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 0x55fc9125a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.488+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 0x55fc9125a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38212/0 (socket says 172.21.15.62:38212) 2024-09-11T23:59:09.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.488+0000 7fe146ffd640 1 -- 172.21.15.62:0/1947609557 learned_addr learned my addr 172.21.15.62:0/1947609557 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:09.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.488+0000 7fe146ffd640 1 --2- 172.21.15.62:0/1947609557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231680 0x55fc91241070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.489+0000 7fe1477fe640 1 -- 172.21.15.62:0/1947609557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231680 msgr2=0x55fc91241070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:09.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.489+0000 7fe1477fe640 1 --2- 172.21.15.62:0/1947609557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231680 0x55fc91241070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.489+0000 7fe1477fe640 1 -- 172.21.15.62:0/1947609557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 msgr2=0x55fc9125a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:09.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.489+0000 7fe1477fe640 1 --2- 172.21.15.62:0/1947609557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 0x55fc9125a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.489+0000 7fe1477fe640 1 -- 172.21.15.62:0/1947609557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc91229260 con 0x55fc91231fa0 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.490+0000 7fe1477fe640 1 --2- 172.21.15.62:0/1947609557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231fa0 0x55fc91259e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe138005380 tx=0x7fe13803c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f63702ed55ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.490+0000 7fe12d7fa640 1 -- 172.21.15.62:0/1947609557 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13803f070 con 0x55fc91231fa0 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.490+0000 7fe12d7fa640 1 -- 172.21.15.62:0/1947609557 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1380317b0 con 0x55fc91231fa0 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.490+0000 7fe12d7fa640 1 -- 172.21.15.62:0/1947609557 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138066960 con 0x55fc91231fa0 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.491+0000 7fe150636740 1 -- 172.21.15.62:0/1947609557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231fa0 msgr2=0x55fc91259e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.491+0000 7fe150636740 1 --2- 172.21.15.62:0/1947609557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231fa0 0x55fc91259e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe138005380 tx=0x7fe13803c000 comp rx=0 tx=0).stop 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.492+0000 7fe150636740 1 -- 172.21.15.62:0/1947609557 shutdown_connections 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.492+0000 7fe150636740 1 --2- 172.21.15.62:0/1947609557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91245150 0x55fc9125a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.492+0000 7fe150636740 1 --2- 172.21.15.62:0/1947609557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231fa0 0x55fc91259e50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.492+0000 7fe150636740 1 --2- 172.21.15.62:0/1947609557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91231680 0x55fc91241070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.492+0000 7fe150636740 1 -- 172.21.15.62:0/1947609557 >> 172.21.15.62:0/1947609557 conn(0x55fc9122dcf0 msgr2=0x55fc912508c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:09.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.492+0000 7fe150636740 1 -- 172.21.15.62:0/1947609557 shutdown_connections 2024-09-11T23:59:09.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.492+0000 7fe150636740 1 -- 172.21.15.62:0/1947609557 wait complete. 2024-09-11T23:59:09.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.492+0000 7fe150636740 1 -- start start 2024-09-11T23:59:09.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.492+0000 7fe150636740 1 Processor -- start 2024-09-11T23:59:09.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe150636740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc9125a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe150636740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91231fa0 0x55fc91257780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe150636740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91245150 0x55fc91257cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe150636740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe14806e540 con 0x55fc91231fa0 2024-09-11T23:59:09.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe150636740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe14806f9b0 con 0x55fc91245150 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe150636740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe14806ec10 con 0x55fc91231680 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe150636740 10 client.0 _ll_register_callbacks cb 0x7fe138004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc9125a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc9125a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:57692/0 (socket says 172.21.15.62:57692) 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe146ffd640 1 -- 172.21.15.62:0/108750966 learned_addr learned my addr 172.21.15.62:0/108750966 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.493+0000 7fe146ffd640 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91245150 0x55fc91257cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.494+0000 7fe1477fe640 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91231fa0 0x55fc91257780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.494+0000 7fe146ffd640 1 -- 172.21.15.62:0/108750966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91245150 msgr2=0x55fc91257cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.494+0000 7fe146ffd640 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91245150 0x55fc91257cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.494+0000 7fe146ffd640 1 -- 172.21.15.62:0/108750966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91231fa0 msgr2=0x55fc91257780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.494+0000 7fe146ffd640 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91231fa0 0x55fc91257780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.494+0000 7fe146ffd640 1 -- 172.21.15.62:0/108750966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc91258360 con 0x55fc91231680 2024-09-11T23:59:09.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.494+0000 7fe146ffd640 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc9125a430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00be00 tx=0x7fe13c0047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=686e4de9a1c2b55c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:09.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.494+0000 7fe126ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:59:09.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.494+0000 7fe126ffd640 1 -- 172.21.15.62:0/108750966 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c054060 con 0x55fc91231680 2024-09-11T23:59:09.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.494+0000 7fe126ffd640 1 -- 172.21.15.62:0/108750966 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8557+0+0 (secure 0 0 0) 0x7fe13c00f040 con 0x55fc91231680 2024-09-11T23:59:09.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.495+0000 7fe150636740 1 -- 172.21.15.62:0/108750966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1380317b0 con 0x55fc91231680 2024-09-11T23:59:09.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.495+0000 7fe126ffd640 1 -- 172.21.15.62:0/108750966 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe13c019d70 con 0x55fc91231680 2024-09-11T23:59:09.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.496+0000 7fe150636740 10 client.26212 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:09.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.496+0000 7fe150636740 1 -- 172.21.15.62:0/108750966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc912bf5b0 con 0x55fc91231680 2024-09-11T23:59:09.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.496+0000 7fe126ffd640 1 -- 172.21.15.62:0/108750966 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1380317b0 con 0x55fc91231680 2024-09-11T23:59:09.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.496+0000 7fe126ffd640 1 -- 172.21.15.62:0/108750966 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fe13c006660 con 0x55fc91231680 2024-09-11T23:59:09.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.497+0000 7fe150636740 10 client.26212 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:09.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.497+0000 7fe150636740 10 client.26212 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:59:09.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.497+0000 7fe150636740 20 client.26212 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:09.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.497+0000 7fe150636740 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fc912bfca0 0x55fc912c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:09.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.497+0000 7fe150636740 4 client.26212 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-11T23:59:09.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.497+0000 7fe150636740 1 -- 172.21.15.62:0/108750966 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x55fc9119ed20 con 0x55fc912bfca0 2024-09-11T23:59:09.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.498+0000 7fe1477fe640 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fc912bfca0 0x55fc912c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.500+0000 7fe1477fe640 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fc912bfca0 0x55fc912c2090 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717d5f7b761b88db server_cookie=9b665c0cccb3adb4 in_seq=0 out_seq=0 2024-09-11T23:59:09.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.500+0000 7fe126ffd640 10 client.26212 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe126ffd640 1 -- 172.21.15.62:0/108750966 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55fc9119ed20 con 0x55fc912bfca0 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe126ffd640 10 client.26212 handle_command_reply: tid=0 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe150636740 1 client.26212 shutdown 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe150636740 1 -- 172.21.15.62:0/108750966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 msgr2=0x55fc9125a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe150636740 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc9125a430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00be00 tx=0x7fe13c0047d0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe150636740 1 -- 172.21.15.62:0/108750966 shutdown_connections 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe150636740 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fc912bfca0 0x55fc912c2090 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe150636740 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc91245150 0x55fc91257cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe150636740 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc91231fa0 0x55fc91257780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe150636740 1 --2- 172.21.15.62:0/108750966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc91231680 0x55fc9125a430 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe126ffd640 0 client.26212 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:09.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe150636740 1 -- 172.21.15.62:0/108750966 >> 172.21.15.62:0/108750966 conn(0x55fc9122dcf0 msgr2=0x55fc912420f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:09.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.788+0000 7fe150636740 1 -- 172.21.15.62:0/108750966 shutdown_connections 2024-09-11T23:59:09.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe150636740 1 -- 172.21.15.62:0/108750966 wait complete. 2024-09-11T23:59:09.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe150636740 20 client.26212 trim_cache size 0 max 16384 2024-09-11T23:59:09.790 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:59:09.790 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-09-11T23:59:09.790 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:59:09.790 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 26, 2024-09-11T23:59:09.790 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "63b8b04": { 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 26, 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 136.6, 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:59:09.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:59:09.792 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:59:09.792 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:09.792 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:09.792 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:09.792 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:09.792 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:09.792 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:59:09.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 -- 172.21.15.62:0/1352342168 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1300a2d70 msgr2=0x7fe1300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:09.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1300a2d70 0x7fe1300a5230 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe140009270 tx=0x7fe140097eb0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 -- 172.21.15.62:0/1352342168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe148106f10 msgr2=0x7fe148074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:09.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe148106f10 0x7fe148074f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe14000eaa0 tx=0x7fe14000ef60 comp rx=0 tx=0).stop 2024-09-11T23:59:09.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 -- 172.21.15.62:0/1352342168 shutdown_connections 2024-09-11T23:59:09.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1300a2d70 0x7fe1300a5230 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe148106f10 0x7fe148074f70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe148102790 0x7fe148074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 --2- 172.21.15.62:0/1352342168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe148101de0 0x7fe1480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:09.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 -- 172.21.15.62:0/1352342168 >> 172.21.15.62:0/1352342168 conn(0x7fe1480fd880 msgr2=0x7fe1480fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:09.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 -- 172.21.15.62:0/1352342168 shutdown_connections 2024-09-11T23:59:09.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:09.789+0000 7fe14e25d640 1 -- 172.21.15.62:0/1352342168 wait complete. 2024-09-11T23:59:10.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:10 smithi062 ceph-mon[30958]: pgmap v216: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 MiB/s wr, 1.69k op/s 2024-09-11T23:59:10.886 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '63b8b04', quiesce took 6.1 seconds. Will release after: 10.500000000000002 2024-09-11T23:59:11.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:10 smithi120 ceph-mon[35182]: pgmap v216: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 MiB/s wr, 1.69k op/s 2024-09-11T23:59:11.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:10 smithi083 ceph-mon[34805]: pgmap v216: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 MiB/s wr, 1.69k op/s 2024-09-11T23:59:11.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:11 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:59:11.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:11 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:59:11.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:11 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:59:12.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:11 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:59:12.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:11 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:59:12.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:11 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:59:12.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:11 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:59:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:11 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:59:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:11 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:59:12.447 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 390.0 s, 1070.8 tps, lat 38.476 ms stddev 42.138 2024-09-11T23:59:12.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:12 smithi062 ceph-mon[30958]: pgmap v217: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 MiB/s wr, 1.68k op/s 2024-09-11T23:59:13.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:12 smithi120 ceph-mon[35182]: pgmap v217: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 MiB/s wr, 1.68k op/s 2024-09-11T23:59:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:12 smithi083 ceph-mon[34805]: pgmap v217: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 MiB/s wr, 1.68k op/s 2024-09-11T23:59:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:14 smithi062 ceph-mon[30958]: pgmap v218: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 1.40k op/s 2024-09-11T23:59:15.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:14 smithi120 ceph-mon[35182]: pgmap v218: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 1.40k op/s 2024-09-11T23:59:15.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:14 smithi083 ceph-mon[34805]: pgmap v218: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 1.40k op/s 2024-09-11T23:59:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 395.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:59:17.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:17 smithi062 ceph-mon[30958]: pgmap v219: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 1.40k op/s 2024-09-11T23:59:17.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:17 smithi062 ceph-mon[30958]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:59:17.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:17 smithi120 ceph-mon[35182]: pgmap v219: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 1.40k op/s 2024-09-11T23:59:17.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:17 smithi120 ceph-mon[35182]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:59:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:17 smithi083 ceph-mon[34805]: pgmap v219: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 1.40k op/s 2024-09-11T23:59:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:17 smithi083 ceph-mon[34805]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:59:19.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:19 smithi120 ceph-mon[35182]: pgmap v221: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 383 op/s 2024-09-11T23:59:19.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:19 smithi083 ceph-mon[34805]: pgmap v221: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 383 op/s 2024-09-11T23:59:19.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:19 smithi062 ceph-mon[30958]: pgmap v221: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 383 op/s 2024-09-11T23:59:20.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:20 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:59:20.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:20 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:59:20.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:20 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:59:20.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:20 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:59:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:20 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:59:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:20 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:59:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:20 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:59:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:20 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:59:20.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:20 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:59:20.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:20 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:59:20.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:20 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:59:20.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:20 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-11T23:59:21.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:59:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:21 smithi120 ceph-mon[35182]: pgmap v222: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-09-11T23:59:21.683 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:21 smithi083 ceph-mon[34805]: pgmap v222: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-09-11T23:59:21.684 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:21.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:21 smithi062 ceph-mon[30958]: pgmap v222: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-09-11T23:59:22.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.117+0000 7fe41c417640 1 -- 172.21.15.62:0/3478298511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4141036e0 msgr2=0x7fe414107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:22.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.117+0000 7fe41c417640 1 --2- 172.21.15.62:0/3478298511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4141036e0 0x7fe414107b90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe404009930 tx=0x7fe40402a420 comp rx=0 tx=0).stop 2024-09-11T23:59:22.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.118+0000 7fe41c417640 1 -- 172.21.15.62:0/3478298511 shutdown_connections 2024-09-11T23:59:22.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.118+0000 7fe41c417640 1 --2- 172.21.15.62:0/3478298511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4141082a0 0x7fe41410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:22.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.118+0000 7fe41c417640 1 --2- 172.21.15.62:0/3478298511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4141036e0 0x7fe414107b90 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:22.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.118+0000 7fe41c417640 1 --2- 172.21.15.62:0/3478298511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe414102d30 0x7fe414103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:22.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.118+0000 7fe41c417640 1 -- 172.21.15.62:0/3478298511 >> 172.21.15.62:0/3478298511 conn(0x7fe4140fc7d0 msgr2=0x7fe4140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:22.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.118+0000 7fe41c417640 1 -- 172.21.15.62:0/3478298511 shutdown_connections 2024-09-11T23:59:22.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.118+0000 7fe41c417640 1 -- 172.21.15.62:0/3478298511 wait complete. 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.118+0000 7fe41c417640 1 Processor -- start 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.118+0000 7fe41c417640 1 -- start start 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41c417640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe414102d30 0x7fe414100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41c417640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4141036e0 0x7fe414100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41c417640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4141082a0 0x7fe414101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41c417640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141120a0 con 0x7fe4141082a0 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41c417640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe414111f20 con 0x7fe414102d30 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41c417640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe414112220 con 0x7fe4141036e0 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41a9b4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4141082a0 0x7fe414101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41a9b4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4141082a0 0x7fe414101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55412/0 (socket says 172.21.15.62:55412) 2024-09-11T23:59:22.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41a9b4640 1 -- 172.21.15.62:0/1633332024 learned_addr learned my addr 172.21.15.62:0/1633332024 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe4199b2640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4141036e0 0x7fe414100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41a1b3640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe414102d30 0x7fe414100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41a9b4640 1 -- 172.21.15.62:0/1633332024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4141036e0 msgr2=0x7fe414100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41a9b4640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4141036e0 0x7fe414100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41a9b4640 1 -- 172.21.15.62:0/1633332024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe414102d30 msgr2=0x7fe414100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41a9b4640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe414102d30 0x7fe414100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41a9b4640 1 -- 172.21.15.62:0/1633332024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4141a6ac0 con 0x7fe4141082a0 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe41a9b4640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4141082a0 0x7fe414101410 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00eb10 tx=0x7fe40c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.119+0000 7fe40b7fe640 1 -- 172.21.15.62:0/1633332024 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c004b20 con 0x7fe4141082a0 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.120+0000 7fe40b7fe640 1 -- 172.21.15.62:0/1633332024 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40c004cc0 con 0x7fe4141082a0 2024-09-11T23:59:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.120+0000 7fe41c417640 1 -- 172.21.15.62:0/1633332024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4141a6db0 con 0x7fe4141082a0 2024-09-11T23:59:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.120+0000 7fe40b7fe640 1 -- 172.21.15.62:0/1633332024 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c010760 con 0x7fe4141082a0 2024-09-11T23:59:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.120+0000 7fe41c417640 1 -- 172.21.15.62:0/1633332024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4141a72c0 con 0x7fe4141082a0 2024-09-11T23:59:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.121+0000 7fe41c417640 1 -- 172.21.15.62:0/1633332024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe414103110 con 0x7fe4141082a0 2024-09-11T23:59:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.123+0000 7fe40b7fe640 1 -- 172.21.15.62:0/1633332024 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe40c010900 con 0x7fe4141082a0 2024-09-11T23:59:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.123+0000 7fe40b7fe640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4000a2d70 0x7fe4000a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.124+0000 7fe41a1b3640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4000a2d70 0x7fe4000a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.124+0000 7fe40b7fe640 1 -- 172.21.15.62:0/1633332024 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7fe40c0cfad0 con 0x7fe4141082a0 2024-09-11T23:59:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.124+0000 7fe41a1b3640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4000a2d70 0x7fe4000a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe410005fd0 tx=0x7fe41000b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.126+0000 7fe40b7fe640 1 -- 172.21.15.62:0/1633332024 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe40c0541b0 con 0x7fe4141082a0 2024-09-11T23:59:22.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.339+0000 7fe41c417640 1 -- 172.21.15.62:0/1633332024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe414101ec0 con 0x7fe4141082a0 2024-09-11T23:59:22.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.340+0000 7fe40b7fe640 1 -- 172.21.15.62:0/1633332024 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11910 (secure 0 0 0) 0x7fe414101ec0 con 0x7fe4141082a0 2024-09-11T23:59:22.341 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:59:22.341 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-09-11T23:57:32:617369+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:57:32.075842+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:59:22.341 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-09-11T23:59:22.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.342+0000 7fe41c417640 1 -- 172.21.15.62:0/1633332024 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4000a2d70 msgr2=0x7fe4000a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:22.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.342+0000 7fe41c417640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4000a2d70 0x7fe4000a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe410005fd0 tx=0x7fe41000b390 comp rx=0 tx=0).stop 2024-09-11T23:59:22.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.342+0000 7fe41c417640 1 -- 172.21.15.62:0/1633332024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4141082a0 msgr2=0x7fe414101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:22.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.342+0000 7fe41c417640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4141082a0 0x7fe414101410 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00eb10 tx=0x7fe40c00c2d0 comp rx=0 tx=0).stop 2024-09-11T23:59:22.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.342+0000 7fe41c417640 1 -- 172.21.15.62:0/1633332024 shutdown_connections 2024-09-11T23:59:22.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.342+0000 7fe41c417640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe4000a2d70 0x7fe4000a5230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:22.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.342+0000 7fe41c417640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4141082a0 0x7fe414101410 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:22.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.342+0000 7fe41c417640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4141036e0 0x7fe414100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:22.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.342+0000 7fe41c417640 1 --2- 172.21.15.62:0/1633332024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe414102d30 0x7fe414100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:22.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.342+0000 7fe41c417640 1 -- 172.21.15.62:0/1633332024 >> 172.21.15.62:0/1633332024 conn(0x7fe4140fc7d0 msgr2=0x7fe414105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:22.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.343+0000 7fe41c417640 1 -- 172.21.15.62:0/1633332024 shutdown_connections 2024-09-11T23:59:22.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:22.343+0000 7fe41c417640 1 -- 172.21.15.62:0/1633332024 wait complete. 2024-09-11T23:59:22.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 400.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-11T23:59:23.074 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 63b8b04 --release --await' 2024-09-11T23:59:23.075 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 63b8b04 --release --await 2024-09-11T23:59:23.339 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:23.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:23 smithi062 ceph-mon[30958]: pgmap v223: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-09-11T23:59:23.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:23 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1633332024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:23.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:23 smithi120 ceph-mon[35182]: pgmap v223: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-09-11T23:59:23.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:23 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1633332024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:23.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:23 smithi083 ceph-mon[34805]: pgmap v223: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-09-11T23:59:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:23 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1633332024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.805+0000 7f0f04163640 1 -- 172.21.15.62:0/447511774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101400 msgr2=0x7f0efc10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.805+0000 7f0f04163640 1 --2- 172.21.15.62:0/447511774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101400 0x7f0efc10d930 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009cd0 tx=0x7f0ef402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.805+0000 7f0f04163640 1 -- 172.21.15.62:0/447511774 shutdown_connections 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.805+0000 7f0f04163640 1 --2- 172.21.15.62:0/447511774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101400 0x7f0efc10d930 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.805+0000 7f0f04163640 1 --2- 172.21.15.62:0/447511774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc100a60 0x7f0efc100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.805+0000 7f0f04163640 1 --2- 172.21.15.62:0/447511774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc106a60 0x7f0efc106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.805+0000 7f0f04163640 1 -- 172.21.15.62:0/447511774 >> 172.21.15.62:0/447511774 conn(0x7f0efc0fc780 msgr2=0x7f0efc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.805+0000 7f0f04163640 1 -- 172.21.15.62:0/447511774 shutdown_connections 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.805+0000 7f0f04163640 1 -- 172.21.15.62:0/447511774 wait complete. 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f04163640 1 Processor -- start 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f04163640 1 -- start start 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f04163640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc100a60 0x7f0efc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f04163640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101400 0x7f0efc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f04163640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc106a60 0x7f0efc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f04163640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc110e80 con 0x7f0efc101400 2024-09-11T23:59:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f04163640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc110d00 con 0x7f0efc106a60 2024-09-11T23:59:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f04163640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc111000 con 0x7f0efc100a60 2024-09-11T23:59:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f016fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101400 0x7f0efc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f016fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101400 0x7f0efc1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55440/0 (socket says 172.21.15.62:55440) 2024-09-11T23:59:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f016fe640 1 -- 172.21.15.62:0/3534133088 learned_addr learned my addr 172.21.15.62:0/3534133088 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f02700640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc106a60 0x7f0efc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f01eff640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc100a60 0x7f0efc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f016fe640 1 -- 172.21.15.62:0/3534133088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc100a60 msgr2=0x7f0efc198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f016fe640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc100a60 0x7f0efc198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f016fe640 1 -- 172.21.15.62:0/3534133088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc106a60 msgr2=0x7f0efc19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.806+0000 7f0f016fe640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc106a60 0x7f0efc19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.807+0000 7f0f016fe640 1 -- 172.21.15.62:0/3534133088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc19d430 con 0x7f0efc101400 2024-09-11T23:59:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.807+0000 7f0f016fe640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101400 0x7f0efc1989c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00c890 tx=0x7f0eec00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.807+0000 7f0ef2ffd640 1 -- 172.21.15.62:0/3534133088 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec004220 con 0x7f0efc101400 2024-09-11T23:59:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.807+0000 7f0ef2ffd640 1 -- 172.21.15.62:0/3534133088 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eec0043c0 con 0x7f0efc101400 2024-09-11T23:59:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.807+0000 7f0f04163640 1 -- 172.21.15.62:0/3534133088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc19d720 con 0x7f0efc101400 2024-09-11T23:59:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.807+0000 7f0ef2ffd640 1 -- 172.21.15.62:0/3534133088 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec004a00 con 0x7f0efc101400 2024-09-11T23:59:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.807+0000 7f0f04163640 1 -- 172.21.15.62:0/3534133088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc1a5000 con 0x7f0efc101400 2024-09-11T23:59:23.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.809+0000 7f0ef2ffd640 1 -- 172.21.15.62:0/3534133088 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0eec0079b0 con 0x7f0efc101400 2024-09-11T23:59:23.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.809+0000 7f0f0653c740 1 Processor -- start 2024-09-11T23:59:23.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f0653c740 1 -- start start 2024-09-11T23:59:23.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 0x55ac12de7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12df1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 0x55ac12df4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f0653c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ac12d4c620 con 0x55ac12de7980 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f0653c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ac12cf3e50 con 0x55ac12de7060 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f0653c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ac12d485e0 con 0x55ac12df2240 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f01eff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12df1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f01eff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12df1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55444/0 (socket says 172.21.15.62:55444) 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f01eff640 1 -- 172.21.15.62:0/4208690291 learned_addr learned my addr 172.21.15.62:0/4208690291 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f02700640 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 0x55ac12de7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0ef2ffd640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0ee80a2cf0 0x7f0ee80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f016fe640 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 0x55ac12df4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f01eff640 1 -- 172.21.15.62:0/4208690291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 msgr2=0x55ac12df4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f01eff640 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 0x55ac12df4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f01eff640 1 -- 172.21.15.62:0/4208690291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 msgr2=0x55ac12de7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f01eff640 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 0x55ac12de7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.810+0000 7f0f01eff640 1 -- 172.21.15.62:0/4208690291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac12ce4d40 con 0x55ac12de7980 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.811+0000 7f0f02700640 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 0x55ac12de7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.811+0000 7f0f016fe640 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 0x55ac12df4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.811+0000 7f0f02700640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0ee80a2cf0 0x7f0ee80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.811+0000 7f0ef2ffd640 1 -- 172.21.15.62:0/3534133088 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f0eec0cf080 con 0x7f0efc101400 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.811+0000 7f0f01eff640 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12df1d00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009a70 tx=0x7f0ef8009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6abb6094bcc95e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.811+0000 7f0ee7fff640 1 -- 172.21.15.62:0/4208690291 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8017070 con 0x55ac12de7980 2024-09-11T23:59:23.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.811+0000 7f0ee7fff640 1 -- 172.21.15.62:0/4208690291 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef8013470 con 0x55ac12de7980 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.811+0000 7f0ee7fff640 1 -- 172.21.15.62:0/4208690291 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8012630 con 0x55ac12de7980 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 -- 172.21.15.62:0/4208690291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 msgr2=0x55ac12df1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12df1d00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009a70 tx=0x7f0ef8009f30 comp rx=0 tx=0).stop 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f02700640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0ee80a2cf0 0x7f0ee80a51b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40098d0 tx=0x7f0ef403e040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 -- 172.21.15.62:0/4208690291 shutdown_connections 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 0x55ac12df4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12df1d00 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 --2- 172.21.15.62:0/4208690291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 0x55ac12de7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 -- 172.21.15.62:0/4208690291 >> 172.21.15.62:0/4208690291 conn(0x55ac12de35f0 msgr2=0x55ac12de39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 -- 172.21.15.62:0/4208690291 shutdown_connections 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 -- 172.21.15.62:0/4208690291 wait complete. 2024-09-11T23:59:23.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 -- start start 2024-09-11T23:59:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.812+0000 7f0f0653c740 1 Processor -- start 2024-09-11T23:59:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12def0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12de7980 0x55ac12def5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 0x55ac12defb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f0653c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ac12df4e10 con 0x55ac12de7060 2024-09-11T23:59:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f0653c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ac12cd4f10 con 0x55ac12df2240 2024-09-11T23:59:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f0653c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ac12df4f90 con 0x55ac12de7980 2024-09-11T23:59:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f0653c740 10 client.0 _ll_register_callbacks cb 0x55ac12cf8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f01eff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12def0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f01eff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12def0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55446/0 (socket says 172.21.15.62:55446) 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f01eff640 1 -- 172.21.15.62:0/655670946 learned_addr learned my addr 172.21.15.62:0/655670946 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f016fe640 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12de7980 0x55ac12def5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f02700640 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 0x55ac12defb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f01eff640 1 -- 172.21.15.62:0/655670946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12de7980 msgr2=0x55ac12def5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f01eff640 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12de7980 0x55ac12def5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f01eff640 1 -- 172.21.15.62:0/655670946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 msgr2=0x55ac12defb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f01eff640 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 0x55ac12defb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f01eff640 1 -- 172.21.15.62:0/655670946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac12a5b580 con 0x55ac12de7060 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.813+0000 7f0f02700640 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 0x55ac12defb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0f01eff640 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12def0a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000c00 tx=0x7f0ef8013cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7025a5d7aeeb007 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0ee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0ee57fa640 1 -- 172.21.15.62:0/655670946 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8017050 con 0x55ac12de7060 2024-09-11T23:59:23.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0f0653c740 1 -- 172.21.15.62:0/655670946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef8013470 con 0x55ac12de7060 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0ee57fa640 1 -- 172.21.15.62:0/655670946 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f0ef801c650 con 0x55ac12de7060 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0ee57fa640 1 -- 172.21.15.62:0/655670946 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ef8013a70 con 0x55ac12de7060 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0f0653c740 10 client.16671 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0f0653c740 1 -- 172.21.15.62:0/655670946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac12b25960 con 0x55ac12de7060 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0ee57fa640 1 -- 172.21.15.62:0/655670946 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef8013470 con 0x55ac12de7060 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0ee57fa640 1 -- 172.21.15.62:0/655670946 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f0ef8026ab0 con 0x55ac12de7060 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0f0653c740 10 client.16671 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0f0653c740 10 client.16671 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0f0653c740 20 client.16671 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0f0653c740 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ac12e01c40 0x55ac12e04030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0f0653c740 4 client.16671 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:23.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.814+0000 7f0f0653c740 1 -- 172.21.15.62:0/655670946 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac12d59b90 con 0x55ac12e01c40 2024-09-11T23:59:23.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.815+0000 7f0f016fe640 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ac12e01c40 0x55ac12e04030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.815+0000 7f0f016fe640 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ac12e01c40 0x55ac12e04030 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=224f73a69326481f server_cookie=74d66336fabf92fd in_seq=0 out_seq=0 2024-09-11T23:59:23.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.815+0000 7f0ee57fa640 10 client.16671 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:23.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0ee57fa640 1 -- 172.21.15.62:0/655670946 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac12d59b90 con 0x55ac12e01c40 2024-09-11T23:59:23.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0ee57fa640 10 client.16671 handle_command_reply: tid=0 2024-09-11T23:59:23.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 client.16671 shutdown 2024-09-11T23:59:23.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 -- 172.21.15.62:0/655670946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 msgr2=0x55ac12def0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12def0a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000c00 tx=0x7f0ef8013cd0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 -- 172.21.15.62:0/655670946 shutdown_connections 2024-09-11T23:59:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ac12e01c40 0x55ac12e04030 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 0x55ac12defb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12de7980 0x55ac12def5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 --2- 172.21.15.62:0/655670946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12def0a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0ee57fa640 0 client.16671 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 -- 172.21.15.62:0/655670946 >> 172.21.15.62:0/655670946 conn(0x55ac12de35f0 msgr2=0x55ac12de9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 -- 172.21.15.62:0/655670946 shutdown_connections 2024-09-11T23:59:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 1 -- 172.21.15.62:0/655670946 wait complete. 2024-09-11T23:59:23.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.816+0000 7f0f0653c740 20 client.16671 trim_cache size 0 max 16384 2024-09-11T23:59:23.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.833+0000 7f0f0653c740 1 Processor -- start 2024-09-11T23:59:23.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.833+0000 7f0f0653c740 1 -- start start 2024-09-11T23:59:23.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.833+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 0x55ac12de9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.833+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12de2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.833+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 0x55ac12e09660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.833+0000 7f0f0653c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1b02e0 con 0x55ac12de7980 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.833+0000 7f0f0653c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1adb80 con 0x55ac12de7060 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.833+0000 7f0f0653c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1ab9f0 con 0x55ac12df2240 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f016fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12de2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f016fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12de2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55456/0 (socket says 172.21.15.62:55456) 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f016fe640 1 -- 172.21.15.62:0/4021421486 learned_addr learned my addr 172.21.15.62:0/4021421486 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f02700640 1 --2- 172.21.15.62:0/4021421486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 0x55ac12e09660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f01eff640 1 --2- 172.21.15.62:0/4021421486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 0x55ac12de9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f016fe640 1 -- 172.21.15.62:0/4021421486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 msgr2=0x55ac12e09660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f016fe640 1 --2- 172.21.15.62:0/4021421486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 0x55ac12e09660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f016fe640 1 -- 172.21.15.62:0/4021421486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 msgr2=0x55ac12de9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f016fe640 1 --2- 172.21.15.62:0/4021421486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 0x55ac12de9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f016fe640 1 -- 172.21.15.62:0/4021421486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac12de3420 con 0x55ac12de7980 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f01eff640 1 --2- 172.21.15.62:0/4021421486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 0x55ac12de9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0f016fe640 1 --2- 172.21.15.62:0/4021421486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12de2640 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0eec097f70 tx=0x7f0eec0d1600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4970ed61841ed49d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0ee77fe640 1 -- 172.21.15.62:0/4021421486 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec01d040 con 0x55ac12de7980 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0ee77fe640 1 -- 172.21.15.62:0/4021421486 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eec09f950 con 0x55ac12de7980 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.834+0000 7f0ee77fe640 1 -- 172.21.15.62:0/4021421486 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0cca50 con 0x55ac12de7980 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 -- 172.21.15.62:0/4021421486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 msgr2=0x55ac12de2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 --2- 172.21.15.62:0/4021421486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12de2640 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0eec097f70 tx=0x7f0eec0d1600 comp rx=0 tx=0).stop 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 -- 172.21.15.62:0/4021421486 shutdown_connections 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 --2- 172.21.15.62:0/4021421486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12df2240 0x55ac12e09660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 --2- 172.21.15.62:0/4021421486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7980 0x55ac12de2640 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 --2- 172.21.15.62:0/4021421486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12de7060 0x55ac12de9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 -- 172.21.15.62:0/4021421486 >> 172.21.15.62:0/4021421486 conn(0x55ac12de35f0 msgr2=0x55ac12de1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 -- 172.21.15.62:0/4021421486 shutdown_connections 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 -- 172.21.15.62:0/4021421486 wait complete. 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 -- start start 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.835+0000 7f0f0653c740 1 Processor -- start 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12dfa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12de7980 0x55ac12dfa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f0653c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 0x55ac12df71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f0653c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1ac040 con 0x55ac12de7060 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f0653c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1ad4b0 con 0x55ac12df2240 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f0653c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1ac710 con 0x55ac12de7980 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f01eff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12dfa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f0653c740 10 client.0 _ll_register_callbacks cb 0x7f0ef800ef20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f01eff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12dfa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55462/0 (socket says 172.21.15.62:55462) 2024-09-11T23:59:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f01eff640 1 -- 172.21.15.62:0/3267126118 learned_addr learned my addr 172.21.15.62:0/3267126118 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f02700640 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 0x55ac12df71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f016fe640 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12de7980 0x55ac12dfa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f01eff640 1 -- 172.21.15.62:0/3267126118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12de7980 msgr2=0x55ac12dfa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f01eff640 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12de7980 0x55ac12dfa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f01eff640 1 -- 172.21.15.62:0/3267126118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 msgr2=0x55ac12df71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f01eff640 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 0x55ac12df71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f01eff640 1 -- 172.21.15.62:0/3267126118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac12dfae60 con 0x55ac12de7060 2024-09-11T23:59:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.836+0000 7f0f02700640 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 0x55ac12df71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0f01eff640 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12dfa1e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8004810 tx=0x7f0ef8009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551b0633edfdc4ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0ee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:59:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0ee57fa640 1 -- 172.21.15.62:0/3267126118 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8033070 con 0x55ac12de7060 2024-09-11T23:59:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0ee57fa640 1 -- 172.21.15.62:0/3267126118 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f0ef802ec80 con 0x55ac12de7060 2024-09-11T23:59:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0f0653c740 1 -- 172.21.15.62:0/3267126118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eec09f950 con 0x55ac12de7060 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0ee57fa640 1 -- 172.21.15.62:0/3267126118 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ef803f050 con 0x55ac12de7060 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0f0653c740 10 client.16677 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0f0653c740 1 -- 172.21.15.62:0/3267126118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac12e6cc30 con 0x55ac12de7060 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0ee57fa640 1 -- 172.21.15.62:0/3267126118 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eec09f950 con 0x55ac12de7060 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0ee57fa640 1 -- 172.21.15.62:0/3267126118 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f0ef801c450 con 0x55ac12de7060 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0f0653c740 10 client.16677 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0f0653c740 10 client.16677 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0f0653c740 20 client.16677 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0f0653c740 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ac12e6d290 0x55ac12e6f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0f0653c740 4 client.16677 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "63b8b04", "release": true, "await": true}] 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.837+0000 7f0f0653c740 1 -- 172.21.15.62:0/3267126118 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "63b8b04", "release": true, "await": true}) -- 0x55ac12dde950 con 0x55ac12e6d290 2024-09-11T23:59:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.838+0000 7f0f016fe640 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ac12e6d290 0x55ac12e6f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:23.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.838+0000 7f0ee57fa640 10 client.16677 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:23.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.838+0000 7f0f016fe640 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ac12e6d290 0x55ac12e6f680 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d6a7fb5005b123 server_cookie=eac7b06a0d02ff9a in_seq=0 out_seq=0 2024-09-11T23:59:23.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0ee57fa640 1 -- 172.21.15.62:0/3267126118 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ac12dde950 con 0x55ac12e6d290 2024-09-11T23:59:23.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0ee57fa640 10 client.16677 handle_command_reply: tid=0 2024-09-11T23:59:23.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0f0653c740 1 client.16677 shutdown 2024-09-11T23:59:23.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0f0653c740 1 -- 172.21.15.62:0/3267126118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 msgr2=0x55ac12dfa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:23.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0f0653c740 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12dfa1e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8004810 tx=0x7f0ef8009110 comp rx=0 tx=0).stop 2024-09-11T23:59:23.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0f0653c740 1 -- 172.21.15.62:0/3267126118 shutdown_connections 2024-09-11T23:59:23.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0f0653c740 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55ac12e6d290 0x55ac12e6f680 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0f0653c740 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac12df2240 0x55ac12df71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0ee57fa640 0 client.16677 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:23.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0f0653c740 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac12de7980 0x55ac12dfa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0f0653c740 1 --2- 172.21.15.62:0/3267126118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12de7060 0x55ac12dfa1e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.887+0000 7f0f0653c740 1 -- 172.21.15.62:0/3267126118 >> 172.21.15.62:0/3267126118 conn(0x55ac12de35f0 msgr2=0x55ac12de2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:23.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.888+0000 7f0f0653c740 1 -- 172.21.15.62:0/3267126118 shutdown_connections 2024-09-11T23:59:23.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.888+0000 7f0f0653c740 1 -- 172.21.15.62:0/3267126118 wait complete. 2024-09-11T23:59:23.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.888+0000 7f0f0653c740 20 client.16677 trim_cache size 0 max 16384 2024-09-11T23:59:23.890 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:59:23.890 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-09-11T23:59:23.890 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:59:23.890 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 28, 2024-09-11T23:59:23.890 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:59:23.890 INFO:teuthology.orchestra.run.smithi062.stdout: "63b8b04": { 2024-09-11T23:59:23.891 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 28, 2024-09-11T23:59:23.891 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:59:23.891 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:59:23.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:59:23.891 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:59:23.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:59:23.891 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:59:23.891 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 136.6, 2024-09-11T23:59:23.891 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:23.892 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:59:23.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.889+0000 7f0f04163640 1 -- 172.21.15.62:0/3534133088 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0ee80a2cf0 msgr2=0x7f0ee80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:23.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.889+0000 7f0f04163640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0ee80a2cf0 0x7f0ee80a51b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40098d0 tx=0x7f0ef403e040 comp rx=0 tx=0).stop 2024-09-11T23:59:23.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.889+0000 7f0f04163640 1 -- 172.21.15.62:0/3534133088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101400 msgr2=0x7f0efc1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:23.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.889+0000 7f0f04163640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101400 0x7f0efc1989c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00c890 tx=0x7f0eec00cc90 comp rx=0 tx=0).stop 2024-09-11T23:59:23.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.889+0000 7f0f04163640 1 -- 172.21.15.62:0/3534133088 shutdown_connections 2024-09-11T23:59:23.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.889+0000 7f0f04163640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0ee80a2cf0 0x7f0ee80a51b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.889+0000 7f0f04163640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc106a60 0x7f0efc19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.889+0000 7f0f04163640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101400 0x7f0efc1989c0 secure :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00c890 tx=0x7f0eec00cc90 comp rx=0 tx=0).stop 2024-09-11T23:59:23.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.889+0000 7f0f04163640 1 --2- 172.21.15.62:0/3534133088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc100a60 0x7f0efc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:23.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.889+0000 7f0f04163640 1 -- 172.21.15.62:0/3534133088 >> 172.21.15.62:0/3534133088 conn(0x7f0efc0fc780 msgr2=0x7f0efc102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:23.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.890+0000 7f0f04163640 1 -- 172.21.15.62:0/3534133088 shutdown_connections 2024-09-11T23:59:23.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:23.890+0000 7f0f04163640 1 -- 172.21.15.62:0/3534133088 wait complete. 2024-09-11T23:59:24.737 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '63b8b04', total seconds elapsed: 19.9 2024-09-11T23:59:25.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:25 smithi120 ceph-mon[35182]: pgmap v224: 97 pgs: 97 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-11T23:59:25.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:25 smithi083 ceph-mon[34805]: pgmap v224: 97 pgs: 97 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-11T23:59:25.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:25 smithi062 ceph-mon[30958]: pgmap v224: 97 pgs: 97 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-11T23:59:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 405.0 s, 2043.9 tps, lat 176.026 ms stddev 1417.049 2024-09-11T23:59:27.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:27 smithi120 ceph-mon[35182]: pgmap v225: 97 pgs: 97 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-11T23:59:27.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:27 smithi083 ceph-mon[34805]: pgmap v225: 97 pgs: 97 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-11T23:59:27.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:27 smithi062 ceph-mon[30958]: pgmap v225: 97 pgs: 97 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-11T23:59:29.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:29 smithi120 ceph-mon[35182]: pgmap v226: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 1.05k op/s 2024-09-11T23:59:29.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:29 smithi083 ceph-mon[34805]: pgmap v226: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 1.05k op/s 2024-09-11T23:59:29.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:29 smithi062 ceph-mon[30958]: pgmap v226: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 1.05k op/s 2024-09-11T23:59:31.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:31 smithi062 ceph-mon[30958]: pgmap v227: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-11T23:59:31.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:31 smithi120 ceph-mon[35182]: pgmap v227: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-11T23:59:31.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:31 smithi083 ceph-mon[34805]: pgmap v227: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-09-11T23:59:32.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 410.0 s, 3104.2 tps, lat 32.060 ms stddev 34.926 2024-09-11T23:59:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:33 smithi120 ceph-mon[35182]: pgmap v228: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-09-11T23:59:33.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:33 smithi083 ceph-mon[34805]: pgmap v228: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-09-11T23:59:33.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:33 smithi062 ceph-mon[30958]: pgmap v228: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-09-11T23:59:35.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:35 smithi120 ceph-mon[35182]: pgmap v229: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-11T23:59:35.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:35 smithi083 ceph-mon[34805]: pgmap v229: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-11T23:59:35.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:35 smithi062 ceph-mon[30958]: pgmap v229: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-11T23:59:36.063 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:59:36.450 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.000+0000 7ff510db2640 1 -- 172.21.15.62:0/2476955214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c106a70 msgr2=0x7ff50c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.000+0000 7ff510db2640 1 --2- 172.21.15.62:0/2476955214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c106a70 0x7ff50c106e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00d940 tx=0x7ff4fc02c3a0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.000+0000 7ff510db2640 1 -- 172.21.15.62:0/2476955214 shutdown_connections 2024-09-11T23:59:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.000+0000 7ff510db2640 1 --2- 172.21.15.62:0/2476955214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101410 0x7ff50c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.000+0000 7ff510db2640 1 --2- 172.21.15.62:0/2476955214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff50c100a70 0x7ff50c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.001+0000 7ff510db2640 1 --2- 172.21.15.62:0/2476955214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c106a70 0x7ff50c106e50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.001+0000 7ff510db2640 1 -- 172.21.15.62:0/2476955214 >> 172.21.15.62:0/2476955214 conn(0x7ff50c0fc7d0 msgr2=0x7ff50c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.001+0000 7ff510db2640 1 -- 172.21.15.62:0/2476955214 shutdown_connections 2024-09-11T23:59:37.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.001+0000 7ff510db2640 1 -- 172.21.15.62:0/2476955214 wait complete. 2024-09-11T23:59:37.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.002+0000 7ff510db2640 1 Processor -- start 2024-09-11T23:59:37.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.002+0000 7ff510db2640 1 -- start start 2024-09-11T23:59:37.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.002+0000 7ff510db2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff50c100a70 0x7ff50c19d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.002+0000 7ff510db2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101410 0x7ff50c196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff510db2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c106a70 0x7ff50c196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff510db2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c10fe00 con 0x7ff50c100a70 2024-09-11T23:59:37.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff510db2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c10fc80 con 0x7ff50c101410 2024-09-11T23:59:37.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff510db2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c10ff80 con 0x7ff50c106a70 2024-09-11T23:59:37.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff50a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff50c100a70 0x7ff50c19d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff50a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff50c100a70 0x7ff50c19d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53464/0 (socket says 172.21.15.62:53464) 2024-09-11T23:59:37.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff50a59c640 1 -- 172.21.15.62:0/4254346048 learned_addr learned my addr 172.21.15.62:0/4254346048 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:37.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff50ad9d640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c106a70 0x7ff50c196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff501d9b640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101410 0x7ff50c196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff50ad9d640 1 -- 172.21.15.62:0/4254346048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101410 msgr2=0x7ff50c196120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:37.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff50ad9d640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101410 0x7ff50c196120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff50ad9d640 1 -- 172.21.15.62:0/4254346048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff50c100a70 msgr2=0x7ff50c19d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:37.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff50ad9d640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff50c100a70 0x7ff50c19d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.003+0000 7ff50ad9d640 1 -- 172.21.15.62:0/4254346048 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c196bf0 con 0x7ff50c106a70 2024-09-11T23:59:37.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.004+0000 7ff50a59c640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff50c100a70 0x7ff50c19d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:59:37.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.004+0000 7ff50ad9d640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c106a70 0x7ff50c196660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff504009fd0 tx=0x7ff50400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:37.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.004+0000 7ff503fff640 1 -- 172.21.15.62:0/4254346048 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50400ce20 con 0x7ff50c106a70 2024-09-11T23:59:37.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.004+0000 7ff503fff640 1 -- 172.21.15.62:0/4254346048 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff504005020 con 0x7ff50c106a70 2024-09-11T23:59:37.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.004+0000 7ff503fff640 1 -- 172.21.15.62:0/4254346048 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504010720 con 0x7ff50c106a70 2024-09-11T23:59:37.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.004+0000 7ff510db2640 1 -- 172.21.15.62:0/4254346048 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c196ee0 con 0x7ff50c106a70 2024-09-11T23:59:37.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.004+0000 7ff510db2640 1 -- 172.21.15.62:0/4254346048 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c19ae80 con 0x7ff50c106a70 2024-09-11T23:59:37.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.009+0000 7ff51318b740 1 Processor -- start 2024-09-11T23:59:37.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.009+0000 7ff503fff640 1 -- 172.21.15.62:0/4254346048 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff504005540 con 0x7ff50c106a70 2024-09-11T23:59:37.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff51318b740 1 -- start start 2024-09-11T23:59:37.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff51318b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbd4a0 0x557fe2bbd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff51318b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbddc0 0x557fe2bc8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff51318b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bc8680 0x557fe2bcaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff51318b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557fe2b232e0 con 0x557fe2bbddc0 2024-09-11T23:59:37.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff51318b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557fe2acace0 con 0x557fe2bbd4a0 2024-09-11T23:59:37.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff51318b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557fe2b1f2a0 con 0x557fe2bc8680 2024-09-11T23:59:37.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff50a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbddc0 0x557fe2bc8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff50a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbddc0 0x557fe2bc8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53466/0 (socket says 172.21.15.62:53466) 2024-09-11T23:59:37.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff50a59c640 1 -- 172.21.15.62:0/2793305740 learned_addr learned my addr 172.21.15.62:0/2793305740 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:37.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.010+0000 7ff503fff640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff4f80a2e40 0x7ff4f80a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff503fff640 1 -- 172.21.15.62:0/4254346048 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7ff5040cfaf0 con 0x7ff50c106a70 2024-09-11T23:59:37.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff50ad9d640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff4f80a2e40 0x7ff4f80a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff50a59c640 1 -- 172.21.15.62:0/2793305740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bc8680 msgr2=0x557fe2bcaa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:59:37.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff50a59c640 1 --2- 172.21.15.62:0/2793305740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bc8680 0x557fe2bcaa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff50a59c640 1 -- 172.21.15.62:0/2793305740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbd4a0 msgr2=0x557fe2bbd880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:59:37.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff501d9b640 1 --2- 172.21.15.62:0/2793305740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbd4a0 0x557fe2bbd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff50a59c640 1 --2- 172.21.15.62:0/2793305740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbd4a0 0x557fe2bbd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff50a59c640 1 -- 172.21.15.62:0/2793305740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fe2aaf440 con 0x557fe2bbddc0 2024-09-11T23:59:37.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff50a59c640 1 --2- 172.21.15.62:0/2793305740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbddc0 0x557fe2bc8140 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00d050 tx=0x7ff4fc00d080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5c035794521dda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:37.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff50ad9d640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff4f80a2e40 0x7ff4f80a5300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff504000f80 tx=0x7ff5040054b0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:37.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.011+0000 7ff500d99640 1 -- 172.21.15.62:0/2793305740 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00d2a0 con 0x557fe2bbddc0 2024-09-11T23:59:37.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.012+0000 7ff500d99640 1 -- 172.21.15.62:0/2793305740 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4fc030050 con 0x557fe2bbddc0 2024-09-11T23:59:37.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.012+0000 7ff500d99640 1 -- 172.21.15.62:0/2793305740 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc034440 con 0x557fe2bbddc0 2024-09-11T23:59:37.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.013+0000 7ff51318b740 1 -- 172.21.15.62:0/2793305740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbddc0 msgr2=0x557fe2bc8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:37.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.013+0000 7ff51318b740 1 --2- 172.21.15.62:0/2793305740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbddc0 0x557fe2bc8140 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00d050 tx=0x7ff4fc00d080 comp rx=0 tx=0).stop 2024-09-11T23:59:37.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.014+0000 7ff51318b740 1 -- 172.21.15.62:0/2793305740 shutdown_connections 2024-09-11T23:59:37.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.014+0000 7ff51318b740 1 --2- 172.21.15.62:0/2793305740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bc8680 0x557fe2bcaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.014+0000 7ff51318b740 1 --2- 172.21.15.62:0/2793305740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbddc0 0x557fe2bc8140 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.014+0000 7ff51318b740 1 --2- 172.21.15.62:0/2793305740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbd4a0 0x557fe2bbd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.014+0000 7ff51318b740 1 -- 172.21.15.62:0/2793305740 >> 172.21.15.62:0/2793305740 conn(0x557fe2bb9a30 msgr2=0x557fe2bb9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:37.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.014+0000 7ff51318b740 1 -- 172.21.15.62:0/2793305740 shutdown_connections 2024-09-11T23:59:37.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.014+0000 7ff51318b740 1 -- 172.21.15.62:0/2793305740 wait complete. 2024-09-11T23:59:37.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff51318b740 1 -- start start 2024-09-11T23:59:37.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff51318b740 1 Processor -- start 2024-09-11T23:59:37.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff51318b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff51318b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbddc0 0x557fe2bc5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff51318b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bc8680 0x557fe2bc5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff51318b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557fe2bcb2f0 con 0x557fe2bbd4a0 2024-09-11T23:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff51318b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557fe2ade050 con 0x557fe2bbddc0 2024-09-11T23:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff51318b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557fe2bcb470 con 0x557fe2bc8680 2024-09-11T23:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff51318b740 10 client.0 _ll_register_callbacks cb 0x557fe2b1e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff501d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff501d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53482/0 (socket says 172.21.15.62:53482) 2024-09-11T23:59:37.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff50a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbddc0 0x557fe2bc5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff501d9b640 1 -- 172.21.15.62:0/2899698150 learned_addr learned my addr 172.21.15.62:0/2899698150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:37.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.015+0000 7ff501d9b640 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bc8680 0x557fe2bc5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.016+0000 7ff50a59c640 1 -- 172.21.15.62:0/2899698150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bc8680 msgr2=0x557fe2bc5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:37.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.016+0000 7ff50a59c640 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bc8680 0x557fe2bc5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.016+0000 7ff50a59c640 1 -- 172.21.15.62:0/2899698150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 msgr2=0x557fe2bc7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:37.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.016+0000 7ff50a59c640 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.016+0000 7ff50a59c640 1 -- 172.21.15.62:0/2899698150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fe28943c0 con 0x557fe2bbddc0 2024-09-11T23:59:37.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.016+0000 7ff50a59c640 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbddc0 0x557fe2bc5220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00d910 tx=0x7ff4fc03e660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1395c2ad4c20c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:37.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.016+0000 7ff4f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:59:37.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.016+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/2899698150 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc03ea60 con 0x557fe2bbddc0 2024-09-11T23:59:37.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.016+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/2899698150 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7ff4fc03ec00 con 0x557fe2bbddc0 2024-09-11T23:59:37.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.016+0000 7ff51318b740 1 -- 172.21.15.62:0/2899698150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc030050 con 0x557fe2bbddc0 2024-09-11T23:59:37.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.017+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/2899698150 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4fc004020 con 0x557fe2bbddc0 2024-09-11T23:59:37.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.017+0000 7ff51318b740 10 client.26285 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.017+0000 7ff51318b740 1 -- 172.21.15.62:0/2899698150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557fe2bd6c60 con 0x557fe2bbddc0 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.018+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/2899698150 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4fc030050 con 0x557fe2bbddc0 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.018+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/2899698150 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7ff4fc03d070 con 0x557fe2bbddc0 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.019+0000 7ff51318b740 10 client.26285 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.019+0000 7ff51318b740 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.019+0000 7ff51318b740 20 client.26285 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.019+0000 7ff51318b740 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fe2bd7fb0 0x557fe2bda3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.019+0000 7ff51318b740 4 client.26285 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.019+0000 7ff51318b740 1 -- 172.21.15.62:0/2899698150 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fe2b30850 con 0x557fe2bd7fb0 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.020+0000 7ff501d9b640 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fe2bd7fb0 0x557fe2bda3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.020+0000 7ff501d9b640 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fe2bd7fb0 0x557fe2bda3a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca4e69f69af0a1c server_cookie=4eeaa35e616dec5d in_seq=0 out_seq=0 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.020+0000 7ff4f1ffb640 10 client.26285 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/2899698150 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fe2b30850 con 0x557fe2bd7fb0 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff4f1ffb640 10 client.26285 handle_command_reply: tid=0 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 client.26285 shutdown 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 -- 172.21.15.62:0/2899698150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbddc0 msgr2=0x557fe2bc5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbddc0 0x557fe2bc5220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00d910 tx=0x7ff4fc03e660 comp rx=0 tx=0).stop 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 -- 172.21.15.62:0/2899698150 shutdown_connections 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fe2bd7fb0 0x557fe2bda3a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bc8680 0x557fe2bc5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbddc0 0x557fe2bc5220 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 --2- 172.21.15.62:0/2899698150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff4f1ffb640 0 client.26285 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 -- 172.21.15.62:0/2899698150 >> 172.21.15.62:0/2899698150 conn(0x557fe2bb9a30 msgr2=0x557fe2bbf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 -- 172.21.15.62:0/2899698150 shutdown_connections 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 1 -- 172.21.15.62:0/2899698150 wait complete. 2024-09-11T23:59:37.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.021+0000 7ff51318b740 20 client.26285 trim_cache size 0 max 16384 2024-09-11T23:59:37.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.050+0000 7ff51318b740 1 Processor -- start 2024-09-11T23:59:37.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.051+0000 7ff51318b740 1 -- start start 2024-09-11T23:59:37.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.051+0000 7ff51318b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.051+0000 7ff51318b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbddc0 0x557fe2bc29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.051+0000 7ff501d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff501d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:54140/0 (socket says 172.21.15.62:54140) 2024-09-11T23:59:37.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.051+0000 7ff51318b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bc8680 0x557fe2bc39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff501d9b640 1 -- 172.21.15.62:0/3408986939 learned_addr learned my addr 172.21.15.62:0/3408986939 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:37.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff51318b740 1 -- 172.21.15.62:0/3408986939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1bef10 con 0x557fe2bc8680 2024-09-11T23:59:37.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff51318b740 1 -- 172.21.15.62:0/3408986939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1bc7b0 con 0x557fe2bbddc0 2024-09-11T23:59:37.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff51318b740 1 -- 172.21.15.62:0/3408986939 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1ba620 con 0x557fe2bbd4a0 2024-09-11T23:59:37.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff501d9b640 1 -- 172.21.15.62:0/3408986939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbddc0 msgr2=0x557fe2bc29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:37.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff501d9b640 1 --2- 172.21.15.62:0/3408986939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbddc0 0x557fe2bc29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff501d9b640 1 -- 172.21.15.62:0/3408986939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bc8680 msgr2=0x557fe2bc39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:37.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff501d9b640 1 --2- 172.21.15.62:0/3408986939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bc8680 0x557fe2bc39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff501d9b640 1 -- 172.21.15.62:0/3408986939 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fe2bba6c0 con 0x557fe2bbd4a0 2024-09-11T23:59:37.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.052+0000 7ff501d9b640 1 --2- 172.21.15.62:0/3408986939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc2480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x557fe2bcb4f0 tx=0x7ff4f4010720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50432c37130c45d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:37.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.053+0000 7ff4f3fff640 1 -- 172.21.15.62:0/3408986939 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4028420 con 0x557fe2bbd4a0 2024-09-11T23:59:37.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.054+0000 7ff4f3fff640 1 -- 172.21.15.62:0/3408986939 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f4023030 con 0x557fe2bbd4a0 2024-09-11T23:59:37.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.054+0000 7ff4f3fff640 1 -- 172.21.15.62:0/3408986939 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4017670 con 0x557fe2bbd4a0 2024-09-11T23:59:37.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.055+0000 7ff51318b740 1 -- 172.21.15.62:0/3408986939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbd4a0 msgr2=0x557fe2bc2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:37.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.055+0000 7ff51318b740 1 --2- 172.21.15.62:0/3408986939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc2480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x557fe2bcb4f0 tx=0x7ff4f4010720 comp rx=0 tx=0).stop 2024-09-11T23:59:37.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.057+0000 7ff51318b740 1 -- 172.21.15.62:0/3408986939 shutdown_connections 2024-09-11T23:59:37.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.057+0000 7ff51318b740 1 --2- 172.21.15.62:0/3408986939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bc8680 0x557fe2bc39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.057+0000 7ff51318b740 1 --2- 172.21.15.62:0/3408986939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bbddc0 0x557fe2bc29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.057+0000 7ff51318b740 1 --2- 172.21.15.62:0/3408986939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbd4a0 0x557fe2bc2480 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.057+0000 7ff51318b740 1 -- 172.21.15.62:0/3408986939 >> 172.21.15.62:0/3408986939 conn(0x557fe2bb9a30 msgr2=0x557fe2bc08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:37.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.062+0000 7ff51318b740 1 -- 172.21.15.62:0/3408986939 shutdown_connections 2024-09-11T23:59:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.064+0000 7ff51318b740 1 -- 172.21.15.62:0/3408986939 wait complete. 2024-09-11T23:59:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.064+0000 7ff51318b740 1 -- start start 2024-09-11T23:59:37.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff51318b740 1 Processor -- start 2024-09-11T23:59:37.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff51318b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bdaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff51318b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbddc0 0x557fe2bdb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff501d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bdaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff51318b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bc8680 0x557fe2c3a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff51318b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1bac70 con 0x557fe2bbd4a0 2024-09-11T23:59:37.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff501d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bdaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53504/0 (socket says 172.21.15.62:53504) 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff51318b740 1 -- 172.21.15.62:0/241630198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1bc0e0 con 0x557fe2bc8680 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff501d9b640 1 -- 172.21.15.62:0/241630198 learned_addr learned my addr 172.21.15.62:0/241630198 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff51318b740 1 -- 172.21.15.62:0/241630198 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1bb340 con 0x557fe2bbddc0 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff51318b740 10 client.0 _ll_register_callbacks cb 0x7ff4fc00d790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.065+0000 7ff50a59c640 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbddc0 0x557fe2bdb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.066+0000 7ff501d9b640 1 -- 172.21.15.62:0/241630198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbddc0 msgr2=0x557fe2bdb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.066+0000 7ff501d9b640 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbddc0 0x557fe2bdb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.066+0000 7ff501d9b640 1 -- 172.21.15.62:0/241630198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bc8680 msgr2=0x557fe2c3a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.066+0000 7ff501d9b640 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bc8680 0x557fe2c3a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.066+0000 7ff501d9b640 1 -- 172.21.15.62:0/241630198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fe2c3af50 con 0x557fe2bbd4a0 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.066+0000 7ff501d9b640 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bdaac0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4006d60 tx=0x7ff4f4027530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0c1488e800ce92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.066+0000 7ff4f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.067+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/241630198 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f400f040 con 0x557fe2bbd4a0 2024-09-11T23:59:37.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.067+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/241630198 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7ff4f40278a0 con 0x557fe2bbd4a0 2024-09-11T23:59:37.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.067+0000 7ff51318b740 1 -- 172.21.15.62:0/241630198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f4023030 con 0x557fe2bbd4a0 2024-09-11T23:59:37.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.067+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/241630198 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4f4027aa0 con 0x557fe2bbd4a0 2024-09-11T23:59:37.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.070+0000 7ff51318b740 10 client.16689 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:37.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.070+0000 7ff51318b740 1 -- 172.21.15.62:0/241630198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557fe2c64320 con 0x557fe2bbd4a0 2024-09-11T23:59:37.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.071+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/241630198 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f4027c40 con 0x557fe2bbd4a0 2024-09-11T23:59:37.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.071+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/241630198 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7ff4f4017650 con 0x557fe2bbd4a0 2024-09-11T23:59:37.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.071+0000 7ff51318b740 10 client.16689 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:37.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.071+0000 7ff51318b740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:37.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.071+0000 7ff51318b740 20 client.16689 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:37.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.071+0000 7ff51318b740 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fe2c647d0 0x557fe2c66c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:37.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.071+0000 7ff51318b740 4 client.16689 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:59:37.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.071+0000 7ff51318b740 1 -- 172.21.15.62:0/241630198 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff4f4023030 con 0x557fe2c647d0 2024-09-11T23:59:37.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.071+0000 7ff50a59c640 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fe2c647d0 0x557fe2c66c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:37.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.076+0000 7ff50a59c640 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fe2c647d0 0x557fe2c66c10 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6440a5f48f9dc541 server_cookie=40460b59ec0f5ce5 in_seq=0 out_seq=0 2024-09-11T23:59:37.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.076+0000 7ff4f1ffb640 10 client.16689 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:37.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.077+0000 7ff4f1ffb640 1 -- 172.21.15.62:0/241630198 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7ff4f4023030 con 0x557fe2c647d0 2024-09-11T23:59:37.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.077+0000 7ff4f1ffb640 10 client.16689 handle_command_reply: tid=0 2024-09-11T23:59:37.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.077+0000 7ff51318b740 1 client.16689 shutdown 2024-09-11T23:59:37.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.081+0000 7ff51318b740 1 -- 172.21.15.62:0/241630198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 msgr2=0x557fe2bdaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:37.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.081+0000 7ff51318b740 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bdaac0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4006d60 tx=0x7ff4f4027530 comp rx=0 tx=0).stop 2024-09-11T23:59:37.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.084+0000 7ff51318b740 1 -- 172.21.15.62:0/241630198 shutdown_connections 2024-09-11T23:59:37.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.084+0000 7ff51318b740 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fe2c647d0 0x557fe2c66c10 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.084+0000 7ff51318b740 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fe2bc8680 0x557fe2c3a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.084+0000 7ff51318b740 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fe2bbddc0 0x557fe2bdb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.084+0000 7ff51318b740 1 --2- 172.21.15.62:0/241630198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fe2bbd4a0 0x557fe2bdaac0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.084+0000 7ff51318b740 1 -- 172.21.15.62:0/241630198 >> 172.21.15.62:0/241630198 conn(0x557fe2bb9a30 msgr2=0x557fe2bbee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:37.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.086+0000 7ff4f1ffb640 0 client.16689 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:37.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.089+0000 7ff51318b740 1 -- 172.21.15.62:0/241630198 shutdown_connections 2024-09-11T23:59:37.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.091+0000 7ff51318b740 1 -- 172.21.15.62:0/241630198 wait complete. 2024-09-11T23:59:37.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.092+0000 7ff51318b740 20 client.16689 trim_cache size 0 max 16384 2024-09-11T23:59:37.095 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:59:37.095 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-11T23:59:37.096 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-11T23:59:37.096 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:59:37.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.096+0000 7ff510db2640 1 -- 172.21.15.62:0/4254346048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff4f80a2e40 msgr2=0x7ff4f80a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:37.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.096+0000 7ff510db2640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff4f80a2e40 0x7ff4f80a5300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff504000f80 tx=0x7ff5040054b0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.096+0000 7ff510db2640 1 -- 172.21.15.62:0/4254346048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c106a70 msgr2=0x7ff50c196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:37.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.096+0000 7ff510db2640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c106a70 0x7ff50c196660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff504009fd0 tx=0x7ff50400ef10 comp rx=0 tx=0).stop 2024-09-11T23:59:37.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.097+0000 7ff510db2640 1 -- 172.21.15.62:0/4254346048 shutdown_connections 2024-09-11T23:59:37.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.097+0000 7ff510db2640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff4f80a2e40 0x7ff4f80a5300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.097+0000 7ff510db2640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c106a70 0x7ff50c196660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.098+0000 7ff510db2640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101410 0x7ff50c196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.098+0000 7ff510db2640 1 --2- 172.21.15.62:0/4254346048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff50c100a70 0x7ff50c19d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:37.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.098+0000 7ff510db2640 1 -- 172.21.15.62:0/4254346048 >> 172.21.15.62:0/4254346048 conn(0x7ff50c0fc7d0 msgr2=0x7ff50c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:37.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.099+0000 7ff510db2640 1 -- 172.21.15.62:0/4254346048 shutdown_connections 2024-09-11T23:59:37.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:37.099+0000 7ff510db2640 1 -- 172.21.15.62:0/4254346048 wait complete. 2024-09-11T23:59:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 415.0 s, 2478.0 tps, lat 40.532 ms stddev 42.641 2024-09-11T23:59:37.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:37 smithi062 ceph-mon[30958]: pgmap v230: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-09-11T23:59:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:37 smithi120 ceph-mon[35182]: pgmap v230: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-09-11T23:59:37.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:37 smithi083 ceph-mon[34805]: pgmap v230: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-09-11T23:59:37.928 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:59:37.928 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:59:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:38.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.868+0000 7f76188a3640 1 -- 172.21.15.62:0/2005075588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614108980 msgr2=0x7f7614114ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:38.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.868+0000 7f76188a3640 1 --2- 172.21.15.62:0/2005075588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614108980 0x7f7614114ea0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f760c009cd0 tx=0x7f760c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:59:38.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.870+0000 7f76188a3640 1 -- 172.21.15.62:0/2005075588 shutdown_connections 2024-09-11T23:59:38.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.870+0000 7f76188a3640 1 --2- 172.21.15.62:0/2005075588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614108980 0x7f7614114ea0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:38.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.870+0000 7f76188a3640 1 --2- 172.21.15.62:0/2005075588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7614108020 0x7f7614108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:38.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.870+0000 7f76188a3640 1 --2- 172.21.15.62:0/2005075588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f761410e060 0x7f761410e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:38.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.870+0000 7f76188a3640 1 -- 172.21.15.62:0/2005075588 >> 172.21.15.62:0/2005075588 conn(0x7f7614077980 msgr2=0x7f7614079da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:38.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.870+0000 7f76188a3640 1 -- 172.21.15.62:0/2005075588 shutdown_connections 2024-09-11T23:59:38.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.870+0000 7f76188a3640 1 -- 172.21.15.62:0/2005075588 wait complete. 2024-09-11T23:59:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.870+0000 7f76188a3640 1 Processor -- start 2024-09-11T23:59:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f76188a3640 1 -- start start 2024-09-11T23:59:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f76188a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7614108020 0x7f76141a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f76188a3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614108980 0x7f76141a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f76188a3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f761410e060 0x7f761419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f76188a3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76141183c0 con 0x7f7614108980 2024-09-11T23:59:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f76137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7614108020 0x7f76141a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f76188a3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7614118240 con 0x7f7614108020 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f76188a3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7614118540 con 0x7f761410e060 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f76137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7614108020 0x7f76141a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:48010/0 (socket says 172.21.15.62:48010) 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f7612ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614108980 0x7f76141a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.871+0000 7f76137fe640 1 -- 172.21.15.62:0/3021291415 learned_addr learned my addr 172.21.15.62:0/3021291415 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.872+0000 7f7613fff640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f761410e060 0x7f761419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.872+0000 7f7612ffd640 1 -- 172.21.15.62:0/3021291415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f761410e060 msgr2=0x7f761419e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.872+0000 7f7612ffd640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f761410e060 0x7f761419e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.872+0000 7f7612ffd640 1 -- 172.21.15.62:0/3021291415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7614108020 msgr2=0x7f76141a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.872+0000 7f7612ffd640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7614108020 0x7f76141a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.872+0000 7f7612ffd640 1 -- 172.21.15.62:0/3021291415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761419eed0 con 0x7f7614108980 2024-09-11T23:59:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.872+0000 7f7612ffd640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614108980 0x7f76141a4ad0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f760800e9f0 tx=0x7f760800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.873+0000 7f7610ff9640 1 -- 172.21.15.62:0/3021291415 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760800ccc0 con 0x7f7614108980 2024-09-11T23:59:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.873+0000 7f7610ff9640 1 -- 172.21.15.62:0/3021291415 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7608010910 con 0x7f7614108980 2024-09-11T23:59:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.873+0000 7f7610ff9640 1 -- 172.21.15.62:0/3021291415 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608018960 con 0x7f7614108980 2024-09-11T23:59:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.873+0000 7f76188a3640 1 -- 172.21.15.62:0/3021291415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761419f1c0 con 0x7f7614108980 2024-09-11T23:59:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.873+0000 7f76188a3640 1 -- 172.21.15.62:0/3021291415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76141ab340 con 0x7f7614108980 2024-09-11T23:59:38.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.874+0000 7f76188a3640 1 -- 172.21.15.62:0/3021291415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7614109710 con 0x7f7614108980 2024-09-11T23:59:38.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.876+0000 7f7610ff9640 1 -- 172.21.15.62:0/3021291415 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7608010ac0 con 0x7f7614108980 2024-09-11T23:59:38.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.877+0000 7f7610ff9640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f75fc0a2e40 0x7f75fc0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:38.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.878+0000 7f7610ff9640 1 -- 172.21.15.62:0/3021291415 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f76080cfc00 con 0x7f7614108980 2024-09-11T23:59:38.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.878+0000 7f76137fe640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f75fc0a2e40 0x7f75fc0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:38.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.879+0000 7f76137fe640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f75fc0a2e40 0x7f75fc0a5300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7604009c10 tx=0x7f7604005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:38.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:38.882+0000 7f7610ff9640 1 -- 172.21.15.62:0/3021291415 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f760809c1f0 con 0x7f7614108980 2024-09-11T23:59:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.231+0000 7f76188a3640 1 -- 172.21.15.62:0/3021291415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7614108440 con 0x7f7614108980 2024-09-11T23:59:39.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.233+0000 7f7610ff9640 1 -- 172.21.15.62:0/3021291415 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11910 (secure 0 0 0) 0x7f7614108440 con 0x7f7614108980 2024-09-11T23:59:39.233 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:59:39.233 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-09-11T23:57:32:617369+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:57:32.075842+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:59:39.234 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.236+0000 7f76188a3640 1 -- 172.21.15.62:0/3021291415 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f75fc0a2e40 msgr2=0x7f75fc0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.236+0000 7f76188a3640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f75fc0a2e40 0x7f75fc0a5300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7604009c10 tx=0x7f7604005f90 comp rx=0 tx=0).stop 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.236+0000 7f76188a3640 1 -- 172.21.15.62:0/3021291415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614108980 msgr2=0x7f76141a4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.236+0000 7f76188a3640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614108980 0x7f76141a4ad0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f760800e9f0 tx=0x7f760800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.237+0000 7f76188a3640 1 -- 172.21.15.62:0/3021291415 shutdown_connections 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.237+0000 7f76188a3640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f75fc0a2e40 0x7f75fc0a5300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.237+0000 7f76188a3640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f761410e060 0x7f761419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.237+0000 7f76188a3640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614108980 0x7f76141a4ad0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.237+0000 7f76188a3640 1 --2- 172.21.15.62:0/3021291415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7614108020 0x7f76141a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.237+0000 7f76188a3640 1 -- 172.21.15.62:0/3021291415 >> 172.21.15.62:0/3021291415 conn(0x7f7614077980 msgr2=0x7f76140791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.237+0000 7f76188a3640 1 -- 172.21.15.62:0/3021291415 shutdown_connections 2024-09-11T23:59:39.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:39.237+0000 7f76188a3640 1 -- 172.21.15.62:0/3021291415 wait complete. 2024-09-11T23:59:39.563 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:39 smithi062 ceph-mon[30958]: pgmap v231: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 761 op/s 2024-09-11T23:59:39.563 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:39 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3021291415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:39.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:39 smithi120 ceph-mon[35182]: pgmap v231: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 761 op/s 2024-09-11T23:59:39.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:39 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3021291415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:39.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:39 smithi083 ceph-mon[34805]: pgmap v231: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 761 op/s 2024-09-11T23:59:39.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:39 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3021291415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:40.394 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:59:40.778 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:41.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:41 smithi062 ceph-mon[30958]: pgmap v232: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.6 MiB/s wr, 656 op/s 2024-09-11T23:59:41.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:41 smithi120 ceph-mon[35182]: pgmap v232: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.6 MiB/s wr, 656 op/s 2024-09-11T23:59:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.622+0000 7f984e86c640 1 -- 172.21.15.62:0/159500978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9848103ff0 msgr2=0x7f98481043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.622+0000 7f984e86c640 1 --2- 172.21.15.62:0/159500978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9848103ff0 0x7f98481043d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9838009900 tx=0x7f983802a410 comp rx=0 tx=0).stop 2024-09-11T23:59:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.622+0000 7f984e86c640 1 -- 172.21.15.62:0/159500978 shutdown_connections 2024-09-11T23:59:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.622+0000 7f984e86c640 1 --2- 172.21.15.62:0/159500978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9848109120 0x7f9848110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.622+0000 7f984e86c640 1 --2- 172.21.15.62:0/159500978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98481049a0 0x7f98481089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.622+0000 7f984e86c640 1 --2- 172.21.15.62:0/159500978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9848103ff0 0x7f98481043d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.622+0000 7f984e86c640 1 -- 172.21.15.62:0/159500978 >> 172.21.15.62:0/159500978 conn(0x7f98480ffa70 msgr2=0x7f9848101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:41.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.622+0000 7f984e86c640 1 -- 172.21.15.62:0/159500978 shutdown_connections 2024-09-11T23:59:41.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.622+0000 7f984e86c640 1 -- 172.21.15.62:0/159500978 wait complete. 2024-09-11T23:59:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.623+0000 7f984e86c640 1 Processor -- start 2024-09-11T23:59:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.623+0000 7f984e86c640 1 -- start start 2024-09-11T23:59:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.623+0000 7f984e86c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9848103ff0 0x7f984819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.623+0000 7f984e86c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98481049a0 0x7f984819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.623+0000 7f984e86c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9848109120 0x7f984819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.623+0000 7f984e86c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9848114150 con 0x7f9848103ff0 2024-09-11T23:59:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.623+0000 7f984e86c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9848113fd0 con 0x7f98481049a0 2024-09-11T23:59:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.623+0000 7f984e86c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98481142d0 con 0x7f9848109120 2024-09-11T23:59:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f9847fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9848103ff0 0x7f984819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f98477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98481049a0 0x7f984819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f98477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98481049a0 0x7f984819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58646/0 (socket says 172.21.15.62:58646) 2024-09-11T23:59:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f98477fe640 1 -- 172.21.15.62:0/3476118998 learned_addr learned my addr 172.21.15.62:0/3476118998 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f984ce09640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9848109120 0x7f984819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f984ce09640 1 -- 172.21.15.62:0/3476118998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98481049a0 msgr2=0x7f984819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f984ce09640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98481049a0 0x7f984819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f984ce09640 1 -- 172.21.15.62:0/3476118998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9848103ff0 msgr2=0x7f984819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:41.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f984ce09640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9848103ff0 0x7f984819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f984ce09640 1 -- 172.21.15.62:0/3476118998 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984819f5a0 con 0x7f9848109120 2024-09-11T23:59:41.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.624+0000 7f98477fe640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98481049a0 0x7f984819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.626+0000 7f984ce09640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9848109120 0x7f984819eec0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9840009fd0 tx=0x7f984000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:41.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.626+0000 7f98457fa640 1 -- 172.21.15.62:0/3476118998 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984000ce20 con 0x7f9848109120 2024-09-11T23:59:41.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.626+0000 7f98457fa640 1 -- 172.21.15.62:0/3476118998 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9840005020 con 0x7f9848109120 2024-09-11T23:59:41.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.626+0000 7f98457fa640 1 -- 172.21.15.62:0/3476118998 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840010720 con 0x7f9848109120 2024-09-11T23:59:41.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.627+0000 7f984e86c640 1 -- 172.21.15.62:0/3476118998 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984819f890 con 0x7f9848109120 2024-09-11T23:59:41.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.627+0000 7f984e86c640 1 -- 172.21.15.62:0/3476118998 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98481a7170 con 0x7f9848109120 2024-09-11T23:59:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.629+0000 7f98457fa640 1 -- 172.21.15.62:0/3476118998 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9840005540 con 0x7f9848109120 2024-09-11T23:59:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.630+0000 7f98457fa640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f98300a2e40 0x7f98300a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.630+0000 7f98457fa640 1 -- 172.21.15.62:0/3476118998 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f98400cf930 con 0x7f9848109120 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.630+0000 7f9847fff640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f98300a2e40 0x7f98300a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.630+0000 7f9850c45740 1 Processor -- start 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.630+0000 7f9850c45740 1 -- start start 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f9850c45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f9850c45740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b65d0 0x56148b5b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f9850c45740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 0x56148b5cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f9850c45740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56148b51c2f0 con 0x56148b5b65d0 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f9850c45740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56148b4c3cf0 con 0x56148b5c9b80 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f9850c45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56148b5182b0 con 0x56148b5b7b40 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f9847fff640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f98300a2e40 0x7f98300a5300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838009620 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f984ce09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 0x56148b5cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f98477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b65d0 0x56148b5b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f984ce09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 0x56148b5cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58660/0 (socket says 172.21.15.62:58660) 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f984ce09640 1 -- 172.21.15.62:0/412632701 learned_addr learned my addr 172.21.15.62:0/412632701 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.631+0000 7f9847fff640 1 --2- 172.21.15.62:0/412632701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.632+0000 7f9847fff640 1 -- 172.21.15.62:0/412632701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 msgr2=0x56148b5cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.632+0000 7f9847fff640 1 --2- 172.21.15.62:0/412632701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 0x56148b5cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.632+0000 7f9847fff640 1 -- 172.21.15.62:0/412632701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b65d0 msgr2=0x56148b5b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.632+0000 7f9847fff640 1 --2- 172.21.15.62:0/412632701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b65d0 0x56148b5b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.632+0000 7f9847fff640 1 -- 172.21.15.62:0/412632701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148b4a83a0 con 0x56148b5b7b40 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.632+0000 7f984ce09640 1 --2- 172.21.15.62:0/412632701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 0x56148b5cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.632+0000 7f9847fff640 1 --2- 172.21.15.62:0/412632701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5b6090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9838009230 tx=0x7f983803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46632c69986621cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.632+0000 7f982e7fc640 1 -- 172.21.15.62:0/412632701 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838035020 con 0x56148b5b7b40 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.632+0000 7f982e7fc640 1 -- 172.21.15.62:0/412632701 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98380317a0 con 0x56148b5b7b40 2024-09-11T23:59:41.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.632+0000 7f982e7fc640 1 -- 172.21.15.62:0/412632701 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838004a70 con 0x56148b5b7b40 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.633+0000 7f9850c45740 1 -- 172.21.15.62:0/412632701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 msgr2=0x56148b5b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.633+0000 7f9850c45740 1 --2- 172.21.15.62:0/412632701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5b6090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9838009230 tx=0x7f983803f000 comp rx=0 tx=0).stop 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.633+0000 7f9850c45740 1 -- 172.21.15.62:0/412632701 shutdown_connections 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.633+0000 7f9850c45740 1 --2- 172.21.15.62:0/412632701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 0x56148b5cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.633+0000 7f9850c45740 1 --2- 172.21.15.62:0/412632701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b65d0 0x56148b5b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.633+0000 7f9850c45740 1 --2- 172.21.15.62:0/412632701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5b6090 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.633+0000 7f9850c45740 1 -- 172.21.15.62:0/412632701 >> 172.21.15.62:0/412632701 conn(0x56148b57f310 msgr2=0x56148b5b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.633+0000 7f9850c45740 1 -- 172.21.15.62:0/412632701 shutdown_connections 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.633+0000 7f9850c45740 1 -- 172.21.15.62:0/412632701 wait complete. 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.634+0000 7f9850c45740 1 -- start start 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.634+0000 7f9850c45740 1 Processor -- start 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.634+0000 7f9850c45740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b65d0 0x56148b5c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.634+0000 7f9850c45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.634+0000 7f9850c45740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 0x56148b5c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.634+0000 7f9850c45740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56148b5cc820 con 0x56148b5b65d0 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.634+0000 7f9850c45740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56148b4d7060 con 0x56148b5c9b80 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.634+0000 7f9850c45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56148b5cc9a0 con 0x56148b5b7b40 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f9850c45740 10 client.0 _ll_register_callbacks cb 0x56148b22a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98477fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f9847fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b65d0 0x56148b5c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98477fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:55110/0 (socket says 172.21.15.62:55110) 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98477fe640 1 -- 172.21.15.62:0/449436550 learned_addr learned my addr 172.21.15.62:0/449436550 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f984ce09640 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 0x56148b5c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98477fe640 1 -- 172.21.15.62:0/449436550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 msgr2=0x56148b5c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98477fe640 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 0x56148b5c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98477fe640 1 -- 172.21.15.62:0/449436550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b65d0 msgr2=0x56148b5c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98477fe640 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b65d0 0x56148b5c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98477fe640 1 -- 172.21.15.62:0/449436550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148b28d3c0 con 0x56148b5b7b40 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98477fe640 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5c67b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f983c012d20 tx=0x7f983c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9332ccfc8b96d905 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98277fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98277fe640 1 -- 172.21.15.62:0/449436550 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c00fe20 con 0x56148b5b7b40 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f9850c45740 1 -- 172.21.15.62:0/449436550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98380317a0 con 0x56148b5b7b40 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.635+0000 7f98277fe640 1 -- 172.21.15.62:0/449436550 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f983c004050 con 0x56148b5b7b40 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f98277fe640 1 -- 172.21.15.62:0/449436550 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f983c004250 con 0x56148b5b7b40 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f9850c45740 10 client.26260 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f9850c45740 1 -- 172.21.15.62:0/449436550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56148b5d82b0 con 0x56148b5b7b40 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f98277fe640 1 -- 172.21.15.62:0/449436550 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98380317a0 con 0x56148b5b7b40 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f98277fe640 1 -- 172.21.15.62:0/449436550 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f983c025640 con 0x56148b5b7b40 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f9850c45740 10 client.26260 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:41.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f9850c45740 10 client.26260 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:59:41.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f9850c45740 20 client.26260 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:41.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f9850c45740 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56148b5d96c0 0x56148b5dbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f9850c45740 4 client.26260 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:41.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f9850c45740 1 -- 172.21.15.62:0/449436550 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56148b529860 con 0x56148b5d96c0 2024-09-11T23:59:41.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.636+0000 7f9847fff640 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56148b5d96c0 0x56148b5dbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.637+0000 7f9847fff640 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56148b5d96c0 0x56148b5dbb20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d3cb5a5e481c9cf server_cookie=6ca6a2c6ad785892 in_seq=0 out_seq=0 2024-09-11T23:59:41.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.637+0000 7f98277fe640 10 client.26260 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:59:41.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f98277fe640 1 -- 172.21.15.62:0/449436550 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56148b529860 con 0x56148b5d96c0 2024-09-11T23:59:41.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f98277fe640 10 client.26260 handle_command_reply: tid=0 2024-09-11T23:59:41.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f9850c45740 1 client.26260 shutdown 2024-09-11T23:59:41.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f9850c45740 1 -- 172.21.15.62:0/449436550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 msgr2=0x56148b5c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:41.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f9850c45740 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5c67b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f983c012d20 tx=0x7f983c00f790 comp rx=0 tx=0).stop 2024-09-11T23:59:41.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f9850c45740 1 -- 172.21.15.62:0/449436550 shutdown_connections 2024-09-11T23:59:41.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f9850c45740 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56148b5d96c0 0x56148b5dbb20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f9850c45740 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5c9b80 0x56148b5c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f9850c45740 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5b7b40 0x56148b5c67b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f9850c45740 1 --2- 172.21.15.62:0/449436550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b65d0 0x56148b5c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f9850c45740 1 -- 172.21.15.62:0/449436550 >> 172.21.15.62:0/449436550 conn(0x56148b57f310 msgr2=0x56148b5b8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.638+0000 7f98277fe640 0 client.26260 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:59:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.639+0000 7f9850c45740 1 -- 172.21.15.62:0/449436550 shutdown_connections 2024-09-11T23:59:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.639+0000 7f9850c45740 1 -- 172.21.15.62:0/449436550 wait complete. 2024-09-11T23:59:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.641+0000 7f9850c45740 20 client.26260 trim_cache size 0 max 16384 2024-09-11T23:59:41.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.662+0000 7f9850c45740 1 Processor -- start 2024-09-11T23:59:41.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.662+0000 7f9850c45740 1 -- start start 2024-09-11T23:59:41.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.662+0000 7f9850c45740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 0x56148b5d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.662+0000 7f9850c45740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 0x56148b5db6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.663+0000 7f9847fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 0x56148b5d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.663+0000 7f98477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 0x56148b5db6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.663+0000 7f9850c45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5c9b80 0x56148b5c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.663+0000 7f9850c45740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f98481adf30 con 0x56148b5b7b40 2024-09-11T23:59:41.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.663+0000 7f9850c45740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98481ab7d0 con 0x56148b5b65d0 2024-09-11T23:59:41.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.663+0000 7f9850c45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98481a9640 con 0x56148b5c9b80 2024-09-11T23:59:41.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.663+0000 7f98477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 0x56148b5db6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39428/0 (socket says 172.21.15.62:39428) 2024-09-11T23:59:41.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.663+0000 7f98477fe640 1 -- 172.21.15.62:0/1254369175 learned_addr learned my addr 172.21.15.62:0/1254369175 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:41.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.663+0000 7f984ce09640 1 --2- 172.21.15.62:0/1254369175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5c9b80 0x56148b5c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.664+0000 7f98477fe640 1 -- 172.21.15.62:0/1254369175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5c9b80 msgr2=0x56148b5c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:41.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.664+0000 7f98477fe640 1 --2- 172.21.15.62:0/1254369175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5c9b80 0x56148b5c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.664+0000 7f98477fe640 1 -- 172.21.15.62:0/1254369175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 msgr2=0x56148b5d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:41.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.664+0000 7f98477fe640 1 --2- 172.21.15.62:0/1254369175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 0x56148b5d5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.664+0000 7f98477fe640 1 -- 172.21.15.62:0/1254369175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148b5c9870 con 0x56148b5b7b40 2024-09-11T23:59:41.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.664+0000 7f98477fe640 1 --2- 172.21.15.62:0/1254369175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 0x56148b5db6d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f983c012870 tx=0x7f983c012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07d26cd0f0ae07b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:41.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.665+0000 7f982dffb640 1 -- 172.21.15.62:0/1254369175 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c025480 con 0x56148b5b7b40 2024-09-11T23:59:41.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.665+0000 7f982dffb640 1 -- 172.21.15.62:0/1254369175 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c005520 con 0x56148b5b7b40 2024-09-11T23:59:41.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.665+0000 7f982dffb640 1 -- 172.21.15.62:0/1254369175 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c010890 con 0x56148b5b7b40 2024-09-11T23:59:41.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.666+0000 7f9850c45740 1 -- 172.21.15.62:0/1254369175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 msgr2=0x56148b5db6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.666+0000 7f9850c45740 1 --2- 172.21.15.62:0/1254369175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 0x56148b5db6d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f983c012870 tx=0x7f983c012910 comp rx=0 tx=0).stop 2024-09-11T23:59:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.667+0000 7f9850c45740 1 -- 172.21.15.62:0/1254369175 shutdown_connections 2024-09-11T23:59:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.667+0000 7f9850c45740 1 --2- 172.21.15.62:0/1254369175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5c9b80 0x56148b5c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.667+0000 7f9850c45740 1 --2- 172.21.15.62:0/1254369175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 0x56148b5db6d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.667+0000 7f9850c45740 1 --2- 172.21.15.62:0/1254369175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 0x56148b5d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.667+0000 7f9850c45740 1 -- 172.21.15.62:0/1254369175 >> 172.21.15.62:0/1254369175 conn(0x56148b57f310 msgr2=0x56148b5c5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.667+0000 7f9850c45740 1 -- 172.21.15.62:0/1254369175 shutdown_connections 2024-09-11T23:59:41.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.667+0000 7f9850c45740 1 -- 172.21.15.62:0/1254369175 wait complete. 2024-09-11T23:59:41.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.667+0000 7f9850c45740 1 -- start start 2024-09-11T23:59:41.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.667+0000 7f9850c45740 1 Processor -- start 2024-09-11T23:59:41.670 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:41 smithi083 ceph-mon[34805]: pgmap v232: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.6 MiB/s wr, 656 op/s 2024-09-11T23:59:41.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9850c45740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 0x56148b5d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9850c45740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 0x56148b5cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9850c45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5c9b80 0x56148b5cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9850c45740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f98481a9c90 con 0x56148b5b7b40 2024-09-11T23:59:41.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9850c45740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98481ab100 con 0x56148b5b65d0 2024-09-11T23:59:41.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9850c45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98481aa360 con 0x56148b5c9b80 2024-09-11T23:59:41.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9850c45740 10 client.0 _ll_register_callbacks cb 0x7f983c009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:41.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9847fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 0x56148b5d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9847fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 0x56148b5d22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58698/0 (socket says 172.21.15.62:58698) 2024-09-11T23:59:41.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9847fff640 1 -- 172.21.15.62:0/105051705 learned_addr learned my addr 172.21.15.62:0/105051705 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:41.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f98477fe640 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 0x56148b5cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f984ce09640 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5c9b80 0x56148b5cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9847fff640 1 -- 172.21.15.62:0/105051705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5c9b80 msgr2=0x56148b5cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:41.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9847fff640 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5c9b80 0x56148b5cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9847fff640 1 -- 172.21.15.62:0/105051705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 msgr2=0x56148b5cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:41.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9847fff640 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 0x56148b5cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.668+0000 7f9847fff640 1 -- 172.21.15.62:0/105051705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148b5ce390 con 0x56148b5b65d0 2024-09-11T23:59:41.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f9847fff640 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 0x56148b5d22a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f983802a450 tx=0x7f9838031600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcf865d1bbebba53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:41.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f98277fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:59:41.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f98277fe640 1 -- 172.21.15.62:0/105051705 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838035040 con 0x56148b5b65d0 2024-09-11T23:59:41.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f98277fe640 1 -- 172.21.15.62:0/105051705 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f9838031bb0 con 0x56148b5b65d0 2024-09-11T23:59:41.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f9850c45740 1 -- 172.21.15.62:0/105051705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983c005520 con 0x56148b5b65d0 2024-09-11T23:59:41.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f98277fe640 1 -- 172.21.15.62:0/105051705 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f983806e050 con 0x56148b5b65d0 2024-09-11T23:59:41.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f9850c45740 10 client.26306 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:41.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f9850c45740 1 -- 172.21.15.62:0/105051705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56148b6442f0 con 0x56148b5b65d0 2024-09-11T23:59:41.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f98277fe640 1 -- 172.21.15.62:0/105051705 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c005520 con 0x56148b5b65d0 2024-09-11T23:59:41.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f98277fe640 1 -- 172.21.15.62:0/105051705 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f983803a040 con 0x56148b5b65d0 2024-09-11T23:59:41.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f9850c45740 10 client.26306 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:41.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f9850c45740 10 client.26306 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T23:59:41.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f9850c45740 20 client.26306 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.669+0000 7f9850c45740 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56148b6445d0 0x56148b6469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.670+0000 7f9850c45740 4 client.26306 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:59:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.670+0000 7f9850c45740 1 -- 172.21.15.62:0/105051705 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56148b523d70 con 0x56148b6445d0 2024-09-11T23:59:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.670+0000 7f98477fe640 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56148b6445d0 0x56148b6469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.670+0000 7f98277fe640 10 client.26306 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-11T23:59:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f98477fe640 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56148b6445d0 0x56148b6469c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a05f21478989e6dc server_cookie=c3ee5ef59d826cee in_seq=0 out_seq=0 2024-09-11T23:59:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f98277fe640 1 -- 172.21.15.62:0/105051705 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56148b523d70 con 0x56148b6445d0 2024-09-11T23:59:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f98277fe640 10 client.26306 handle_command_reply: tid=0 2024-09-11T23:59:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f9850c45740 1 client.26306 shutdown 2024-09-11T23:59:41.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f9850c45740 1 -- 172.21.15.62:0/105051705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 msgr2=0x56148b5d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:41.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f9850c45740 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 0x56148b5d22a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f983802a450 tx=0x7f9838031600 comp rx=0 tx=0).stop 2024-09-11T23:59:41.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f9850c45740 1 -- 172.21.15.62:0/105051705 shutdown_connections 2024-09-11T23:59:41.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f9850c45740 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56148b6445d0 0x56148b6469c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f98277fe640 0 client.26306 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-11T23:59:41.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f9850c45740 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56148b5c9b80 0x56148b5cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f9850c45740 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148b5b7b40 0x56148b5cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f9850c45740 1 --2- 172.21.15.62:0/105051705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56148b5b65d0 0x56148b5d22a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.671+0000 7f9850c45740 1 -- 172.21.15.62:0/105051705 >> 172.21.15.62:0/105051705 conn(0x56148b57f310 msgr2=0x56148b5c58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:41.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.672+0000 7f9850c45740 1 -- 172.21.15.62:0/105051705 shutdown_connections 2024-09-11T23:59:41.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.672+0000 7f9850c45740 1 -- 172.21.15.62:0/105051705 wait complete. 2024-09-11T23:59:41.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.672+0000 7f9850c45740 20 client.26306 trim_cache size 0 max 16384 2024-09-11T23:59:41.681 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:59:41.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.672+0000 7f984e86c640 1 -- 172.21.15.62:0/3476118998 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f98300a2e40 msgr2=0x7f98300a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:41.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.672+0000 7f984e86c640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f98300a2e40 0x7f98300a5300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838009620 comp rx=0 tx=0).stop 2024-09-11T23:59:41.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.672+0000 7f984e86c640 1 -- 172.21.15.62:0/3476118998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9848109120 msgr2=0x7f984819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:41.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.672+0000 7f984e86c640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9848109120 0x7f984819eec0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9840009fd0 tx=0x7f984000ef10 comp rx=0 tx=0).stop 2024-09-11T23:59:41.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.673+0000 7f984e86c640 1 -- 172.21.15.62:0/3476118998 shutdown_connections 2024-09-11T23:59:41.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.673+0000 7f984e86c640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f98300a2e40 0x7f98300a5300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.673+0000 7f984e86c640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9848109120 0x7f984819eec0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.673+0000 7f984e86c640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98481049a0 0x7f984819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.673+0000 7f984e86c640 1 --2- 172.21.15.62:0/3476118998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9848103ff0 0x7f984819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:41.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.673+0000 7f984e86c640 1 -- 172.21.15.62:0/3476118998 >> 172.21.15.62:0/3476118998 conn(0x7f98480ffa70 msgr2=0x7f9848100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:41.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.673+0000 7f984e86c640 1 -- 172.21.15.62:0/3476118998 shutdown_connections 2024-09-11T23:59:41.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:41.673+0000 7f984e86c640 1 -- 172.21.15.62:0/3476118998 wait complete. 2024-09-11T23:59:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 420.0 s, 2699.0 tps, lat 36.808 ms stddev 41.805 2024-09-11T23:59:42.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:59:43.289 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:43.569 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:43 smithi062 ceph-mon[30958]: pgmap v233: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 639 op/s 2024-09-11T23:59:43.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:43 smithi120 ceph-mon[35182]: pgmap v233: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 639 op/s 2024-09-11T23:59:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:43 smithi083 ceph-mon[34805]: pgmap v233: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 639 op/s 2024-09-11T23:59:43.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.828+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1591090247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4aac109450 msgr2=0x7f4aac109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:43.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.828+0000 7f4ab32ab640 1 --2- 172.21.15.62:0/1591090247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4aac109450 0x7f4aac109830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009900 tx=0x7f4a9c02a440 comp rx=0 tx=0).stop 2024-09-11T23:59:43.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.829+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1591090247 shutdown_connections 2024-09-11T23:59:43.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.829+0000 7f4ab32ab640 1 --2- 172.21.15.62:0/1591090247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4aac074990 0x7f4aac07b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.829+0000 7f4ab32ab640 1 --2- 172.21.15.62:0/1591090247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac109e00 0x7f4aac0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.829+0000 7f4ab32ab640 1 --2- 172.21.15.62:0/1591090247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4aac109450 0x7f4aac109830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.829+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1591090247 >> 172.21.15.62:0/1591090247 conn(0x7f4aac06e140 msgr2=0x7f4aac06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:43.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.842+0000 7f4ab0846640 1 --2- 172.21.15.62:0/1591090247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac109e00 0x7f4aac0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:43.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.843+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1591090247 shutdown_connections 2024-09-11T23:59:43.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.843+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1591090247 wait complete. 2024-09-11T23:59:43.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.843+0000 7f4ab32ab640 1 Processor -- start 2024-09-11T23:59:43.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.844+0000 7f4ab32ab640 1 -- start start 2024-09-11T23:59:43.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.844+0000 7f4ab32ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4aac074990 0x7f4aac07a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.844+0000 7f4ab32ab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac109450 0x7f4aac07ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.844+0000 7f4ab32ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4aac1b8770 0x7f4aac1b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.844+0000 7f4ab32ab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac07e810 con 0x7f4aac109450 2024-09-11T23:59:43.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.844+0000 7f4ab32ab640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac07e690 con 0x7f4aac1b8770 2024-09-11T23:59:43.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.844+0000 7f4ab32ab640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac07e990 con 0x7f4aac074990 2024-09-11T23:59:43.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.844+0000 7f4ab0846640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac109450 0x7f4aac07ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.844+0000 7f4ab0846640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac109450 0x7f4aac07ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39450/0 (socket says 172.21.15.62:39450) 2024-09-11T23:59:43.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.844+0000 7f4ab0846640 1 -- 172.21.15.62:0/1914681580 learned_addr learned my addr 172.21.15.62:0/1914681580 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:43.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.845+0000 7f4ab1848640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4aac1b8770 0x7f4aac1b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.846+0000 7f4ab1047640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4aac074990 0x7f4aac07a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.849+0000 7f4ab1848640 1 -- 172.21.15.62:0/1914681580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4aac074990 msgr2=0x7f4aac07a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.849+0000 7f4ab1848640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4aac074990 0x7f4aac07a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.849+0000 7f4ab1848640 1 -- 172.21.15.62:0/1914681580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac109450 msgr2=0x7f4aac07ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:43.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.849+0000 7f4ab1848640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac109450 0x7f4aac07ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.849+0000 7f4ab1848640 1 -- 172.21.15.62:0/1914681580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac1b9230 con 0x7f4aac1b8770 2024-09-11T23:59:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.849+0000 7f4ab1848640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4aac1b8770 0x7f4aac1b8c20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800e970 tx=0x7f4aa800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.850+0000 7f4aa27fc640 1 -- 172.21.15.62:0/1914681580 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800ccc0 con 0x7f4aac1b8770 2024-09-11T23:59:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.850+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1914681580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aac1b9520 con 0x7f4aac1b8770 2024-09-11T23:59:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.850+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1914681580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aac07fe20 con 0x7f4aac1b8770 2024-09-11T23:59:43.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.850+0000 7f4aa27fc640 1 -- 172.21.15.62:0/1914681580 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa8005020 con 0x7f4aac1b8770 2024-09-11T23:59:43.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.851+0000 7f4aa27fc640 1 -- 172.21.15.62:0/1914681580 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8004a00 con 0x7f4aac1b8770 2024-09-11T23:59:43.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.854+0000 7f4ab5684740 1 Processor -- start 2024-09-11T23:59:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.855+0000 7f4aa27fc640 1 -- 172.21.15.62:0/1914681580 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4aa8004ba0 con 0x7f4aac1b8770 2024-09-11T23:59:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.855+0000 7f4ab5684740 1 -- start start 2024-09-11T23:59:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.855+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f3844090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.856+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 0x55a1f3844a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.856+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f384f370 0x55a1f3851760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.856+0000 7f4ab5684740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a1f37aa2f0 con 0x55a1f384f370 2024-09-11T23:59:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.856+0000 7f4ab5684740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1f3751cf0 con 0x55a1f3845b40 2024-09-11T23:59:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.856+0000 7f4ab5684740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1f37a62b0 con 0x55a1f38445d0 2024-09-11T23:59:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.856+0000 7f4ab1047640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 0x55a1f3844a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.857+0000 7f4ab1047640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 0x55a1f3844a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:55152/0 (socket says 172.21.15.62:55152) 2024-09-11T23:59:43.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.857+0000 7f4ab1047640 1 -- 172.21.15.62:0/4111221484 learned_addr learned my addr 172.21.15.62:0/4111221484 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:43.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.857+0000 7f4aa27fc640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a980a2e90 0x7f4a980a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.857+0000 7f4aa27fc640 1 -- 172.21.15.62:0/1914681580 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f4aa80cf120 con 0x7f4aac1b8770 2024-09-11T23:59:43.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.857+0000 7f4ab0846640 1 --2- 172.21.15.62:0/4111221484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f384f370 0x55a1f3851760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.857+0000 7f4ab0846640 1 --2- 172.21.15.62:0/4111221484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f3844090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.860+0000 7f4ab1047640 1 -- 172.21.15.62:0/4111221484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 msgr2=0x55a1f3844090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.860+0000 7f4ab1047640 1 --2- 172.21.15.62:0/4111221484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f3844090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.860+0000 7f4ab1047640 1 -- 172.21.15.62:0/4111221484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f384f370 msgr2=0x55a1f3851760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.860+0000 7f4ab1047640 1 --2- 172.21.15.62:0/4111221484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f384f370 0x55a1f3851760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.860+0000 7f4ab1047640 1 -- 172.21.15.62:0/4111221484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f37363a0 con 0x55a1f38445d0 2024-09-11T23:59:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.860+0000 7f4ab1848640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a980a2e90 0x7f4a980a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.862+0000 7f4ab1848640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a980a2e90 0x7f4a980a5350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8097aa0 tx=0x7f4aa80046c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:43.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.862+0000 7f4ab1047640 1 --2- 172.21.15.62:0/4111221484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 0x55a1f3844a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c004fd0 tx=0x7f4a9c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1998b59c8b13f0a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:43.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.863+0000 7f4a977fe640 1 -- 172.21.15.62:0/4111221484 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0048c0 con 0x55a1f38445d0 2024-09-11T23:59:43.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.863+0000 7f4a977fe640 1 -- 172.21.15.62:0/4111221484 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a9c0054e0 con 0x55a1f38445d0 2024-09-11T23:59:43.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.864+0000 7f4ab5684740 1 -- 172.21.15.62:0/4111221484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 msgr2=0x55a1f3844a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:43.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.864+0000 7f4ab5684740 1 --2- 172.21.15.62:0/4111221484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 0x55a1f3844a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c004fd0 tx=0x7f4a9c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.864+0000 7f4ab5684740 1 -- 172.21.15.62:0/4111221484 shutdown_connections 2024-09-11T23:59:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.864+0000 7f4ab5684740 1 --2- 172.21.15.62:0/4111221484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f384f370 0x55a1f3851760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.864+0000 7f4ab5684740 1 --2- 172.21.15.62:0/4111221484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 0x55a1f3844a10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.864+0000 7f4ab5684740 1 --2- 172.21.15.62:0/4111221484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f3844090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.864+0000 7f4ab5684740 1 -- 172.21.15.62:0/4111221484 >> 172.21.15.62:0/4111221484 conn(0x55a1f380d310 msgr2=0x55a1f3840c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:43.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.866+0000 7f4ab5684740 1 -- 172.21.15.62:0/4111221484 shutdown_connections 2024-09-11T23:59:43.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.866+0000 7f4ab5684740 1 -- 172.21.15.62:0/4111221484 wait complete. 2024-09-11T23:59:43.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.867+0000 7f4ab5684740 1 -- start start 2024-09-11T23:59:43.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.867+0000 7f4ab5684740 1 Processor -- start 2024-09-11T23:59:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.868+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 0x55a1f384ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.868+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f384beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.868+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 0x55a1f384c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.868+0000 7f4ab0846640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 0x55a1f384ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.868+0000 7f4ab0846640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 0x55a1f384ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39470/0 (socket says 172.21.15.62:39470) 2024-09-11T23:59:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.868+0000 7f4ab0846640 1 -- 172.21.15.62:0/4173156635 learned_addr learned my addr 172.21.15.62:0/4173156635 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.868+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a1f3863070 con 0x55a1f38445d0 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.869+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1f3765060 con 0x55a1f3845b40 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.869+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1f38631f0 con 0x55a1f384f370 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.869+0000 7f4ab5684740 10 client.0 _ll_register_callbacks cb 0x55a1f34b88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.869+0000 7f4ab1047640 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f384beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.869+0000 7f4ab0846640 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 0x55a1f384c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.869+0000 7f4ab1047640 1 -- 172.21.15.62:0/4173156635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 msgr2=0x55a1f384c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.870+0000 7f4ab1047640 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 0x55a1f384c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.870+0000 7f4ab1047640 1 -- 172.21.15.62:0/4173156635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 msgr2=0x55a1f384ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.870+0000 7f4ab1047640 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 0x55a1f384ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.870+0000 7f4ab1047640 1 -- 172.21.15.62:0/4173156635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f351b3c0 con 0x55a1f3845b40 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.870+0000 7f4ab0846640 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 0x55a1f384c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.870+0000 7f4ab1047640 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f384beb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009900 tx=0x7f4a9c005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd3e5b431cbdc1ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.870+0000 7f4a94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:59:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.870+0000 7f4a94ff9640 1 -- 172.21.15.62:0/4173156635 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c04b040 con 0x55a1f3845b40 2024-09-11T23:59:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.870+0000 7f4a94ff9640 1 -- 172.21.15.62:0/4173156635 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f4a9c005000 con 0x55a1f3845b40 2024-09-11T23:59:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.871+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a9c0054e0 con 0x55a1f3845b40 2024-09-11T23:59:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.871+0000 7f4a94ff9640 1 -- 172.21.15.62:0/4173156635 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a9c009520 con 0x55a1f3845b40 2024-09-11T23:59:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.871+0000 7f4ab5684740 10 client.26318 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.871+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1f386eaa0 con 0x55a1f3845b40 2024-09-11T23:59:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.873+0000 7f4a94ff9640 1 -- 172.21.15.62:0/4173156635 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a9c005200 con 0x55a1f3845b40 2024-09-11T23:59:43.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.874+0000 7f4a94ff9640 1 -- 172.21.15.62:0/4173156635 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f4a9c04e070 con 0x55a1f3845b40 2024-09-11T23:59:43.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.874+0000 7f4ab5684740 10 client.26318 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.874+0000 7f4ab5684740 10 client.26318 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:59:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.874+0000 7f4ab5684740 20 client.26318 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.875+0000 7f4ab5684740 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a1f386ffa0 0x55a1f38723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.875+0000 7f4ab5684740 4 client.26318 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.875+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4a9c0054e0 con 0x55a1f386ffa0 2024-09-11T23:59:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.875+0000 7f4ab0846640 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a1f386ffa0 0x55a1f38723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.878+0000 7f4ab0846640 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a1f386ffa0 0x55a1f38723c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f1d7e7ab640f921 server_cookie=e27a6c369cc6e5b3 in_seq=0 out_seq=0 2024-09-11T23:59:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.878+0000 7f4a94ff9640 10 client.26318 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:59:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.879+0000 7f4a94ff9640 1 -- 172.21.15.62:0/4173156635 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4a9c0054e0 con 0x55a1f386ffa0 2024-09-11T23:59:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.879+0000 7f4a94ff9640 10 client.26318 handle_command_reply: tid=0 2024-09-11T23:59:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.879+0000 7f4ab5684740 1 client.26318 shutdown 2024-09-11T23:59:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 msgr2=0x55a1f384beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4ab5684740 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f384beb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009900 tx=0x7f4a9c005d10 comp rx=0 tx=0).stop 2024-09-11T23:59:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 shutdown_connections 2024-09-11T23:59:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4ab5684740 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a1f386ffa0 0x55a1f38723c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4ab5684740 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 0x55a1f384c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4ab5684740 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f384beb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4ab5684740 1 --2- 172.21.15.62:0/4173156635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 0x55a1f384ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4a94ff9640 0 client.26318 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:59:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 >> 172.21.15.62:0/4173156635 conn(0x55a1f380d310 msgr2=0x55a1f3846960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 shutdown_connections 2024-09-11T23:59:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.880+0000 7f4ab5684740 1 -- 172.21.15.62:0/4173156635 wait complete. 2024-09-11T23:59:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.881+0000 7f4ab5684740 20 client.26318 trim_cache size 0 max 16384 2024-09-11T23:59:43.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab5684740 1 Processor -- start 2024-09-11T23:59:43.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab5684740 1 -- start start 2024-09-11T23:59:43.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 0x55a1f386c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f3845b40 0x55a1f3850760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f384f370 0x55a1f3850ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab5684740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a90011070 con 0x55a1f3845b40 2024-09-11T23:59:43.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab5684740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000e910 con 0x55a1f384f370 2024-09-11T23:59:43.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab5684740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000c810 con 0x55a1f38445d0 2024-09-11T23:59:43.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab1047640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f3845b40 0x55a1f3850760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab0846640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f384f370 0x55a1f3850ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab0846640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f384f370 0x55a1f3850ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58752/0 (socket says 172.21.15.62:58752) 2024-09-11T23:59:43.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab1047640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f3845b40 0x55a1f3850760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39486/0 (socket says 172.21.15.62:39486) 2024-09-11T23:59:43.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab0846640 1 -- 172.21.15.62:0/961765873 learned_addr learned my addr 172.21.15.62:0/961765873 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:43.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.899+0000 7f4ab0846640 1 --2- 172.21.15.62:0/961765873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 0x55a1f386c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.900+0000 7f4ab0846640 1 -- 172.21.15.62:0/961765873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 msgr2=0x55a1f386c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:43.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.900+0000 7f4ab0846640 1 --2- 172.21.15.62:0/961765873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 0x55a1f386c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.900+0000 7f4ab0846640 1 -- 172.21.15.62:0/961765873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f3845b40 msgr2=0x55a1f3850760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:43.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.900+0000 7f4ab0846640 1 --2- 172.21.15.62:0/961765873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f3845b40 0x55a1f3850760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.900+0000 7f4ab0846640 1 -- 172.21.15.62:0/961765873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f384ef70 con 0x55a1f384f370 2024-09-11T23:59:43.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.900+0000 7f4ab0846640 1 --2- 172.21.15.62:0/961765873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f384f370 0x55a1f3850ca0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55a1f3872be0 tx=0x7f4aa4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2f83a254185cab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:43.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.901+0000 7f4a96ffd640 1 -- 172.21.15.62:0/961765873 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40130f0 con 0x55a1f384f370 2024-09-11T23:59:43.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.901+0000 7f4a96ffd640 1 -- 172.21.15.62:0/961765873 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa4003e50 con 0x55a1f384f370 2024-09-11T23:59:43.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.901+0000 7f4a96ffd640 1 -- 172.21.15.62:0/961765873 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40130f0 con 0x55a1f384f370 2024-09-11T23:59:43.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.902+0000 7f4ab5684740 1 -- 172.21.15.62:0/961765873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f384f370 msgr2=0x55a1f3850ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:43.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.902+0000 7f4ab5684740 1 --2- 172.21.15.62:0/961765873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f384f370 0x55a1f3850ca0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55a1f3872be0 tx=0x7f4aa4004ab0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.902+0000 7f4ab5684740 1 -- 172.21.15.62:0/961765873 shutdown_connections 2024-09-11T23:59:43.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.902+0000 7f4ab5684740 1 --2- 172.21.15.62:0/961765873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f384f370 0x55a1f3850ca0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.902+0000 7f4ab5684740 1 --2- 172.21.15.62:0/961765873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f3845b40 0x55a1f3850760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.902+0000 7f4ab5684740 1 --2- 172.21.15.62:0/961765873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f38445d0 0x55a1f386c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.902+0000 7f4ab5684740 1 -- 172.21.15.62:0/961765873 >> 172.21.15.62:0/961765873 conn(0x55a1f380d310 msgr2=0x55a1f384df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:43.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.903+0000 7f4ab5684740 1 -- 172.21.15.62:0/961765873 shutdown_connections 2024-09-11T23:59:43.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.903+0000 7f4ab5684740 1 -- 172.21.15.62:0/961765873 wait complete. 2024-09-11T23:59:43.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.903+0000 7f4ab5684740 1 -- start start 2024-09-11T23:59:43.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.903+0000 7f4ab5684740 1 Processor -- start 2024-09-11T23:59:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 0x55a1f3877200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f3877740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab5684740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 0x55a1f3867560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab0846640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 0x55a1f3877200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab0846640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 0x55a1f3877200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39490/0 (socket says 172.21.15.62:39490) 2024-09-11T23:59:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab0846640 1 -- 172.21.15.62:0/3888492624 learned_addr learned my addr 172.21.15.62:0/3888492624 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000cdd0 con 0x55a1f38445d0 2024-09-11T23:59:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000e240 con 0x55a1f3845b40 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab1047640 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f3877740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000d4a0 con 0x55a1f384f370 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.904+0000 7f4ab0846640 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 0x55a1f3867560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4ab5684740 10 client.0 _ll_register_callbacks cb 0x7f4a9c003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4ab1047640 1 -- 172.21.15.62:0/3888492624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 msgr2=0x55a1f3867560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4ab1047640 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 0x55a1f3867560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4ab1047640 1 -- 172.21.15.62:0/3888492624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 msgr2=0x55a1f3877200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4ab1047640 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 0x55a1f3877200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4ab1047640 1 -- 172.21.15.62:0/3888492624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f3867c00 con 0x55a1f3845b40 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4ab1047640 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f3877740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0317a0 tx=0x7f4a9c0048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52665251adb4db2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4a94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4a94ff9640 1 -- 172.21.15.62:0/3888492624 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c060030 con 0x55a1f3845b40 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4a94ff9640 1 -- 172.21.15.62:0/3888492624 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f4a9c054040 con 0x55a1f3845b40 2024-09-11T23:59:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa4003e50 con 0x55a1f3845b40 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4a94ff9640 1 -- 172.21.15.62:0/3888492624 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a9c004a60 con 0x55a1f3845b40 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4ab5684740 10 client.26324 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.905+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1f3868ea0 con 0x55a1f3845b40 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.906+0000 7f4a94ff9640 1 -- 172.21.15.62:0/3888492624 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa4003e50 con 0x55a1f3845b40 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.906+0000 7f4a94ff9640 1 -- 172.21.15.62:0/3888492624 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f4a9c04e050 con 0x55a1f3845b40 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.906+0000 7f4ab5684740 10 client.26324 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.906+0000 7f4ab5684740 10 client.26324 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.906+0000 7f4ab5684740 20 client.26324 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.906+0000 7f4ab5684740 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a1f38daef0 0x55a1f38dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.906+0000 7f4ab5684740 4 client.26324 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.906+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1f37b7860 con 0x55a1f38daef0 2024-09-11T23:59:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.906+0000 7f4ab0846640 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a1f38daef0 0x55a1f38dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4a94ff9640 10 client.26324 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-11T23:59:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab0846640 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a1f38daef0 0x55a1f38dd310 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9ab5f395d46bec3 server_cookie=482dce544dd4f79 in_seq=0 out_seq=0 2024-09-11T23:59:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4a94ff9640 1 -- 172.21.15.62:0/3888492624 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1f37b7860 con 0x55a1f38daef0 2024-09-11T23:59:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4a94ff9640 10 client.26324 handle_command_reply: tid=0 2024-09-11T23:59:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 client.26324 shutdown 2024-09-11T23:59:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 msgr2=0x55a1f3877740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f3877740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0317a0 tx=0x7f4a9c0048f0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 shutdown_connections 2024-09-11T23:59:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55a1f38daef0 0x55a1f38dd310 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f384f370 0x55a1f3867560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1f3845b40 0x55a1f3877740 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 --2- 172.21.15.62:0/3888492624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1f38445d0 0x55a1f3877200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 >> 172.21.15.62:0/3888492624 conn(0x55a1f380d310 msgr2=0x55a1f384b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4a94ff9640 0 client.26324 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-11T23:59:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 shutdown_connections 2024-09-11T23:59:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.907+0000 7f4ab5684740 1 -- 172.21.15.62:0/3888492624 wait complete. 2024-09-11T23:59:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.908+0000 7f4ab5684740 20 client.26324 trim_cache size 0 max 16384 2024-09-11T23:59:43.910 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:59:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.909+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1914681580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a980a2e90 msgr2=0x7f4a980a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.909+0000 7f4ab32ab640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a980a2e90 0x7f4a980a5350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8097aa0 tx=0x7f4aa80046c0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.909+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1914681580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4aac1b8770 msgr2=0x7f4aac1b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.909+0000 7f4ab32ab640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4aac1b8770 0x7f4aac1b8c20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800e970 tx=0x7f4aa800ee30 comp rx=0 tx=0).stop 2024-09-11T23:59:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.910+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1914681580 shutdown_connections 2024-09-11T23:59:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.910+0000 7f4ab32ab640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a980a2e90 0x7f4a980a5350 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.910+0000 7f4ab32ab640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4aac1b8770 0x7f4aac1b8c20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.910+0000 7f4ab32ab640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac109450 0x7f4aac07ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.910+0000 7f4ab32ab640 1 --2- 172.21.15.62:0/1914681580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4aac074990 0x7f4aac07a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.910+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1914681580 >> 172.21.15.62:0/1914681580 conn(0x7f4aac06e140 msgr2=0x7f4aac070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.910+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1914681580 shutdown_connections 2024-09-11T23:59:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:43.910+0000 7f4ab32ab640 1 -- 172.21.15.62:0/1914681580 wait complete. 2024-09-11T23:59:44.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:59:45.035 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:45.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:45 smithi062 ceph-mon[30958]: pgmap v234: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.5 MiB/s wr, 868 op/s 2024-09-11T23:59:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:45 smithi120 ceph-mon[35182]: pgmap v234: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.5 MiB/s wr, 868 op/s 2024-09-11T23:59:45.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:45 smithi083 ceph-mon[34805]: pgmap v234: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.5 MiB/s wr, 868 op/s 2024-09-11T23:59:45.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.731+0000 7f1417fff640 1 -- 172.21.15.62:0/2354318973 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14100319b0 con 0x7f1420109b10 2024-09-11T23:59:45.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.732+0000 7f14273dc640 1 -- 172.21.15.62:0/2354318973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1420109b10 msgr2=0x7f1420114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:45.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.732+0000 7f14273dc640 1 --2- 172.21.15.62:0/2354318973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1420109b10 0x7f1420114570 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1410009900 tx=0x7f141002a480 comp rx=0 tx=0).stop 2024-09-11T23:59:45.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.732+0000 7f14273dc640 1 -- 172.21.15.62:0/2354318973 shutdown_connections 2024-09-11T23:59:45.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.732+0000 7f14273dc640 1 --2- 172.21.15.62:0/2354318973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1420114ab0 0x7f1420116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.732+0000 7f14273dc640 1 --2- 172.21.15.62:0/2354318973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1420109b10 0x7f1420114570 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.732+0000 7f14273dc640 1 --2- 172.21.15.62:0/2354318973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14201091f0 0x7f14201095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.732+0000 7f14273dc640 1 -- 172.21.15.62:0/2354318973 >> 172.21.15.62:0/2354318973 conn(0x7f142006e140 msgr2=0x7f142006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:45.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.739+0000 7f14273dc640 1 -- 172.21.15.62:0/2354318973 shutdown_connections 2024-09-11T23:59:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.739+0000 7f14273dc640 1 -- 172.21.15.62:0/2354318973 wait complete. 2024-09-11T23:59:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f14273dc640 1 Processor -- start 2024-09-11T23:59:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f14273dc640 1 -- start start 2024-09-11T23:59:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f14273dc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14201091f0 0x7f1420111690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f14273dc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420109b10 0x7f1420111bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f14273dc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1420114ab0 0x7f1420112110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f14273dc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f142010be80 con 0x7f14201091f0 2024-09-11T23:59:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f14273dc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f142010bd00 con 0x7f1420109b10 2024-09-11T23:59:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f14273dc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f142010c000 con 0x7f1420114ab0 2024-09-11T23:59:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f1424977640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420109b10 0x7f1420111bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f1424977640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420109b10 0x7f1420111bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58772/0 (socket says 172.21.15.62:58772) 2024-09-11T23:59:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f1424977640 1 -- 172.21.15.62:0/1357927129 learned_addr learned my addr 172.21.15.62:0/1357927129 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.740+0000 7f1425979640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1420114ab0 0x7f1420112110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.741+0000 7f1425178640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14201091f0 0x7f1420111690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.743+0000 7f1424977640 1 -- 172.21.15.62:0/1357927129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1420114ab0 msgr2=0x7f1420112110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.743+0000 7f1424977640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1420114ab0 0x7f1420112110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.743+0000 7f1424977640 1 -- 172.21.15.62:0/1357927129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14201091f0 msgr2=0x7f1420111690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.743+0000 7f1424977640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14201091f0 0x7f1420111690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.743+0000 7f1424977640 1 -- 172.21.15.62:0/1357927129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14201240a0 con 0x7f1420109b10 2024-09-11T23:59:45.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.749+0000 7f1424977640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420109b10 0x7f1420111bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1410006010 tx=0x7f14100042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:45.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.751+0000 7f14167fc640 1 -- 172.21.15.62:0/1357927129 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141003f030 con 0x7f1420109b10 2024-09-11T23:59:45.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.751+0000 7f14167fc640 1 -- 172.21.15.62:0/1357927129 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1410003ce0 con 0x7f1420109b10 2024-09-11T23:59:45.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.751+0000 7f14167fc640 1 -- 172.21.15.62:0/1357927129 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410035070 con 0x7f1420109b10 2024-09-11T23:59:45.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.752+0000 7f14273dc640 1 -- 172.21.15.62:0/1357927129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1420124330 con 0x7f1420109b10 2024-09-11T23:59:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.752+0000 7f14273dc640 1 -- 172.21.15.62:0/1357927129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14201247c0 con 0x7f1420109b10 2024-09-11T23:59:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.758+0000 7f14297b5740 1 Processor -- start 2024-09-11T23:59:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.760+0000 7f14297b5740 1 -- start start 2024-09-11T23:59:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.761+0000 7f14167fc640 1 -- 172.21.15.62:0/1357927129 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1410042030 con 0x7f1420109b10 2024-09-11T23:59:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.761+0000 7f14297b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5f0b40 0x561d0f5ef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.761+0000 7f14297b5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f5efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.761+0000 7f14297b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5fa370 0x561d0f5fc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.761+0000 7f14297b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561d0f5552f0 con 0x561d0f5fa370 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.761+0000 7f14297b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d0f4fccf0 con 0x561d0f5f0b40 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.761+0000 7f14297b5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561d0f5512b0 con 0x561d0f5ef5d0 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.763+0000 7f1424977640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5fa370 0x561d0f5fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.765+0000 7f1425979640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5f0b40 0x561d0f5ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.765+0000 7f1425979640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5f0b40 0x561d0f5ef090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58786/0 (socket says 172.21.15.62:58786) 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.766+0000 7f1425979640 1 -- 172.21.15.62:0/2875734734 learned_addr learned my addr 172.21.15.62:0/2875734734 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.766+0000 7f14167fc640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f140c0a2df0 0x7f140c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.766+0000 7f1425178640 1 --2- 172.21.15.62:0/2875734734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f5efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.767+0000 7f1425979640 1 -- 172.21.15.62:0/2875734734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 msgr2=0x561d0f5efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.767+0000 7f1425979640 1 --2- 172.21.15.62:0/2875734734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f5efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.767+0000 7f1425979640 1 -- 172.21.15.62:0/2875734734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5fa370 msgr2=0x561d0f5fc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.767+0000 7f1425979640 1 --2- 172.21.15.62:0/2875734734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5fa370 0x561d0f5fc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.767+0000 7f1425979640 1 -- 172.21.15.62:0/2875734734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0f4e13a0 con 0x561d0f5f0b40 2024-09-11T23:59:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.767+0000 7f1425979640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f140c0a2df0 0x7f140c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.767+0000 7f1424977640 1 --2- 172.21.15.62:0/2875734734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5fa370 0x561d0f5fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.768+0000 7f14167fc640 1 -- 172.21.15.62:0/1357927129 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f14100ef5f0 con 0x7f1420109b10 2024-09-11T23:59:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.771+0000 7f1425979640 1 --2- 172.21.15.62:0/2875734734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5f0b40 0x561d0f5ef090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f141800af30 tx=0x7f141800f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed64c7bf317ac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:45.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.773+0000 7f140b7fe640 1 -- 172.21.15.62:0/2875734734 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141801ac60 con 0x561d0f5f0b40 2024-09-11T23:59:45.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.774+0000 7f140b7fe640 1 -- 172.21.15.62:0/2875734734 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141801ae00 con 0x561d0f5f0b40 2024-09-11T23:59:45.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.774+0000 7f140b7fe640 1 -- 172.21.15.62:0/2875734734 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141800d560 con 0x561d0f5f0b40 2024-09-11T23:59:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.775+0000 7f1425979640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f140c0a2df0 0x7f140c0a52b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1418020000 tx=0x7f141801c030 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:45.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.776+0000 7f14297b5740 1 -- 172.21.15.62:0/2875734734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5f0b40 msgr2=0x561d0f5ef090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:45.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.776+0000 7f14297b5740 1 --2- 172.21.15.62:0/2875734734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5f0b40 0x561d0f5ef090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f141800af30 tx=0x7f141800f850 comp rx=0 tx=0).stop 2024-09-11T23:59:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.778+0000 7f14297b5740 1 -- 172.21.15.62:0/2875734734 shutdown_connections 2024-09-11T23:59:45.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.778+0000 7f14297b5740 1 --2- 172.21.15.62:0/2875734734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5fa370 0x561d0f5fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.779+0000 7f14297b5740 1 --2- 172.21.15.62:0/2875734734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f5efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.779+0000 7f14297b5740 1 --2- 172.21.15.62:0/2875734734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5f0b40 0x561d0f5ef090 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.779+0000 7f14297b5740 1 -- 172.21.15.62:0/2875734734 >> 172.21.15.62:0/2875734734 conn(0x561d0f5b8310 msgr2=0x561d0f5ebc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:45.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.781+0000 7f14297b5740 1 -- 172.21.15.62:0/2875734734 shutdown_connections 2024-09-11T23:59:45.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.783+0000 7f14297b5740 1 -- 172.21.15.62:0/2875734734 wait complete. 2024-09-11T23:59:45.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.783+0000 7f14297b5740 1 -- start start 2024-09-11T23:59:45.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.783+0000 7f14297b5740 1 Processor -- start 2024-09-11T23:59:45.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.784+0000 7f14297b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5ef5d0 0x561d0f5f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.785+0000 7f14297b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f5f7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.785+0000 7f1425178640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5ef5d0 0x561d0f5f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.785+0000 7f1425178640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5ef5d0 0x561d0f5f9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58802/0 (socket says 172.21.15.62:58802) 2024-09-11T23:59:45.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.785+0000 7f1425178640 1 -- 172.21.15.62:0/3549884245 learned_addr learned my addr 172.21.15.62:0/3549884245 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:45.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.785+0000 7f14297b5740 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5fa370 0x561d0f5f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.785+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561d0f5fd070 con 0x561d0f5f0b40 2024-09-11T23:59:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.785+0000 7f1425979640 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5fa370 0x561d0f5f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.785+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d0f510060 con 0x561d0f5ef5d0 2024-09-11T23:59:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.786+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561d0f5fd1f0 con 0x561d0f5fa370 2024-09-11T23:59:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.786+0000 7f14297b5740 10 client.0 _ll_register_callbacks cb 0x561d0f2638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.786+0000 7f1425979640 1 -- 172.21.15.62:0/3549884245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5ef5d0 msgr2=0x561d0f5f9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.786+0000 7f1425979640 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5ef5d0 0x561d0f5f9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.786+0000 7f1425979640 1 -- 172.21.15.62:0/3549884245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 msgr2=0x561d0f5f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:45.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.786+0000 7f1425979640 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f5f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.786+0000 7f1425979640 1 -- 172.21.15.62:0/3549884245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0f2c63c0 con 0x561d0f5fa370 2024-09-11T23:59:45.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.786+0000 7f1425979640 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5fa370 0x561d0f5f7600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f141800fd50 tx=0x7f1418012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2d06ff439a2b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.787+0000 7f1408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.787+0000 7f1408ff9640 1 -- 172.21.15.62:0/3549884245 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141800aa30 con 0x561d0f5fa370 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.787+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141801ae00 con 0x561d0f5fa370 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.787+0000 7f1408ff9640 1 -- 172.21.15.62:0/3549884245 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f1418004c10 con 0x561d0f5fa370 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.787+0000 7f1408ff9640 1 -- 172.21.15.62:0/3549884245 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1418004e10 con 0x561d0f5fa370 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.787+0000 7f14297b5740 10 client.26290 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.787+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561d0f608ac0 con 0x561d0f5fa370 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.788+0000 7f1408ff9640 1 -- 172.21.15.62:0/3549884245 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141801ae00 con 0x561d0f5fa370 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.788+0000 7f1408ff9640 1 -- 172.21.15.62:0/3549884245 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f1418023070 con 0x561d0f5fa370 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.788+0000 7f14297b5740 10 client.26290 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.788+0000 7f14297b5740 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.788+0000 7f14297b5740 20 client.26290 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.788+0000 7f14297b5740 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561d0f609f20 0x561d0f60c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.788+0000 7f14297b5740 4 client.26290 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.788+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d0f562860 con 0x561d0f609f20 2024-09-11T23:59:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.789+0000 7f1425178640 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561d0f609f20 0x561d0f60c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.789+0000 7f1425178640 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561d0f609f20 0x561d0f60c310 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=213424e0c7f22fc0 server_cookie=a9fa5dfec07b229c in_seq=0 out_seq=0 2024-09-11T23:59:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.789+0000 7f1408ff9640 10 client.26290 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:45.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.790+0000 7f1408ff9640 1 -- 172.21.15.62:0/3549884245 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d0f562860 con 0x561d0f609f20 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.790+0000 7f1408ff9640 10 client.26290 handle_command_reply: tid=0 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.790+0000 7f14297b5740 1 client.26290 shutdown 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.791+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5fa370 msgr2=0x561d0f5f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.791+0000 7f14297b5740 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5fa370 0x561d0f5f7600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f141800fd50 tx=0x7f1418012730 comp rx=0 tx=0).stop 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.792+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 shutdown_connections 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.792+0000 7f14297b5740 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561d0f609f20 0x561d0f60c310 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.792+0000 7f14297b5740 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5fa370 0x561d0f5f7600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.792+0000 7f14297b5740 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f5f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.792+0000 7f14297b5740 1 --2- 172.21.15.62:0/3549884245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5ef5d0 0x561d0f5f9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.792+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 >> 172.21.15.62:0/3549884245 conn(0x561d0f5b8310 msgr2=0x561d0f5f1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.792+0000 7f1408ff9640 0 client.26290 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.793+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 shutdown_connections 2024-09-11T23:59:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.793+0000 7f14297b5740 1 -- 172.21.15.62:0/3549884245 wait complete. 2024-09-11T23:59:45.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.793+0000 7f14297b5740 20 client.26290 trim_cache size 0 max 16384 2024-09-11T23:59:45.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.815+0000 7f14297b5740 1 Processor -- start 2024-09-11T23:59:45.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.815+0000 7f14297b5740 1 -- start start 2024-09-11T23:59:45.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.815+0000 7f14297b5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f606370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.815+0000 7f14297b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f5eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.815+0000 7f14297b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5fa370 0x561d0f5ebc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.815+0000 7f14297b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1404011110 con 0x561d0f5f0b40 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.815+0000 7f14297b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f140400e9b0 con 0x561d0f5fa370 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.815+0000 7f14297b5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f140400c820 con 0x561d0f5ef5d0 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.816+0000 7f1424977640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f5eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.816+0000 7f1424977640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f5eb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39552/0 (socket says 172.21.15.62:39552) 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.816+0000 7f1424977640 1 -- 172.21.15.62:0/1033357277 learned_addr learned my addr 172.21.15.62:0/1033357277 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.816+0000 7f1425178640 1 --2- 172.21.15.62:0/1033357277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f606370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.816+0000 7f1425178640 1 -- 172.21.15.62:0/1033357277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5fa370 msgr2=0x561d0f5ebc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.816+0000 7f1425178640 1 --2- 172.21.15.62:0/1033357277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5fa370 0x561d0f5ebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.816+0000 7f1425178640 1 -- 172.21.15.62:0/1033357277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 msgr2=0x561d0f5eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:45.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.816+0000 7f1425178640 1 --2- 172.21.15.62:0/1033357277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f5eb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.816+0000 7f1425178640 1 -- 172.21.15.62:0/1033357277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0f5ec180 con 0x561d0f5ef5d0 2024-09-11T23:59:45.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.816+0000 7f1425178640 1 --2- 172.21.15.62:0/1033357277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f606370 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x561d0f60c9f0 tx=0x7f141c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6b30668cc0b127b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:45.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.818+0000 7f140affd640 1 -- 172.21.15.62:0/1033357277 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c005250 con 0x561d0f5ef5d0 2024-09-11T23:59:45.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.818+0000 7f140affd640 1 -- 172.21.15.62:0/1033357277 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141c004c80 con 0x561d0f5ef5d0 2024-09-11T23:59:45.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.818+0000 7f140affd640 1 -- 172.21.15.62:0/1033357277 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c01f8d0 con 0x561d0f5ef5d0 2024-09-11T23:59:45.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 -- 172.21.15.62:0/1033357277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 msgr2=0x561d0f606370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:45.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 --2- 172.21.15.62:0/1033357277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f606370 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x561d0f60c9f0 tx=0x7f141c005f90 comp rx=0 tx=0).stop 2024-09-11T23:59:45.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 -- 172.21.15.62:0/1033357277 shutdown_connections 2024-09-11T23:59:45.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 --2- 172.21.15.62:0/1033357277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5fa370 0x561d0f5ebc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 --2- 172.21.15.62:0/1033357277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f5eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 --2- 172.21.15.62:0/1033357277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f606370 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 -- 172.21.15.62:0/1033357277 >> 172.21.15.62:0/1033357277 conn(0x561d0f5b8310 msgr2=0x561d0f5f58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:45.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 -- 172.21.15.62:0/1033357277 shutdown_connections 2024-09-11T23:59:45.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 -- 172.21.15.62:0/1033357277 wait complete. 2024-09-11T23:59:45.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 -- start start 2024-09-11T23:59:45.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.819+0000 7f14297b5740 1 Processor -- start 2024-09-11T23:59:45.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.820+0000 7f14297b5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f5f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.820+0000 7f14297b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f6024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.820+0000 7f14297b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5fa370 0x561d0f602a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.820+0000 7f14297b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f140400ce70 con 0x561d0f5f0b40 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.820+0000 7f14297b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f140400e2e0 con 0x561d0f5fa370 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.820+0000 7f14297b5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f140400d540 con 0x561d0f5ef5d0 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.820+0000 7f1424977640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f6024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.820+0000 7f1425979640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5fa370 0x561d0f602a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.820+0000 7f1424977640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f6024e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39562/0 (socket says 172.21.15.62:39562) 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.820+0000 7f1424977640 1 -- 172.21.15.62:0/3070298824 learned_addr learned my addr 172.21.15.62:0/3070298824 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.821+0000 7f1425178640 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f5f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.821+0000 7f14297b5740 10 client.0 _ll_register_callbacks cb 0x7f1418012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.821+0000 7f1425979640 1 -- 172.21.15.62:0/3070298824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 msgr2=0x561d0f5f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.821+0000 7f1425979640 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f5f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.821+0000 7f1425979640 1 -- 172.21.15.62:0/3070298824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 msgr2=0x561d0f6024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.821+0000 7f1425979640 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f6024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.821+0000 7f1425979640 1 -- 172.21.15.62:0/3070298824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0f5f3280 con 0x561d0f5fa370 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.821+0000 7f1425979640 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5fa370 0x561d0f602a80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f141800fd50 tx=0x7f1418004020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3cca4c1f17a4fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.822+0000 7f1408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.822+0000 7f1408ff9640 1 -- 172.21.15.62:0/3070298824 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141800aa30 con 0x561d0f5fa370 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.822+0000 7f1408ff9640 1 -- 172.21.15.62:0/3070298824 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f1418004c10 con 0x561d0f5fa370 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.822+0000 7f14297b5740 1 -- 172.21.15.62:0/3070298824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141c004c80 con 0x561d0f5fa370 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.822+0000 7f1408ff9640 1 -- 172.21.15.62:0/3070298824 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1418004110 con 0x561d0f5fa370 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f14297b5740 10 client.26339 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f14297b5740 1 -- 172.21.15.62:0/3070298824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561d0f673cc0 con 0x561d0f5fa370 2024-09-11T23:59:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f1408ff9640 1 -- 172.21.15.62:0/3070298824 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141c004c80 con 0x561d0f5fa370 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f1408ff9640 1 -- 172.21.15.62:0/3070298824 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f1418023070 con 0x561d0f5fa370 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f14297b5740 10 client.26339 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f14297b5740 10 client.26339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f14297b5740 20 client.26339 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f14297b5740 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561d0f6744c0 0x561d0f676920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f14297b5740 4 client.26339 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f14297b5740 1 -- 172.21.15.62:0/3070298824 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d0f55cd70 con 0x561d0f6744c0 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.823+0000 7f1425178640 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561d0f6744c0 0x561d0f676920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.824+0000 7f1408ff9640 10 client.26339 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.824+0000 7f1425178640 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561d0f6744c0 0x561d0f676920 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d05ab87ca6c99ec server_cookie=709c8e1929f999fb in_seq=0 out_seq=0 2024-09-11T23:59:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.824+0000 7f1408ff9640 1 -- 172.21.15.62:0/3070298824 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d0f55cd70 con 0x561d0f6744c0 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.824+0000 7f1408ff9640 10 client.26339 handle_command_reply: tid=0 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.824+0000 7f14297b5740 1 client.26339 shutdown 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.824+0000 7f14297b5740 1 -- 172.21.15.62:0/3070298824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5fa370 msgr2=0x561d0f602a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.825+0000 7f14297b5740 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5fa370 0x561d0f602a80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f141800fd50 tx=0x7f1418004020 comp rx=0 tx=0).stop 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.825+0000 7f14297b5740 1 -- 172.21.15.62:0/3070298824 shutdown_connections 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.825+0000 7f14297b5740 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561d0f6744c0 0x561d0f676920 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.825+0000 7f14297b5740 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0f5fa370 0x561d0f602a80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.825+0000 7f14297b5740 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d0f5f0b40 0x561d0f6024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.825+0000 7f14297b5740 1 --2- 172.21.15.62:0/3070298824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d0f5ef5d0 0x561d0f5f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.825+0000 7f14297b5740 1 -- 172.21.15.62:0/3070298824 >> 172.21.15.62:0/3070298824 conn(0x561d0f5b8310 msgr2=0x561d0f5f5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.825+0000 7f1408ff9640 0 client.26339 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.825+0000 7f14297b5740 1 -- 172.21.15.62:0/3070298824 shutdown_connections 2024-09-11T23:59:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.826+0000 7f14297b5740 1 -- 172.21.15.62:0/3070298824 wait complete. 2024-09-11T23:59:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.826+0000 7f14297b5740 20 client.26339 trim_cache size 0 max 16384 2024-09-11T23:59:45.828 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-11T23:59:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.827+0000 7f14273dc640 1 -- 172.21.15.62:0/1357927129 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f140c0a2df0 msgr2=0x7f140c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.827+0000 7f14273dc640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f140c0a2df0 0x7f140c0a52b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1418020000 tx=0x7f141801c030 comp rx=0 tx=0).stop 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.827+0000 7f14273dc640 1 -- 172.21.15.62:0/1357927129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420109b10 msgr2=0x7f1420111bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.827+0000 7f14273dc640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420109b10 0x7f1420111bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1410006010 tx=0x7f14100042f0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.827+0000 7f14273dc640 1 -- 172.21.15.62:0/1357927129 shutdown_connections 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.827+0000 7f14273dc640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f140c0a2df0 0x7f140c0a52b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.828+0000 7f14273dc640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1420114ab0 0x7f1420112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.828+0000 7f14273dc640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420109b10 0x7f1420111bd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.828+0000 7f14273dc640 1 --2- 172.21.15.62:0/1357927129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14201091f0 0x7f1420111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.828+0000 7f14273dc640 1 -- 172.21.15.62:0/1357927129 >> 172.21.15.62:0/1357927129 conn(0x7f142006e140 msgr2=0x7f1420070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.828+0000 7f14273dc640 1 -- 172.21.15.62:0/1357927129 shutdown_connections 2024-09-11T23:59:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:45.828+0000 7f14273dc640 1 -- 172.21.15.62:0/1357927129 wait complete. 2024-09-11T23:59:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 425.0 s, 2209.6 tps, lat 45.268 ms stddev 47.138 2024-09-11T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:47 smithi120 ceph-mon[35182]: pgmap v235: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.6 MiB/s wr, 574 op/s 2024-09-11T23:59:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:47 smithi083 ceph-mon[34805]: pgmap v235: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.6 MiB/s wr, 574 op/s 2024-09-11T23:59:47.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:47 smithi062 ceph-mon[30958]: pgmap v235: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.6 MiB/s wr, 574 op/s 2024-09-11T23:59:47.723 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:59:47.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:59:48.095 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:48.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.612+0000 7f21fbe58640 1 -- 172.21.15.62:0/4235772666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21f4102d90 msgr2=0x7f21f4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:48.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.612+0000 7f21fbe58640 1 --2- 172.21.15.62:0/4235772666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21f4102d90 0x7f21f4103170 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009cd0 tx=0x7f21ec02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.612+0000 7f21fbe58640 1 -- 172.21.15.62:0/4235772666 shutdown_connections 2024-09-11T23:59:48.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.612+0000 7f21fbe58640 1 --2- 172.21.15.62:0/4235772666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4108290 0x7f21f410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.612+0000 7f21fbe58640 1 --2- 172.21.15.62:0/4235772666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21f4103740 0x7f21f4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.612+0000 7f21fbe58640 1 --2- 172.21.15.62:0/4235772666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21f4102d90 0x7f21f4103170 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.612+0000 7f21fbe58640 1 -- 172.21.15.62:0/4235772666 >> 172.21.15.62:0/4235772666 conn(0x7f21f40fc7d0 msgr2=0x7f21f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:48.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.612+0000 7f21fbe58640 1 -- 172.21.15.62:0/4235772666 shutdown_connections 2024-09-11T23:59:48.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.612+0000 7f21fbe58640 1 -- 172.21.15.62:0/4235772666 wait complete. 2024-09-11T23:59:48.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.613+0000 7f21fbe58640 1 Processor -- start 2024-09-11T23:59:48.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.613+0000 7f21fbe58640 1 -- start start 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.613+0000 7f21fbe58640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21f4102d90 0x7f21f41a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.613+0000 7f21fbe58640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4103740 0x7f21f41a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.613+0000 7f21fbe58640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21f4108290 0x7f21f419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.613+0000 7f21fbe58640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41120d0 con 0x7f21f4108290 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.613+0000 7f21fbe58640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111f50 con 0x7f21f4103740 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.613+0000 7f21fbe58640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4112250 con 0x7f21f4102d90 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.613+0000 7f21fa3f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21f4108290 0x7f21f419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f9bf4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21f4102d90 0x7f21f41a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21fa3f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21f4108290 0x7f21f419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39584/0 (socket says 172.21.15.62:39584) 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f9bf4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21f4102d90 0x7f21f41a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:55242/0 (socket says 172.21.15.62:55242) 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21fa3f5640 1 -- 172.21.15.62:0/1655771952 learned_addr learned my addr 172.21.15.62:0/1655771952 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f93f3640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4103740 0x7f21f41a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f9bf4640 1 -- 172.21.15.62:0/1655771952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4103740 msgr2=0x7f21f41a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4103740 0x7f21f41a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f9bf4640 1 -- 172.21.15.62:0/1655771952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21f4108290 msgr2=0x7f21f419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21f4108290 0x7f21f419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f9bf4640 1 -- 172.21.15.62:0/1655771952 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f419ad50 con 0x7f21f4102d90 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21fa3f5640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21f4108290 0x7f21f419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21f4102d90 0x7f21f41a0460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f21ec000f80 tx=0x7f21ec031e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f2ffd640 1 -- 172.21.15.62:0/1655771952 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec003d30 con 0x7f21f4102d90 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21fbe58640 1 -- 172.21.15.62:0/1655771952 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f419afe0 con 0x7f21f4102d90 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f2ffd640 1 -- 172.21.15.62:0/1655771952 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ec004520 con 0x7f21f4102d90 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.614+0000 7f21f2ffd640 1 -- 172.21.15.62:0/1655771952 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec030dc0 con 0x7f21f4102d90 2024-09-11T23:59:48.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.615+0000 7f21fbe58640 1 -- 172.21.15.62:0/1655771952 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41a7330 con 0x7f21f4102d90 2024-09-11T23:59:48.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.618+0000 7f21fe231740 1 Processor -- start 2024-09-11T23:59:48.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.618+0000 7f21fe231740 1 -- start start 2024-09-11T23:59:48.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.618+0000 7f21fe231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 0x55f669970f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.618+0000 7f21fe231740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.618+0000 7f21fe231740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 0x55f66997e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21fe231740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f6698d6280 con 0x55f669971480 2024-09-11T23:59:48.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21fe231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f66987db10 con 0x55f669970b60 2024-09-11T23:59:48.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21fe231740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f6698d2240 con 0x55f66997bd40 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f93f3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 0x55f66997e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f93f3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 0x55f66997e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:55246/0 (socket says 172.21.15.62:55246) 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f93f3640 1 -- 172.21.15.62:0/396693005 learned_addr learned my addr 172.21.15.62:0/396693005 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21fa3f5640 1 --2- 172.21.15.62:0/396693005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 0x55f669970f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f2ffd640 1 -- 172.21.15.62:0/1655771952 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f21ec004040 con 0x7f21f4102d90 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f93f3640 1 -- 172.21.15.62:0/396693005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 msgr2=0x55f669970f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f93f3640 1 --2- 172.21.15.62:0/396693005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 0x55f669970f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f93f3640 1 -- 172.21.15.62:0/396693005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 msgr2=0x55f66997b800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/396693005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f93f3640 1 --2- 172.21.15.62:0/396693005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f93f3640 1 -- 172.21.15.62:0/396693005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66988a1f0 con 0x55f66997bd40 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.619+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/396693005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:48.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.620+0000 7f21f93f3640 1 --2- 172.21.15.62:0/396693005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 0x55f66997e130 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009c20 tx=0x7f21e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=407204dcf9744af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:48.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.620+0000 7f21dbfff640 1 -- 172.21.15.62:0/396693005 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8004230 con 0x55f66997bd40 2024-09-11T23:59:48.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.620+0000 7f21dbfff640 1 -- 172.21.15.62:0/396693005 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e80043d0 con 0x55f66997bd40 2024-09-11T23:59:48.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.620+0000 7f21dbfff640 1 -- 172.21.15.62:0/396693005 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8004d70 con 0x55f66997bd40 2024-09-11T23:59:48.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.620+0000 7f21f2ffd640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f21dc0a2cf0 0x7f21dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.620+0000 7f21f2ffd640 1 -- 172.21.15.62:0/1655771952 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f21ec02e030 con 0x7f21f4102d90 2024-09-11T23:59:48.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.620+0000 7f21fa3f5640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f21dc0a2cf0 0x7f21dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.621+0000 7f21fe231740 1 -- 172.21.15.62:0/396693005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 msgr2=0x55f66997e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.621+0000 7f21fa3f5640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f21dc0a2cf0 0x7f21dc0a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009640 tx=0x7f21e4005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.621+0000 7f21fe231740 1 --2- 172.21.15.62:0/396693005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 0x55f66997e130 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009c20 tx=0x7f21e8011810 comp rx=0 tx=0).stop 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.621+0000 7f21fe231740 1 -- 172.21.15.62:0/396693005 shutdown_connections 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.621+0000 7f21fe231740 1 --2- 172.21.15.62:0/396693005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 0x55f66997e130 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.621+0000 7f21fe231740 1 --2- 172.21.15.62:0/396693005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.621+0000 7f21fe231740 1 --2- 172.21.15.62:0/396693005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 0x55f669970f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.621+0000 7f21fe231740 1 -- 172.21.15.62:0/396693005 >> 172.21.15.62:0/396693005 conn(0x55f66996d0f0 msgr2=0x55f66996d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.621+0000 7f21fe231740 1 -- 172.21.15.62:0/396693005 shutdown_connections 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.622+0000 7f21fe231740 1 -- 172.21.15.62:0/396693005 wait complete. 2024-09-11T23:59:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.622+0000 7f21fe231740 1 -- start start 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.623+0000 7f21fe231740 1 Processor -- start 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.623+0000 7f21fe231740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669978bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21fe231740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 0x55f669979100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21fe231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 0x55f669979640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21fe231740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f66997e910 con 0x55f669970b60 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21fe231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f669890f10 con 0x55f66997bd40 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21fe231740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f66997ea90 con 0x55f669971480 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21f9bf4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669978bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21f9bf4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669978bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39596/0 (socket says 172.21.15.62:39596) 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21fe231740 10 client.0 _ll_register_callbacks cb 0x55f669859a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21f9bf4640 1 -- 172.21.15.62:0/2217645682 learned_addr learned my addr 172.21.15.62:0/2217645682 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21f93f3640 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 0x55f669979100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21fa3f5640 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 0x55f669979640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21f9bf4640 1 -- 172.21.15.62:0/2217645682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 msgr2=0x55f669979100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 0x55f669979100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21f9bf4640 1 -- 172.21.15.62:0/2217645682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 msgr2=0x55f669979640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 0x55f669979640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21f9bf4640 1 -- 172.21.15.62:0/2217645682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6696476a0 con 0x55f669970b60 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669978bc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f21ec006010 tx=0x7f21ec0b8ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5934b56fc219385 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.624+0000 7f21d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.625+0000 7f21d97fa640 1 -- 172.21.15.62:0/2217645682 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0bcf10 con 0x55f669970b60 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.625+0000 7f21d97fa640 1 -- 172.21.15.62:0/2217645682 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f21ec0a70e0 con 0x55f669970b60 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.625+0000 7f21fe231740 1 -- 172.21.15.62:0/2217645682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e80043d0 con 0x55f669970b60 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.625+0000 7f21d97fa640 1 -- 172.21.15.62:0/2217645682 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f21ec0bc810 con 0x55f669970b60 2024-09-11T23:59:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.625+0000 7f21fe231740 10 client.16749 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.625+0000 7f21fe231740 1 -- 172.21.15.62:0/2217645682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f66998a460 con 0x55f669970b60 2024-09-11T23:59:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.626+0000 7f21d97fa640 1 -- 172.21.15.62:0/2217645682 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e80043d0 con 0x55f669970b60 2024-09-11T23:59:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.626+0000 7f21d97fa640 1 -- 172.21.15.62:0/2217645682 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f21ec030070 con 0x55f669970b60 2024-09-11T23:59:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.626+0000 7f21fe231740 10 client.16749 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.626+0000 7f21fe231740 10 client.16749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.626+0000 7f21fe231740 20 client.16749 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.626+0000 7f21fe231740 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f66998bb40 0x55f66998df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.626+0000 7f21fe231740 4 client.16749 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.626+0000 7f21fe231740 1 -- 172.21.15.62:0/2217645682 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6698e37f0 con 0x55f66998bb40 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.627+0000 7f21f93f3640 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f66998bb40 0x55f66998df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.628+0000 7f21f93f3640 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f66998bb40 0x55f66998df90 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c96f3898245351 server_cookie=5eee71c67526deda in_seq=0 out_seq=0 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.628+0000 7f21d97fa640 10 client.16749 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.628+0000 7f21d97fa640 1 -- 172.21.15.62:0/2217645682 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6698e37f0 con 0x55f66998bb40 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.628+0000 7f21d97fa640 10 client.16749 handle_command_reply: tid=0 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.628+0000 7f21fe231740 1 client.16749 shutdown 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 1 -- 172.21.15.62:0/2217645682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 msgr2=0x55f669978bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669978bc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f21ec006010 tx=0x7f21ec0b8ba0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 1 -- 172.21.15.62:0/2217645682 shutdown_connections 2024-09-11T23:59:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f66998bb40 0x55f66998df90 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21d97fa640 0 client.16749 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 0x55f669979640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 0x55f669979100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 1 --2- 172.21.15.62:0/2217645682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669978bc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 1 -- 172.21.15.62:0/2217645682 >> 172.21.15.62:0/2217645682 conn(0x55f66996d0f0 msgr2=0x55f669972e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 1 -- 172.21.15.62:0/2217645682 shutdown_connections 2024-09-11T23:59:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 1 -- 172.21.15.62:0/2217645682 wait complete. 2024-09-11T23:59:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.629+0000 7f21fe231740 20 client.16749 trim_cache size 0 max 16384 2024-09-11T23:59:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21fe231740 1 Processor -- start 2024-09-11T23:59:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21fe231740 1 -- start start 2024-09-11T23:59:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21fe231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 0x55f6699735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21fe231740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21fe231740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 0x55f669993830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21fe231740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41b2310 con 0x55f669971480 2024-09-11T23:59:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21fe231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41afbb0 con 0x55f669970b60 2024-09-11T23:59:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21fe231740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41ada20 con 0x55f66997bd40 2024-09-11T23:59:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21f93f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21f93f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39600/0 (socket says 172.21.15.62:39600) 2024-09-11T23:59:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21f93f3640 1 -- 172.21.15.62:0/1611419452 learned_addr learned my addr 172.21.15.62:0/1611419452 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.650+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/1611419452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 0x55f6699735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21fa3f5640 1 --2- 172.21.15.62:0/1611419452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 0x55f669993830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21f93f3640 1 -- 172.21.15.62:0/1611419452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 msgr2=0x55f669993830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21f93f3640 1 --2- 172.21.15.62:0/1611419452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 0x55f669993830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21f93f3640 1 -- 172.21.15.62:0/1611419452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 msgr2=0x55f6699735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21f93f3640 1 --2- 172.21.15.62:0/1611419452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 0x55f6699735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21f93f3640 1 -- 172.21.15.62:0/1611419452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6699712b0 con 0x55f669971480 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/1611419452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 0x55f6699735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21f93f3640 1 --2- 172.21.15.62:0/1611419452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997aa20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009b40 tx=0x7f21e8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cf9759687598ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21db7fe640 1 -- 172.21.15.62:0/1611419452 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e802a070 con 0x55f669971480 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21db7fe640 1 -- 172.21.15.62:0/1611419452 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e801f040 con 0x55f669971480 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.651+0000 7f21db7fe640 1 -- 172.21.15.62:0/1611419452 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e801e040 con 0x55f669971480 2024-09-11T23:59:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.653+0000 7f21fe231740 1 -- 172.21.15.62:0/1611419452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 msgr2=0x55f66997aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:48.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.653+0000 7f21fe231740 1 --2- 172.21.15.62:0/1611419452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997aa20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009b40 tx=0x7f21e8009920 comp rx=0 tx=0).stop 2024-09-11T23:59:48.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.653+0000 7f21fe231740 1 -- 172.21.15.62:0/1611419452 shutdown_connections 2024-09-11T23:59:48.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.653+0000 7f21fe231740 1 --2- 172.21.15.62:0/1611419452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f66997bd40 0x55f669993830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.653+0000 7f21fe231740 1 --2- 172.21.15.62:0/1611419452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669971480 0x55f66997aa20 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.653+0000 7f21fe231740 1 --2- 172.21.15.62:0/1611419452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f669970b60 0x55f6699735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.653+0000 7f21fe231740 1 -- 172.21.15.62:0/1611419452 >> 172.21.15.62:0/1611419452 conn(0x55f66996d0f0 msgr2=0x55f66997a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:48.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.653+0000 7f21fe231740 1 -- 172.21.15.62:0/1611419452 shutdown_connections 2024-09-11T23:59:48.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.653+0000 7f21fe231740 1 -- 172.21.15.62:0/1611419452 wait complete. 2024-09-11T23:59:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.653+0000 7f21fe231740 1 -- start start 2024-09-11T23:59:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.654+0000 7f21fe231740 1 Processor -- start 2024-09-11T23:59:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.654+0000 7f21fe231740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669983d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.654+0000 7f21fe231740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 0x55f669984250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.654+0000 7f21f9bf4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669983d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.654+0000 7f21f9bf4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669983d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39616/0 (socket says 172.21.15.62:39616) 2024-09-11T23:59:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.654+0000 7f21f9bf4640 1 -- 172.21.15.62:0/3345066991 learned_addr learned my addr 172.21.15.62:0/3345066991 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21fe231740 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 0x55f669980ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41ae070 con 0x55f669970b60 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41af4e0 con 0x55f66997bd40 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21f93f3640 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 0x55f669984250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41ae740 con 0x55f669971480 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21fe231740 10 client.0 _ll_register_callbacks cb 0x7f21ec030f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21fa3f5640 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 0x55f669980ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21f93f3640 1 -- 172.21.15.62:0/3345066991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 msgr2=0x55f669980ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21f93f3640 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 0x55f669980ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21f93f3640 1 -- 172.21.15.62:0/3345066991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 msgr2=0x55f669983d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21f93f3640 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669983d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21f93f3640 1 -- 172.21.15.62:0/3345066991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f669981220 con 0x55f669971480 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21fa3f5640 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 0x55f669980ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669983d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21f93f3640 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 0x55f669984250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55f669978a90 tx=0x7f21e8011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f615c3cbc57fcb87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.655+0000 7f21d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:59:48.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.656+0000 7f21d97fa640 1 -- 172.21.15.62:0/3345066991 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e802a070 con 0x55f669971480 2024-09-11T23:59:48.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.656+0000 7f21d97fa640 1 -- 172.21.15.62:0/3345066991 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f21e8005440 con 0x55f669971480 2024-09-11T23:59:48.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.656+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e801f040 con 0x55f669971480 2024-09-11T23:59:48.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.656+0000 7f21d97fa640 1 -- 172.21.15.62:0/3345066991 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f21e80035f0 con 0x55f669971480 2024-09-11T23:59:48.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.657+0000 7f21fe231740 10 client.26314 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:48.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.657+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f6699f6f70 con 0x55f669971480 2024-09-11T23:59:48.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.657+0000 7f21d97fa640 1 -- 172.21.15.62:0/3345066991 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e803f480 con 0x55f669971480 2024-09-11T23:59:48.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.657+0000 7f21d97fa640 1 -- 172.21.15.62:0/3345066991 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f21e8024040 con 0x55f669971480 2024-09-11T23:59:48.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.657+0000 7f21fe231740 10 client.26314 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:48.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.658+0000 7f21fe231740 10 client.26314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:48.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.658+0000 7f21fe231740 20 client.26314 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:48.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.658+0000 7f21fe231740 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f66998bb40 0x55f6699f9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:48.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.658+0000 7f21fe231740 4 client.26314 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:59:48.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.658+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f21e801f040 con 0x55f66998bb40 2024-09-11T23:59:48.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.658+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f66998bb40 0x55f6699f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:48.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.658+0000 7f21f9bf4640 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f66998bb40 0x55f6699f9360 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d39d6b42b96ddb8 server_cookie=7a9fb75fad02e7c7 in_seq=0 out_seq=0 2024-09-11T23:59:48.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.658+0000 7f21d97fa640 10 client.26314 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:48.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.659+0000 7f21d97fa640 1 -- 172.21.15.62:0/3345066991 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f21e801f040 con 0x55f66998bb40 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.659+0000 7f21d97fa640 10 client.26314 handle_command_reply: tid=0 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.659+0000 7f21fe231740 1 client.26314 shutdown 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.659+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 msgr2=0x55f669984250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.659+0000 7f21fe231740 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 0x55f669984250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55f669978a90 tx=0x7f21e8011d10 comp rx=0 tx=0).stop 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.660+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 shutdown_connections 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.660+0000 7f21fe231740 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f66998bb40 0x55f6699f9360 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.660+0000 7f21fe231740 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66997bd40 0x55f669980ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.660+0000 7f21d97fa640 0 client.26314 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.660+0000 7f21fe231740 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f669971480 0x55f669984250 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.660+0000 7f21fe231740 1 --2- 172.21.15.62:0/3345066991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f669970b60 0x55f669983d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.660+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 >> 172.21.15.62:0/3345066991 conn(0x55f66996d0f0 msgr2=0x55f6699793a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.660+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 shutdown_connections 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.660+0000 7f21fe231740 1 -- 172.21.15.62:0/3345066991 wait complete. 2024-09-11T23:59:48.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.660+0000 7f21fe231740 20 client.26314 trim_cache size 0 max 16384 2024-09-11T23:59:48.662 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:59:48.662 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-11T23:59:48.662 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "3a7d8094-1e2f-490f-8cb8-4c640eedfde9", 2024-09-11T23:59:48.662 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-11T23:59:48.662 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:59:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 -- 172.21.15.62:0/1655771952 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f21dc0a2cf0 msgr2=0x7f21dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:48.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f21dc0a2cf0 0x7f21dc0a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009640 tx=0x7f21e4005f90 comp rx=0 tx=0).stop 2024-09-11T23:59:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 -- 172.21.15.62:0/1655771952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21f4102d90 msgr2=0x7f21f41a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21f4102d90 0x7f21f41a0460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f21ec000f80 tx=0x7f21ec031e60 comp rx=0 tx=0).stop 2024-09-11T23:59:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 -- 172.21.15.62:0/1655771952 shutdown_connections 2024-09-11T23:59:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f21dc0a2cf0 0x7f21dc0a51b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21f4108290 0x7f21f419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4103740 0x7f21f41a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 --2- 172.21.15.62:0/1655771952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21f4102d90 0x7f21f41a0460 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 -- 172.21.15.62:0/1655771952 >> 172.21.15.62:0/1655771952 conn(0x7f21f40fc7d0 msgr2=0x7f21f4100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 -- 172.21.15.62:0/1655771952 shutdown_connections 2024-09-11T23:59:48.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:48.661+0000 7f21fbe58640 1 -- 172.21.15.62:0/1655771952 wait complete. 2024-09-11T23:59:49.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:49 smithi062 ceph-mon[30958]: pgmap v236: 97 pgs: 97 active+clean; 4.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.8 MiB/s wr, 810 op/s 2024-09-11T23:59:49.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:49 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-11T23:59:49.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:49 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:49.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:49 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:59:49.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:49 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:59:49.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:49 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:49.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:49 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-11T23:59:49.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:49 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:49.528 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:59:49.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:49 smithi120 ceph-mon[35182]: pgmap v236: 97 pgs: 97 active+clean; 4.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.8 MiB/s wr, 810 op/s 2024-09-11T23:59:49.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:49 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-11T23:59:49.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:49 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:49.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:49 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:59:49.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:49 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:59:49.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:49 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:49.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:49 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-11T23:59:49.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:49 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:49 smithi083 ceph-mon[34805]: pgmap v236: 97 pgs: 97 active+clean; 4.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.8 MiB/s wr, 810 op/s 2024-09-11T23:59:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:49 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-11T23:59:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:49 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:49 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:59:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:49 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:59:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:49 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:49 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-11T23:59:49.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:49 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:49.916 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:50.349 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:50 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.478+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/3603119576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc081082a0 msgr2=0x7fbc0810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.478+0000 7fbc0ff7d640 1 --2- 172.21.15.62:0/3603119576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc081082a0 0x7fbc0810ec40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009cd0 tx=0x7fbc0002a910 comp rx=0 tx=0).stop 2024-09-11T23:59:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.478+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/3603119576 shutdown_connections 2024-09-11T23:59:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.478+0000 7fbc0ff7d640 1 --2- 172.21.15.62:0/3603119576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc081082a0 0x7fbc0810ec40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.478+0000 7fbc0ff7d640 1 --2- 172.21.15.62:0/3603119576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc081036e0 0x7fbc08107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.478+0000 7fbc0ff7d640 1 --2- 172.21.15.62:0/3603119576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc08102d30 0x7fbc08103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.478+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/3603119576 >> 172.21.15.62:0/3603119576 conn(0x7fbc080fc7d0 msgr2=0x7fbc080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.478+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/3603119576 shutdown_connections 2024-09-11T23:59:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.478+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/3603119576 wait complete. 2024-09-11T23:59:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0ff7d640 1 Processor -- start 2024-09-11T23:59:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0ff7d640 1 -- start start 2024-09-11T23:59:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0ff7d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc08102d30 0x7fbc081a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0ff7d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc081036e0 0x7fbc081a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0ff7d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc081082a0 0x7fbc0819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0ff7d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081120e0 con 0x7fbc081082a0 2024-09-11T23:59:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0ff7d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc08111f60 con 0x7fbc081036e0 2024-09-11T23:59:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0ff7d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc08112260 con 0x7fbc08102d30 2024-09-11T23:59:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0e51a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc081082a0 0x7fbc0819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0dd19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc08102d30 0x7fbc081a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0e51a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc081082a0 0x7fbc0819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39630/0 (socket says 172.21.15.62:39630) 2024-09-11T23:59:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0e51a640 1 -- 172.21.15.62:0/1807052206 learned_addr learned my addr 172.21.15.62:0/1807052206 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.479+0000 7fbc0d518640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc081036e0 0x7fbc081a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.480+0000 7fbc0e51a640 1 -- 172.21.15.62:0/1807052206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc08102d30 msgr2=0x7fbc081a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:50.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.480+0000 7fbc0e51a640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc08102d30 0x7fbc081a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.480+0000 7fbc0e51a640 1 -- 172.21.15.62:0/1807052206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc081036e0 msgr2=0x7fbc081a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:50.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.480+0000 7fbc0e51a640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc081036e0 0x7fbc081a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.480+0000 7fbc0e51a640 1 -- 172.21.15.62:0/1807052206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0819af00 con 0x7fbc081082a0 2024-09-11T23:59:50.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.481+0000 7fbc0e51a640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc081082a0 0x7fbc0819a690 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005e30 tx=0x7fbc00003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:50.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.481+0000 7fbbfeffd640 1 -- 172.21.15.62:0/1807052206 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00031660 con 0x7fbc081082a0 2024-09-11T23:59:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.481+0000 7fbbfeffd640 1 -- 172.21.15.62:0/1807052206 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc00031800 con 0x7fbc081082a0 2024-09-11T23:59:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.481+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/1807052206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0819b190 con 0x7fbc081082a0 2024-09-11T23:59:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.481+0000 7fbbfeffd640 1 -- 172.21.15.62:0/1807052206 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0002fb10 con 0x7fbc081082a0 2024-09-11T23:59:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.481+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/1807052206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc081a7540 con 0x7fbc081082a0 2024-09-11T23:59:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.485+0000 7fbc12356740 1 Processor -- start 2024-09-11T23:59:50.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.486+0000 7fbbfeffd640 1 -- 172.21.15.62:0/1807052206 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbc0002fcf0 con 0x7fbc081082a0 2024-09-11T23:59:50.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.486+0000 7fbc12356740 1 -- start start 2024-09-11T23:59:50.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.486+0000 7fbc12356740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f7c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.486+0000 7fbc12356740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f87140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.486+0000 7fbc12356740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 0x55b9f0f89a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.486+0000 7fbc12356740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b9f0ee22e0 con 0x55b9f0f7c4a0 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.486+0000 7fbc12356740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b9f0e89ce0 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.486+0000 7fbc12356740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b9f0ede2a0 con 0x55b9f0f87680 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0d518640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f7c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0d518640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f7c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39640/0 (socket says 172.21.15.62:39640) 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0d518640 1 -- 172.21.15.62:0/2281829187 learned_addr learned my addr 172.21.15.62:0/2281829187 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0d518640 1 --2- 172.21.15.62:0/2281829187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 0x55b9f0f89a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0dd19640 1 --2- 172.21.15.62:0/2281829187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f87140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbbfeffd640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbf40a2d70 0x7fbbf40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0d518640 1 -- 172.21.15.62:0/2281829187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7cdc0 msgr2=0x55b9f0f87140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0d518640 1 --2- 172.21.15.62:0/2281829187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f87140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0d518640 1 -- 172.21.15.62:0/2281829187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 msgr2=0x55b9f0f7c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0d518640 1 --2- 172.21.15.62:0/2281829187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f7c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0d518640 1 -- 172.21.15.62:0/2281829187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9f0e6e440 con 0x55b9f0f87680 2024-09-11T23:59:50.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0e51a640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbf40a2d70 0x7fbbf40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbc0d518640 1 --2- 172.21.15.62:0/2281829187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 0x55b9f0f89a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80176a0 tx=0x7fbbf80046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b6ed97a67554d2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:50.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.487+0000 7fbbfeffd640 1 -- 172.21.15.62:0/1807052206 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7fbc000f05d0 con 0x7fbc081082a0 2024-09-11T23:59:50.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.488+0000 7fbbf3fff640 1 -- 172.21.15.62:0/2281829187 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8015830 con 0x55b9f0f87680 2024-09-11T23:59:50.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.488+0000 7fbbf3fff640 1 -- 172.21.15.62:0/2281829187 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf801d070 con 0x55b9f0f87680 2024-09-11T23:59:50.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.488+0000 7fbbf3fff640 1 -- 172.21.15.62:0/2281829187 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8018b60 con 0x55b9f0f87680 2024-09-11T23:59:50.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.488+0000 7fbc0e51a640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbf40a2d70 0x7fbbf40a5230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbc000b9060 tx=0x7fbc000b8f20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:50.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.488+0000 7fbc12356740 1 -- 172.21.15.62:0/2281829187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 msgr2=0x55b9f0f89a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.488+0000 7fbc12356740 1 --2- 172.21.15.62:0/2281829187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 0x55b9f0f89a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80176a0 tx=0x7fbbf80046e0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.488+0000 7fbc12356740 1 -- 172.21.15.62:0/2281829187 shutdown_connections 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.488+0000 7fbc12356740 1 --2- 172.21.15.62:0/2281829187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 0x55b9f0f89a70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.489+0000 7fbc12356740 1 --2- 172.21.15.62:0/2281829187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f87140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.489+0000 7fbc12356740 1 --2- 172.21.15.62:0/2281829187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f7c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.489+0000 7fbc12356740 1 -- 172.21.15.62:0/2281829187 >> 172.21.15.62:0/2281829187 conn(0x55b9f0f78a30 msgr2=0x55b9f0f78e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.489+0000 7fbc12356740 1 -- 172.21.15.62:0/2281829187 shutdown_connections 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.489+0000 7fbc12356740 1 -- 172.21.15.62:0/2281829187 wait complete. 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.489+0000 7fbc12356740 1 -- start start 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.490+0000 7fbc12356740 1 Processor -- start 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.490+0000 7fbc12356740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f86ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc12356740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f842a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc12356740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f87680 0x55b9f0f84800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc12356740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b9f0f8a250 con 0x55b9f0f7c4a0 2024-09-11T23:59:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc12356740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b9f0e9d050 con 0x55b9f0f87680 2024-09-11T23:59:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc12356740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b9f0f8a3d0 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc12356740 10 client.0 _ll_register_callbacks cb 0x55b9f0edd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0dd19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f842a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0dd19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f842a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:55316/0 (socket says 172.21.15.62:55316) 2024-09-11T23:59:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0dd19640 1 -- 172.21.15.62:0/1129479944 learned_addr learned my addr 172.21.15.62:0/1129479944 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0d518640 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f86ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0dd19640 1 -- 172.21.15.62:0/1129479944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f87680 msgr2=0x55b9f0f84800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:50.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0d518640 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f87680 0x55b9f0f84800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0dd19640 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f87680 0x55b9f0f84800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0dd19640 1 -- 172.21.15.62:0/1129479944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 msgr2=0x55b9f0f86ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0dd19640 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f86ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0dd19640 1 -- 172.21.15.62:0/1129479944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9f0c533c0 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0d518640 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f87680 0x55b9f0f84800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbc0dd19640 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f842a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc040126d0 tx=0x7fbc04012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43bf60d083ddf9dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.491+0000 7fbbf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbbf17fa640 1 -- 172.21.15.62:0/1129479944 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04004050 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbbf17fa640 1 -- 172.21.15.62:0/1129479944 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7fbc040041f0 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbc12356740 1 -- 172.21.15.62:0/1129479944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf801d070 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbbf17fa640 1 -- 172.21.15.62:0/1129479944 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc040043f0 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbc12356740 10 client.26323 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbc12356740 1 -- 172.21.15.62:0/1129479944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b9f0f95cb0 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbbf17fa640 1 -- 172.21.15.62:0/1129479944 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf801d070 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbbf17fa640 1 -- 172.21.15.62:0/1129479944 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fbc04003c00 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbc12356740 10 client.26323 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:50.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbc12356740 10 client.26323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:50.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbc12356740 20 client.26323 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:50.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbc12356740 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9f0f97110 0x55b9f0f99530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbc12356740 4 client.26323 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:50.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.492+0000 7fbc12356740 1 -- 172.21.15.62:0/1129479944 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9f0eef850 con 0x55b9f0f97110 2024-09-11T23:59:50.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.493+0000 7fbc0d518640 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9f0f97110 0x55b9f0f99530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.494+0000 7fbbf17fa640 10 client.26323 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:50.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.494+0000 7fbc0d518640 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9f0f97110 0x55b9f0f99530 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73731cc1472c36 server_cookie=a4a9dff286b689b1 in_seq=0 out_seq=0 2024-09-11T23:59:50.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.495+0000 7fbbf17fa640 1 -- 172.21.15.62:0/1129479944 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9f0eef850 con 0x55b9f0f97110 2024-09-11T23:59:50.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.495+0000 7fbbf17fa640 10 client.26323 handle_command_reply: tid=0 2024-09-11T23:59:50.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.495+0000 7fbc12356740 1 client.26323 shutdown 2024-09-11T23:59:50.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.495+0000 7fbc12356740 1 -- 172.21.15.62:0/1129479944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 msgr2=0x55b9f0f842a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:50.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.495+0000 7fbc12356740 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f842a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc040126d0 tx=0x7fbc04012b90 comp rx=0 tx=0).stop 2024-09-11T23:59:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.496+0000 7fbc12356740 1 -- 172.21.15.62:0/1129479944 shutdown_connections 2024-09-11T23:59:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.496+0000 7fbc12356740 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9f0f97110 0x55b9f0f99530 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.496+0000 7fbc12356740 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f87680 0x55b9f0f84800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.496+0000 7fbc12356740 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f842a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.496+0000 7fbc12356740 1 --2- 172.21.15.62:0/1129479944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f86ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.496+0000 7fbc12356740 1 -- 172.21.15.62:0/1129479944 >> 172.21.15.62:0/1129479944 conn(0x55b9f0f78a30 msgr2=0x55b9f0f7e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.496+0000 7fbbf17fa640 0 client.26323 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.496+0000 7fbc12356740 1 -- 172.21.15.62:0/1129479944 shutdown_connections 2024-09-11T23:59:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.496+0000 7fbc12356740 1 -- 172.21.15.62:0/1129479944 wait complete. 2024-09-11T23:59:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.496+0000 7fbc12356740 20 client.26323 trim_cache size 0 max 16384 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.521+0000 7fbc12356740 1 Processor -- start 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc12356740 1 -- start start 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc12356740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f934e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc12356740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f7e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc12356740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 0x55b9f0f9ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc12356740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081b23d0 con 0x55b9f0f7c4a0 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc12356740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081afc70 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc12356740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081adae0 con 0x55b9f0f87680 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc0d518640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f934e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc0d518640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f934e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39656/0 (socket says 172.21.15.62:39656) 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc0d518640 1 -- 172.21.15.62:0/1324765464 learned_addr learned my addr 172.21.15.62:0/1324765464 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc0dd19640 1 --2- 172.21.15.62:0/1324765464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f7e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc0d518640 1 --2- 172.21.15.62:0/1324765464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 0x55b9f0f9ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc0d518640 1 -- 172.21.15.62:0/1324765464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 msgr2=0x55b9f0f9ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc0d518640 1 --2- 172.21.15.62:0/1324765464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 0x55b9f0f9ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc0d518640 1 -- 172.21.15.62:0/1324765464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7cdc0 msgr2=0x55b9f0f7e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc0d518640 1 --2- 172.21.15.62:0/1324765464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f7e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.522+0000 7fbc0d518640 1 -- 172.21.15.62:0/1324765464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9f0f827f0 con 0x55b9f0f7c4a0 2024-09-11T23:59:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.523+0000 7fbc0d518640 1 --2- 172.21.15.62:0/1324765464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f934e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80176a0 tx=0x7fbbf8009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc10a7ed62c1e140 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.523+0000 7fbbf37fe640 1 -- 172.21.15.62:0/1324765464 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8018bd0 con 0x55b9f0f7c4a0 2024-09-11T23:59:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.523+0000 7fbbf37fe640 1 -- 172.21.15.62:0/1324765464 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf8028030 con 0x55b9f0f7c4a0 2024-09-11T23:59:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.523+0000 7fbbf37fe640 1 -- 172.21.15.62:0/1324765464 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80039d0 con 0x55b9f0f7c4a0 2024-09-11T23:59:50.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 -- 172.21.15.62:0/1324765464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 msgr2=0x55b9f0f934e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:50.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 --2- 172.21.15.62:0/1324765464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f934e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80176a0 tx=0x7fbbf8009940 comp rx=0 tx=0).stop 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 -- 172.21.15.62:0/1324765464 shutdown_connections 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 --2- 172.21.15.62:0/1324765464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f87680 0x55b9f0f9ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 --2- 172.21.15.62:0/1324765464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f7e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 --2- 172.21.15.62:0/1324765464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f934e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 -- 172.21.15.62:0/1324765464 >> 172.21.15.62:0/1324765464 conn(0x55b9f0f78a30 msgr2=0x55b9f0f7def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 -- 172.21.15.62:0/1324765464 shutdown_connections 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 -- 172.21.15.62:0/1324765464 wait complete. 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 -- start start 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.524+0000 7fbc12356740 1 Processor -- start 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.525+0000 7fbc12356740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f8f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.525+0000 7fbc12356740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f8fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.526+0000 7fbc12356740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f87680 0x55b9f0f8c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.526+0000 7fbc0dd19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f8fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.526+0000 7fbc0dd19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f8fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:55332/0 (socket says 172.21.15.62:55332) 2024-09-11T23:59:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.526+0000 7fbc0dd19640 1 -- 172.21.15.62:0/4132816951 learned_addr learned my addr 172.21.15.62:0/4132816951 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.526+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081ae130 con 0x55b9f0f87680 2024-09-11T23:59:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.526+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081af5a0 con 0x55b9f0f7c4a0 2024-09-11T23:59:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.526+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081ae800 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.526+0000 7fbc12356740 10 client.0 _ll_register_callbacks cb 0x7fbc040097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.527+0000 7fbc0d518640 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f8f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.527+0000 7fbc0dd19640 1 -- 172.21.15.62:0/4132816951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7c4a0 msgr2=0x55b9f0f8f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.527+0000 7fbc0dd19640 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f8f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.527+0000 7fbc0dd19640 1 -- 172.21.15.62:0/4132816951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f87680 msgr2=0x55b9f0f8c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.527+0000 7fbc0dd19640 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f87680 0x55b9f0f8c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.527+0000 7fbc0dd19640 1 -- 172.21.15.62:0/4132816951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9f0f90280 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.528+0000 7fbc0dd19640 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f8fb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55b9f0f858e0 tx=0x7fbc04025dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fbb6364d0082ede server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.528+0000 7fbbf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:59:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.528+0000 7fbbf17fa640 1 -- 172.21.15.62:0/4132816951 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0402b070 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.528+0000 7fbbf17fa640 1 -- 172.21.15.62:0/4132816951 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7fbc0400fb30 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.528+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf8028030 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.528+0000 7fbbf17fa640 1 -- 172.21.15.62:0/4132816951 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc04005c50 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.528+0000 7fbc12356740 10 client.26329 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.528+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b9f100abe0 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.528+0000 7fbbf17fa640 1 -- 172.21.15.62:0/4132816951 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf8028030 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.529+0000 7fbbf17fa640 1 -- 172.21.15.62:0/4132816951 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fbc04004740 con 0x55b9f0f7cdc0 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.529+0000 7fbc12356740 10 client.26329 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.529+0000 7fbc12356740 10 client.26329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.529+0000 7fbc12356740 20 client.26329 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.529+0000 7fbc12356740 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9f100b3e0 0x55b9f100d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.529+0000 7fbc12356740 4 client.26329 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:59:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.529+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9f0f795e0 con 0x55b9f100b3e0 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.529+0000 7fbc0d518640 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9f100b3e0 0x55b9f100d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.529+0000 7fbbf17fa640 10 client.26329 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.529+0000 7fbc0d518640 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9f100b3e0 0x55b9f100d800 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b1796cc071f491 server_cookie=d81f4fd9feed1131 in_seq=0 out_seq=0 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbbf17fa640 1 -- 172.21.15.62:0/4132816951 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b9f0f795e0 con 0x55b9f100b3e0 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbbf17fa640 10 client.26329 handle_command_reply: tid=0 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 client.26329 shutdown 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 msgr2=0x55b9f0f8fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f8fb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55b9f0f858e0 tx=0x7fbc04025dc0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 shutdown_connections 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b9f100b3e0 0x55b9f100d800 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b9f0f87680 0x55b9f0f8c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9f0f7cdc0 0x55b9f0f8fb90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 --2- 172.21.15.62:0/4132816951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9f0f7c4a0 0x55b9f0f8f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbbf17fa640 0 client.26329 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 >> 172.21.15.62:0/4132816951 conn(0x55b9f0f78a30 msgr2=0x55b9f0f7e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 shutdown_connections 2024-09-11T23:59:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 1 -- 172.21.15.62:0/4132816951 wait complete. 2024-09-11T23:59:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.530+0000 7fbc12356740 20 client.26329 trim_cache size 0 max 16384 2024-09-11T23:59:50.534 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:59:50.534 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:59:50.534 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-11T23:59:50.534 INFO:teuthology.orchestra.run.smithi062.stdout: "3a7d8094-1e2f-490f-8cb8-4c640eedfde9": { 2024-09-11T23:59:50.534 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-11T23:59:50.535 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "3a7d8094-1e2f-490f-8cb8-4c640eedfde9", 2024-09-11T23:59:50.535 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-11T23:59:50.535 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:50.535 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:50.535 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:59:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/1807052206 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbf40a2d70 msgr2=0x7fbbf40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbf40a2d70 0x7fbbf40a5230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbc000b9060 tx=0x7fbc000b8f20 comp rx=0 tx=0).stop 2024-09-11T23:59:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/1807052206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc081082a0 msgr2=0x7fbc0819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc081082a0 0x7fbc0819a690 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005e30 tx=0x7fbc00003d10 comp rx=0 tx=0).stop 2024-09-11T23:59:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/1807052206 shutdown_connections 2024-09-11T23:59:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbf40a2d70 0x7fbbf40a5230 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc081082a0 0x7fbc0819a690 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc081036e0 0x7fbc081a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 --2- 172.21.15.62:0/1807052206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc08102d30 0x7fbc081a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:50.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/1807052206 >> 172.21.15.62:0/1807052206 conn(0x7fbc080fc7d0 msgr2=0x7fbc080fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:50.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/1807052206 shutdown_connections 2024-09-11T23:59:50.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:50.531+0000 7fbc0ff7d640 1 -- 172.21.15.62:0/1807052206 wait complete. 2024-09-11T23:59:50.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:50 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:50.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:50 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-11T23:59:51.278 INFO:tasks.cephfs.filesystem:scrub status for tag:3a7d8094-1e2f-490f-8cb8-4c640eedfde9 - {'path': '/', 'tag': '3a7d8094-1e2f-490f-8cb8-4c640eedfde9', 'options': 'recursive,force'} 2024-09-11T23:59:51.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:51 smithi120 ceph-mon[35182]: pgmap v237: 97 pgs: 97 active+clean; 4.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.7 MiB/s wr, 601 op/s 2024-09-11T23:59:51.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:51 smithi083 ceph-mon[34805]: pgmap v237: 97 pgs: 97 active+clean; 4.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.7 MiB/s wr, 601 op/s 2024-09-11T23:59:51.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:51 smithi062 ceph-mon[30958]: pgmap v237: 97 pgs: 97 active+clean; 4.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.7 MiB/s wr, 601 op/s 2024-09-11T23:59:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 430.0 s, 2464.4 tps, lat 40.637 ms stddev 42.646 2024-09-11T23:59:53.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:53 smithi120 ceph-mon[35182]: pgmap v238: 97 pgs: 97 active+clean; 4.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.5 MiB/s wr, 575 op/s 2024-09-11T23:59:53.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:53 smithi083 ceph-mon[34805]: pgmap v238: 97 pgs: 97 active+clean; 4.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.5 MiB/s wr, 575 op/s 2024-09-11T23:59:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:53 smithi062 ceph-mon[30958]: pgmap v238: 97 pgs: 97 active+clean; 4.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.5 MiB/s wr, 575 op/s 2024-09-11T23:59:55.566 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:59:55.567 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-11T23:59:55.567 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:59:55.598 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:55 smithi062 ceph-mon[30958]: pgmap v239: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.3 MiB/s wr, 850 op/s 2024-09-11T23:59:55.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:55 smithi120 ceph-mon[35182]: pgmap v239: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.3 MiB/s wr, 850 op/s 2024-09-11T23:59:55.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:55 smithi083 ceph-mon[34805]: pgmap v239: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.3 MiB/s wr, 850 op/s 2024-09-11T23:59:55.927 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:56.417 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:56 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:59:56.417 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:56 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:59:56.417 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:56 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-11T23:59:56.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.433+0000 7f22c1e06640 1 -- 172.21.15.62:0/1646235102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc102570 msgr2=0x7f22bc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:56.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.433+0000 7f22c1e06640 1 --2- 172.21.15.62:0/1646235102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc102570 0x7f22bc1029b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22b000ba20 tx=0x7f22b002c160 comp rx=0 tx=0).stop 2024-09-11T23:59:56.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.433+0000 7f22c1e06640 1 -- 172.21.15.62:0/1646235102 shutdown_connections 2024-09-11T23:59:56.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.433+0000 7f22c1e06640 1 --2- 172.21.15.62:0/1646235102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22bc102ef0 0x7f22bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:56.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.433+0000 7f22c1e06640 1 --2- 172.21.15.62:0/1646235102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc102570 0x7f22bc1029b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:56.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.433+0000 7f22c1e06640 1 --2- 172.21.15.62:0/1646235102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22bc100e20 0x7f22bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:56.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.433+0000 7f22c1e06640 1 -- 172.21.15.62:0/1646235102 >> 172.21.15.62:0/1646235102 conn(0x7f22bc0fc7d0 msgr2=0x7f22bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:56.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.433+0000 7f22c1e06640 1 -- 172.21.15.62:0/1646235102 shutdown_connections 2024-09-11T23:59:56.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.433+0000 7f22c1e06640 1 -- 172.21.15.62:0/1646235102 wait complete. 2024-09-11T23:59:56.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.433+0000 7f22c1e06640 1 Processor -- start 2024-09-11T23:59:56.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22c1e06640 1 -- start start 2024-09-11T23:59:56.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22c1e06640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc100e20 0x7f22bc1a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:56.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22c1e06640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22bc102570 0x7f22bc1a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:56.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22c1e06640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22bc102ef0 0x7f22bc19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:56.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22bb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc100e20 0x7f22bc1a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:56.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22bb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc100e20 0x7f22bc1a47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:52802/0 (socket says 172.21.15.62:52802) 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22bb7fe640 1 -- 172.21.15.62:0/2933991231 learned_addr learned my addr 172.21.15.62:0/2933991231 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22baffd640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22bc102570 0x7f22bc1a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc118690 con 0x7f22bc102570 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc118510 con 0x7f22bc100e20 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc118810 con 0x7f22bc102ef0 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.434+0000 7f22bbfff640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22bc102ef0 0x7f22bc19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22bb7fe640 1 -- 172.21.15.62:0/2933991231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22bc102ef0 msgr2=0x7f22bc19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22bb7fe640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22bc102ef0 0x7f22bc19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22bb7fe640 1 -- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22bc102570 msgr2=0x7f22bc1a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22bb7fe640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22bc102570 0x7f22bc1a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22bb7fe640 1 -- 172.21.15.62:0/2933991231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc19f0d0 con 0x7f22bc100e20 2024-09-11T23:59:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22baffd640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22bc102570 0x7f22bc1a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:56.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22bbfff640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22bc102ef0 0x7f22bc19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:56.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22bb7fe640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc100e20 0x7f22bc1a47e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00e9f0 tx=0x7f22ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:56.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22b8ff9640 1 -- 172.21.15.62:0/2933991231 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00cd80 con 0x7f22bc100e20 2024-09-11T23:59:56.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22b8ff9640 1 -- 172.21.15.62:0/2933991231 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22ac010910 con 0x7f22bc100e20 2024-09-11T23:59:56.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc19f3c0 con 0x7f22bc100e20 2024-09-11T23:59:56.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22b8ff9640 1 -- 172.21.15.62:0/2933991231 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac018990 con 0x7f22bc100e20 2024-09-11T23:59:56.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.435+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc1ab6b0 con 0x7f22bc100e20 2024-09-11T23:59:56.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.437+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc101200 con 0x7f22bc100e20 2024-09-11T23:59:56.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.438+0000 7f22b8ff9640 1 -- 172.21.15.62:0/2933991231 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f22ac010ac0 con 0x7f22bc100e20 2024-09-11T23:59:56.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.439+0000 7f22b8ff9640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f22a40a2dc0 0x7f22a40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:56.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.439+0000 7f22b8ff9640 1 -- 172.21.15.62:0/2933991231 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f22ac0cfc50 con 0x7f22bc100e20 2024-09-11T23:59:56.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.439+0000 7f22baffd640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f22a40a2dc0 0x7f22a40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:56.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.440+0000 7f22baffd640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f22a40a2dc0 0x7f22a40a5280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0fec10 tx=0x7f22b002e040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:56.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.443+0000 7f22b8ff9640 1 -- 172.21.15.62:0/2933991231 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22ac09c1c0 con 0x7f22bc100e20 2024-09-11T23:59:56.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:56 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:59:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:56 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:59:56.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:56 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-11T23:59:56.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:56 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:59:56.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:56 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:59:56.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:56 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-11T23:59:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.775+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22bc19fed0 con 0x7f22bc100e20 2024-09-11T23:59:56.778 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-11T23:59:56.778 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-09-11T23:57:32:617369+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-11T23:57:32.075842+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-11T23:59:56.778 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-09-11T23:59:56.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.777+0000 7f22b8ff9640 1 -- 172.21.15.62:0/2933991231 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11910 (secure 0 0 0) 0x7f22bc19fed0 con 0x7f22bc100e20 2024-09-11T23:59:56.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.783+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f22a40a2dc0 msgr2=0x7f22a40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:56.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.783+0000 7f22c1e06640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f22a40a2dc0 0x7f22a40a5280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0fec10 tx=0x7f22b002e040 comp rx=0 tx=0).stop 2024-09-11T23:59:56.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.783+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc100e20 msgr2=0x7f22bc1a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:56.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.783+0000 7f22c1e06640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc100e20 0x7f22bc1a47e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00e9f0 tx=0x7f22ac00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:59:56.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.783+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 shutdown_connections 2024-09-11T23:59:56.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.783+0000 7f22c1e06640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f22a40a2dc0 0x7f22a40a5280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:56.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.783+0000 7f22c1e06640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22bc102ef0 0x7f22bc19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:56.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.783+0000 7f22c1e06640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22bc102570 0x7f22bc1a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:56.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.783+0000 7f22c1e06640 1 --2- 172.21.15.62:0/2933991231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc100e20 0x7f22bc1a47e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:56.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.783+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 >> 172.21.15.62:0/2933991231 conn(0x7f22bc0fc7d0 msgr2=0x7f22bc104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:56.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.784+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 shutdown_connections 2024-09-11T23:59:56.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:56.784+0000 7f22c1e06640 1 -- 172.21.15.62:0/2933991231 wait complete. 2024-09-11T23:59:57.439 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:57 smithi062 ceph-mon[30958]: pgmap v240: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.6 MiB/s wr, 566 op/s 2024-09-11T23:59:57.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:57 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2933991231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 435.0 s, 2633.8 tps, lat 37.574 ms stddev 43.016 2024-09-11T23:59:57.484 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-11T23:59:57.485 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-11T23:59:57.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:57 smithi120 ceph-mon[35182]: pgmap v240: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.6 MiB/s wr, 566 op/s 2024-09-11T23:59:57.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:57 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2933991231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:57.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:57 smithi083 ceph-mon[34805]: pgmap v240: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.6 MiB/s wr, 566 op/s 2024-09-11T23:59:57.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:57 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2933991231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:59:57.948 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:59:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.532+0000 7f9184e5f640 1 -- 172.21.15.62:0/107642972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9180109830 msgr2=0x7f9180109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.532+0000 7f9184e5f640 1 --2- 172.21.15.62:0/107642972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9180109830 0x7f9180109c90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9178009ca0 tx=0x7f917802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.532+0000 7f9184e5f640 1 -- 172.21.15.62:0/107642972 shutdown_connections 2024-09-11T23:59:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.532+0000 7f9184e5f640 1 --2- 172.21.15.62:0/107642972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f918011b140 0x7f918011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.532+0000 7f9184e5f640 1 --2- 172.21.15.62:0/107642972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9180109830 0x7f9180109c90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.532+0000 7f9184e5f640 1 --2- 172.21.15.62:0/107642972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180109450 0x7f9180110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.532+0000 7f9184e5f640 1 -- 172.21.15.62:0/107642972 >> 172.21.15.62:0/107642972 conn(0x7f918006e140 msgr2=0x7f918006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.532+0000 7f9184e5f640 1 -- 172.21.15.62:0/107642972 shutdown_connections 2024-09-11T23:59:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.532+0000 7f9184e5f640 1 -- 172.21.15.62:0/107642972 wait complete. 2024-09-11T23:59:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.533+0000 7f9184e5f640 1 Processor -- start 2024-09-11T23:59:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.533+0000 7f9184e5f640 1 -- start start 2024-09-11T23:59:58.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f9184e5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180109450 0x7f91801a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f9184e5f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9180109830 0x7f91801a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f9184e5f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f918011b140 0x7f91801abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f9184e5f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9180120df0 con 0x7f9180109830 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f9184e5f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9180120c70 con 0x7f9180109450 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f9184e5f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9180120f70 con 0x7f918011b140 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f917dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9180109830 0x7f91801a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f917dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9180109830 0x7f91801a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36332/0 (socket says 172.21.15.62:36332) 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f917dd9b640 1 -- 172.21.15.62:0/3832153689 learned_addr learned my addr 172.21.15.62:0/3832153689 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f917ed9d640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f918011b140 0x7f91801abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f917e59c640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180109450 0x7f91801a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f917dd9b640 1 -- 172.21.15.62:0/3832153689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f918011b140 msgr2=0x7f91801abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.534+0000 7f917dd9b640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f918011b140 0x7f91801abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.535+0000 7f917dd9b640 1 -- 172.21.15.62:0/3832153689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180109450 msgr2=0x7f91801a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.535+0000 7f917dd9b640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180109450 0x7f91801a72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.535+0000 7f917dd9b640 1 -- 172.21.15.62:0/3832153689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91801ac2a0 con 0x7f9180109830 2024-09-11T23:59:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.535+0000 7f917ed9d640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f918011b140 0x7f91801abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.535+0000 7f917dd9b640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9180109830 0x7f91801a7830 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9178005e30 tx=0x7f9178003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.535+0000 7f916f7fe640 1 -- 172.21.15.62:0/3832153689 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178031650 con 0x7f9180109830 2024-09-11T23:59:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.535+0000 7f916f7fe640 1 -- 172.21.15.62:0/3832153689 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91780317f0 con 0x7f9180109830 2024-09-11T23:59:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.535+0000 7f9184e5f640 1 -- 172.21.15.62:0/3832153689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91801ac530 con 0x7f9180109830 2024-09-11T23:59:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.536+0000 7f9184e5f640 1 -- 172.21.15.62:0/3832153689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91801b3e70 con 0x7f9180109830 2024-09-11T23:59:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.536+0000 7f916f7fe640 1 -- 172.21.15.62:0/3832153689 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917802fa30 con 0x7f9180109830 2024-09-11T23:59:58.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.538+0000 7f916f7fe640 1 -- 172.21.15.62:0/3832153689 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f917802fc30 con 0x7f9180109830 2024-09-11T23:59:58.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.539+0000 7f9187238740 1 Processor -- start 2024-09-11T23:59:58.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.541+0000 7f916f7fe640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91680a2cf0 0x7f91680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.541+0000 7f916f7fe640 1 -- 172.21.15.62:0/3832153689 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f91780f0630 con 0x7f9180109830 2024-09-11T23:59:58.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.542+0000 7f917ed9d640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91680a2cf0 0x7f91680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.542+0000 7f9187238740 1 -- start start 2024-09-11T23:59:58.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.542+0000 7f9187238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed1760 0x559648ed1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.542+0000 7f9187238740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed2080 0x559648edc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.542+0000 7f9187238740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.542+0000 7f9187238740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559648e372a0 con 0x559648edc940 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.542+0000 7f9187238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559648ddeb10 con 0x559648ed1760 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.542+0000 7f9187238740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559648e33260 con 0x559648ed2080 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.542+0000 7f917ed9d640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91680a2cf0 0x7f91680a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9174009f30 tx=0x7f9174005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed1760 0x559648ed1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed1760 0x559648ed1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:52832/0 (socket says 172.21.15.62:52832) 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917e59c640 1 -- 172.21.15.62:0/2499845164 learned_addr learned my addr 172.21.15.62:0/2499845164 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917dd9b640 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed2080 0x559648edc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917e59c640 1 -- 172.21.15.62:0/2499845164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed2080 msgr2=0x559648edc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917ed9d640 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917e59c640 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed2080 0x559648edc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917e59c640 1 -- 172.21.15.62:0/2499845164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 msgr2=0x559648eded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917e59c640 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eded30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917e59c640 1 -- 172.21.15.62:0/2499845164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559648ba8b90 con 0x559648ed1760 2024-09-11T23:59:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917ed9d640 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917dd9b640 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed2080 0x559648edc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:59:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.543+0000 7f917e59c640 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed1760 0x559648ed1b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9170009cf0 tx=0x7f9170011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aec3d1cab91ebd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.544+0000 7f916cff9640 1 -- 172.21.15.62:0/2499845164 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170004230 con 0x559648ed1760 2024-09-11T23:59:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.544+0000 7f916cff9640 1 -- 172.21.15.62:0/2499845164 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91700043d0 con 0x559648ed1760 2024-09-11T23:59:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.544+0000 7f916cff9640 1 -- 172.21.15.62:0/2499845164 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170019920 con 0x559648ed1760 2024-09-11T23:59:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.545+0000 7f9187238740 1 -- 172.21.15.62:0/2499845164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed1760 msgr2=0x559648ed1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.545+0000 7f9187238740 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed1760 0x559648ed1b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9170009cf0 tx=0x7f9170011810 comp rx=0 tx=0).stop 2024-09-11T23:59:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.545+0000 7f9187238740 1 -- 172.21.15.62:0/2499845164 shutdown_connections 2024-09-11T23:59:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.545+0000 7f9187238740 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.545+0000 7f9187238740 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed2080 0x559648edc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.545+0000 7f9187238740 1 --2- 172.21.15.62:0/2499845164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed1760 0x559648ed1b40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.545+0000 7f9187238740 1 -- 172.21.15.62:0/2499845164 >> 172.21.15.62:0/2499845164 conn(0x559648ecdcf0 msgr2=0x559648ece0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.545+0000 7f9187238740 1 -- 172.21.15.62:0/2499845164 shutdown_connections 2024-09-11T23:59:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.545+0000 7f9187238740 1 -- 172.21.15.62:0/2499845164 wait complete. 2024-09-11T23:59:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f9187238740 1 -- start start 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f9187238740 1 Processor -- start 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f9187238740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed1760 0x559648ed9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f9187238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 0x559648ed9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f9187238740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eda1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f9187238740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559648edf510 con 0x559648edc940 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f9187238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559648dbf6e0 con 0x559648ed2080 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f9187238740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559648edf690 con 0x559648ed1760 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f9187238740 10 client.0 _ll_register_callbacks cb 0x559648de3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f917ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eda1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f917ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eda1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36358/0 (socket says 172.21.15.62:36358) 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f917ed9d640 1 -- 172.21.15.62:0/628522139 learned_addr learned my addr 172.21.15.62:0/628522139 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f917dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 0x559648ed9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.546+0000 7f917e59c640 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed1760 0x559648ed9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f917e59c640 1 -- 172.21.15.62:0/628522139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 msgr2=0x559648ed9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f917e59c640 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 0x559648ed9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f917e59c640 1 -- 172.21.15.62:0/628522139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 msgr2=0x559648eda1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f917e59c640 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eda1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f917e59c640 1 -- 172.21.15.62:0/628522139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559648ee8980 con 0x559648ed1760 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f917ed9d640 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eda1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f917e59c640 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed1760 0x559648ed9740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9170009cf0 tx=0x7f9170003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f23097dedfc9a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f9165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f9165ffb640 1 -- 172.21.15.62:0/628522139 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170027070 con 0x559648ed1760 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f9187238740 1 -- 172.21.15.62:0/628522139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91700043d0 con 0x559648ed1760 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f9165ffb640 1 -- 172.21.15.62:0/628522139 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f9170005240 con 0x559648ed1760 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f9165ffb640 1 -- 172.21.15.62:0/628522139 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9170033050 con 0x559648ed1760 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f9187238740 10 client.26344 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.547+0000 7f9187238740 1 -- 172.21.15.62:0/628522139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559648eeb220 con 0x559648ed1760 2024-09-11T23:59:58.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.548+0000 7f9165ffb640 1 -- 172.21.15.62:0/628522139 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91700043d0 con 0x559648ed1760 2024-09-11T23:59:58.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.548+0000 7f9165ffb640 1 -- 172.21.15.62:0/628522139 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f9170004ba0 con 0x559648ed1760 2024-09-11T23:59:58.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.548+0000 7f9187238740 10 client.26344 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:58.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.548+0000 7f9187238740 10 client.26344 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:59:58.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.548+0000 7f9187238740 20 client.26344 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:58.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.548+0000 7f9187238740 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559648eec630 0x559648eeea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.548+0000 7f9187238740 4 client.26344 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:59:58.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.548+0000 7f9187238740 1 -- 172.21.15.62:0/628522139 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559648e449d0 con 0x559648eec630 2024-09-11T23:59:58.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.549+0000 7f917dd9b640 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559648eec630 0x559648eeea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.550+0000 7f917dd9b640 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559648eec630 0x559648eeea20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537e5c8ccbe44400 server_cookie=97eb24873dac4d0b in_seq=0 out_seq=0 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.550+0000 7f9165ffb640 10 client.26344 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.551+0000 7f9165ffb640 1 -- 172.21.15.62:0/628522139 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559648e449d0 con 0x559648eec630 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.551+0000 7f9165ffb640 10 client.26344 handle_command_reply: tid=0 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.551+0000 7f9187238740 1 client.26344 shutdown 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.551+0000 7f9187238740 1 -- 172.21.15.62:0/628522139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed1760 msgr2=0x559648ed9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.551+0000 7f9187238740 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed1760 0x559648ed9740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9170009cf0 tx=0x7f9170003820 comp rx=0 tx=0).stop 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.552+0000 7f9187238740 1 -- 172.21.15.62:0/628522139 shutdown_connections 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.552+0000 7f9187238740 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559648eec630 0x559648eeea20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.552+0000 7f9187238740 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edc940 0x559648eda1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.552+0000 7f9187238740 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 0x559648ed9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.552+0000 7f9187238740 1 --2- 172.21.15.62:0/628522139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648ed1760 0x559648ed9740 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9170009cf0 tx=0x7f9170003820 comp rx=0 tx=0).stop 2024-09-11T23:59:58.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.552+0000 7f9187238740 1 -- 172.21.15.62:0/628522139 >> 172.21.15.62:0/628522139 conn(0x559648ecdcf0 msgr2=0x559648ed39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:58.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.552+0000 7f9165ffb640 0 client.26344 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:58.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.552+0000 7f9187238740 1 -- 172.21.15.62:0/628522139 shutdown_connections 2024-09-11T23:59:58.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.552+0000 7f9187238740 1 -- 172.21.15.62:0/628522139 wait complete. 2024-09-11T23:59:58.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.552+0000 7f9187238740 20 client.26344 trim_cache size 0 max 16384 2024-09-11T23:59:58.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f9187238740 1 Processor -- start 2024-09-11T23:59:58.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f9187238740 1 -- start start 2024-09-11T23:59:58.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f9187238740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648ed2080 0x559648ed41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f9187238740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 0x559648ef4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f9187238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648edb6b0 0x559648ef4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f9187238740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91801bf020 con 0x559648ed2080 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f9187238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91801bc8c0 con 0x559648edb6b0 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f9187238740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91801ba730 con 0x559648edc940 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f917e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648ed2080 0x559648ed41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f917e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648ed2080 0x559648ed41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36370/0 (socket says 172.21.15.62:36370) 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f917e59c640 1 -- 172.21.15.62:0/1122999239 learned_addr learned my addr 172.21.15.62:0/1122999239 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f917dd9b640 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 0x559648ef4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.574+0000 7f917ed9d640 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648edb6b0 0x559648ef4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f917e59c640 1 -- 172.21.15.62:0/1122999239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 msgr2=0x559648ef4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f917e59c640 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 0x559648ef4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f917e59c640 1 -- 172.21.15.62:0/1122999239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648edb6b0 msgr2=0x559648ef4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f917e59c640 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648edb6b0 0x559648ef4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f917e59c640 1 -- 172.21.15.62:0/1122999239 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559648ecdb20 con 0x559648ed2080 2024-09-11T23:59:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f917ed9d640 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648edb6b0 0x559648ef4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f917dd9b640 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 0x559648ef4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:59:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f917e59c640 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648ed2080 0x559648ed41c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9170000c00 tx=0x7f9170011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3d78c0bc91758d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f9167fff640 1 -- 172.21.15.62:0/1122999239 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170027070 con 0x559648ed2080 2024-09-11T23:59:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f9167fff640 1 -- 172.21.15.62:0/1122999239 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9170009910 con 0x559648ed2080 2024-09-11T23:59:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.575+0000 7f9167fff640 1 -- 172.21.15.62:0/1122999239 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170004650 con 0x559648ed2080 2024-09-11T23:59:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.576+0000 7f9187238740 1 -- 172.21.15.62:0/1122999239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648ed2080 msgr2=0x559648ed41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.576+0000 7f9187238740 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648ed2080 0x559648ed41c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9170000c00 tx=0x7f9170011810 comp rx=0 tx=0).stop 2024-09-11T23:59:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.576+0000 7f9187238740 1 -- 172.21.15.62:0/1122999239 shutdown_connections 2024-09-11T23:59:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.576+0000 7f9187238740 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648edb6b0 0x559648ef4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.576+0000 7f9187238740 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 0x559648ef4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.576+0000 7f9187238740 1 --2- 172.21.15.62:0/1122999239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648ed2080 0x559648ed41c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.576+0000 7f9187238740 1 -- 172.21.15.62:0/1122999239 >> 172.21.15.62:0/1122999239 conn(0x559648ecdcf0 msgr2=0x559648edafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.576+0000 7f9187238740 1 -- 172.21.15.62:0/1122999239 shutdown_connections 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.576+0000 7f9187238740 1 -- 172.21.15.62:0/1122999239 wait complete. 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.577+0000 7f9187238740 1 -- start start 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.577+0000 7f9187238740 1 Processor -- start 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f9187238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 0x559648ee48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f9187238740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edb6b0 0x559648ee4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f9187238740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 0x559648edf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f917e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 0x559648ee48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f917e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 0x559648ee48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:52852/0 (socket says 172.21.15.62:52852) 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f917e59c640 1 -- 172.21.15.62:0/2787780562 learned_addr learned my addr 172.21.15.62:0/2787780562 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-11T23:59:58.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91801bad80 con 0x559648edb6b0 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91801bc1f0 con 0x559648ed2080 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91801bb450 con 0x559648edc940 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f917dd9b640 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edb6b0 0x559648ee4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f9187238740 10 client.0 _ll_register_callbacks cb 0x7f9170005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.578+0000 7f917ed9d640 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 0x559648edf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f917e59c640 1 -- 172.21.15.62:0/2787780562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 msgr2=0x559648edf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f917e59c640 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 0x559648edf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f917e59c640 1 -- 172.21.15.62:0/2787780562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edb6b0 msgr2=0x559648ee4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f917e59c640 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edb6b0 0x559648ee4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f917e59c640 1 -- 172.21.15.62:0/2787780562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559648ec9260 con 0x559648ed2080 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f917ed9d640 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 0x559648edf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f917e59c640 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 0x559648ee48f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9170011d10 tx=0x7f9170003e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=934e167f8a1821ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:59:58.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f9165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-11T23:59:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f9165ffb640 1 -- 172.21.15.62:0/2787780562 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170027070 con 0x559648ed2080 2024-09-11T23:59:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f9165ffb640 1 -- 172.21.15.62:0/2787780562 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8513+0+0 (secure 0 0 0) 0x7f9170005450 con 0x559648ed2080 2024-09-11T23:59:58.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.579+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9170009910 con 0x559648ed2080 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9165ffb640 1 -- 172.21.15.62:0/2787780562 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9170005650 con 0x559648ed2080 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9187238740 10 client.26393 fetch_fsmap learned FSMap version 30 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559648f577f0 con 0x559648ed2080 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9165ffb640 1 -- 172.21.15.62:0/2787780562 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9170009910 con 0x559648ed2080 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9165ffb640 1 -- 172.21.15.62:0/2787780562 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f917002c560 con 0x559648ed2080 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9187238740 10 client.26393 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9187238740 10 client.26393 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9187238740 20 client.26393 populate_metadata read hostname 'smithi062' 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9187238740 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559648f57eb0 0x559648f5a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9187238740 4 client.26393 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.581+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x559648e3ed20 con 0x559648f57eb0 2024-09-11T23:59:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.582+0000 7f917dd9b640 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559648f57eb0 0x559648f5a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:59:58.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.582+0000 7f9165ffb640 10 client.26393 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:58.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.582+0000 7f917dd9b640 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559648f57eb0 0x559648f5a2a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b85015224210e3c server_cookie=b506b88a4462fe6a in_seq=0 out_seq=0 2024-09-11T23:59:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9165ffb640 1 -- 172.21.15.62:0/2787780562 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559648e3ed20 con 0x559648f57eb0 2024-09-11T23:59:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9165ffb640 10 client.26393 handle_command_reply: tid=0 2024-09-11T23:59:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 client.26393 shutdown 2024-09-11T23:59:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 msgr2=0x559648ee48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 0x559648ee48f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9170011d10 tx=0x7f9170003e50 comp rx=0 tx=0).stop 2024-09-11T23:59:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 shutdown_connections 2024-09-11T23:59:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559648f57eb0 0x559648f5a2a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559648edc940 0x559648edf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9165ffb640 0 client.26393 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-11T23:59:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559648edb6b0 0x559648ee4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 --2- 172.21.15.62:0/2787780562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559648ed2080 0x559648ee48f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 >> 172.21.15.62:0/2787780562 conn(0x559648ecdcf0 msgr2=0x559648ed96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 shutdown_connections 2024-09-11T23:59:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.855+0000 7f9187238740 1 -- 172.21.15.62:0/2787780562 wait complete. 2024-09-11T23:59:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9187238740 20 client.26393 trim_cache size 0 max 16384 2024-09-11T23:59:58.890 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-11T23:59:58.890 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-09-11T23:59:58.890 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-11T23:59:58.890 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 30, 2024-09-11T23:59:58.890 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-11T23:59:58.890 INFO:teuthology.orchestra.run.smithi062.stdout: "2615c871": { 2024-09-11T23:59:58.890 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 30, 2024-09-11T23:59:58.890 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-11T23:59:58.890 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:59:58.890 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 139.4, 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:58.891 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:58.892 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:58.892 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-11T23:59:58.892 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-11T23:59:58.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 -- 172.21.15.62:0/3832153689 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91680a2cf0 msgr2=0x7f91680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:58.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91680a2cf0 0x7f91680a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9174009f30 tx=0x7f9174005f90 comp rx=0 tx=0).stop 2024-09-11T23:59:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 -- 172.21.15.62:0/3832153689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9180109830 msgr2=0x7f91801a7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:59:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9180109830 0x7f91801a7830 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9178005e30 tx=0x7f9178003f90 comp rx=0 tx=0).stop 2024-09-11T23:59:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 -- 172.21.15.62:0/3832153689 shutdown_connections 2024-09-11T23:59:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91680a2cf0 0x7f91680a51b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f918011b140 0x7f91801abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9180109830 0x7f91801a7830 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 --2- 172.21.15.62:0/3832153689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180109450 0x7f91801a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:59:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 -- 172.21.15.62:0/3832153689 >> 172.21.15.62:0/3832153689 conn(0x7f918006e140 msgr2=0x7f918011be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:59:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 -- 172.21.15.62:0/3832153689 shutdown_connections 2024-09-11T23:59:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-11T23:59:58.856+0000 7f9184e5f640 1 -- 172.21.15.62:0/3832153689 wait complete. 2024-09-11T23:59:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 11 23:59:59 smithi062 ceph-mon[30958]: pgmap v241: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.3 MiB/s wr, 768 op/s 2024-09-11T23:59:59.513 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2615c871', quiesce took 3.9 seconds. Will release after: 15.499999999999998 2024-09-11T23:59:59.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 11 23:59:59 smithi120 ceph-mon[35182]: pgmap v241: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.3 MiB/s wr, 768 op/s 2024-09-11T23:59:59.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 11 23:59:59 smithi083 ceph-mon[34805]: pgmap v241: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.3 MiB/s wr, 768 op/s 2024-09-12T00:00:00.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:00 smithi120 ceph-mon[35182]: overall HEALTH_OK 2024-09-12T00:00:00.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:00 smithi083 ceph-mon[34805]: overall HEALTH_OK 2024-09-12T00:00:00.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:00 smithi062 ceph-mon[30958]: overall HEALTH_OK 2024-09-12T00:00:01.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:01 smithi062 ceph-mon[30958]: pgmap v242: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 621 op/s 2024-09-12T00:00:01.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:01 smithi120 ceph-mon[35182]: pgmap v242: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 621 op/s 2024-09-12T00:00:01.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:01 smithi083 ceph-mon[34805]: pgmap v242: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 621 op/s 2024-09-12T00:00:02.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 440.0 s, 470.6 tps, lat 51.381 ms stddev 55.226 2024-09-12T00:00:03.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:03 smithi120 ceph-mon[35182]: pgmap v243: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2024-09-12T00:00:03.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:03 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:00:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:03 smithi083 ceph-mon[34805]: pgmap v243: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2024-09-12T00:00:03.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:03 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:00:03.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:03 smithi062 ceph-mon[30958]: pgmap v243: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2024-09-12T00:00:03.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:03 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:00:05.585 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:05 smithi120 ceph-mon[35182]: pgmap v244: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 722 op/s 2024-09-12T00:00:05.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:05 smithi083 ceph-mon[34805]: pgmap v244: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 722 op/s 2024-09-12T00:00:05.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:05 smithi062 ceph-mon[30958]: pgmap v244: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 722 op/s 2024-09-12T00:00:07.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 445.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:00:07.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:07 smithi120 ceph-mon[35182]: pgmap v245: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.3 MiB/s wr, 436 op/s 2024-09-12T00:00:07.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:07 smithi120 ceph-mon[35182]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T00:00:07.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:07 smithi083 ceph-mon[34805]: pgmap v245: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.3 MiB/s wr, 436 op/s 2024-09-12T00:00:07.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:07 smithi083 ceph-mon[34805]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T00:00:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:07 smithi062 ceph-mon[30958]: pgmap v245: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.3 MiB/s wr, 436 op/s 2024-09-12T00:00:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:07 smithi062 ceph-mon[30958]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T00:00:09.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:09 smithi120 ceph-mon[35182]: pgmap v247: 97 pgs: 97 active+clean; 3.8 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-09-12T00:00:09.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:09 smithi083 ceph-mon[34805]: pgmap v247: 97 pgs: 97 active+clean; 3.8 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-09-12T00:00:09.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:09 smithi062 ceph-mon[30958]: pgmap v247: 97 pgs: 97 active+clean; 3.8 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-09-12T00:00:11.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:11 smithi120 ceph-mon[35182]: pgmap v248: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-09-12T00:00:11.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:11 smithi120 ceph-mon[35182]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T00:00:11.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:11 smithi083 ceph-mon[34805]: pgmap v248: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-09-12T00:00:11.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:11 smithi083 ceph-mon[34805]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T00:00:11.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:11 smithi062 ceph-mon[30958]: pgmap v248: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-09-12T00:00:11.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:11 smithi062 ceph-mon[30958]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T00:00:12.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 450.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:00:13.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:13 smithi062 ceph-mon[30958]: pgmap v250: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 485 KiB/s wr, 33 op/s 2024-09-12T00:00:13.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:13 smithi120 ceph-mon[35182]: pgmap v250: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 485 KiB/s wr, 33 op/s 2024-09-12T00:00:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:13 smithi083 ceph-mon[34805]: pgmap v250: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 485 KiB/s wr, 33 op/s 2024-09-12T00:00:15.015 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:00:15.278 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:15.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:15 smithi120 ceph-mon[35182]: pgmap v251: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.7 MiB/s wr, 238 op/s 2024-09-12T00:00:15.638 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:15 smithi062 ceph-mon[30958]: pgmap v251: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.7 MiB/s wr, 238 op/s 2024-09-12T00:00:15.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:15 smithi083 ceph-mon[34805]: pgmap v251: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.7 MiB/s wr, 238 op/s 2024-09-12T00:00:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.723+0000 7efc414cc640 1 -- 172.21.15.62:0/3458392185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc3c102570 msgr2=0x7efc3c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.723+0000 7efc414cc640 1 --2- 172.21.15.62:0/3458392185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc3c102570 0x7efc3c1029b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efc24009900 tx=0x7efc2402a480 comp rx=0 tx=0).stop 2024-09-12T00:00:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.723+0000 7efc414cc640 1 -- 172.21.15.62:0/3458392185 shutdown_connections 2024-09-12T00:00:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.723+0000 7efc414cc640 1 --2- 172.21.15.62:0/3458392185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c102ef0 0x7efc3c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.723+0000 7efc414cc640 1 --2- 172.21.15.62:0/3458392185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc3c102570 0x7efc3c1029b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.723+0000 7efc414cc640 1 --2- 172.21.15.62:0/3458392185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc3c100e20 0x7efc3c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.723+0000 7efc414cc640 1 -- 172.21.15.62:0/3458392185 >> 172.21.15.62:0/3458392185 conn(0x7efc3c0fc7d0 msgr2=0x7efc3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.723+0000 7efc414cc640 1 -- 172.21.15.62:0/3458392185 shutdown_connections 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.723+0000 7efc414cc640 1 -- 172.21.15.62:0/3458392185 wait complete. 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.723+0000 7efc414cc640 1 Processor -- start 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc414cc640 1 -- start start 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc414cc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc3c100e20 0x7efc3c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc414cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c102570 0x7efc3c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc414cc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc3c102ef0 0x7efc3c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc414cc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c110f50 con 0x7efc3c100e20 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc414cc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c110dd0 con 0x7efc3c102570 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc414cc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c1110d0 con 0x7efc3c102ef0 2024-09-12T00:00:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc3affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc3c100e20 0x7efc3c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc3affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc3c100e20 0x7efc3c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45910/0 (socket says 172.21.15.62:45910) 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc3affd640 1 -- 172.21.15.62:0/1958615505 learned_addr learned my addr 172.21.15.62:0/1958615505 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc3a7fc640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c102570 0x7efc3c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc3affd640 1 -- 172.21.15.62:0/1958615505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc3c102ef0 msgr2=0x7efc3c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc3affd640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc3c102ef0 0x7efc3c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc3affd640 1 -- 172.21.15.62:0/1958615505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c102570 msgr2=0x7efc3c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc3affd640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c102570 0x7efc3c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc3affd640 1 -- 172.21.15.62:0/1958615505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c1a68c0 con 0x7efc3c100e20 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.724+0000 7efc3a7fc640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c102570 0x7efc3c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.725+0000 7efc3affd640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc3c100e20 0x7efc3c0fcc80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7efc3400cce0 tx=0x7efc3400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.725+0000 7efc33fff640 1 -- 172.21.15.62:0/1958615505 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3400f650 con 0x7efc3c100e20 2024-09-12T00:00:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.725+0000 7efc33fff640 1 -- 172.21.15.62:0/1958615505 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3400fcd0 con 0x7efc3c100e20 2024-09-12T00:00:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.725+0000 7efc414cc640 1 -- 172.21.15.62:0/1958615505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c1a6bb0 con 0x7efc3c100e20 2024-09-12T00:00:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.725+0000 7efc33fff640 1 -- 172.21.15.62:0/1958615505 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34018c70 con 0x7efc3c100e20 2024-09-12T00:00:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.725+0000 7efc414cc640 1 -- 172.21.15.62:0/1958615505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c1a70f0 con 0x7efc3c100e20 2024-09-12T00:00:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.726+0000 7efc414cc640 1 -- 172.21.15.62:0/1958615505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c101200 con 0x7efc3c100e20 2024-09-12T00:00:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.727+0000 7efc33fff640 1 -- 172.21.15.62:0/1958615505 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efc3400f7f0 con 0x7efc3c100e20 2024-09-12T00:00:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.728+0000 7efc33fff640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efc280a2cf0 0x7efc280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.728+0000 7efc3a7fc640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efc280a2cf0 0x7efc280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.728+0000 7efc33fff640 1 -- 172.21.15.62:0/1958615505 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7efc340cf9c0 con 0x7efc3c100e20 2024-09-12T00:00:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.728+0000 7efc3a7fc640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efc280a2cf0 0x7efc280a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0fe710 tx=0x7efc24005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.732+0000 7efc33fff640 1 -- 172.21.15.62:0/1958615505 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc340d1e20 con 0x7efc3c100e20 2024-09-12T00:00:15.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.946+0000 7efc414cc640 1 -- 172.21.15.62:0/1958615505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc3c0fe1b0 con 0x7efc3c100e20 2024-09-12T00:00:15.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.946+0000 7efc33fff640 1 -- 172.21.15.62:0/1958615505 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11909 (secure 0 0 0) 0x7efc34014030 con 0x7efc3c100e20 2024-09-12T00:00:15.946 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:00:15.947 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"btime":"2024-09-12T00:00:02:308078+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:00:02.079738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:00:15.947 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2024-09-12T00:00:15.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 -- 172.21.15.62:0/1958615505 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efc280a2cf0 msgr2=0x7efc280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efc280a2cf0 0x7efc280a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0fe710 tx=0x7efc24005a20 comp rx=0 tx=0).stop 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 -- 172.21.15.62:0/1958615505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc3c100e20 msgr2=0x7efc3c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc3c100e20 0x7efc3c0fcc80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7efc3400cce0 tx=0x7efc3400a620 comp rx=0 tx=0).stop 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 -- 172.21.15.62:0/1958615505 shutdown_connections 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efc280a2cf0 0x7efc280a51b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc3c102ef0 0x7efc3c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c102570 0x7efc3c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 --2- 172.21.15.62:0/1958615505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc3c100e20 0x7efc3c0fcc80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 -- 172.21.15.62:0/1958615505 >> 172.21.15.62:0/1958615505 conn(0x7efc3c0fc7d0 msgr2=0x7efc3c06a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 -- 172.21.15.62:0/1958615505 shutdown_connections 2024-09-12T00:00:15.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:15.949+0000 7efc414cc640 1 -- 172.21.15.62:0/1958615505 wait complete. 2024-09-12T00:00:16.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:16 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1958615505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:16.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:16 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1958615505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:16.711 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:16 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1958615505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:16.743 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 2615c871 --release --await' 2024-09-12T00:00:16.743 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 2615c871 --release --await 2024-09-12T00:00:17.049 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 455.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:00:17.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:17 smithi062 ceph-mon[30958]: pgmap v252: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.2 MiB/s wr, 205 op/s 2024-09-12T00:00:17.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.504+0000 7f3afc564640 1 -- 172.21.15.62:0/2804156806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3af4100ca0 msgr2=0x7f3af4106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:17.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.504+0000 7f3afc564640 1 --2- 172.21.15.62:0/2804156806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3af4100ca0 0x7f3af4106840 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009930 tx=0x7f3ae802a420 comp rx=0 tx=0).stop 2024-09-12T00:00:17.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.504+0000 7f3afc564640 1 -- 172.21.15.62:0/2804156806 shutdown_connections 2024-09-12T00:00:17.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.504+0000 7f3afc564640 1 --2- 172.21.15.62:0/2804156806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af4106f50 0x7f3af410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.504+0000 7f3afc564640 1 --2- 172.21.15.62:0/2804156806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3af4100ca0 0x7f3af4106840 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.504+0000 7f3afc564640 1 --2- 172.21.15.62:0/2804156806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af41002f0 0x7f3af41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.504+0000 7f3afc564640 1 -- 172.21.15.62:0/2804156806 >> 172.21.15.62:0/2804156806 conn(0x7f3af4077210 msgr2=0x7f3af40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:17.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.504+0000 7f3afc564640 1 -- 172.21.15.62:0/2804156806 shutdown_connections 2024-09-12T00:00:17.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.504+0000 7f3afc564640 1 -- 172.21.15.62:0/2804156806 wait complete. 2024-09-12T00:00:17.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afc564640 1 Processor -- start 2024-09-12T00:00:17.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afc564640 1 -- start start 2024-09-12T00:00:17.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afc564640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af41002f0 0x7f3af4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afc564640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af4100ca0 0x7f3af4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afc564640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3af4106f50 0x7f3af419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afc564640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3af4111fe0 con 0x7f3af4106f50 2024-09-12T00:00:17.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afc564640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3af4111e60 con 0x7f3af41002f0 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afc564640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3af4112160 con 0x7f3af4100ca0 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afab01640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3af4106f50 0x7f3af419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afab01640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3af4106f50 0x7f3af419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45936/0 (socket says 172.21.15.62:45936) 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afab01640 1 -- 172.21.15.62:0/2945949510 learned_addr learned my addr 172.21.15.62:0/2945949510 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3afa300640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af41002f0 0x7f3af4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.505+0000 7f3af9aff640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af4100ca0 0x7f3af4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3afab01640 1 -- 172.21.15.62:0/2945949510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af4100ca0 msgr2=0x7f3af4198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3afab01640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af4100ca0 0x7f3af4198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3afab01640 1 -- 172.21.15.62:0/2945949510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af41002f0 msgr2=0x7f3af4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3afab01640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af41002f0 0x7f3af4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3afab01640 1 -- 172.21.15.62:0/2945949510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af419d4d0 con 0x7f3af4106f50 2024-09-12T00:00:17.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3af9aff640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af4100ca0 0x7f3af4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3afa300640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af41002f0 0x7f3af4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3afab01640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3af4106f50 0x7f3af419cdf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00eaa0 tx=0x7f3aec00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3ae77fe640 1 -- 172.21.15.62:0/2945949510 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004d20 con 0x7f3af4106f50 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3ae77fe640 1 -- 172.21.15.62:0/2945949510 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aec005020 con 0x7f3af4106f50 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3afc564640 1 -- 172.21.15.62:0/2945949510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af419d7c0 con 0x7f3af4106f50 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3ae77fe640 1 -- 172.21.15.62:0/2945949510 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0116d0 con 0x7f3af4106f50 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.506+0000 7f3afc564640 1 -- 172.21.15.62:0/2945949510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af41a50a0 con 0x7f3af4106f50 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.508+0000 7f3ae77fe640 1 -- 172.21.15.62:0/2945949510 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3aec0051d0 con 0x7f3af4106f50 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3afe93d740 1 Processor -- start 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3afe93d740 1 -- start start 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16433a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d164349110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d16434ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3afe93d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d16429f620 con 0x55d164339f40 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3afe93d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d164246e50 con 0x55d164349650 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3afe93d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d16429b5e0 con 0x55d16433a860 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3ae77fe640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3ae00a2cf0 0x7f3ae00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3af9aff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16433a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3af9aff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16433a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45940/0 (socket says 172.21.15.62:45940) 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3af9aff640 1 -- 172.21.15.62:0/1430938915 learned_addr learned my addr 172.21.15.62:0/1430938915 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3afab01640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3ae00a2cf0 0x7f3ae00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.509+0000 7f3afa300640 1 --2- 172.21.15.62:0/1430938915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d164349110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3ae77fe640 1 -- 172.21.15.62:0/2945949510 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f3aec0cf6e0 con 0x7f3af4106f50 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3af9aff640 1 --2- 172.21.15.62:0/1430938915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d16434ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3af9aff640 1 -- 172.21.15.62:0/1430938915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 msgr2=0x55d164349110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3af9aff640 1 --2- 172.21.15.62:0/1430938915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d164349110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3af9aff640 1 -- 172.21.15.62:0/1430938915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 msgr2=0x55d16434ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3af9aff640 1 --2- 172.21.15.62:0/1430938915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d16434ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3af9aff640 1 -- 172.21.15.62:0/1430938915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d164237d40 con 0x55d164339f40 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3afab01640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3ae00a2cf0 0x7f3ae00a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3aec098060 tx=0x7f3aec066c20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3af9aff640 1 --2- 172.21.15.62:0/1430938915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16433a320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8005940 tx=0x7f3ae8031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b03a3932c00f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3ae4ff9640 1 -- 172.21.15.62:0/1430938915 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8043020 con 0x55d164339f40 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3ae4ff9640 1 -- 172.21.15.62:0/1430938915 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae8005040 con 0x55d164339f40 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.510+0000 7f3ae4ff9640 1 -- 172.21.15.62:0/1430938915 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8035050 con 0x55d164339f40 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.511+0000 7f3afe93d740 1 -- 172.21.15.62:0/1430938915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 msgr2=0x55d16433a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.511+0000 7f3afe93d740 1 --2- 172.21.15.62:0/1430938915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16433a320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8005940 tx=0x7f3ae8031480 comp rx=0 tx=0).stop 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.511+0000 7f3afe93d740 1 -- 172.21.15.62:0/1430938915 shutdown_connections 2024-09-12T00:00:17.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.511+0000 7f3afe93d740 1 --2- 172.21.15.62:0/1430938915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d16434ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.511+0000 7f3afe93d740 1 --2- 172.21.15.62:0/1430938915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d164349110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.511+0000 7f3afe93d740 1 --2- 172.21.15.62:0/1430938915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16433a320 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.511+0000 7f3afe93d740 1 -- 172.21.15.62:0/1430938915 >> 172.21.15.62:0/1430938915 conn(0x55d1643365f0 msgr2=0x55d1643369d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.511+0000 7f3afe93d740 1 -- 172.21.15.62:0/1430938915 shutdown_connections 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.511+0000 7f3afe93d740 1 -- 172.21.15.62:0/1430938915 wait complete. 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.511+0000 7f3afe93d740 1 -- start start 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3afe93d740 1 Processor -- start 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d1643465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d164346b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d164347040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3afe93d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d16434c220 con 0x55d164339f40 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3afe93d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d164227f10 con 0x55d164349650 2024-09-12T00:00:17.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3afe93d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d16434c3a0 con 0x55d16433a860 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3afe93d740 10 client.0 _ll_register_callbacks cb 0x55d16424bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3af9aff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d1643465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3af9aff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d1643465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45954/0 (socket says 172.21.15.62:45954) 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3af9aff640 1 -- 172.21.15.62:0/808214774 learned_addr learned my addr 172.21.15.62:0/808214774 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3af9aff640 1 -- 172.21.15.62:0/808214774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 msgr2=0x55d164346b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3af9aff640 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d164346b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3af9aff640 1 -- 172.21.15.62:0/808214774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 msgr2=0x55d164347040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3af9aff640 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d164347040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3af9aff640 1 -- 172.21.15.62:0/808214774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d163fae580 con 0x55d164339f40 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3af9aff640 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d1643465c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8031ac0 tx=0x7f3ae80044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54b6f73c1d9fecf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.512+0000 7f3addffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:17.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3addffb640 1 -- 172.21.15.62:0/808214774 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8035030 con 0x55d164339f40 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3afe93d740 1 -- 172.21.15.62:0/808214774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae8005040 con 0x55d164339f40 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3addffb640 1 -- 172.21.15.62:0/808214774 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f3ae80046c0 con 0x55d164339f40 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3addffb640 1 -- 172.21.15.62:0/808214774 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3ae8043040 con 0x55d164339f40 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3afe93d740 10 client.16809 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3afe93d740 1 -- 172.21.15.62:0/808214774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d164078960 con 0x55d164339f40 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3addffb640 1 -- 172.21.15.62:0/808214774 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae8005040 con 0x55d164339f40 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3addffb640 1 -- 172.21.15.62:0/808214774 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f3ae8003950 con 0x55d164339f40 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3afe93d740 10 client.16809 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3afe93d740 10 client.16809 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3afe93d740 20 client.16809 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3afe93d740 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d164359100 0x55d16435b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3afe93d740 4 client.16809 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:00:17.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3afe93d740 1 -- 172.21.15.62:0/808214774 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1642acb90 con 0x55d164359100 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.513+0000 7f3afa300640 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d164359100 0x55d16435b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.514+0000 7f3afa300640 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d164359100 0x55d16435b4f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17ed263a451d569b server_cookie=890ff7ff4a8784d8 in_seq=0 out_seq=0 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.514+0000 7f3addffb640 10 client.16809 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3addffb640 1 -- 172.21.15.62:0/808214774 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1642acb90 con 0x55d164359100 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3addffb640 10 client.16809 handle_command_reply: tid=0 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 client.16809 shutdown 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 -- 172.21.15.62:0/808214774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 msgr2=0x55d1643465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d1643465c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8031ac0 tx=0x7f3ae80044b0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 -- 172.21.15.62:0/808214774 shutdown_connections 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d164359100 0x55d16435b4f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3addffb640 0 client.16809 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d164347040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d164346b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 --2- 172.21.15.62:0/808214774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d1643465c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 -- 172.21.15.62:0/808214774 >> 172.21.15.62:0/808214774 conn(0x55d1643365f0 msgr2=0x55d16433c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:17.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 -- 172.21.15.62:0/808214774 shutdown_connections 2024-09-12T00:00:17.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 1 -- 172.21.15.62:0/808214774 wait complete. 2024-09-12T00:00:17.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.515+0000 7f3afe93d740 20 client.16809 trim_cache size 0 max 16384 2024-09-12T00:00:17.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afe93d740 1 Processor -- start 2024-09-12T00:00:17.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afe93d740 1 -- start start 2024-09-12T00:00:17.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d164339f40 0x55d164354d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d16433a860 0x55d164343820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d164343d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afe93d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41adf00 con 0x55d16433a860 2024-09-12T00:00:17.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afe93d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41ab7d0 con 0x55d164349650 2024-09-12T00:00:17.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afe93d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41a9640 con 0x55d164339f40 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afa300640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d16433a860 0x55d164343820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afa300640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d16433a860 0x55d164343820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45964/0 (socket says 172.21.15.62:45964) 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3afa300640 1 -- 172.21.15.62:0/2576648218 learned_addr learned my addr 172.21.15.62:0/2576648218 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.532+0000 7f3af9aff640 1 --2- 172.21.15.62:0/2576648218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d164339f40 0x55d164354d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.533+0000 7f3afa300640 1 -- 172.21.15.62:0/2576648218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d164339f40 msgr2=0x55d164354d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.533+0000 7f3afa300640 1 --2- 172.21.15.62:0/2576648218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d164339f40 0x55d164354d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.533+0000 7f3afa300640 1 -- 172.21.15.62:0/2576648218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 msgr2=0x55d164343d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.533+0000 7f3afa300640 1 --2- 172.21.15.62:0/2576648218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d164343d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.533+0000 7f3afa300640 1 -- 172.21.15.62:0/2576648218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d164337550 con 0x55d16433a860 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.533+0000 7f3af9aff640 1 --2- 172.21.15.62:0/2576648218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d164339f40 0x55d164354d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.533+0000 7f3afa300640 1 --2- 172.21.15.62:0/2576648218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d16433a860 0x55d164343820 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3af0000c00 tx=0x7f3af0016620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a92759500aa4eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.533+0000 7f3adffff640 1 -- 172.21.15.62:0/2576648218 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0024070 con 0x55d16433a860 2024-09-12T00:00:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.533+0000 7f3adffff640 1 -- 172.21.15.62:0/2576648218 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af00149b0 con 0x55d16433a860 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.533+0000 7f3adffff640 1 -- 172.21.15.62:0/2576648218 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af001f960 con 0x55d16433a860 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 -- 172.21.15.62:0/2576648218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d16433a860 msgr2=0x55d164343820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 --2- 172.21.15.62:0/2576648218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d16433a860 0x55d164343820 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3af0000c00 tx=0x7f3af0016620 comp rx=0 tx=0).stop 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 -- 172.21.15.62:0/2576648218 shutdown_connections 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 --2- 172.21.15.62:0/2576648218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d164343d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 --2- 172.21.15.62:0/2576648218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d16433a860 0x55d164343820 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 --2- 172.21.15.62:0/2576648218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d164339f40 0x55d164354d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 -- 172.21.15.62:0/2576648218 >> 172.21.15.62:0/2576648218 conn(0x55d1643365f0 msgr2=0x55d164354630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 -- 172.21.15.62:0/2576648218 shutdown_connections 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 -- 172.21.15.62:0/2576648218 wait complete. 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 -- start start 2024-09-12T00:00:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.534+0000 7f3afe93d740 1 Processor -- start 2024-09-12T00:00:17.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16435c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d16435cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3afe93d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d1643c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3afe93d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41a9c90 con 0x55d164339f40 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3afe93d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41ab100 con 0x55d164349650 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3afe93d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41aa360 con 0x55d16433a860 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3af9aff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16435c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3afe93d740 10 client.0 _ll_register_callbacks cb 0x7f3ae8004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3af9aff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16435c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45976/0 (socket says 172.21.15.62:45976) 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3af9aff640 1 -- 172.21.15.62:0/2926103142 learned_addr learned my addr 172.21.15.62:0/2926103142 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3af9aff640 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d1643c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3afa300640 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d16435cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3af9aff640 1 -- 172.21.15.62:0/2926103142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 msgr2=0x55d16435cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3af9aff640 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d16435cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3af9aff640 1 -- 172.21.15.62:0/2926103142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 msgr2=0x55d1643c2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3af9aff640 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d1643c2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3af9aff640 1 -- 172.21.15.62:0/2926103142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1643c2f80 con 0x55d164339f40 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3af9aff640 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16435c6e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8004650 tx=0x7f3ae803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2c8fc146725582 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3addffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.535+0000 7f3addffb640 1 -- 172.21.15.62:0/2926103142 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8035070 con 0x55d164339f40 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3addffb640 1 -- 172.21.15.62:0/2926103142 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f3ae80035f0 con 0x55d164339f40 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3afe93d740 1 -- 172.21.15.62:0/2926103142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af00149b0 con 0x55d164339f40 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3addffb640 1 -- 172.21.15.62:0/2926103142 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3ae8005600 con 0x55d164339f40 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3afe93d740 10 client.16815 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3afe93d740 1 -- 172.21.15.62:0/2926103142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d1643c4110 con 0x55d164339f40 2024-09-12T00:00:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3addffb640 1 -- 172.21.15.62:0/2926103142 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af00149b0 con 0x55d164339f40 2024-09-12T00:00:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3addffb640 1 -- 172.21.15.62:0/2926103142 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f3ae8059030 con 0x55d164339f40 2024-09-12T00:00:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3afe93d740 10 client.16815 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3afe93d740 10 client.16815 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:00:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3afe93d740 20 client.16815 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3afe93d740 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d1643c4770 0x55d1643c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3afe93d740 4 client.16815 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2615c871", "release": true, "await": true}] 2024-09-12T00:00:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3afe93d740 1 -- 172.21.15.62:0/2926103142 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2615c871", "release": true, "await": true}) -- 0x55d1642a70a0 con 0x55d1643c4770 2024-09-12T00:00:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.536+0000 7f3afa300640 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d1643c4770 0x55d1643c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.537+0000 7f3afa300640 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d1643c4770 0x55d1643c6b60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc7fb73877e2c6e server_cookie=ca85100521ac90ab in_seq=0 out_seq=0 2024-09-12T00:00:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.537+0000 7f3addffb640 10 client.16815 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.583+0000 7f3addffb640 1 -- 172.21.15.62:0/2926103142 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d1642a70a0 con 0x55d1643c4770 2024-09-12T00:00:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.583+0000 7f3addffb640 10 client.16815 handle_command_reply: tid=0 2024-09-12T00:00:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.583+0000 7f3afe93d740 1 client.16815 shutdown 2024-09-12T00:00:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.583+0000 7f3afe93d740 1 -- 172.21.15.62:0/2926103142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 msgr2=0x55d16435c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.583+0000 7f3afe93d740 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16435c6e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8004650 tx=0x7f3ae803f040 comp rx=0 tx=0).stop 2024-09-12T00:00:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.583+0000 7f3afe93d740 1 -- 172.21.15.62:0/2926103142 shutdown_connections 2024-09-12T00:00:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.583+0000 7f3afe93d740 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d1643c4770 0x55d1643c6b60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afe93d740 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d164349650 0x55d1643c2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afe93d740 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d16433a860 0x55d16435cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afe93d740 1 --2- 172.21.15.62:0/2926103142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d164339f40 0x55d16435c6e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afe93d740 1 -- 172.21.15.62:0/2926103142 >> 172.21.15.62:0/2926103142 conn(0x55d1643365f0 msgr2=0x55d164354e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3addffb640 0 client.16815 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afe93d740 1 -- 172.21.15.62:0/2926103142 shutdown_connections 2024-09-12T00:00:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afe93d740 1 -- 172.21.15.62:0/2926103142 wait complete. 2024-09-12T00:00:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afe93d740 20 client.16815 trim_cache size 0 max 16384 2024-09-12T00:00:17.585 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 31, 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 32, 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "2615c871": { 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 32, 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:00:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:00:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:00:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 139.4, 2024-09-12T00:00:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:00:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:00:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:00:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:00:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:00:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:00:17.588 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:17.588 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:17.588 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:17.588 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:17.588 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:17.588 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:00:17.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afc564640 1 -- 172.21.15.62:0/2945949510 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3ae00a2cf0 msgr2=0x7f3ae00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:17.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afc564640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3ae00a2cf0 0x7f3ae00a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3aec098060 tx=0x7f3aec066c20 comp rx=0 tx=0).stop 2024-09-12T00:00:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afc564640 1 -- 172.21.15.62:0/2945949510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3af4106f50 msgr2=0x7f3af419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afc564640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3af4106f50 0x7f3af419cdf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00eaa0 tx=0x7f3aec00eea0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afc564640 1 -- 172.21.15.62:0/2945949510 shutdown_connections 2024-09-12T00:00:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afc564640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3ae00a2cf0 0x7f3ae00a51b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afc564640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3af4106f50 0x7f3af419cdf0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afc564640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3af4100ca0 0x7f3af4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afc564640 1 --2- 172.21.15.62:0/2945949510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af41002f0 0x7f3af4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.584+0000 7f3afc564640 1 -- 172.21.15.62:0/2945949510 >> 172.21.15.62:0/2945949510 conn(0x7f3af4077210 msgr2=0x7f3af41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.585+0000 7f3afc564640 1 -- 172.21.15.62:0/2945949510 shutdown_connections 2024-09-12T00:00:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:17.585+0000 7f3afc564640 1 -- 172.21.15.62:0/2945949510 wait complete. 2024-09-12T00:00:17.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:17 smithi120 ceph-mon[35182]: pgmap v252: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.2 MiB/s wr, 205 op/s 2024-09-12T00:00:17.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:17 smithi083 ceph-mon[34805]: pgmap v252: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.2 MiB/s wr, 205 op/s 2024-09-12T00:00:18.655 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2615c871', total seconds elapsed: 23.1 2024-09-12T00:00:18.655 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T00:00:18.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:00:19.058 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:19.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:19 smithi062 ceph-mon[30958]: pgmap v253: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-09-12T00:00:19.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:19 smithi120 ceph-mon[35182]: pgmap v253: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-09-12T00:00:19.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:19 smithi083 ceph-mon[34805]: pgmap v253: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-09-12T00:00:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.748+0000 7f96ab92c640 1 -- 172.21.15.62:0/3451281211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a4102ef0 msgr2=0x7f96a410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.748+0000 7f96ab92c640 1 --2- 172.21.15.62:0/3451281211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a4102ef0 0x7f96a410da40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f969c009cd0 tx=0x7f969c02a980 comp rx=0 tx=0).stop 2024-09-12T00:00:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.749+0000 7f96ab92c640 1 -- 172.21.15.62:0/3451281211 shutdown_connections 2024-09-12T00:00:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.749+0000 7f96ab92c640 1 --2- 172.21.15.62:0/3451281211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a4102ef0 0x7f96a410da40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.749+0000 7f96ab92c640 1 --2- 172.21.15.62:0/3451281211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a4102570 0x7f96a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.749+0000 7f96ab92c640 1 --2- 172.21.15.62:0/3451281211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4100e20 0x7f96a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.749+0000 7f96ab92c640 1 -- 172.21.15.62:0/3451281211 >> 172.21.15.62:0/3451281211 conn(0x7f96a40fc7d0 msgr2=0x7f96a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.749+0000 7f96ab92c640 1 -- 172.21.15.62:0/3451281211 shutdown_connections 2024-09-12T00:00:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.751+0000 7f96ab92c640 1 -- 172.21.15.62:0/3451281211 wait complete. 2024-09-12T00:00:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96ab92c640 1 Processor -- start 2024-09-12T00:00:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96ab92c640 1 -- start start 2024-09-12T00:00:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96ab92c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a4100e20 0x7f96a419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96ab92c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a4102570 0x7f96a419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96ab92c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102ef0 0x7f96a419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96ab92c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41142a0 con 0x7f96a4100e20 2024-09-12T00:00:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96ab92c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4114120 con 0x7f96a4102570 2024-09-12T00:00:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96ab92c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4114420 con 0x7f96a4102ef0 2024-09-12T00:00:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96a96c8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a4100e20 0x7f96a419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96a8ec7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a4102570 0x7f96a419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96a8ec7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a4102570 0x7f96a419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:59478/0 (socket says 172.21.15.62:59478) 2024-09-12T00:00:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96a96c8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a4100e20 0x7f96a419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45986/0 (socket says 172.21.15.62:45986) 2024-09-12T00:00:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96a8ec7640 1 -- 172.21.15.62:0/336088683 learned_addr learned my addr 172.21.15.62:0/336088683 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.752+0000 7f96a9ec9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102ef0 0x7f96a419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f96a8ec7640 1 -- 172.21.15.62:0/336088683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102ef0 msgr2=0x7f96a419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f96a8ec7640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102ef0 0x7f96a419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f96a8ec7640 1 -- 172.21.15.62:0/336088683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a4100e20 msgr2=0x7f96a419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f96a8ec7640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a4100e20 0x7f96a419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f96a8ec7640 1 -- 172.21.15.62:0/336088683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a419f6d0 con 0x7f96a4102570 2024-09-12T00:00:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f96a96c8640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a4100e20 0x7f96a419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:00:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f96a8ec7640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a4102570 0x7f96a419ac60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f969400e9f0 tx=0x7f969400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f969a7fc640 1 -- 172.21.15.62:0/336088683 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969400ccc0 con 0x7f96a4102570 2024-09-12T00:00:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f969a7fc640 1 -- 172.21.15.62:0/336088683 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9694010910 con 0x7f96a4102570 2024-09-12T00:00:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f96ab92c640 1 -- 172.21.15.62:0/336088683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a419f9c0 con 0x7f96a4102570 2024-09-12T00:00:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f969a7fc640 1 -- 172.21.15.62:0/336088683 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940188a0 con 0x7f96a4102570 2024-09-12T00:00:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.753+0000 7f96ab92c640 1 -- 172.21.15.62:0/336088683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41a72a0 con 0x7f96a4102570 2024-09-12T00:00:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.755+0000 7f96ab92c640 1 -- 172.21.15.62:0/336088683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a4101200 con 0x7f96a4102570 2024-09-12T00:00:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.756+0000 7f969a7fc640 1 -- 172.21.15.62:0/336088683 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9694018a40 con 0x7f96a4102570 2024-09-12T00:00:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.757+0000 7f969a7fc640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f96900a2cf0 0x7f96900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.757+0000 7f969a7fc640 1 -- 172.21.15.62:0/336088683 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f96940cf8c0 con 0x7f96a4102570 2024-09-12T00:00:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.757+0000 7f96a96c8640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f96900a2cf0 0x7f96900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.758+0000 7f96a96c8640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f96900a2cf0 0x7f96900a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009d60 tx=0x7f96a0009110 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:19.761+0000 7f969a7fc640 1 -- 172.21.15.62:0/336088683 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f969409c4e0 con 0x7f96a4102570 2024-09-12T00:00:20.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.091+0000 7f96ab92c640 1 -- 172.21.15.62:0/336088683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96a4109090 con 0x7f96a4102570 2024-09-12T00:00:20.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.096+0000 7f969a7fc640 1 -- 172.21.15.62:0/336088683 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11909 (secure 0 0 0) 0x7f969409bde0 con 0x7f96a4102570 2024-09-12T00:00:20.096 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:00:20.096 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"btime":"2024-09-12T00:00:02:308078+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:00:02.079738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:00:20.097 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2024-09-12T00:00:20.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.100+0000 7f96ab92c640 1 -- 172.21.15.62:0/336088683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f96900a2cf0 msgr2=0x7f96900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:20.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.100+0000 7f96ab92c640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f96900a2cf0 0x7f96900a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009d60 tx=0x7f96a0009110 comp rx=0 tx=0).stop 2024-09-12T00:00:20.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.100+0000 7f96ab92c640 1 -- 172.21.15.62:0/336088683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a4102570 msgr2=0x7f96a419ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:20.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.100+0000 7f96ab92c640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a4102570 0x7f96a419ac60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f969400e9f0 tx=0x7f969400edf0 comp rx=0 tx=0).stop 2024-09-12T00:00:20.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.106+0000 7f96ab92c640 1 -- 172.21.15.62:0/336088683 shutdown_connections 2024-09-12T00:00:20.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.106+0000 7f96ab92c640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f96900a2cf0 0x7f96900a51b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:20.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.106+0000 7f96ab92c640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102ef0 0x7f96a419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:20.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.106+0000 7f96ab92c640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a4102570 0x7f96a419ac60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:20.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.106+0000 7f96ab92c640 1 --2- 172.21.15.62:0/336088683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a4100e20 0x7f96a419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:20.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.106+0000 7f96ab92c640 1 -- 172.21.15.62:0/336088683 >> 172.21.15.62:0/336088683 conn(0x7f96a40fc7d0 msgr2=0x7f96a4104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:20.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.112+0000 7f96ab92c640 1 -- 172.21.15.62:0/336088683 shutdown_connections 2024-09-12T00:00:20.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:20.112+0000 7f96ab92c640 1 -- 172.21.15.62:0/336088683 wait complete. 2024-09-12T00:00:20.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:20 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:00:20.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:20 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/336088683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:20.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:20 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:00:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:20 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/336088683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:20.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:20 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:00:20.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:20 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/336088683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:20.853 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T00:00:20.854 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T00:00:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:21.278 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:00:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:21 smithi120 ceph-mon[35182]: pgmap v254: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-09-12T00:00:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:21 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:00:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:21 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:00:21.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:21 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:00:21.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:21 smithi062 ceph-mon[30958]: pgmap v254: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-09-12T00:00:21.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:21 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:00:21.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:21 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:00:21.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:21 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:00:21.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:21 smithi083 ceph-mon[34805]: pgmap v254: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-09-12T00:00:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:21 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:00:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:21 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:00:21.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:21 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:00:21.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.818+0000 7f2604622640 1 -- 172.21.15.62:0/3233417784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25fc109450 msgr2=0x7f25fc110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:21.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.818+0000 7f2604622640 1 --2- 172.21.15.62:0/3233417784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25fc109450 0x7f25fc110250 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f25ec009be0 tx=0x7f25ec02a6f0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.820+0000 7f2604622640 1 -- 172.21.15.62:0/3233417784 shutdown_connections 2024-09-12T00:00:21.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.820+0000 7f2604622640 1 --2- 172.21.15.62:0/3233417784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc11b140 0x7f25fc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.820+0000 7f2604622640 1 --2- 172.21.15.62:0/3233417784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25fc109830 0x7f25fc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.820+0000 7f2604622640 1 --2- 172.21.15.62:0/3233417784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25fc109450 0x7f25fc110250 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.820+0000 7f2604622640 1 -- 172.21.15.62:0/3233417784 >> 172.21.15.62:0/3233417784 conn(0x7f25fc06e140 msgr2=0x7f25fc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:21.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.820+0000 7f2604622640 1 -- 172.21.15.62:0/3233417784 shutdown_connections 2024-09-12T00:00:21.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.820+0000 7f2604622640 1 -- 172.21.15.62:0/3233417784 wait complete. 2024-09-12T00:00:21.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.821+0000 7f2604622640 1 Processor -- start 2024-09-12T00:00:21.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.827+0000 7f2604622640 1 -- start start 2024-09-12T00:00:21.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.827+0000 7f2604622640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25fc109450 0x7f25fc11a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.827+0000 7f2604622640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25fc109830 0x7f25fc1168b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.827+0000 7f2604622640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc11b140 0x7f25fc116df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.827+0000 7f2604622640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc113300 con 0x7f25fc109450 2024-09-12T00:00:21.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.827+0000 7f2604622640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc113180 con 0x7f25fc11b140 2024-09-12T00:00:21.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.827+0000 7f2604622640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc113480 con 0x7f25fc109830 2024-09-12T00:00:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.827+0000 7f2602bbf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc11b140 0x7f25fc116df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.827+0000 7f2602bbf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc11b140 0x7f25fc116df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:36158/0 (socket says 172.21.15.62:36158) 2024-09-12T00:00:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.827+0000 7f2602bbf640 1 -- 172.21.15.62:0/292896414 learned_addr learned my addr 172.21.15.62:0/292896414 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.828+0000 7f2602bbf640 1 -- 172.21.15.62:0/292896414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25fc109830 msgr2=0x7f25fc1168b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T00:00:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.828+0000 7f2602bbf640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25fc109830 0x7f25fc1168b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.828+0000 7f2602bbf640 1 -- 172.21.15.62:0/292896414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25fc109450 msgr2=0x7f25fc11a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.828+0000 7f2602bbf640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25fc109450 0x7f25fc11a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.828+0000 7f2602bbf640 1 -- 172.21.15.62:0/292896414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc1174f0 con 0x7f25fc11b140 2024-09-12T00:00:21.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.830+0000 7f2602bbf640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc11b140 0x7f25fc116df0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e970 tx=0x7f25f400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:21.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.831+0000 7f25fb7fe640 1 -- 172.21.15.62:0/292896414 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f400ccc0 con 0x7f25fc11b140 2024-09-12T00:00:21.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.831+0000 7f2604622640 1 -- 172.21.15.62:0/292896414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc1ab170 con 0x7f25fc11b140 2024-09-12T00:00:21.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.831+0000 7f2604622640 1 -- 172.21.15.62:0/292896414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc1ab560 con 0x7f25fc11b140 2024-09-12T00:00:21.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.834+0000 7f26069fb740 1 Processor -- start 2024-09-12T00:00:21.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f26069fb740 1 -- start start 2024-09-12T00:00:21.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f26069fb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d2760 0x563d280d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f26069fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f26069fb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 0x563d280dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f26069fb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563d280382a0 con 0x563d280dd940 2024-09-12T00:00:21.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f26069fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563d27fdfb10 con 0x563d280d3080 2024-09-12T00:00:21.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f26069fb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d28034260 con 0x563d280d2760 2024-09-12T00:00:21.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f25fb7fe640 1 -- 172.21.15.62:0/292896414 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f400ce60 con 0x7f25fc11b140 2024-09-12T00:00:21.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f25fb7fe640 1 -- 172.21.15.62:0/292896414 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4010690 con 0x7f25fc11b140 2024-09-12T00:00:21.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f2601bbd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d2760 0x563d280d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f2601bbd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d2760 0x563d280d2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:54854/0 (socket says 172.21.15.62:54854) 2024-09-12T00:00:21.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.835+0000 7f2601bbd640 1 -- 172.21.15.62:0/1407822344 learned_addr learned my addr 172.21.15.62:0/1407822344 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:21.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.836+0000 7f2601bbd640 1 --2- 172.21.15.62:0/1407822344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 0x563d280dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.837+0000 7f2601bbd640 1 -- 172.21.15.62:0/1407822344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 msgr2=0x563d280dd400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:00:21.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.837+0000 7f2601bbd640 1 --2- 172.21.15.62:0/1407822344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280dd400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.837+0000 7f2601bbd640 1 -- 172.21.15.62:0/1407822344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 msgr2=0x563d280dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:21.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.837+0000 7f2601bbd640 1 --2- 172.21.15.62:0/1407822344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 0x563d280dfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.837+0000 7f2601bbd640 1 -- 172.21.15.62:0/1407822344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d27da9b90 con 0x563d280d2760 2024-09-12T00:00:21.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.837+0000 7f2601bbd640 1 --2- 172.21.15.62:0/1407822344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d2760 0x563d280d2b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f25f001afd0 tx=0x7f25f0005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca4507b54ae93949 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:21.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.838+0000 7f25f8ff9640 1 -- 172.21.15.62:0/1407822344 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0024110 con 0x563d280d2760 2024-09-12T00:00:21.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.838+0000 7f25f8ff9640 1 -- 172.21.15.62:0/1407822344 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f00242b0 con 0x563d280d2760 2024-09-12T00:00:21.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.839+0000 7f26069fb740 1 -- 172.21.15.62:0/1407822344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d2760 msgr2=0x563d280d2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:21.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.839+0000 7f26069fb740 1 --2- 172.21.15.62:0/1407822344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d2760 0x563d280d2b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f25f001afd0 tx=0x7f25f0005a40 comp rx=0 tx=0).stop 2024-09-12T00:00:21.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.840+0000 7f26069fb740 1 -- 172.21.15.62:0/1407822344 shutdown_connections 2024-09-12T00:00:21.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.840+0000 7f26069fb740 1 --2- 172.21.15.62:0/1407822344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 0x563d280dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.840+0000 7f26069fb740 1 --2- 172.21.15.62:0/1407822344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280dd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.840+0000 7f26069fb740 1 --2- 172.21.15.62:0/1407822344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d2760 0x563d280d2b40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.840+0000 7f26069fb740 1 -- 172.21.15.62:0/1407822344 >> 172.21.15.62:0/1407822344 conn(0x563d280cecf0 msgr2=0x563d280cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:21.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.840+0000 7f25fb7fe640 1 -- 172.21.15.62:0/292896414 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f25f40046c0 con 0x7f25fc11b140 2024-09-12T00:00:21.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.840+0000 7f26069fb740 1 -- 172.21.15.62:0/1407822344 shutdown_connections 2024-09-12T00:00:21.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.840+0000 7f26069fb740 1 -- 172.21.15.62:0/1407822344 wait complete. 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.841+0000 7f26069fb740 1 -- start start 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.841+0000 7f26069fb740 1 Processor -- start 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.841+0000 7f26069fb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 0x563d280da790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.841+0000 7f26069fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.841+0000 7f26069fb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280dd940 0x563d280db210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.841+0000 7f26069fb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563d280e0560 con 0x563d280d2760 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.841+0000 7f26069fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563d27fc06e0 con 0x563d280d3080 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.841+0000 7f26069fb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d280e06e0 con 0x563d280dd940 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.842+0000 7f26069fb740 10 client.0 _ll_register_callbacks cb 0x563d27fe4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.842+0000 7f26023be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.842+0000 7f2601bbd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280dd940 0x563d280db210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.842+0000 7f2601bbd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 0x563d280da790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.842+0000 7f2601bbd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 0x563d280da790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41256/0 (socket says 172.21.15.62:41256) 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.842+0000 7f2601bbd640 1 -- 172.21.15.62:0/687788042 learned_addr learned my addr 172.21.15.62:0/687788042 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.842+0000 7f2601bbd640 1 -- 172.21.15.62:0/687788042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 msgr2=0x563d280dacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.842+0000 7f2601bbd640 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280dacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.842+0000 7f2601bbd640 1 -- 172.21.15.62:0/687788042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 msgr2=0x563d280da790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.842+0000 7f2601bbd640 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 0x563d280da790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.843+0000 7f2601bbd640 1 -- 172.21.15.62:0/687788042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d280e9ab0 con 0x563d280dd940 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.843+0000 7f2601bbd640 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280dd940 0x563d280db210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f25f001ab60 tx=0x7f25f00054c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c758507bd0b9919 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.843+0000 7f25e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.843+0000 7f25e1ffb640 1 -- 172.21.15.62:0/687788042 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0024110 con 0x563d280dd940 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.843+0000 7f26069fb740 1 -- 172.21.15.62:0/687788042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25f00242b0 con 0x563d280dd940 2024-09-12T00:00:21.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.843+0000 7f25fb7fe640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25e40a2d00 0x7f25e40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.843+0000 7f25e1ffb640 1 -- 172.21.15.62:0/687788042 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f25f0004030 con 0x563d280dd940 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.843+0000 7f26023be640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25e40a2d00 0x7f25e40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.843+0000 7f25e1ffb640 1 -- 172.21.15.62:0/687788042 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25f0041050 con 0x563d280dd940 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f26069fb740 10 client.26371 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f26069fb740 1 -- 172.21.15.62:0/687788042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563d280ec450 con 0x563d280dd940 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f25fb7fe640 1 -- 172.21.15.62:0/292896414 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f25f40cf800 con 0x7f25fc11b140 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f25e1ffb640 1 -- 172.21.15.62:0/687788042 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f00242b0 con 0x563d280dd940 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f25e1ffb640 1 -- 172.21.15.62:0/687788042 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f25f001fa90 con 0x563d280dd940 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f26069fb740 10 client.26371 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f26069fb740 10 client.26371 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f26069fb740 20 client.26371 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f26069fb740 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563d280edce0 0x563d280f0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f26069fb740 4 client.26371 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:00:21.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.844+0000 7f26069fb740 1 -- 172.21.15.62:0/687788042 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d280459d0 con 0x563d280edce0 2024-09-12T00:00:21.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.845+0000 7f2601bbd640 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563d280edce0 0x563d280f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.845+0000 7f25e1ffb640 10 client.26371 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:21.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.846+0000 7f26023be640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25e40a2d00 0x7f25e40a51c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25ec034040 tx=0x7f25ec000f40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:21.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.846+0000 7f2601bbd640 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563d280edce0 0x563d280f0100 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=672128624741845b server_cookie=e1bdb832d61ccfc9 in_seq=0 out_seq=0 2024-09-12T00:00:21.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f25e1ffb640 1 -- 172.21.15.62:0/687788042 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d280459d0 con 0x563d280edce0 2024-09-12T00:00:21.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f25e1ffb640 10 client.26371 handle_command_reply: tid=0 2024-09-12T00:00:21.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 client.26371 shutdown 2024-09-12T00:00:21.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 -- 172.21.15.62:0/687788042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280dd940 msgr2=0x563d280db210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280dd940 0x563d280db210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f25f001ab60 tx=0x7f25f00054c0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 -- 172.21.15.62:0/687788042 shutdown_connections 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563d280edce0 0x563d280f0100 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280dd940 0x563d280db210 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 --2- 172.21.15.62:0/687788042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 0x563d280da790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 -- 172.21.15.62:0/687788042 >> 172.21.15.62:0/687788042 conn(0x563d280cecf0 msgr2=0x563d280d4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f25e1ffb640 0 client.26371 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 -- 172.21.15.62:0/687788042 shutdown_connections 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 1 -- 172.21.15.62:0/687788042 wait complete. 2024-09-12T00:00:21.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.847+0000 7f26069fb740 20 client.26371 trim_cache size 0 max 16384 2024-09-12T00:00:21.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.876+0000 7f26069fb740 1 Processor -- start 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.876+0000 7f26069fb740 1 -- start start 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.876+0000 7f26069fb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 0x563d280d51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.876+0000 7f26069fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.876+0000 7f26069fb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280dd940 0x563d280f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.876+0000 7f26069fb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8011190 con 0x563d280d2760 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.876+0000 7f26069fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25e800ea60 con 0x563d280d3080 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.876+0000 7f26069fb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f25e800c8d0 con 0x563d280dd940 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.876+0000 7f26023be640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 0x563d280d51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.877+0000 7f26023be640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 0x563d280d51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41260/0 (socket says 172.21.15.62:41260) 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.877+0000 7f26023be640 1 -- 172.21.15.62:0/1071221839 learned_addr learned my addr 172.21.15.62:0/1071221839 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.877+0000 7f2601bbd640 1 --2- 172.21.15.62:0/1071221839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.877+0000 7f2601bbd640 1 -- 172.21.15.62:0/1071221839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280dd940 msgr2=0x563d280f5f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:00:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.877+0000 7f2601bbd640 1 --2- 172.21.15.62:0/1071221839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280dd940 0x563d280f5f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.877+0000 7f2601bbd640 1 -- 172.21.15.62:0/1071221839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 msgr2=0x563d280d51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:21.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.877+0000 7f2601bbd640 1 --2- 172.21.15.62:0/1071221839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 0x563d280d51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.877+0000 7f2601bbd640 1 -- 172.21.15.62:0/1071221839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d280ceb20 con 0x563d280d3080 2024-09-12T00:00:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.879+0000 7f2601bbd640 1 --2- 172.21.15.62:0/1071221839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280f5a10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25f0020010 tx=0x7f25f0036040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683aecb7482bc50e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.879+0000 7f25e3fff640 1 -- 172.21.15.62:0/1071221839 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f004c070 con 0x563d280d3080 2024-09-12T00:00:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.879+0000 7f25e3fff640 1 -- 172.21.15.62:0/1071221839 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f002d600 con 0x563d280d3080 2024-09-12T00:00:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.879+0000 7f25e3fff640 1 -- 172.21.15.62:0/1071221839 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f001a2e0 con 0x563d280d3080 2024-09-12T00:00:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.880+0000 7f26069fb740 1 -- 172.21.15.62:0/1071221839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 msgr2=0x563d280f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.880+0000 7f26069fb740 1 --2- 172.21.15.62:0/1071221839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280f5a10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25f0020010 tx=0x7f25f0036040 comp rx=0 tx=0).stop 2024-09-12T00:00:21.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.881+0000 7f26069fb740 1 -- 172.21.15.62:0/1071221839 shutdown_connections 2024-09-12T00:00:21.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.881+0000 7f26069fb740 1 --2- 172.21.15.62:0/1071221839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280dd940 0x563d280f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.881+0000 7f26069fb740 1 --2- 172.21.15.62:0/1071221839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d3080 0x563d280f5a10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.881+0000 7f26069fb740 1 --2- 172.21.15.62:0/1071221839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280d2760 0x563d280d51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.881+0000 7f26069fb740 1 -- 172.21.15.62:0/1071221839 >> 172.21.15.62:0/1071221839 conn(0x563d280cecf0 msgr2=0x563d280dc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:21.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.881+0000 7f26069fb740 1 -- 172.21.15.62:0/1071221839 shutdown_connections 2024-09-12T00:00:21.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.881+0000 7f26069fb740 1 -- 172.21.15.62:0/1071221839 wait complete. 2024-09-12T00:00:21.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26069fb740 1 -- start start 2024-09-12T00:00:21.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26069fb740 1 Processor -- start 2024-09-12T00:00:21.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26069fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d2760 0x563d280e6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26069fb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d3080 0x563d280d7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26069fb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 0x563d280d79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26069fb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f25e800cf20 con 0x563d280dd940 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26023be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d2760 0x563d280e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f2601bbd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d3080 0x563d280d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26069fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25e800e390 con 0x563d280d2760 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26069fb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f25e800d5f0 con 0x563d280d3080 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f2601bbd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d3080 0x563d280d7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:54878/0 (socket says 172.21.15.62:54878) 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26023be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d2760 0x563d280e6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:36178/0 (socket says 172.21.15.62:36178) 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f2601bbd640 1 -- 172.21.15.62:0/2098267240 learned_addr learned my addr 172.21.15.62:0/2098267240 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f2602bbf640 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 0x563d280d79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.882+0000 7f26069fb740 10 client.0 _ll_register_callbacks cb 0x7f25f00052e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:21.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f2602bbf640 1 -- 172.21.15.62:0/2098267240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d3080 msgr2=0x563d280d7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f2602bbf640 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d3080 0x563d280d7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f2602bbf640 1 -- 172.21.15.62:0/2098267240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d2760 msgr2=0x563d280e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f2602bbf640 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d2760 0x563d280e6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f2602bbf640 1 -- 172.21.15.62:0/2098267240 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d280d81a0 con 0x563d280dd940 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f2602bbf640 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 0x563d280d79a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f25f40ccef0 tx=0x7f25f409be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f27bde497a9d747 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f25e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f25e1ffb640 1 -- 172.21.15.62:0/2098267240 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f40cd170 con 0x563d280dd940 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f26069fb740 1 -- 172.21.15.62:0/2098267240 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25f002d600 con 0x563d280dd940 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f25e1ffb640 1 -- 172.21.15.62:0/2098267240 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f25f40cd310 con 0x563d280dd940 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f25e1ffb640 1 -- 172.21.15.62:0/2098267240 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25f409bf20 con 0x563d280dd940 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f26069fb740 10 client.16836 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.883+0000 7f26069fb740 1 -- 172.21.15.62:0/2098267240 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563d281589b0 con 0x563d280dd940 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.884+0000 7f25e1ffb640 1 -- 172.21.15.62:0/2098267240 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f002d600 con 0x563d280dd940 2024-09-12T00:00:21.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.884+0000 7f25e1ffb640 1 -- 172.21.15.62:0/2098267240 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f25f409ab80 con 0x563d280dd940 2024-09-12T00:00:21.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.884+0000 7f26069fb740 10 client.16836 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:21.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.884+0000 7f26069fb740 10 client.16836 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:00:21.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.884+0000 7f26069fb740 20 client.16836 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:21.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.884+0000 7f26069fb740 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563d281590d0 0x563d2815b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:21.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.884+0000 7f26069fb740 4 client.16836 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T00:00:21.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.884+0000 7f26069fb740 1 -- 172.21.15.62:0/2098267240 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x563d2803fd20 con 0x563d281590d0 2024-09-12T00:00:21.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.884+0000 7f26023be640 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563d281590d0 0x563d2815b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:21.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.885+0000 7f26023be640 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563d281590d0 0x563d2815b4f0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b79b4934895bd033 server_cookie=ac30ffb0d90439fb in_seq=0 out_seq=0 2024-09-12T00:00:21.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:21.885+0000 7f25e1ffb640 10 client.16836 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:21.899 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:22.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.132+0000 7f25e1ffb640 1 -- 172.21.15.62:0/2098267240 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563d2803fd20 con 0x563d281590d0 2024-09-12T00:00:22.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.132+0000 7f25e1ffb640 10 client.16836 handle_command_reply: tid=0 2024-09-12T00:00:22.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.132+0000 7f26069fb740 1 client.16836 shutdown 2024-09-12T00:00:22.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 1 -- 172.21.15.62:0/2098267240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 msgr2=0x563d280d79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 0x563d280d79a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f25f40ccef0 tx=0x7f25f409be00 comp rx=0 tx=0).stop 2024-09-12T00:00:22.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 1 -- 172.21.15.62:0/2098267240 shutdown_connections 2024-09-12T00:00:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x563d281590d0 0x563d2815b4f0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d280dd940 0x563d280d79a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d280d3080 0x563d280d7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f25e1ffb640 0 client.16836 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 1 --2- 172.21.15.62:0/2098267240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d280d2760 0x563d280e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 1 -- 172.21.15.62:0/2098267240 >> 172.21.15.62:0/2098267240 conn(0x563d280cecf0 msgr2=0x563d280dbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 1 -- 172.21.15.62:0/2098267240 shutdown_connections 2024-09-12T00:00:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 1 -- 172.21.15.62:0/2098267240 wait complete. 2024-09-12T00:00:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.133+0000 7f26069fb740 20 client.16836 trim_cache size 0 max 16384 2024-09-12T00:00:22.137 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:00:22.137 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 31, 2024-09-12T00:00:22.137 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:00:22.137 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 34, 2024-09-12T00:00:22.137 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:00:22.137 INFO:teuthology.orchestra.run.smithi062.stdout: "ccd456ca": { 2024-09-12T00:00:22.137 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 34, 2024-09-12T00:00:22.137 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:00:22.137 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:00:22.137 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 139.4, 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:22.138 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:22.139 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:22.139 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:22.139 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:00:22.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 -- 172.21.15.62:0/292896414 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25e40a2d00 msgr2=0x7f25e40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25e40a2d00 0x7f25e40a51c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25ec034040 tx=0x7f25ec000f40 comp rx=0 tx=0).stop 2024-09-12T00:00:22.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 -- 172.21.15.62:0/292896414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc11b140 msgr2=0x7f25fc116df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc11b140 0x7f25fc116df0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e970 tx=0x7f25f400ee30 comp rx=0 tx=0).stop 2024-09-12T00:00:22.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 -- 172.21.15.62:0/292896414 shutdown_connections 2024-09-12T00:00:22.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25e40a2d00 0x7f25e40a51c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25fc11b140 0x7f25fc116df0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25fc109830 0x7f25fc1168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 --2- 172.21.15.62:0/292896414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25fc109450 0x7f25fc11a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 -- 172.21.15.62:0/292896414 >> 172.21.15.62:0/292896414 conn(0x7f25fc06e140 msgr2=0x7f25fc072350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:22.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 -- 172.21.15.62:0/292896414 shutdown_connections 2024-09-12T00:00:22.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.134+0000 7f2604622640 1 -- 172.21.15.62:0/292896414 wait complete. 2024-09-12T00:00:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 460.0 s, 1264.1 tps, lat 369.158 ms stddev 2371.520 2024-09-12T00:00:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 -- 172.21.15.62:0/3956192544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a0102570 msgr2=0x7fc4a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 --2- 172.21.15.62:0/3956192544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a0102570 0x7fc4a01029b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc490009900 tx=0x7fc49002a480 comp rx=0 tx=0).stop 2024-09-12T00:00:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 -- 172.21.15.62:0/3956192544 shutdown_connections 2024-09-12T00:00:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 --2- 172.21.15.62:0/3956192544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a0102ef0 0x7fc4a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 --2- 172.21.15.62:0/3956192544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a0102570 0x7fc4a01029b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 --2- 172.21.15.62:0/3956192544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4a0100e20 0x7fc4a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 -- 172.21.15.62:0/3956192544 >> 172.21.15.62:0/3956192544 conn(0x7fc4a00fc7d0 msgr2=0x7fc4a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 -- 172.21.15.62:0/3956192544 shutdown_connections 2024-09-12T00:00:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 -- 172.21.15.62:0/3956192544 wait complete. 2024-09-12T00:00:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 Processor -- start 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.752+0000 7fc4a80d5640 1 -- start start 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a80d5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4a0100e20 0x7fc4a0197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a80d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a0102570 0x7fc4a0197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a80d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a0102ef0 0x7fc4a0197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a80d5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0110f80 con 0x7fc4a0102ef0 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a80d5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0110e00 con 0x7fc4a0102570 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a80d5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0111100 con 0x7fc4a0100e20 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a6672640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a0102ef0 0x7fc4a0197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a6672640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a0102ef0 0x7fc4a0197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41274/0 (socket says 172.21.15.62:41274) 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a6672640 1 -- 172.21.15.62:0/1733973442 learned_addr learned my addr 172.21.15.62:0/1733973442 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4a0100e20 0x7fc4a0197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a5670640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a0102570 0x7fc4a0197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a6672640 1 -- 172.21.15.62:0/1733973442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4a0100e20 msgr2=0x7fc4a0197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a6672640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4a0100e20 0x7fc4a0197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a6672640 1 -- 172.21.15.62:0/1733973442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a0102570 msgr2=0x7fc4a0197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a6672640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a0102570 0x7fc4a0197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a6672640 1 -- 172.21.15.62:0/1733973442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a01a69b0 con 0x7fc4a0102ef0 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4a0100e20 0x7fc4a0197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.753+0000 7fc4a6672640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a0102ef0 0x7fc4a0197bc0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc49800cce0 tx=0x7fc49800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.754+0000 7fc496ffd640 1 -- 172.21.15.62:0/1733973442 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49800f450 con 0x7fc4a0102ef0 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.754+0000 7fc496ffd640 1 -- 172.21.15.62:0/1733973442 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49800fad0 con 0x7fc4a0102ef0 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.754+0000 7fc4a80d5640 1 -- 172.21.15.62:0/1733973442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a01a6ca0 con 0x7fc4a0102ef0 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.754+0000 7fc496ffd640 1 -- 172.21.15.62:0/1733973442 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498018c10 con 0x7fc4a0102ef0 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.754+0000 7fc4a80d5640 1 -- 172.21.15.62:0/1733973442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a01a7160 con 0x7fc4a0102ef0 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.756+0000 7fc496ffd640 1 -- 172.21.15.62:0/1733973442 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc49800f5f0 con 0x7fc4a0102ef0 2024-09-12T00:00:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.756+0000 7fc4aa4ae740 1 Processor -- start 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.756+0000 7fc4aa4ae740 1 -- start start 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.756+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f67880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f72140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 0x55a423f74a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a423ecd2e0 con 0x55a423f67dc0 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a423e74ce0 con 0x55a423f72680 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a423ec92a0 con 0x55a423f674a0 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5e71640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f72140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5e71640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f72140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41278/0 (socket says 172.21.15.62:41278) 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5e71640 1 -- 172.21.15.62:0/1037785782 learned_addr learned my addr 172.21.15.62:0/1037785782 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5670640 1 --2- 172.21.15.62:0/1037785782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 0x55a423f74a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5670640 1 --2- 172.21.15.62:0/1037785782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f67880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5e71640 1 -- 172.21.15.62:0/1037785782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 msgr2=0x55a423f67880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/1037785782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f67880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5e71640 1 -- 172.21.15.62:0/1037785782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 msgr2=0x55a423f74a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/1037785782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 0x55a423f74a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5e71640 1 -- 172.21.15.62:0/1037785782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a423e59440 con 0x55a423f67dc0 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5670640 1 --2- 172.21.15.62:0/1037785782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f67880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc496ffd640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc48c0a2cf0 0x7fc48c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/1037785782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f72140 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0104d0 tx=0x7fc49c010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bee60b38e822d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc4a5670640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc48c0a2cf0 0x7fc48c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc496ffd640 1 -- 172.21.15.62:0/1733973442 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fc4980cfa20 con 0x7fc4a0102ef0 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc48bfff640 1 -- 172.21.15.62:0/1037785782 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c011760 con 0x55a423f67dc0 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc48bfff640 1 -- 172.21.15.62:0/1037785782 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49c011de0 con 0x55a423f67dc0 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.757+0000 7fc48bfff640 1 -- 172.21.15.62:0/1037785782 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00e430 con 0x55a423f67dc0 2024-09-12T00:00:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.758+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1037785782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 msgr2=0x55a423f72140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.758+0000 7fc4a5670640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc48c0a2cf0 0x7fc48c0a51b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc490009230 tx=0x7fc49003f040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:22.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.758+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/1037785782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f72140 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0104d0 tx=0x7fc49c010990 comp rx=0 tx=0).stop 2024-09-12T00:00:22.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.758+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1037785782 shutdown_connections 2024-09-12T00:00:22.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.758+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/1037785782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 0x55a423f74a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.758+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/1037785782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f72140 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.758+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/1037785782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f67880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.758+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1037785782 >> 172.21.15.62:0/1037785782 conn(0x55a423f63a30 msgr2=0x55a423f63e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:22.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.758+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1037785782 shutdown_connections 2024-09-12T00:00:22.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.758+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1037785782 wait complete. 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.759+0000 7fc4aa4ae740 1 -- start start 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.759+0000 7fc4aa4ae740 1 Processor -- start 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.759+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f71f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.759+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f6f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.759+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 0x55a423f6f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.759+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a423f75250 con 0x55a423f67dc0 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.759+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a423e88050 con 0x55a423f72680 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.759+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a423f753d0 con 0x55a423f674a0 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4aa4ae740 10 client.0 _ll_register_callbacks cb 0x55a423ec8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a5670640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f6f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a5670640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f6f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41286/0 (socket says 172.21.15.62:41286) 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a5670640 1 -- 172.21.15.62:0/1811305138 learned_addr learned my addr 172.21.15.62:0/1811305138 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a6672640 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 0x55a423f6f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f71f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a6672640 1 -- 172.21.15.62:0/1811305138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 msgr2=0x55a423f71f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a6672640 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f71f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a6672640 1 -- 172.21.15.62:0/1811305138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 msgr2=0x55a423f6f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a6672640 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f6f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a6672640 1 -- 172.21.15.62:0/1811305138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a423c3e3c0 con 0x55a423f72680 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f71f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a5670640 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f6f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4a6672640 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 0x55a423f6f740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc4980cd170 tx=0x7fc49809bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=917f173c97a30dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4897fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:00:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.760+0000 7fc4897fa640 1 -- 172.21.15.62:0/1811305138 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49809bf40 con 0x55a423f72680 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4897fa640 1 -- 172.21.15.62:0/1811305138 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fc4980a4720 con 0x55a423f72680 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1811305138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc49c011de0 con 0x55a423f72680 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4897fa640 1 -- 172.21.15.62:0/1811305138 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc4980a02f0 con 0x55a423f72680 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4aa4ae740 10 client.26435 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1811305138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a423f91b20 con 0x55a423f72680 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4897fa640 1 -- 172.21.15.62:0/1811305138 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49c011de0 con 0x55a423f72680 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4897fa640 1 -- 172.21.15.62:0/1811305138 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc4980baca0 con 0x55a423f72680 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4aa4ae740 10 client.26435 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4aa4ae740 10 client.26435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4aa4ae740 20 client.26435 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a423f92fe0 0x55a423f95430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4aa4ae740 4 client.26435 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.761+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1811305138 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a423eda850 con 0x55a423f92fe0 2024-09-12T00:00:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.762+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a423f92fe0 0x55a423f95430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.762+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a423f92fe0 0x55a423f95430 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1327b7765c5d010e server_cookie=8dd532c4989d9754 in_seq=0 out_seq=0 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.762+0000 7fc4897fa640 10 client.26435 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.762+0000 7fc4897fa640 1 -- 172.21.15.62:0/1811305138 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a423eda850 con 0x55a423f92fe0 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.762+0000 7fc4897fa640 10 client.26435 handle_command_reply: tid=0 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 client.26435 shutdown 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1811305138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 msgr2=0x55a423f6f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 0x55a423f6f740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc4980cd170 tx=0x7fc49809bb40 comp rx=0 tx=0).stop 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1811305138 shutdown_connections 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a423f92fe0 0x55a423f95430 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f72680 0x55a423f6f740 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f6f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/1811305138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f71f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4897fa640 0 client.26435 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1811305138 >> 172.21.15.62:0/1811305138 conn(0x55a423f63a30 msgr2=0x55a423f69710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1811305138 shutdown_connections 2024-09-12T00:00:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/1811305138 wait complete. 2024-09-12T00:00:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.763+0000 7fc4aa4ae740 20 client.26435 trim_cache size 0 max 16384 2024-09-12T00:00:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.780+0000 7fc4aa4ae740 1 Processor -- start 2024-09-12T00:00:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.780+0000 7fc4aa4ae740 1 -- start start 2024-09-12T00:00:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f62a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f67dc0 0x55a423f9ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f72680 0x55a423f9b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01a9a30 con 0x55a423f72680 2024-09-12T00:00:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01a73f0 con 0x55a423f67dc0 2024-09-12T00:00:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0072190 con 0x55a423f674a0 2024-09-12T00:00:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a6672640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f72680 0x55a423f9b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a6672640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f72680 0x55a423f9b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41300/0 (socket says 172.21.15.62:41300) 2024-09-12T00:00:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/2416333904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f62a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a6672640 1 -- 172.21.15.62:0/2416333904 learned_addr learned my addr 172.21.15.62:0/2416333904 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a5670640 1 --2- 172.21.15.62:0/2416333904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f67dc0 0x55a423f9ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a6672640 1 -- 172.21.15.62:0/2416333904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 msgr2=0x55a423f62a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a6672640 1 --2- 172.21.15.62:0/2416333904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f62a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a6672640 1 -- 172.21.15.62:0/2416333904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f67dc0 msgr2=0x55a423f9ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a6672640 1 --2- 172.21.15.62:0/2416333904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f67dc0 0x55a423f9ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a6672640 1 -- 172.21.15.62:0/2416333904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a423f6e1b0 con 0x55a423f72680 2024-09-12T00:00:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/2416333904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f62a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:00:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.781+0000 7fc4a6672640 1 --2- 172.21.15.62:0/2416333904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f72680 0x55a423f9b240 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc4980cd770 tx=0x7fc4980bc170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9595926ec9f211 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc48b7fe640 1 -- 172.21.15.62:0/2416333904 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49800f8e0 con 0x55a423f72680 2024-09-12T00:00:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc48b7fe640 1 -- 172.21.15.62:0/2416333904 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4980a4bb0 con 0x55a423f72680 2024-09-12T00:00:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc48b7fe640 1 -- 172.21.15.62:0/2416333904 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980bab80 con 0x55a423f72680 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/2416333904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f72680 msgr2=0x55a423f9b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/2416333904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f72680 0x55a423f9b240 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc4980cd770 tx=0x7fc4980bc170 comp rx=0 tx=0).stop 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/2416333904 shutdown_connections 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/2416333904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f72680 0x55a423f9b240 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/2416333904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f67dc0 0x55a423f9ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/2416333904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f674a0 0x55a423f62a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/2416333904 >> 172.21.15.62:0/2416333904 conn(0x55a423f63a30 msgr2=0x55a423f62350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.782+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/2416333904 shutdown_connections 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/2416333904 wait complete. 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4aa4ae740 1 -- start start 2024-09-12T00:00:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4aa4ae740 1 Processor -- start 2024-09-12T00:00:22.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f674a0 0x55a423f873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4aa4ae740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f72680 0x55a423f87e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a00727e0 con 0x55a423f67dc0 2024-09-12T00:00:22.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0073c50 con 0x55a423f674a0 2024-09-12T00:00:22.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4aa4ae740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0072eb0 con 0x55a423f72680 2024-09-12T00:00:22.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4aa4ae740 10 client.0 _ll_register_callbacks cb 0x7fc4980185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.783+0000 7fc4a5670640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a5670640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f878e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41310/0 (socket says 172.21.15.62:41310) 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a5670640 1 -- 172.21.15.62:0/140964555 learned_addr learned my addr 172.21.15.62:0/140964555 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f674a0 0x55a423f873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a6672640 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f72680 0x55a423f87e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a5670640 1 -- 172.21.15.62:0/140964555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f72680 msgr2=0x55a423f87e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a5670640 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f72680 0x55a423f87e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a5670640 1 -- 172.21.15.62:0/140964555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f674a0 msgr2=0x55a423f873a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a5670640 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f674a0 0x55a423f873a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a5670640 1 -- 172.21.15.62:0/140964555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a423f722c0 con 0x55a423f67dc0 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f674a0 0x55a423f873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4a5670640 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f878e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc490005940 tx=0x7fc490000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17a0c38c18c7d4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4897fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4897fa640 1 -- 172.21.15.62:0/140964555 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490035040 con 0x55a423f67dc0 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4897fa640 1 -- 172.21.15.62:0/140964555 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fc490044070 con 0x55a423f67dc0 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/140964555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4980a4bb0 con 0x55a423f67dc0 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4897fa640 1 -- 172.21.15.62:0/140964555 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc490031a10 con 0x55a423f67dc0 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4aa4ae740 10 client.16854 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.784+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/140964555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a423ffddc0 con 0x55a423f67dc0 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.785+0000 7fc4897fa640 1 -- 172.21.15.62:0/140964555 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4980a4bb0 con 0x55a423f67dc0 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.785+0000 7fc4897fa640 1 -- 172.21.15.62:0/140964555 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc49004ace0 con 0x55a423f67dc0 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.785+0000 7fc4aa4ae740 10 client.16854 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.785+0000 7fc4aa4ae740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.785+0000 7fc4aa4ae740 20 client.16854 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.785+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a423ffe5c0 0x55a4240009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.785+0000 7fc4aa4ae740 4 client.16854 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.785+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/140964555 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a423ed4d60 con 0x55a423ffe5c0 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.785+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a423ffe5c0 0x55a4240009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4a5e71640 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a423ffe5c0 0x55a4240009e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0bf0e02db74243 server_cookie=6ff9d7f137ba79c4 in_seq=0 out_seq=0 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4897fa640 10 client.16854 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4897fa640 1 -- 172.21.15.62:0/140964555 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a423ed4d60 con 0x55a423ffe5c0 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4897fa640 10 client.16854 handle_command_reply: tid=0 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 client.16854 shutdown 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/140964555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 msgr2=0x55a423f878e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f878e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc490005940 tx=0x7fc490000bc0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/140964555 shutdown_connections 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a423ffe5c0 0x55a4240009e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a423f72680 0x55a423f87e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a423f67dc0 0x55a423f878e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 --2- 172.21.15.62:0/140964555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a423f674a0 0x55a423f873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4897fa640 0 client.16854 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:22.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/140964555 >> 172.21.15.62:0/140964555 conn(0x55a423f63a30 msgr2=0x55a423f71280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:22.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/140964555 shutdown_connections 2024-09-12T00:00:22.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 1 -- 172.21.15.62:0/140964555 wait complete. 2024-09-12T00:00:22.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.786+0000 7fc4aa4ae740 20 client.16854 trim_cache size 0 max 16384 2024-09-12T00:00:22.795 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:00:22.795 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:00:22.795 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:00:22.795 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:00:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 -- 172.21.15.62:0/1733973442 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc48c0a2cf0 msgr2=0x7fc48c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc48c0a2cf0 0x7fc48c0a51b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc490009230 tx=0x7fc49003f040 comp rx=0 tx=0).stop 2024-09-12T00:00:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 -- 172.21.15.62:0/1733973442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a0102ef0 msgr2=0x7fc4a0197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a0102ef0 0x7fc4a0197bc0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc49800cce0 tx=0x7fc49800a620 comp rx=0 tx=0).stop 2024-09-12T00:00:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 -- 172.21.15.62:0/1733973442 shutdown_connections 2024-09-12T00:00:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc48c0a2cf0 0x7fc48c0a51b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4a0102ef0 0x7fc4a0197bc0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4a0102570 0x7fc4a0197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 --2- 172.21.15.62:0/1733973442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4a0100e20 0x7fc4a0197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 -- 172.21.15.62:0/1733973442 >> 172.21.15.62:0/1733973442 conn(0x7fc4a00fc7d0 msgr2=0x7fc4a0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 -- 172.21.15.62:0/1733973442 shutdown_connections 2024-09-12T00:00:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:22.787+0000 7fc4a80d5640 1 -- 172.21.15.62:0/1733973442 wait complete. 2024-09-12T00:00:23.517 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ccd456ca', quiesce took 4.9 seconds. Will release after: 14.499999999999998 2024-09-12T00:00:23.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:23 smithi120 ceph-mon[35182]: pgmap v255: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.9 MiB/s wr, 237 op/s 2024-09-12T00:00:23.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:23 smithi083 ceph-mon[34805]: pgmap v255: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.9 MiB/s wr, 237 op/s 2024-09-12T00:00:23.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:23 smithi062 ceph-mon[30958]: pgmap v255: 97 pgs: 97 active+clean; 3.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.9 MiB/s wr, 237 op/s 2024-09-12T00:00:25.130 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:00:25.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:00:25.396 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:25.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:25 smithi120 ceph-mon[35182]: pgmap v256: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 1.33k op/s 2024-09-12T00:00:25.651 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:25 smithi062 ceph-mon[30958]: pgmap v256: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 1.33k op/s 2024-09-12T00:00:25.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:25 smithi083 ceph-mon[34805]: pgmap v256: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 1.33k op/s 2024-09-12T00:00:25.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.864+0000 7f7dcb744640 1 -- 172.21.15.62:0/3978069852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dc4101280 msgr2=0x7f7dc410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.864+0000 7f7dcb744640 1 --2- 172.21.15.62:0/3978069852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dc4101280 0x7f7dc410df40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009900 tx=0x7f7db402a410 comp rx=0 tx=0).stop 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.865+0000 7f7dcb744640 1 -- 172.21.15.62:0/3978069852 shutdown_connections 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.865+0000 7f7dcb744640 1 --2- 172.21.15.62:0/3978069852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dc410e480 0x7f7dc4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.865+0000 7f7dcb744640 1 --2- 172.21.15.62:0/3978069852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dc4101280 0x7f7dc410df40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.865+0000 7f7dcb744640 1 --2- 172.21.15.62:0/3978069852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dc41008d0 0x7f7dc4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.865+0000 7f7dcb744640 1 -- 172.21.15.62:0/3978069852 >> 172.21.15.62:0/3978069852 conn(0x7f7dc40fc7a0 msgr2=0x7f7dc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.865+0000 7f7dcb744640 1 -- 172.21.15.62:0/3978069852 shutdown_connections 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.865+0000 7f7dcb744640 1 -- 172.21.15.62:0/3978069852 wait complete. 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.865+0000 7f7dcb744640 1 Processor -- start 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.865+0000 7f7dcb744640 1 -- start start 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dcb744640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dc41008d0 0x7f7dc419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:25.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dcb744640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dc4101280 0x7f7dc419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dcb744640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dc410e480 0x7f7dc419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dcb744640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc41141b0 con 0x7f7dc410e480 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dcb744640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4114030 con 0x7f7dc41008d0 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dcb744640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4114330 con 0x7f7dc4101280 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc9ce1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dc410e480 0x7f7dc419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc9ce1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dc410e480 0x7f7dc419f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41330/0 (socket says 172.21.15.62:41330) 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc9ce1640 1 -- 172.21.15.62:0/2459132905 learned_addr learned my addr 172.21.15.62:0/2459132905 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc8cdf640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dc4101280 0x7f7dc419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc94e0640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dc41008d0 0x7f7dc419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc9ce1640 1 -- 172.21.15.62:0/2459132905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dc4101280 msgr2=0x7f7dc419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc9ce1640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dc4101280 0x7f7dc419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:25.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc9ce1640 1 -- 172.21.15.62:0/2459132905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dc41008d0 msgr2=0x7f7dc419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc9ce1640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dc41008d0 0x7f7dc419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc9ce1640 1 -- 172.21.15.62:0/2459132905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc419f700 con 0x7f7dc410e480 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dc9ce1640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dc410e480 0x7f7dc419f020 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00cab0 tx=0x7f7dbc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dba7fc640 1 -- 172.21.15.62:0/2459132905 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc007e10 con 0x7f7dc410e480 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dba7fc640 1 -- 172.21.15.62:0/2459132905 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dbc00f930 con 0x7f7dc410e480 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dcb744640 1 -- 172.21.15.62:0/2459132905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc419f9f0 con 0x7f7dc410e480 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.866+0000 7f7dba7fc640 1 -- 172.21.15.62:0/2459132905 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc018b80 con 0x7f7dc410e480 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.867+0000 7f7dcb744640 1 -- 172.21.15.62:0/2459132905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc41a72d0 con 0x7f7dc410e480 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.867+0000 7f7dcb744640 1 -- 172.21.15.62:0/2459132905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc4100cb0 con 0x7f7dc410e480 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.869+0000 7f7dba7fc640 1 -- 172.21.15.62:0/2459132905 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7dbc00fae0 con 0x7f7dc410e480 2024-09-12T00:00:25.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.869+0000 7f7dba7fc640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7db00a2d70 0x7f7db00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:25.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.869+0000 7f7dc94e0640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7db00a2d70 0x7f7db00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:25.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.869+0000 7f7dba7fc640 1 -- 172.21.15.62:0/2459132905 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f7dbc0cf740 con 0x7f7dc410e480 2024-09-12T00:00:25.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.870+0000 7f7dc94e0640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7db00a2d70 0x7f7db00a5230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0005fd0 tx=0x7f7dc0005850 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:25.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:25.874+0000 7f7dba7fc640 1 -- 172.21.15.62:0/2459132905 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7dbc014030 con 0x7f7dc410e480 2024-09-12T00:00:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.084+0000 7f7dcb744640 1 -- 172.21.15.62:0/2459132905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dc4109080 con 0x7f7dc410e480 2024-09-12T00:00:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.085+0000 7f7dba7fc640 1 -- 172.21.15.62:0/2459132905 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11909 (secure 0 0 0) 0x7f7dbc09c300 con 0x7f7dc410e480 2024-09-12T00:00:26.085 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:00:26.085 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"btime":"2024-09-12T00:00:02:308078+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:00:02.079738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:00:26.086 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2024-09-12T00:00:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 -- 172.21.15.62:0/2459132905 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7db00a2d70 msgr2=0x7f7db00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7db00a2d70 0x7f7db00a5230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0005fd0 tx=0x7f7dc0005850 comp rx=0 tx=0).stop 2024-09-12T00:00:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 -- 172.21.15.62:0/2459132905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dc410e480 msgr2=0x7f7dc419f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dc410e480 0x7f7dc419f020 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00cab0 tx=0x7f7dbc00cf70 comp rx=0 tx=0).stop 2024-09-12T00:00:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 -- 172.21.15.62:0/2459132905 shutdown_connections 2024-09-12T00:00:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7db00a2d70 0x7f7db00a5230 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dc410e480 0x7f7dc419f020 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dc4101280 0x7f7dc419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 --2- 172.21.15.62:0/2459132905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dc41008d0 0x7f7dc419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 -- 172.21.15.62:0/2459132905 >> 172.21.15.62:0/2459132905 conn(0x7f7dc40fc7a0 msgr2=0x7f7dc40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 -- 172.21.15.62:0/2459132905 shutdown_connections 2024-09-12T00:00:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:26.087+0000 7f7dcb744640 1 -- 172.21.15.62:0/2459132905 wait complete. 2024-09-12T00:00:26.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:26 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2459132905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:26.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:26 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2459132905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:26.758 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:00:26.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:26 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2459132905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:27.018 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:27.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:27 smithi062 ceph-mon[30958]: pgmap v257: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-12T00:00:27.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 465.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:00:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.492+0000 7f767148b640 1 -- 172.21.15.62:0/3359775821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f766c1082a0 msgr2=0x7f766c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.492+0000 7f767148b640 1 --2- 172.21.15.62:0/3359775821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f766c1082a0 0x7f766c10ec40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7664009930 tx=0x7f766402a420 comp rx=0 tx=0).stop 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.492+0000 7f767148b640 1 -- 172.21.15.62:0/3359775821 shutdown_connections 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.492+0000 7f767148b640 1 --2- 172.21.15.62:0/3359775821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f766c1082a0 0x7f766c10ec40 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.492+0000 7f767148b640 1 --2- 172.21.15.62:0/3359775821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f766c1036e0 0x7f766c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.492+0000 7f767148b640 1 --2- 172.21.15.62:0/3359775821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f766c102d30 0x7f766c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.492+0000 7f767148b640 1 -- 172.21.15.62:0/3359775821 >> 172.21.15.62:0/3359775821 conn(0x7f766c0fc7d0 msgr2=0x7f766c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.492+0000 7f767148b640 1 -- 172.21.15.62:0/3359775821 shutdown_connections 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.492+0000 7f767148b640 1 -- 172.21.15.62:0/3359775821 wait complete. 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.492+0000 7f767148b640 1 Processor -- start 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f767148b640 1 -- start start 2024-09-12T00:00:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f767148b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f766c102d30 0x7f766c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f767148b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f766c1036e0 0x7f766c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f767148b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f766c1082a0 0x7f766c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f767148b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1120a0 con 0x7f766c1036e0 2024-09-12T00:00:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f767148b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f766c111f20 con 0x7f766c102d30 2024-09-12T00:00:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f767148b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f766c112220 con 0x7f766c1082a0 2024-09-12T00:00:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f766c1036e0 0x7f766c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f766c1036e0 0x7f766c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41348/0 (socket says 172.21.15.62:41348) 2024-09-12T00:00:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766a7fc640 1 -- 172.21.15.62:0/3193016560 learned_addr learned my addr 172.21.15.62:0/3193016560 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:27.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766affd640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f766c102d30 0x7f766c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766b7fe640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f766c1082a0 0x7f766c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766a7fc640 1 -- 172.21.15.62:0/3193016560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f766c1082a0 msgr2=0x7f766c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766a7fc640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f766c1082a0 0x7f766c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766a7fc640 1 -- 172.21.15.62:0/3193016560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f766c102d30 msgr2=0x7f766c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766a7fc640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f766c102d30 0x7f766c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766a7fc640 1 -- 172.21.15.62:0/3193016560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766c1a6980 con 0x7f766c1036e0 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766affd640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f766c102d30 0x7f766c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.493+0000 7f766a7fc640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f766c1036e0 0x7f766c100ed0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f765400c8c0 tx=0x7f765400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.494+0000 7f7663fff640 1 -- 172.21.15.62:0/3193016560 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7654007bb0 con 0x7f766c1036e0 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.494+0000 7f767148b640 1 -- 172.21.15.62:0/3193016560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766c1a6c70 con 0x7f766c1036e0 2024-09-12T00:00:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.494+0000 7f7663fff640 1 -- 172.21.15.62:0/3193016560 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7654007d50 con 0x7f766c1036e0 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.494+0000 7f7663fff640 1 -- 172.21.15.62:0/3193016560 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7654018830 con 0x7f766c1036e0 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.494+0000 7f767148b640 1 -- 172.21.15.62:0/3193016560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f766c1a71b0 con 0x7f766c1036e0 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.496+0000 7f7663fff640 1 -- 172.21.15.62:0/3193016560 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7654018a10 con 0x7f766c1036e0 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.496+0000 7f7673864740 1 Processor -- start 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.496+0000 7f7673864740 1 -- start start 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f7673864740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 0x55e6cb11e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f7673864740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11e5d0 0x55e6cb11ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f7673864740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb129370 0x55e6cb12b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f7673864740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e6cb0842f0 con 0x55e6cb11e5d0 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f7673864740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e6cb02bcf0 con 0x55e6cb129370 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f7673864740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e6cb0802b0 con 0x55e6cb11fb40 2024-09-12T00:00:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f7663fff640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76580a2cf0 0x7f76580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11e5d0 0x55e6cb11ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11e5d0 0x55e6cb11ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41358/0 (socket says 172.21.15.62:41358) 2024-09-12T00:00:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766affd640 1 -- 172.21.15.62:0/2104147686 learned_addr learned my addr 172.21.15.62:0/2104147686 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766b7fe640 1 --2- 172.21.15.62:0/2104147686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 0x55e6cb11e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766b7fe640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76580a2cf0 0x7f76580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f7663fff640 1 -- 172.21.15.62:0/3193016560 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f76540cf5a0 con 0x7f766c1036e0 2024-09-12T00:00:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766a7fc640 1 --2- 172.21.15.62:0/2104147686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb129370 0x55e6cb12b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766affd640 1 -- 172.21.15.62:0/2104147686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 msgr2=0x55e6cb11e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766affd640 1 --2- 172.21.15.62:0/2104147686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 0x55e6cb11e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766affd640 1 -- 172.21.15.62:0/2104147686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb129370 msgr2=0x55e6cb12b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766affd640 1 --2- 172.21.15.62:0/2104147686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb129370 0x55e6cb12b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766affd640 1 -- 172.21.15.62:0/2104147686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6cb0103a0 con 0x55e6cb11e5d0 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766a7fc640 1 --2- 172.21.15.62:0/2104147686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb129370 0x55e6cb12b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.497+0000 7f766affd640 1 --2- 172.21.15.62:0/2104147686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11e5d0 0x55e6cb11ea10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f765c0103a0 tx=0x7f765c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53e7fa09f63711e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.498+0000 7f766b7fe640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76580a2cf0 0x7f76580a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7664031550 tx=0x7f7664009260 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.498+0000 7f76617fa640 1 -- 172.21.15.62:0/2104147686 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c016070 con 0x55e6cb11e5d0 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.498+0000 7f76617fa640 1 -- 172.21.15.62:0/2104147686 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f765c012bf0 con 0x55e6cb11e5d0 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.498+0000 7f76617fa640 1 -- 172.21.15.62:0/2104147686 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c01bd40 con 0x55e6cb11e5d0 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.498+0000 7f7673864740 1 -- 172.21.15.62:0/2104147686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11e5d0 msgr2=0x55e6cb11ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.498+0000 7f7673864740 1 --2- 172.21.15.62:0/2104147686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11e5d0 0x55e6cb11ea10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f765c0103a0 tx=0x7f765c010860 comp rx=0 tx=0).stop 2024-09-12T00:00:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 -- 172.21.15.62:0/2104147686 shutdown_connections 2024-09-12T00:00:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 --2- 172.21.15.62:0/2104147686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb129370 0x55e6cb12b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 --2- 172.21.15.62:0/2104147686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11e5d0 0x55e6cb11ea10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 --2- 172.21.15.62:0/2104147686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 0x55e6cb11e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 -- 172.21.15.62:0/2104147686 >> 172.21.15.62:0/2104147686 conn(0x55e6cb0e7310 msgr2=0x55e6cb11ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 -- 172.21.15.62:0/2104147686 shutdown_connections 2024-09-12T00:00:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 -- 172.21.15.62:0/2104147686 wait complete. 2024-09-12T00:00:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 -- start start 2024-09-12T00:00:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 Processor -- start 2024-09-12T00:00:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb128d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb125fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb129370 0x55e6cb1265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e6cb12c070 con 0x55e6cb11fb40 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e6cb03f060 con 0x55e6cb11e5d0 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e6cb12c1f0 con 0x55e6cb129370 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f7673864740 10 client.0 _ll_register_callbacks cb 0x55e6cad928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f766a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb125fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f766a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb125fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41368/0 (socket says 172.21.15.62:41368) 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f766a7fc640 1 -- 172.21.15.62:0/3586105251 learned_addr learned my addr 172.21.15.62:0/3586105251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.499+0000 7f766affd640 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb128d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f766b7fe640 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb129370 0x55e6cb1265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f766a7fc640 1 -- 172.21.15.62:0/3586105251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb129370 msgr2=0x55e6cb1265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f766a7fc640 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb129370 0x55e6cb1265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f766a7fc640 1 -- 172.21.15.62:0/3586105251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 msgr2=0x55e6cb128d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f766a7fc640 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb128d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f766a7fc640 1 -- 172.21.15.62:0/3586105251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6cadf53c0 con 0x55e6cb11fb40 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f766a7fc640 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb125fe0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f76540ccd50 tx=0x7f765409c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995c55c31495fc38 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f76527fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f76527fc640 1 -- 172.21.15.62:0/3586105251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765409fe70 con 0x55e6cb11fb40 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f76527fc640 1 -- 172.21.15.62:0/3586105251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f7654018d00 con 0x55e6cb11fb40 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f7673864740 1 -- 172.21.15.62:0/3586105251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f765c012bf0 con 0x55e6cb11fb40 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f76527fc640 1 -- 172.21.15.62:0/3586105251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f76540a4730 con 0x55e6cb11fb40 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f7673864740 10 client.16872 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.500+0000 7f7673864740 1 -- 172.21.15.62:0/3586105251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e6cb137aa0 con 0x55e6cb11fb40 2024-09-12T00:00:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.501+0000 7f76527fc640 1 -- 172.21.15.62:0/3586105251 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f765c012bf0 con 0x55e6cb11fb40 2024-09-12T00:00:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.501+0000 7f76527fc640 1 -- 172.21.15.62:0/3586105251 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f76540a42a0 con 0x55e6cb11fb40 2024-09-12T00:00:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.501+0000 7f7673864740 10 client.16872 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.501+0000 7f7673864740 10 client.16872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:00:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.501+0000 7f7673864740 20 client.16872 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.501+0000 7f7673864740 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e6cb138f60 0x55e6cb13b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.501+0000 7f7673864740 4 client.16872 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:00:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.501+0000 7f7673864740 1 -- 172.21.15.62:0/3586105251 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6cb091860 con 0x55e6cb138f60 2024-09-12T00:00:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.501+0000 7f766affd640 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e6cb138f60 0x55e6cb13b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.502+0000 7f766affd640 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e6cb138f60 0x55e6cb13b350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5818bf9f21376392 server_cookie=a5fc3eee8f20f3e4 in_seq=0 out_seq=0 2024-09-12T00:00:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.502+0000 7f76527fc640 10 client.16872 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:00:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.502+0000 7f76527fc640 1 -- 172.21.15.62:0/3586105251 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6cb091860 con 0x55e6cb138f60 2024-09-12T00:00:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.502+0000 7f76527fc640 10 client.16872 handle_command_reply: tid=0 2024-09-12T00:00:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.502+0000 7f7673864740 1 client.16872 shutdown 2024-09-12T00:00:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 1 -- 172.21.15.62:0/3586105251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 msgr2=0x55e6cb125fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb125fe0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f76540ccd50 tx=0x7f765409c1c0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 1 -- 172.21.15.62:0/3586105251 shutdown_connections 2024-09-12T00:00:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e6cb138f60 0x55e6cb13b350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb129370 0x55e6cb1265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb125fe0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f76527fc640 0 client.16872 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:00:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 1 --2- 172.21.15.62:0/3586105251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb128d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 1 -- 172.21.15.62:0/3586105251 >> 172.21.15.62:0/3586105251 conn(0x55e6cb0e7310 msgr2=0x55e6cb120960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 1 -- 172.21.15.62:0/3586105251 shutdown_connections 2024-09-12T00:00:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 1 -- 172.21.15.62:0/3586105251 wait complete. 2024-09-12T00:00:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.503+0000 7f7673864740 20 client.16872 trim_cache size 0 max 16384 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f7673864740 1 Processor -- start 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f7673864740 1 -- start start 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f7673864740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb135310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f7673864740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 0x55e6cb1258a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f7673864740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb129370 0x55e6cb121120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f7673864740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1b24b0 con 0x55e6cb129370 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f7673864740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1afd80 con 0x55e6cb11e5d0 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f7673864740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1adbf0 con 0x55e6cb11fb40 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f766b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb129370 0x55e6cb121120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f766b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb129370 0x55e6cb121120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41370/0 (socket says 172.21.15.62:41370) 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f766b7fe640 1 -- 172.21.15.62:0/109741313 learned_addr learned my addr 172.21.15.62:0/109741313 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f766a7fc640 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 0x55e6cb1258a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.516+0000 7f766affd640 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb135310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f766b7fe640 1 -- 172.21.15.62:0/109741313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 msgr2=0x55e6cb1258a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f766b7fe640 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 0x55e6cb1258a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f766b7fe640 1 -- 172.21.15.62:0/109741313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 msgr2=0x55e6cb135310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f766b7fe640 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb135310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f766b7fe640 1 -- 172.21.15.62:0/109741313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6cb124530 con 0x55e6cb129370 2024-09-12T00:00:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f766a7fc640 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 0x55e6cb1258a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:00:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f766affd640 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb135310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:00:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f766b7fe640 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb129370 0x55e6cb121120 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7664009930 tx=0x7f7664040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43c701b66d9f50d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f7660ff9640 1 -- 172.21.15.62:0/109741313 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664035040 con 0x55e6cb129370 2024-09-12T00:00:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f7660ff9640 1 -- 172.21.15.62:0/109741313 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7664050070 con 0x55e6cb129370 2024-09-12T00:00:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.517+0000 7f7660ff9640 1 -- 172.21.15.62:0/109741313 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766404b870 con 0x55e6cb129370 2024-09-12T00:00:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 -- 172.21.15.62:0/109741313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb129370 msgr2=0x55e6cb121120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb129370 0x55e6cb121120 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7664009930 tx=0x7f7664040070 comp rx=0 tx=0).stop 2024-09-12T00:00:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 -- 172.21.15.62:0/109741313 shutdown_connections 2024-09-12T00:00:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb129370 0x55e6cb121120 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb11fb40 0x55e6cb1258a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 --2- 172.21.15.62:0/109741313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb135310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 -- 172.21.15.62:0/109741313 >> 172.21.15.62:0/109741313 conn(0x55e6cb0e7310 msgr2=0x55e6cb1250d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 -- 172.21.15.62:0/109741313 shutdown_connections 2024-09-12T00:00:27.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 -- 172.21.15.62:0/109741313 wait complete. 2024-09-12T00:00:27.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 -- start start 2024-09-12T00:00:27.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 Processor -- start 2024-09-12T00:00:27.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb131920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb12cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb129370 0x55e6cb12d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1ae240 con 0x55e6cb11fb40 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1af6b0 con 0x55e6cb11e5d0 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.518+0000 7f7673864740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1ae910 con 0x55e6cb129370 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f7673864740 10 client.0 _ll_register_callbacks cb 0x7f7654097e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb12cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb12cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41378/0 (socket says 172.21.15.62:41378) 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766a7fc640 1 -- 172.21.15.62:0/3556792015 learned_addr learned my addr 172.21.15.62:0/3556792015 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766b7fe640 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb129370 0x55e6cb12d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766affd640 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766a7fc640 1 -- 172.21.15.62:0/3556792015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb129370 msgr2=0x55e6cb12d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766a7fc640 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb129370 0x55e6cb12d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766a7fc640 1 -- 172.21.15.62:0/3556792015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 msgr2=0x55e6cb131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766a7fc640 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766a7fc640 1 -- 172.21.15.62:0/3556792015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6cb12d6d0 con 0x55e6cb11fb40 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f766a7fc640 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb12cc50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f76540ccda0 tx=0x7f765400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1f41e68bc3632b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f76527fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f76527fc640 1 -- 172.21.15.62:0/3556792015 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765409b890 con 0x55e6cb11fb40 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f76527fc640 1 -- 172.21.15.62:0/3556792015 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f7654018d00 con 0x55e6cb11fb40 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f7673864740 1 -- 172.21.15.62:0/3556792015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7664050070 con 0x55e6cb11fb40 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f76527fc640 1 -- 172.21.15.62:0/3556792015 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f765409fe70 con 0x55e6cb11fb40 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f7673864740 10 client.16878 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.519+0000 7f7673864740 1 -- 172.21.15.62:0/3556792015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e6cb1a38c0 con 0x55e6cb11fb40 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.520+0000 7f76527fc640 1 -- 172.21.15.62:0/3556792015 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7664050070 con 0x55e6cb11fb40 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.520+0000 7f76527fc640 1 -- 172.21.15.62:0/3556792015 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f765409a980 con 0x55e6cb11fb40 2024-09-12T00:00:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.520+0000 7f7673864740 10 client.16878 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.520+0000 7f7673864740 10 client.16878 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.520+0000 7f7673864740 20 client.16878 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.520+0000 7f7673864740 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e6cb1a3e60 0x55e6cb1a6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.520+0000 7f7673864740 4 client.16878 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.520+0000 7f7673864740 1 -- 172.21.15.62:0/3556792015 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6cb08bd70 con 0x55e6cb1a3e60 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.520+0000 7f766affd640 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e6cb1a3e60 0x55e6cb1a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f766affd640 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e6cb1a3e60 0x55e6cb1a6280 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9cbe6bf2c04af991 server_cookie=19e1278ab251eb61 in_seq=0 out_seq=0 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f76527fc640 10 client.16878 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f76527fc640 1 -- 172.21.15.62:0/3556792015 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6cb08bd70 con 0x55e6cb1a3e60 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f76527fc640 10 client.16878 handle_command_reply: tid=0 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 client.16878 shutdown 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 -- 172.21.15.62:0/3556792015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 msgr2=0x55e6cb12cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb12cc50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f76540ccda0 tx=0x7f765400a5f0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 -- 172.21.15.62:0/3556792015 shutdown_connections 2024-09-12T00:00:27.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e6cb1a3e60 0x55e6cb1a6280 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6cb129370 0x55e6cb12d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e6cb11fb40 0x55e6cb12cc50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 --2- 172.21.15.62:0/3556792015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6cb11e5d0 0x55e6cb131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 -- 172.21.15.62:0/3556792015 >> 172.21.15.62:0/3556792015 conn(0x55e6cb0e7310 msgr2=0x55e6cb1194e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:27.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f76527fc640 0 client.16878 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:00:27.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 -- 172.21.15.62:0/3556792015 shutdown_connections 2024-09-12T00:00:27.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 1 -- 172.21.15.62:0/3556792015 wait complete. 2024-09-12T00:00:27.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.521+0000 7f7673864740 20 client.16878 trim_cache size 0 max 16384 2024-09-12T00:00:27.530 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:00:27.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 -- 172.21.15.62:0/3193016560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76580a2cf0 msgr2=0x7f76580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76580a2cf0 0x7f76580a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7664031550 tx=0x7f7664009260 comp rx=0 tx=0).stop 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 -- 172.21.15.62:0/3193016560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f766c1036e0 msgr2=0x7f766c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f766c1036e0 0x7f766c100ed0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f765400c8c0 tx=0x7f765400cd80 comp rx=0 tx=0).stop 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 -- 172.21.15.62:0/3193016560 shutdown_connections 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76580a2cf0 0x7f76580a51b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f766c1082a0 0x7f766c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f766c1036e0 0x7f766c100ed0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 --2- 172.21.15.62:0/3193016560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f766c102d30 0x7f766c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 -- 172.21.15.62:0/3193016560 >> 172.21.15.62:0/3193016560 conn(0x7f766c0fc7d0 msgr2=0x7f766c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 -- 172.21.15.62:0/3193016560 shutdown_connections 2024-09-12T00:00:27.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:27.522+0000 7f767148b640 1 -- 172.21.15.62:0/3193016560 wait complete. 2024-09-12T00:00:27.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:27 smithi083 ceph-mon[34805]: pgmap v257: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-12T00:00:27.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:27 smithi120 ceph-mon[35182]: pgmap v257: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-12T00:00:28.438 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:00:28.707 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.155+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/1359060299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2ac102d30 msgr2=0x7fc2ac103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.155+0000 7fc2b0cf3640 1 --2- 172.21.15.62:0/1359060299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2ac102d30 0x7fc2ac103110 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009930 tx=0x7fc29c02a420 comp rx=0 tx=0).stop 2024-09-12T00:00:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.155+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/1359060299 shutdown_connections 2024-09-12T00:00:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.155+0000 7fc2b0cf3640 1 --2- 172.21.15.62:0/1359060299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac1082a0 0x7fc2ac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.155+0000 7fc2b0cf3640 1 --2- 172.21.15.62:0/1359060299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2ac1036e0 0x7fc2ac107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.155+0000 7fc2b0cf3640 1 --2- 172.21.15.62:0/1359060299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2ac102d30 0x7fc2ac103110 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.155+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/1359060299 >> 172.21.15.62:0/1359060299 conn(0x7fc2ac0fc7d0 msgr2=0x7fc2ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.155+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/1359060299 shutdown_connections 2024-09-12T00:00:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.155+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/1359060299 wait complete. 2024-09-12T00:00:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2b0cf3640 1 Processor -- start 2024-09-12T00:00:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2b0cf3640 1 -- start start 2024-09-12T00:00:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2b0cf3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2ac102d30 0x7fc2ac100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2b0cf3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac1036e0 0x7fc2ac100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2b0cf3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2ac1082a0 0x7fc2ac101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2b0cf3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1120a0 con 0x7fc2ac102d30 2024-09-12T00:00:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2b0cf3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac111f20 con 0x7fc2ac1036e0 2024-09-12T00:00:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2b0cf3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac112220 con 0x7fc2ac1082a0 2024-09-12T00:00:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2ac102d30 0x7fc2ac100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2ac102d30 0x7fc2ac100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41410/0 (socket says 172.21.15.62:41410) 2024-09-12T00:00:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2aa59c640 1 -- 172.21.15.62:0/2856897539 learned_addr learned my addr 172.21.15.62:0/2856897539 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac1036e0 0x7fc2ac100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2aad9d640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2ac1082a0 0x7fc2ac101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2aa59c640 1 -- 172.21.15.62:0/2856897539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2ac1082a0 msgr2=0x7fc2ac101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2ac1082a0 0x7fc2ac101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2aa59c640 1 -- 172.21.15.62:0/2856897539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac1036e0 msgr2=0x7fc2ac100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.156+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac1036e0 0x7fc2ac100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.157+0000 7fc2aa59c640 1 -- 172.21.15.62:0/2856897539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac1a6a60 con 0x7fc2ac102d30 2024-09-12T00:00:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.157+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2ac102d30 0x7fc2ac100990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009230 tx=0x7fc29c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.157+0000 7fc2a37fe640 1 -- 172.21.15.62:0/2856897539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c03f030 con 0x7fc2ac102d30 2024-09-12T00:00:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.157+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/2856897539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac1a6cf0 con 0x7fc2ac102d30 2024-09-12T00:00:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.157+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/2856897539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac1a71c0 con 0x7fc2ac102d30 2024-09-12T00:00:29.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.157+0000 7fc2a37fe640 1 -- 172.21.15.62:0/2856897539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c003ad0 con 0x7fc2ac102d30 2024-09-12T00:00:29.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.157+0000 7fc2a37fe640 1 -- 172.21.15.62:0/2856897539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c035070 con 0x7fc2ac102d30 2024-09-12T00:00:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.159+0000 7fc2a37fe640 1 -- 172.21.15.62:0/2856897539 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc29c042030 con 0x7fc2ac102d30 2024-09-12T00:00:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.160+0000 7fc2b30cc740 1 Processor -- start 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.160+0000 7fc2b30cc740 1 -- start start 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.160+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 0x56350e9b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.160+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.160+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 0x56350e9c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.160+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56350e91d2f0 con 0x56350e9b75d0 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.160+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56350e8c4cf0 con 0x56350e9b8b40 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.160+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56350e9192b0 con 0x56350e9c2370 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.160+0000 7fc2a37fe640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2980a2d70 0x7fc2980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.160+0000 7fc2aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41420/0 (socket says 172.21.15.62:41420) 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2aa59c640 1 -- 172.21.15.62:0/1084529412 learned_addr learned my addr 172.21.15.62:0/1084529412 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2aad9d640 1 --2- 172.21.15.62:0/1084529412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 0x56350e9b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/1084529412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 0x56350e9c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2aad9d640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2980a2d70 0x7fc2980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a37fe640 1 -- 172.21.15.62:0/2856897539 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fc29c0ef7d0 con 0x7fc2ac102d30 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a9d9b640 1 -- 172.21.15.62:0/1084529412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 msgr2=0x56350e9b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/1084529412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 0x56350e9b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a9d9b640 1 -- 172.21.15.62:0/1084529412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 msgr2=0x56350e9b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/1084529412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a9d9b640 1 -- 172.21.15.62:0/1084529412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350e8a93a0 con 0x56350e9c2370 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2aad9d640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2980a2d70 0x7fc2980a5230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009fd0 tx=0x7fc2a4009600 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/1084529412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 0x56350e9c4760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac06a370 tx=0x7fc29400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52f1ae19a62846f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:29.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a0ff9640 1 -- 172.21.15.62:0/1084529412 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294005220 con 0x56350e9c2370 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a0ff9640 1 -- 172.21.15.62:0/1084529412 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc294004c80 con 0x56350e9c2370 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.161+0000 7fc2a0ff9640 1 -- 172.21.15.62:0/1084529412 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2940187a0 con 0x56350e9c2370 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.162+0000 7fc2b30cc740 1 -- 172.21.15.62:0/1084529412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 msgr2=0x56350e9c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.162+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/1084529412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 0x56350e9c4760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac06a370 tx=0x7fc29400f790 comp rx=0 tx=0).stop 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.162+0000 7fc2b30cc740 1 -- 172.21.15.62:0/1084529412 shutdown_connections 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.162+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/1084529412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 0x56350e9c4760 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.162+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/1084529412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.162+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/1084529412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 0x56350e9b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.162+0000 7fc2b30cc740 1 -- 172.21.15.62:0/1084529412 >> 172.21.15.62:0/1084529412 conn(0x56350e980310 msgr2=0x56350e9b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.162+0000 7fc2b30cc740 1 -- 172.21.15.62:0/1084529412 shutdown_connections 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.162+0000 7fc2b30cc740 1 -- 172.21.15.62:0/1084529412 wait complete. 2024-09-12T00:00:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2b30cc740 1 -- start start 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2b30cc740 1 Processor -- start 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b75d0 0x56350e9c1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9bef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9c2370 0x56350e9bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56350e9c5070 con 0x56350e9c2370 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56350e8d8060 con 0x56350e9b75d0 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56350e9c51f0 con 0x56350e9b8b40 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2b30cc740 10 client.0 _ll_register_callbacks cb 0x56350e62b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2a9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9bef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2aad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9c2370 0x56350e9bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2a9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9bef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:55026/0 (socket says 172.21.15.62:55026) 2024-09-12T00:00:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b75d0 0x56350e9c1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.163+0000 7fc2a9d9b640 1 -- 172.21.15.62:0/2557045394 learned_addr learned my addr 172.21.15.62:0/2557045394 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2aad9d640 1 -- 172.21.15.62:0/2557045394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 msgr2=0x56350e9bef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2aad9d640 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9bef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2aad9d640 1 -- 172.21.15.62:0/2557045394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b75d0 msgr2=0x56350e9c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2aad9d640 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b75d0 0x56350e9c1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2aad9d640 1 -- 172.21.15.62:0/2557045394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350e68e3c0 con 0x56350e9c2370 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9bef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b75d0 0x56350e9c1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2aad9d640 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9c2370 0x56350e9bf490 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4005fd0 tx=0x7fc2a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108efa95f38f83ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc291ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc291ffb640 1 -- 172.21.15.62:0/2557045394 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a401eb30 con 0x56350e9c2370 2024-09-12T00:00:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2557045394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc294004c80 con 0x56350e9c2370 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc291ffb640 1 -- 172.21.15.62:0/2557045394 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fc2a401ecd0 con 0x56350e9c2370 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc291ffb640 1 -- 172.21.15.62:0/2557045394 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc2a4004b20 con 0x56350e9c2370 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2b30cc740 10 client.16890 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2557045394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56350e9d0ae0 con 0x56350e9c2370 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.164+0000 7fc291ffb640 1 -- 172.21.15.62:0/2557045394 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc294004c80 con 0x56350e9c2370 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.165+0000 7fc291ffb640 1 -- 172.21.15.62:0/2557045394 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc2a400f040 con 0x56350e9c2370 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.165+0000 7fc2b30cc740 10 client.16890 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.165+0000 7fc2b30cc740 10 client.16890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.165+0000 7fc2b30cc740 20 client.16890 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.165+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56350e9d1ef0 0x56350e9d43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.165+0000 7fc2b30cc740 4 client.16890 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.165+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2557045394 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56350e92a860 con 0x56350e9d1ef0 2024-09-12T00:00:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.165+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56350e9d1ef0 0x56350e9d43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.166+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56350e9d1ef0 0x56350e9d43b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7e4d739424c7389 server_cookie=bace8f5f33161cea in_seq=0 out_seq=0 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.166+0000 7fc291ffb640 10 client.16890 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.166+0000 7fc291ffb640 1 -- 172.21.15.62:0/2557045394 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56350e92a860 con 0x56350e9d1ef0 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.166+0000 7fc291ffb640 10 client.16890 handle_command_reply: tid=0 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.166+0000 7fc2b30cc740 1 client.16890 shutdown 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.166+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2557045394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9c2370 msgr2=0x56350e9bf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.166+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9c2370 0x56350e9bf490 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4005fd0 tx=0x7fc2a4004a40 comp rx=0 tx=0).stop 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.167+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2557045394 shutdown_connections 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.167+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56350e9d1ef0 0x56350e9d43b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.167+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9c2370 0x56350e9bf490 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.167+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9bef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.167+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2557045394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b75d0 0x56350e9c1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.167+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2557045394 >> 172.21.15.62:0/2557045394 conn(0x56350e980310 msgr2=0x56350e9b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.167+0000 7fc291ffb640 0 client.16890 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:00:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.167+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2557045394 shutdown_connections 2024-09-12T00:00:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.167+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2557045394 wait complete. 2024-09-12T00:00:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.167+0000 7fc2b30cc740 20 client.16890 trim_cache size 0 max 16384 2024-09-12T00:00:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2b30cc740 1 Processor -- start 2024-09-12T00:00:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2b30cc740 1 -- start start 2024-09-12T00:00:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9d9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9c2370 0x56350e9da1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1b2310 con 0x56350e9b75d0 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1afbe0 con 0x56350e9c2370 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1ada50 con 0x56350e9b8b40 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41438/0 (socket says 172.21.15.62:41438) 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2aa59c640 1 -- 172.21.15.62:0/3293981620 learned_addr learned my addr 172.21.15.62:0/3293981620 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/3293981620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9d9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2aad9d640 1 --2- 172.21.15.62:0/3293981620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9c2370 0x56350e9da1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2aa59c640 1 -- 172.21.15.62:0/3293981620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 msgr2=0x56350e9d9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/3293981620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9d9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2aa59c640 1 -- 172.21.15.62:0/3293981620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9c2370 msgr2=0x56350e9da1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:29.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.193+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/3293981620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9c2370 0x56350e9da1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.194+0000 7fc2aa59c640 1 -- 172.21.15.62:0/3293981620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350e9ba120 con 0x56350e9b75d0 2024-09-12T00:00:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.194+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/3293981620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9d9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:00:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.194+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/3293981620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9ce340 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0b8150 tx=0x7fc29c0f1ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e431ae19628d29e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.194+0000 7fc293fff640 1 -- 172.21.15.62:0/3293981620 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c049040 con 0x56350e9b75d0 2024-09-12T00:00:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.194+0000 7fc293fff640 1 -- 172.21.15.62:0/3293981620 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c0bbcf0 con 0x56350e9b75d0 2024-09-12T00:00:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.194+0000 7fc293fff640 1 -- 172.21.15.62:0/3293981620 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0ed1a0 con 0x56350e9b75d0 2024-09-12T00:00:29.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.195+0000 7fc2b30cc740 1 -- 172.21.15.62:0/3293981620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 msgr2=0x56350e9ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:29.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.195+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/3293981620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9ce340 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0b8150 tx=0x7fc29c0f1ce0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.195+0000 7fc2b30cc740 1 -- 172.21.15.62:0/3293981620 shutdown_connections 2024-09-12T00:00:29.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.195+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/3293981620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9c2370 0x56350e9da1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.195+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/3293981620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9b8b40 0x56350e9d9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.195+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/3293981620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9ce340 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.195+0000 7fc2b30cc740 1 -- 172.21.15.62:0/3293981620 >> 172.21.15.62:0/3293981620 conn(0x56350e980310 msgr2=0x56350e9bd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.195+0000 7fc2b30cc740 1 -- 172.21.15.62:0/3293981620 shutdown_connections 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.195+0000 7fc2b30cc740 1 -- 172.21.15.62:0/3293981620 wait complete. 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.195+0000 7fc2b30cc740 1 -- start start 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2b30cc740 1 Processor -- start 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 0x56350e9bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2b30cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 0x56350e9bc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1ae0a0 con 0x56350e9b75d0 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1af510 con 0x56350e9b8b40 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2b30cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1ae770 con 0x56350e9c2370 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2b30cc740 10 client.0 _ll_register_callbacks cb 0x7fc2a4012760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41452/0 (socket says 172.21.15.62:41452) 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2aa59c640 1 -- 172.21.15.62:0/2614247040 learned_addr learned my addr 172.21.15.62:0/2614247040 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 0x56350e9bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2aad9d640 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 0x56350e9bc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2aa59c640 1 -- 172.21.15.62:0/2614247040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 msgr2=0x56350e9bc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 0x56350e9bc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2aa59c640 1 -- 172.21.15.62:0/2614247040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 msgr2=0x56350e9bbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 0x56350e9bbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2aa59c640 1 -- 172.21.15.62:0/2614247040 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350e9bc650 con 0x56350e9b75d0 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 0x56350e9bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.196+0000 7fc2aa59c640 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9caaa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0bb9f0 tx=0x7fc29c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887873af1606551d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc291ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc291ffb640 1 -- 172.21.15.62:0/2614247040 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c049070 con 0x56350e9b75d0 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc291ffb640 1 -- 172.21.15.62:0/2614247040 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fc29c0ed1a0 con 0x56350e9b75d0 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2614247040 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc29c0bbcf0 con 0x56350e9b75d0 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc291ffb640 1 -- 172.21.15.62:0/2614247040 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc29c0c4960 con 0x56350e9b75d0 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc2b30cc740 10 client.16896 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2614247040 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56350ea3c9f0 con 0x56350e9b75d0 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc291ffb640 1 -- 172.21.15.62:0/2614247040 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c0bbcf0 con 0x56350e9b75d0 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc291ffb640 1 -- 172.21.15.62:0/2614247040 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc29c0c44d0 con 0x56350e9b75d0 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc2b30cc740 10 client.16896 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc2b30cc740 10 client.16896 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc2b30cc740 20 client.16896 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56350ea3d0f0 0x56350ea3f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc2b30cc740 4 client.16896 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:00:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.197+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2614247040 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56350e924d70 con 0x56350ea3d0f0 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.198+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56350ea3d0f0 0x56350ea3f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.198+0000 7fc2a9d9b640 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56350ea3d0f0 0x56350ea3f510 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c492a313dbefc7b server_cookie=7eb5e072b26f8476 in_seq=0 out_seq=0 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.198+0000 7fc291ffb640 10 client.16896 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.198+0000 7fc291ffb640 1 -- 172.21.15.62:0/2614247040 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56350e924d70 con 0x56350ea3d0f0 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.198+0000 7fc291ffb640 10 client.16896 handle_command_reply: tid=0 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.198+0000 7fc2b30cc740 1 client.16896 shutdown 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2614247040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 msgr2=0x56350e9caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9caaa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0bb9f0 tx=0x7fc29c0045f0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2614247040 shutdown_connections 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56350ea3d0f0 0x56350ea3f510 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc291ffb640 0 client.16896 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56350e9c2370 0x56350e9bc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56350e9b8b40 0x56350e9bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 1 --2- 172.21.15.62:0/2614247040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350e9b75d0 0x56350e9caaa0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2614247040 >> 172.21.15.62:0/2614247040 conn(0x56350e980310 msgr2=0x56350e9c07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:29.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2614247040 shutdown_connections 2024-09-12T00:00:29.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 1 -- 172.21.15.62:0/2614247040 wait complete. 2024-09-12T00:00:29.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.199+0000 7fc2b30cc740 20 client.16896 trim_cache size 0 max 16384 2024-09-12T00:00:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:00:29.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/2856897539 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2980a2d70 msgr2=0x7fc2980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:29.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2980a2d70 0x7fc2980a5230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009fd0 tx=0x7fc2a4009600 comp rx=0 tx=0).stop 2024-09-12T00:00:29.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/2856897539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2ac102d30 msgr2=0x7fc2ac100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:29.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2ac102d30 0x7fc2ac100990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009230 tx=0x7fc29c004270 comp rx=0 tx=0).stop 2024-09-12T00:00:29.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/2856897539 shutdown_connections 2024-09-12T00:00:29.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc2980a2d70 0x7fc2980a5230 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2ac1082a0 0x7fc2ac101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac1036e0 0x7fc2ac100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 --2- 172.21.15.62:0/2856897539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2ac102d30 0x7fc2ac100990 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:29.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/2856897539 >> 172.21.15.62:0/2856897539 conn(0x7fc2ac0fc7d0 msgr2=0x7fc2ac105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:29.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/2856897539 shutdown_connections 2024-09-12T00:00:29.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:29.200+0000 7fc2b0cf3640 1 -- 172.21.15.62:0/2856897539 wait complete. 2024-09-12T00:00:29.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:29 smithi083 ceph-mon[34805]: pgmap v258: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 23 MiB/s wr, 1.22k op/s 2024-09-12T00:00:29.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:29 smithi062 ceph-mon[30958]: pgmap v258: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 23 MiB/s wr, 1.22k op/s 2024-09-12T00:00:29.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:29 smithi120 ceph-mon[35182]: pgmap v258: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 23 MiB/s wr, 1.22k op/s 2024-09-12T00:00:29.993 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:00:30.252 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:30.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.713+0000 7f822d0f6640 1 -- 172.21.15.62:0/4252671399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8228100e20 msgr2=0x7f8228101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:30.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.713+0000 7f822d0f6640 1 --2- 172.21.15.62:0/4252671399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8228100e20 0x7f8228101200 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8218009900 tx=0x7f821802a480 comp rx=0 tx=0).stop 2024-09-12T00:00:30.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.713+0000 7f822d0f6640 1 -- 172.21.15.62:0/4252671399 shutdown_connections 2024-09-12T00:00:30.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.713+0000 7f822d0f6640 1 --2- 172.21.15.62:0/4252671399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228102ef0 0x7f822810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.713+0000 7f822d0f6640 1 --2- 172.21.15.62:0/4252671399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228102570 0x7f82281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.713+0000 7f822d0f6640 1 --2- 172.21.15.62:0/4252671399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8228100e20 0x7f8228101200 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.713+0000 7f822d0f6640 1 -- 172.21.15.62:0/4252671399 >> 172.21.15.62:0/4252671399 conn(0x7f82280fc7d0 msgr2=0x7f82280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:30.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.713+0000 7f822d0f6640 1 -- 172.21.15.62:0/4252671399 shutdown_connections 2024-09-12T00:00:30.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.713+0000 7f822d0f6640 1 -- 172.21.15.62:0/4252671399 wait complete. 2024-09-12T00:00:30.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822d0f6640 1 Processor -- start 2024-09-12T00:00:30.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822d0f6640 1 -- start start 2024-09-12T00:00:30.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822d0f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228100e20 0x7f82281973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822d0f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8228102570 0x7f8228197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822d0f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228102ef0 0x7f822819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822d0f6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8228110f80 con 0x7f8228102570 2024-09-12T00:00:30.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822d0f6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8228110e00 con 0x7f8228100e20 2024-09-12T00:00:30.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822d0f6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8228111100 con 0x7f8228102ef0 2024-09-12T00:00:30.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8228102570 0x7f8228197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8228102570 0x7f8228197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41468/0 (socket says 172.21.15.62:41468) 2024-09-12T00:00:30.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.714+0000 7f822659c640 1 -- 172.21.15.62:0/3309943507 learned_addr learned my addr 172.21.15.62:0/3309943507 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:30.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f822759e640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228102ef0 0x7f822819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f8226d9d640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228100e20 0x7f82281973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f822659c640 1 -- 172.21.15.62:0/3309943507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228102ef0 msgr2=0x7f822819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:30.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f822659c640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228102ef0 0x7f822819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f822659c640 1 -- 172.21.15.62:0/3309943507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228100e20 msgr2=0x7f82281973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:30.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f822659c640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228100e20 0x7f82281973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f822659c640 1 -- 172.21.15.62:0/3309943507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822819c390 con 0x7f8228102570 2024-09-12T00:00:30.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f822659c640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8228102570 0x7f8228197920 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f821c00c890 tx=0x7f821c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:30.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f820ffff640 1 -- 172.21.15.62:0/3309943507 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c004220 con 0x7f8228102570 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f820ffff640 1 -- 172.21.15.62:0/3309943507 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821c0043c0 con 0x7f8228102570 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f822d0f6640 1 -- 172.21.15.62:0/3309943507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822819c680 con 0x7f8228102570 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f820ffff640 1 -- 172.21.15.62:0/3309943507 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0049e0 con 0x7f8228102570 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.715+0000 7f822d0f6640 1 -- 172.21.15.62:0/3309943507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82281a3f60 con 0x7f8228102570 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.718+0000 7f820ffff640 1 -- 172.21.15.62:0/3309943507 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f821c0079b0 con 0x7f8228102570 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.718+0000 7f822f4cf740 1 Processor -- start 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.718+0000 7f822f4cf740 1 -- start start 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.718+0000 7f820ffff640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f82100a2cf0 0x7f82100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.718+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a170e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.718+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a170ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.718+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 0x55d0a171b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.718+0000 7f822f4cf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d0a16722f0 con 0x55d0a170fd50 2024-09-12T00:00:30.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.718+0000 7f822f4cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d0a1619cf0 con 0x55d0a1719580 2024-09-12T00:00:30.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.718+0000 7f822f4cf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d0a166e2b0 con 0x55d0a170e7e0 2024-09-12T00:00:30.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f822759e640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f82100a2cf0 0x7f82100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a170e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f820ffff640 1 -- 172.21.15.62:0/3309943507 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f821c0cf460 con 0x7f8228102570 2024-09-12T00:00:30.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a170e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41482/0 (socket says 172.21.15.62:41482) 2024-09-12T00:00:30.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f8226d9d640 1 -- 172.21.15.62:0/2758352937 learned_addr learned my addr 172.21.15.62:0/2758352937 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:30.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f822659c640 1 --2- 172.21.15.62:0/2758352937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a170ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f822759e640 1 --2- 172.21.15.62:0/2758352937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 0x55d0a171b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f8226d9d640 1 -- 172.21.15.62:0/2758352937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 msgr2=0x55d0a170ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f8226d9d640 1 --2- 172.21.15.62:0/2758352937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a170ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f8226d9d640 1 -- 172.21.15.62:0/2758352937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 msgr2=0x55d0a171b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f8226d9d640 1 --2- 172.21.15.62:0/2758352937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 0x55d0a171b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f8226d9d640 1 -- 172.21.15.62:0/2758352937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a15fe3a0 con 0x55d0a170fd50 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f822759e640 1 --2- 172.21.15.62:0/2758352937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 0x55d0a171b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f822759e640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f82100a2cf0 0x7f82100a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8220005fd0 tx=0x7f8220004410 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f8226d9d640 1 --2- 172.21.15.62:0/2758352937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a170e2a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8218009900 tx=0x7f8218004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6095829a6d0e39d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f820d7fa640 1 -- 172.21.15.62:0/2758352937 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218035030 con 0x55d0a170fd50 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f820d7fa640 1 -- 172.21.15.62:0/2758352937 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218040070 con 0x55d0a170fd50 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.719+0000 7f820d7fa640 1 -- 172.21.15.62:0/2758352937 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218031a70 con 0x55d0a170fd50 2024-09-12T00:00:30.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.720+0000 7f822f4cf740 1 -- 172.21.15.62:0/2758352937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 msgr2=0x55d0a170e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.720+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2758352937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a170e2a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8218009900 tx=0x7f8218004170 comp rx=0 tx=0).stop 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.720+0000 7f822f4cf740 1 -- 172.21.15.62:0/2758352937 shutdown_connections 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.720+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2758352937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 0x55d0a171b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.720+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2758352937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a170ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.720+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2758352937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a170e2a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.720+0000 7f822f4cf740 1 -- 172.21.15.62:0/2758352937 >> 172.21.15.62:0/2758352937 conn(0x55d0a16d5310 msgr2=0x55d0a170ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.720+0000 7f822f4cf740 1 -- 172.21.15.62:0/2758352937 shutdown_connections 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.720+0000 7f822f4cf740 1 -- 172.21.15.62:0/2758352937 wait complete. 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.720+0000 7f822f4cf740 1 -- start start 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822f4cf740 1 Processor -- start 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170e7e0 0x55d0a1718f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a17161e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a1719580 0x55d0a1716780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822f4cf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d0a171c220 con 0x55d0a170fd50 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822f4cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d0a162d060 con 0x55d0a170e7e0 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822f4cf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d0a171c3a0 con 0x55d0a1719580 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822f4cf740 10 client.0 _ll_register_callbacks cb 0x55d0a13808c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a17161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a17161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41486/0 (socket says 172.21.15.62:41486) 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822659c640 1 -- 172.21.15.62:0/3007043909 learned_addr learned my addr 172.21.15.62:0/3007043909 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f8226d9d640 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170e7e0 0x55d0a1718f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822759e640 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a1719580 0x55d0a1716780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822659c640 1 -- 172.21.15.62:0/3007043909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a1719580 msgr2=0x55d0a1716780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822659c640 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a1719580 0x55d0a1716780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822659c640 1 -- 172.21.15.62:0/3007043909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170e7e0 msgr2=0x55d0a1718f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:30.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822659c640 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170e7e0 0x55d0a1718f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822659c640 1 -- 172.21.15.62:0/3007043909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a13e33c0 con 0x55d0a170fd50 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f8226d9d640 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170e7e0 0x55d0a1718f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f822659c640 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a17161e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f821c007590 tx=0x7f821c09b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4093ec92dcd2a2af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.721+0000 7f820aff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f820aff5640 1 -- 172.21.15.62:0/3007043909 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c01d040 con 0x55d0a170fd50 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f822f4cf740 1 -- 172.21.15.62:0/3007043909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8218040070 con 0x55d0a170fd50 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f820aff5640 1 -- 172.21.15.62:0/3007043909 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f821c06e2b0 con 0x55d0a170fd50 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f820aff5640 1 -- 172.21.15.62:0/3007043909 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f821c063c40 con 0x55d0a170fd50 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f822f4cf740 10 client.16908 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f822f4cf740 1 -- 172.21.15.62:0/3007043909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d0a1727c70 con 0x55d0a170fd50 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f820aff5640 1 -- 172.21.15.62:0/3007043909 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218040070 con 0x55d0a170fd50 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f820aff5640 1 -- 172.21.15.62:0/3007043909 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f821c09bc30 con 0x55d0a170fd50 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f822f4cf740 10 client.16908 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:30.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f822f4cf740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f822f4cf740 20 client.16908 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f822f4cf740 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d0a1729440 0x55d0a172b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f822f4cf740 4 client.16908 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f822f4cf740 1 -- 172.21.15.62:0/3007043909 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0a167f860 con 0x55d0a1729440 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.722+0000 7f8226d9d640 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d0a1729440 0x55d0a172b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.723+0000 7f820aff5640 10 client.16908 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.723+0000 7f8226d9d640 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d0a1729440 0x55d0a172b8e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695716c883d6e3c1 server_cookie=4b8c37c497130064 in_seq=0 out_seq=0 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.723+0000 7f820aff5640 1 -- 172.21.15.62:0/3007043909 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0a167f860 con 0x55d0a1729440 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.723+0000 7f820aff5640 10 client.16908 handle_command_reply: tid=0 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.723+0000 7f822f4cf740 1 client.16908 shutdown 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 1 -- 172.21.15.62:0/3007043909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 msgr2=0x55d0a17161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a17161e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f821c007590 tx=0x7f821c09b3f0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 1 -- 172.21.15.62:0/3007043909 shutdown_connections 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d0a1729440 0x55d0a172b8e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a1719580 0x55d0a1716780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a17161e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 1 --2- 172.21.15.62:0/3007043909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170e7e0 0x55d0a1718f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 1 -- 172.21.15.62:0/3007043909 >> 172.21.15.62:0/3007043909 conn(0x55d0a16d5310 msgr2=0x55d0a1710b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f820aff5640 0 client.16908 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 1 -- 172.21.15.62:0/3007043909 shutdown_connections 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 1 -- 172.21.15.62:0/3007043909 wait complete. 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.724+0000 7f822f4cf740 20 client.16908 trim_cache size 0 max 16384 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822f4cf740 1 Processor -- start 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822f4cf740 1 -- start start 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a17254c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170fd50 0x55d0a1714ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a1719580 0x55d0a1714ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822f4cf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f82281adee0 con 0x55d0a1719580 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822f4cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82281ab7b0 con 0x55d0a170fd50 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822f4cf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82281a9620 con 0x55d0a170e7e0 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a1719580 0x55d0a1714ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a1719580 0x55d0a1714ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41494/0 (socket says 172.21.15.62:41494) 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.737+0000 7f822759e640 1 -- 172.21.15.62:0/2487868311 learned_addr learned my addr 172.21.15.62:0/2487868311 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f8226d9d640 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a17254c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f822659c640 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170fd50 0x55d0a1714ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f822759e640 1 -- 172.21.15.62:0/2487868311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 msgr2=0x55d0a17254c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f822759e640 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a17254c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f822759e640 1 -- 172.21.15.62:0/2487868311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170fd50 msgr2=0x55d0a1714ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:30.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f822759e640 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170fd50 0x55d0a1714ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f822759e640 1 -- 172.21.15.62:0/2487868311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a170f5b0 con 0x55d0a1719580 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f822659c640 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170fd50 0x55d0a1714ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f8226d9d640 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a17254c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f822759e640 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a1719580 0x55d0a1714ff0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8220012770 tx=0x7f822001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c550a8114ffde3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f820cff9640 1 -- 172.21.15.62:0/2487868311 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82200206c0 con 0x55d0a1719580 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f820cff9640 1 -- 172.21.15.62:0/2487868311 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8220024070 con 0x55d0a1719580 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.738+0000 7f820cff9640 1 -- 172.21.15.62:0/2487868311 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220029bd0 con 0x55d0a1719580 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 -- 172.21.15.62:0/2487868311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a1719580 msgr2=0x55d0a1714ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a1719580 0x55d0a1714ff0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8220012770 tx=0x7f822001d000 comp rx=0 tx=0).stop 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 -- 172.21.15.62:0/2487868311 shutdown_connections 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a1719580 0x55d0a1714ff0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a170fd50 0x55d0a1714ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2487868311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a17254c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 -- 172.21.15.62:0/2487868311 >> 172.21.15.62:0/2487868311 conn(0x55d0a16d5310 msgr2=0x55d0a17114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 -- 172.21.15.62:0/2487868311 shutdown_connections 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 -- 172.21.15.62:0/2487868311 wait complete. 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 -- start start 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.739+0000 7f822f4cf740 1 Processor -- start 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a171cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a171d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822f4cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 0x55d0a17921b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822f4cf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f82281a9c70 con 0x55d0a170fd50 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822f4cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82281ab0e0 con 0x55d0a1719580 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822f4cf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82281aa340 con 0x55d0a170e7e0 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822f4cf740 10 client.0 _ll_register_callbacks cb 0x7f821c097c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:30.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a171d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a171d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41502/0 (socket says 172.21.15.62:41502) 2024-09-12T00:00:30.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822659c640 1 -- 172.21.15.62:0/2958992594 learned_addr learned my addr 172.21.15.62:0/2958992594 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:30.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822759e640 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 0x55d0a17921b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f8226d9d640 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a171cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822659c640 1 -- 172.21.15.62:0/2958992594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 msgr2=0x55d0a171cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:30.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822659c640 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a171cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822659c640 1 -- 172.21.15.62:0/2958992594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 msgr2=0x55d0a17921b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:30.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822659c640 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 0x55d0a17921b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822659c640 1 -- 172.21.15.62:0/2958992594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a1792770 con 0x55d0a170fd50 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822659c640 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a171d0d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f821c000c00 tx=0x7f821c09be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a61a10435c0a2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f820aff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f820aff5640 1 -- 172.21.15.62:0/2958992594 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c01d040 con 0x55d0a170fd50 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f820aff5640 1 -- 172.21.15.62:0/2958992594 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f821c0a4300 con 0x55d0a170fd50 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822f4cf740 1 -- 172.21.15.62:0/2958992594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8220024070 con 0x55d0a170fd50 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f820aff5640 1 -- 172.21.15.62:0/2958992594 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f821c06e6b0 con 0x55d0a170fd50 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.740+0000 7f822f4cf740 10 client.16914 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f822f4cf740 1 -- 172.21.15.62:0/2958992594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d0a17089b0 con 0x55d0a170fd50 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f820aff5640 1 -- 172.21.15.62:0/2958992594 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8220024070 con 0x55d0a170fd50 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f820aff5640 1 -- 172.21.15.62:0/2958992594 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f821c0860d0 con 0x55d0a170fd50 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f822f4cf740 10 client.16914 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f822f4cf740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:00:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f822f4cf740 20 client.16914 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d0a1709130 0x55d0a1709540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f822f4cf740 4 client.16914 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f822f4cf740 1 -- 172.21.15.62:0/2958992594 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0a1679d70 con 0x55d0a1709130 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f8226d9d640 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d0a1709130 0x55d0a1709540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f820aff5640 10 client.16914 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.741+0000 7f8226d9d640 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d0a1709130 0x55d0a1709540 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d82fe7af04adec server_cookie=28518de9e332704c in_seq=0 out_seq=0 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f820aff5640 1 -- 172.21.15.62:0/2958992594 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0a1679d70 con 0x55d0a1709130 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f820aff5640 10 client.16914 handle_command_reply: tid=0 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 client.16914 shutdown 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 -- 172.21.15.62:0/2958992594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 msgr2=0x55d0a171d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a171d0d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f821c000c00 tx=0x7f821c09be60 comp rx=0 tx=0).stop 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 -- 172.21.15.62:0/2958992594 shutdown_connections 2024-09-12T00:00:30.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55d0a1709130 0x55d0a1709540 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0a1719580 0x55d0a17921b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0a170fd50 0x55d0a171d0d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f820aff5640 0 client.16914 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:30.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 --2- 172.21.15.62:0/2958992594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d0a170e7e0 0x55d0a171cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 -- 172.21.15.62:0/2958992594 >> 172.21.15.62:0/2958992594 conn(0x55d0a16d5310 msgr2=0x55d0a17074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:30.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 -- 172.21.15.62:0/2958992594 shutdown_connections 2024-09-12T00:00:30.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 1 -- 172.21.15.62:0/2958992594 wait complete. 2024-09-12T00:00:30.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.742+0000 7f822f4cf740 20 client.16914 trim_cache size 0 max 16384 2024-09-12T00:00:30.760 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:00:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 -- 172.21.15.62:0/3309943507 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f82100a2cf0 msgr2=0x7f82100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f82100a2cf0 0x7f82100a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8220005fd0 tx=0x7f8220004410 comp rx=0 tx=0).stop 2024-09-12T00:00:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 -- 172.21.15.62:0/3309943507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8228102570 msgr2=0x7f8228197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8228102570 0x7f8228197920 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f821c00c890 tx=0x7f821c00cd50 comp rx=0 tx=0).stop 2024-09-12T00:00:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 -- 172.21.15.62:0/3309943507 shutdown_connections 2024-09-12T00:00:30.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f82100a2cf0 0x7f82100a51b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8228102ef0 0x7f822819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8228102570 0x7f8228197920 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 --2- 172.21.15.62:0/3309943507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228100e20 0x7f82281973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:30.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 -- 172.21.15.62:0/3309943507 >> 172.21.15.62:0/3309943507 conn(0x7f82280fc7d0 msgr2=0x7f82280fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:30.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 -- 172.21.15.62:0/3309943507 shutdown_connections 2024-09-12T00:00:30.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:30.743+0000 7f822d0f6640 1 -- 172.21.15.62:0/3309943507 wait complete. 2024-09-12T00:00:31.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:31 smithi083 ceph-mon[34805]: pgmap v259: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-09-12T00:00:31.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:31 smithi062 ceph-mon[30958]: pgmap v259: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-09-12T00:00:31.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:31 smithi120 ceph-mon[35182]: pgmap v259: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-09-12T00:00:32.431 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:00:32.432 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:00:32.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 470.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:00:32.703 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:32 smithi062 ceph-mon[30958]: pgmap v260: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-09-12T00:00:32.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:32 smithi120 ceph-mon[35182]: pgmap v260: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-09-12T00:00:32.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:32 smithi083 ceph-mon[34805]: pgmap v260: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-09-12T00:00:33.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.209+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3054830904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7781082a0 msgr2=0x7fe77810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:33.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.209+0000 7fe77d7f2640 1 --2- 172.21.15.62:0/3054830904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7781082a0 0x7fe77810ec40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe768009cd0 tx=0x7fe76802a8e0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.209+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3054830904 shutdown_connections 2024-09-12T00:00:33.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.209+0000 7fe77d7f2640 1 --2- 172.21.15.62:0/3054830904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7781082a0 0x7fe77810ec40 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.209+0000 7fe77d7f2640 1 --2- 172.21.15.62:0/3054830904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7781036e0 0x7fe778107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.209+0000 7fe77d7f2640 1 --2- 172.21.15.62:0/3054830904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe778102d30 0x7fe778103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.209+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3054830904 >> 172.21.15.62:0/3054830904 conn(0x7fe7780fc7d0 msgr2=0x7fe7780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.209+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3054830904 shutdown_connections 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.209+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3054830904 wait complete. 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe77d7f2640 1 Processor -- start 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe77d7f2640 1 -- start start 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe77d7f2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe778102d30 0x7fe778198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe77d7f2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7781036e0 0x7fe778198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe77d7f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7781082a0 0x7fe77819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe77d7f2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781120a0 con 0x7fe7781036e0 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe77d7f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe778111f20 con 0x7fe7781082a0 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe77d7f2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe778112220 con 0x7fe778102d30 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe7767fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7781036e0 0x7fe778198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe7767fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7781036e0 0x7fe778198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57416/0 (socket says 172.21.15.62:57416) 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe7767fc640 1 -- 172.21.15.62:0/3008626214 learned_addr learned my addr 172.21.15.62:0/3008626214 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe776ffd640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe778102d30 0x7fe778198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe7777fe640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7781082a0 0x7fe77819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe7767fc640 1 -- 172.21.15.62:0/3008626214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe778102d30 msgr2=0x7fe778198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe7767fc640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe778102d30 0x7fe778198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe7767fc640 1 -- 172.21.15.62:0/3008626214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7781082a0 msgr2=0x7fe77819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe7767fc640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7781082a0 0x7fe77819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.210+0000 7fe7767fc640 1 -- 172.21.15.62:0/3008626214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77819d4e0 con 0x7fe7781036e0 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.211+0000 7fe7767fc640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7781036e0 0x7fe778198a70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe76000d950 tx=0x7fe76000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.211+0000 7fe76ffff640 1 -- 172.21.15.62:0/3008626214 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760017070 con 0x7fe7781036e0 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.211+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3008626214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77819d7d0 con 0x7fe7781036e0 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.211+0000 7fe76ffff640 1 -- 172.21.15.62:0/3008626214 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe760009c30 con 0x7fe7781036e0 2024-09-12T00:00:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.211+0000 7fe76ffff640 1 -- 172.21.15.62:0/3008626214 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600047a0 con 0x7fe7781036e0 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.211+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3008626214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7781a50b0 con 0x7fe7781036e0 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.213+0000 7fe76ffff640 1 -- 172.21.15.62:0/3008626214 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe760004020 con 0x7fe7781036e0 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.213+0000 7fe77fbcb740 1 Processor -- start 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe77fbcb740 1 -- start start 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 0x55c15780ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c15781a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 0x55c15781d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c157775280 con 0x55c15781ad40 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c15771cb10 con 0x55c157810480 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c157771240 con 0x55c15780fb60 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe7767fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 0x55c15781d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c15781a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c15781a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:51304/0 (socket says 172.21.15.62:51304) 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe776ffd640 1 -- 172.21.15.62:0/3424950774 learned_addr learned my addr 172.21.15.62:0/3424950774 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe7777fe640 1 --2- 172.21.15.62:0/3424950774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 0x55c15780ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.214+0000 7fe76ffff640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7640a2cf0 0x7fe7640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe776ffd640 1 -- 172.21.15.62:0/3424950774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 msgr2=0x55c15780ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe7777fe640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7640a2cf0 0x7fe7640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe776ffd640 1 --2- 172.21.15.62:0/3424950774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 0x55c15780ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe776ffd640 1 -- 172.21.15.62:0/3424950774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 msgr2=0x55c15781d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe776ffd640 1 --2- 172.21.15.62:0/3424950774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 0x55c15781d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe76ffff640 1 -- 172.21.15.62:0/3008626214 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fe7600ceff0 con 0x7fe7781036e0 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe776ffd640 1 -- 172.21.15.62:0/3424950774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1577291f0 con 0x55c157810480 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe776ffd640 1 --2- 172.21.15.62:0/3424950774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c15781a800 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe77000b760 tx=0x7fe77000bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767045132fceb3df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe76d7fa640 1 -- 172.21.15.62:0/3424950774 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770003dd0 con 0x55c157810480 2024-09-12T00:00:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe7777fe640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7640a2cf0 0x7fe7640a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe768004ab0 tx=0x7fe76802a8e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe76d7fa640 1 -- 172.21.15.62:0/3424950774 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe770015070 con 0x55c157810480 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.215+0000 7fe76d7fa640 1 -- 172.21.15.62:0/3424950774 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770011440 con 0x55c157810480 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3424950774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 msgr2=0x55c15781a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/3424950774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c15781a800 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe77000b760 tx=0x7fe77000bc20 comp rx=0 tx=0).stop 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3424950774 shutdown_connections 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/3424950774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 0x55c15781d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/3424950774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c15781a800 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/3424950774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 0x55c15780ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3424950774 >> 172.21.15.62:0/3424950774 conn(0x55c15780c0f0 msgr2=0x55c15780c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3424950774 shutdown_connections 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3424950774 wait complete. 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 -- start start 2024-09-12T00:00:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.216+0000 7fe77fbcb740 1 Processor -- start 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c157817bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c157818130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 0x55c157818670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c15781d910 con 0x55c15780fb60 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c15772ff10 con 0x55c157810480 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c15781da90 con 0x55c15781ad40 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe77fbcb740 10 client.0 _ll_register_callbacks cb 0x55c1576f8a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c157817bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c157817bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57442/0 (socket says 172.21.15.62:57442) 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe776ffd640 1 -- 172.21.15.62:0/1624181822 learned_addr learned my addr 172.21.15.62:0/1624181822 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe7767fc640 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c157818130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe7777fe640 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 0x55c157818670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe776ffd640 1 -- 172.21.15.62:0/1624181822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 msgr2=0x55c157818670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe776ffd640 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 0x55c157818670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe776ffd640 1 -- 172.21.15.62:0/1624181822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 msgr2=0x55c157818130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe776ffd640 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c157818130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe776ffd640 1 -- 172.21.15.62:0/1624181822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1574e66a0 con 0x55c15780fb60 2024-09-12T00:00:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe7777fe640 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 0x55c157818670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:00:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe776ffd640 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c157817bf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe7700039c0 tx=0x7fe7700048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e7ae566d306ab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe75e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe75e7fc640 1 -- 172.21.15.62:0/1624181822 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770028060 con 0x55c15780fb60 2024-09-12T00:00:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe75e7fc640 1 -- 172.21.15.62:0/1624181822 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fe770011440 con 0x55c15780fb60 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.217+0000 7fe77fbcb740 1 -- 172.21.15.62:0/1624181822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe770015070 con 0x55c15780fb60 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe75e7fc640 1 -- 172.21.15.62:0/1624181822 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe770011640 con 0x55c15780fb60 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe77fbcb740 10 client.16926 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe77fbcb740 1 -- 172.21.15.62:0/1624181822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c1578294e0 con 0x55c15780fb60 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe75e7fc640 1 -- 172.21.15.62:0/1624181822 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe770015070 con 0x55c15780fb60 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe75e7fc640 1 -- 172.21.15.62:0/1624181822 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe770022070 con 0x55c15780fb60 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe77fbcb740 10 client.16926 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe77fbcb740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe77fbcb740 20 client.16926 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c15782a9a0 0x55c15782cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe77fbcb740 4 client.16926 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe77fbcb740 1 -- 172.21.15.62:0/1624181822 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1577827f0 con 0x55c15782a9a0 2024-09-12T00:00:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.218+0000 7fe7767fc640 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c15782a9a0 0x55c15782cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.219+0000 7fe7767fc640 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c15782a9a0 0x55c15782cd90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c2aefa53d4411a server_cookie=29ef35ad079db952 in_seq=0 out_seq=0 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.219+0000 7fe75e7fc640 10 client.16926 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.219+0000 7fe75e7fc640 1 -- 172.21.15.62:0/1624181822 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1577827f0 con 0x55c15782a9a0 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.219+0000 7fe75e7fc640 10 client.16926 handle_command_reply: tid=0 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.219+0000 7fe77fbcb740 1 client.16926 shutdown 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.219+0000 7fe77fbcb740 1 -- 172.21.15.62:0/1624181822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 msgr2=0x55c157817bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.219+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c157817bf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe7700039c0 tx=0x7fe7700048d0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.220+0000 7fe77fbcb740 1 -- 172.21.15.62:0/1624181822 shutdown_connections 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.220+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c15782a9a0 0x55c15782cd90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.220+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 0x55c157818670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.220+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c157818130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.220+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/1624181822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c157817bf0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.220+0000 7fe75e7fc640 0 client.16926 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.220+0000 7fe77fbcb740 1 -- 172.21.15.62:0/1624181822 >> 172.21.15.62:0/1624181822 conn(0x55c15780c0f0 msgr2=0x55c157811dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.220+0000 7fe77fbcb740 1 -- 172.21.15.62:0/1624181822 shutdown_connections 2024-09-12T00:00:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.220+0000 7fe77fbcb740 1 -- 172.21.15.62:0/1624181822 wait complete. 2024-09-12T00:00:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.220+0000 7fe77fbcb740 20 client.16926 trim_cache size 0 max 16384 2024-09-12T00:00:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe77fbcb740 1 Processor -- start 2024-09-12T00:00:33.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe77fbcb740 1 -- start start 2024-09-12T00:00:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 0x55c1578125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c157816aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 0x55c157832330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781b0210 con 0x55c15781ad40 2024-09-12T00:00:33.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781adae0 con 0x55c157810480 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781ab950 con 0x55c15780fb60 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe7777fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 0x55c157832330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe7777fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 0x55c157832330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57456/0 (socket says 172.21.15.62:57456) 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.238+0000 7fe7777fe640 1 -- 172.21.15.62:0/868475593 learned_addr learned my addr 172.21.15.62:0/868475593 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe7767fc640 1 --2- 172.21.15.62:0/868475593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c157816aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe776ffd640 1 --2- 172.21.15.62:0/868475593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 0x55c1578125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe7777fe640 1 -- 172.21.15.62:0/868475593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 msgr2=0x55c1578125c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe7777fe640 1 --2- 172.21.15.62:0/868475593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 0x55c1578125c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe7777fe640 1 -- 172.21.15.62:0/868475593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 msgr2=0x55c157816aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe7777fe640 1 --2- 172.21.15.62:0/868475593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c157816aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe7777fe640 1 -- 172.21.15.62:0/868475593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c15780bf20 con 0x55c15781ad40 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe776ffd640 1 --2- 172.21.15.62:0/868475593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 0x55c1578125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:33.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe7777fe640 1 --2- 172.21.15.62:0/868475593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 0x55c157832330 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe768035000 tx=0x7fe768005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312c89a981523d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:33.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe76cff9640 1 -- 172.21.15.62:0/868475593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768004540 con 0x55c15781ad40 2024-09-12T00:00:33.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe76cff9640 1 -- 172.21.15.62:0/868475593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe768036050 con 0x55c15781ad40 2024-09-12T00:00:33.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.239+0000 7fe76cff9640 1 -- 172.21.15.62:0/868475593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76804a6c0 con 0x55c15781ad40 2024-09-12T00:00:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.240+0000 7fe77fbcb740 1 -- 172.21.15.62:0/868475593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 msgr2=0x55c157832330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.240+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/868475593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 0x55c157832330 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe768035000 tx=0x7fe768005ef0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.240+0000 7fe77fbcb740 1 -- 172.21.15.62:0/868475593 shutdown_connections 2024-09-12T00:00:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.240+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/868475593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15781ad40 0x55c157832330 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.240+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/868475593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c157816aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.240+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/868475593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15780fb60 0x55c1578125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.240+0000 7fe77fbcb740 1 -- 172.21.15.62:0/868475593 >> 172.21.15.62:0/868475593 conn(0x55c15780c0f0 msgr2=0x55c157816390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.240+0000 7fe77fbcb740 1 -- 172.21.15.62:0/868475593 shutdown_connections 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.240+0000 7fe77fbcb740 1 -- 172.21.15.62:0/868475593 wait complete. 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe77fbcb740 1 -- start start 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe77fbcb740 1 Processor -- start 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c15781ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c15781f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe77fbcb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 0x55c157894300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781abfa0 con 0x55c15780fb60 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781ad410 con 0x55c157810480 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe77fbcb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781ac670 con 0x55c15781ad40 2024-09-12T00:00:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe77fbcb740 10 client.0 _ll_register_callbacks cb 0x7fe7600975a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c15781ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c15781ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57468/0 (socket says 172.21.15.62:57468) 2024-09-12T00:00:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe776ffd640 1 -- 172.21.15.62:0/3771146434 learned_addr learned my addr 172.21.15.62:0/3771146434 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe7767fc640 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c15781f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe7777fe640 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 0x55c157894300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe776ffd640 1 -- 172.21.15.62:0/3771146434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 msgr2=0x55c157894300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe776ffd640 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 0x55c157894300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe776ffd640 1 -- 172.21.15.62:0/3771146434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 msgr2=0x55c15781f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe776ffd640 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c15781f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe776ffd640 1 -- 172.21.15.62:0/3771146434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c15781fc20 con 0x55c15780fb60 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.241+0000 7fe7777fe640 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 0x55c157894300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe776ffd640 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c15781ede0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe7700039c0 tx=0x7fe770004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b970bc344aecef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe75e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe75e7fc640 1 -- 172.21.15.62:0/3771146434 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7700119f0 con 0x55c15780fb60 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe75e7fc640 1 -- 172.21.15.62:0/3771146434 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fe770011b90 con 0x55c15780fb60 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3771146434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe768036050 con 0x55c15780fb60 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe75e7fc640 1 -- 172.21.15.62:0/3771146434 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe770042050 con 0x55c15780fb60 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe77fbcb740 10 client.16932 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3771146434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c1578959b0 con 0x55c15780fb60 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe75e7fc640 1 -- 172.21.15.62:0/3771146434 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe768036050 con 0x55c15780fb60 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe75e7fc640 1 -- 172.21.15.62:0/3771146434 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe770022040 con 0x55c15780fb60 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe77fbcb740 10 client.16932 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.242+0000 7fe77fbcb740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.243+0000 7fe77fbcb740 20 client.16932 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.243+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c157896060 0x55c157898450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.243+0000 7fe77fbcb740 4 client.16932 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.243+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3771146434 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c15777cd00 con 0x55c157896060 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.243+0000 7fe7767fc640 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c157896060 0x55c157898450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.243+0000 7fe75e7fc640 10 client.16932 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.243+0000 7fe7767fc640 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c157896060 0x55c157898450 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e573df3a9ebd33e8 server_cookie=c2e6fd598a6d60c8 in_seq=0 out_seq=0 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe75e7fc640 1 -- 172.21.15.62:0/3771146434 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c15777cd00 con 0x55c157896060 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe75e7fc640 10 client.16932 handle_command_reply: tid=0 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 client.16932 shutdown 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3771146434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 msgr2=0x55c15781ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c15781ede0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe7700039c0 tx=0x7fe770004770 comp rx=0 tx=0).stop 2024-09-12T00:00:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3771146434 shutdown_connections 2024-09-12T00:00:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c157896060 0x55c157898450 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c15781ad40 0x55c157894300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c157810480 0x55c15781f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 --2- 172.21.15.62:0/3771146434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c15780fb60 0x55c15781ede0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3771146434 >> 172.21.15.62:0/3771146434 conn(0x55c15780c0f0 msgr2=0x55c15780b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe75e7fc640 0 client.16932 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3771146434 shutdown_connections 2024-09-12T00:00:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.244+0000 7fe77fbcb740 1 -- 172.21.15.62:0/3771146434 wait complete. 2024-09-12T00:00:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77fbcb740 20 client.16932 trim_cache size 0 max 16384 2024-09-12T00:00:33.249 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:00:33.250 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-12T00:00:33.250 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "32966ae3-b13c-4bcc-9d1c-7d179f8db01d", 2024-09-12T00:00:33.250 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-12T00:00:33.250 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:00:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3008626214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7640a2cf0 msgr2=0x7fe7640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7640a2cf0 0x7fe7640a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe768004ab0 tx=0x7fe76802a8e0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3008626214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7781036e0 msgr2=0x7fe778198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7781036e0 0x7fe778198a70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe76000d950 tx=0x7fe76000de10 comp rx=0 tx=0).stop 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3008626214 shutdown_connections 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe7640a2cf0 0x7fe7640a51b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7781082a0 0x7fe77819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7781036e0 0x7fe778198a70 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 --2- 172.21.15.62:0/3008626214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe778102d30 0x7fe778198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3008626214 >> 172.21.15.62:0/3008626214 conn(0x7fe7780fc7d0 msgr2=0x7fe778105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3008626214 shutdown_connections 2024-09-12T00:00:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:33.245+0000 7fe77d7f2640 1 -- 172.21.15.62:0/3008626214 wait complete. 2024-09-12T00:00:33.948 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:00:34.245 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:34.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.699+0000 7f6f52fea640 1 -- 172.21.15.62:0/3993254434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4c106f10 msgr2=0x7f6f4c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:34.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.699+0000 7f6f52fea640 1 --2- 172.21.15.62:0/3993254434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4c106f10 0x7f6f4c10ea90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009ca0 tx=0x7f6f4402a8c0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.699+0000 7f6f52fea640 1 -- 172.21.15.62:0/3993254434 shutdown_connections 2024-09-12T00:00:34.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.699+0000 7f6f52fea640 1 --2- 172.21.15.62:0/3993254434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4c106f10 0x7f6f4c10ea90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.699+0000 7f6f52fea640 1 --2- 172.21.15.62:0/3993254434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c102790 0x7f6f4c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.699+0000 7f6f52fea640 1 --2- 172.21.15.62:0/3993254434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4c101de0 0x7f6f4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.699+0000 7f6f52fea640 1 -- 172.21.15.62:0/3993254434 >> 172.21.15.62:0/3993254434 conn(0x7f6f4c0fd8c0 msgr2=0x7f6f4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.699+0000 7f6f52fea640 1 -- 172.21.15.62:0/3993254434 shutdown_connections 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.699+0000 7f6f52fea640 1 -- 172.21.15.62:0/3993254434 wait complete. 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f52fea640 1 Processor -- start 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f52fea640 1 -- start start 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f52fea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c101de0 0x7f6f4c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f52fea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4c102790 0x7f6f4c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f52fea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4c106f10 0x7f6f4c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f52fea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c111f70 con 0x7f6f4c102790 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f52fea640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c111df0 con 0x7f6f4c106f10 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f52fea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1120f0 con 0x7f6f4c101de0 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4c102790 0x7f6f4c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4c102790 0x7f6f4c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57496/0 (socket says 172.21.15.62:57496) 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f4bfff640 1 -- 172.21.15.62:0/256903250 learned_addr learned my addr 172.21.15.62:0/256903250 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.700+0000 7f6f51587640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4c106f10 0x7f6f4c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f50d86640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c101de0 0x7f6f4c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f4bfff640 1 -- 172.21.15.62:0/256903250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c101de0 msgr2=0x7f6f4c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c101de0 0x7f6f4c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f4bfff640 1 -- 172.21.15.62:0/256903250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4c106f10 msgr2=0x7f6f4c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4c106f10 0x7f6f4c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f4bfff640 1 -- 172.21.15.62:0/256903250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c19d300 con 0x7f6f4c102790 2024-09-12T00:00:34.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f50d86640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c101de0 0x7f6f4c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:00:34.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4c102790 0x7f6f4c198890 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800d950 tx=0x7f6f3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:34.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f49ffb640 1 -- 172.21.15.62:0/256903250 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38017030 con 0x7f6f4c102790 2024-09-12T00:00:34.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f49ffb640 1 -- 172.21.15.62:0/256903250 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f38004020 con 0x7f6f4c102790 2024-09-12T00:00:34.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f52fea640 1 -- 172.21.15.62:0/256903250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c19d5f0 con 0x7f6f4c102790 2024-09-12T00:00:34.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f49ffb640 1 -- 172.21.15.62:0/256903250 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f380047a0 con 0x7f6f4c102790 2024-09-12T00:00:34.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.701+0000 7f6f52fea640 1 -- 172.21.15.62:0/256903250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c1a4ed0 con 0x7f6f4c102790 2024-09-12T00:00:34.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f49ffb640 1 -- 172.21.15.62:0/256903250 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6f380041d0 con 0x7f6f4c102790 2024-09-12T00:00:34.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f553c3740 1 Processor -- start 2024-09-12T00:00:34.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f553c3740 1 -- start start 2024-09-12T00:00:34.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a99b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99bdc0 0x56130a9a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f553c3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56130a9012e0 con 0x56130a9a6680 2024-09-12T00:00:34.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f553c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56130a8a8ce0 con 0x56130a99b4a0 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f553c3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56130a8fd2a0 con 0x56130a99bdc0 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f49ffb640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f3c0a2d70 0x7f6f3c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f51587640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f3c0a2d70 0x7f6f3c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9a8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57512/0 (socket says 172.21.15.62:57512) 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f4bfff640 1 -- 172.21.15.62:0/3266337639 learned_addr learned my addr 172.21.15.62:0/3266337639 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.704+0000 7f6f50d86640 1 --2- 172.21.15.62:0/3266337639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99bdc0 0x56130a9a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f51587640 1 --2- 172.21.15.62:0/3266337639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a99b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f49ffb640 1 -- 172.21.15.62:0/256903250 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f6f380cedd0 con 0x7f6f4c102790 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f4bfff640 1 -- 172.21.15.62:0/3266337639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99bdc0 msgr2=0x56130a9a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:34.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/3266337639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99bdc0 0x56130a9a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f4bfff640 1 -- 172.21.15.62:0/3266337639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 msgr2=0x56130a99b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:34.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/3266337639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a99b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f4bfff640 1 -- 172.21.15.62:0/3266337639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130a88d440 con 0x56130a9a6680 2024-09-12T00:00:34.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/3266337639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9a8a70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6f3806a1f0 tx=0x7f6f3806e360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995a1bf2705b9471 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:34.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f51587640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f3c0a2d70 0x7f6f3c0a5230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f4402f540 tx=0x7f6f4402f410 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:34.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3266337639 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38095e60 con 0x56130a9a6680 2024-09-12T00:00:34.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3266337639 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3809b9f0 con 0x56130a9a6680 2024-09-12T00:00:34.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.705+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3266337639 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38097960 con 0x56130a9a6680 2024-09-12T00:00:34.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 -- 172.21.15.62:0/3266337639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 msgr2=0x56130a9a8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 --2- 172.21.15.62:0/3266337639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9a8a70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6f3806a1f0 tx=0x7f6f3806e360 comp rx=0 tx=0).stop 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 -- 172.21.15.62:0/3266337639 shutdown_connections 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 --2- 172.21.15.62:0/3266337639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9a8a70 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 --2- 172.21.15.62:0/3266337639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99bdc0 0x56130a9a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 --2- 172.21.15.62:0/3266337639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a99b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 -- 172.21.15.62:0/3266337639 >> 172.21.15.62:0/3266337639 conn(0x56130a997a30 msgr2=0x56130a997e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 -- 172.21.15.62:0/3266337639 shutdown_connections 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 -- 172.21.15.62:0/3266337639 wait complete. 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 -- start start 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.706+0000 7f6f553c3740 1 Processor -- start 2024-09-12T00:00:34.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99b4a0 0x56130a9a6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a9a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a9a6680 0x56130a9a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f553c3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56130a9a9250 con 0x56130a99bdc0 2024-09-12T00:00:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f553c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56130a8bc050 con 0x56130a9a6680 2024-09-12T00:00:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f553c3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56130a9a93d0 con 0x56130a99b4a0 2024-09-12T00:00:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f553c3740 10 client.0 _ll_register_callbacks cb 0x56130a8fc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a9a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a9a32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57514/0 (socket says 172.21.15.62:57514) 2024-09-12T00:00:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f4bfff640 1 -- 172.21.15.62:0/405474677 learned_addr learned my addr 172.21.15.62:0/405474677 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f50d86640 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99b4a0 0x56130a9a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f51587640 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a9a6680 0x56130a9a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f4bfff640 1 -- 172.21.15.62:0/405474677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99b4a0 msgr2=0x56130a9a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99b4a0 0x56130a9a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f4bfff640 1 -- 172.21.15.62:0/405474677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a9a6680 msgr2=0x56130a9a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a9a6680 0x56130a9a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f4bfff640 1 -- 172.21.15.62:0/405474677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130a6723c0 con 0x56130a99bdc0 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f50d86640 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99b4a0 0x56130a9a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f51587640 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a9a6680 0x56130a9a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a9a32e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6f38097cc0 tx=0x7f6f380cc9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8d40a0fee8d884 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.707+0000 7f6f2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f2bfff640 1 -- 172.21.15.62:0/405474677 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38014070 con 0x56130a99bdc0 2024-09-12T00:00:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f2bfff640 1 -- 172.21.15.62:0/405474677 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f6f38099e20 con 0x56130a99bdc0 2024-09-12T00:00:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f553c3740 1 -- 172.21.15.62:0/405474677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f3809b9f0 con 0x56130a99bdc0 2024-09-12T00:00:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f2bfff640 1 -- 172.21.15.62:0/405474677 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6f38095e60 con 0x56130a99bdc0 2024-09-12T00:00:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f553c3740 10 client.16944 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f553c3740 1 -- 172.21.15.62:0/405474677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56130a9b4c60 con 0x56130a99bdc0 2024-09-12T00:00:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f2bfff640 1 -- 172.21.15.62:0/405474677 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3809b9f0 con 0x56130a99bdc0 2024-09-12T00:00:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f2bfff640 1 -- 172.21.15.62:0/405474677 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6f380b40f0 con 0x56130a99bdc0 2024-09-12T00:00:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f553c3740 10 client.16944 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f553c3740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f553c3740 20 client.16944 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f553c3740 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56130a9b5fb0 0x56130a9b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f553c3740 4 client.16944 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f553c3740 1 -- 172.21.15.62:0/405474677 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56130a90e850 con 0x56130a9b5fb0 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.708+0000 7f6f50d86640 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56130a9b5fb0 0x56130a9b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.709+0000 7f6f50d86640 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56130a9b5fb0 0x56130a9b83a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4780b7760917ba0d server_cookie=89daae8d408af4b4 in_seq=0 out_seq=0 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.709+0000 7f6f2bfff640 10 client.16944 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.709+0000 7f6f2bfff640 1 -- 172.21.15.62:0/405474677 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56130a90e850 con 0x56130a9b5fb0 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.709+0000 7f6f2bfff640 10 client.16944 handle_command_reply: tid=0 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 client.16944 shutdown 2024-09-12T00:00:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 -- 172.21.15.62:0/405474677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 msgr2=0x56130a9a32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a9a32e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6f38097cc0 tx=0x7f6f380cc9b0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 -- 172.21.15.62:0/405474677 shutdown_connections 2024-09-12T00:00:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56130a9b5fb0 0x56130a9b83a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a9a6680 0x56130a9a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a9a32e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 --2- 172.21.15.62:0/405474677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99b4a0 0x56130a9a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f2bfff640 0 client.16944 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 -- 172.21.15.62:0/405474677 >> 172.21.15.62:0/405474677 conn(0x56130a997a30 msgr2=0x56130a99d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 -- 172.21.15.62:0/405474677 shutdown_connections 2024-09-12T00:00:34.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 1 -- 172.21.15.62:0/405474677 wait complete. 2024-09-12T00:00:34.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.710+0000 7f6f553c3740 20 client.16944 trim_cache size 0 max 16384 2024-09-12T00:00:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.727+0000 7f6f553c3740 1 Processor -- start 2024-09-12T00:00:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.727+0000 7f6f553c3740 1 -- start start 2024-09-12T00:00:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.727+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a9a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.727+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a996a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.727+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a9a6680 0x56130a9a0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.727+0000 7f6f553c3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1ade80 con 0x56130a99bdc0 2024-09-12T00:00:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.727+0000 7f6f553c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1ab750 con 0x56130a99b4a0 2024-09-12T00:00:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.727+0000 7f6f553c3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a95e0 con 0x56130a9a6680 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a996a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a996a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57516/0 (socket says 172.21.15.62:57516) 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f4bfff640 1 -- 172.21.15.62:0/3055611745 learned_addr learned my addr 172.21.15.62:0/3055611745 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f50d86640 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a9a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f4bfff640 1 -- 172.21.15.62:0/3055611745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a9a6680 msgr2=0x56130a9a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f51587640 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a9a6680 0x56130a9a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a9a6680 0x56130a9a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f4bfff640 1 -- 172.21.15.62:0/3055611745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 msgr2=0x56130a9a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a9a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f4bfff640 1 -- 172.21.15.62:0/3055611745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130a99ac90 con 0x56130a99bdc0 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f50d86640 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a9a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f51587640 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a9a6680 0x56130a9a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:00:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a996a60 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6f380045f0 tx=0x7f6f380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10483eac2cf47412 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f367fc640 1 -- 172.21.15.62:0/3055611745 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38014070 con 0x56130a99bdc0 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f367fc640 1 -- 172.21.15.62:0/3055611745 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3809b9f0 con 0x56130a99bdc0 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.728+0000 7f6f367fc640 1 -- 172.21.15.62:0/3055611745 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f380998b0 con 0x56130a99bdc0 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 -- 172.21.15.62:0/3055611745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 msgr2=0x56130a996a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a996a60 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6f380045f0 tx=0x7f6f380046c0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 -- 172.21.15.62:0/3055611745 shutdown_connections 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a9a6680 0x56130a9a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a99bdc0 0x56130a996a60 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 --2- 172.21.15.62:0/3055611745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a9a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 -- 172.21.15.62:0/3055611745 >> 172.21.15.62:0/3055611745 conn(0x56130a997a30 msgr2=0x56130a996350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 -- 172.21.15.62:0/3055611745 shutdown_connections 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 -- 172.21.15.62:0/3055611745 wait complete. 2024-09-12T00:00:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 -- start start 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.729+0000 7f6f553c3740 1 Processor -- start 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a9b9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99bdc0 0x56130a9b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f553c3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f553c3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a9c30 con 0x56130a9a6680 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f553c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1ab080 con 0x56130a99b4a0 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f553c3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1aa2e0 con 0x56130a99bdc0 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f553c3740 10 client.0 _ll_register_callbacks cb 0x7f6f44009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f51587640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f51587640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9b9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57532/0 (socket says 172.21.15.62:57532) 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f51587640 1 -- 172.21.15.62:0/83344129 learned_addr learned my addr 172.21.15.62:0/83344129 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f4bfff640 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99bdc0 0x56130a9b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f50d86640 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a9b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f51587640 1 -- 172.21.15.62:0/83344129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99bdc0 msgr2=0x56130a9b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f51587640 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99bdc0 0x56130a9b9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f51587640 1 -- 172.21.15.62:0/83344129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 msgr2=0x56130a9b9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f51587640 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a9b9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f51587640 1 -- 172.21.15.62:0/83344129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130aa1fbc0 con 0x56130a9a6680 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.730+0000 7f6f50d86640 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a9b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f51587640 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9b9ec0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009ca0 tx=0x7f6f4404a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b008c20f6138bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f2bfff640 1 -- 172.21.15.62:0/83344129 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44003a30 con 0x56130a9a6680 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f2bfff640 1 -- 172.21.15.62:0/83344129 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f6f44003bd0 con 0x56130a9a6680 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f553c3740 1 -- 172.21.15.62:0/83344129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f3809b9f0 con 0x56130a9a6680 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f2bfff640 1 -- 172.21.15.62:0/83344129 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6f44003dd0 con 0x56130a9a6680 2024-09-12T00:00:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f553c3740 10 client.16950 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f553c3740 1 -- 172.21.15.62:0/83344129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56130aa20e40 con 0x56130a9a6680 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f2bfff640 1 -- 172.21.15.62:0/83344129 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3809b9f0 con 0x56130a9a6680 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f2bfff640 1 -- 172.21.15.62:0/83344129 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6f4404a070 con 0x56130a9a6680 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f553c3740 10 client.16950 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f553c3740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f553c3740 20 client.16950 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f553c3740 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56130aa21570 0x56130aa23990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f553c3740 4 client.16950 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.731+0000 7f6f553c3740 1 -- 172.21.15.62:0/83344129 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56130a908d60 con 0x56130aa21570 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.732+0000 7f6f50d86640 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56130aa21570 0x56130aa23990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.732+0000 7f6f2bfff640 10 client.16950 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.732+0000 7f6f50d86640 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56130aa21570 0x56130aa23990 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42f48e147f93bb3 server_cookie=691c623368767173 in_seq=0 out_seq=0 2024-09-12T00:00:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f2bfff640 1 -- 172.21.15.62:0/83344129 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56130a908d60 con 0x56130aa21570 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f2bfff640 10 client.16950 handle_command_reply: tid=0 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 client.16950 shutdown 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 -- 172.21.15.62:0/83344129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 msgr2=0x56130a9b9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9b9ec0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009ca0 tx=0x7f6f4404a000 comp rx=0 tx=0).stop 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 -- 172.21.15.62:0/83344129 shutdown_connections 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56130aa21570 0x56130aa23990 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56130a9a6680 0x56130a9b9ec0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56130a99bdc0 0x56130a9b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 --2- 172.21.15.62:0/83344129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130a99b4a0 0x56130a9b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f2bfff640 0 client.16950 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 -- 172.21.15.62:0/83344129 >> 172.21.15.62:0/83344129 conn(0x56130a997a30 msgr2=0x56130a99d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 -- 172.21.15.62:0/83344129 shutdown_connections 2024-09-12T00:00:34.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 1 -- 172.21.15.62:0/83344129 wait complete. 2024-09-12T00:00:34.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.733+0000 7f6f553c3740 20 client.16950 trim_cache size 0 max 16384 2024-09-12T00:00:34.748 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:00:34.748 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:00:34.748 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-12T00:00:34.748 INFO:teuthology.orchestra.run.smithi062.stdout: "32966ae3-b13c-4bcc-9d1c-7d179f8db01d": { 2024-09-12T00:00:34.748 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-12T00:00:34.748 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "32966ae3-b13c-4bcc-9d1c-7d179f8db01d", 2024-09-12T00:00:34.748 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-12T00:00:34.749 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:34.749 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:34.749 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:00:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 -- 172.21.15.62:0/256903250 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f3c0a2d70 msgr2=0x7f6f3c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f3c0a2d70 0x7f6f3c0a5230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f4402f540 tx=0x7f6f4402f410 comp rx=0 tx=0).stop 2024-09-12T00:00:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 -- 172.21.15.62:0/256903250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4c102790 msgr2=0x7f6f4c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4c102790 0x7f6f4c198890 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800d950 tx=0x7f6f3800de10 comp rx=0 tx=0).stop 2024-09-12T00:00:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 -- 172.21.15.62:0/256903250 shutdown_connections 2024-09-12T00:00:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f3c0a2d70 0x7f6f3c0a5230 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4c106f10 0x7f6f4c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4c102790 0x7f6f4c198890 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 --2- 172.21.15.62:0/256903250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f4c101de0 0x7f6f4c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 -- 172.21.15.62:0/256903250 >> 172.21.15.62:0/256903250 conn(0x7f6f4c0fd8c0 msgr2=0x7f6f4c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 -- 172.21.15.62:0/256903250 shutdown_connections 2024-09-12T00:00:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:34.734+0000 7f6f52fea640 1 -- 172.21.15.62:0/256903250 wait complete. 2024-09-12T00:00:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:34 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-12T00:00:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:34 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:34 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:00:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:34 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:00:34.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:34 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:34.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:34 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:00:34.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:34 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:34.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:34 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:34.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:34 smithi083 ceph-mon[34805]: pgmap v261: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 1.17k op/s 2024-09-12T00:00:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:34 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-12T00:00:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:34 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:34 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:00:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:34 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:00:34.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:34 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:34.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:34 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:00:34.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:34 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:34.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:34 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:34.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:34 smithi062 ceph-mon[30958]: pgmap v261: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 1.17k op/s 2024-09-12T00:00:35.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:34 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-12T00:00:35.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:34 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:35.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:34 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:00:35.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:34 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:00:35.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:34 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:35.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:34 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:00:35.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:34 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:35.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:34 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:00:35.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:34 smithi120 ceph-mon[35182]: pgmap v261: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 1.17k op/s 2024-09-12T00:00:35.379 INFO:tasks.cephfs.filesystem:scrub status for tag:32966ae3-b13c-4bcc-9d1c-7d179f8db01d - {'path': '/', 'tag': '32966ae3-b13c-4bcc-9d1c-7d179f8db01d', 'options': 'recursive,force'} 2024-09-12T00:00:36.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:36 smithi083 ceph-mon[34805]: pgmap v262: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 157 KiB/s wr, 18 op/s 2024-09-12T00:00:36.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:36 smithi062 ceph-mon[30958]: pgmap v262: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 157 KiB/s wr, 18 op/s 2024-09-12T00:00:37.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:36 smithi120 ceph-mon[35182]: pgmap v262: 97 pgs: 97 active+clean; 3.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 157 KiB/s wr, 18 op/s 2024-09-12T00:00:37.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 475.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:00:38.018 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:00:38.289 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:38.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:38 smithi062 ceph-mon[30958]: pgmap v263: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-09-12T00:00:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.765+0000 7fb3121be640 1 -- 172.21.15.62:0/3332274455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb30c102790 msgr2=0x7fb30c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:38.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.765+0000 7fb3121be640 1 --2- 172.21.15.62:0/3332274455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb30c102790 0x7fb30c1067e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009900 tx=0x7fb2f402a410 comp rx=0 tx=0).stop 2024-09-12T00:00:38.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.765+0000 7fb3121be640 1 -- 172.21.15.62:0/3332274455 shutdown_connections 2024-09-12T00:00:38.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.765+0000 7fb3121be640 1 --2- 172.21.15.62:0/3332274455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb30c106f10 0x7fb30c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:38.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.765+0000 7fb3121be640 1 --2- 172.21.15.62:0/3332274455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb30c102790 0x7fb30c1067e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:38.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.765+0000 7fb3121be640 1 --2- 172.21.15.62:0/3332274455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c101de0 0x7fb30c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:38.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.765+0000 7fb3121be640 1 -- 172.21.15.62:0/3332274455 >> 172.21.15.62:0/3332274455 conn(0x7fb30c0fd8c0 msgr2=0x7fb30c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:38.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.765+0000 7fb3121be640 1 -- 172.21.15.62:0/3332274455 shutdown_connections 2024-09-12T00:00:38.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.765+0000 7fb3121be640 1 -- 172.21.15.62:0/3332274455 wait complete. 2024-09-12T00:00:38.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb3121be640 1 Processor -- start 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb3121be640 1 -- start start 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb3121be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb30c101de0 0x7fb30c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb3121be640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb30c102790 0x7fb30c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb3121be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c106f10 0x7fb30c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb3121be640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c111f70 con 0x7fb30c102790 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb3121be640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c111df0 con 0x7fb30c106f10 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb3121be640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1120f0 con 0x7fb30c101de0 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb30affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb30c102790 0x7fb30c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb30affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb30c102790 0x7fb30c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57552/0 (socket says 172.21.15.62:57552) 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb30affd640 1 -- 172.21.15.62:0/1982807090 learned_addr learned my addr 172.21.15.62:0/1982807090 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb30b7fe640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb30c101de0 0x7fb30c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.766+0000 7fb30bfff640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c106f10 0x7fb30c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:38.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb30affd640 1 -- 172.21.15.62:0/1982807090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb30c101de0 msgr2=0x7fb30c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:38.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb30affd640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb30c101de0 0x7fb30c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:38.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb30affd640 1 -- 172.21.15.62:0/1982807090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c106f10 msgr2=0x7fb30c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:38.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb30affd640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c106f10 0x7fb30c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:38.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb30affd640 1 -- 172.21.15.62:0/1982807090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c19d440 con 0x7fb30c102790 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb30affd640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb30c102790 0x7fb30c1989d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4006010 tx=0x7fb2f4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb308ff9640 1 -- 172.21.15.62:0/1982807090 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f403f030 con 0x7fb30c102790 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb308ff9640 1 -- 172.21.15.62:0/1982807090 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2f4004440 con 0x7fb30c102790 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb3121be640 1 -- 172.21.15.62:0/1982807090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c19d6d0 con 0x7fb30c102790 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb308ff9640 1 -- 172.21.15.62:0/1982807090 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4035070 con 0x7fb30c102790 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.767+0000 7fb3121be640 1 -- 172.21.15.62:0/1982807090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c1a5010 con 0x7fb30c102790 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.768+0000 7fb3121be640 1 -- 172.21.15.62:0/1982807090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30c1032c0 con 0x7fb30c102790 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.769+0000 7fb308ff9640 1 -- 172.21.15.62:0/1982807090 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb2f4042030 con 0x7fb30c102790 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.770+0000 7fb308ff9640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb2f80a2cf0 0x7fb2f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.770+0000 7fb30b7fe640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb2f80a2cf0 0x7fb2f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.770+0000 7fb308ff9640 1 -- 172.21.15.62:0/1982807090 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7fb2f40ef860 con 0x7fb30c102790 2024-09-12T00:00:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.771+0000 7fb30b7fe640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb2f80a2cf0 0x7fb2f80a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009bd0 tx=0x7fb2fc005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.774+0000 7fb308ff9640 1 -- 172.21.15.62:0/1982807090 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2f40bc480 con 0x7fb30c102790 2024-09-12T00:00:38.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:38 smithi083 ceph-mon[34805]: pgmap v263: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-09-12T00:00:38.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.987+0000 7fb3121be640 1 -- 172.21.15.62:0/1982807090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb30c10a0f0 con 0x7fb30c102790 2024-09-12T00:00:38.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.987+0000 7fb308ff9640 1 -- 172.21.15.62:0/1982807090 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11909 (secure 0 0 0) 0x7fb2f40bbd80 con 0x7fb30c102790 2024-09-12T00:00:38.987 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:00:38.988 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"btime":"2024-09-12T00:00:02:308078+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:00:02.079738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:00:38.988 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 -- 172.21.15.62:0/1982807090 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb2f80a2cf0 msgr2=0x7fb2f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb2f80a2cf0 0x7fb2f80a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009bd0 tx=0x7fb2fc005f90 comp rx=0 tx=0).stop 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 -- 172.21.15.62:0/1982807090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb30c102790 msgr2=0x7fb30c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb30c102790 0x7fb30c1989d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4006010 tx=0x7fb2f4004040 comp rx=0 tx=0).stop 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 -- 172.21.15.62:0/1982807090 shutdown_connections 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fb2f80a2cf0 0x7fb2f80a51b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c106f10 0x7fb30c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb30c102790 0x7fb30c1989d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 --2- 172.21.15.62:0/1982807090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb30c101de0 0x7fb30c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 -- 172.21.15.62:0/1982807090 >> 172.21.15.62:0/1982807090 conn(0x7fb30c0fd8c0 msgr2=0x7fb30c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 -- 172.21.15.62:0/1982807090 shutdown_connections 2024-09-12T00:00:38.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:38.990+0000 7fb3121be640 1 -- 172.21.15.62:0/1982807090 wait complete. 2024-09-12T00:00:39.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:38 smithi120 ceph-mon[35182]: pgmap v263: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-09-12T00:00:39.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:39 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1982807090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:39.717 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id ccd456ca --release --await' 2024-09-12T00:00:39.718 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id ccd456ca --release --await 2024-09-12T00:00:39.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:39 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1982807090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:39.976 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:00:40.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:39 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1982807090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:00:40.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.447+0000 7f7495007640 1 -- 172.21.15.62:0/942080795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490100e20 msgr2=0x7f7490101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.447+0000 7f7495007640 1 --2- 172.21.15.62:0/942080795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490100e20 0x7f7490101200 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7480009900 tx=0x7f748002a480 comp rx=0 tx=0).stop 2024-09-12T00:00:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.447+0000 7f7495007640 1 -- 172.21.15.62:0/942080795 shutdown_connections 2024-09-12T00:00:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.447+0000 7f7495007640 1 --2- 172.21.15.62:0/942080795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490102ef0 0x7f749010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.447+0000 7f7495007640 1 --2- 172.21.15.62:0/942080795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7490102570 0x7f74901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.447+0000 7f7495007640 1 --2- 172.21.15.62:0/942080795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490100e20 0x7f7490101200 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.447+0000 7f7495007640 1 -- 172.21.15.62:0/942080795 >> 172.21.15.62:0/942080795 conn(0x7f74900fc7d0 msgr2=0x7f74900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.447+0000 7f7495007640 1 -- 172.21.15.62:0/942080795 shutdown_connections 2024-09-12T00:00:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.447+0000 7f7495007640 1 -- 172.21.15.62:0/942080795 wait complete. 2024-09-12T00:00:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.448+0000 7f7495007640 1 Processor -- start 2024-09-12T00:00:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.448+0000 7f7495007640 1 -- start start 2024-09-12T00:00:40.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.448+0000 7f7495007640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7490100e20 0x7f74901992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.448+0000 7f7495007640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490102570 0x7f7490199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.448+0000 7f7495007640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490102ef0 0x7f74901a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.448+0000 7f7495007640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7490110f50 con 0x7f7490102ef0 2024-09-12T00:00:40.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.448+0000 7f7495007640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7490110dd0 con 0x7f7490102570 2024-09-12T00:00:40.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.448+0000 7f7495007640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f74901110d0 con 0x7f7490100e20 2024-09-12T00:00:40.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.448+0000 7f748f59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490102ef0 0x7f74901a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748f59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490102ef0 0x7f74901a4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57566/0 (socket says 172.21.15.62:57566) 2024-09-12T00:00:40.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748f59e640 1 -- 172.21.15.62:0/1850489754 learned_addr learned my addr 172.21.15.62:0/1850489754 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:40.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748ed9d640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7490100e20 0x7f74901992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748e59c640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490102570 0x7f7490199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748f59e640 1 -- 172.21.15.62:0/1850489754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7490100e20 msgr2=0x7f74901992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:40.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748f59e640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7490100e20 0x7f74901992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748f59e640 1 -- 172.21.15.62:0/1850489754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490102570 msgr2=0x7f7490199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:40.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748f59e640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490102570 0x7f7490199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748f59e640 1 -- 172.21.15.62:0/1850489754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74901a48a0 con 0x7f7490102ef0 2024-09-12T00:00:40.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748ed9d640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7490100e20 0x7f74901992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:40.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f748f59e640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490102ef0 0x7f74901a4360 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f748800cad0 tx=0x7f748800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:40.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f7477fff640 1 -- 172.21.15.62:0/1850489754 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748800ae20 con 0x7f7490102ef0 2024-09-12T00:00:40.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f7495007640 1 -- 172.21.15.62:0/1850489754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74901a4a70 con 0x7f7490102ef0 2024-09-12T00:00:40.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f7477fff640 1 -- 172.21.15.62:0/1850489754 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7488005020 con 0x7f7490102ef0 2024-09-12T00:00:40.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f7477fff640 1 -- 172.21.15.62:0/1850489754 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748800f6f0 con 0x7f7490102ef0 2024-09-12T00:00:40.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.449+0000 7f7495007640 1 -- 172.21.15.62:0/1850489754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901a4f80 con 0x7f7490102ef0 2024-09-12T00:00:40.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f7477fff640 1 -- 172.21.15.62:0/1850489754 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f74880051d0 con 0x7f7490102ef0 2024-09-12T00:00:40.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f74973e0740 1 Processor -- start 2024-09-12T00:00:40.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f74973e0740 1 -- start start 2024-09-12T00:00:40.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f74973e0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 0x55811824e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f74973e0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e980 0x558118258d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f74973e0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558118259240 0x55811825b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f74973e0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5581181b3620 con 0x55811824e980 2024-09-12T00:00:40.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f74973e0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55811815ae50 con 0x558118259240 2024-09-12T00:00:40.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f74973e0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5581181af5e0 con 0x55811824e060 2024-09-12T00:00:40.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f7477fff640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74780a2cf0 0x7f74780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f748ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e980 0x558118258d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f748ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e980 0x558118258d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57568/0 (socket says 172.21.15.62:57568) 2024-09-12T00:00:40.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f748ed9d640 1 -- 172.21.15.62:0/1880670018 learned_addr learned my addr 172.21.15.62:0/1880670018 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:40.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.452+0000 7f748e59c640 1 --2- 172.21.15.62:0/1880670018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 0x55811824e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f748f59e640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74780a2cf0 0x7f74780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f748e59c640 1 --2- 172.21.15.62:0/1880670018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558118259240 0x55811825b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f748ed9d640 1 -- 172.21.15.62:0/1880670018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 msgr2=0x55811824e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f748ed9d640 1 --2- 172.21.15.62:0/1880670018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 0x55811824e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f748ed9d640 1 -- 172.21.15.62:0/1880670018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558118259240 msgr2=0x55811825b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:40.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f748ed9d640 1 --2- 172.21.15.62:0/1880670018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558118259240 0x55811825b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f748ed9d640 1 -- 172.21.15.62:0/1880670018 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55811814bd40 con 0x55811824e980 2024-09-12T00:00:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f7477fff640 1 -- 172.21.15.62:0/1850489754 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f74880cf4f0 con 0x7f7490102ef0 2024-09-12T00:00:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f748e59c640 1 --2- 172.21.15.62:0/1880670018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 0x55811824e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f748ed9d640 1 --2- 172.21.15.62:0/1880670018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e980 0x558118258d00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f74800314e0 tx=0x7f7480004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c218c85cc9d9c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f748f59e640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74780a2cf0 0x7f74780a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f74880cccf0 tx=0x7f74880976b0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f74757fa640 1 -- 172.21.15.62:0/1880670018 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480035030 con 0x55811824e980 2024-09-12T00:00:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f74757fa640 1 -- 172.21.15.62:0/1880670018 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7480003ad0 con 0x55811824e980 2024-09-12T00:00:40.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.453+0000 7f74757fa640 1 -- 172.21.15.62:0/1880670018 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480031c70 con 0x55811824e980 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 -- 172.21.15.62:0/1880670018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e980 msgr2=0x558118258d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 --2- 172.21.15.62:0/1880670018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e980 0x558118258d00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f74800314e0 tx=0x7f7480004200 comp rx=0 tx=0).stop 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 -- 172.21.15.62:0/1880670018 shutdown_connections 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 --2- 172.21.15.62:0/1880670018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558118259240 0x55811825b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 --2- 172.21.15.62:0/1880670018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e980 0x558118258d00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 --2- 172.21.15.62:0/1880670018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 0x55811824e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 -- 172.21.15.62:0/1880670018 >> 172.21.15.62:0/1880670018 conn(0x55811824a5f0 msgr2=0x55811824a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 -- 172.21.15.62:0/1880670018 shutdown_connections 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 -- 172.21.15.62:0/1880670018 wait complete. 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 -- start start 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.454+0000 7f74973e0740 1 Processor -- start 2024-09-12T00:00:40.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f74973e0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e060 0x5581182560e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f74973e0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e980 0x558118256620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f74973e0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558118259240 0x558118256b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f74973e0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55811825be10 con 0x55811824e060 2024-09-12T00:00:40.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f74973e0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55811813bf10 con 0x558118259240 2024-09-12T00:00:40.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f74973e0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55811825bf90 con 0x55811824e980 2024-09-12T00:00:40.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f74973e0740 10 client.0 _ll_register_callbacks cb 0x55811815fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:40.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f748e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e060 0x5581182560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f748e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e060 0x5581182560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57570/0 (socket says 172.21.15.62:57570) 2024-09-12T00:00:40.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f748e59c640 1 -- 172.21.15.62:0/724508836 learned_addr learned my addr 172.21.15.62:0/724508836 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:40.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f748ed9d640 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e980 0x558118256620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f748e59c640 1 -- 172.21.15.62:0/724508836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e980 msgr2=0x558118256620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f748e59c640 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e980 0x558118256620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f748e59c640 1 -- 172.21.15.62:0/724508836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558118259240 msgr2=0x558118256b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f748e59c640 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558118259240 0x558118256b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f748e59c640 1 -- 172.21.15.62:0/724508836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558117ec2580 con 0x55811824e060 2024-09-12T00:00:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f748e59c640 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e060 0x5581182560e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7484013c20 tx=0x7f7484010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3084a9b369309d74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.455+0000 7f7472ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f7472ff5640 1 -- 172.21.15.62:0/724508836 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748400f5d0 con 0x55811824e060 2024-09-12T00:00:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f7472ff5640 1 -- 172.21.15.62:0/724508836 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f748400f770 con 0x55811824e060 2024-09-12T00:00:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f74973e0740 1 -- 172.21.15.62:0/724508836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7480003ad0 con 0x55811824e060 2024-09-12T00:00:40.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f7472ff5640 1 -- 172.21.15.62:0/724508836 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f748400f970 con 0x55811824e060 2024-09-12T00:00:40.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f74973e0740 10 client.16968 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:40.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f74973e0740 1 -- 172.21.15.62:0/724508836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558117f8c960 con 0x55811824e060 2024-09-12T00:00:40.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f7472ff5640 1 -- 172.21.15.62:0/724508836 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7480003ad0 con 0x55811824e060 2024-09-12T00:00:40.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f7472ff5640 1 -- 172.21.15.62:0/724508836 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7484025b60 con 0x55811824e060 2024-09-12T00:00:40.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f74973e0740 10 client.16968 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:40.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f74973e0740 10 client.16968 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:00:40.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f74973e0740 20 client.16968 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:40.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f74973e0740 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558118268d40 0x55811826b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f74973e0740 4 client.16968 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:00:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f74973e0740 1 -- 172.21.15.62:0/724508836 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581181c0b90 con 0x558118268d40 2024-09-12T00:00:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.456+0000 7f748ed9d640 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558118268d40 0x55811826b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.457+0000 7f7472ff5640 10 client.16968 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.457+0000 7f748ed9d640 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558118268d40 0x55811826b170 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=174ab325f5449487 server_cookie=ef11a5eca305c339 in_seq=0 out_seq=0 2024-09-12T00:00:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f7472ff5640 1 -- 172.21.15.62:0/724508836 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581181c0b90 con 0x558118268d40 2024-09-12T00:00:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f7472ff5640 10 client.16968 handle_command_reply: tid=0 2024-09-12T00:00:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 client.16968 shutdown 2024-09-12T00:00:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 -- 172.21.15.62:0/724508836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e060 msgr2=0x5581182560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e060 0x5581182560e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7484013c20 tx=0x7f7484010800 comp rx=0 tx=0).stop 2024-09-12T00:00:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 -- 172.21.15.62:0/724508836 shutdown_connections 2024-09-12T00:00:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x558118268d40 0x55811826b170 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558118259240 0x558118256b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e980 0x558118256620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 --2- 172.21.15.62:0/724508836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55811824e060 0x5581182560e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 -- 172.21.15.62:0/724508836 >> 172.21.15.62:0/724508836 conn(0x55811824a5f0 msgr2=0x5581182502d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f7472ff5640 0 client.16968 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 -- 172.21.15.62:0/724508836 shutdown_connections 2024-09-12T00:00:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 1 -- 172.21.15.62:0/724508836 wait complete. 2024-09-12T00:00:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.458+0000 7f74973e0740 20 client.16968 trim_cache size 0 max 16384 2024-09-12T00:00:40.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f74973e0740 1 Processor -- start 2024-09-12T00:00:40.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f74973e0740 1 -- start start 2024-09-12T00:00:40.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f74973e0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 0x558118250ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f74973e0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e980 0x5581182615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f74973e0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x558118261b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f74973e0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74901b0060 con 0x558118259240 2024-09-12T00:00:40.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f74973e0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ad930 con 0x55811824e980 2024-09-12T00:00:40.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f74973e0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ab7a0 con 0x55811824e060 2024-09-12T00:00:40.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f748e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x558118261b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f748e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x558118261b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57572/0 (socket says 172.21.15.62:57572) 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.475+0000 7f748e59c640 1 -- 172.21.15.62:0/2333411618 learned_addr learned my addr 172.21.15.62:0/2333411618 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f748ed9d640 1 --2- 172.21.15.62:0/2333411618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e980 0x5581182615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f748e59c640 1 --2- 172.21.15.62:0/2333411618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 0x558118250ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f748e59c640 1 -- 172.21.15.62:0/2333411618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 msgr2=0x558118250ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f748e59c640 1 --2- 172.21.15.62:0/2333411618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 0x558118250ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f748e59c640 1 -- 172.21.15.62:0/2333411618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e980 msgr2=0x5581182615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f748e59c640 1 --2- 172.21.15.62:0/2333411618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e980 0x5581182615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f748e59c640 1 -- 172.21.15.62:0/2333411618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55811824b1a0 con 0x558118259240 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f748ed9d640 1 --2- 172.21.15.62:0/2333411618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e980 0x5581182615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:00:40.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f748e59c640 1 --2- 172.21.15.62:0/2333411618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x558118261b30 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7484025f00 tx=0x7f7484013910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2302e8481bb4a406 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:40.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f7474ff9640 1 -- 172.21.15.62:0/2333411618 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748402b040 con 0x558118259240 2024-09-12T00:00:40.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f7474ff9640 1 -- 172.21.15.62:0/2333411618 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f748402c040 con 0x558118259240 2024-09-12T00:00:40.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.476+0000 7f7474ff9640 1 -- 172.21.15.62:0/2333411618 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7484011c00 con 0x558118259240 2024-09-12T00:00:40.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 -- 172.21.15.62:0/2333411618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 msgr2=0x558118261b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:40.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 --2- 172.21.15.62:0/2333411618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x558118261b30 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7484025f00 tx=0x7f7484013910 comp rx=0 tx=0).stop 2024-09-12T00:00:40.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 -- 172.21.15.62:0/2333411618 shutdown_connections 2024-09-12T00:00:40.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 --2- 172.21.15.62:0/2333411618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x558118261b30 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 --2- 172.21.15.62:0/2333411618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e980 0x5581182615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 --2- 172.21.15.62:0/2333411618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e060 0x558118250ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 -- 172.21.15.62:0/2333411618 >> 172.21.15.62:0/2333411618 conn(0x55811824a5f0 msgr2=0x5581182549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:40.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 -- 172.21.15.62:0/2333411618 shutdown_connections 2024-09-12T00:00:40.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 -- 172.21.15.62:0/2333411618 wait complete. 2024-09-12T00:00:40.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 -- start start 2024-09-12T00:00:40.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 Processor -- start 2024-09-12T00:00:40.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e060 0x5581182cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.477+0000 7f74973e0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e980 0x5581182cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f74973e0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x5581182d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f74973e0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74901abdf0 con 0x558118259240 2024-09-12T00:00:40.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f74973e0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ad260 con 0x55811824e060 2024-09-12T00:00:40.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f74973e0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ac4c0 con 0x55811824e980 2024-09-12T00:00:40.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f74973e0740 10 client.0 _ll_register_callbacks cb 0x7f7484013860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:00:40.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x5581182d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x5581182d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57574/0 (socket says 172.21.15.62:57574) 2024-09-12T00:00:40.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748e59c640 1 -- 172.21.15.62:0/259556150 learned_addr learned my addr 172.21.15.62:0/259556150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:00:40.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748ed9d640 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e980 0x5581182cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748e59c640 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e060 0x5581182cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748e59c640 1 -- 172.21.15.62:0/259556150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e980 msgr2=0x5581182cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:40.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748e59c640 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e980 0x5581182cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748e59c640 1 -- 172.21.15.62:0/259556150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e060 msgr2=0x5581182cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:00:40.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748e59c640 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e060 0x5581182cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748e59c640 1 -- 172.21.15.62:0/259556150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581182d1130 con 0x558118259240 2024-09-12T00:00:40.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f748e59c640 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x5581182d0a90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7484001eb0 tx=0x7f74840056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97425d22aedb00d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:00:40.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f7472ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:00:40.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f7472ff5640 1 -- 172.21.15.62:0/259556150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748402b040 con 0x558118259240 2024-09-12T00:00:40.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f74973e0740 1 -- 172.21.15.62:0/259556150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f748402c040 con 0x558118259240 2024-09-12T00:00:40.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.478+0000 7f7472ff5640 1 -- 172.21.15.62:0/259556150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8513+0+0 (secure 0 0 0) 0x7f7484010d00 con 0x558118259240 2024-09-12T00:00:40.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f7472ff5640 1 -- 172.21.15.62:0/259556150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f748400fb10 con 0x558118259240 2024-09-12T00:00:40.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f74973e0740 10 client.16974 fetch_fsmap learned FSMap version 31 2024-09-12T00:00:40.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f74973e0740 1 -- 172.21.15.62:0/259556150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5581182d8520 con 0x558118259240 2024-09-12T00:00:40.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f7472ff5640 1 -- 172.21.15.62:0/259556150 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f748402c040 con 0x558118259240 2024-09-12T00:00:40.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f7472ff5640 1 -- 172.21.15.62:0/259556150 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7484033420 con 0x558118259240 2024-09-12T00:00:40.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f74973e0740 10 client.16974 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T00:00:40.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f74973e0740 10 client.16974 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:00:40.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f74973e0740 20 client.16974 populate_metadata read hostname 'smithi062' 2024-09-12T00:00:40.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f74973e0740 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581182d8a80 0x5581182dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:00:40.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f74973e0740 4 client.16974 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ccd456ca", "release": true, "await": true}] 2024-09-12T00:00:40.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f74973e0740 1 -- 172.21.15.62:0/259556150 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ccd456ca", "release": true, "await": true}) -- 0x5581181bb0a0 con 0x5581182d8a80 2024-09-12T00:00:40.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.479+0000 7f748ed9d640 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581182d8a80 0x5581182dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:00:40.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.480+0000 7f748ed9d640 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581182d8a80 0x5581182dae70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ceb559308d1b747 server_cookie=16168ca86c7f3a84 in_seq=0 out_seq=0 2024-09-12T00:00:40.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.480+0000 7f7472ff5640 10 client.16974 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:40.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.524+0000 7f7472ff5640 1 -- 172.21.15.62:0/259556150 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5581181bb0a0 con 0x5581182d8a80 2024-09-12T00:00:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.524+0000 7f7472ff5640 10 client.16974 handle_command_reply: tid=0 2024-09-12T00:00:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.524+0000 7f74973e0740 1 client.16974 shutdown 2024-09-12T00:00:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.524+0000 7f74973e0740 1 -- 172.21.15.62:0/259556150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 msgr2=0x5581182d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.524+0000 7f74973e0740 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x5581182d0a90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7484001eb0 tx=0x7f74840056d0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.525+0000 7f74973e0740 1 -- 172.21.15.62:0/259556150 shutdown_connections 2024-09-12T00:00:40.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.525+0000 7f74973e0740 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581182d8a80 0x5581182dae70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.525+0000 7f74973e0740 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558118259240 0x5581182d0a90 secure :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7484001eb0 tx=0x7f74840056d0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.525+0000 7f7472ff5640 0 client.16974 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:00:40.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.525+0000 7f74973e0740 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811824e980 0x5581182cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.525+0000 7f74973e0740 1 --2- 172.21.15.62:0/259556150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55811824e060 0x5581182cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.525+0000 7f74973e0740 1 -- 172.21.15.62:0/259556150 >> 172.21.15.62:0/259556150 conn(0x55811824a5f0 msgr2=0x5581182560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:40.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.526+0000 7f74973e0740 1 -- 172.21.15.62:0/259556150 shutdown_connections 2024-09-12T00:00:40.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.526+0000 7f74973e0740 1 -- 172.21.15.62:0/259556150 wait complete. 2024-09-12T00:00:40.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.526+0000 7f74973e0740 20 client.16974 trim_cache size 0 max 16384 2024-09-12T00:00:40.528 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:00:40.528 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 31, 2024-09-12T00:00:40.528 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:00:40.528 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 36, 2024-09-12T00:00:40.528 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:00:40.528 INFO:teuthology.orchestra.run.smithi062.stdout: "ccd456ca": { 2024-09-12T00:00:40.528 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 36, 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 139.4, 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:00:40.529 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:00:40.530 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:40.530 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:40.530 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:40.530 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:40.530 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:00:40.530 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.527+0000 7f7495007640 1 -- 172.21.15.62:0/1850489754 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74780a2cf0 msgr2=0x7f74780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.527+0000 7f7495007640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74780a2cf0 0x7f74780a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f74880cccf0 tx=0x7f74880976b0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.527+0000 7f7495007640 1 -- 172.21.15.62:0/1850489754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490102ef0 msgr2=0x7f74901a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.527+0000 7f7495007640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490102ef0 0x7f74901a4360 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f748800cad0 tx=0x7f748800cf90 comp rx=0 tx=0).stop 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.527+0000 7f7495007640 1 -- 172.21.15.62:0/1850489754 shutdown_connections 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.527+0000 7f7495007640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f74780a2cf0 0x7f74780a51b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.527+0000 7f7495007640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490102ef0 0x7f74901a4360 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.527+0000 7f7495007640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490102570 0x7f7490199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.527+0000 7f7495007640 1 --2- 172.21.15.62:0/1850489754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7490100e20 0x7f74901992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.527+0000 7f7495007640 1 -- 172.21.15.62:0/1850489754 >> 172.21.15.62:0/1850489754 conn(0x7f74900fc7d0 msgr2=0x7f74900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.528+0000 7f7495007640 1 -- 172.21.15.62:0/1850489754 shutdown_connections 2024-09-12T00:00:40.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:00:40.529+0000 7f7495007640 1 -- 172.21.15.62:0/1850489754 wait complete. 2024-09-12T00:00:40.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:40 smithi083 ceph-mon[34805]: pgmap v264: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-09-12T00:00:40.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:40 smithi062 ceph-mon[30958]: pgmap v264: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-09-12T00:00:41.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:40 smithi120 ceph-mon[35182]: pgmap v264: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-09-12T00:00:41.280 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ccd456ca', total seconds elapsed: 22.6 2024-09-12T00:00:41.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:41 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:00:41.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:41 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:00:41.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:41 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:00:41.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:41 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:00:41.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:41 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:00:41.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:41 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:00:42.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:41 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:00:42.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:41 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:00:42.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:41 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:00:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 480.0 s, 1346.1 tps, lat 305.502 ms stddev 2252.309 2024-09-12T00:00:43.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:43 smithi120 ceph-mon[35182]: pgmap v265: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-09-12T00:00:43.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:43 smithi083 ceph-mon[34805]: pgmap v265: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-09-12T00:00:43.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:43 smithi062 ceph-mon[30958]: pgmap v265: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-09-12T00:00:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:45 smithi120 ceph-mon[35182]: pgmap v266: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-09-12T00:00:45.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:45 smithi083 ceph-mon[34805]: pgmap v266: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-09-12T00:00:45.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:45 smithi062 ceph-mon[30958]: pgmap v266: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-09-12T00:00:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 485.0 s, 2634.0 tps, lat 37.901 ms stddev 40.969 2024-09-12T00:00:47.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:47 smithi120 ceph-mon[35182]: pgmap v267: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-09-12T00:00:47.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:47 smithi083 ceph-mon[34805]: pgmap v267: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-09-12T00:00:47.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:47 smithi062 ceph-mon[30958]: pgmap v267: 97 pgs: 97 active+clean; 3.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-09-12T00:00:49.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:49 smithi120 ceph-mon[35182]: pgmap v268: 97 pgs: 97 active+clean; 4.0 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.4 MiB/s wr, 593 op/s 2024-09-12T00:00:49.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:49 smithi083 ceph-mon[34805]: pgmap v268: 97 pgs: 97 active+clean; 4.0 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.4 MiB/s wr, 593 op/s 2024-09-12T00:00:49.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:49 smithi062 ceph-mon[30958]: pgmap v268: 97 pgs: 97 active+clean; 4.0 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.4 MiB/s wr, 593 op/s 2024-09-12T00:00:51.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:51 smithi120 ceph-mon[35182]: pgmap v269: 97 pgs: 97 active+clean; 4.0 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-09-12T00:00:51.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:51 smithi083 ceph-mon[34805]: pgmap v269: 97 pgs: 97 active+clean; 4.0 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-09-12T00:00:51.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:51 smithi062 ceph-mon[30958]: pgmap v269: 97 pgs: 97 active+clean; 4.0 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-09-12T00:00:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 490.0 s, 2076.0 tps, lat 48.105 ms stddev 51.095 2024-09-12T00:00:52.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:52 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:00:52.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:52 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:00:52.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:52 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:00:53.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:53 smithi120 ceph-mon[35182]: pgmap v270: 97 pgs: 97 active+clean; 4.0 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-12T00:00:53.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:53 smithi083 ceph-mon[34805]: pgmap v270: 97 pgs: 97 active+clean; 4.0 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-12T00:00:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:53 smithi062 ceph-mon[30958]: pgmap v270: 97 pgs: 97 active+clean; 4.0 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-12T00:00:55.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:55 smithi120 ceph-mon[35182]: pgmap v271: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 7.3 MiB/s wr, 724 op/s 2024-09-12T00:00:55.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:55 smithi083 ceph-mon[34805]: pgmap v271: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 7.3 MiB/s wr, 724 op/s 2024-09-12T00:00:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:55 smithi062 ceph-mon[30958]: pgmap v271: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 7.3 MiB/s wr, 724 op/s 2024-09-12T00:00:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 495.0 s, 2689.6 tps, lat 37.241 ms stddev 41.725 2024-09-12T00:00:57.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:57 smithi120 ceph-mon[35182]: pgmap v272: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.5 MiB/s wr, 635 op/s 2024-09-12T00:00:57.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:57 smithi083 ceph-mon[34805]: pgmap v272: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.5 MiB/s wr, 635 op/s 2024-09-12T00:00:57.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:57 smithi062 ceph-mon[30958]: pgmap v272: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.5 MiB/s wr, 635 op/s 2024-09-12T00:00:59.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:00:59 smithi120 ceph-mon[35182]: pgmap v273: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.9 MiB/s wr, 928 op/s 2024-09-12T00:00:59.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:00:59 smithi083 ceph-mon[34805]: pgmap v273: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.9 MiB/s wr, 928 op/s 2024-09-12T00:00:59.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:00:59 smithi062 ceph-mon[30958]: pgmap v273: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.9 MiB/s wr, 928 op/s 2024-09-12T00:01:01.610 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:01 smithi062 ceph-mon[30958]: pgmap v274: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 702 op/s 2024-09-12T00:01:01.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:01 smithi120 ceph-mon[35182]: pgmap v274: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 702 op/s 2024-09-12T00:01:01.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:01 smithi083 ceph-mon[34805]: pgmap v274: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 702 op/s 2024-09-12T00:01:02.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 500.0 s, 1601.8 tps, lat 61.782 ms stddev 74.423 2024-09-12T00:01:03.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:03 smithi120 ceph-mon[35182]: pgmap v275: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 682 op/s 2024-09-12T00:01:03.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:03 smithi083 ceph-mon[34805]: pgmap v275: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 682 op/s 2024-09-12T00:01:03.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:03 smithi062 ceph-mon[30958]: pgmap v275: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 682 op/s 2024-09-12T00:01:05.381 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:01:05.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:05 smithi120 ceph-mon[35182]: pgmap v276: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 9.4 MiB/s wr, 924 op/s 2024-09-12T00:01:05.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:05 smithi083 ceph-mon[34805]: pgmap v276: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 9.4 MiB/s wr, 924 op/s 2024-09-12T00:01:05.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:05 smithi062 ceph-mon[30958]: pgmap v276: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 9.4 MiB/s wr, 924 op/s 2024-09-12T00:01:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:06.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.621+0000 7f6990862640 1 -- 172.21.15.62:0/4231506956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c101410 msgr2=0x7f698c10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:06.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.621+0000 7f6990862640 1 --2- 172.21.15.62:0/4231506956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c101410 0x7f698c10d940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6984009ca0 tx=0x7f698402a8d0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.622+0000 7f6990862640 1 -- 172.21.15.62:0/4231506956 shutdown_connections 2024-09-12T00:01:06.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.622+0000 7f6990862640 1 --2- 172.21.15.62:0/4231506956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c101410 0x7f698c10d940 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.622+0000 7f6990862640 1 --2- 172.21.15.62:0/4231506956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f698c100a70 0x7f698c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.622+0000 7f6990862640 1 --2- 172.21.15.62:0/4231506956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c106a70 0x7f698c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.622+0000 7f6990862640 1 -- 172.21.15.62:0/4231506956 >> 172.21.15.62:0/4231506956 conn(0x7f698c0fc790 msgr2=0x7f698c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:06.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.622+0000 7f6990862640 1 -- 172.21.15.62:0/4231506956 shutdown_connections 2024-09-12T00:01:06.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.622+0000 7f6990862640 1 -- 172.21.15.62:0/4231506956 wait complete. 2024-09-12T00:01:06.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.624+0000 7f6990862640 1 Processor -- start 2024-09-12T00:01:06.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.624+0000 7f6990862640 1 -- start start 2024-09-12T00:01:06.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f6990862640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c100a70 0x7f698c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f6990862640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c101410 0x7f698c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f6990862640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f698c106a70 0x7f698c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f698a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c100a70 0x7f698c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f698a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c100a70 0x7f698c1972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:38110/0 (socket says 172.21.15.62:38110) 2024-09-12T00:01:06.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c101410 0x7f698c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f6990862640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f698c110e70 con 0x7f698c106a70 2024-09-12T00:01:06.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f698a59c640 1 -- 172.21.15.62:0/1796333576 learned_addr learned my addr 172.21.15.62:0/1796333576 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:06.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f6990862640 1 -- 172.21.15.62:0/1796333576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f698c110cf0 con 0x7f698c100a70 2024-09-12T00:01:06.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f698ad9d640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f698c106a70 0x7f698c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.625+0000 7f6990862640 1 -- 172.21.15.62:0/1796333576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f698c110ff0 con 0x7f698c101410 2024-09-12T00:01:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.627+0000 7f698ad9d640 1 -- 172.21.15.62:0/1796333576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c101410 msgr2=0x7f698c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.627+0000 7f698ad9d640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c101410 0x7f698c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.627+0000 7f698ad9d640 1 -- 172.21.15.62:0/1796333576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c100a70 msgr2=0x7f698c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.627+0000 7f698ad9d640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c100a70 0x7f698c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.627+0000 7f698ad9d640 1 -- 172.21.15.62:0/1796333576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c19c260 con 0x7f698c106a70 2024-09-12T00:01:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.627+0000 7f6989d9b640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c101410 0x7f698c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:01:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.628+0000 7f698ad9d640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f698c106a70 0x7f698c19bb80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6984005cc0 tx=0x7f6984031710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.629+0000 7f697b7fe640 1 -- 172.21.15.62:0/1796333576 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984003b70 con 0x7f698c106a70 2024-09-12T00:01:06.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.629+0000 7f697b7fe640 1 -- 172.21.15.62:0/1796333576 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6984031900 con 0x7f698c106a70 2024-09-12T00:01:06.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.629+0000 7f697b7fe640 1 -- 172.21.15.62:0/1796333576 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698402fa10 con 0x7f698c106a70 2024-09-12T00:01:06.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.629+0000 7f6990862640 1 -- 172.21.15.62:0/1796333576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c19c4f0 con 0x7f698c106a70 2024-09-12T00:01:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.629+0000 7f6990862640 1 -- 172.21.15.62:0/1796333576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c19c8f0 con 0x7f698c106a70 2024-09-12T00:01:06.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.632+0000 7f6992c3b740 1 Processor -- start 2024-09-12T00:01:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.634+0000 7f697b7fe640 1 -- 172.21.15.62:0/1796333576 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f698402fc50 con 0x7f698c106a70 2024-09-12T00:01:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f6992c3b740 1 -- start start 2024-09-12T00:01:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 0x5604b8af7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8af2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f6992c3b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5604b8a4e2e0 con 0x5604b8af8000 2024-09-12T00:01:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f6992c3b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5604b89f5ce0 con 0x5604b8af26a0 2024-09-12T00:01:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f6992c3b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5604b8a4a2a0 con 0x5604b8af76e0 2024-09-12T00:01:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f698a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8af2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 0x5604b8af7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 0x5604b8af7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:39212/0 (socket says 172.21.15.62:39212) 2024-09-12T00:01:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f698a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8af2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56592/0 (socket says 172.21.15.62:56592) 2024-09-12T00:01:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.635+0000 7f6989d9b640 1 -- 172.21.15.62:0/1785517770 learned_addr learned my addr 172.21.15.62:0/1785517770 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.636+0000 7f6989d9b640 1 --2- 172.21.15.62:0/1785517770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.636+0000 7f698a59c640 1 -- 172.21.15.62:0/1785517770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 msgr2=0x5604b8af7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.636+0000 7f698a59c640 1 --2- 172.21.15.62:0/1785517770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 0x5604b8af7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.636+0000 7f698a59c640 1 -- 172.21.15.62:0/1785517770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 msgr2=0x5604b8af2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.636+0000 7f698a59c640 1 --2- 172.21.15.62:0/1785517770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.636+0000 7f698a59c640 1 -- 172.21.15.62:0/1785517770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b89da440 con 0x5604b8af8000 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.636+0000 7f6989d9b640 1 --2- 172.21.15.62:0/1785517770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.636+0000 7f697b7fe640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69740a2ec0 0x7f69740a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.637+0000 7f697b7fe640 1 -- 172.21.15.62:0/1796333576 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f69840f0540 con 0x7f698c106a70 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.637+0000 7f6989d9b640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69740a2ec0 0x7f69740a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.637+0000 7f698a59c640 1 --2- 172.21.15.62:0/1785517770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8af2030 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f697c005fd0 tx=0x7f697c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba7598e41364a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.637+0000 7f6978ff9640 1 -- 172.21.15.62:0/1785517770 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c01d070 con 0x5604b8af8000 2024-09-12T00:01:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.637+0000 7f6978ff9640 1 -- 172.21.15.62:0/1785517770 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697c018b20 con 0x5604b8af8000 2024-09-12T00:01:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.637+0000 7f6978ff9640 1 -- 172.21.15.62:0/1785517770 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c010a20 con 0x5604b8af8000 2024-09-12T00:01:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.638+0000 7f6992c3b740 1 -- 172.21.15.62:0/1785517770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 msgr2=0x5604b8af2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.638+0000 7f6992c3b740 1 --2- 172.21.15.62:0/1785517770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8af2030 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f697c005fd0 tx=0x7f697c00f790 comp rx=0 tx=0).stop 2024-09-12T00:01:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.638+0000 7f6992c3b740 1 -- 172.21.15.62:0/1785517770 shutdown_connections 2024-09-12T00:01:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.638+0000 7f6992c3b740 1 --2- 172.21.15.62:0/1785517770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.638+0000 7f6992c3b740 1 --2- 172.21.15.62:0/1785517770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8af2030 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.638+0000 7f6992c3b740 1 --2- 172.21.15.62:0/1785517770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 0x5604b8af7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.638+0000 7f6992c3b740 1 -- 172.21.15.62:0/1785517770 >> 172.21.15.62:0/1785517770 conn(0x5604b8ae44e0 msgr2=0x5604b8ae48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.639+0000 7f6989d9b640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69740a2ec0 0x7f69740a5380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5604b8ae5d30 tx=0x7f6980005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.639+0000 7f6992c3b740 1 -- 172.21.15.62:0/1785517770 shutdown_connections 2024-09-12T00:01:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.639+0000 7f6992c3b740 1 -- 172.21.15.62:0/1785517770 wait complete. 2024-09-12T00:01:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.640+0000 7f6992c3b740 1 -- start start 2024-09-12T00:01:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.640+0000 7f6992c3b740 1 Processor -- start 2024-09-12T00:01:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.640+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8afdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.640+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 0x5604b8afb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.640+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8afb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.641+0000 7f6992c3b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5604b8afea20 con 0x5604b8af8000 2024-09-12T00:01:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.641+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8afb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.641+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8afb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56598/0 (socket says 172.21.15.62:56598) 2024-09-12T00:01:06.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.641+0000 7f698ad9d640 1 -- 172.21.15.62:0/659991997 learned_addr learned my addr 172.21.15.62:0/659991997 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:06.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.641+0000 7f6992c3b740 1 -- 172.21.15.62:0/659991997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5604b8a09050 con 0x5604b8af26a0 2024-09-12T00:01:06.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.641+0000 7f6992c3b740 1 -- 172.21.15.62:0/659991997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5604b8afeba0 con 0x5604b8af76e0 2024-09-12T00:01:06.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.641+0000 7f6992c3b740 10 client.0 _ll_register_callbacks cb 0x5604b8a49160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:06.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.641+0000 7f6989d9b640 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 0x5604b8afb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.642+0000 7f698ad9d640 1 -- 172.21.15.62:0/659991997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 msgr2=0x5604b8afb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.642+0000 7f698ad9d640 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 0x5604b8afb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.642+0000 7f698ad9d640 1 -- 172.21.15.62:0/659991997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 msgr2=0x5604b8afdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.642+0000 7f698ad9d640 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8afdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.642+0000 7f698ad9d640 1 -- 172.21.15.62:0/659991997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b87bf3c0 con 0x5604b8af8000 2024-09-12T00:01:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.642+0000 7f698ad9d640 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8afb7f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f69840c5230 tx=0x7f69840bd1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac583c32f852c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.642+0000 7f6971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:01:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.643+0000 7f6971ffb640 1 -- 172.21.15.62:0/659991997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69840c5760 con 0x5604b8af8000 2024-09-12T00:01:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.643+0000 7f6971ffb640 1 -- 172.21.15.62:0/659991997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f69840ede90 con 0x5604b8af8000 2024-09-12T00:01:06.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.643+0000 7f6992c3b740 1 -- 172.21.15.62:0/659991997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f697c018b20 con 0x5604b8af8000 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.643+0000 7f6971ffb640 1 -- 172.21.15.62:0/659991997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69840ee090 con 0x5604b8af8000 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.643+0000 7f6992c3b740 10 client.16986 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.644+0000 7f6992c3b740 1 -- 172.21.15.62:0/659991997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5604b8b0a4a0 con 0x5604b8af8000 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.644+0000 7f6971ffb640 1 -- 172.21.15.62:0/659991997 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697c018b20 con 0x5604b8af8000 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.644+0000 7f6971ffb640 1 -- 172.21.15.62:0/659991997 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f69840bbf40 con 0x5604b8af8000 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.644+0000 7f6992c3b740 10 client.16986 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.644+0000 7f6992c3b740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.644+0000 7f6992c3b740 20 client.16986 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.644+0000 7f6992c3b740 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5604b8b0ba10 0x5604b8b0deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.644+0000 7f6992c3b740 4 client.16986 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.644+0000 7f6992c3b740 1 -- 172.21.15.62:0/659991997 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604b8a5b850 con 0x5604b8b0ba10 2024-09-12T00:01:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.645+0000 7f698a59c640 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5604b8b0ba10 0x5604b8b0deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.645+0000 7f6971ffb640 10 client.16986 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:06.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.646+0000 7f698a59c640 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5604b8b0ba10 0x5604b8b0deb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1ad8fb08ccef0c server_cookie=5d1e62cdf7595fa9 in_seq=0 out_seq=0 2024-09-12T00:01:06.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.647+0000 7f6971ffb640 1 -- 172.21.15.62:0/659991997 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604b8a5b850 con 0x5604b8b0ba10 2024-09-12T00:01:06.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.647+0000 7f6971ffb640 10 client.16986 handle_command_reply: tid=0 2024-09-12T00:01:06.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.647+0000 7f6992c3b740 1 client.16986 shutdown 2024-09-12T00:01:06.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.647+0000 7f6992c3b740 1 -- 172.21.15.62:0/659991997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 msgr2=0x5604b8afb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:06.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.647+0000 7f6992c3b740 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8afb7f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f69840c5230 tx=0x7f69840bd1f0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.648+0000 7f6992c3b740 1 -- 172.21.15.62:0/659991997 shutdown_connections 2024-09-12T00:01:06.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.648+0000 7f6992c3b740 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5604b8b0ba10 0x5604b8b0deb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.648+0000 7f6992c3b740 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af8000 0x5604b8afb7f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.648+0000 7f6971ffb640 0 client.16986 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:06.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.648+0000 7f6992c3b740 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af76e0 0x5604b8afb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.648+0000 7f6992c3b740 1 --2- 172.21.15.62:0/659991997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8afdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.648+0000 7f6992c3b740 1 -- 172.21.15.62:0/659991997 >> 172.21.15.62:0/659991997 conn(0x5604b8ae44e0 msgr2=0x5604b8ae3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:06.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.648+0000 7f6992c3b740 1 -- 172.21.15.62:0/659991997 shutdown_connections 2024-09-12T00:01:06.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.648+0000 7f6992c3b740 1 -- 172.21.15.62:0/659991997 wait complete. 2024-09-12T00:01:06.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.648+0000 7f6992c3b740 20 client.16986 trim_cache size 0 max 16384 2024-09-12T00:01:06.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.686+0000 7f6992c3b740 1 Processor -- start 2024-09-12T00:01:06.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.688+0000 7f6992c3b740 1 -- start start 2024-09-12T00:01:06.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.689+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.689+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af76e0 0x5604b8b07cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.689+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 0x5604b8af6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.689+0000 7f698a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.689+0000 7f698a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:38132/0 (socket says 172.21.15.62:38132) 2024-09-12T00:01:06.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.689+0000 7f698a59c640 1 -- 172.21.15.62:0/2207866089 learned_addr learned my addr 172.21.15.62:0/2207866089 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:06.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.689+0000 7f6992c3b740 1 -- 172.21.15.62:0/2207866089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1ae050 con 0x5604b8af76e0 2024-09-12T00:01:06.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.689+0000 7f6992c3b740 1 -- 172.21.15.62:0/2207866089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1ab920 con 0x5604b8af26a0 2024-09-12T00:01:06.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.689+0000 7f6992c3b740 1 -- 172.21.15.62:0/2207866089 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1a9790 con 0x5604b8af8000 2024-09-12T00:01:06.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.691+0000 7f6989d9b640 1 --2- 172.21.15.62:0/2207866089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af76e0 0x5604b8b07cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.691+0000 7f698ad9d640 1 --2- 172.21.15.62:0/2207866089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 0x5604b8af6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.692+0000 7f698a59c640 1 -- 172.21.15.62:0/2207866089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 msgr2=0x5604b8af6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:06.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.692+0000 7f698a59c640 1 --2- 172.21.15.62:0/2207866089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 0x5604b8af6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.692+0000 7f698a59c640 1 -- 172.21.15.62:0/2207866089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af76e0 msgr2=0x5604b8b07cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:06.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.692+0000 7f698a59c640 1 --2- 172.21.15.62:0/2207866089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af76e0 0x5604b8b07cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.692+0000 7f698a59c640 1 -- 172.21.15.62:0/2207866089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b8af9770 con 0x5604b8af26a0 2024-09-12T00:01:06.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.693+0000 7f698a59c640 1 --2- 172.21.15.62:0/2207866089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f697c00fc90 tx=0x7f697c010680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c6ece770bb04ade server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:06.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.695+0000 7f6973fff640 1 -- 172.21.15.62:0/2207866089 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c01d030 con 0x5604b8af26a0 2024-09-12T00:01:06.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.695+0000 7f6973fff640 1 -- 172.21.15.62:0/2207866089 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697c01e040 con 0x5604b8af26a0 2024-09-12T00:01:06.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.695+0000 7f6973fff640 1 -- 172.21.15.62:0/2207866089 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c010c50 con 0x5604b8af26a0 2024-09-12T00:01:06.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.696+0000 7f6992c3b740 1 -- 172.21.15.62:0/2207866089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 msgr2=0x5604b8af2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:06.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.696+0000 7f6992c3b740 1 --2- 172.21.15.62:0/2207866089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f697c00fc90 tx=0x7f697c010680 comp rx=0 tx=0).stop 2024-09-12T00:01:06.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.700+0000 7f6992c3b740 1 -- 172.21.15.62:0/2207866089 shutdown_connections 2024-09-12T00:01:06.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.700+0000 7f6992c3b740 1 --2- 172.21.15.62:0/2207866089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 0x5604b8af6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.700+0000 7f6992c3b740 1 --2- 172.21.15.62:0/2207866089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af76e0 0x5604b8b07cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.700+0000 7f6992c3b740 1 --2- 172.21.15.62:0/2207866089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af26a0 0x5604b8af2030 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.700+0000 7f6992c3b740 1 -- 172.21.15.62:0/2207866089 >> 172.21.15.62:0/2207866089 conn(0x5604b8ae44e0 msgr2=0x5604b8afca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:06.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.703+0000 7f6992c3b740 1 -- 172.21.15.62:0/2207866089 shutdown_connections 2024-09-12T00:01:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.707+0000 7f6992c3b740 1 -- 172.21.15.62:0/2207866089 wait complete. 2024-09-12T00:01:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.707+0000 7f6992c3b740 1 -- start start 2024-09-12T00:01:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.707+0000 7f6992c3b740 1 Processor -- start 2024-09-12T00:01:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af26a0 0x5604b8b13300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af76e0 0x5604b8b13840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f6992c3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 0x5604b8b13d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f6992c3b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1a9de0 con 0x5604b8af26a0 2024-09-12T00:01:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f6992c3b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1ab250 con 0x5604b8af76e0 2024-09-12T00:01:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f6992c3b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1aa4b0 con 0x5604b8af8000 2024-09-12T00:01:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f6992c3b740 10 client.0 _ll_register_callbacks cb 0x7f698402ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:06.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 0x5604b8b13d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f698a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af26a0 0x5604b8b13300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f698ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 0x5604b8b13d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:39234/0 (socket says 172.21.15.62:39234) 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f698ad9d640 1 -- 172.21.15.62:0/447220080 learned_addr learned my addr 172.21.15.62:0/447220080 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.708+0000 7f6989d9b640 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af76e0 0x5604b8b13840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.709+0000 7f698ad9d640 1 -- 172.21.15.62:0/447220080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af76e0 msgr2=0x5604b8b13840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.709+0000 7f698ad9d640 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af76e0 0x5604b8b13840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.709+0000 7f698ad9d640 1 -- 172.21.15.62:0/447220080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af26a0 msgr2=0x5604b8b13300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.709+0000 7f698ad9d640 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af26a0 0x5604b8b13300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.709+0000 7f698ad9d640 1 -- 172.21.15.62:0/447220080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b8aef850 con 0x5604b8af8000 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.709+0000 7f698ad9d640 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 0x5604b8b13d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6984009ca0 tx=0x7f69840c5500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577434373e50ded9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.709+0000 7f6971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:01:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.712+0000 7f6971ffb640 1 -- 172.21.15.62:0/447220080 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69840b8d90 con 0x5604b8af8000 2024-09-12T00:01:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.712+0000 7f6992c3b740 1 -- 172.21.15.62:0/447220080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f697c01e040 con 0x5604b8af8000 2024-09-12T00:01:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.712+0000 7f6971ffb640 1 -- 172.21.15.62:0/447220080 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f69840bbb70 con 0x5604b8af8000 2024-09-12T00:01:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.713+0000 7f6971ffb640 1 -- 172.21.15.62:0/447220080 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69840ede90 con 0x5604b8af8000 2024-09-12T00:01:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.713+0000 7f6992c3b740 10 client.26491 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.713+0000 7f6992c3b740 1 -- 172.21.15.62:0/447220080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5604b8aef850 con 0x5604b8af8000 2024-09-12T00:01:06.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.719+0000 7f6971ffb640 1 -- 172.21.15.62:0/447220080 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697c01e040 con 0x5604b8af8000 2024-09-12T00:01:06.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.719+0000 7f6971ffb640 1 -- 172.21.15.62:0/447220080 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f69840cd030 con 0x5604b8af8000 2024-09-12T00:01:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.720+0000 7f6992c3b740 10 client.26491 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.720+0000 7f6992c3b740 10 client.26491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:01:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.720+0000 7f6992c3b740 20 client.26491 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.720+0000 7f6992c3b740 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5604b8b0ba10 0x5604b8b81770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.720+0000 7f6992c3b740 4 client.26491 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:01:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.720+0000 7f6992c3b740 1 -- 172.21.15.62:0/447220080 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5604b8a55d60 con 0x5604b8b0ba10 2024-09-12T00:01:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.721+0000 7f698a59c640 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5604b8b0ba10 0x5604b8b81770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.722+0000 7f698a59c640 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5604b8b0ba10 0x5604b8b81770 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76fdaaa833de242 server_cookie=c231eb962dff330b in_seq=0 out_seq=0 2024-09-12T00:01:06.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.723+0000 7f6971ffb640 10 client.26491 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:06.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.723+0000 7f6971ffb640 1 -- 172.21.15.62:0/447220080 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5604b8a55d60 con 0x5604b8b0ba10 2024-09-12T00:01:06.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.723+0000 7f6971ffb640 10 client.26491 handle_command_reply: tid=0 2024-09-12T00:01:06.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.723+0000 7f6992c3b740 1 client.26491 shutdown 2024-09-12T00:01:06.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.726+0000 7f6992c3b740 1 -- 172.21.15.62:0/447220080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 msgr2=0x5604b8b13d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:06.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.726+0000 7f6992c3b740 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 0x5604b8b13d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6984009ca0 tx=0x7f69840c5500 comp rx=0 tx=0).stop 2024-09-12T00:01:06.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.730+0000 7f6992c3b740 1 -- 172.21.15.62:0/447220080 shutdown_connections 2024-09-12T00:01:06.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.730+0000 7f6992c3b740 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5604b8b0ba10 0x5604b8b81770 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.730+0000 7f6992c3b740 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604b8af8000 0x5604b8b13d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.730+0000 7f6992c3b740 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604b8af76e0 0x5604b8b13840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.730+0000 7f6992c3b740 1 --2- 172.21.15.62:0/447220080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604b8af26a0 0x5604b8b13300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.730+0000 7f6992c3b740 1 -- 172.21.15.62:0/447220080 >> 172.21.15.62:0/447220080 conn(0x5604b8ae44e0 msgr2=0x5604b8ae3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:06.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.730+0000 7f6971ffb640 0 client.26491 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:06.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.730+0000 7f6992c3b740 1 -- 172.21.15.62:0/447220080 shutdown_connections 2024-09-12T00:01:06.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.730+0000 7f6992c3b740 1 -- 172.21.15.62:0/447220080 wait complete. 2024-09-12T00:01:06.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.730+0000 7f6992c3b740 20 client.26491 trim_cache size 0 max 16384 2024-09-12T00:01:06.733 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:01:06.733 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:01:06.733 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:01:06.733 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:01:06.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.735+0000 7f6990862640 1 -- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69740a2ec0 msgr2=0x7f69740a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:06.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.735+0000 7f6990862640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69740a2ec0 0x7f69740a5380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5604b8ae5d30 tx=0x7f6980005f90 comp rx=0 tx=0).stop 2024-09-12T00:01:06.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.735+0000 7f6990862640 1 -- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f698c106a70 msgr2=0x7f698c19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:06.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.735+0000 7f6990862640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f698c106a70 0x7f698c19bb80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6984005cc0 tx=0x7f6984031710 comp rx=0 tx=0).stop 2024-09-12T00:01:06.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.739+0000 7f6990862640 1 -- 172.21.15.62:0/1796333576 shutdown_connections 2024-09-12T00:01:06.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.739+0000 7f6990862640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f69740a2ec0 0x7f69740a5380 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.739+0000 7f6990862640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f698c106a70 0x7f698c19bb80 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.739+0000 7f6990862640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f698c101410 0x7f698c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.739+0000 7f6990862640 1 --2- 172.21.15.62:0/1796333576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c100a70 0x7f698c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:06.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.739+0000 7f6990862640 1 -- 172.21.15.62:0/1796333576 >> 172.21.15.62:0/1796333576 conn(0x7f698c0fc790 msgr2=0x7f698c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:06.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.739+0000 7f6990862640 1 -- 172.21.15.62:0/1796333576 shutdown_connections 2024-09-12T00:01:06.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:06.739+0000 7f6990862640 1 -- 172.21.15.62:0/1796333576 wait complete. 2024-09-12T00:01:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 505.0 s, 2009.4 tps, lat 50.422 ms stddev 102.424 2024-09-12T00:01:07.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:07 smithi120 ceph-mon[35182]: pgmap v277: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.8 MiB/s wr, 591 op/s 2024-09-12T00:01:07.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:07 smithi120 ceph-mon[35182]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T00:01:07.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:07 smithi083 ceph-mon[34805]: pgmap v277: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.8 MiB/s wr, 591 op/s 2024-09-12T00:01:07.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:07 smithi083 ceph-mon[34805]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T00:01:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:07 smithi062 ceph-mon[30958]: pgmap v277: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.8 MiB/s wr, 591 op/s 2024-09-12T00:01:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:07 smithi062 ceph-mon[30958]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T00:01:08.405 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:01:08.405 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:01:08.774 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:09.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.361+0000 7fa019b8f640 1 -- 172.21.15.62:0/1856354547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa014102570 msgr2=0x7fa0141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:09.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.361+0000 7fa019b8f640 1 --2- 172.21.15.62:0/1856354547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa014102570 0x7fa0141029b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa008009900 tx=0x7fa00802a480 comp rx=0 tx=0).stop 2024-09-12T00:01:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.362+0000 7fa019b8f640 1 -- 172.21.15.62:0/1856354547 shutdown_connections 2024-09-12T00:01:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.362+0000 7fa019b8f640 1 --2- 172.21.15.62:0/1856354547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014102ef0 0x7fa01410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.362+0000 7fa019b8f640 1 --2- 172.21.15.62:0/1856354547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa014102570 0x7fa0141029b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.362+0000 7fa019b8f640 1 --2- 172.21.15.62:0/1856354547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa014100e20 0x7fa014101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.362+0000 7fa019b8f640 1 -- 172.21.15.62:0/1856354547 >> 172.21.15.62:0/1856354547 conn(0x7fa0140fc7d0 msgr2=0x7fa0140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.362+0000 7fa019b8f640 1 -- 172.21.15.62:0/1856354547 shutdown_connections 2024-09-12T00:01:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.362+0000 7fa019b8f640 1 -- 172.21.15.62:0/1856354547 wait complete. 2024-09-12T00:01:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa019b8f640 1 Processor -- start 2024-09-12T00:01:09.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa019b8f640 1 -- start start 2024-09-12T00:01:09.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa019b8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014100e20 0x7fa014103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:09.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa019b8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa014102570 0x7fa014104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:09.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa019b8f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa014102ef0 0x7fa0141046c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:09.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa019b8f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa014110f80 con 0x7fa014102ef0 2024-09-12T00:01:09.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa019b8f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa014110e00 con 0x7fa014100e20 2024-09-12T00:01:09.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa019b8f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa014111100 con 0x7fa014102570 2024-09-12T00:01:09.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa013fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa014102ef0 0x7fa0141046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:09.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa0137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014100e20 0x7fa014103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:09.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa013fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa014102ef0 0x7fa0141046c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56642/0 (socket says 172.21.15.62:56642) 2024-09-12T00:01:09.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa0137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014100e20 0x7fa014103c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:38156/0 (socket says 172.21.15.62:38156) 2024-09-12T00:01:09.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa013fff640 1 -- 172.21.15.62:0/2229295759 learned_addr learned my addr 172.21.15.62:0/2229295759 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:09.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.363+0000 7fa012ffd640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa014102570 0x7fa014104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa0137fe640 1 -- 172.21.15.62:0/2229295759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa014102570 msgr2=0x7fa014104180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa0137fe640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa014102570 0x7fa014104180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa0137fe640 1 -- 172.21.15.62:0/2229295759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa014102ef0 msgr2=0x7fa0141046c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa0137fe640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa014102ef0 0x7fa0141046c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa0137fe640 1 -- 172.21.15.62:0/2229295759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0141a6920 con 0x7fa014100e20 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa0137fe640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014100e20 0x7fa014103c40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa00400e9f0 tx=0x7fa00400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa010ff9640 1 -- 172.21.15.62:0/2229295759 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00400ccc0 con 0x7fa014100e20 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa010ff9640 1 -- 172.21.15.62:0/2229295759 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa004005020 con 0x7fa014100e20 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa010ff9640 1 -- 172.21.15.62:0/2229295759 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0040049e0 con 0x7fa014100e20 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa019b8f640 1 -- 172.21.15.62:0/2229295759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0141a6c10 con 0x7fa014100e20 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.364+0000 7fa019b8f640 1 -- 172.21.15.62:0/2229295759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0141a70d0 con 0x7fa014100e20 2024-09-12T00:01:09.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.366+0000 7fa019b8f640 1 -- 172.21.15.62:0/2229295759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa014101200 con 0x7fa014100e20 2024-09-12T00:01:09.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.367+0000 7fa010ff9640 1 -- 172.21.15.62:0/2229295759 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa004005560 con 0x7fa014100e20 2024-09-12T00:01:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.368+0000 7fa010ff9640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9ffc0a2cf0 0x7f9ffc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.368+0000 7fa010ff9640 1 -- 172.21.15.62:0/2229295759 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7fa0040cf420 con 0x7fa014100e20 2024-09-12T00:01:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.368+0000 7fa012ffd640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9ffc0a2cf0 0x7f9ffc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:09.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.369+0000 7fa012ffd640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9ffc0a2cf0 0x7f9ffc0a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa008006010 tx=0x7fa008031440 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:09.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.372+0000 7fa010ff9640 1 -- 172.21.15.62:0/2229295759 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0040051d0 con 0x7fa014100e20 2024-09-12T00:01:09.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:09 smithi120 ceph-mon[35182]: pgmap v279: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 21 MiB/s wr, 1.69k op/s 2024-09-12T00:01:09.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:09 smithi083 ceph-mon[34805]: pgmap v279: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 21 MiB/s wr, 1.69k op/s 2024-09-12T00:01:09.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:09 smithi062 ceph-mon[30958]: pgmap v279: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 21 MiB/s wr, 1.69k op/s 2024-09-12T00:01:09.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.734+0000 7fa019b8f640 1 -- 172.21.15.62:0/2229295759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0141a7330 con 0x7fa014100e20 2024-09-12T00:01:09.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.734+0000 7fa010ff9640 1 -- 172.21.15.62:0/2229295759 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11910 (secure 0 0 0) 0x7fa00409bfc0 con 0x7fa014100e20 2024-09-12T00:01:09.734 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:01:09.735 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":32,"btime":"2024-09-12T00:00:51:267719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:00:50.703604+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:01:09.735 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 32 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 -- 172.21.15.62:0/2229295759 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9ffc0a2cf0 msgr2=0x7f9ffc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9ffc0a2cf0 0x7f9ffc0a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa008006010 tx=0x7fa008031440 comp rx=0 tx=0).stop 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 -- 172.21.15.62:0/2229295759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014100e20 msgr2=0x7fa014103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014100e20 0x7fa014103c40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa00400e9f0 tx=0x7fa00400eeb0 comp rx=0 tx=0).stop 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 -- 172.21.15.62:0/2229295759 shutdown_connections 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9ffc0a2cf0 0x7f9ffc0a51b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa014102ef0 0x7fa0141046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa014102570 0x7fa014104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 --2- 172.21.15.62:0/2229295759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014100e20 0x7fa014103c40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 -- 172.21.15.62:0/2229295759 >> 172.21.15.62:0/2229295759 conn(0x7fa0140fc7d0 msgr2=0x7fa014105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:09.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 -- 172.21.15.62:0/2229295759 shutdown_connections 2024-09-12T00:01:09.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:09.737+0000 7fa019b8f640 1 -- 172.21.15.62:0/2229295759 wait complete. 2024-09-12T00:01:10.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:10 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2229295759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:01:10.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:10 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2229295759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:01:10.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:10 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2229295759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:01:11.528 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:01:11.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:11 smithi120 ceph-mon[35182]: pgmap v280: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 25 MiB/s wr, 1.93k op/s 2024-09-12T00:01:11.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:11 smithi083 ceph-mon[34805]: pgmap v280: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 25 MiB/s wr, 1.93k op/s 2024-09-12T00:01:11.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:11 smithi062 ceph-mon[30958]: pgmap v280: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 25 MiB/s wr, 1.93k op/s 2024-09-12T00:01:11.922 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:12.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 510.0 s, 3095.2 tps, lat 31.855 ms stddev 34.177 2024-09-12T00:01:13.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.181+0000 7f713ac10640 1 -- 172.21.15.62:0/3475969445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134106f10 msgr2=0x7f713410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:13.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.181+0000 7f713ac10640 1 --2- 172.21.15.62:0/3475969445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134106f10 0x7f713410ea90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f712c009900 tx=0x7f712c02a400 comp rx=0 tx=0).stop 2024-09-12T00:01:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.182+0000 7f713ac10640 1 -- 172.21.15.62:0/3475969445 shutdown_connections 2024-09-12T00:01:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.182+0000 7f713ac10640 1 --2- 172.21.15.62:0/3475969445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134106f10 0x7f713410ea90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.182+0000 7f713ac10640 1 --2- 172.21.15.62:0/3475969445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7134102790 0x7f71341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.182+0000 7f713ac10640 1 --2- 172.21.15.62:0/3475969445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134101de0 0x7f71341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.182+0000 7f713ac10640 1 -- 172.21.15.62:0/3475969445 >> 172.21.15.62:0/3475969445 conn(0x7f71340fd880 msgr2=0x7f71340ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.182+0000 7f713ac10640 1 -- 172.21.15.62:0/3475969445 shutdown_connections 2024-09-12T00:01:13.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.183+0000 7f713ac10640 1 -- 172.21.15.62:0/3475969445 wait complete. 2024-09-12T00:01:13.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.183+0000 7f713ac10640 1 Processor -- start 2024-09-12T00:01:13.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.183+0000 7f713ac10640 1 -- start start 2024-09-12T00:01:13.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.184+0000 7f713ac10640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134101de0 0x7f7134198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.184+0000 7f713ac10640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134102790 0x7f71341988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.184+0000 7f713ac10640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7134106f10 0x7f713419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.184+0000 7f713ac10640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7134111f70 con 0x7f7134102790 2024-09-12T00:01:13.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.184+0000 7f713ac10640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7134111df0 con 0x7f7134106f10 2024-09-12T00:01:13.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.184+0000 7f713ac10640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71341120f0 con 0x7f7134101de0 2024-09-12T00:01:13.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.184+0000 7f71391ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7134106f10 0x7f713419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.184+0000 7f71391ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7134106f10 0x7f713419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58230/0 (socket says 172.21.15.62:58230) 2024-09-12T00:01:13.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.184+0000 7f71391ad640 1 -- 172.21.15.62:0/4184951594 learned_addr learned my addr 172.21.15.62:0/4184951594 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:13.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.184+0000 7f7133fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134102790 0x7f71341988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.185+0000 7f71391ad640 1 -- 172.21.15.62:0/4184951594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134101de0 msgr2=0x7f7134198370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:01:13.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.185+0000 7f71391ad640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134101de0 0x7f7134198370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.185+0000 7f71391ad640 1 -- 172.21.15.62:0/4184951594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134102790 msgr2=0x7f71341988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:13.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.185+0000 7f71391ad640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134102790 0x7f71341988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.185+0000 7f71391ad640 1 -- 172.21.15.62:0/4184951594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713419d320 con 0x7f7134106f10 2024-09-12T00:01:13.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.185+0000 7f71391ad640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7134106f10 0x7f713419cc40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f712c009230 tx=0x7f712c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:13.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.185+0000 7f7131ffb640 1 -- 172.21.15.62:0/4184951594 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c03f030 con 0x7f7134106f10 2024-09-12T00:01:13.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.185+0000 7f7131ffb640 1 -- 172.21.15.62:0/4184951594 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f712c003ce0 con 0x7f7134106f10 2024-09-12T00:01:13.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.185+0000 7f7131ffb640 1 -- 172.21.15.62:0/4184951594 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c035070 con 0x7f7134106f10 2024-09-12T00:01:13.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.186+0000 7f713ac10640 1 -- 172.21.15.62:0/4184951594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713419d5b0 con 0x7f7134106f10 2024-09-12T00:01:13.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.186+0000 7f713ac10640 1 -- 172.21.15.62:0/4184951594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713419da40 con 0x7f7134106f10 2024-09-12T00:01:13.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.189+0000 7f7131ffb640 1 -- 172.21.15.62:0/4184951594 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f712c042030 con 0x7f7134106f10 2024-09-12T00:01:13.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.190+0000 7f7131ffb640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71240a2ec0 0x7f71240a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.190+0000 7f7131ffb640 1 -- 172.21.15.62:0/4184951594 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f712c0ef9d0 con 0x7f7134106f10 2024-09-12T00:01:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.190+0000 7f71389ac640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71240a2ec0 0x7f71240a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.190+0000 7f713cfe9740 1 Processor -- start 2024-09-12T00:01:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.190+0000 7f713cfe9740 1 -- start start 2024-09-12T00:01:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f71389ac640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71240a2ec0 0x7f71240a5380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7128005fd0 tx=0x7f7128005f00 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:13.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 0x55b1d7a638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a63de0 0x55b1d7a64220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a70f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f713cfe9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b1d79c12f0 con 0x55b1d7a6eb80 2024-09-12T00:01:13.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f713cfe9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b1d7968cf0 con 0x55b1d7a65350 2024-09-12T00:01:13.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f713cfe9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b1d79bd2b0 con 0x55b1d7a63de0 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f7133fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a63de0 0x55b1d7a64220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f7133fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a63de0 0x55b1d7a64220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:39680/0 (socket says 172.21.15.62:39680) 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f7133fff640 1 -- 172.21.15.62:0/2747109495 learned_addr learned my addr 172.21.15.62:0/2747109495 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f71391ad640 1 --2- 172.21.15.62:0/2747109495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a70f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.191+0000 7f71389ac640 1 --2- 172.21.15.62:0/2747109495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 0x55b1d7a638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.192+0000 7f7133fff640 1 -- 172.21.15.62:0/2747109495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 msgr2=0x55b1d7a638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.192+0000 7f7133fff640 1 --2- 172.21.15.62:0/2747109495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 0x55b1d7a638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.192+0000 7f7133fff640 1 -- 172.21.15.62:0/2747109495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a6eb80 msgr2=0x55b1d7a70f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.192+0000 7f7133fff640 1 --2- 172.21.15.62:0/2747109495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a70f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.192+0000 7f7133fff640 1 -- 172.21.15.62:0/2747109495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d794d3a0 con 0x55b1d7a63de0 2024-09-12T00:01:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.192+0000 7f7133fff640 1 --2- 172.21.15.62:0/2747109495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a63de0 0x55b1d7a64220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f712000ac20 tx=0x7f712000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7633aeca1d63a5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.192+0000 7f711effd640 1 -- 172.21.15.62:0/2747109495 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712002e070 con 0x55b1d7a63de0 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.192+0000 7f711effd640 1 -- 172.21.15.62:0/2747109495 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7120029b30 con 0x55b1d7a63de0 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.192+0000 7f711effd640 1 -- 172.21.15.62:0/2747109495 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120010b90 con 0x55b1d7a63de0 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.193+0000 7f713cfe9740 1 -- 172.21.15.62:0/2747109495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a63de0 msgr2=0x55b1d7a64220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.193+0000 7f713cfe9740 1 --2- 172.21.15.62:0/2747109495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a63de0 0x55b1d7a64220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f712000ac20 tx=0x7f712000f790 comp rx=0 tx=0).stop 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.193+0000 7f713cfe9740 1 -- 172.21.15.62:0/2747109495 shutdown_connections 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.193+0000 7f713cfe9740 1 --2- 172.21.15.62:0/2747109495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a70f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.193+0000 7f713cfe9740 1 --2- 172.21.15.62:0/2747109495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a63de0 0x55b1d7a64220 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.193+0000 7f713cfe9740 1 --2- 172.21.15.62:0/2747109495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 0x55b1d7a638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.193+0000 7f713cfe9740 1 -- 172.21.15.62:0/2747109495 >> 172.21.15.62:0/2747109495 conn(0x55b1d7a24310 msgr2=0x55b1d7a60470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.194+0000 7f713cfe9740 1 -- 172.21.15.62:0/2747109495 shutdown_connections 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.194+0000 7f713cfe9740 1 -- 172.21.15.62:0/2747109495 wait complete. 2024-09-12T00:01:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.194+0000 7f713cfe9740 1 -- start start 2024-09-12T00:01:13.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.194+0000 7f713cfe9740 1 Processor -- start 2024-09-12T00:01:13.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a6b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a65350 0x55b1d7a6be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a6c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f713cfe9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b1d7a57870 con 0x55b1d7a65350 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f713cfe9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b1d797c060 con 0x55b1d7a63de0 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f713cfe9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b1d7a579f0 con 0x55b1d7a6eb80 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f71389ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a6b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f71389ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a6b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58244/0 (socket says 172.21.15.62:58244) 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f71389ac640 1 -- 172.21.15.62:0/3833844083 learned_addr learned my addr 172.21.15.62:0/3833844083 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f713cfe9740 10 client.0 _ll_register_callbacks cb 0x55b1d76cf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f71389ac640 1 -- 172.21.15.62:0/3833844083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a6eb80 msgr2=0x55b1d7a6c3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f71389ac640 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a6c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f71389ac640 1 -- 172.21.15.62:0/3833844083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a65350 msgr2=0x55b1d7a6be90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:01:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f71389ac640 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a65350 0x55b1d7a6be90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.195+0000 7f71389ac640 1 -- 172.21.15.62:0/3833844083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d77323c0 con 0x55b1d7a63de0 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.196+0000 7f71389ac640 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a6b950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f712800fab0 tx=0x7f7128004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66cb7e3bd86ca162 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.196+0000 7f7113fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.196+0000 7f7113fff640 1 -- 172.21.15.62:0/3833844083 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128021070 con 0x55b1d7a63de0 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.196+0000 7f713cfe9740 1 -- 172.21.15.62:0/3833844083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7120029b30 con 0x55b1d7a63de0 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.196+0000 7f7113fff640 1 -- 172.21.15.62:0/3833844083 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f712800ee00 con 0x55b1d7a63de0 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.196+0000 7f7113fff640 1 -- 172.21.15.62:0/3833844083 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f712802c050 con 0x55b1d7a63de0 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.196+0000 7f713cfe9740 10 client.26564 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.196+0000 7f713cfe9740 1 -- 172.21.15.62:0/3833844083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b1d7a74ca0 con 0x55b1d7a63de0 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.197+0000 7f7113fff640 1 -- 172.21.15.62:0/3833844083 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7120029b30 con 0x55b1d7a63de0 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.197+0000 7f7113fff640 1 -- 172.21.15.62:0/3833844083 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f7128005870 con 0x55b1d7a63de0 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.197+0000 7f713cfe9740 10 client.26564 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.197+0000 7f713cfe9740 10 client.26564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:01:13.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.197+0000 7f713cfe9740 20 client.26564 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:13.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.197+0000 7f713cfe9740 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1d7a760a0 0x55b1d7a78560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.197+0000 7f713cfe9740 4 client.26564 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:01:13.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.197+0000 7f713cfe9740 1 -- 172.21.15.62:0/3833844083 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1d79ce860 con 0x55b1d7a760a0 2024-09-12T00:01:13.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.198+0000 7f7133fff640 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1d7a760a0 0x55b1d7a78560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.198+0000 7f7133fff640 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1d7a760a0 0x55b1d7a78560 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8c071ed41f0f663 server_cookie=deb2bfba57cb5055 in_seq=0 out_seq=0 2024-09-12T00:01:13.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.198+0000 7f7113fff640 10 client.26564 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:01:13.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.199+0000 7f7113fff640 1 -- 172.21.15.62:0/3833844083 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1d79ce860 con 0x55b1d7a760a0 2024-09-12T00:01:13.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.199+0000 7f7113fff640 10 client.26564 handle_command_reply: tid=0 2024-09-12T00:01:13.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.199+0000 7f713cfe9740 1 client.26564 shutdown 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.199+0000 7f713cfe9740 1 -- 172.21.15.62:0/3833844083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 msgr2=0x55b1d7a6b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.199+0000 7f713cfe9740 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a6b950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f712800fab0 tx=0x7f7128004ab0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.200+0000 7f713cfe9740 1 -- 172.21.15.62:0/3833844083 shutdown_connections 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.200+0000 7f713cfe9740 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1d7a760a0 0x55b1d7a78560 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.200+0000 7f713cfe9740 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a6c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.200+0000 7f713cfe9740 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a65350 0x55b1d7a6be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.200+0000 7f7113fff640 0 client.26564 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.200+0000 7f713cfe9740 1 --2- 172.21.15.62:0/3833844083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a6b950 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.200+0000 7f713cfe9740 1 -- 172.21.15.62:0/3833844083 >> 172.21.15.62:0/3833844083 conn(0x55b1d7a24310 msgr2=0x55b1d7a66170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.200+0000 7f713cfe9740 1 -- 172.21.15.62:0/3833844083 shutdown_connections 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.200+0000 7f713cfe9740 1 -- 172.21.15.62:0/3833844083 wait complete. 2024-09-12T00:01:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.200+0000 7f713cfe9740 20 client.26564 trim_cache size 0 max 16384 2024-09-12T00:01:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.220+0000 7f713cfe9740 1 Processor -- start 2024-09-12T00:01:13.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.224+0000 7f713cfe9740 1 -- start start 2024-09-12T00:01:13.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.225+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a56fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.225+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a65350 0x55b1d7a6c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.225+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a7de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.225+0000 7f713cfe9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f71341adef0 con 0x55b1d7a6eb80 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.225+0000 7f713cfe9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71341ab7c0 con 0x55b1d7a63de0 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.225+0000 7f713cfe9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71341a9630 con 0x55b1d7a65350 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.225+0000 7f71389ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a56fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.225+0000 7f71389ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a56fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58256/0 (socket says 172.21.15.62:58256) 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.225+0000 7f71389ac640 1 -- 172.21.15.62:0/2677175268 learned_addr learned my addr 172.21.15.62:0/2677175268 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.225+0000 7f71391ad640 1 --2- 172.21.15.62:0/2677175268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a7de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.226+0000 7f71389ac640 1 -- 172.21.15.62:0/2677175268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a65350 msgr2=0x55b1d7a6c220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.226+0000 7f71389ac640 1 --2- 172.21.15.62:0/2677175268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a65350 0x55b1d7a6c220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.226+0000 7f71389ac640 1 -- 172.21.15.62:0/2677175268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a6eb80 msgr2=0x55b1d7a7de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.226+0000 7f71389ac640 1 --2- 172.21.15.62:0/2677175268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a7de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.226+0000 7f71389ac640 1 -- 172.21.15.62:0/2677175268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d7a69ea0 con 0x55b1d7a63de0 2024-09-12T00:01:13.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.226+0000 7f71389ac640 1 --2- 172.21.15.62:0/2677175268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a56fb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f712800c760 tx=0x7f7128007360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c71714965c40ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:13.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.226+0000 7f711e7fc640 1 -- 172.21.15.62:0/2677175268 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128021070 con 0x55b1d7a63de0 2024-09-12T00:01:13.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.227+0000 7f711e7fc640 1 -- 172.21.15.62:0/2677175268 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7128015040 con 0x55b1d7a63de0 2024-09-12T00:01:13.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.227+0000 7f711e7fc640 1 -- 172.21.15.62:0/2677175268 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128003d50 con 0x55b1d7a63de0 2024-09-12T00:01:13.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.228+0000 7f713cfe9740 1 -- 172.21.15.62:0/2677175268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 msgr2=0x55b1d7a56fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:13.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.228+0000 7f713cfe9740 1 --2- 172.21.15.62:0/2677175268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a56fb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f712800c760 tx=0x7f7128007360 comp rx=0 tx=0).stop 2024-09-12T00:01:13.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.235+0000 7f713cfe9740 1 -- 172.21.15.62:0/2677175268 shutdown_connections 2024-09-12T00:01:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.235+0000 7f713cfe9740 1 --2- 172.21.15.62:0/2677175268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a6eb80 0x55b1d7a7de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.235+0000 7f713cfe9740 1 --2- 172.21.15.62:0/2677175268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a65350 0x55b1d7a6c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.235+0000 7f713cfe9740 1 --2- 172.21.15.62:0/2677175268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a63de0 0x55b1d7a56fb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.235+0000 7f713cfe9740 1 -- 172.21.15.62:0/2677175268 >> 172.21.15.62:0/2677175268 conn(0x55b1d7a24310 msgr2=0x55b1d7a6d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:13.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.239+0000 7f713cfe9740 1 -- 172.21.15.62:0/2677175268 shutdown_connections 2024-09-12T00:01:13.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.244+0000 7f713cfe9740 1 -- 172.21.15.62:0/2677175268 wait complete. 2024-09-12T00:01:13.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.244+0000 7f713cfe9740 1 -- start start 2024-09-12T00:01:13.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.244+0000 7f713cfe9740 1 Processor -- start 2024-09-12T00:01:13.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.245+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a63de0 0x55b1d7adc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.245+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 0x55b1d7adadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.245+0000 7f713cfe9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a6eb80 0x55b1d7adb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.245+0000 7f7133fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 0x55b1d7adadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.245+0000 7f7133fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 0x55b1d7adadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58262/0 (socket says 172.21.15.62:58262) 2024-09-12T00:01:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.246+0000 7f71391ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a6eb80 0x55b1d7adb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.246+0000 7f7133fff640 1 -- 172.21.15.62:0/1370214591 learned_addr learned my addr 172.21.15.62:0/1370214591 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.246+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f71341a9c80 con 0x55b1d7a63de0 2024-09-12T00:01:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.246+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71341ab0f0 con 0x55b1d7a65350 2024-09-12T00:01:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.246+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71341aa350 con 0x55b1d7a6eb80 2024-09-12T00:01:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.246+0000 7f713cfe9740 10 client.0 _ll_register_callbacks cb 0x7f71280078b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.247+0000 7f7133fff640 1 -- 172.21.15.62:0/1370214591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a6eb80 msgr2=0x55b1d7adb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.247+0000 7f7133fff640 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a6eb80 0x55b1d7adb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.247+0000 7f7133fff640 1 -- 172.21.15.62:0/1370214591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a63de0 msgr2=0x55b1d7adc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.247+0000 7f7133fff640 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a63de0 0x55b1d7adc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.247+0000 7f7133fff640 1 -- 172.21.15.62:0/1370214591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d7adbb80 con 0x55b1d7a65350 2024-09-12T00:01:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.247+0000 7f7133fff640 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 0x55b1d7adadd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f712002c040 tx=0x7f71200058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75e0dc1f23a53f1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.247+0000 7f7113fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:01:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.247+0000 7f7113fff640 1 -- 172.21.15.62:0/1370214591 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120042060 con 0x55b1d7a65350 2024-09-12T00:01:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.248+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7128015040 con 0x55b1d7a65350 2024-09-12T00:01:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.248+0000 7f7113fff640 1 -- 172.21.15.62:0/1370214591 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f7120005450 con 0x55b1d7a65350 2024-09-12T00:01:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.248+0000 7f7113fff640 1 -- 172.21.15.62:0/1370214591 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f712002e030 con 0x55b1d7a65350 2024-09-12T00:01:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.248+0000 7f713cfe9740 10 client.26570 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.248+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b1d7af1e70 con 0x55b1d7a65350 2024-09-12T00:01:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.249+0000 7f7113fff640 1 -- 172.21.15.62:0/1370214591 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f712002f040 con 0x55b1d7a65350 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.249+0000 7f7113fff640 1 -- 172.21.15.62:0/1370214591 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f7120029ce0 con 0x55b1d7a65350 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.249+0000 7f713cfe9740 10 client.26570 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.249+0000 7f713cfe9740 10 client.26570 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.249+0000 7f713cfe9740 20 client.26570 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.249+0000 7f713cfe9740 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1d7a760a0 0x55b1d7af4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.249+0000 7f713cfe9740 4 client.26570 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.249+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7128015040 con 0x55b1d7a760a0 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.250+0000 7f71389ac640 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1d7a760a0 0x55b1d7af4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.250+0000 7f7113fff640 10 client.26570 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.250+0000 7f71389ac640 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1d7a760a0 0x55b1d7af4320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f21d714b0d7feb2a server_cookie=2883536ae9a68ac9 in_seq=0 out_seq=0 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.250+0000 7f7113fff640 1 -- 172.21.15.62:0/1370214591 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7128015040 con 0x55b1d7a760a0 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.250+0000 7f7113fff640 10 client.26570 handle_command_reply: tid=0 2024-09-12T00:01:13.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.250+0000 7f713cfe9740 1 client.26570 shutdown 2024-09-12T00:01:13.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.251+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 msgr2=0x55b1d7adadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:13.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.251+0000 7f713cfe9740 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 0x55b1d7adadd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f712002c040 tx=0x7f71200058d0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.252+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 shutdown_connections 2024-09-12T00:01:13.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.252+0000 7f713cfe9740 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b1d7a760a0 0x55b1d7af4320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.252+0000 7f713cfe9740 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1d7a6eb80 0x55b1d7adb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.252+0000 7f713cfe9740 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1d7a65350 0x55b1d7adadd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.252+0000 7f713cfe9740 1 --2- 172.21.15.62:0/1370214591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1d7a63de0 0x55b1d7adc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.252+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 >> 172.21.15.62:0/1370214591 conn(0x55b1d7a24310 msgr2=0x55b1d7a6b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:13.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.252+0000 7f7113fff640 0 client.26570 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:01:13.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.253+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 shutdown_connections 2024-09-12T00:01:13.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.253+0000 7f713cfe9740 1 -- 172.21.15.62:0/1370214591 wait complete. 2024-09-12T00:01:13.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.253+0000 7f713cfe9740 20 client.26570 trim_cache size 0 max 16384 2024-09-12T00:01:13.254 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:01:13.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.256+0000 7f713ac10640 1 -- 172.21.15.62:0/4184951594 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71240a2ec0 msgr2=0x7f71240a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:13.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.256+0000 7f713ac10640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71240a2ec0 0x7f71240a5380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7128005fd0 tx=0x7f7128005f00 comp rx=0 tx=0).stop 2024-09-12T00:01:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.256+0000 7f713ac10640 1 -- 172.21.15.62:0/4184951594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7134106f10 msgr2=0x7f713419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.256+0000 7f713ac10640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7134106f10 0x7f713419cc40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f712c009230 tx=0x7f712c0042f0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.256+0000 7f713ac10640 1 -- 172.21.15.62:0/4184951594 shutdown_connections 2024-09-12T00:01:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.256+0000 7f713ac10640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f71240a2ec0 0x7f71240a5380 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.256+0000 7f713ac10640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7134106f10 0x7f713419cc40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.256+0000 7f713ac10640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134102790 0x7f71341988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.256+0000 7f713ac10640 1 --2- 172.21.15.62:0/4184951594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134101de0 0x7f7134198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:13.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.256+0000 7f713ac10640 1 -- 172.21.15.62:0/4184951594 >> 172.21.15.62:0/4184951594 conn(0x7f71340fd880 msgr2=0x7f71340fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:13.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.257+0000 7f713ac10640 1 -- 172.21.15.62:0/4184951594 shutdown_connections 2024-09-12T00:01:13.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:13.257+0000 7f713ac10640 1 -- 172.21.15.62:0/4184951594 wait complete. 2024-09-12T00:01:13.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:13 smithi062 ceph-mon[30958]: pgmap v281: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 25 MiB/s wr, 1.93k op/s 2024-09-12T00:01:13.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:13 smithi120 ceph-mon[35182]: pgmap v281: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 25 MiB/s wr, 1.93k op/s 2024-09-12T00:01:13.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:13 smithi083 ceph-mon[34805]: pgmap v281: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 25 MiB/s wr, 1.93k op/s 2024-09-12T00:01:13.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:01:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.904+0000 7f172795b640 1 -- 172.21.15.62:0/823056358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1720100860 msgr2=0x7f17201045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.904+0000 7f172795b640 1 --2- 172.21.15.62:0/823056358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1720100860 0x7f17201045a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f171c009900 tx=0x7f171c02a410 comp rx=0 tx=0).stop 2024-09-12T00:01:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.905+0000 7f172795b640 1 -- 172.21.15.62:0/823056358 shutdown_connections 2024-09-12T00:01:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.905+0000 7f172795b640 1 --2- 172.21.15.62:0/823056358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1720100860 0x7f17201045a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.905+0000 7f172795b640 1 --2- 172.21.15.62:0/823056358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1720107440 0x7f1720100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.905+0000 7f172795b640 1 --2- 172.21.15.62:0/823056358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1720106a90 0x7f1720106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.905+0000 7f172795b640 1 -- 172.21.15.62:0/823056358 >> 172.21.15.62:0/823056358 conn(0x7f1720077230 msgr2=0x7f17200fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.905+0000 7f172795b640 1 -- 172.21.15.62:0/823056358 shutdown_connections 2024-09-12T00:01:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.905+0000 7f172795b640 1 -- 172.21.15.62:0/823056358 wait complete. 2024-09-12T00:01:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.905+0000 7f172795b640 1 Processor -- start 2024-09-12T00:01:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.905+0000 7f172795b640 1 -- start start 2024-09-12T00:01:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f172795b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1720100860 0x7f17201a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f172795b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1720106a90 0x7f17201a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f172795b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172019e660 0x7f172019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f172795b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1720118590 con 0x7f1720100860 2024-09-12T00:01:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f172795b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1720118410 con 0x7f1720106a90 2024-09-12T00:01:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f172795b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1720118710 con 0x7f172019e660 2024-09-12T00:01:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1720106a90 0x7f17201a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f1725ef8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172019e660 0x7f172019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1720106a90 0x7f17201a4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58286/0 (socket says 172.21.15.62:58286) 2024-09-12T00:01:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f17256f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1720100860 0x7f17201a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f1724ef6640 1 -- 172.21.15.62:0/3850940706 learned_addr learned my addr 172.21.15.62:0/3850940706 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:14.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f1725ef8640 1 -- 172.21.15.62:0/3850940706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1720106a90 msgr2=0x7f17201a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.906+0000 7f1725ef8640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1720106a90 0x7f17201a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.907+0000 7f1725ef8640 1 -- 172.21.15.62:0/3850940706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1720100860 msgr2=0x7f17201a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.907+0000 7f1725ef8640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1720100860 0x7f17201a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.907+0000 7f1725ef8640 1 -- 172.21.15.62:0/3850940706 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172019f210 con 0x7f172019e660 2024-09-12T00:01:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.908+0000 7f1725ef8640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172019e660 0x7f172019eae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f171c02aa00 tx=0x7f171c0305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.909+0000 7f17167fc640 1 -- 172.21.15.62:0/3850940706 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c03f030 con 0x7f172019e660 2024-09-12T00:01:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.909+0000 7f17167fc640 1 -- 172.21.15.62:0/3850940706 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f171c030db0 con 0x7f172019e660 2024-09-12T00:01:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.909+0000 7f17167fc640 1 -- 172.21.15.62:0/3850940706 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c035070 con 0x7f172019e660 2024-09-12T00:01:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.909+0000 7f172795b640 1 -- 172.21.15.62:0/3850940706 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172019f4a0 con 0x7f172019e660 2024-09-12T00:01:14.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.909+0000 7f172795b640 1 -- 172.21.15.62:0/3850940706 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17201ab5d0 con 0x7f172019e660 2024-09-12T00:01:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.912+0000 7f17167fc640 1 -- 172.21.15.62:0/3850940706 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f171c042030 con 0x7f172019e660 2024-09-12T00:01:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.913+0000 7f1729d34740 1 Processor -- start 2024-09-12T00:01:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.913+0000 7f1729d34740 1 -- start start 2024-09-12T00:01:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.913+0000 7f1729d34740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a32a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.913+0000 7f1729d34740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a32aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.913+0000 7f1729d34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a337760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.913+0000 7f1729d34740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55671a2902f0 con 0x55671a32a5d0 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.913+0000 7f1729d34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55671a237cf0 con 0x55671a335370 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.913+0000 7f1729d34740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55671a28c2b0 con 0x55671a32bb40 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.914+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a337760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.914+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a337760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58296/0 (socket says 172.21.15.62:58296) 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.914+0000 7f1724ef6640 1 -- 172.21.15.62:0/763791871 learned_addr learned my addr 172.21.15.62:0/763791871 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.914+0000 7f1724ef6640 1 --2- 172.21.15.62:0/763791871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a32a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.914+0000 7f17167fc640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f170c0a2dc0 0x7f170c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.914+0000 7f1725ef8640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f170c0a2dc0 0x7f170c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.915+0000 7f17167fc640 1 -- 172.21.15.62:0/3850940706 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f171c0ef910 con 0x7f172019e660 2024-09-12T00:01:14.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.915+0000 7f1725ef8640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f170c0a2dc0 0x7f170c0a5280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f171c04a000 tx=0x7f171c03bab0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.917+0000 7f1724ef6640 1 -- 172.21.15.62:0/763791871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 msgr2=0x55671a32a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.917+0000 7f1724ef6640 1 --2- 172.21.15.62:0/763791871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a32a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.917+0000 7f1724ef6640 1 -- 172.21.15.62:0/763791871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 msgr2=0x55671a32aa10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.917+0000 7f1724ef6640 1 --2- 172.21.15.62:0/763791871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a32aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.917+0000 7f1724ef6640 1 -- 172.21.15.62:0/763791871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55671a21c3a0 con 0x55671a335370 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.917+0000 7f1724ef6640 1 --2- 172.21.15.62:0/763791871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a337760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f17100095e0 tx=0x7f1710009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b6c5dc0014d0957 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.917+0000 7f170b7fe640 1 -- 172.21.15.62:0/763791871 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171000f790 con 0x55671a335370 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.917+0000 7f170b7fe640 1 -- 172.21.15.62:0/763791871 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f171000fe10 con 0x55671a335370 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.917+0000 7f170b7fe640 1 -- 172.21.15.62:0/763791871 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171001fd10 con 0x55671a335370 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.918+0000 7f1729d34740 1 -- 172.21.15.62:0/763791871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 msgr2=0x55671a337760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.918+0000 7f1729d34740 1 --2- 172.21.15.62:0/763791871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a337760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f17100095e0 tx=0x7f1710009530 comp rx=0 tx=0).stop 2024-09-12T00:01:14.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.919+0000 7f1729d34740 1 -- 172.21.15.62:0/763791871 shutdown_connections 2024-09-12T00:01:14.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.919+0000 7f1729d34740 1 --2- 172.21.15.62:0/763791871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a337760 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.919+0000 7f1729d34740 1 --2- 172.21.15.62:0/763791871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a32aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.919+0000 7f1729d34740 1 --2- 172.21.15.62:0/763791871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a32a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.919+0000 7f1729d34740 1 -- 172.21.15.62:0/763791871 >> 172.21.15.62:0/763791871 conn(0x55671a2f3310 msgr2=0x55671a326c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:14.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.920+0000 7f1729d34740 1 -- 172.21.15.62:0/763791871 shutdown_connections 2024-09-12T00:01:14.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.920+0000 7f1729d34740 1 -- 172.21.15.62:0/763791871 wait complete. 2024-09-12T00:01:14.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.920+0000 7f1729d34740 1 -- start start 2024-09-12T00:01:14.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.920+0000 7f1729d34740 1 Processor -- start 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1729d34740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a334e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1729d34740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a3320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1729d34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a332860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1729d34740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55671a338070 con 0x55671a32a5d0 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1729d34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55671a24b060 con 0x55671a335370 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1729d34740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55671a3381f0 con 0x55671a32bb40 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1729d34740 10 client.0 _ll_register_callbacks cb 0x556719f9e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a334e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a334e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49050/0 (socket says 172.21.15.62:49050) 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1724ef6640 1 -- 172.21.15.62:0/2096919763 learned_addr learned my addr 172.21.15.62:0/2096919763 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f17256f7640 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a3320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.921+0000 7f1724ef6640 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a332860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f17256f7640 1 -- 172.21.15.62:0/2096919763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 msgr2=0x55671a332860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f17256f7640 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a332860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f17256f7640 1 -- 172.21.15.62:0/2096919763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 msgr2=0x55671a334e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f17256f7640 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a334e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f17256f7640 1 -- 172.21.15.62:0/2096919763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55671a0013c0 con 0x55671a32bb40 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f1724ef6640 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a334e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f17256f7640 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a3320d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1718005fd0 tx=0x7f1718011dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ff099be7a6a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f1708ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f1708ff9640 1 -- 172.21.15.62:0/2096919763 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718003e40 con 0x55671a32bb40 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f1708ff9640 1 -- 172.21.15.62:0/2096919763 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f1718004b20 con 0x55671a32bb40 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f1729d34740 1 -- 172.21.15.62:0/2096919763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f171000fe10 con 0x55671a32bb40 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f1708ff9640 1 -- 172.21.15.62:0/2096919763 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1718004d20 con 0x55671a32bb40 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f1729d34740 10 client.26515 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:14.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.922+0000 7f1729d34740 1 -- 172.21.15.62:0/2096919763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55671a343c40 con 0x55671a32bb40 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.923+0000 7f1708ff9640 1 -- 172.21.15.62:0/2096919763 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f171800fde0 con 0x55671a32bb40 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.923+0000 7f1708ff9640 1 -- 172.21.15.62:0/2096919763 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f1718018070 con 0x55671a32bb40 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.923+0000 7f1729d34740 10 client.26515 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.923+0000 7f1729d34740 10 client.26515 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.923+0000 7f1729d34740 20 client.26515 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.923+0000 7f1729d34740 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55671a345100 0x55671a3474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.923+0000 7f1729d34740 4 client.26515 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.923+0000 7f1729d34740 1 -- 172.21.15.62:0/2096919763 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f171000fe10 con 0x55671a345100 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.923+0000 7f1724ef6640 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55671a345100 0x55671a3474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.924+0000 7f1724ef6640 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55671a345100 0x55671a3474f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f03f68d8e4ff224c server_cookie=3eb6f112be3f2c1f in_seq=0 out_seq=0 2024-09-12T00:01:14.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.924+0000 7f1708ff9640 10 client.26515 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:01:14.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.925+0000 7f1708ff9640 1 -- 172.21.15.62:0/2096919763 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f171000fe10 con 0x55671a345100 2024-09-12T00:01:14.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.925+0000 7f1708ff9640 10 client.26515 handle_command_reply: tid=0 2024-09-12T00:01:14.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.925+0000 7f1729d34740 1 client.26515 shutdown 2024-09-12T00:01:14.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.925+0000 7f1729d34740 1 -- 172.21.15.62:0/2096919763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 msgr2=0x55671a3320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:14.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.925+0000 7f1729d34740 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a3320d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1718005fd0 tx=0x7f1718011dd0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.932+0000 7f1729d34740 1 -- 172.21.15.62:0/2096919763 shutdown_connections 2024-09-12T00:01:14.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.932+0000 7f1729d34740 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55671a345100 0x55671a3474f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.932+0000 7f1729d34740 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a332860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.932+0000 7f1729d34740 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a3320d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.932+0000 7f1729d34740 1 --2- 172.21.15.62:0/2096919763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a334e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.932+0000 7f1729d34740 1 -- 172.21.15.62:0/2096919763 >> 172.21.15.62:0/2096919763 conn(0x55671a2f3310 msgr2=0x55671a32c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:14.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.932+0000 7f1708ff9640 0 client.26515 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:01:14.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.932+0000 7f1729d34740 1 -- 172.21.15.62:0/2096919763 shutdown_connections 2024-09-12T00:01:14.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.937+0000 7f1729d34740 1 -- 172.21.15.62:0/2096919763 wait complete. 2024-09-12T00:01:14.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.939+0000 7f1729d34740 20 client.26515 trim_cache size 0 max 16384 2024-09-12T00:01:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.971+0000 7f1729d34740 1 Processor -- start 2024-09-12T00:01:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.972+0000 7f1729d34740 1 -- start start 2024-09-12T00:01:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.972+0000 7f1729d34740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a3320a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.972+0000 7f1729d34740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a331010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.972+0000 7f1729d34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a32d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.972+0000 7f1729d34740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f17201b67c0 con 0x55671a32a5d0 2024-09-12T00:01:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.972+0000 7f1729d34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17201b4090 con 0x55671a335370 2024-09-12T00:01:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.972+0000 7f1729d34740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17201b1f00 con 0x55671a32bb40 2024-09-12T00:01:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.974+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a32d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.974+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a3320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.974+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a3320a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49064/0 (socket says 172.21.15.62:49064) 2024-09-12T00:01:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.974+0000 7f1724ef6640 1 -- 172.21.15.62:0/3495109254 learned_addr learned my addr 172.21.15.62:0/3495109254 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.974+0000 7f17256f7640 1 --2- 172.21.15.62:0/3495109254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a331010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.975+0000 7f17256f7640 1 -- 172.21.15.62:0/3495109254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 msgr2=0x55671a32d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.975+0000 7f17256f7640 1 --2- 172.21.15.62:0/3495109254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a32d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.975+0000 7f17256f7640 1 -- 172.21.15.62:0/3495109254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 msgr2=0x55671a3320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.975+0000 7f17256f7640 1 --2- 172.21.15.62:0/3495109254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a3320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.975+0000 7f17256f7640 1 -- 172.21.15.62:0/3495109254 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55671a332bd0 con 0x55671a32bb40 2024-09-12T00:01:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.975+0000 7f17256f7640 1 --2- 172.21.15.62:0/3495109254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a331010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1718000f80 tx=0x7f171800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61546c90d5f6ff59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.976+0000 7f170affd640 1 -- 172.21.15.62:0/3495109254 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718005460 con 0x55671a32bb40 2024-09-12T00:01:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.976+0000 7f170affd640 1 -- 172.21.15.62:0/3495109254 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1718011040 con 0x55671a32bb40 2024-09-12T00:01:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.976+0000 7f170affd640 1 -- 172.21.15.62:0/3495109254 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718010430 con 0x55671a32bb40 2024-09-12T00:01:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.980+0000 7f1729d34740 1 -- 172.21.15.62:0/3495109254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 msgr2=0x55671a331010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.981+0000 7f1729d34740 1 --2- 172.21.15.62:0/3495109254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a331010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1718000f80 tx=0x7f171800f790 comp rx=0 tx=0).stop 2024-09-12T00:01:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.982+0000 7f1729d34740 1 -- 172.21.15.62:0/3495109254 shutdown_connections 2024-09-12T00:01:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.982+0000 7f1729d34740 1 --2- 172.21.15.62:0/3495109254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a335370 0x55671a32d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.982+0000 7f1729d34740 1 --2- 172.21.15.62:0/3495109254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a331010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.982+0000 7f1729d34740 1 --2- 172.21.15.62:0/3495109254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a32a5d0 0x55671a3320a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.982+0000 7f1729d34740 1 -- 172.21.15.62:0/3495109254 >> 172.21.15.62:0/3495109254 conn(0x55671a2f3310 msgr2=0x55671a331990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:14.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.983+0000 7f1729d34740 1 -- 172.21.15.62:0/3495109254 shutdown_connections 2024-09-12T00:01:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.987+0000 7f1729d34740 1 -- 172.21.15.62:0/3495109254 wait complete. 2024-09-12T00:01:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.987+0000 7f1729d34740 1 -- start start 2024-09-12T00:01:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.987+0000 7f1729d34740 1 Processor -- start 2024-09-12T00:01:14.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.988+0000 7f1729d34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a32a5d0 0x55671a3a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.988+0000 7f1729d34740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a3a8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.988+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a32a5d0 0x55671a3a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.989+0000 7f1724ef6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a32a5d0 0x55671a3a8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58330/0 (socket says 172.21.15.62:58330) 2024-09-12T00:01:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.989+0000 7f1724ef6640 1 -- 172.21.15.62:0/1425614926 learned_addr learned my addr 172.21.15.62:0/1425614926 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.989+0000 7f17256f7640 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a3a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.989+0000 7f1729d34740 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a335370 0x55671a33c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.989+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f17201b2550 con 0x55671a335370 2024-09-12T00:01:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.989+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17201b39c0 con 0x55671a32a5d0 2024-09-12T00:01:14.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.989+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17201b2c20 con 0x55671a32bb40 2024-09-12T00:01:14.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.989+0000 7f1729d34740 10 client.0 _ll_register_callbacks cb 0x7f1718009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.990+0000 7f17256f7640 1 -- 172.21.15.62:0/1425614926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a32a5d0 msgr2=0x55671a3a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.990+0000 7f17256f7640 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a32a5d0 0x55671a3a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.990+0000 7f17256f7640 1 -- 172.21.15.62:0/1425614926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a335370 msgr2=0x55671a33c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.990+0000 7f17256f7640 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a335370 0x55671a33c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.990+0000 7f17256f7640 1 -- 172.21.15.62:0/1425614926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55671a3a8f90 con 0x55671a32bb40 2024-09-12T00:01:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.991+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1718011040 con 0x55671a32bb40 2024-09-12T00:01:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.991+0000 7f17256f7640 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a3a8690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55671a3325e0 tx=0x7f1718011420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=632baee60521ee74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.991+0000 7f1708ff9640 10 client.26521 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:01:14.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.992+0000 7f1708ff9640 1 -- 172.21.15.62:0/1425614926 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171800fdd0 con 0x55671a32bb40 2024-09-12T00:01:14.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.992+0000 7f1708ff9640 1 -- 172.21.15.62:0/1425614926 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8557+0+0 (secure 0 0 0) 0x7f17180051e0 con 0x55671a32bb40 2024-09-12T00:01:14.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.992+0000 7f1708ff9640 1 -- 172.21.15.62:0/1425614926 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1718005400 con 0x55671a32bb40 2024-09-12T00:01:14.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.992+0000 7f1729d34740 10 client.26521 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:14.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.992+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55671a33da20 con 0x55671a32bb40 2024-09-12T00:01:14.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.993+0000 7f1708ff9640 1 -- 172.21.15.62:0/1425614926 <== mon.1 v2:172.21.15.120:3300/0 4 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f1718028070 con 0x55671a32bb40 2024-09-12T00:01:14.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.993+0000 7f1729d34740 10 client.26521 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:14.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.993+0000 7f1729d34740 10 client.26521 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:01:14.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.993+0000 7f1729d34740 20 client.26521 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:14.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.993+0000 7f1729d34740 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55671a3c0de0 0x55671a3c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:14.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.993+0000 7f1729d34740 4 client.26521 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:01:14.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.993+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55671a29d860 con 0x55671a3c0de0 2024-09-12T00:01:14.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.993+0000 7f1724ef6640 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55671a3c0de0 0x55671a3c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.994+0000 7f1724ef6640 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55671a3c0de0 0x55671a3c31d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=760cc16626a8dad1 server_cookie=17bafcff8b843a59 in_seq=0 out_seq=0 2024-09-12T00:01:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.994+0000 7f1708ff9640 10 client.26521 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:01:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.994+0000 7f1708ff9640 1 -- 172.21.15.62:0/1425614926 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55671a29d860 con 0x55671a3c0de0 2024-09-12T00:01:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.994+0000 7f1708ff9640 10 client.26521 handle_command_reply: tid=0 2024-09-12T00:01:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.994+0000 7f1729d34740 1 client.26521 shutdown 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 msgr2=0x55671a3a8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a3a8690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55671a3325e0 tx=0x7f1718011420 comp rx=0 tx=0).stop 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 shutdown_connections 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55671a3c0de0 0x55671a3c31d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671a335370 0x55671a33c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55671a32bb40 0x55671a3a8690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 1 --2- 172.21.15.62:0/1425614926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55671a32a5d0 0x55671a3a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 >> 172.21.15.62:0/1425614926 conn(0x55671a2f3310 msgr2=0x55671a333930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1708ff9640 0 client.26521 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 shutdown_connections 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 1 -- 172.21.15.62:0/1425614926 wait complete. 2024-09-12T00:01:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.995+0000 7f1729d34740 20 client.26521 trim_cache size 0 max 16384 2024-09-12T00:01:14.998 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.998+0000 7f172795b640 1 -- 172.21.15.62:0/3850940706 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f170c0a2dc0 msgr2=0x7f170c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.998+0000 7f172795b640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f170c0a2dc0 0x7f170c0a5280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f171c04a000 tx=0x7f171c03bab0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.998+0000 7f172795b640 1 -- 172.21.15.62:0/3850940706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172019e660 msgr2=0x7f172019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.998+0000 7f172795b640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172019e660 0x7f172019eae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f171c02aa00 tx=0x7f171c0305d0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.998+0000 7f172795b640 1 -- 172.21.15.62:0/3850940706 shutdown_connections 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.998+0000 7f172795b640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f170c0a2dc0 0x7f170c0a5280 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.998+0000 7f172795b640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172019e660 0x7f172019eae0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.998+0000 7f172795b640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1720106a90 0x7f17201a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.998+0000 7f172795b640 1 --2- 172.21.15.62:0/3850940706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1720100860 0x7f17201a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.998+0000 7f172795b640 1 -- 172.21.15.62:0/3850940706 >> 172.21.15.62:0/3850940706 conn(0x7f1720077230 msgr2=0x7f17201024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.999+0000 7f172795b640 1 -- 172.21.15.62:0/3850940706 shutdown_connections 2024-09-12T00:01:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:14.999+0000 7f172795b640 1 -- 172.21.15.62:0/3850940706 wait complete. 2024-09-12T00:01:15.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:15 smithi062 ceph-mon[30958]: pgmap v282: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-09-12T00:01:15.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:15 smithi120 ceph-mon[35182]: pgmap v282: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-09-12T00:01:15.648 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:01:15.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:15 smithi083 ceph-mon[34805]: pgmap v282: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-09-12T00:01:16.038 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:16.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.589+0000 7f76ae99f640 1 -- 172.21.15.62:0/1705444246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a8100e20 msgr2=0x7f76a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:16.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.589+0000 7f76ae99f640 1 --2- 172.21.15.62:0/1705444246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a8100e20 0x7f76a8101200 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7698009900 tx=0x7f769802a4a0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.589+0000 7f76ae99f640 1 -- 172.21.15.62:0/1705444246 shutdown_connections 2024-09-12T00:01:16.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.589+0000 7f76ae99f640 1 --2- 172.21.15.62:0/1705444246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a8102ef0 0x7f76a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.589+0000 7f76ae99f640 1 --2- 172.21.15.62:0/1705444246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76a8102570 0x7f76a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.589+0000 7f76ae99f640 1 --2- 172.21.15.62:0/1705444246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a8100e20 0x7f76a8101200 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.589+0000 7f76ae99f640 1 -- 172.21.15.62:0/1705444246 >> 172.21.15.62:0/1705444246 conn(0x7f76a80fc7d0 msgr2=0x7f76a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:16.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.589+0000 7f76ae99f640 1 -- 172.21.15.62:0/1705444246 shutdown_connections 2024-09-12T00:01:16.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.589+0000 7f76ae99f640 1 -- 172.21.15.62:0/1705444246 wait complete. 2024-09-12T00:01:16.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.590+0000 7f76ae99f640 1 Processor -- start 2024-09-12T00:01:16.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.590+0000 7f76ae99f640 1 -- start start 2024-09-12T00:01:16.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.590+0000 7f76ae99f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76a8100e20 0x7f76a81962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.590+0000 7f76ae99f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a8102570 0x7f76a8196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.590+0000 7f76ae99f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a8102ef0 0x7f76a819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.590+0000 7f76ae99f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76a810fe80 con 0x7f76a8102570 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.590+0000 7f76ae99f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76a810fd00 con 0x7f76a8102ef0 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.590+0000 7f76ae99f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8110000 con 0x7f76a8100e20 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a8102570 0x7f76a8196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a8102570 0x7f76a8196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49092/0 (socket says 172.21.15.62:49092) 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76a77fe640 1 -- 172.21.15.62:0/3592390028 learned_addr learned my addr 172.21.15.62:0/3592390028 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76acf3c640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a8102ef0 0x7f76a819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76a7fff640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76a8100e20 0x7f76a81962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76a77fe640 1 -- 172.21.15.62:0/3592390028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76a8100e20 msgr2=0x7f76a81962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76a77fe640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76a8100e20 0x7f76a81962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76a77fe640 1 -- 172.21.15.62:0/3592390028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a8102ef0 msgr2=0x7f76a819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:16.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76a77fe640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a8102ef0 0x7f76a819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76a77fe640 1 -- 172.21.15.62:0/3592390028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a819b290 con 0x7f76a8102570 2024-09-12T00:01:16.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76a7fff640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76a8100e20 0x7f76a81962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:01:16.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.591+0000 7f76acf3c640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a8102ef0 0x7f76a819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:16.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.592+0000 7f76a77fe640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a8102570 0x7f76a8196820 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f769c00daf0 tx=0x7f769c00af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:16.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.592+0000 7f76a57fa640 1 -- 172.21.15.62:0/3592390028 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c01a070 con 0x7f76a8102570 2024-09-12T00:01:16.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.592+0000 7f76ae99f640 1 -- 172.21.15.62:0/3592390028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a819b580 con 0x7f76a8102570 2024-09-12T00:01:16.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.592+0000 7f76a57fa640 1 -- 172.21.15.62:0/3592390028 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f769c00cc70 con 0x7f76a8102570 2024-09-12T00:01:16.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.592+0000 7f76ae99f640 1 -- 172.21.15.62:0/3592390028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81a2e60 con 0x7f76a8102570 2024-09-12T00:01:16.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.592+0000 7f76a57fa640 1 -- 172.21.15.62:0/3592390028 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c01a070 con 0x7f76a8102570 2024-09-12T00:01:16.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.596+0000 7f76a57fa640 1 -- 172.21.15.62:0/3592390028 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f769c01d070 con 0x7f76a8102570 2024-09-12T00:01:16.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.596+0000 7f76b0d78740 1 Processor -- start 2024-09-12T00:01:16.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.597+0000 7f76a57fa640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76900a3130 0x7f76900a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.597+0000 7f76acf3c640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76900a3130 0x7f76900a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.597+0000 7f76a57fa640 1 -- 172.21.15.62:0/3592390028 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f769c012070 con 0x7f76a8102570 2024-09-12T00:01:16.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.597+0000 7f76b0d78740 1 -- start start 2024-09-12T00:01:16.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 0x55baec7a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a85d0 0x55baec7a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76b0d78740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55baec70e2f0 con 0x55baec7a9b40 2024-09-12T00:01:16.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76b0d78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55baec6b5cf0 con 0x55baec7a85d0 2024-09-12T00:01:16.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76b0d78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55baec70a2b0 con 0x55baec7b3370 2024-09-12T00:01:16.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76acf3c640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76900a3130 0x7f76900a55f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005fd0 tx=0x7f76a0016040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:16.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a85d0 0x55baec7a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a85d0 0x55baec7a8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58356/0 (socket says 172.21.15.62:58356) 2024-09-12T00:01:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76a77fe640 1 -- 172.21.15.62:0/65520277 learned_addr learned my addr 172.21.15.62:0/65520277 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76a7fff640 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 0x55baec7a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76acf3c640 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76a77fe640 1 -- 172.21.15.62:0/65520277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 msgr2=0x55baec7b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76a77fe640 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.598+0000 7f76a77fe640 1 -- 172.21.15.62:0/65520277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 msgr2=0x55baec7a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.599+0000 7f76a77fe640 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 0x55baec7a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.599+0000 7f76a77fe640 1 -- 172.21.15.62:0/65520277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baec69a3a0 con 0x55baec7a85d0 2024-09-12T00:01:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.599+0000 7f76acf3c640 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:01:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.599+0000 7f76a7fff640 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 0x55baec7a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:16.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.599+0000 7f76a77fe640 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a85d0 0x55baec7a8a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f769c098390 tx=0x7f769c09fe00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f62292a6d726048 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:16.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.599+0000 7f768e7fc640 1 -- 172.21.15.62:0/65520277 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c0b4070 con 0x55baec7a85d0 2024-09-12T00:01:16.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.599+0000 7f768e7fc640 1 -- 172.21.15.62:0/65520277 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f769c0ccdc0 con 0x55baec7a85d0 2024-09-12T00:01:16.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.599+0000 7f768e7fc640 1 -- 172.21.15.62:0/65520277 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c01a050 con 0x55baec7a85d0 2024-09-12T00:01:16.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.600+0000 7f76b0d78740 1 -- 172.21.15.62:0/65520277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a85d0 msgr2=0x55baec7a8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.600+0000 7f76b0d78740 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a85d0 0x55baec7a8a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f769c098390 tx=0x7f769c09fe00 comp rx=0 tx=0).stop 2024-09-12T00:01:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.601+0000 7f76b0d78740 1 -- 172.21.15.62:0/65520277 shutdown_connections 2024-09-12T00:01:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.601+0000 7f76b0d78740 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.601+0000 7f76b0d78740 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a85d0 0x55baec7a8a10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.601+0000 7f76b0d78740 1 --2- 172.21.15.62:0/65520277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 0x55baec7a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.601+0000 7f76b0d78740 1 -- 172.21.15.62:0/65520277 >> 172.21.15.62:0/65520277 conn(0x55baec771310 msgr2=0x55baec7a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.601+0000 7f76b0d78740 1 -- 172.21.15.62:0/65520277 shutdown_connections 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.601+0000 7f76b0d78740 1 -- 172.21.15.62:0/65520277 wait complete. 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.601+0000 7f76b0d78740 1 -- start start 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.601+0000 7f76b0d78740 1 Processor -- start 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 0x55baec7b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76b0d78740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55baec7b6070 con 0x55baec7a85d0 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76b0d78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55baec6c9060 con 0x55baec7a9b40 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 0x55baec7b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 0x55baec7b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49106/0 (socket says 172.21.15.62:49106) 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76a7fff640 1 -- 172.21.15.62:0/408072654 learned_addr learned my addr 172.21.15.62:0/408072654 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76a77fe640 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76b0d78740 1 -- 172.21.15.62:0/408072654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55baec7b61f0 con 0x55baec7b3370 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76acf3c640 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76a77fe640 1 -- 172.21.15.62:0/408072654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 msgr2=0x55baec7b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76a77fe640 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76a77fe640 1 -- 172.21.15.62:0/408072654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 msgr2=0x55baec7b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76a77fe640 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 0x55baec7b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.603+0000 7f76a77fe640 1 -- 172.21.15.62:0/408072654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baec47f3c0 con 0x55baec7a9b40 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.602+0000 7f76b0d78740 10 client.0 _ll_register_callbacks cb 0x55baec41c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.603+0000 7f76b0d78740 1 -- 172.21.15.62:0/408072654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f769c0ccdc0 con 0x55baec7a9b40 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.603+0000 7f76a77fe640 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b0070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f769c097cd0 tx=0x7f769c09b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1731ed2440c8937f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.603+0000 7f76877fe640 10 client.26600 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.603+0000 7f76877fe640 1 -- 172.21.15.62:0/408072654 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c0b8020 con 0x55baec7a9b40 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.603+0000 7f76877fe640 1 -- 172.21.15.62:0/408072654 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f769c0ba2c0 con 0x55baec7a9b40 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.603+0000 7f76877fe640 1 -- 172.21.15.62:0/408072654 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f769c0b4050 con 0x55baec7a9b40 2024-09-12T00:01:16.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.603+0000 7f76b0d78740 10 client.26600 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.603+0000 7f76b0d78740 1 -- 172.21.15.62:0/408072654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55baec7b6070 con 0x55baec7a9b40 2024-09-12T00:01:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.604+0000 7f76877fe640 1 -- 172.21.15.62:0/408072654 <== mon.2 v2:172.21.15.83:3300/0 4 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f769c097710 con 0x55baec7a9b40 2024-09-12T00:01:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.604+0000 7f76b0d78740 10 client.26600 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.604+0000 7f76b0d78740 10 client.26600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:01:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.604+0000 7f76b0d78740 20 client.26600 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.604+0000 7f76b0d78740 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55baec7c2ba0 0x55baec7c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.604+0000 7f76b0d78740 4 client.26600 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:01:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.604+0000 7f76a7fff640 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55baec7c2ba0 0x55baec7c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.605+0000 7f76b0d78740 1 -- 172.21.15.62:0/408072654 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baec71b860 con 0x55baec7c2ba0 2024-09-12T00:01:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.605+0000 7f76a7fff640 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55baec7c2ba0 0x55baec7c4f90 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969d50c589d0cfa0 server_cookie=8525f4af69309bb4 in_seq=0 out_seq=0 2024-09-12T00:01:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.605+0000 7f76877fe640 10 client.26600 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:16.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76877fe640 1 -- 172.21.15.62:0/408072654 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55baec71b860 con 0x55baec7c2ba0 2024-09-12T00:01:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76877fe640 10 client.26600 handle_command_reply: tid=0 2024-09-12T00:01:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76b0d78740 1 client.26600 shutdown 2024-09-12T00:01:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76b0d78740 1 -- 172.21.15.62:0/408072654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 msgr2=0x55baec7b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76b0d78740 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b0070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f769c097cd0 tx=0x7f769c09b040 comp rx=0 tx=0).stop 2024-09-12T00:01:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76b0d78740 1 -- 172.21.15.62:0/408072654 shutdown_connections 2024-09-12T00:01:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76b0d78740 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55baec7c2ba0 0x55baec7c4f90 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76b0d78740 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76b0d78740 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b0070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76b0d78740 1 --2- 172.21.15.62:0/408072654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 0x55baec7b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76b0d78740 1 -- 172.21.15.62:0/408072654 >> 172.21.15.62:0/408072654 conn(0x55baec771310 msgr2=0x55baec7aa930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:16.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.606+0000 7f76877fe640 0 client.26600 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:16.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.607+0000 7f76b0d78740 1 -- 172.21.15.62:0/408072654 shutdown_connections 2024-09-12T00:01:16.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.607+0000 7f76b0d78740 1 -- 172.21.15.62:0/408072654 wait complete. 2024-09-12T00:01:16.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.607+0000 7f76b0d78740 20 client.26600 trim_cache size 0 max 16384 2024-09-12T00:01:16.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.625+0000 7f76b0d78740 1 Processor -- start 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76b0d78740 1 -- start start 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7a85d0 0x55baec7af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 0x55baec7ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7b3370 0x55baec7cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76b0d78740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7694010f80 con 0x55baec7a9b40 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76b0d78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f769400e850 con 0x55baec7b3370 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76b0d78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f769400c6a0 con 0x55baec7a85d0 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 0x55baec7ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76acf3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7b3370 0x55baec7cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76acf3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7b3370 0x55baec7cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58380/0 (socket says 172.21.15.62:58380) 2024-09-12T00:01:16.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76acf3c640 1 -- 172.21.15.62:0/3636638309 learned_addr learned my addr 172.21.15.62:0/3636638309 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76a7fff640 1 --2- 172.21.15.62:0/3636638309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7a85d0 0x55baec7af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76acf3c640 1 -- 172.21.15.62:0/3636638309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7a85d0 msgr2=0x55baec7af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76acf3c640 1 --2- 172.21.15.62:0/3636638309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7a85d0 0x55baec7af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76acf3c640 1 -- 172.21.15.62:0/3636638309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 msgr2=0x55baec7ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76acf3c640 1 --2- 172.21.15.62:0/3636638309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 0x55baec7ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76acf3c640 1 -- 172.21.15.62:0/3636638309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baec7ae5c0 con 0x55baec7b3370 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76a7fff640 1 --2- 172.21.15.62:0/3636638309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7a85d0 0x55baec7af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.626+0000 7f76a77fe640 1 --2- 172.21.15.62:0/3636638309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 0x55baec7ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.627+0000 7f76acf3c640 1 --2- 172.21.15.62:0/3636638309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7b3370 0x55baec7cac10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f76a0003d20 tx=0x7f76a001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c76cc8b9d02c4f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.627+0000 7f768dffb640 1 -- 172.21.15.62:0/3636638309 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0007d10 con 0x55baec7b3370 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.627+0000 7f768dffb640 1 -- 172.21.15.62:0/3636638309 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a0034070 con 0x55baec7b3370 2024-09-12T00:01:16.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.627+0000 7f768dffb640 1 -- 172.21.15.62:0/3636638309 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a001fcd0 con 0x55baec7b3370 2024-09-12T00:01:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.628+0000 7f76b0d78740 1 -- 172.21.15.62:0/3636638309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7b3370 msgr2=0x55baec7cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.628+0000 7f76b0d78740 1 --2- 172.21.15.62:0/3636638309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7b3370 0x55baec7cac10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f76a0003d20 tx=0x7f76a001d000 comp rx=0 tx=0).stop 2024-09-12T00:01:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.628+0000 7f76b0d78740 1 -- 172.21.15.62:0/3636638309 shutdown_connections 2024-09-12T00:01:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.628+0000 7f76b0d78740 1 --2- 172.21.15.62:0/3636638309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7b3370 0x55baec7cac10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.628+0000 7f76b0d78740 1 --2- 172.21.15.62:0/3636638309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a9b40 0x55baec7ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.629+0000 7f76b0d78740 1 --2- 172.21.15.62:0/3636638309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7a85d0 0x55baec7af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.629+0000 7f76b0d78740 1 -- 172.21.15.62:0/3636638309 >> 172.21.15.62:0/3636638309 conn(0x55baec771310 msgr2=0x55baec7ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.629+0000 7f76b0d78740 1 -- 172.21.15.62:0/3636638309 shutdown_connections 2024-09-12T00:01:16.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.629+0000 7f76b0d78740 1 -- 172.21.15.62:0/3636638309 wait complete. 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.629+0000 7f76b0d78740 1 -- start start 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76b0d78740 1 Processor -- start 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 0x55baec7b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76b0d78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76b0d78740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f769400cd10 con 0x55baec7a85d0 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76b0d78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f769400e180 con 0x55baec7a9b40 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76b0d78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f769400d3e0 con 0x55baec7b3370 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76b0d78740 10 client.0 _ll_register_callbacks cb 0x7f769c0b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58396/0 (socket says 172.21.15.62:58396) 2024-09-12T00:01:16.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76a77fe640 1 -- 172.21.15.62:0/3961390018 learned_addr learned my addr 172.21.15.62:0/3961390018 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76a7fff640 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 0x55baec7b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.630+0000 7f76acf3c640 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76a77fe640 1 -- 172.21.15.62:0/3961390018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 msgr2=0x55baec7b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76a77fe640 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76a77fe640 1 -- 172.21.15.62:0/3961390018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 msgr2=0x55baec7b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76a77fe640 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 0x55baec7b72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76a77fe640 1 -- 172.21.15.62:0/3961390018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baec82c750 con 0x55baec7a9b40 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76a77fe640 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b7830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f769c0a4740 tx=0x7f769c097540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb1803ba9f0951d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76877fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76877fe640 1 -- 172.21.15.62:0/3961390018 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c0b8040 con 0x55baec7a9b40 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76877fe640 1 -- 172.21.15.62:0/3961390018 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f769c0b4b10 con 0x55baec7a9b40 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76b0d78740 1 -- 172.21.15.62:0/3961390018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76a0034070 con 0x55baec7a9b40 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76877fe640 1 -- 172.21.15.62:0/3961390018 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f769c01a040 con 0x55baec7a9b40 2024-09-12T00:01:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76b0d78740 10 client.26606 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:16.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76b0d78740 1 -- 172.21.15.62:0/3961390018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55baec82c750 con 0x55baec7a9b40 2024-09-12T00:01:16.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.631+0000 7f76877fe640 1 -- 172.21.15.62:0/3961390018 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a0034070 con 0x55baec7a9b40 2024-09-12T00:01:16.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.632+0000 7f76877fe640 1 -- 172.21.15.62:0/3961390018 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f769c0a0e50 con 0x55baec7a9b40 2024-09-12T00:01:16.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.632+0000 7f76b0d78740 10 client.26606 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:16.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.632+0000 7f76b0d78740 10 client.26606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:01:16.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.632+0000 7f76b0d78740 20 client.26606 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:16.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.632+0000 7f76b0d78740 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55baec82ded0 0x55baec8302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:16.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.632+0000 7f76b0d78740 4 client.26606 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:01:16.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.632+0000 7f76b0d78740 1 -- 172.21.15.62:0/3961390018 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55baec715d70 con 0x55baec82ded0 2024-09-12T00:01:16.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.632+0000 7f76a7fff640 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55baec82ded0 0x55baec8302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:16.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.634+0000 7f76a7fff640 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55baec82ded0 0x55baec8302f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81be41fddcfcfd3 server_cookie=464c3d8d558a4076 in_seq=0 out_seq=0 2024-09-12T00:01:16.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.634+0000 7f76877fe640 10 client.26606 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:16.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.634+0000 7f76877fe640 1 -- 172.21.15.62:0/3961390018 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55baec715d70 con 0x55baec82ded0 2024-09-12T00:01:16.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.634+0000 7f76877fe640 10 client.26606 handle_command_reply: tid=0 2024-09-12T00:01:16.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.634+0000 7f76b0d78740 1 client.26606 shutdown 2024-09-12T00:01:16.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.635+0000 7f76b0d78740 1 -- 172.21.15.62:0/3961390018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 msgr2=0x55baec7b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:16.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.635+0000 7f76b0d78740 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b7830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f769c0a4740 tx=0x7f769c097540 comp rx=0 tx=0).stop 2024-09-12T00:01:16.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.635+0000 7f76b0d78740 1 -- 172.21.15.62:0/3961390018 shutdown_connections 2024-09-12T00:01:16.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.635+0000 7f76b0d78740 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55baec82ded0 0x55baec8302f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.635+0000 7f76b0d78740 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55baec7b3370 0x55baec7b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.635+0000 7f76877fe640 0 client.26606 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.635+0000 7f76b0d78740 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55baec7a9b40 0x55baec7b7830 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.635+0000 7f76b0d78740 1 --2- 172.21.15.62:0/3961390018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baec7a85d0 0x55baec7b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.635+0000 7f76b0d78740 1 -- 172.21.15.62:0/3961390018 >> 172.21.15.62:0/3961390018 conn(0x55baec771310 msgr2=0x55baec7af160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.635+0000 7f76b0d78740 1 -- 172.21.15.62:0/3961390018 shutdown_connections 2024-09-12T00:01:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.636+0000 7f76b0d78740 1 -- 172.21.15.62:0/3961390018 wait complete. 2024-09-12T00:01:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.638+0000 7f76b0d78740 20 client.26606 trim_cache size 0 max 16384 2024-09-12T00:01:16.639 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:01:16.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.643+0000 7f76ae99f640 1 -- 172.21.15.62:0/3592390028 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76900a3130 msgr2=0x7f76900a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:16.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.643+0000 7f76ae99f640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76900a3130 0x7f76900a55f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005fd0 tx=0x7f76a0016040 comp rx=0 tx=0).stop 2024-09-12T00:01:16.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.643+0000 7f76ae99f640 1 -- 172.21.15.62:0/3592390028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a8102570 msgr2=0x7f76a8196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:16.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.643+0000 7f76ae99f640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a8102570 0x7f76a8196820 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f769c00daf0 tx=0x7f769c00af60 comp rx=0 tx=0).stop 2024-09-12T00:01:16.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.646+0000 7f76ae99f640 1 -- 172.21.15.62:0/3592390028 shutdown_connections 2024-09-12T00:01:16.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.646+0000 7f76ae99f640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f76900a3130 0x7f76900a55f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.646+0000 7f76ae99f640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a8102ef0 0x7f76a819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.646+0000 7f76ae99f640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a8102570 0x7f76a8196820 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.646+0000 7f76ae99f640 1 --2- 172.21.15.62:0/3592390028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76a8100e20 0x7f76a81962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:16.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.646+0000 7f76ae99f640 1 -- 172.21.15.62:0/3592390028 >> 172.21.15.62:0/3592390028 conn(0x7f76a80fc7d0 msgr2=0x7f76a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:16.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.649+0000 7f76ae99f640 1 -- 172.21.15.62:0/3592390028 shutdown_connections 2024-09-12T00:01:16.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:16.649+0000 7f76ae99f640 1 -- 172.21.15.62:0/3592390028 wait complete. 2024-09-12T00:01:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 515.0 s, 2543.4 tps, lat 39.811 ms stddev 42.826 2024-09-12T00:01:17.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:17 smithi120 ceph-mon[35182]: pgmap v283: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-09-12T00:01:17.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:17 smithi120 ceph-mon[35182]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T00:01:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:17 smithi083 ceph-mon[34805]: pgmap v283: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-09-12T00:01:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:17 smithi083 ceph-mon[34805]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T00:01:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:17 smithi062 ceph-mon[30958]: pgmap v283: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-09-12T00:01:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:17 smithi062 ceph-mon[30958]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T00:01:18.387 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:01:18.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:01:18.732 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:19.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.276+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2602000493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd194106f10 msgr2=0x7fd19410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:19.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.277+0000 7fd19a9d2640 1 --2- 172.21.15.62:0/2602000493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd194106f10 0x7fd19410ea90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009ca0 tx=0x7fd18c02a8c0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.277+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2602000493 shutdown_connections 2024-09-12T00:01:19.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.277+0000 7fd19a9d2640 1 --2- 172.21.15.62:0/2602000493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd194106f10 0x7fd19410ea90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.277+0000 7fd19a9d2640 1 --2- 172.21.15.62:0/2602000493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd194102790 0x7fd1941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.277+0000 7fd19a9d2640 1 --2- 172.21.15.62:0/2602000493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd194101de0 0x7fd1941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.277+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2602000493 >> 172.21.15.62:0/2602000493 conn(0x7fd1940fd8c0 msgr2=0x7fd1940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:19.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.277+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2602000493 shutdown_connections 2024-09-12T00:01:19.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.277+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2602000493 wait complete. 2024-09-12T00:01:19.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd19a9d2640 1 Processor -- start 2024-09-12T00:01:19.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd19a9d2640 1 -- start start 2024-09-12T00:01:19.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd19a9d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd194101de0 0x7fd1940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd19a9d2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd194102790 0x7fd194074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd19a9d2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd194106f10 0x7fd194074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd19a9d2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd194111f70 con 0x7fd194102790 2024-09-12T00:01:19.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd19a9d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd194111df0 con 0x7fd194101de0 2024-09-12T00:01:19.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd19a9d2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1941120f0 con 0x7fd194106f10 2024-09-12T00:01:19.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd193fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd194101de0 0x7fd1940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd193fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd194101de0 0x7fd1940744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58412/0 (socket says 172.21.15.62:58412) 2024-09-12T00:01:19.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.278+0000 7fd193fff640 1 -- 172.21.15.62:0/2858877049 learned_addr learned my addr 172.21.15.62:0/2858877049 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:19.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.279+0000 7fd18bfff640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd194102790 0x7fd194074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.279+0000 7fd198f6f640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd194106f10 0x7fd194074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.279+0000 7fd193fff640 1 -- 172.21.15.62:0/2858877049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd194106f10 msgr2=0x7fd194074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.279+0000 7fd193fff640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd194106f10 0x7fd194074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.279+0000 7fd193fff640 1 -- 172.21.15.62:0/2858877049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd194102790 msgr2=0x7fd194074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.279+0000 7fd193fff640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd194102790 0x7fd194074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.279+0000 7fd193fff640 1 -- 172.21.15.62:0/2858877049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1941aad60 con 0x7fd194101de0 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.279+0000 7fd193fff640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd194101de0 0x7fd1940744f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd18400e9f0 tx=0x7fd18400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.280+0000 7fd191ffb640 1 -- 172.21.15.62:0/2858877049 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18400cd80 con 0x7fd194101de0 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.280+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2858877049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19419c580 con 0x7fd194101de0 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.280+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2858877049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19419cb10 con 0x7fd194101de0 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.280+0000 7fd191ffb640 1 -- 172.21.15.62:0/2858877049 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd184029910 con 0x7fd194101de0 2024-09-12T00:01:19.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.281+0000 7fd191ffb640 1 -- 172.21.15.62:0/2858877049 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184010960 con 0x7fd194101de0 2024-09-12T00:01:19.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.283+0000 7fd19cdab740 1 Processor -- start 2024-09-12T00:01:19.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.283+0000 7fd19cdab740 1 -- start start 2024-09-12T00:01:19.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a57b60 0x55b341a57f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a58480 0x55b341a62800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a65130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd19cdab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b3419bd280 con 0x55b341a58480 2024-09-12T00:01:19.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd19cdab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b341964b10 con 0x55b341a62d40 2024-09-12T00:01:19.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd19cdab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b3419b9240 con 0x55b341a57b60 2024-09-12T00:01:19.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd198f6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a57b60 0x55b341a57f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd198f6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a57b60 0x55b341a57f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:39830/0 (socket says 172.21.15.62:39830) 2024-09-12T00:01:19.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd198f6f640 1 -- 172.21.15.62:0/2805588991 learned_addr learned my addr 172.21.15.62:0/2805588991 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:19.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd18bfff640 1 --2- 172.21.15.62:0/2805588991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a65130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd191ffb640 1 -- 172.21.15.62:0/2858877049 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd184029430 con 0x7fd194101de0 2024-09-12T00:01:19.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd193fff640 1 --2- 172.21.15.62:0/2805588991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a58480 0x55b341a62800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd198f6f640 1 -- 172.21.15.62:0/2805588991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 msgr2=0x55b341a65130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:19.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd198f6f640 1 --2- 172.21.15.62:0/2805588991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a65130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd198f6f640 1 -- 172.21.15.62:0/2805588991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a58480 msgr2=0x55b341a62800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:19.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd198f6f640 1 --2- 172.21.15.62:0/2805588991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a58480 0x55b341a62800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd198f6f640 1 -- 172.21.15.62:0/2805588991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3419711f0 con 0x55b341a57b60 2024-09-12T00:01:19.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.284+0000 7fd193fff640 1 --2- 172.21.15.62:0/2805588991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a58480 0x55b341a62800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:01:19.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.285+0000 7fd198f6f640 1 --2- 172.21.15.62:0/2805588991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a57b60 0x55b341a57f40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0093e0 tx=0x7fd18c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=742dd1a10785baf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:19.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.285+0000 7fd18a7fc640 1 -- 172.21.15.62:0/2805588991 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c003b70 con 0x55b341a57b60 2024-09-12T00:01:19.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.285+0000 7fd18a7fc640 1 -- 172.21.15.62:0/2805588991 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18c036040 con 0x55b341a57b60 2024-09-12T00:01:19.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.285+0000 7fd191ffb640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1800a2df0 0x7fd1800a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.285+0000 7fd18a7fc640 1 -- 172.21.15.62:0/2805588991 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c004680 con 0x55b341a57b60 2024-09-12T00:01:19.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.285+0000 7fd193fff640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1800a2df0 0x7fd1800a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd191ffb640 1 -- 172.21.15.62:0/2858877049 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fd1840e0840 con 0x7fd194101de0 2024-09-12T00:01:19.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd19cdab740 1 -- 172.21.15.62:0/2805588991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a57b60 msgr2=0x55b341a57f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:19.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2805588991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a57b60 0x55b341a57f40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0093e0 tx=0x7fd18c005bd0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd19cdab740 1 -- 172.21.15.62:0/2805588991 shutdown_connections 2024-09-12T00:01:19.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2805588991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a65130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2805588991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a58480 0x55b341a62800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2805588991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a57b60 0x55b341a57f40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd19cdab740 1 -- 172.21.15.62:0/2805588991 >> 172.21.15.62:0/2805588991 conn(0x55b341a540f0 msgr2=0x55b341a544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:19.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd19cdab740 1 -- 172.21.15.62:0/2805588991 shutdown_connections 2024-09-12T00:01:19.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd19cdab740 1 -- 172.21.15.62:0/2805588991 wait complete. 2024-09-12T00:01:19.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd193fff640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1800a2df0 0x7fd1800a52b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55b341a54f50 tx=0x7fd1840779f0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:19.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.286+0000 7fd19cdab740 1 -- start start 2024-09-12T00:01:19.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.287+0000 7fd19cdab740 1 Processor -- start 2024-09-12T00:01:19.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.287+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a62750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.287+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a5faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.288+0000 7fd18bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a5faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.288+0000 7fd18bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a5faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:39842/0 (socket says 172.21.15.62:39842) 2024-09-12T00:01:19.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.288+0000 7fd18bfff640 1 -- 172.21.15.62:0/2581610186 learned_addr learned my addr 172.21.15.62:0/2581610186 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:19.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.288+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.288+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b341a65960 con 0x55b341a57b60 2024-09-12T00:01:19.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.288+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b341977f10 con 0x55b341a62d40 2024-09-12T00:01:19.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.288+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b341a65ae0 con 0x55b341a58480 2024-09-12T00:01:19.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.288+0000 7fd19cdab740 10 client.0 _ll_register_callbacks cb 0x55b341940a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:19.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd198f6f640 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a62750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd198f6f640 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd18bfff640 1 -- 172.21.15.62:0/2581610186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 msgr2=0x55b341a600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:19.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd18bfff640 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd18bfff640 1 -- 172.21.15.62:0/2581610186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 msgr2=0x55b341a62750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:19.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd18bfff640 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a62750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd18bfff640 1 -- 172.21.15.62:0/2581610186 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b34172e6a0 con 0x55b341a58480 2024-09-12T00:01:19.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd198f6f640 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:01:19.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd18bfff640 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a5faa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd17c012700 tx=0x7fd17c012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bf798c4c8c121e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:19.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd1737fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:01:19.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd1737fe640 1 -- 172.21.15.62:0/2581610186 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c004050 con 0x55b341a58480 2024-09-12T00:01:19.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18c036040 con 0x55b341a58480 2024-09-12T00:01:19.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.289+0000 7fd1737fe640 1 -- 172.21.15.62:0/2581610186 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fd17c0041f0 con 0x55b341a58480 2024-09-12T00:01:19.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd1737fe640 1 -- 172.21.15.62:0/2581610186 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd17c0043f0 con 0x55b341a58480 2024-09-12T00:01:19.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd19cdab740 10 client.26551 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:19.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b341a71520 con 0x55b341a58480 2024-09-12T00:01:19.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd1737fe640 1 -- 172.21.15.62:0/2581610186 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18c036040 con 0x55b341a58480 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd1737fe640 1 -- 172.21.15.62:0/2581610186 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fd17c003bf0 con 0x55b341a58480 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd19cdab740 10 client.26551 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd19cdab740 10 client.26551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd19cdab740 20 client.26551 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b341a72930 0x55b341a74d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd19cdab740 4 client.26551 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3419ca7f0 con 0x55b341a72930 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.290+0000 7fd198f6f640 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b341a72930 0x55b341a74d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.291+0000 7fd198f6f640 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b341a72930 0x55b341a74d80 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a2136bf0cc672 server_cookie=a653504b362e4e7a in_seq=0 out_seq=0 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.291+0000 7fd1737fe640 10 client.26551 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd1737fe640 1 -- 172.21.15.62:0/2581610186 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3419ca7f0 con 0x55b341a72930 2024-09-12T00:01:19.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd1737fe640 10 client.26551 handle_command_reply: tid=0 2024-09-12T00:01:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 client.26551 shutdown 2024-09-12T00:01:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 msgr2=0x55b341a5faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a5faa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd17c012700 tx=0x7fd17c012bc0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 shutdown_connections 2024-09-12T00:01:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b341a72930 0x55b341a74d80 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a5faa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 --2- 172.21.15.62:0/2581610186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a62750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 >> 172.21.15.62:0/2581610186 conn(0x55b341a540f0 msgr2=0x55b341a59e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:19.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd1737fe640 0 client.26551 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:19.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 shutdown_connections 2024-09-12T00:01:19.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 1 -- 172.21.15.62:0/2581610186 wait complete. 2024-09-12T00:01:19.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.292+0000 7fd19cdab740 20 client.26551 trim_cache size 0 max 16384 2024-09-12T00:01:19.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.318+0000 7fd19cdab740 1 Processor -- start 2024-09-12T00:01:19.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.318+0000 7fd19cdab740 1 -- start start 2024-09-12T00:01:19.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a7a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a7abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd19cdab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd174011020 con 0x55b341a57b60 2024-09-12T00:01:19.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd19cdab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd17400e8f0 con 0x55b341a62d40 2024-09-12T00:01:19.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd19cdab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd17400c760 con 0x55b341a58480 2024-09-12T00:01:19.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd198f6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd198f6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49168/0 (socket says 172.21.15.62:49168) 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd198f6f640 1 -- 172.21.15.62:0/231270450 learned_addr learned my addr 172.21.15.62:0/231270450 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd18bfff640 1 --2- 172.21.15.62:0/231270450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a7a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd198f6f640 1 --2- 172.21.15.62:0/231270450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a7abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd18bfff640 1 -- 172.21.15.62:0/231270450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 msgr2=0x55b341a7abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd18bfff640 1 --2- 172.21.15.62:0/231270450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a7abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd18bfff640 1 -- 172.21.15.62:0/231270450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 msgr2=0x55b341a6eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd18bfff640 1 --2- 172.21.15.62:0/231270450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd18bfff640 1 -- 172.21.15.62:0/231270450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b341a60420 con 0x55b341a58480 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.319+0000 7fd198f6f640 1 --2- 172.21.15.62:0/231270450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.320+0000 7fd18bfff640 1 --2- 172.21.15.62:0/231270450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a7a690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0126d0 tx=0x7fd17c0108f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b8d20deaea00890 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.320+0000 7fd189ffb640 1 -- 172.21.15.62:0/231270450 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c032070 con 0x55b341a58480 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.320+0000 7fd189ffb640 1 -- 172.21.15.62:0/231270450 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c003fe0 con 0x55b341a58480 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.320+0000 7fd189ffb640 1 -- 172.21.15.62:0/231270450 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c029040 con 0x55b341a58480 2024-09-12T00:01:19.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.320+0000 7fd19cdab740 1 -- 172.21.15.62:0/231270450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 msgr2=0x55b341a7a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.320+0000 7fd19cdab740 1 --2- 172.21.15.62:0/231270450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a7a690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0126d0 tx=0x7fd17c0108f0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.321+0000 7fd19cdab740 1 -- 172.21.15.62:0/231270450 shutdown_connections 2024-09-12T00:01:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.321+0000 7fd19cdab740 1 --2- 172.21.15.62:0/231270450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a62d40 0x55b341a7abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.321+0000 7fd19cdab740 1 --2- 172.21.15.62:0/231270450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a58480 0x55b341a7a690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.321+0000 7fd19cdab740 1 --2- 172.21.15.62:0/231270450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.321+0000 7fd19cdab740 1 -- 172.21.15.62:0/231270450 >> 172.21.15.62:0/231270450 conn(0x55b341a540f0 msgr2=0x55b341a5e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.321+0000 7fd19cdab740 1 -- 172.21.15.62:0/231270450 shutdown_connections 2024-09-12T00:01:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.321+0000 7fd19cdab740 1 -- 172.21.15.62:0/231270450 wait complete. 2024-09-12T00:01:19.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.321+0000 7fd19cdab740 1 -- start start 2024-09-12T00:01:19.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.321+0000 7fd19cdab740 1 Processor -- start 2024-09-12T00:01:19.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a58480 0x55b341a6b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd19cdab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a62d40 0x55b341a5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd19cdab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd17400cdb0 con 0x55b341a57b60 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd19cdab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd17400e220 con 0x55b341a58480 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd19cdab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd17400d480 con 0x55b341a62d40 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd19cdab740 10 client.0 _ll_register_callbacks cb 0x7fd17c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd198f6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd198f6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49174/0 (socket says 172.21.15.62:49174) 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd198f6f640 1 -- 172.21.15.62:0/184667335 learned_addr learned my addr 172.21.15.62:0/184667335 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd18bfff640 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a58480 0x55b341a6b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd198f6f640 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a62d40 0x55b341a5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd198f6f640 1 -- 172.21.15.62:0/184667335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a58480 msgr2=0x55b341a6b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd198f6f640 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a58480 0x55b341a6b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd198f6f640 1 -- 172.21.15.62:0/184667335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 msgr2=0x55b341a6aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd198f6f640 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.322+0000 7fd198f6f640 1 -- 172.21.15.62:0/184667335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b341a5c3f0 con 0x55b341a62d40 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.323+0000 7fd198f6f640 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a62d40 0x55b341a5bd30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd18c02ade0 tx=0x7fd18c047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc456b9c3225a32c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd1737fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd1737fe640 1 -- 172.21.15.62:0/184667335 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c04a040 con 0x55b341a62d40 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd1737fe640 1 -- 172.21.15.62:0/184667335 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fd18c031760 con 0x55b341a62d40 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd19cdab740 1 -- 172.21.15.62:0/184667335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd17c003fe0 con 0x55b341a62d40 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd1737fe640 1 -- 172.21.15.62:0/184667335 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd18c004b40 con 0x55b341a62d40 2024-09-12T00:01:19.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd19cdab740 10 client.26557 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd19cdab740 1 -- 172.21.15.62:0/184667335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b341a5d730 con 0x55b341a62d40 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd1737fe640 1 -- 172.21.15.62:0/184667335 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c003fe0 con 0x55b341a62d40 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd1737fe640 1 -- 172.21.15.62:0/184667335 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fd18c05b040 con 0x55b341a62d40 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd19cdab740 10 client.26557 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd19cdab740 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd19cdab740 20 client.26557 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd19cdab740 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b341ade670 0x55b341ae0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd19cdab740 4 client.26557 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.324+0000 7fd19cdab740 1 -- 172.21.15.62:0/184667335 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b341a75110 con 0x55b341ade670 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.325+0000 7fd18bfff640 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b341ade670 0x55b341ae0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.325+0000 7fd1737fe640 10 client.26557 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:19.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.325+0000 7fd18bfff640 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b341ade670 0x55b341ae0a60 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a0c59b7d9b6b06 server_cookie=bcba4ee4643c8a54 in_seq=0 out_seq=0 2024-09-12T00:01:19.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.326+0000 7fd1737fe640 1 -- 172.21.15.62:0/184667335 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b341a75110 con 0x55b341ade670 2024-09-12T00:01:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.326+0000 7fd1737fe640 10 client.26557 handle_command_reply: tid=0 2024-09-12T00:01:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.326+0000 7fd19cdab740 1 client.26557 shutdown 2024-09-12T00:01:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 1 -- 172.21.15.62:0/184667335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a62d40 msgr2=0x55b341a5bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a62d40 0x55b341a5bd30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd18c02ade0 tx=0x7fd18c047040 comp rx=0 tx=0).stop 2024-09-12T00:01:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 1 -- 172.21.15.62:0/184667335 shutdown_connections 2024-09-12T00:01:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55b341ade670 0x55b341ae0a60 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b341a62d40 0x55b341a5bd30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b341a58480 0x55b341a6b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd1737fe640 0 client.26557 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:19.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 1 --2- 172.21.15.62:0/184667335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b341a57b60 0x55b341a6aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 1 -- 172.21.15.62:0/184667335 >> 172.21.15.62:0/184667335 conn(0x55b341a540f0 msgr2=0x55b341a595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:19.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 1 -- 172.21.15.62:0/184667335 shutdown_connections 2024-09-12T00:01:19.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 1 -- 172.21.15.62:0/184667335 wait complete. 2024-09-12T00:01:19.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.327+0000 7fd19cdab740 20 client.26557 trim_cache size 0 max 16384 2024-09-12T00:01:19.332 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:01:19.332 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-12T00:01:19.332 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "89b23aa2-5c0f-47a0-9112-10d5669b3151", 2024-09-12T00:01:19.332 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-12T00:01:19.332 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:01:19.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2858877049 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1800a2df0 msgr2=0x7fd1800a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:19.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1800a2df0 0x7fd1800a52b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55b341a54f50 tx=0x7fd1840779f0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2858877049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd194101de0 msgr2=0x7fd1940744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:19.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd194101de0 0x7fd1940744f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd18400e9f0 tx=0x7fd18400eeb0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2858877049 shutdown_connections 2024-09-12T00:01:19.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1800a2df0 0x7fd1800a52b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd194106f10 0x7fd194074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd194102790 0x7fd194074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 --2- 172.21.15.62:0/2858877049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd194101de0 0x7fd1940744f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:19.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2858877049 >> 172.21.15.62:0/2858877049 conn(0x7fd1940fd8c0 msgr2=0x7fd1940fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:19.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2858877049 shutdown_connections 2024-09-12T00:01:19.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:19.328+0000 7fd19a9d2640 1 -- 172.21.15.62:0/2858877049 wait complete. 2024-09-12T00:01:19.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:19 smithi062 ceph-mon[30958]: pgmap v285: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 955 op/s 2024-09-12T00:01:19.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:19 smithi120 ceph-mon[35182]: pgmap v285: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 955 op/s 2024-09-12T00:01:19.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:19 smithi083 ceph-mon[34805]: pgmap v285: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 955 op/s 2024-09-12T00:01:20.057 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:01:20.430 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:20.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:20 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-12T00:01:20.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:20 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:20 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:01:20.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:20 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:01:20.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:20 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:20 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:01:20.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:20 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:20 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:20 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-12T00:01:20.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:20 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:20 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:01:20.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:20 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:01:20.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:20 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:20 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:01:20.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:20 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:20 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:20 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-12T00:01:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:20 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:20 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:01:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:20 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:01:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:20 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:20 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:01:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:20 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:20.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:20 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:01:21.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.012+0000 7f80ce6da640 1 -- 172.21.15.62:0/3326769967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80c81008d0 msgr2=0x7f80c8100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:21.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.012+0000 7f80ce6da640 1 --2- 172.21.15.62:0/3326769967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80c81008d0 0x7f80c8100d30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009890 tx=0x7f80b802a360 comp rx=0 tx=0).stop 2024-09-12T00:01:21.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.012+0000 7f80ce6da640 1 -- 172.21.15.62:0/3326769967 shutdown_connections 2024-09-12T00:01:21.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.012+0000 7f80ce6da640 1 --2- 172.21.15.62:0/3326769967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c8101270 0x7f80c810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.012+0000 7f80ce6da640 1 --2- 172.21.15.62:0/3326769967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80c81008d0 0x7f80c8100d30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.012+0000 7f80ce6da640 1 --2- 172.21.15.62:0/3326769967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c81068d0 0x7f80c8106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.012+0000 7f80ce6da640 1 -- 172.21.15.62:0/3326769967 >> 172.21.15.62:0/3326769967 conn(0x7f80c80fc610 msgr2=0x7f80c80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:21.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.012+0000 7f80ce6da640 1 -- 172.21.15.62:0/3326769967 shutdown_connections 2024-09-12T00:01:21.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.012+0000 7f80ce6da640 1 -- 172.21.15.62:0/3326769967 wait complete. 2024-09-12T00:01:21.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.014+0000 7f80ce6da640 1 Processor -- start 2024-09-12T00:01:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.014+0000 7f80ce6da640 1 -- start start 2024-09-12T00:01:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.015+0000 7f80ce6da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80c81008d0 0x7f80c81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.015+0000 7f80ce6da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c8101270 0x7f80c81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.015+0000 7f80ce6da640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c81068d0 0x7f80c819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.015+0000 7f80ce6da640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f80c8111df0 con 0x7f80c81068d0 2024-09-12T00:01:21.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.015+0000 7f80ce6da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80c8111c70 con 0x7f80c8101270 2024-09-12T00:01:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.015+0000 7f80ce6da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f80c8111f70 con 0x7f80c81008d0 2024-09-12T00:01:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.016+0000 7f80cced7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c8101270 0x7f80c81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.016+0000 7f80cced7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c8101270 0x7f80c81987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58448/0 (socket says 172.21.15.62:58448) 2024-09-12T00:01:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.016+0000 7f80cced7640 1 -- 172.21.15.62:0/3113239743 learned_addr learned my addr 172.21.15.62:0/3113239743 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.016+0000 7f80cded9640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c81068d0 0x7f80c819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.016+0000 7f80cced7640 1 -- 172.21.15.62:0/3113239743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80c81008d0 msgr2=0x7f80c81982b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T00:01:21.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.016+0000 7f80cced7640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80c81008d0 0x7f80c81982b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.016+0000 7f80cced7640 1 -- 172.21.15.62:0/3113239743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c81068d0 msgr2=0x7f80c819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:21.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.016+0000 7f80cced7640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c81068d0 0x7f80c819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.016+0000 7f80cced7640 1 -- 172.21.15.62:0/3113239743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c819d0c0 con 0x7f80c8101270 2024-09-12T00:01:21.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.017+0000 7f80cced7640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c8101270 0x7f80c81987f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f80b8000c00 tx=0x7f80b8033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:21.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.017+0000 7f80be7fc640 1 -- 172.21.15.62:0/3113239743 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b803f020 con 0x7f80c8101270 2024-09-12T00:01:21.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.017+0000 7f80be7fc640 1 -- 172.21.15.62:0/3113239743 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80b8032770 con 0x7f80c8101270 2024-09-12T00:01:21.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.017+0000 7f80be7fc640 1 -- 172.21.15.62:0/3113239743 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8037040 con 0x7f80c8101270 2024-09-12T00:01:21.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.017+0000 7f80ce6da640 1 -- 172.21.15.62:0/3113239743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c819d2f0 con 0x7f80c8101270 2024-09-12T00:01:21.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.017+0000 7f80ce6da640 1 -- 172.21.15.62:0/3113239743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c819d6c0 con 0x7f80c8101270 2024-09-12T00:01:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.021+0000 7f80be7fc640 1 -- 172.21.15.62:0/3113239743 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f80b8042020 con 0x7f80c8101270 2024-09-12T00:01:21.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.022+0000 7f80d1d15740 1 Processor -- start 2024-09-12T00:01:21.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.022+0000 7f80d1d15740 1 -- start start 2024-09-12T00:01:21.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4e670 0x5631fbf4ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4ef90 0x5631fbf59310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf5bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80d1d15740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631fbeb42e0 con 0x5631fbf4e670 2024-09-12T00:01:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80d1d15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631fbe5bce0 con 0x5631fbf59850 2024-09-12T00:01:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80d1d15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5631fbeb02a0 con 0x5631fbf4ef90 2024-09-12T00:01:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80cced7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf5bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80cced7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf5bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58462/0 (socket says 172.21.15.62:58462) 2024-09-12T00:01:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80cced7640 1 -- 172.21.15.62:0/3863792540 learned_addr learned my addr 172.21.15.62:0/3863792540 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80cd6d8640 1 --2- 172.21.15.62:0/3863792540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4ef90 0x5631fbf59310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80cced7640 1 -- 172.21.15.62:0/3863792540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4ef90 msgr2=0x5631fbf59310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80cced7640 1 --2- 172.21.15.62:0/3863792540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4ef90 0x5631fbf59310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80cced7640 1 -- 172.21.15.62:0/3863792540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4e670 msgr2=0x5631fbf4ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80cced7640 1 --2- 172.21.15.62:0/3863792540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4e670 0x5631fbf4ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.023+0000 7f80cced7640 1 -- 172.21.15.62:0/3863792540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631fbe40440 con 0x5631fbf59850 2024-09-12T00:01:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.024+0000 7f80cced7640 1 --2- 172.21.15.62:0/3863792540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf5bc40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80b80ecb40 tx=0x7f80b80f7470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5502eb55117dcac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.025+0000 7f80b37fe640 1 -- 172.21.15.62:0/3863792540 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80f1310 con 0x5631fbf59850 2024-09-12T00:01:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.025+0000 7f80b37fe640 1 -- 172.21.15.62:0/3863792540 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80b80f14b0 con 0x5631fbf59850 2024-09-12T00:01:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.025+0000 7f80be7fc640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f80b40a2df0 0x7f80b40a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.025+0000 7f80be7fc640 1 -- 172.21.15.62:0/3113239743 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f80b80ef400 con 0x7f80c8101270 2024-09-12T00:01:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.025+0000 7f80cded9640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f80b40a2df0 0x7f80b40a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.026+0000 7f80b37fe640 1 -- 172.21.15.62:0/3863792540 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8100a00 con 0x5631fbf59850 2024-09-12T00:01:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.026+0000 7f80d1d15740 1 -- 172.21.15.62:0/3863792540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 msgr2=0x5631fbf5bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.026+0000 7f80d1d15740 1 --2- 172.21.15.62:0/3863792540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf5bc40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80b80ecb40 tx=0x7f80b80f7470 comp rx=0 tx=0).stop 2024-09-12T00:01:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.026+0000 7f80d1d15740 1 -- 172.21.15.62:0/3863792540 shutdown_connections 2024-09-12T00:01:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.026+0000 7f80d1d15740 1 --2- 172.21.15.62:0/3863792540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf5bc40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.026+0000 7f80d1d15740 1 --2- 172.21.15.62:0/3863792540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4ef90 0x5631fbf59310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.026+0000 7f80d1d15740 1 --2- 172.21.15.62:0/3863792540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4e670 0x5631fbf4ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.026+0000 7f80d1d15740 1 -- 172.21.15.62:0/3863792540 >> 172.21.15.62:0/3863792540 conn(0x5631fbf4ac00 msgr2=0x5631fbf4afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:21.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.026+0000 7f80d1d15740 1 -- 172.21.15.62:0/3863792540 shutdown_connections 2024-09-12T00:01:21.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.026+0000 7f80d1d15740 1 -- 172.21.15.62:0/3863792540 wait complete. 2024-09-12T00:01:21.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.027+0000 7f80d1d15740 1 -- start start 2024-09-12T00:01:21.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.027+0000 7f80d1d15740 1 Processor -- start 2024-09-12T00:01:21.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.027+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4e670 0x5631fbf590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.027+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ef90 0x5631fbf56360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.027+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.027+0000 7f80d1d15740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631fbf5c1e0 con 0x5631fbf4ef90 2024-09-12T00:01:21.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.027+0000 7f80d1d15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631fbe6f050 con 0x5631fbf59850 2024-09-12T00:01:21.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.027+0000 7f80d1d15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5631fbf5c360 con 0x5631fbf4e670 2024-09-12T00:01:21.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.027+0000 7f80d1d15740 10 client.0 _ll_register_callbacks cb 0x5631fbeaf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:21.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80cded9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80cded9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58474/0 (socket says 172.21.15.62:58474) 2024-09-12T00:01:21.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80cded9640 1 -- 172.21.15.62:0/4277316392 learned_addr learned my addr 172.21.15.62:0/4277316392 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:21.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80cced7640 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ef90 0x5631fbf56360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80cded9640 1 -- 172.21.15.62:0/4277316392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4e670 msgr2=0x5631fbf590b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:01:21.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80cded9640 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4e670 0x5631fbf590b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80cded9640 1 -- 172.21.15.62:0/4277316392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ef90 msgr2=0x5631fbf56360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:21.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80cded9640 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ef90 0x5631fbf56360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80cded9640 1 -- 172.21.15.62:0/4277316392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631fbc253c0 con 0x5631fbf59850 2024-09-12T00:01:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80cded9640 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf568a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f80c001ad70 tx=0x7f80c0009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a78cd1f77e2c669 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.028+0000 7f80b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:01:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.029+0000 7f80b0ff9640 1 -- 172.21.15.62:0/4277316392 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0003970 con 0x5631fbf59850 2024-09-12T00:01:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.029+0000 7f80cded9640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f80b40a2df0 0x7f80b40a52b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009ba0 tx=0x7f80c0010040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.029+0000 7f80d1d15740 1 -- 172.21.15.62:0/4277316392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80b80f14b0 con 0x5631fbf59850 2024-09-12T00:01:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.029+0000 7f80b0ff9640 1 -- 172.21.15.62:0/4277316392 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f80c0003b10 con 0x5631fbf59850 2024-09-12T00:01:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.029+0000 7f80b0ff9640 1 -- 172.21.15.62:0/4277316392 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80c002f050 con 0x5631fbf59850 2024-09-12T00:01:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.029+0000 7f80d1d15740 10 client.26633 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.029+0000 7f80d1d15740 1 -- 172.21.15.62:0/4277316392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631fbf70420 con 0x5631fbf59850 2024-09-12T00:01:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.030+0000 7f80b0ff9640 1 -- 172.21.15.62:0/4277316392 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80b80f14b0 con 0x5631fbf59850 2024-09-12T00:01:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.030+0000 7f80b0ff9640 1 -- 172.21.15.62:0/4277316392 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f80c0021030 con 0x5631fbf59850 2024-09-12T00:01:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.030+0000 7f80d1d15740 10 client.26633 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.030+0000 7f80d1d15740 10 client.26633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:01:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.030+0000 7f80d1d15740 20 client.26633 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.030+0000 7f80d1d15740 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5631fbf71e20 0x5631fbf742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.030+0000 7f80d1d15740 4 client.26633 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:01:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.030+0000 7f80d1d15740 1 -- 172.21.15.62:0/4277316392 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631fbec1850 con 0x5631fbf71e20 2024-09-12T00:01:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.031+0000 7f80cd6d8640 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5631fbf71e20 0x5631fbf742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.031+0000 7f80cd6d8640 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5631fbf71e20 0x5631fbf742b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1a0bc8f38f70570 server_cookie=6110d60fac6c8ea in_seq=0 out_seq=0 2024-09-12T00:01:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.031+0000 7f80b0ff9640 10 client.26633 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.032+0000 7f80b0ff9640 1 -- 172.21.15.62:0/4277316392 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631fbec1850 con 0x5631fbf71e20 2024-09-12T00:01:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.032+0000 7f80b0ff9640 10 client.26633 handle_command_reply: tid=0 2024-09-12T00:01:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.032+0000 7f80d1d15740 1 client.26633 shutdown 2024-09-12T00:01:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.033+0000 7f80d1d15740 1 -- 172.21.15.62:0/4277316392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 msgr2=0x5631fbf568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.033+0000 7f80d1d15740 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf568a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f80c001ad70 tx=0x7f80c0009b30 comp rx=0 tx=0).stop 2024-09-12T00:01:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.033+0000 7f80d1d15740 1 -- 172.21.15.62:0/4277316392 shutdown_connections 2024-09-12T00:01:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.033+0000 7f80d1d15740 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5631fbf71e20 0x5631fbf742b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.033+0000 7f80d1d15740 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf59850 0x5631fbf568a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.033+0000 7f80d1d15740 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ef90 0x5631fbf56360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.033+0000 7f80d1d15740 1 --2- 172.21.15.62:0/4277316392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4e670 0x5631fbf590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.033+0000 7f80d1d15740 1 -- 172.21.15.62:0/4277316392 >> 172.21.15.62:0/4277316392 conn(0x5631fbf4ac00 msgr2=0x5631fbf508e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.033+0000 7f80b0ff9640 0 client.26633 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.033+0000 7f80d1d15740 1 -- 172.21.15.62:0/4277316392 shutdown_connections 2024-09-12T00:01:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.034+0000 7f80d1d15740 1 -- 172.21.15.62:0/4277316392 wait complete. 2024-09-12T00:01:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.034+0000 7f80d1d15740 20 client.26633 trim_cache size 0 max 16384 2024-09-12T00:01:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.064+0000 7f80d1d15740 1 Processor -- start 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.064+0000 7f80d1d15740 1 -- start start 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.064+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ac00 0x5631fbf4a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.064+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4e670 0x5631fbf79bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.064+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ef90 0x5631fbf7a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80d1d15740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81ade30 con 0x5631fbf4ef90 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80cd6d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ac00 0x5631fbf4a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80cd6d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ac00 0x5631fbf4a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58484/0 (socket says 172.21.15.62:58484) 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80d1d15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81ab700 con 0x5631fbf4ac00 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80cded9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ef90 0x5631fbf7a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80cd6d8640 1 -- 172.21.15.62:0/1810731288 learned_addr learned my addr 172.21.15.62:0/1810731288 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80d1d15740 1 -- 172.21.15.62:0/1810731288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81a9570 con 0x5631fbf4e670 2024-09-12T00:01:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80cced7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4e670 0x5631fbf79bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80cced7640 1 -- 172.21.15.62:0/1810731288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ac00 msgr2=0x5631fbf4a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80cced7640 1 --2- 172.21.15.62:0/1810731288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ac00 0x5631fbf4a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80cced7640 1 -- 172.21.15.62:0/1810731288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ef90 msgr2=0x5631fbf7a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80cced7640 1 --2- 172.21.15.62:0/1810731288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ef90 0x5631fbf7a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.065+0000 7f80cced7640 1 -- 172.21.15.62:0/1810731288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631fbf5bfb0 con 0x5631fbf4e670 2024-09-12T00:01:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.066+0000 7f80cced7640 1 --2- 172.21.15.62:0/1810731288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4e670 0x5631fbf79bc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005e00 tx=0x7f80b8004240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26b39e4a99ec8c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.066+0000 7f80b2ffd640 1 -- 172.21.15.62:0/1810731288 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80bb7a0 con 0x5631fbf4e670 2024-09-12T00:01:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.066+0000 7f80b2ffd640 1 -- 172.21.15.62:0/1810731288 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80b80bc0d0 con 0x5631fbf4e670 2024-09-12T00:01:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.066+0000 7f80b2ffd640 1 -- 172.21.15.62:0/1810731288 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80bb330 con 0x5631fbf4e670 2024-09-12T00:01:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.067+0000 7f80d1d15740 1 -- 172.21.15.62:0/1810731288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4e670 msgr2=0x5631fbf79bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.067+0000 7f80d1d15740 1 --2- 172.21.15.62:0/1810731288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4e670 0x5631fbf79bc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005e00 tx=0x7f80b8004240 comp rx=0 tx=0).stop 2024-09-12T00:01:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.068+0000 7f80d1d15740 1 -- 172.21.15.62:0/1810731288 shutdown_connections 2024-09-12T00:01:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.068+0000 7f80d1d15740 1 --2- 172.21.15.62:0/1810731288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ef90 0x5631fbf7a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.068+0000 7f80d1d15740 1 --2- 172.21.15.62:0/1810731288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf4e670 0x5631fbf79bc0 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005e00 tx=0x7f80b8004240 comp rx=0 tx=0).stop 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.068+0000 7f80d1d15740 1 --2- 172.21.15.62:0/1810731288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ac00 0x5631fbf4a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.068+0000 7f80d1d15740 1 -- 172.21.15.62:0/1810731288 >> 172.21.15.62:0/1810731288 conn(0x5631fbf56a00 msgr2=0x5631fbf54c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.068+0000 7f80d1d15740 1 -- 172.21.15.62:0/1810731288 shutdown_connections 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.068+0000 7f80d1d15740 1 -- 172.21.15.62:0/1810731288 wait complete. 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.068+0000 7f80d1d15740 1 -- start start 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.069+0000 7f80d1d15740 1 Processor -- start 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.069+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ac00 0x5631fbf6a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.069+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ef90 0x5631fbf53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.069+0000 7f80d1d15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf59850 0x5631fbf53730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.069+0000 7f80d1d15740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81a9bc0 con 0x5631fbf4ac00 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.069+0000 7f80d1d15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81ab030 con 0x5631fbf4ef90 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.069+0000 7f80d1d15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81aa290 con 0x5631fbf59850 2024-09-12T00:01:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.069+0000 7f80d1d15740 10 client.0 _ll_register_callbacks cb 0x7f80c0009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.071+0000 7f80cced7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ef90 0x5631fbf53120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.071+0000 7f80cced7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ef90 0x5631fbf53120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:58494/0 (socket says 172.21.15.62:58494) 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.071+0000 7f80cced7640 1 -- 172.21.15.62:0/2173318737 learned_addr learned my addr 172.21.15.62:0/2173318737 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.071+0000 7f80cced7640 1 -- 172.21.15.62:0/2173318737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf59850 msgr2=0x5631fbf53730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.071+0000 7f80cced7640 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf59850 0x5631fbf53730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.071+0000 7f80cced7640 1 -- 172.21.15.62:0/2173318737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ac00 msgr2=0x5631fbf6a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.071+0000 7f80cced7640 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ac00 0x5631fbf6a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.071+0000 7f80cced7640 1 -- 172.21.15.62:0/2173318737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631fbf53f00 con 0x5631fbf4ef90 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.072+0000 7f80cced7640 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ef90 0x5631fbf53120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f80b80b83a0 tx=0x7f80b802a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c797251c09b16b50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.072+0000 7f80b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.072+0000 7f80b0ff9640 1 -- 172.21.15.62:0/2173318737 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80bb510 con 0x5631fbf4ef90 2024-09-12T00:01:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.072+0000 7f80b0ff9640 1 -- 172.21.15.62:0/2173318737 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f80b80b9cd0 con 0x5631fbf4ef90 2024-09-12T00:01:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.072+0000 7f80d1d15740 1 -- 172.21.15.62:0/2173318737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80b80bc0d0 con 0x5631fbf4ef90 2024-09-12T00:01:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.072+0000 7f80b0ff9640 1 -- 172.21.15.62:0/2173318737 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80b80bb330 con 0x5631fbf4ef90 2024-09-12T00:01:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.073+0000 7f80d1d15740 10 client.26639 fetch_fsmap learned FSMap version 32 2024-09-12T00:01:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.073+0000 7f80d1d15740 1 -- 172.21.15.62:0/2173318737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631fbfdcc70 con 0x5631fbf4ef90 2024-09-12T00:01:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.073+0000 7f80b0ff9640 1 -- 172.21.15.62:0/2173318737 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80b80bb9d0 con 0x5631fbf4ef90 2024-09-12T00:01:21.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.074+0000 7f80b0ff9640 1 -- 172.21.15.62:0/2173318737 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f80b803a030 con 0x5631fbf4ef90 2024-09-12T00:01:21.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.074+0000 7f80d1d15740 10 client.26639 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T00:01:21.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.074+0000 7f80d1d15740 10 client.26639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:01:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.074+0000 7f80d1d15740 20 client.26639 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.074+0000 7f80d1d15740 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5631fbfdcdd0 0x5631fbfdf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.074+0000 7f80d1d15740 4 client.26639 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:01:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.074+0000 7f80d1d15740 1 -- 172.21.15.62:0/2173318737 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f80b80bc0d0 con 0x5631fbfdcdd0 2024-09-12T00:01:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.075+0000 7f80cd6d8640 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5631fbfdcdd0 0x5631fbfdf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.076+0000 7f80cd6d8640 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5631fbfdcdd0 0x5631fbfdf1c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c06e35cc056beb2 server_cookie=beb9d83b3b61c487 in_seq=0 out_seq=0 2024-09-12T00:01:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.076+0000 7f80b0ff9640 10 client.26639 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.076+0000 7f80b0ff9640 1 -- 172.21.15.62:0/2173318737 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f80b80bc0d0 con 0x5631fbfdcdd0 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.076+0000 7f80b0ff9640 10 client.26639 handle_command_reply: tid=0 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.077+0000 7f80d1d15740 1 client.26639 shutdown 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.077+0000 7f80d1d15740 1 -- 172.21.15.62:0/2173318737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ef90 msgr2=0x5631fbf53120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.077+0000 7f80d1d15740 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ef90 0x5631fbf53120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f80b80b83a0 tx=0x7f80b802a360 comp rx=0 tx=0).stop 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.078+0000 7f80d1d15740 1 -- 172.21.15.62:0/2173318737 shutdown_connections 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.078+0000 7f80d1d15740 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5631fbfdcdd0 0x5631fbfdf1c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.078+0000 7f80d1d15740 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5631fbf59850 0x5631fbf53730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.078+0000 7f80b0ff9640 0 client.26639 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.078+0000 7f80d1d15740 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fbf4ef90 0x5631fbf53120 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.078+0000 7f80d1d15740 1 --2- 172.21.15.62:0/2173318737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631fbf4ac00 0x5631fbf6a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.078+0000 7f80d1d15740 1 -- 172.21.15.62:0/2173318737 >> 172.21.15.62:0/2173318737 conn(0x5631fbf56a00 msgr2=0x5631fbf57bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.078+0000 7f80d1d15740 1 -- 172.21.15.62:0/2173318737 shutdown_connections 2024-09-12T00:01:21.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.078+0000 7f80d1d15740 1 -- 172.21.15.62:0/2173318737 wait complete. 2024-09-12T00:01:21.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.078+0000 7f80d1d15740 20 client.26639 trim_cache size 0 max 16384 2024-09-12T00:01:21.080 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:01:21.080 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:01:21.080 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-12T00:01:21.080 INFO:teuthology.orchestra.run.smithi062.stdout: "89b23aa2-5c0f-47a0-9112-10d5669b3151": { 2024-09-12T00:01:21.080 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-12T00:01:21.080 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "89b23aa2-5c0f-47a0-9112-10d5669b3151", 2024-09-12T00:01:21.080 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-12T00:01:21.081 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:01:21.081 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:01:21.081 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:01:21.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.079+0000 7f80ce6da640 1 -- 172.21.15.62:0/3113239743 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f80b40a2df0 msgr2=0x7f80b40a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:21.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.079+0000 7f80ce6da640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f80b40a2df0 0x7f80b40a52b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009ba0 tx=0x7f80c0010040 comp rx=0 tx=0).stop 2024-09-12T00:01:21.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.079+0000 7f80ce6da640 1 -- 172.21.15.62:0/3113239743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c8101270 msgr2=0x7f80c81987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.079+0000 7f80ce6da640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c8101270 0x7f80c81987f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f80b8000c00 tx=0x7f80b8033d10 comp rx=0 tx=0).stop 2024-09-12T00:01:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.080+0000 7f80ce6da640 1 -- 172.21.15.62:0/3113239743 shutdown_connections 2024-09-12T00:01:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.080+0000 7f80ce6da640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f80b40a2df0 0x7f80b40a52b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.080+0000 7f80ce6da640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c81068d0 0x7f80c819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.080+0000 7f80ce6da640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c8101270 0x7f80c81987f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.080+0000 7f80ce6da640 1 --2- 172.21.15.62:0/3113239743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80c81008d0 0x7f80c81982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.080+0000 7f80ce6da640 1 -- 172.21.15.62:0/3113239743 >> 172.21.15.62:0/3113239743 conn(0x7f80c80fc610 msgr2=0x7f80c80fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.080+0000 7f80ce6da640 1 -- 172.21.15.62:0/3113239743 shutdown_connections 2024-09-12T00:01:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:21.080+0000 7f80ce6da640 1 -- 172.21.15.62:0/3113239743 wait complete. 2024-09-12T00:01:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:21 smithi120 ceph-mon[35182]: pgmap v286: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 729 op/s 2024-09-12T00:01:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:21 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:01:21.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:21 smithi083 ceph-mon[34805]: pgmap v286: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 729 op/s 2024-09-12T00:01:21.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:21 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:01:21.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:21 smithi062 ceph-mon[30958]: pgmap v286: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 729 op/s 2024-09-12T00:01:21.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:21 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:01:21.827 INFO:tasks.cephfs.filesystem:scrub status for tag:89b23aa2-5c0f-47a0-9112-10d5669b3151 - {'path': '/', 'tag': '89b23aa2-5c0f-47a0-9112-10d5669b3151', 'options': 'recursive,force'} 2024-09-12T00:01:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 520.0 s, 2685.2 tps, lat 37.128 ms stddev 39.551 2024-09-12T00:01:22.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:22 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:01:22.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:22 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:01:22.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:22 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:01:22.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:22 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:01:22.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:22 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:01:22.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:22 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:01:22.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:22 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:01:22.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:22 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:01:22.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:22 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:01:23.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:23 smithi120 ceph-mon[35182]: pgmap v287: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 728 op/s 2024-09-12T00:01:23.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:23 smithi083 ceph-mon[34805]: pgmap v287: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 728 op/s 2024-09-12T00:01:23.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:23 smithi062 ceph-mon[30958]: pgmap v287: 97 pgs: 97 active+clean; 3.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 728 op/s 2024-09-12T00:01:25.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:25 smithi120 ceph-mon[35182]: pgmap v288: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-12T00:01:25.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:25 smithi083 ceph-mon[34805]: pgmap v288: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-12T00:01:25.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:25 smithi062 ceph-mon[30958]: pgmap v288: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-12T00:01:26.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:26 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:01:26.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:26 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:01:26.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:26 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:01:26.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:26 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:01:26.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:26 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:01:26.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:26 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:01:26.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:26 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:01:26.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:26 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:01:26.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:26 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:01:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 525.0 s, 2793.0 tps, lat 35.921 ms stddev 40.025 2024-09-12T00:01:27.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:27 smithi120 ceph-mon[35182]: pgmap v289: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-12T00:01:27.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:27 smithi083 ceph-mon[34805]: pgmap v289: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-12T00:01:27.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:27 smithi062 ceph-mon[30958]: pgmap v289: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-12T00:01:29.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:29 smithi120 ceph-mon[35182]: pgmap v290: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.0 MiB/s wr, 920 op/s 2024-09-12T00:01:29.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:29 smithi083 ceph-mon[34805]: pgmap v290: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.0 MiB/s wr, 920 op/s 2024-09-12T00:01:29.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:29 smithi062 ceph-mon[30958]: pgmap v290: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.0 MiB/s wr, 920 op/s 2024-09-12T00:01:31.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:31 smithi120 ceph-mon[35182]: pgmap v291: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 667 op/s 2024-09-12T00:01:31.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:31 smithi083 ceph-mon[34805]: pgmap v291: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 667 op/s 2024-09-12T00:01:31.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:31 smithi062 ceph-mon[30958]: pgmap v291: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 667 op/s 2024-09-12T00:01:32.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 530.0 s, 2565.8 tps, lat 38.664 ms stddev 46.752 2024-09-12T00:01:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:33 smithi120 ceph-mon[35182]: pgmap v292: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 685 op/s 2024-09-12T00:01:33.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:33 smithi083 ceph-mon[34805]: pgmap v292: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 685 op/s 2024-09-12T00:01:33.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:33 smithi062 ceph-mon[30958]: pgmap v292: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 685 op/s 2024-09-12T00:01:35.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:35 smithi120 ceph-mon[35182]: pgmap v293: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 987 op/s 2024-09-12T00:01:35.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:35 smithi083 ceph-mon[34805]: pgmap v293: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 987 op/s 2024-09-12T00:01:35.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:35 smithi062 ceph-mon[30958]: pgmap v293: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 987 op/s 2024-09-12T00:01:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 535.0 s, 2072.2 tps, lat 48.505 ms stddev 87.150 2024-09-12T00:01:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:37 smithi120 ceph-mon[35182]: pgmap v294: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-12T00:01:37.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:37 smithi083 ceph-mon[34805]: pgmap v294: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-12T00:01:37.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:37 smithi062 ceph-mon[30958]: pgmap v294: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-12T00:01:39.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:39 smithi120 ceph-mon[35182]: pgmap v295: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 37 MiB/s wr, 1.88k op/s 2024-09-12T00:01:39.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:39 smithi083 ceph-mon[34805]: pgmap v295: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 37 MiB/s wr, 1.88k op/s 2024-09-12T00:01:39.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:39 smithi062 ceph-mon[30958]: pgmap v295: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 37 MiB/s wr, 1.88k op/s 2024-09-12T00:01:41.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:41 smithi120 ceph-mon[35182]: pgmap v296: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 MiB/s wr, 1.83k op/s 2024-09-12T00:01:41.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:41 smithi083 ceph-mon[34805]: pgmap v296: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 MiB/s wr, 1.83k op/s 2024-09-12T00:01:41.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:41 smithi062 ceph-mon[30958]: pgmap v296: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 MiB/s wr, 1.83k op/s 2024-09-12T00:01:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 540.0 s, 3064.6 tps, lat 32.574 ms stddev 38.353 2024-09-12T00:01:43.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:43 smithi120 ceph-mon[35182]: pgmap v297: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 40 MiB/s wr, 1.82k op/s 2024-09-12T00:01:43.666 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:43 smithi062 ceph-mon[30958]: pgmap v297: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 40 MiB/s wr, 1.82k op/s 2024-09-12T00:01:43.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:43 smithi083 ceph-mon[34805]: pgmap v297: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 40 MiB/s wr, 1.82k op/s 2024-09-12T00:01:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:45 smithi120 ceph-mon[35182]: pgmap v298: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 46 MiB/s wr, 2.03k op/s 2024-09-12T00:01:45.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:45 smithi083 ceph-mon[34805]: pgmap v298: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 46 MiB/s wr, 2.03k op/s 2024-09-12T00:01:45.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:45 smithi062 ceph-mon[30958]: pgmap v298: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 46 MiB/s wr, 2.03k op/s 2024-09-12T00:01:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 545.0 s, 2747.0 tps, lat 36.414 ms stddev 39.803 2024-09-12T00:01:47.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:47 smithi120 ceph-mon[35182]: pgmap v299: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-09-12T00:01:47.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:47 smithi083 ceph-mon[34805]: pgmap v299: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-09-12T00:01:47.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:47 smithi062 ceph-mon[30958]: pgmap v299: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-09-12T00:01:49.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:49 smithi120 ceph-mon[35182]: pgmap v300: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 47 MiB/s wr, 1.98k op/s 2024-09-12T00:01:49.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:49 smithi083 ceph-mon[34805]: pgmap v300: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 47 MiB/s wr, 1.98k op/s 2024-09-12T00:01:49.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:49 smithi062 ceph-mon[30958]: pgmap v300: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 47 MiB/s wr, 1.98k op/s 2024-09-12T00:01:51.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:51 smithi120 ceph-mon[35182]: pgmap v301: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 852 op/s 2024-09-12T00:01:51.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:51 smithi083 ceph-mon[34805]: pgmap v301: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 852 op/s 2024-09-12T00:01:51.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:51 smithi062 ceph-mon[30958]: pgmap v301: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 852 op/s 2024-09-12T00:01:51.828 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:01:52.220 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 550.0 s, 2653.2 tps, lat 37.537 ms stddev 46.691 2024-09-12T00:01:52.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.749+0000 7f6b5685a640 1 -- 172.21.15.62:0/3860821443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b50102ef0 msgr2=0x7f6b5010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:52.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.749+0000 7f6b5685a640 1 --2- 172.21.15.62:0/3860821443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b50102ef0 0x7f6b5010da40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009900 tx=0x7f6b3802a480 comp rx=0 tx=0).stop 2024-09-12T00:01:52.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.749+0000 7f6b5685a640 1 -- 172.21.15.62:0/3860821443 shutdown_connections 2024-09-12T00:01:52.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.749+0000 7f6b5685a640 1 --2- 172.21.15.62:0/3860821443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b50102ef0 0x7f6b5010da40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.749+0000 7f6b5685a640 1 --2- 172.21.15.62:0/3860821443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b50102570 0x7f6b501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.749+0000 7f6b5685a640 1 --2- 172.21.15.62:0/3860821443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b50100e20 0x7f6b50101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.749+0000 7f6b5685a640 1 -- 172.21.15.62:0/3860821443 >> 172.21.15.62:0/3860821443 conn(0x7f6b500fc7d0 msgr2=0x7f6b500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:52.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.749+0000 7f6b5685a640 1 -- 172.21.15.62:0/3860821443 shutdown_connections 2024-09-12T00:01:52.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.749+0000 7f6b5685a640 1 -- 172.21.15.62:0/3860821443 wait complete. 2024-09-12T00:01:52.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.750+0000 7f6b5685a640 1 Processor -- start 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.750+0000 7f6b5685a640 1 -- start start 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.750+0000 7f6b5685a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b50100e20 0x7f6b50198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.750+0000 7f6b5685a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b50102570 0x7f6b50198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.750+0000 7f6b5685a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b50102ef0 0x7f6b5019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.750+0000 7f6b5685a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b50110f80 con 0x7f6b50102ef0 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.750+0000 7f6b5685a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b50110e00 con 0x7f6b50102570 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.750+0000 7f6b5685a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b50111100 con 0x7f6b50100e20 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b54df7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b50102ef0 0x7f6b5019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b54df7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b50102ef0 0x7f6b5019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55358/0 (socket says 172.21.15.62:55358) 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b54df7640 1 -- 172.21.15.62:0/2857582481 learned_addr learned my addr 172.21.15.62:0/2857582481 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b50100e20 0x7f6b50198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b50102570 0x7f6b50198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b4ffff640 1 -- 172.21.15.62:0/2857582481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b50102570 msgr2=0x7f6b50198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b50102570 0x7f6b50198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b4ffff640 1 -- 172.21.15.62:0/2857582481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b50102ef0 msgr2=0x7f6b5019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b50102ef0 0x7f6b5019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b4ffff640 1 -- 172.21.15.62:0/2857582481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5019d4b0 con 0x7f6b50100e20 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.751+0000 7f6b54df7640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b50102ef0 0x7f6b5019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.752+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b50100e20 0x7f6b50198500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000eb00 tx=0x7f6b4000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.752+0000 7f6b4d7fa640 1 -- 172.21.15.62:0/2857582481 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40010430 con 0x7f6b50100e20 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.753+0000 7f6b4d7fa640 1 -- 172.21.15.62:0/2857582481 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b40010ab0 con 0x7f6b50100e20 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.753+0000 7f6b5685a640 1 -- 172.21.15.62:0/2857582481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5019d7a0 con 0x7f6b50100e20 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.753+0000 7f6b5685a640 1 -- 172.21.15.62:0/2857582481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b501a5080 con 0x7f6b50100e20 2024-09-12T00:01:52.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.753+0000 7f6b4d7fa640 1 -- 172.21.15.62:0/2857582481 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40017bb0 con 0x7f6b50100e20 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.755+0000 7f6b4d7fa640 1 -- 172.21.15.62:0/2857582481 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6b40010c60 con 0x7f6b50100e20 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b4d7fa640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b3c0a2d70 0x7f6b3c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b58c33740 1 Processor -- start 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b4d7fa640 1 -- 172.21.15.62:0/2857582481 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f6b400cfd70 con 0x7f6b50100e20 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b3c0a2d70 0x7f6b3c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b58c33740 1 -- start start 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22360 0x55fb16d22740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22c80 0x55fb16d31550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d31a90 0x55fb16d33e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b58c33740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fb16c882e0 con 0x55fb16d22360 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b58c33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fb16c2fce0 con 0x55fb16d22c80 2024-09-12T00:01:52.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.756+0000 7f6b58c33740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fb16c842a0 con 0x55fb16d31a90 2024-09-12T00:01:52.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22360 0x55fb16d22740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22360 0x55fb16d22740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55374/0 (socket says 172.21.15.62:55374) 2024-09-12T00:01:52.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4ffff640 1 -- 172.21.15.62:0/1106799012 learned_addr learned my addr 172.21.15.62:0/1106799012 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:52.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b54df7640 1 --2- 172.21.15.62:0/1106799012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d31a90 0x55fb16d33e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b3c0a2d70 0x7f6b3c0a5230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6b50105210 tx=0x7f6b48005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:52.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/1106799012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22c80 0x55fb16d31550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4ffff640 1 -- 172.21.15.62:0/1106799012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d31a90 msgr2=0x55fb16d33e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/1106799012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d31a90 0x55fb16d33e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4ffff640 1 -- 172.21.15.62:0/1106799012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22c80 msgr2=0x55fb16d31550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/1106799012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22c80 0x55fb16d31550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4ffff640 1 -- 172.21.15.62:0/1106799012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb16c14440 con 0x55fb16d22360 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/1106799012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22c80 0x55fb16d31550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/1106799012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22360 0x55fb16d22740 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6b40098c00 tx=0x7f6b4009ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17956ac3c6d38280 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b467fc640 1 -- 172.21.15.62:0/1106799012 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b400b5210 con 0x55fb16d22360 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b467fc640 1 -- 172.21.15.62:0/1106799012 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b400a4ea0 con 0x55fb16d22360 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.757+0000 7f6b467fc640 1 -- 172.21.15.62:0/1106799012 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b400a0170 con 0x55fb16d22360 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.758+0000 7f6b58c33740 1 -- 172.21.15.62:0/1106799012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22360 msgr2=0x55fb16d22740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.758+0000 7f6b58c33740 1 --2- 172.21.15.62:0/1106799012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22360 0x55fb16d22740 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6b40098c00 tx=0x7f6b4009ca70 comp rx=0 tx=0).stop 2024-09-12T00:01:52.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.758+0000 7f6b58c33740 1 -- 172.21.15.62:0/1106799012 shutdown_connections 2024-09-12T00:01:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.758+0000 7f6b58c33740 1 --2- 172.21.15.62:0/1106799012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d31a90 0x55fb16d33e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.758+0000 7f6b58c33740 1 --2- 172.21.15.62:0/1106799012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22c80 0x55fb16d31550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.758+0000 7f6b58c33740 1 --2- 172.21.15.62:0/1106799012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22360 0x55fb16d22740 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.758+0000 7f6b58c33740 1 -- 172.21.15.62:0/1106799012 >> 172.21.15.62:0/1106799012 conn(0x55fb16d1ea30 msgr2=0x55fb16d1ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.758+0000 7f6b58c33740 1 -- 172.21.15.62:0/1106799012 shutdown_connections 2024-09-12T00:01:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.758+0000 7f6b58c33740 1 -- 172.21.15.62:0/1106799012 wait complete. 2024-09-12T00:01:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.759+0000 7f6b58c33740 1 -- start start 2024-09-12T00:01:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.759+0000 7f6b58c33740 1 Processor -- start 2024-09-12T00:01:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.760+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d31410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.760+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22c80 0x55fb16d2e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.760+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d31a90 0x55fb16d2ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.760+0000 7f6b4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d31410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.760+0000 7f6b4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d31410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:53370/0 (socket says 172.21.15.62:53370) 2024-09-12T00:01:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.760+0000 7f6b4f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22c80 0x55fb16d2e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.760+0000 7f6b4f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22c80 0x55fb16d2e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:45626/0 (socket says 172.21.15.62:45626) 2024-09-12T00:01:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b4ffff640 1 -- 172.21.15.62:0/1146267752 learned_addr learned my addr 172.21.15.62:0/1146267752 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fb16d34660 con 0x55fb16d31a90 2024-09-12T00:01:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fb16c43050 con 0x55fb16d22360 2024-09-12T00:01:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b54df7640 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d31a90 0x55fb16d2ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fb16d347e0 con 0x55fb16d22c80 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b58c33740 10 client.0 _ll_register_callbacks cb 0x55fb16c83160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b4ffff640 1 -- 172.21.15.62:0/1146267752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22c80 msgr2=0x55fb16d2e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22c80 0x55fb16d2e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b4ffff640 1 -- 172.21.15.62:0/1146267752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d31a90 msgr2=0x55fb16d2ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d31a90 0x55fb16d2ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b4ffff640 1 -- 172.21.15.62:0/1146267752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb169f93c0 con 0x55fb16d22360 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d31410 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b40098690 tx=0x7f6b4009a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109509890ef984c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.761+0000 7f6b2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/1146267752 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b400b9070 con 0x55fb16d22360 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/1146267752 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f6b400cd6e0 con 0x55fb16d22360 2024-09-12T00:01:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b400a4ea0 con 0x55fb16d22360 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/1146267752 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6b400a0170 con 0x55fb16d22360 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b58c33740 10 client.26651 fetch_fsmap learned FSMap version 33 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fb16d400b0 con 0x55fb16d22360 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/1146267752 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b4007e020 con 0x55fb16d22360 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/1146267752 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f6b40081070 con 0x55fb16d22360 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b58c33740 10 client.26651 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b58c33740 10 client.26651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b58c33740 20 client.26651 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b58c33740 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fb16d41400 0x55fb16d437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b58c33740 4 client.26651 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb16c95850 con 0x55fb16d41400 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.762+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fb16d41400 0x55fb16d437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.763+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fb16d41400 0x55fb16d437f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7a41de630c0b3c1 server_cookie=9bb98430f247b471 in_seq=0 out_seq=0 2024-09-12T00:01:52.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.763+0000 7f6b2f7fe640 10 client.26651 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/1146267752 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb16c95850 con 0x55fb16d41400 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b2f7fe640 10 client.26651 handle_command_reply: tid=0 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 client.26651 shutdown 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 msgr2=0x55fb16d31410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d31410 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b40098690 tx=0x7f6b4009a5c0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 shutdown_connections 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fb16d41400 0x55fb16d437f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d31a90 0x55fb16d2ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22c80 0x55fb16d2e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 --2- 172.21.15.62:0/1146267752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d31410 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 >> 172.21.15.62:0/1146267752 conn(0x55fb16d1ea30 msgr2=0x55fb16d24790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b2f7fe640 0 client.26651 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:52.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 shutdown_connections 2024-09-12T00:01:52.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.764+0000 7f6b58c33740 1 -- 172.21.15.62:0/1146267752 wait complete. 2024-09-12T00:01:52.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.765+0000 7f6b58c33740 20 client.26651 trim_cache size 0 max 16384 2024-09-12T00:01:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b58c33740 1 Processor -- start 2024-09-12T00:01:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b58c33740 1 -- start start 2024-09-12T00:01:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22360 0x55fb16d3d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d2cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d31a90 0x55fb16d1da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b58c33740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b30011050 con 0x55fb16d22c80 2024-09-12T00:01:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b58c33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3000e8f0 con 0x55fb16d31a90 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b58c33740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3000c760 con 0x55fb16d22360 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b4f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d2cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b4f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d2cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55392/0 (socket says 172.21.15.62:55392) 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.787+0000 7f6b4f7fe640 1 -- 172.21.15.62:0/3947776848 learned_addr learned my addr 172.21.15.62:0/3947776848 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.788+0000 7f6b54df7640 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d31a90 0x55fb16d1da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.788+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22360 0x55fb16d3d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.788+0000 7f6b4f7fe640 1 -- 172.21.15.62:0/3947776848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22360 msgr2=0x55fb16d3d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.788+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22360 0x55fb16d3d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.788+0000 7f6b4f7fe640 1 -- 172.21.15.62:0/3947776848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d31a90 msgr2=0x55fb16d1da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.788+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d31a90 0x55fb16d1da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.788+0000 7f6b4f7fe640 1 -- 172.21.15.62:0/3947776848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb16d2d670 con 0x55fb16d22c80 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.788+0000 7f6b54df7640 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d31a90 0x55fb16d1da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:01:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.788+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22360 0x55fb16d3d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:01:52.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.788+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d2cf90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6b480117b0 tx=0x7f6b48014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea151004719586 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.789+0000 7f6b45ffb640 1 -- 172.21.15.62:0/3947776848 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48003b70 con 0x55fb16d22c80 2024-09-12T00:01:52.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.789+0000 7f6b45ffb640 1 -- 172.21.15.62:0/3947776848 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b48003d10 con 0x55fb16d22c80 2024-09-12T00:01:52.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.789+0000 7f6b45ffb640 1 -- 172.21.15.62:0/3947776848 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4801f820 con 0x55fb16d22c80 2024-09-12T00:01:52.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.790+0000 7f6b58c33740 1 -- 172.21.15.62:0/3947776848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 msgr2=0x55fb16d2cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:52.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.790+0000 7f6b58c33740 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d2cf90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6b480117b0 tx=0x7f6b48014000 comp rx=0 tx=0).stop 2024-09-12T00:01:52.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.790+0000 7f6b58c33740 1 -- 172.21.15.62:0/3947776848 shutdown_connections 2024-09-12T00:01:52.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.790+0000 7f6b58c33740 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d31a90 0x55fb16d1da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.790+0000 7f6b58c33740 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d2cf90 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.790+0000 7f6b58c33740 1 --2- 172.21.15.62:0/3947776848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d22360 0x55fb16d3d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.790+0000 7f6b58c33740 1 -- 172.21.15.62:0/3947776848 >> 172.21.15.62:0/3947776848 conn(0x55fb16d1ea30 msgr2=0x55fb16d1d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.790+0000 7f6b58c33740 1 -- 172.21.15.62:0/3947776848 shutdown_connections 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.790+0000 7f6b58c33740 1 -- 172.21.15.62:0/3947776848 wait complete. 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.790+0000 7f6b58c33740 1 -- start start 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b58c33740 1 Processor -- start 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d35ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d36430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b58c33740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d31a90 0x55fb16daadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b58c33740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3000cdb0 con 0x55fb16d22c80 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b58c33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3000e220 con 0x55fb16d22360 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b58c33740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3000d480 con 0x55fb16d31a90 2024-09-12T00:01:52.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b4f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d36430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d35ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b4f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d36430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55398/0 (socket says 172.21.15.62:55398) 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b4f7fe640 1 -- 172.21.15.62:0/2501198756 learned_addr learned my addr 172.21.15.62:0/2501198756 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b54df7640 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d31a90 0x55fb16daadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.791+0000 7f6b58c33740 10 client.0 _ll_register_callbacks cb 0x7f6b48005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b4ffff640 1 -- 172.21.15.62:0/2501198756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d31a90 msgr2=0x55fb16daadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d31a90 0x55fb16daadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b4ffff640 1 -- 172.21.15.62:0/2501198756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 msgr2=0x55fb16d36430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d36430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b4ffff640 1 -- 172.21.15.62:0/2501198756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb16dab300 con 0x55fb16d22360 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b4ffff640 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d35ef0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b400bc200 tx=0x7f6b4009c0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1db9595df04f4f50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:01:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/2501198756 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b400b9070 con 0x55fb16d22360 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/2501198756 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f6b400cd6e0 con 0x55fb16d22360 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b58c33740 1 -- 172.21.15.62:0/2501198756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b48003d10 con 0x55fb16d22360 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/2501198756 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6b400984d0 con 0x55fb16d22360 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b58c33740 10 client.26657 fetch_fsmap learned FSMap version 33 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.792+0000 7f6b58c33740 1 -- 172.21.15.62:0/2501198756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fb16dac570 con 0x55fb16d22360 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.793+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/2501198756 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b48003d10 con 0x55fb16d22360 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.793+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/2501198756 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f6b40081070 con 0x55fb16d22360 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.793+0000 7f6b58c33740 10 client.26657 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.793+0000 7f6b58c33740 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.793+0000 7f6b58c33740 20 client.26657 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.793+0000 7f6b58c33740 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fb16dacb50 0x55fb16daef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.793+0000 7f6b58c33740 4 client.26657 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:01:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.793+0000 7f6b58c33740 1 -- 172.21.15.62:0/2501198756 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb16c8fd60 con 0x55fb16dacb50 2024-09-12T00:01:52.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.793+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fb16dacb50 0x55fb16daef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:52.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.794+0000 7f6b2f7fe640 10 client.26657 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:52.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.794+0000 7f6b4f7fe640 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fb16dacb50 0x55fb16daef40 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c076787a4204cc8 server_cookie=c466f2c9740369b in_seq=0 out_seq=0 2024-09-12T00:01:52.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.794+0000 7f6b2f7fe640 1 -- 172.21.15.62:0/2501198756 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fb16c8fd60 con 0x55fb16dacb50 2024-09-12T00:01:52.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.794+0000 7f6b2f7fe640 10 client.26657 handle_command_reply: tid=0 2024-09-12T00:01:52.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.794+0000 7f6b58c33740 1 client.26657 shutdown 2024-09-12T00:01:52.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b58c33740 1 -- 172.21.15.62:0/2501198756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 msgr2=0x55fb16d35ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:52.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b58c33740 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d35ef0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b400bc200 tx=0x7f6b4009c0c0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b58c33740 1 -- 172.21.15.62:0/2501198756 shutdown_connections 2024-09-12T00:01:52.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b58c33740 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fb16dacb50 0x55fb16daef40 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b58c33740 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb16d31a90 0x55fb16daadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b58c33740 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb16d22c80 0x55fb16d36430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b58c33740 1 --2- 172.21.15.62:0/2501198756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb16d22360 0x55fb16d35ef0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b58c33740 1 -- 172.21.15.62:0/2501198756 >> 172.21.15.62:0/2501198756 conn(0x55fb16d1ea30 msgr2=0x55fb16d1d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b2f7fe640 0 client.26657 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b58c33740 1 -- 172.21.15.62:0/2501198756 shutdown_connections 2024-09-12T00:01:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.795+0000 7f6b58c33740 1 -- 172.21.15.62:0/2501198756 wait complete. 2024-09-12T00:01:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.796+0000 7f6b58c33740 20 client.26657 trim_cache size 0 max 16384 2024-09-12T00:01:52.801 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:01:52.801 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:01:52.802 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:01:52.802 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:01:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.797+0000 7f6b5685a640 1 -- 172.21.15.62:0/2857582481 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b3c0a2d70 msgr2=0x7f6b3c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.797+0000 7f6b5685a640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b3c0a2d70 0x7f6b3c0a5230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6b50105210 tx=0x7f6b48005f90 comp rx=0 tx=0).stop 2024-09-12T00:01:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.797+0000 7f6b5685a640 1 -- 172.21.15.62:0/2857582481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b50100e20 msgr2=0x7f6b50198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.797+0000 7f6b5685a640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b50100e20 0x7f6b50198500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000eb00 tx=0x7f6b4000c2d0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.797+0000 7f6b5685a640 1 -- 172.21.15.62:0/2857582481 shutdown_connections 2024-09-12T00:01:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.797+0000 7f6b5685a640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b3c0a2d70 0x7f6b3c0a5230 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.797+0000 7f6b5685a640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b50102ef0 0x7f6b5019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.797+0000 7f6b5685a640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b50102570 0x7f6b50198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.797+0000 7f6b5685a640 1 --2- 172.21.15.62:0/2857582481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b50100e20 0x7f6b50198500 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.797+0000 7f6b5685a640 1 -- 172.21.15.62:0/2857582481 >> 172.21.15.62:0/2857582481 conn(0x7f6b500fc7d0 msgr2=0x7f6b5006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.798+0000 7f6b5685a640 1 -- 172.21.15.62:0/2857582481 shutdown_connections 2024-09-12T00:01:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:52.798+0000 7f6b5685a640 1 -- 172.21.15.62:0/2857582481 wait complete. 2024-09-12T00:01:53.338 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T00:01:53.338 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T00:01:53.339 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:01:53.566 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:53 smithi062 ceph-mon[30958]: pgmap v302: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 614 op/s 2024-09-12T00:01:53.566 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:53 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:01:53.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:53 smithi120 ceph-mon[35182]: pgmap v302: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 614 op/s 2024-09-12T00:01:53.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:53 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:01:53.626 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:01:53.627 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:01:53.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:53 smithi083 ceph-mon[34805]: pgmap v302: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 614 op/s 2024-09-12T00:01:53.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:53 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:01:53.839 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:54.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.430+0000 7fd51e606640 1 -- 172.21.15.62:0/127625945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5100964a0 msgr2=0x7fd5100a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:54.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.430+0000 7fd51e606640 1 --2- 172.21.15.62:0/127625945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5100964a0 0x7fd5100a2500 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd508009890 tx=0x7fd50802a380 comp rx=0 tx=0).stop 2024-09-12T00:01:54.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.435+0000 7fd51e606640 1 -- 172.21.15.62:0/127625945 shutdown_connections 2024-09-12T00:01:54.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.435+0000 7fd51e606640 1 --2- 172.21.15.62:0/127625945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5100964a0 0x7fd5100a2500 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:54.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.435+0000 7fd51e606640 1 --2- 172.21.15.62:0/127625945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd510095b00 0x7fd510095f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:54.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.435+0000 7fd51e606640 1 --2- 172.21.15.62:0/127625945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51009b630 0x7fd51009ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:54.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.435+0000 7fd51e606640 1 -- 172.21.15.62:0/127625945 >> 172.21.15.62:0/127625945 conn(0x7fd51000b3d0 msgr2=0x7fd51000b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:54.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.435+0000 7fd51e606640 1 -- 172.21.15.62:0/127625945 shutdown_connections 2024-09-12T00:01:54.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.437+0000 7fd51e606640 1 -- 172.21.15.62:0/127625945 wait complete. 2024-09-12T00:01:54.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.438+0000 7fd51e606640 1 Processor -- start 2024-09-12T00:01:54.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.438+0000 7fd51e606640 1 -- start start 2024-09-12T00:01:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.439+0000 7fd51e606640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd510095b00 0x7fd5100a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.439+0000 7fd51e606640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5100964a0 0x7fd5100a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.439+0000 7fd51e606640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd51009b630 0x7fd510136de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.440+0000 7fd51ce03640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5100964a0 0x7fd5100a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:54.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.440+0000 7fd51e606640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5100a5a30 con 0x7fd51009b630 2024-09-12T00:01:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.440+0000 7fd51e606640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5100a58b0 con 0x7fd510095b00 2024-09-12T00:01:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.440+0000 7fd51e606640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd5100a5bb0 con 0x7fd5100964a0 2024-09-12T00:01:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.440+0000 7fd51ce03640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5100964a0 0x7fd5100a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:45646/0 (socket says 172.21.15.62:45646) 2024-09-12T00:01:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.440+0000 7fd51ce03640 1 -- 172.21.15.62:0/1055966450 learned_addr learned my addr 172.21.15.62:0/1055966450 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.440+0000 7fd51d604640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd510095b00 0x7fd5100a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:54.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.441+0000 7fd51de05640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd51009b630 0x7fd510136de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:54.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.446+0000 7fd51de05640 1 -- 172.21.15.62:0/1055966450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5100964a0 msgr2=0x7fd5100a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:54.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.446+0000 7fd51de05640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5100964a0 0x7fd5100a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:54.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.446+0000 7fd51de05640 1 -- 172.21.15.62:0/1055966450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd510095b00 msgr2=0x7fd5100a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:54.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.447+0000 7fd51de05640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd510095b00 0x7fd5100a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:54.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.447+0000 7fd51de05640 1 -- 172.21.15.62:0/1055966450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5101374c0 con 0x7fd51009b630 2024-09-12T00:01:54.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.452+0000 7fd51de05640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd51009b630 0x7fd510136de0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd50802a970 tx=0x7fd5080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:54.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.453+0000 7fd51ce03640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5100964a0 0x7fd5100a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:01:54.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.453+0000 7fd51d604640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd510095b00 0x7fd5100a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:01:54.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.453+0000 7fd50e7fc640 1 -- 172.21.15.62:0/1055966450 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50803f070 con 0x7fd51009b630 2024-09-12T00:01:54.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.453+0000 7fd51e606640 1 -- 172.21.15.62:0/1055966450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd510137750 con 0x7fd51009b630 2024-09-12T00:01:54.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.453+0000 7fd51e606640 1 -- 172.21.15.62:0/1055966450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51013f090 con 0x7fd51009b630 2024-09-12T00:01:54.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.454+0000 7fd50e7fc640 1 -- 172.21.15.62:0/1055966450 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd508033e50 con 0x7fd51009b630 2024-09-12T00:01:54.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.455+0000 7fd50e7fc640 1 -- 172.21.15.62:0/1055966450 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50803f070 con 0x7fd51009b630 2024-09-12T00:01:54.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.459+0000 7fd503fff640 1 -- 172.21.15.62:0/1055966450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd510095f60 con 0x7fd51009b630 2024-09-12T00:01:54.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.464+0000 7fd50e7fc640 1 -- 172.21.15.62:0/1055966450 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd508042070 con 0x7fd51009b630 2024-09-12T00:01:54.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.465+0000 7fd50e7fc640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd5040a3100 0x7fd5040a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:54.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.465+0000 7fd50e7fc640 1 -- 172.21.15.62:0/1055966450 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fd50802ac80 con 0x7fd51009b630 2024-09-12T00:01:54.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.470+0000 7fd51d604640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd5040a3100 0x7fd5040a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:54.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.472+0000 7fd51d604640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd5040a3100 0x7fd5040a55c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd5180679d0 tx=0x7fd518067a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:54.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.477+0000 7fd50e7fc640 1 -- 172.21.15.62:0/1055966450 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5080bac30 con 0x7fd51009b630 2024-09-12T00:01:54.497 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:54.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.851+0000 7fd503fff640 1 -- 172.21.15.62:0/1055966450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5100035e0 con 0x7fd51009b630 2024-09-12T00:01:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.856+0000 7fd50e7fc640 1 -- 172.21.15.62:0/1055966450 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11911 (secure 0 0 0) 0x7fd5080304c0 con 0x7fd51009b630 2024-09-12T00:01:54.855 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:01:54.855 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":33,"btime":"2024-09-12T00:01:52:327104+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:01:52.081596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:01:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 33 2024-09-12T00:01:54.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 -- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd5040a3100 msgr2=0x7fd5040a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:54.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd5040a3100 0x7fd5040a55c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd5180679d0 tx=0x7fd518067a20 comp rx=0 tx=0).stop 2024-09-12T00:01:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 -- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd51009b630 msgr2=0x7fd510136de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd51009b630 0x7fd510136de0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd50802a970 tx=0x7fd5080042f0 comp rx=0 tx=0).stop 2024-09-12T00:01:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 -- 172.21.15.62:0/1055966450 shutdown_connections 2024-09-12T00:01:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd5040a3100 0x7fd5040a55c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd51009b630 0x7fd510136de0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5100964a0 0x7fd5100a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 --2- 172.21.15.62:0/1055966450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd510095b00 0x7fd5100a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 -- 172.21.15.62:0/1055966450 >> 172.21.15.62:0/1055966450 conn(0x7fd51000b3d0 msgr2=0x7fd5100076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 -- 172.21.15.62:0/1055966450 shutdown_connections 2024-09-12T00:01:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:54.866+0000 7fd503fff640 1 -- 172.21.15.62:0/1055966450 wait complete. 2024-09-12T00:01:55.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:55 smithi120 ceph-mon[35182]: pgmap v303: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 872 op/s 2024-09-12T00:01:55.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:55 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1055966450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:01:55.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:55 smithi083 ceph-mon[34805]: pgmap v303: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 872 op/s 2024-09-12T00:01:55.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:55 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1055966450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:01:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:55 smithi062 ceph-mon[30958]: pgmap v303: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 872 op/s 2024-09-12T00:01:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:55 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1055966450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:01:55.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.859+0000 7fc0025b0640 1 -- 172.21.15.62:0/3873968486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbffc1008d0 msgr2=0x7fbffc100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:55.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.859+0000 7fc0025b0640 1 --2- 172.21.15.62:0/3873968486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbffc1008d0 0x7fbffc100cb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009900 tx=0x7fbfec02a410 comp rx=0 tx=0).stop 2024-09-12T00:01:55.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.859+0000 7fc0025b0640 1 -- 172.21.15.62:0/3873968486 shutdown_connections 2024-09-12T00:01:55.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.859+0000 7fc0025b0640 1 --2- 172.21.15.62:0/3873968486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbffc10e480 0x7fbffc110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:55.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.859+0000 7fc0025b0640 1 --2- 172.21.15.62:0/3873968486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbffc101280 0x7fbffc10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:55.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.859+0000 7fc0025b0640 1 --2- 172.21.15.62:0/3873968486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbffc1008d0 0x7fbffc100cb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:55.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.859+0000 7fc0025b0640 1 -- 172.21.15.62:0/3873968486 >> 172.21.15.62:0/3873968486 conn(0x7fbffc0fc7a0 msgr2=0x7fbffc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:55.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.859+0000 7fc0025b0640 1 -- 172.21.15.62:0/3873968486 shutdown_connections 2024-09-12T00:01:55.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.859+0000 7fc0025b0640 1 -- 172.21.15.62:0/3873968486 wait complete. 2024-09-12T00:01:55.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fc0025b0640 1 Processor -- start 2024-09-12T00:01:55.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fc0025b0640 1 -- start start 2024-09-12T00:01:55.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fc0025b0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbffc1008d0 0x7fbffc1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:55.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fc0025b0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbffc101280 0x7fbffc1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fc0025b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbffc10e480 0x7fbffc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fc0025b0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc114200 con 0x7fbffc101280 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fc0025b0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc114080 con 0x7fbffc10e480 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fc0025b0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc114380 con 0x7fbffc1008d0 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fbffb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbffc101280 0x7fbffc1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fbffb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbffc101280 0x7fbffc1a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55426/0 (socket says 172.21.15.62:55426) 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fbffb7fe640 1 -- 172.21.15.62:0/1654891933 learned_addr learned my addr 172.21.15.62:0/1654891933 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fbffbfff640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbffc1008d0 0x7fbffc1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.860+0000 7fc000b4d640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbffc10e480 0x7fbffc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fbffb7fe640 1 -- 172.21.15.62:0/1654891933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbffc1008d0 msgr2=0x7fbffc1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fbffb7fe640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbffc1008d0 0x7fbffc1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fbffb7fe640 1 -- 172.21.15.62:0/1654891933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbffc10e480 msgr2=0x7fbffc19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:55.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fbffb7fe640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbffc10e480 0x7fbffc19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:55.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fbffb7fe640 1 -- 172.21.15.62:0/1654891933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc19acc0 con 0x7fbffc101280 2024-09-12T00:01:55.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fc000b4d640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbffc10e480 0x7fbffc19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:55.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fbffb7fe640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbffc101280 0x7fbffc1a0910 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fbff000b4a0 tx=0x7fbff000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:55.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fbff97fa640 1 -- 172.21.15.62:0/1654891933 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0014070 con 0x7fbffc101280 2024-09-12T00:01:55.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fbff97fa640 1 -- 172.21.15.62:0/1654891933 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff0010af0 con 0x7fbffc101280 2024-09-12T00:01:55.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fbff97fa640 1 -- 172.21.15.62:0/1654891933 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0017d10 con 0x7fbffc101280 2024-09-12T00:01:55.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fc0025b0640 1 -- 172.21.15.62:0/1654891933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc19afb0 con 0x7fbffc101280 2024-09-12T00:01:55.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.861+0000 7fc0025b0640 1 -- 172.21.15.62:0/1654891933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc1a72a0 con 0x7fbffc101280 2024-09-12T00:01:55.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.862+0000 7fc0025b0640 1 -- 172.21.15.62:0/1654891933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbffc100cb0 con 0x7fbffc101280 2024-09-12T00:01:55.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.864+0000 7fbff97fa640 1 -- 172.21.15.62:0/1654891933 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbff0017070 con 0x7fbffc101280 2024-09-12T00:01:55.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.865+0000 7fbff97fa640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 0x7fbfe40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:55.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.865+0000 7fbff97fa640 1 -- 172.21.15.62:0/1654891933 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fbff00ce6d0 con 0x7fbffc101280 2024-09-12T00:01:55.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.866+0000 7fbffbfff640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 0x7fbfe40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:55.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.867+0000 7fbffbfff640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 0x7fbfe40a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbfec000c00 tx=0x7fbfec0058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:55.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:55.872+0000 7fbff97fa640 1 -- 172.21.15.62:0/1654891933 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbff009adb0 con 0x7fbffc101280 2024-09-12T00:01:56.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.243+0000 7fc0025b0640 1 -- 172.21.15.62:0/1654891933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbffc109670 con 0x7fbffc101280 2024-09-12T00:01:56.248 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:01:56.248 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":33,"btime":"2024-09-12T00:01:52:327104+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:01:52.081596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:01:56.248 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 33 2024-09-12T00:01:56.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.249+0000 7fbff97fa640 1 -- 172.21.15.62:0/1654891933 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11911 (secure 0 0 0) 0x7fbff009f0b0 con 0x7fbffc101280 2024-09-12T00:01:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.255+0000 7fc0025b0640 1 -- 172.21.15.62:0/1654891933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 msgr2=0x7fbfe40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.255+0000 7fc0025b0640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 0x7fbfe40a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbfec000c00 tx=0x7fbfec0058d0 comp rx=0 tx=0).stop 2024-09-12T00:01:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.255+0000 7fc0025b0640 1 -- 172.21.15.62:0/1654891933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbffc101280 msgr2=0x7fbffc1a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.255+0000 7fc0025b0640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbffc101280 0x7fbffc1a0910 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fbff000b4a0 tx=0x7fbff000b960 comp rx=0 tx=0).stop 2024-09-12T00:01:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.257+0000 7fc0025b0640 1 -- 172.21.15.62:0/1654891933 shutdown_connections 2024-09-12T00:01:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.257+0000 7fc0025b0640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 0x7fbfe40a51b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.257+0000 7fc0025b0640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbffc10e480 0x7fbffc19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.257+0000 7fc0025b0640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbffc101280 0x7fbffc1a0910 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.257+0000 7fc0025b0640 1 --2- 172.21.15.62:0/1654891933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbffc1008d0 0x7fbffc1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.257+0000 7fc0025b0640 1 -- 172.21.15.62:0/1654891933 >> 172.21.15.62:0/1654891933 conn(0x7fbffc0fc7a0 msgr2=0x7fbffc10f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.260+0000 7fc0025b0640 1 -- 172.21.15.62:0/1654891933 shutdown_connections 2024-09-12T00:01:56.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:56.261+0000 7fc0025b0640 1 -- 172.21.15.62:0/1654891933 wait complete. 2024-09-12T00:01:56.510 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:56 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1654891933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:01:56.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:56 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1654891933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:01:56.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:56 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1654891933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:01:56.893 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T00:01:56.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T00:01:57.317 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:01:57.430 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:57.447 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 555.0 s, 2016.2 tps, lat 49.365 ms stddev 99.179 2024-09-12T00:01:57.571 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:57 smithi062 ceph-mon[30958]: pgmap v304: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 590 op/s 2024-09-12T00:01:57.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:57 smithi120 ceph-mon[35182]: pgmap v304: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 590 op/s 2024-09-12T00:01:57.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:57 smithi083 ceph-mon[34805]: pgmap v304: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 590 op/s 2024-09-12T00:01:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.054+0000 7f552c82f640 1 -- 172.21.15.62:0/2013010793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55281012a0 msgr2=0x7f5528105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.054+0000 7f552c82f640 1 --2- 172.21.15.62:0/2013010793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55281012a0 0x7f5528105770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5510009860 tx=0x7f551002a3b0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.055+0000 7f552c82f640 1 -- 172.21.15.62:0/2013010793 shutdown_connections 2024-09-12T00:01:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.055+0000 7f552c82f640 1 --2- 172.21.15.62:0/2013010793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5528105d90 0x7f5528106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.055+0000 7f552c82f640 1 --2- 172.21.15.62:0/2013010793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55281012a0 0x7f5528105770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.055+0000 7f552c82f640 1 --2- 172.21.15.62:0/2013010793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5528100980 0x7f5528100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.055+0000 7f552c82f640 1 -- 172.21.15.62:0/2013010793 >> 172.21.15.62:0/2013010793 conn(0x7f55280fc610 msgr2=0x7f55280fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.056+0000 7f552c82f640 1 -- 172.21.15.62:0/2013010793 shutdown_connections 2024-09-12T00:01:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.061+0000 7f552c82f640 1 -- 172.21.15.62:0/2013010793 wait complete. 2024-09-12T00:01:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.061+0000 7f552c82f640 1 Processor -- start 2024-09-12T00:01:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.062+0000 7f552c82f640 1 -- start start 2024-09-12T00:01:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.062+0000 7f552c82f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5528100980 0x7f5528073940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.062+0000 7f552c82f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55281012a0 0x7f5528073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.062+0000 7f552c82f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5528105d90 0x7f5528071c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.062+0000 7f552c82f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5528118390 con 0x7f5528100980 2024-09-12T00:01:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.062+0000 7f552c82f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5528118210 con 0x7f5528105d90 2024-09-12T00:01:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.062+0000 7f552c82f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5528118510 con 0x7f55281012a0 2024-09-12T00:01:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.063+0000 7f5527fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5528105d90 0x7f5528071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.067+0000 7f5527fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5528105d90 0x7f5528071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:53436/0 (socket says 172.21.15.62:53436) 2024-09-12T00:01:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.067+0000 7f5527fff640 1 -- 172.21.15.62:0/533370146 learned_addr learned my addr 172.21.15.62:0/533370146 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.067+0000 7f55277fe640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5528100980 0x7f5528073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.067+0000 7f551effd640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55281012a0 0x7f5528073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.068+0000 7f551effd640 1 -- 172.21.15.62:0/533370146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5528105d90 msgr2=0x7f5528071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.068+0000 7f551effd640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5528105d90 0x7f5528071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.068+0000 7f551effd640 1 -- 172.21.15.62:0/533370146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5528100980 msgr2=0x7f5528073940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.068+0000 7f551effd640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5528100980 0x7f5528073940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.068+0000 7f551effd640 1 -- 172.21.15.62:0/533370146 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5528072510 con 0x7f55281012a0 2024-09-12T00:01:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.070+0000 7f551effd640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55281012a0 0x7f5528073e80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5510035040 tx=0x7f5510030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.071+0000 7f55257fa640 1 -- 172.21.15.62:0/533370146 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551003f020 con 0x7f55281012a0 2024-09-12T00:01:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.071+0000 7f55257fa640 1 -- 172.21.15.62:0/533370146 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5510004340 con 0x7f55281012a0 2024-09-12T00:01:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.072+0000 7f55257fa640 1 -- 172.21.15.62:0/533370146 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510037040 con 0x7f55281012a0 2024-09-12T00:01:58.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.076+0000 7f552c82f640 1 -- 172.21.15.62:0/533370146 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5528072740 con 0x7f55281012a0 2024-09-12T00:01:58.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.076+0000 7f552c82f640 1 -- 172.21.15.62:0/533370146 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5528112020 con 0x7f55281012a0 2024-09-12T00:01:58.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.083+0000 7f552fe6a740 1 Processor -- start 2024-09-12T00:01:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.084+0000 7f55257fa640 1 -- 172.21.15.62:0/533370146 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5510042020 con 0x7f55281012a0 2024-09-12T00:01:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.084+0000 7f552fe6a740 1 -- start start 2024-09-12T00:01:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.084+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x5599703fee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.084+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703ff340 0x5599704096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.084+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x55997040bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.084+0000 7f552fe6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559970363160 con 0x5599703fea20 2024-09-12T00:01:58.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.084+0000 7f552fe6a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55997030a720 con 0x559970409c00 2024-09-12T00:01:58.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.084+0000 7f552fe6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55997035f120 con 0x5599703ff340 2024-09-12T00:01:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.085+0000 7f55257fa640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55140a2cd0 0x7f55140a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.085+0000 7f55257fa640 1 -- 172.21.15.62:0/533370146 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f55100ee5c0 con 0x7f55281012a0 2024-09-12T00:01:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.086+0000 7f55277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703ff340 0x5599704096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.086+0000 7f551effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x55997040bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.087+0000 7f551effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x55997040bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:53438/0 (socket says 172.21.15.62:53438) 2024-09-12T00:01:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.087+0000 7f551effd640 1 -- 172.21.15.62:0/3963212107 learned_addr learned my addr 172.21.15.62:0/3963212107 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.087+0000 7f5527fff640 1 --2- 172.21.15.62:0/3963212107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x5599703fee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.088+0000 7f5527fff640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55140a2cd0 0x7f55140a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.090+0000 7f5527fff640 1 -- 172.21.15.62:0/3963212107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703ff340 msgr2=0x5599704096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.090+0000 7f5527fff640 1 --2- 172.21.15.62:0/3963212107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703ff340 0x5599704096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.090+0000 7f5527fff640 1 -- 172.21.15.62:0/3963212107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 msgr2=0x55997040bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.090+0000 7f5527fff640 1 --2- 172.21.15.62:0/3963212107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x55997040bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.090+0000 7f5527fff640 1 -- 172.21.15.62:0/3963212107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599700d4ba0 con 0x5599703fea20 2024-09-12T00:01:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.090+0000 7f5527fff640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55140a2cd0 0x7f55140a5190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f552000bfd0 tx=0x7f552000b840 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.091+0000 7f5527fff640 1 --2- 172.21.15.62:0/3963212107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x5599703fee00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5520005fd0 tx=0x7f552001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f073fe2cdcb5ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.091+0000 7f551dffb640 1 -- 172.21.15.62:0/3963212107 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520004230 con 0x5599703fea20 2024-09-12T00:01:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.091+0000 7f551dffb640 1 -- 172.21.15.62:0/3963212107 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55200043d0 con 0x5599703fea20 2024-09-12T00:01:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.091+0000 7f551dffb640 1 -- 172.21.15.62:0/3963212107 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520004230 con 0x5599703fea20 2024-09-12T00:01:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.093+0000 7f552fe6a740 1 -- 172.21.15.62:0/3963212107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 msgr2=0x5599703fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.093+0000 7f552fe6a740 1 --2- 172.21.15.62:0/3963212107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x5599703fee00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5520005fd0 tx=0x7f552001e000 comp rx=0 tx=0).stop 2024-09-12T00:01:58.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.093+0000 7f552fe6a740 1 -- 172.21.15.62:0/3963212107 shutdown_connections 2024-09-12T00:01:58.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.093+0000 7f552fe6a740 1 --2- 172.21.15.62:0/3963212107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x55997040bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.093+0000 7f552fe6a740 1 --2- 172.21.15.62:0/3963212107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703ff340 0x5599704096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.093+0000 7f552fe6a740 1 --2- 172.21.15.62:0/3963212107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x5599703fee00 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.093+0000 7f552fe6a740 1 -- 172.21.15.62:0/3963212107 >> 172.21.15.62:0/3963212107 conn(0x5599703fafb0 msgr2=0x5599703fb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.095+0000 7f552fe6a740 1 -- 172.21.15.62:0/3963212107 shutdown_connections 2024-09-12T00:01:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.097+0000 7f552fe6a740 1 -- 172.21.15.62:0/3963212107 wait complete. 2024-09-12T00:01:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.097+0000 7f552fe6a740 1 -- start start 2024-09-12T00:01:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.097+0000 7f552fe6a740 1 Processor -- start 2024-09-12T00:01:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.098+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x559970406aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.098+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703ff340 0x559970406fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.098+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x559970407520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.098+0000 7f552fe6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55997040c7d0 con 0x5599703fea20 2024-09-12T00:01:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.098+0000 7f552fe6a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5599702eb900 con 0x559970409c00 2024-09-12T00:01:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.098+0000 7f552fe6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55997040c950 con 0x5599703ff340 2024-09-12T00:01:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.098+0000 7f552fe6a740 10 client.0 _ll_register_callbacks cb 0x5599702e9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f55277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x559970406aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f5527fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x559970407520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f55277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x559970406aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55468/0 (socket says 172.21.15.62:55468) 2024-09-12T00:01:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f5527fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x559970407520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:53440/0 (socket says 172.21.15.62:53440) 2024-09-12T00:01:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f55277fe640 1 -- 172.21.15.62:0/4181808537 learned_addr learned my addr 172.21.15.62:0/4181808537 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f5527fff640 1 -- 172.21.15.62:0/4181808537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703ff340 msgr2=0x559970406fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:01:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f5527fff640 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703ff340 0x559970406fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f5527fff640 1 -- 172.21.15.62:0/4181808537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 msgr2=0x559970406aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f5527fff640 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x559970406aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f5527fff640 1 -- 172.21.15.62:0/4181808537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55997032bc50 con 0x559970409c00 2024-09-12T00:01:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.099+0000 7f55277fe640 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x559970406aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.100+0000 7f5506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:01:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.100+0000 7f5527fff640 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x559970407520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5520012e60 tx=0x7f5520016040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771b81c3815c4f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.100+0000 7f5506ffd640 1 -- 172.21.15.62:0/4181808537 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520037060 con 0x559970409c00 2024-09-12T00:01:58.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.100+0000 7f552fe6a740 1 -- 172.21.15.62:0/4181808537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55200043d0 con 0x559970409c00 2024-09-12T00:01:58.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.100+0000 7f5506ffd640 1 -- 172.21.15.62:0/4181808537 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f5520004030 con 0x559970409c00 2024-09-12T00:01:58.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.100+0000 7f5506ffd640 1 -- 172.21.15.62:0/4181808537 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5520037060 con 0x559970409c00 2024-09-12T00:01:58.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.100+0000 7f552fe6a740 10 client.26675 fetch_fsmap learned FSMap version 33 2024-09-12T00:01:58.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.100+0000 7f552fe6a740 1 -- 172.21.15.62:0/4181808537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559970418460 con 0x559970409c00 2024-09-12T00:01:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.101+0000 7f5506ffd640 1 -- 172.21.15.62:0/4181808537 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55200043d0 con 0x559970409c00 2024-09-12T00:01:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.102+0000 7f5506ffd640 1 -- 172.21.15.62:0/4181808537 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5520024020 con 0x559970409c00 2024-09-12T00:01:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.102+0000 7f552fe6a740 10 client.26675 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:01:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.102+0000 7f552fe6a740 10 client.26675 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:01:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.102+0000 7f552fe6a740 20 client.26675 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.102+0000 7f552fe6a740 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559970419bd0 0x55997041c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.102+0000 7f552fe6a740 4 client.26675 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:01:58.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.102+0000 7f552fe6a740 1 -- 172.21.15.62:0/4181808537 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559970370890 con 0x559970419bd0 2024-09-12T00:01:58.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.105+0000 7f55277fe640 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559970419bd0 0x55997041c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.110+0000 7f5506ffd640 10 client.26675 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.110+0000 7f55277fe640 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559970419bd0 0x55997041c070 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e61ec82c468b15 server_cookie=1cd7e9cc530d99da in_seq=0 out_seq=0 2024-09-12T00:01:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f5506ffd640 1 -- 172.21.15.62:0/4181808537 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559970370890 con 0x559970419bd0 2024-09-12T00:01:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f5506ffd640 10 client.26675 handle_command_reply: tid=0 2024-09-12T00:01:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f552fe6a740 1 client.26675 shutdown 2024-09-12T00:01:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f552fe6a740 1 -- 172.21.15.62:0/4181808537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 msgr2=0x559970407520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f552fe6a740 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x559970407520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5520012e60 tx=0x7f5520016040 comp rx=0 tx=0).stop 2024-09-12T00:01:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f552fe6a740 1 -- 172.21.15.62:0/4181808537 shutdown_connections 2024-09-12T00:01:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f552fe6a740 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559970419bd0 0x55997041c070 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f552fe6a740 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x559970407520 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f552fe6a740 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703ff340 0x559970406fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f552fe6a740 1 --2- 172.21.15.62:0/4181808537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703fea20 0x559970406aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f552fe6a740 1 -- 172.21.15.62:0/4181808537 >> 172.21.15.62:0/4181808537 conn(0x5599703fafb0 msgr2=0x559970400ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.111+0000 7f5506ffd640 0 client.26675 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.112+0000 7f552fe6a740 1 -- 172.21.15.62:0/4181808537 shutdown_connections 2024-09-12T00:01:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.112+0000 7f552fe6a740 1 -- 172.21.15.62:0/4181808537 wait complete. 2024-09-12T00:01:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.112+0000 7f552fe6a740 20 client.26675 trim_cache size 0 max 16384 2024-09-12T00:01:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:01:58.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.142+0000 7f552fe6a740 1 Processor -- start 2024-09-12T00:01:58.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.142+0000 7f552fe6a740 1 -- start start 2024-09-12T00:01:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.143+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599703fea20 0x559970406210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.143+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703ff340 0x559970421980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.143+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559970409c00 0x559970421ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.143+0000 7f552fe6a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55281c7530 con 0x5599703ff340 2024-09-12T00:01:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.143+0000 7f552fe6a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55281c4dd0 con 0x5599703fea20 2024-09-12T00:01:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.143+0000 7f552fe6a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55281c2c20 con 0x559970409c00 2024-09-12T00:01:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.143+0000 7f551effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703ff340 0x559970421980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.143+0000 7f551effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703ff340 0x559970421980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55482/0 (socket says 172.21.15.62:55482) 2024-09-12T00:01:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.143+0000 7f551effd640 1 -- 172.21.15.62:0/3631415388 learned_addr learned my addr 172.21.15.62:0/3631415388 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.143+0000 7f55277fe640 1 --2- 172.21.15.62:0/3631415388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599703fea20 0x559970406210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.144+0000 7f5527fff640 1 --2- 172.21.15.62:0/3631415388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559970409c00 0x559970421ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.144+0000 7f55277fe640 1 -- 172.21.15.62:0/3631415388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559970409c00 msgr2=0x559970421ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.144+0000 7f55277fe640 1 --2- 172.21.15.62:0/3631415388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559970409c00 0x559970421ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.144+0000 7f55277fe640 1 -- 172.21.15.62:0/3631415388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703ff340 msgr2=0x559970421980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.144+0000 7f55277fe640 1 --2- 172.21.15.62:0/3631415388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703ff340 0x559970421980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.144+0000 7f55277fe640 1 -- 172.21.15.62:0/3631415388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599703fade0 con 0x5599703fea20 2024-09-12T00:01:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.144+0000 7f55277fe640 1 --2- 172.21.15.62:0/3631415388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599703fea20 0x559970406210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5518005fd0 tx=0x7f5518009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9d219121993372e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.145+0000 7f551d7fa640 1 -- 172.21.15.62:0/3631415388 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518003970 con 0x5599703fea20 2024-09-12T00:01:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.145+0000 7f551d7fa640 1 -- 172.21.15.62:0/3631415388 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f551800f8f0 con 0x5599703fea20 2024-09-12T00:01:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.145+0000 7f551d7fa640 1 -- 172.21.15.62:0/3631415388 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518004440 con 0x5599703fea20 2024-09-12T00:01:58.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.146+0000 7f552fe6a740 1 -- 172.21.15.62:0/3631415388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599703fea20 msgr2=0x559970406210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.146+0000 7f552fe6a740 1 --2- 172.21.15.62:0/3631415388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599703fea20 0x559970406210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5518005fd0 tx=0x7f5518009500 comp rx=0 tx=0).stop 2024-09-12T00:01:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.146+0000 7f552fe6a740 1 -- 172.21.15.62:0/3631415388 shutdown_connections 2024-09-12T00:01:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.146+0000 7f552fe6a740 1 --2- 172.21.15.62:0/3631415388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559970409c00 0x559970421ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.146+0000 7f552fe6a740 1 --2- 172.21.15.62:0/3631415388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703ff340 0x559970421980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.146+0000 7f552fe6a740 1 --2- 172.21.15.62:0/3631415388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599703fea20 0x559970406210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.146+0000 7f552fe6a740 1 -- 172.21.15.62:0/3631415388 >> 172.21.15.62:0/3631415388 conn(0x5599703fafb0 msgr2=0x559970405370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.146+0000 7f552fe6a740 1 -- 172.21.15.62:0/3631415388 shutdown_connections 2024-09-12T00:01:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.146+0000 7f552fe6a740 1 -- 172.21.15.62:0/3631415388 wait complete. 2024-09-12T00:01:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.147+0000 7f552fe6a740 1 -- start start 2024-09-12T00:01:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.147+0000 7f552fe6a740 1 Processor -- start 2024-09-12T00:01:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.148+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703fea20 0x55997040cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.148+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703ff340 0x55997040d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.148+0000 7f552fe6a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x559970482940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.148+0000 7f55277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703fea20 0x55997040cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.148+0000 7f55277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703fea20 0x55997040cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:45690/0 (socket says 172.21.15.62:45690) 2024-09-12T00:01:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.148+0000 7f55277fe640 1 -- 172.21.15.62:0/2114206979 learned_addr learned my addr 172.21.15.62:0/2114206979 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.148+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55281c3290 con 0x5599703ff340 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.148+0000 7f5527fff640 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x559970482940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.148+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55281c4700 con 0x559970409c00 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.148+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55281c3960 con 0x5599703fea20 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.149+0000 7f55277fe640 1 -- 172.21.15.62:0/2114206979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 msgr2=0x559970482940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.149+0000 7f55277fe640 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x559970482940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.149+0000 7f55277fe640 1 -- 172.21.15.62:0/2114206979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703ff340 msgr2=0x55997040d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.149+0000 7f55277fe640 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703ff340 0x55997040d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.149+0000 7f55277fe640 1 -- 172.21.15.62:0/2114206979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559970482e80 con 0x5599703fea20 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.149+0000 7f55277fe640 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703fea20 0x55997040cd70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5518012390 tx=0x7f55180123c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d717c5c8d7187439 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.149+0000 7f552fe6a740 10 client.0 _ll_register_callbacks cb 0x7f552000e550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.149+0000 7f5506ffd640 10 client.26602 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:01:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.149+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f551800f8f0 con 0x5599703fea20 2024-09-12T00:01:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.149+0000 7f5506ffd640 1 -- 172.21.15.62:0/2114206979 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518004ab0 con 0x5599703fea20 2024-09-12T00:01:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.150+0000 7f5506ffd640 1 -- 172.21.15.62:0/2114206979 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f5518004c50 con 0x5599703fea20 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.150+0000 7f5506ffd640 1 -- 172.21.15.62:0/2114206979 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5518038050 con 0x5599703fea20 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.150+0000 7f552fe6a740 10 client.26602 fetch_fsmap learned FSMap version 33 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.150+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=33}) -- 0x559970483ed0 con 0x5599703fea20 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.150+0000 7f5506ffd640 1 -- 172.21.15.62:0/2114206979 <== mon.1 v2:172.21.15.120:3300/0 4 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5518025030 con 0x5599703fea20 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.151+0000 7f552fe6a740 10 client.26602 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.151+0000 7f552fe6a740 10 client.26602 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.151+0000 7f552fe6a740 20 client.26602 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.151+0000 7f552fe6a740 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5599704844e0 0x559970486940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.151+0000 7f552fe6a740 4 client.26602 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.151+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x7f551800f8f0 con 0x5599704844e0 2024-09-12T00:01:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.152+0000 7f551effd640 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5599704844e0 0x559970486940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.153+0000 7f551effd640 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5599704844e0 0x559970486940 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71cc26bc5cd7e85 server_cookie=5b713dbf61563c40 in_seq=0 out_seq=0 2024-09-12T00:01:58.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.153+0000 7f5506ffd640 10 client.26602 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:58.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f5506ffd640 1 -- 172.21.15.62:0/2114206979 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f551800f8f0 con 0x5599704844e0 2024-09-12T00:01:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f5506ffd640 10 client.26602 handle_command_reply: tid=0 2024-09-12T00:01:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f552fe6a740 1 client.26602 shutdown 2024-09-12T00:01:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703fea20 msgr2=0x55997040cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f552fe6a740 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703fea20 0x55997040cd70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5518012390 tx=0x7f55180123c0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 shutdown_connections 2024-09-12T00:01:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f552fe6a740 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5599704844e0 0x559970486940 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f552fe6a740 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559970409c00 0x559970482940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f5506ffd640 0 client.26602 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:01:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f552fe6a740 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599703ff340 0x55997040d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f552fe6a740 1 --2- 172.21.15.62:0/2114206979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599703fea20 0x55997040cd70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.395+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 >> 172.21.15.62:0/2114206979 conn(0x5599703fafb0 msgr2=0x559970406a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 shutdown_connections 2024-09-12T00:01:58.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552fe6a740 1 -- 172.21.15.62:0/2114206979 wait complete. 2024-09-12T00:01:58.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552fe6a740 20 client.26602 trim_cache size 0 max 16384 2024-09-12T00:01:58.396 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 33, 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 38, 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "494e38e3": { 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 38, 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:01:58.397 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 142.6, 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:01:58.398 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:01:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552c82f640 1 -- 172.21.15.62:0/533370146 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55140a2cd0 msgr2=0x7f55140a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552c82f640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55140a2cd0 0x7f55140a5190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f552000bfd0 tx=0x7f552000b840 comp rx=0 tx=0).stop 2024-09-12T00:01:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552c82f640 1 -- 172.21.15.62:0/533370146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55281012a0 msgr2=0x7f5528073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552c82f640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55281012a0 0x7f5528073e80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5510035040 tx=0x7f5510030430 comp rx=0 tx=0).stop 2024-09-12T00:01:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552c82f640 1 -- 172.21.15.62:0/533370146 shutdown_connections 2024-09-12T00:01:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552c82f640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f55140a2cd0 0x7f55140a5190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552c82f640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5528105d90 0x7f5528071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552c82f640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55281012a0 0x7f5528073e80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.396+0000 7f552c82f640 1 --2- 172.21.15.62:0/533370146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5528100980 0x7f5528073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.397+0000 7f552c82f640 1 -- 172.21.15.62:0/533370146 >> 172.21.15.62:0/533370146 conn(0x7f55280fc610 msgr2=0x7f55280fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.397+0000 7f552c82f640 1 -- 172.21.15.62:0/533370146 shutdown_connections 2024-09-12T00:01:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.397+0000 7f552c82f640 1 -- 172.21.15.62:0/533370146 wait complete. 2024-09-12T00:01:58.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.844+0000 7fa6deee5640 1 -- 172.21.15.62:0/435857549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d8102570 msgr2=0x7fa6d81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.844+0000 7fa6deee5640 1 --2- 172.21.15.62:0/435857549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d8102570 0x7fa6d81029b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009900 tx=0x7fa6c402a480 comp rx=0 tx=0).stop 2024-09-12T00:01:58.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.844+0000 7fa6deee5640 1 -- 172.21.15.62:0/435857549 shutdown_connections 2024-09-12T00:01:58.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.844+0000 7fa6deee5640 1 --2- 172.21.15.62:0/435857549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8102ef0 0x7fa6d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.844+0000 7fa6deee5640 1 --2- 172.21.15.62:0/435857549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d8102570 0x7fa6d81029b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.844+0000 7fa6deee5640 1 --2- 172.21.15.62:0/435857549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6d8100e20 0x7fa6d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.844+0000 7fa6deee5640 1 -- 172.21.15.62:0/435857549 >> 172.21.15.62:0/435857549 conn(0x7fa6d80fc7d0 msgr2=0x7fa6d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.844+0000 7fa6deee5640 1 -- 172.21.15.62:0/435857549 shutdown_connections 2024-09-12T00:01:58.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.844+0000 7fa6deee5640 1 -- 172.21.15.62:0/435857549 wait complete. 2024-09-12T00:01:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6deee5640 1 Processor -- start 2024-09-12T00:01:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6deee5640 1 -- start start 2024-09-12T00:01:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6deee5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d8100e20 0x7fa6d8197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6deee5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6d8102570 0x7fa6d8197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6deee5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8102ef0 0x7fa6d8197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6deee5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8110f80 con 0x7fa6d8100e20 2024-09-12T00:01:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6deee5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8110e00 con 0x7fa6d8102570 2024-09-12T00:01:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6deee5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8111100 con 0x7fa6d8102ef0 2024-09-12T00:01:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6dcc81640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d8100e20 0x7fa6d8197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6dcc81640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d8100e20 0x7fa6d8197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55510/0 (socket says 172.21.15.62:55510) 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6dcc81640 1 -- 172.21.15.62:0/4268504212 learned_addr learned my addr 172.21.15.62:0/4268504212 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6d7fff640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6d8102570 0x7fa6d8197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.845+0000 7fa6dd482640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8102ef0 0x7fa6d8197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6dcc81640 1 -- 172.21.15.62:0/4268504212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8102ef0 msgr2=0x7fa6d8197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8102ef0 0x7fa6d8197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6dcc81640 1 -- 172.21.15.62:0/4268504212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6d8102570 msgr2=0x7fa6d8197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6d8102570 0x7fa6d8197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6dcc81640 1 -- 172.21.15.62:0/4268504212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d81a69b0 con 0x7fa6d8100e20 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d8100e20 0x7fa6d8197140 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00e9f0 tx=0x7fa6cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6d5ffb640 1 -- 172.21.15.62:0/4268504212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00cd80 con 0x7fa6d8100e20 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6d5ffb640 1 -- 172.21.15.62:0/4268504212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc010910 con 0x7fa6d8100e20 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6deee5640 1 -- 172.21.15.62:0/4268504212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d81a6ca0 con 0x7fa6d8100e20 2024-09-12T00:01:58.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6d5ffb640 1 -- 172.21.15.62:0/4268504212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc018990 con 0x7fa6d8100e20 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.846+0000 7fa6deee5640 1 -- 172.21.15.62:0/4268504212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d81a71b0 con 0x7fa6d8100e20 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6d5ffb640 1 -- 172.21.15.62:0/4268504212 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa6cc010ac0 con 0x7fa6d8100e20 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6e12be740 1 Processor -- start 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6e12be740 1 -- start start 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515b0b40 0x55af515af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515af5d0 0x55af515afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6e12be740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55af515152f0 con 0x55af515af5d0 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6e12be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55af514bccf0 con 0x55af515b0b40 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6e12be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55af515112b0 con 0x55af515ba370 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6d5ffb640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6c80a2cf0 0x7fa6c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6dcc81640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515af5d0 0x55af515afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515bc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:45718/0 (socket says 172.21.15.62:45718) 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6d7fff640 1 -- 172.21.15.62:0/1211761 learned_addr learned my addr 172.21.15.62:0/1211761 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6dcc81640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515af5d0 0x55af515afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55520/0 (socket says 172.21.15.62:55520) 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.849+0000 7fa6dd482640 1 --2- 172.21.15.62:0/1211761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515b0b40 0x55af515af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6dd482640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6c80a2cf0 0x7fa6c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6d5ffb640 1 -- 172.21.15.62:0/4268504212 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fa6cc0cf7a0 con 0x7fa6d8100e20 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6d7fff640 1 -- 172.21.15.62:0/1211761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515b0b40 msgr2=0x55af515af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6d7fff640 1 --2- 172.21.15.62:0/1211761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515b0b40 0x55af515af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6d7fff640 1 -- 172.21.15.62:0/1211761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515af5d0 msgr2=0x55af515afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6d7fff640 1 --2- 172.21.15.62:0/1211761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515af5d0 0x55af515afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6d7fff640 1 -- 172.21.15.62:0/1211761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af514a13a0 con 0x55af515ba370 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6dd482640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6c80a2cf0 0x7fa6c80a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009990 tx=0x7fa6d00096a0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6d7fff640 1 --2- 172.21.15.62:0/1211761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515bc760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa6c402a9a0 tx=0x7fa6c4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d3b6b26db024de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6c2ffd640 1 -- 172.21.15.62:0/1211761 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4035030 con 0x55af515ba370 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6c2ffd640 1 -- 172.21.15.62:0/1211761 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4040070 con 0x55af515ba370 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.850+0000 7fa6c2ffd640 1 -- 172.21.15.62:0/1211761 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4031960 con 0x55af515ba370 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.851+0000 7fa6e12be740 1 -- 172.21.15.62:0/1211761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 msgr2=0x55af515bc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.851+0000 7fa6e12be740 1 --2- 172.21.15.62:0/1211761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515bc760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa6c402a9a0 tx=0x7fa6c4005e80 comp rx=0 tx=0).stop 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.851+0000 7fa6e12be740 1 -- 172.21.15.62:0/1211761 shutdown_connections 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.851+0000 7fa6e12be740 1 --2- 172.21.15.62:0/1211761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515bc760 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.851+0000 7fa6e12be740 1 --2- 172.21.15.62:0/1211761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515af5d0 0x55af515afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.851+0000 7fa6e12be740 1 --2- 172.21.15.62:0/1211761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515b0b40 0x55af515af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.851+0000 7fa6e12be740 1 -- 172.21.15.62:0/1211761 >> 172.21.15.62:0/1211761 conn(0x55af51578310 msgr2=0x55af515abc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.851+0000 7fa6e12be740 1 -- 172.21.15.62:0/1211761 shutdown_connections 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.851+0000 7fa6e12be740 1 -- 172.21.15.62:0/1211761 wait complete. 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.851+0000 7fa6e12be740 1 -- start start 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6e12be740 1 Processor -- start 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515b0b40 0x55af515b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515ba370 0x55af515b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6e12be740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55af515bd010 con 0x55af515ba370 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6e12be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55af514d0060 con 0x55af515af5d0 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6e12be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55af515bd190 con 0x55af515b0b40 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6e12be740 10 client.0 _ll_register_callbacks cb 0x55af512238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6dd482640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515ba370 0x55af515b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6dd482640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515ba370 0x55af515b7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55522/0 (socket says 172.21.15.62:55522) 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515b0b40 0x55af515b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6dd482640 1 -- 172.21.15.62:0/3279629394 learned_addr learned my addr 172.21.15.62:0/3279629394 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6dd482640 1 -- 172.21.15.62:0/3279629394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515b0b40 msgr2=0x55af515b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6dd482640 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515b0b40 0x55af515b6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.852+0000 7fa6dd482640 1 -- 172.21.15.62:0/3279629394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 msgr2=0x55af515b9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6dd482640 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515b9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6dd482640 1 -- 172.21.15.62:0/3279629394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af512863c0 con 0x55af515ba370 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6dd482640 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515ba370 0x55af515b7530 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000bbf0 tx=0x7fa6d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffff0c698338c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3279629394 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0010d60 con 0x55af515ba370 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3279629394 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fa6d001e3f0 con 0x55af515ba370 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6e12be740 1 -- 172.21.15.62:0/3279629394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c4040070 con 0x55af515ba370 2024-09-12T00:01:58.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3279629394 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa6d001e5f0 con 0x55af515ba370 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6e12be740 10 client.17121 fetch_fsmap learned FSMap version 33 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6e12be740 1 -- 172.21.15.62:0/3279629394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55af515c8af0 con 0x55af515ba370 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3279629394 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4040070 con 0x55af515ba370 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3279629394 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa6d0031840 con 0x55af515ba370 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6e12be740 10 client.17121 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.853+0000 7fa6e12be740 10 client.17121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.854+0000 7fa6e12be740 20 client.17121 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.854+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55af515ca000 0x55af515cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.854+0000 7fa6e12be740 4 client.17121 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.854+0000 7fa6e12be740 1 -- 172.21.15.62:0/3279629394 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af51522860 con 0x55af515ca000 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.854+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55af515ca000 0x55af515cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.854+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55af515ca000 0x55af515cc450 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c67917c1562bfcdc server_cookie=942ee78ed60f7fe1 in_seq=0 out_seq=0 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.854+0000 7fa6b7fff640 10 client.17121 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:01:58.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3279629394 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af51522860 con 0x55af515ca000 2024-09-12T00:01:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6b7fff640 10 client.17121 handle_command_reply: tid=0 2024-09-12T00:01:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 client.17121 shutdown 2024-09-12T00:01:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 -- 172.21.15.62:0/3279629394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515ba370 msgr2=0x55af515b7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515ba370 0x55af515b7530 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000bbf0 tx=0x7fa6d0004a40 comp rx=0 tx=0).stop 2024-09-12T00:01:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 -- 172.21.15.62:0/3279629394 shutdown_connections 2024-09-12T00:01:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55af515ca000 0x55af515cc450 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6b7fff640 0 client.17121 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:01:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515ba370 0x55af515b7530 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515b0b40 0x55af515b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3279629394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 -- 172.21.15.62:0/3279629394 >> 172.21.15.62:0/3279629394 conn(0x55af51578310 msgr2=0x55af515b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 -- 172.21.15.62:0/3279629394 shutdown_connections 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 1 -- 172.21.15.62:0/3279629394 wait complete. 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.855+0000 7fa6e12be740 20 client.17121 trim_cache size 0 max 16384 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.868+0000 7fa6e12be740 1 Processor -- start 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.868+0000 7fa6e12be740 1 -- start start 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6e12be740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81b22e0 con 0x55af515b0b40 2024-09-12T00:01:58.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6e12be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81afb80 con 0x55af515af5d0 2024-09-12T00:01:58.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6e12be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ad9f0 con 0x55af515ba370 2024-09-12T00:01:58.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55530/0 (socket says 172.21.15.62:55530) 2024-09-12T00:01:58.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6d7fff640 1 -- 172.21.15.62:0/1908745742 learned_addr learned my addr 172.21.15.62:0/1908745742 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:58.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/1908745742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6dd482640 1 --2- 172.21.15.62:0/1908745742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6dcc81640 1 -- 172.21.15.62:0/1908745742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 msgr2=0x55af515b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/1908745742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6dcc81640 1 -- 172.21.15.62:0/1908745742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 msgr2=0x55af515b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/1908745742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6dcc81640 1 -- 172.21.15.62:0/1908745742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af515ba020 con 0x55af515af5d0 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6d7fff640 1 --2- 172.21.15.62:0/1908745742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/1908745742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515b8f20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0cce90 tx=0x7fa6cc097fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef24cc64f5729a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.869+0000 7fa6c27fc640 1 -- 172.21.15.62:0/1908745742 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc026040 con 0x55af515af5d0 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6c27fc640 1 -- 172.21.15.62:0/1908745742 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc0b1030 con 0x55af515af5d0 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6c27fc640 1 -- 172.21.15.62:0/1908745742 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0cfdf0 con 0x55af515af5d0 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6e12be740 1 -- 172.21.15.62:0/1908745742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 msgr2=0x55af515b8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6e12be740 1 --2- 172.21.15.62:0/1908745742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515b8f20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0cce90 tx=0x7fa6cc097fb0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6e12be740 1 -- 172.21.15.62:0/1908745742 shutdown_connections 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6e12be740 1 --2- 172.21.15.62:0/1908745742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6e12be740 1 --2- 172.21.15.62:0/1908745742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6e12be740 1 --2- 172.21.15.62:0/1908745742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515b8f20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6e12be740 1 -- 172.21.15.62:0/1908745742 >> 172.21.15.62:0/1908745742 conn(0x55af51578310 msgr2=0x55af515b8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6e12be740 1 -- 172.21.15.62:0/1908745742 shutdown_connections 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6e12be740 1 -- 172.21.15.62:0/1908745742 wait complete. 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.870+0000 7fa6e12be740 1 -- start start 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6e12be740 1 Processor -- start 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6e12be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6e12be740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ae040 con 0x55af515b0b40 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6e12be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81af4b0 con 0x55af515af5d0 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6e12be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ae710 con 0x55af515ba370 2024-09-12T00:01:58.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6e12be740 10 client.0 _ll_register_callbacks cb 0x7fa6d0012d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:01:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6dcc81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515be6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55536/0 (socket says 172.21.15.62:55536) 2024-09-12T00:01:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6d7fff640 1 -- 172.21.15.62:0/3900376944 learned_addr learned my addr 172.21.15.62:0/3900376944 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:01:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6dd482640 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6dcc81640 1 -- 172.21.15.62:0/3900376944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 msgr2=0x55af515bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6dcc81640 1 -- 172.21.15.62:0/3900376944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 msgr2=0x55af515be6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:01:58.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515be6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.871+0000 7fa6dcc81640 1 -- 172.21.15.62:0/3900376944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af51633d40 con 0x55af515af5d0 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6d7fff640 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515be6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6dcc81640 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515be1b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0cce90 tx=0x7fa6cc0a4a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a1f2a495fc10ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3900376944 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0b9060 con 0x55af515af5d0 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3900376944 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7fa6cc098380 con 0x55af515af5d0 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6e12be740 1 -- 172.21.15.62:0/3900376944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6cc0b1030 con 0x55af515af5d0 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3900376944 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa6cc0b4e60 con 0x55af515af5d0 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6e12be740 10 client.26699 fetch_fsmap learned FSMap version 33 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6e12be740 1 -- 172.21.15.62:0/3900376944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55af51635000 con 0x55af515af5d0 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3900376944 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc0b1030 con 0x55af515af5d0 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3900376944 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa6cc018bc0 con 0x55af515af5d0 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6e12be740 10 client.26699 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:01:58.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6e12be740 10 client.26699 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6e12be740 20 client.26699 populate_metadata read hostname 'smithi062' 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55af515ca000 0x55af51637770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6e12be740 4 client.26699 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.872+0000 7fa6e12be740 1 -- 172.21.15.62:0/3900376944 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af5151cd70 con 0x55af515ca000 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6d7fff640 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55af515ca000 0x55af51637770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6d7fff640 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55af515ca000 0x55af51637770 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c728d8ae78b34ab8 server_cookie=c90a0a00400842b4 in_seq=0 out_seq=0 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6b7fff640 10 client.26699 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6b7fff640 1 -- 172.21.15.62:0/3900376944 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af5151cd70 con 0x55af515ca000 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6b7fff640 10 client.26699 handle_command_reply: tid=0 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6e12be740 1 client.26699 shutdown 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6e12be740 1 -- 172.21.15.62:0/3900376944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 msgr2=0x55af515be1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515be1b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0cce90 tx=0x7fa6cc0a4a50 comp rx=0 tx=0).stop 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6e12be740 1 -- 172.21.15.62:0/3900376944 shutdown_connections 2024-09-12T00:01:58.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55af515ca000 0x55af51637770 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af515ba370 0x55af515bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6b7fff640 0 client.26699 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:01:58.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af515b0b40 0x55af515be6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6e12be740 1 --2- 172.21.15.62:0/3900376944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af515af5d0 0x55af515be1b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.873+0000 7fa6e12be740 1 -- 172.21.15.62:0/3900376944 >> 172.21.15.62:0/3900376944 conn(0x55af51578310 msgr2=0x55af515b9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6e12be740 1 -- 172.21.15.62:0/3900376944 shutdown_connections 2024-09-12T00:01:58.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6e12be740 1 -- 172.21.15.62:0/3900376944 wait complete. 2024-09-12T00:01:58.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6e12be740 20 client.26699 trim_cache size 0 max 16384 2024-09-12T00:01:58.892 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:01:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 -- 172.21.15.62:0/4268504212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6c80a2cf0 msgr2=0x7fa6c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6c80a2cf0 0x7fa6c80a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009990 tx=0x7fa6d00096a0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 -- 172.21.15.62:0/4268504212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d8100e20 msgr2=0x7fa6d8197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:01:58.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d8100e20 0x7fa6d8197140 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00e9f0 tx=0x7fa6cc00eeb0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 -- 172.21.15.62:0/4268504212 shutdown_connections 2024-09-12T00:01:58.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6c80a2cf0 0x7fa6c80a51b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8102ef0 0x7fa6d8197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6d8102570 0x7fa6d8197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 --2- 172.21.15.62:0/4268504212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d8100e20 0x7fa6d8197140 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:01:58.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 -- 172.21.15.62:0/4268504212 >> 172.21.15.62:0/4268504212 conn(0x7fa6d80fc7d0 msgr2=0x7fa6d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:01:58.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 -- 172.21.15.62:0/4268504212 shutdown_connections 2024-09-12T00:01:58.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:01:58.874+0000 7fa6deee5640 1 -- 172.21.15.62:0/4268504212 wait complete. 2024-09-12T00:01:59.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:01:59 smithi062 ceph-mon[30958]: pgmap v305: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 31 MiB/s wr, 1.72k op/s 2024-09-12T00:01:59.585 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:01:59.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:01:59 smithi120 ceph-mon[35182]: pgmap v305: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 31 MiB/s wr, 1.72k op/s 2024-09-12T00:01:59.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:01:59 smithi083 ceph-mon[34805]: pgmap v305: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 31 MiB/s wr, 1.72k op/s 2024-09-12T00:02:00.013 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '494e38e3', quiesce took 6.7 seconds. Will release after: 15.850000000000001 2024-09-12T00:02:00.074 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:00.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.574+0000 7f8709e8a640 1 -- 172.21.15.62:0/4082533446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87041082a0 msgr2=0x7f870410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:00.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.574+0000 7f8709e8a640 1 --2- 172.21.15.62:0/4082533446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87041082a0 0x7f870410ec40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009cd0 tx=0x7f86fc02a8e0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.575+0000 7f8709e8a640 1 -- 172.21.15.62:0/4082533446 shutdown_connections 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.575+0000 7f8709e8a640 1 --2- 172.21.15.62:0/4082533446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87041082a0 0x7f870410ec40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.575+0000 7f8709e8a640 1 --2- 172.21.15.62:0/4082533446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87041036e0 0x7f8704107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.575+0000 7f8709e8a640 1 --2- 172.21.15.62:0/4082533446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8704102d30 0x7f8704103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.575+0000 7f8709e8a640 1 -- 172.21.15.62:0/4082533446 >> 172.21.15.62:0/4082533446 conn(0x7f87040fc7d0 msgr2=0x7f87040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.575+0000 7f8709e8a640 1 -- 172.21.15.62:0/4082533446 shutdown_connections 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.575+0000 7f8709e8a640 1 -- 172.21.15.62:0/4082533446 wait complete. 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.575+0000 7f8709e8a640 1 Processor -- start 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.575+0000 7f8709e8a640 1 -- start start 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.575+0000 7f8709e8a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8704102d30 0x7f8704198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8709e8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87041036e0 0x7f8704198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8709e8a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87041082a0 0x7f870419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8709e8a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f87041120a0 con 0x7f8704102d30 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8709e8a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8704111f20 con 0x7f87041036e0 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8709e8a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8704112220 con 0x7f87041082a0 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f87037fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8704102d30 0x7f8704198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f87037fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8704102d30 0x7f8704198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55552/0 (socket says 172.21.15.62:55552) 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f87037fe640 1 -- 172.21.15.62:0/1153487302 learned_addr learned my addr 172.21.15.62:0/1153487302 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8702ffd640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87041036e0 0x7f8704198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8703fff640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87041082a0 0x7f870419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f87037fe640 1 -- 172.21.15.62:0/1153487302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87041082a0 msgr2=0x7f870419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f87037fe640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87041082a0 0x7f870419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f87037fe640 1 -- 172.21.15.62:0/1153487302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87041036e0 msgr2=0x7f8704198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f87037fe640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87041036e0 0x7f8704198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f87037fe640 1 -- 172.21.15.62:0/1153487302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870419d500 con 0x7f8704102d30 2024-09-12T00:02:00.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f87037fe640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8704102d30 0x7f8704198550 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f86f400d950 tx=0x7f86f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:00.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8700ff9640 1 -- 172.21.15.62:0/1153487302 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4017070 con 0x7f8704102d30 2024-09-12T00:02:00.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8700ff9640 1 -- 172.21.15.62:0/1153487302 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f4009c30 con 0x7f8704102d30 2024-09-12T00:02:00.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8709e8a640 1 -- 172.21.15.62:0/1153487302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870419d7f0 con 0x7f8704102d30 2024-09-12T00:02:00.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8700ff9640 1 -- 172.21.15.62:0/1153487302 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f400f6b0 con 0x7f8704102d30 2024-09-12T00:02:00.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.576+0000 7f8709e8a640 1 -- 172.21.15.62:0/1153487302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87041a50d0 con 0x7f8704102d30 2024-09-12T00:02:00.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f8700ff9640 1 -- 172.21.15.62:0/1153487302 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f86f40109d0 con 0x7f8704102d30 2024-09-12T00:02:00.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f870c263740 1 Processor -- start 2024-09-12T00:02:00.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f870c263740 1 -- start start 2024-09-12T00:02:00.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f870c263740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff1152b40 0x556ff1151090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f870c263740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff11515d0 0x556ff1151a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f870c263740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff115e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f870c263740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556ff10b72f0 con 0x556ff115c370 2024-09-12T00:02:00.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f870c263740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ff105ecf0 con 0x556ff1152b40 2024-09-12T00:02:00.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f870c263740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556ff10b32b0 con 0x556ff11515d0 2024-09-12T00:02:00.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f8702ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff115e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f8702ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff115e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55554/0 (socket says 172.21.15.62:55554) 2024-09-12T00:02:00.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f8702ffd640 1 -- 172.21.15.62:0/276272959 learned_addr learned my addr 172.21.15.62:0/276272959 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:00.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.579+0000 7f8703fff640 1 --2- 172.21.15.62:0/276272959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff1152b40 0x556ff1151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f87037fe640 1 --2- 172.21.15.62:0/276272959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff11515d0 0x556ff1151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8700ff9640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86ec0a2cf0 0x7f86ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8703fff640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86ec0a2cf0 0x7f86ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8702ffd640 1 -- 172.21.15.62:0/276272959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff11515d0 msgr2=0x556ff1151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:00.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8702ffd640 1 --2- 172.21.15.62:0/276272959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff11515d0 0x556ff1151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8702ffd640 1 -- 172.21.15.62:0/276272959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff1152b40 msgr2=0x556ff1151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8702ffd640 1 --2- 172.21.15.62:0/276272959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff1152b40 0x556ff1151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8702ffd640 1 -- 172.21.15.62:0/276272959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff10433a0 con 0x556ff115c370 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8703fff640 1 --2- 172.21.15.62:0/276272959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff1152b40 0x556ff1151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8700ff9640 1 -- 172.21.15.62:0/1153487302 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f86f40cf030 con 0x7f8704102d30 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8702ffd640 1 --2- 172.21.15.62:0/276272959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff115e760 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f86f800bbb0 tx=0x7f86f80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84b6a14bd18e7d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f8703fff640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86ec0a2cf0 0x7f86ec0a51b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86fc004ab0 tx=0x7f86fc02a8e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f86e9ffb640 1 -- 172.21.15.62:0/276272959 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8004030 con 0x556ff115c370 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f86e9ffb640 1 -- 172.21.15.62:0/276272959 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f8005500 con 0x556ff115c370 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.580+0000 7f86e9ffb640 1 -- 172.21.15.62:0/276272959 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8004c50 con 0x556ff115c370 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.581+0000 7f870c263740 1 -- 172.21.15.62:0/276272959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 msgr2=0x556ff115e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:00.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.581+0000 7f870c263740 1 --2- 172.21.15.62:0/276272959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff115e760 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f86f800bbb0 tx=0x7f86f80107a0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.581+0000 7f870c263740 1 -- 172.21.15.62:0/276272959 shutdown_connections 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.581+0000 7f870c263740 1 --2- 172.21.15.62:0/276272959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff115e760 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.581+0000 7f870c263740 1 --2- 172.21.15.62:0/276272959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff11515d0 0x556ff1151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.581+0000 7f870c263740 1 --2- 172.21.15.62:0/276272959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff1152b40 0x556ff1151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.581+0000 7f870c263740 1 -- 172.21.15.62:0/276272959 >> 172.21.15.62:0/276272959 conn(0x556ff111a310 msgr2=0x556ff114dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.581+0000 7f870c263740 1 -- 172.21.15.62:0/276272959 shutdown_connections 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.581+0000 7f870c263740 1 -- 172.21.15.62:0/276272959 wait complete. 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.581+0000 7f870c263740 1 -- start start 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f870c263740 1 Processor -- start 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f870c263740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff11515d0 0x556ff115bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f870c263740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 0x556ff1158f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f870c263740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff115c370 0x556ff11594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f870c263740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556ff115f070 con 0x556ff11515d0 2024-09-12T00:02:00.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f870c263740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ff1072060 con 0x556ff115c370 2024-09-12T00:02:00.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f870c263740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556ff115f1f0 con 0x556ff1152b40 2024-09-12T00:02:00.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f870c263740 10 client.0 _ll_register_callbacks cb 0x556ff0dc58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:00.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f87037fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff11515d0 0x556ff115bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f87037fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff11515d0 0x556ff115bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55568/0 (socket says 172.21.15.62:55568) 2024-09-12T00:02:00.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f87037fe640 1 -- 172.21.15.62:0/903969498 learned_addr learned my addr 172.21.15.62:0/903969498 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:00.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f8703fff640 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff115c370 0x556ff11594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f8702ffd640 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 0x556ff1158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f87037fe640 1 -- 172.21.15.62:0/903969498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 msgr2=0x556ff1158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f87037fe640 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 0x556ff1158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f87037fe640 1 -- 172.21.15.62:0/903969498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff115c370 msgr2=0x556ff11594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f87037fe640 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff115c370 0x556ff11594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.582+0000 7f87037fe640 1 -- 172.21.15.62:0/903969498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff0e283c0 con 0x556ff11515d0 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.583+0000 7f8702ffd640 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 0x556ff1158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.583+0000 7f87037fe640 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff11515d0 0x556ff115bcd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f86f4097fd0 tx=0x7f86f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2c6bf7e0177c3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.583+0000 7f86e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.583+0000 7f86e2ffd640 1 -- 172.21.15.62:0/903969498 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f409b600 con 0x556ff11515d0 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.583+0000 7f86e2ffd640 1 -- 172.21.15.62:0/903969498 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f86f40d10d0 con 0x556ff11515d0 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.583+0000 7f870c263740 1 -- 172.21.15.62:0/903969498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f8005500 con 0x556ff11515d0 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f86e2ffd640 1 -- 172.21.15.62:0/903969498 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f86f409f900 con 0x556ff11515d0 2024-09-12T00:02:00.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f870c263740 10 client.17139 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f870c263740 1 -- 172.21.15.62:0/903969498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556ff116aa80 con 0x556ff11515d0 2024-09-12T00:02:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f86e2ffd640 1 -- 172.21.15.62:0/903969498 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f8005500 con 0x556ff11515d0 2024-09-12T00:02:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f86e2ffd640 1 -- 172.21.15.62:0/903969498 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f86f4084b40 con 0x556ff11515d0 2024-09-12T00:02:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f870c263740 10 client.17139 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f870c263740 10 client.17139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:02:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f870c263740 20 client.17139 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f870c263740 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ff116be90 0x556ff116e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f870c263740 4 client.17139 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.584+0000 7f870c263740 1 -- 172.21.15.62:0/903969498 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ff10c4860 con 0x556ff116be90 2024-09-12T00:02:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.585+0000 7f8702ffd640 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ff116be90 0x556ff116e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.585+0000 7f8702ffd640 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ff116be90 0x556ff116e280 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4b18b04157801b1 server_cookie=717407ff2ba38954 in_seq=0 out_seq=0 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.585+0000 7f86e2ffd640 10 client.17139 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f86e2ffd640 1 -- 172.21.15.62:0/903969498 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ff10c4860 con 0x556ff116be90 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f86e2ffd640 10 client.17139 handle_command_reply: tid=0 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 client.17139 shutdown 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 -- 172.21.15.62:0/903969498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff11515d0 msgr2=0x556ff115bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff11515d0 0x556ff115bcd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f86f4097fd0 tx=0x7f86f4005880 comp rx=0 tx=0).stop 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 -- 172.21.15.62:0/903969498 shutdown_connections 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ff116be90 0x556ff116e280 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff115c370 0x556ff11594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f86e2ffd640 0 client.17139 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:02:00.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 0x556ff1158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 --2- 172.21.15.62:0/903969498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff11515d0 0x556ff115bcd0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 -- 172.21.15.62:0/903969498 >> 172.21.15.62:0/903969498 conn(0x556ff111a310 msgr2=0x556ff1153960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:00.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 -- 172.21.15.62:0/903969498 shutdown_connections 2024-09-12T00:02:00.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 1 -- 172.21.15.62:0/903969498 wait complete. 2024-09-12T00:02:00.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.586+0000 7f870c263740 20 client.17139 trim_cache size 0 max 16384 2024-09-12T00:02:00.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.599+0000 7f870c263740 1 Processor -- start 2024-09-12T00:02:00.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f870c263740 1 -- start start 2024-09-12T00:02:00.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f870c263740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 0x556ff1168310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f870c263740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff1152b40 0x556ff115d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f870c263740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff115c370 0x556ff115dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f870c263740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f87041b0200 con 0x556ff1152b40 2024-09-12T00:02:00.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f870c263740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87041adaa0 con 0x556ff11515d0 2024-09-12T00:02:00.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f870c263740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87041ab910 con 0x556ff115c370 2024-09-12T00:02:00.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8702ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff1152b40 0x556ff115d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8702ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff1152b40 0x556ff115d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55582/0 (socket says 172.21.15.62:55582) 2024-09-12T00:02:00.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8702ffd640 1 -- 172.21.15.62:0/3641239149 learned_addr learned my addr 172.21.15.62:0/3641239149 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f87037fe640 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 0x556ff1168310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8702ffd640 1 -- 172.21.15.62:0/3641239149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff115c370 msgr2=0x556ff115dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8703fff640 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff115c370 0x556ff115dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8702ffd640 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff115c370 0x556ff115dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8702ffd640 1 -- 172.21.15.62:0/3641239149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 msgr2=0x556ff1168310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8702ffd640 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 0x556ff1168310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8702ffd640 1 -- 172.21.15.62:0/3641239149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff11574d0 con 0x556ff1152b40 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8703fff640 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff115c370 0x556ff115dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f87037fe640 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 0x556ff1168310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.600+0000 7f8702ffd640 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff1152b40 0x556ff115d760 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f86f800bbb0 tx=0x7f86f8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d9f1d42dc00c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:00.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f86e97fa640 1 -- 172.21.15.62:0/3641239149 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f802a430 con 0x556ff1152b40 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f86e97fa640 1 -- 172.21.15.62:0/3641239149 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f802aab0 con 0x556ff1152b40 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f86e97fa640 1 -- 172.21.15.62:0/3641239149 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f80288c0 con 0x556ff1152b40 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f870c263740 1 -- 172.21.15.62:0/3641239149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff1152b40 msgr2=0x556ff115d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f870c263740 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff1152b40 0x556ff115d760 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f86f800bbb0 tx=0x7f86f8013040 comp rx=0 tx=0).stop 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f870c263740 1 -- 172.21.15.62:0/3641239149 shutdown_connections 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f870c263740 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff115c370 0x556ff115dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f870c263740 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff1152b40 0x556ff115d760 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f870c263740 1 --2- 172.21.15.62:0/3641239149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 0x556ff1168310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f870c263740 1 -- 172.21.15.62:0/3641239149 >> 172.21.15.62:0/3641239149 conn(0x556ff111a310 msgr2=0x556ff1157850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f870c263740 1 -- 172.21.15.62:0/3641239149 shutdown_connections 2024-09-12T00:02:00.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.601+0000 7f870c263740 1 -- 172.21.15.62:0/3641239149 wait complete. 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f870c263740 1 -- start start 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f870c263740 1 Processor -- start 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f870c263740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 0x556ff1164c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f870c263740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 0x556ff1173470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f870c263740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff11739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f870c263740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f87041abf60 con 0x556ff115c370 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f870c263740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87041ad3d0 con 0x556ff11515d0 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f870c263740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87041ac630 con 0x556ff1152b40 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f870c263740 10 client.0 _ll_register_callbacks cb 0x7f86f4097210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f8703fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff11739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f8703fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff11739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55598/0 (socket says 172.21.15.62:55598) 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f8703fff640 1 -- 172.21.15.62:0/1746463370 learned_addr learned my addr 172.21.15.62:0/1746463370 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:00.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f8702ffd640 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 0x556ff1173470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.602+0000 7f87037fe640 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 0x556ff1164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f8703fff640 1 -- 172.21.15.62:0/1746463370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 msgr2=0x556ff1173470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f8703fff640 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 0x556ff1173470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f8703fff640 1 -- 172.21.15.62:0/1746463370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 msgr2=0x556ff1164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f8703fff640 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 0x556ff1164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f8703fff640 1 -- 172.21.15.62:0/1746463370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff11741e0 con 0x556ff115c370 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f8703fff640 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff11739b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009cd0 tx=0x7f86fc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef22cb6c6e2b1a4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f86e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f86e2ffd640 1 -- 172.21.15.62:0/1746463370 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0305a0 con 0x556ff115c370 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f86e2ffd640 1 -- 172.21.15.62:0/1746463370 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f86fc030740 con 0x556ff115c370 2024-09-12T00:02:00.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f870c263740 1 -- 172.21.15.62:0/1746463370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f802aab0 con 0x556ff115c370 2024-09-12T00:02:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f86e2ffd640 1 -- 172.21.15.62:0/1746463370 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f86fc030940 con 0x556ff115c370 2024-09-12T00:02:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f870c263740 10 client.17145 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f870c263740 1 -- 172.21.15.62:0/1746463370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556ff11d6b20 con 0x556ff115c370 2024-09-12T00:02:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.603+0000 7f86e2ffd640 1 -- 172.21.15.62:0/1746463370 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f802aab0 con 0x556ff115c370 2024-09-12T00:02:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.604+0000 7f86e2ffd640 1 -- 172.21.15.62:0/1746463370 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f86fc030070 con 0x556ff115c370 2024-09-12T00:02:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.604+0000 7f870c263740 10 client.17145 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.604+0000 7f870c263740 10 client.17145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:02:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.604+0000 7f870c263740 20 client.17145 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.604+0000 7f870c263740 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ff11d7220 0x556ff11d9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.604+0000 7f870c263740 4 client.17145 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:02:00.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.604+0000 7f870c263740 1 -- 172.21.15.62:0/1746463370 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ff10bed70 con 0x556ff11d7220 2024-09-12T00:02:00.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.604+0000 7f87037fe640 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ff11d7220 0x556ff11d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:00.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.604+0000 7f86e2ffd640 10 client.17145 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:02:00.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.604+0000 7f87037fe640 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ff11d7220 0x556ff11d9640 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12a430fe6888b9d7 server_cookie=ff881e5704e3640b in_seq=0 out_seq=0 2024-09-12T00:02:00.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f86e2ffd640 1 -- 172.21.15.62:0/1746463370 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ff10bed70 con 0x556ff11d7220 2024-09-12T00:02:00.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f86e2ffd640 10 client.17145 handle_command_reply: tid=0 2024-09-12T00:02:00.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 client.17145 shutdown 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 -- 172.21.15.62:0/1746463370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 msgr2=0x556ff11739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff11739b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009cd0 tx=0x7f86fc0095c0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 -- 172.21.15.62:0/1746463370 shutdown_connections 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x556ff11d7220 0x556ff11d9640 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ff115c370 0x556ff11739b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556ff1152b40 0x556ff1173470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f86e2ffd640 0 client.17145 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 --2- 172.21.15.62:0/1746463370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff11515d0 0x556ff1164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 -- 172.21.15.62:0/1746463370 >> 172.21.15.62:0/1746463370 conn(0x556ff111a310 msgr2=0x556ff114cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 -- 172.21.15.62:0/1746463370 shutdown_connections 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 1 -- 172.21.15.62:0/1746463370 wait complete. 2024-09-12T00:02:00.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f870c263740 20 client.17145 trim_cache size 0 max 16384 2024-09-12T00:02:00.613 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:02:00.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f8709e8a640 1 -- 172.21.15.62:0/1153487302 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86ec0a2cf0 msgr2=0x7f86ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f8709e8a640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86ec0a2cf0 0x7f86ec0a51b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86fc004ab0 tx=0x7f86fc02a8e0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f8709e8a640 1 -- 172.21.15.62:0/1153487302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8704102d30 msgr2=0x7f8704198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.605+0000 7f8709e8a640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8704102d30 0x7f8704198550 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f86f400d950 tx=0x7f86f400de10 comp rx=0 tx=0).stop 2024-09-12T00:02:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.606+0000 7f8709e8a640 1 -- 172.21.15.62:0/1153487302 shutdown_connections 2024-09-12T00:02:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.606+0000 7f8709e8a640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f86ec0a2cf0 0x7f86ec0a51b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.606+0000 7f8709e8a640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87041082a0 0x7f870419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.606+0000 7f8709e8a640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87041036e0 0x7f8704198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.606+0000 7f8709e8a640 1 --2- 172.21.15.62:0/1153487302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8704102d30 0x7f8704198550 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:00.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.606+0000 7f8709e8a640 1 -- 172.21.15.62:0/1153487302 >> 172.21.15.62:0/1153487302 conn(0x7f87040fc7d0 msgr2=0x7f8704105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:00.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.606+0000 7f8709e8a640 1 -- 172.21.15.62:0/1153487302 shutdown_connections 2024-09-12T00:02:00.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:00.606+0000 7f8709e8a640 1 -- 172.21.15.62:0/1153487302 wait complete. 2024-09-12T00:02:01.375 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:02:01.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:01 smithi120 ceph-mon[35182]: pgmap v306: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2024-09-12T00:02:01.636 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:01.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:01 smithi083 ceph-mon[34805]: pgmap v306: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2024-09-12T00:02:01.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:01 smithi062 ceph-mon[30958]: pgmap v306: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2024-09-12T00:02:02.448 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 560.0 s, 334.4 tps, lat 46.639 ms stddev 51.894 2024-09-12T00:02:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.796+0000 7f4d71644640 1 -- 172.21.15.62:0/3977308537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c10c170 msgr2=0x7f4d6c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.796+0000 7f4d71644640 1 --2- 172.21.15.62:0/3977308537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c10c170 0x7f4d6c10e610 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009900 tx=0x7f4d6402a480 comp rx=0 tx=0).stop 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.796+0000 7f4d71644640 1 -- 172.21.15.62:0/3977308537 shutdown_connections 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.796+0000 7f4d71644640 1 --2- 172.21.15.62:0/3977308537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c10c170 0x7f4d6c10e610 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.796+0000 7f4d71644640 1 --2- 172.21.15.62:0/3977308537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c103660 0x7f4d6c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.796+0000 7f4d71644640 1 --2- 172.21.15.62:0/3977308537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d6c102d40 0x7f4d6c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.796+0000 7f4d71644640 1 -- 172.21.15.62:0/3977308537 >> 172.21.15.62:0/3977308537 conn(0x7f4d6c0fc7a0 msgr2=0x7f4d6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.796+0000 7f4d71644640 1 -- 172.21.15.62:0/3977308537 shutdown_connections 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.796+0000 7f4d71644640 1 -- 172.21.15.62:0/3977308537 wait complete. 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.797+0000 7f4d71644640 1 Processor -- start 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.797+0000 7f4d71644640 1 -- start start 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.797+0000 7f4d71644640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d6c102d40 0x7f4d6c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.797+0000 7f4d71644640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c103660 0x7f4d6c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.797+0000 7f4d71644640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c10c170 0x7f4d6c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.797+0000 7f4d71644640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c112030 con 0x7f4d6c10c170 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.797+0000 7f4d71644640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c111eb0 con 0x7f4d6c103660 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.797+0000 7f4d71644640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1121b0 con 0x7f4d6c102d40 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.797+0000 7f4d6b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c10c170 0x7f4d6c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c10c170 0x7f4d6c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51026/0 (socket says 172.21.15.62:51026) 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6b7fe640 1 -- 172.21.15.62:0/1125493299 learned_addr learned my addr 172.21.15.62:0/1125493299 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6affd640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d6c102d40 0x7f4d6c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c103660 0x7f4d6c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6b7fe640 1 -- 172.21.15.62:0/1125493299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d6c102d40 msgr2=0x7f4d6c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6b7fe640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d6c102d40 0x7f4d6c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6b7fe640 1 -- 172.21.15.62:0/1125493299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c103660 msgr2=0x7f4d6c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6b7fe640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c103660 0x7f4d6c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6b7fe640 1 -- 172.21.15.62:0/1125493299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c1a6ab0 con 0x7f4d6c10c170 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c103660 0x7f4d6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d6b7fe640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c10c170 0x7f4d6c101420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009230 tx=0x7f4d640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d63fff640 1 -- 172.21.15.62:0/1125493299 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6403f030 con 0x7f4d6c10c170 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d71644640 1 -- 172.21.15.62:0/1125493299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c1a6d40 con 0x7f4d6c10c170 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d63fff640 1 -- 172.21.15.62:0/1125493299 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d64003ce0 con 0x7f4d6c10c170 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d63fff640 1 -- 172.21.15.62:0/1125493299 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64035070 con 0x7f4d6c10c170 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.798+0000 7f4d71644640 1 -- 172.21.15.62:0/1125493299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c1a7220 con 0x7f4d6c10c170 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.801+0000 7f4d63fff640 1 -- 172.21.15.62:0/1125493299 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4d64042030 con 0x7f4d6c10c170 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.801+0000 7f4d73a1d740 1 Processor -- start 2024-09-12T00:02:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.801+0000 7f4d73a1d740 1 -- start start 2024-09-12T00:02:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.801+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4eb40 0x559c38c4d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.801+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4d5d0 0x559c38c4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.801+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.801+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559c38bb32f0 con 0x559c38c58370 2024-09-12T00:02:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.801+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559c38b5acf0 con 0x559c38c4d5d0 2024-09-12T00:02:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.801+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559c38baf2b0 con 0x559c38c4eb40 2024-09-12T00:02:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d63fff640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4d580a2cf0 0x7f4d580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c5a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51038/0 (socket says 172.21.15.62:51038) 2024-09-12T00:02:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6a7fc640 1 -- 172.21.15.62:0/2926424072 learned_addr learned my addr 172.21.15.62:0/2926424072 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6affd640 1 --2- 172.21.15.62:0/2926424072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4d5d0 0x559c38c4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/2926424072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4eb40 0x559c38c4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6b7fe640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4d580a2cf0 0x7f4d580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6a7fc640 1 -- 172.21.15.62:0/2926424072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4eb40 msgr2=0x559c38c4d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/2926424072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4eb40 0x559c38c4d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d63fff640 1 -- 172.21.15.62:0/1125493299 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f4d640efd00 con 0x7f4d6c10c170 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6a7fc640 1 -- 172.21.15.62:0/2926424072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4d5d0 msgr2=0x559c38c4da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/2926424072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4d5d0 0x559c38c4da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6a7fc640 1 -- 172.21.15.62:0/2926424072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c38b3f3a0 con 0x559c38c58370 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/2926424072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c5a760 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00acf0 tx=0x7f4d5c009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d81084d6e8e6b98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d617fa640 1 -- 172.21.15.62:0/2926424072 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c013040 con 0x559c38c58370 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.802+0000 7f4d6b7fe640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4d580a2cf0 0x7f4d580a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4d640bbf20 tx=0x7f4d640dc040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d617fa640 1 -- 172.21.15.62:0/2926424072 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5c01a430 con 0x559c38c58370 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d617fa640 1 -- 172.21.15.62:0/2926424072 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0195e0 con 0x559c38c58370 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d73a1d740 1 -- 172.21.15.62:0/2926424072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 msgr2=0x559c38c5a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/2926424072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c5a760 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00acf0 tx=0x7f4d5c009940 comp rx=0 tx=0).stop 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d73a1d740 1 -- 172.21.15.62:0/2926424072 shutdown_connections 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/2926424072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c5a760 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/2926424072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4d5d0 0x559c38c4da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/2926424072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4eb40 0x559c38c4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d73a1d740 1 -- 172.21.15.62:0/2926424072 >> 172.21.15.62:0/2926424072 conn(0x559c38c16310 msgr2=0x559c38c49c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d73a1d740 1 -- 172.21.15.62:0/2926424072 shutdown_connections 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.803+0000 7f4d73a1d740 1 -- 172.21.15.62:0/2926424072 wait complete. 2024-09-12T00:02:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d73a1d740 1 -- start start 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d73a1d740 1 Processor -- start 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4d5d0 0x559c38c57d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4eb40 0x559c38c55020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c58370 0x559c38c55610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559c38c5b010 con 0x559c38c4d5d0 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559c38b6e060 con 0x559c38c58370 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559c38c5b190 con 0x559c38c4eb40 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d73a1d740 10 client.0 _ll_register_callbacks cb 0x559c388c18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d6a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4d5d0 0x559c38c57d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d6a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4d5d0 0x559c38c57d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51052/0 (socket says 172.21.15.62:51052) 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.804+0000 7f4d6a7fc640 1 -- 172.21.15.62:0/530015916 learned_addr learned my addr 172.21.15.62:0/530015916 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d6affd640 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4eb40 0x559c38c55020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d6a7fc640 1 -- 172.21.15.62:0/530015916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4eb40 msgr2=0x559c38c55020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:02.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4eb40 0x559c38c55020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d6a7fc640 1 -- 172.21.15.62:0/530015916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c58370 msgr2=0x559c38c55610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:02.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c58370 0x559c38c55610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d6a7fc640 1 -- 172.21.15.62:0/530015916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c389243c0 con 0x559c38c4d5d0 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4d5d0 0x559c38c57d70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c002230 tx=0x7f4d5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d5fed691452e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d527fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d527fc640 1 -- 172.21.15.62:0/530015916 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0193e0 con 0x559c38c4d5d0 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d527fc640 1 -- 172.21.15.62:0/530015916 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f4d5c012040 con 0x559c38c4d5d0 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d73a1d740 1 -- 172.21.15.62:0/530015916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d5c01a430 con 0x559c38c4d5d0 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d527fc640 1 -- 172.21.15.62:0/530015916 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c031050 con 0x559c38c4d5d0 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d73a1d740 10 client.17157 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d73a1d740 1 -- 172.21.15.62:0/530015916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559c38c66ab0 con 0x559c38c4d5d0 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.805+0000 7f4d527fc640 1 -- 172.21.15.62:0/530015916 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5c01a430 con 0x559c38c4d5d0 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.806+0000 7f4d527fc640 1 -- 172.21.15.62:0/530015916 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f4d5c023070 con 0x559c38c4d5d0 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.806+0000 7f4d73a1d740 10 client.17157 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.806+0000 7f4d73a1d740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.806+0000 7f4d73a1d740 20 client.17157 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.806+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559c38c67e50 0x559c38c6a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.806+0000 7f4d73a1d740 4 client.17157 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.806+0000 7f4d73a1d740 1 -- 172.21.15.62:0/530015916 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c38bc0860 con 0x559c38c67e50 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.806+0000 7f4d6affd640 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559c38c67e50 0x559c38c6a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.806+0000 7f4d527fc640 10 client.17157 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.806+0000 7f4d6affd640 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559c38c67e50 0x559c38c6a240 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=150a2380c5199d86 server_cookie=12e600b828dab239 in_seq=0 out_seq=0 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d527fc640 1 -- 172.21.15.62:0/530015916 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c38bc0860 con 0x559c38c67e50 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d527fc640 10 client.17157 handle_command_reply: tid=0 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 client.17157 shutdown 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 -- 172.21.15.62:0/530015916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4d5d0 msgr2=0x559c38c57d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4d5d0 0x559c38c57d70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c002230 tx=0x7f4d5c004a40 comp rx=0 tx=0).stop 2024-09-12T00:02:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 -- 172.21.15.62:0/530015916 shutdown_connections 2024-09-12T00:02:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559c38c67e50 0x559c38c6a240 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c58370 0x559c38c55610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4eb40 0x559c38c55020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d527fc640 0 client.17157 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/530015916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4d5d0 0x559c38c57d70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 -- 172.21.15.62:0/530015916 >> 172.21.15.62:0/530015916 conn(0x559c38c16310 msgr2=0x559c38c4f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 -- 172.21.15.62:0/530015916 shutdown_connections 2024-09-12T00:02:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.807+0000 7f4d73a1d740 1 -- 172.21.15.62:0/530015916 wait complete. 2024-09-12T00:02:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.808+0000 7f4d73a1d740 20 client.17157 trim_cache size 0 max 16384 2024-09-12T00:02:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d73a1d740 1 Processor -- start 2024-09-12T00:02:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d73a1d740 1 -- start start 2024-09-12T00:02:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4d5d0 0x559c38c642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4eb40 0x559c38c548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c54020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b23d0 con 0x559c38c58370 2024-09-12T00:02:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1afc70 con 0x559c38c4eb40 2024-09-12T00:02:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1adae0 con 0x559c38c4d5d0 2024-09-12T00:02:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d6a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c54020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d6a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c54020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51058/0 (socket says 172.21.15.62:51058) 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d6a7fc640 1 -- 172.21.15.62:0/1001726021 learned_addr learned my addr 172.21.15.62:0/1001726021 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d6affd640 1 --2- 172.21.15.62:0/1001726021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4eb40 0x559c38c548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/1001726021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4d5d0 0x559c38c642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d6affd640 1 -- 172.21.15.62:0/1001726021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4d5d0 msgr2=0x559c38c642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d6affd640 1 --2- 172.21.15.62:0/1001726021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4d5d0 0x559c38c642b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d6affd640 1 -- 172.21.15.62:0/1001726021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 msgr2=0x559c38c54020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d6affd640 1 --2- 172.21.15.62:0/1001726021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c54020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.821+0000 7f4d6affd640 1 -- 172.21.15.62:0/1001726021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c38c53590 con 0x559c38c4eb40 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.822+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/1001726021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c54020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.822+0000 7f4d6affd640 1 --2- 172.21.15.62:0/1001726021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4eb40 0x559c38c548e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x559c38c6a8f0 tx=0x7f4d54008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5cc5c596057ea2a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.822+0000 7f4d60ff9640 1 -- 172.21.15.62:0/1001726021 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5401d250 con 0x559c38c4eb40 2024-09-12T00:02:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.822+0000 7f4d60ff9640 1 -- 172.21.15.62:0/1001726021 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5401d8d0 con 0x559c38c4eb40 2024-09-12T00:02:02.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.822+0000 7f4d60ff9640 1 -- 172.21.15.62:0/1001726021 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d540159d0 con 0x559c38c4eb40 2024-09-12T00:02:02.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 -- 172.21.15.62:0/1001726021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4eb40 msgr2=0x559c38c548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:02.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/1001726021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4eb40 0x559c38c548e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x559c38c6a8f0 tx=0x7f4d54008040 comp rx=0 tx=0).stop 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 -- 172.21.15.62:0/1001726021 shutdown_connections 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/1001726021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c58370 0x559c38c54020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/1001726021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4eb40 0x559c38c548e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/1001726021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c4d5d0 0x559c38c642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 -- 172.21.15.62:0/1001726021 >> 172.21.15.62:0/1001726021 conn(0x559c38c16310 msgr2=0x559c38c53910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 -- 172.21.15.62:0/1001726021 shutdown_connections 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 -- 172.21.15.62:0/1001726021 wait complete. 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 -- start start 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.823+0000 7f4d73a1d740 1 Processor -- start 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4d5d0 0x559c38c60600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4eb40 0x559c38c60b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d73a1d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c58370 0x559c38c518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1ae130 con 0x559c38c4eb40 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1af5a0 con 0x559c38c4d5d0 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d73a1d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1ae800 con 0x559c38c58370 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d73a1d740 10 client.0 _ll_register_callbacks cb 0x7f4d540093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4eb40 0x559c38c60b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4eb40 0x559c38c60b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51064/0 (socket says 172.21.15.62:51064) 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6affd640 1 -- 172.21.15.62:0/580634723 learned_addr learned my addr 172.21.15.62:0/580634723 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4d5d0 0x559c38c60600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c58370 0x559c38c518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6affd640 1 -- 172.21.15.62:0/580634723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c58370 msgr2=0x559c38c518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:02.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6affd640 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c58370 0x559c38c518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6affd640 1 -- 172.21.15.62:0/580634723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4d5d0 msgr2=0x559c38c60600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6affd640 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4d5d0 0x559c38c60600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6affd640 1 -- 172.21.15.62:0/580634723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c38c51fe0 con 0x559c38c4eb40 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c58370 0x559c38c518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d6affd640 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4eb40 0x559c38c60b40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4d54000c00 tx=0x7f4d540096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd57ca05a7c44725 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d527fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d527fc640 1 -- 172.21.15.62:0/580634723 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5401dc80 con 0x559c38c4eb40 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d527fc640 1 -- 172.21.15.62:0/580634723 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f4d54015950 con 0x559c38c4eb40 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.824+0000 7f4d73a1d740 1 -- 172.21.15.62:0/580634723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d5401d8d0 con 0x559c38c4eb40 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d527fc640 1 -- 172.21.15.62:0/580634723 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4d54015b50 con 0x559c38c4eb40 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d73a1d740 10 client.17163 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:02.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d73a1d740 1 -- 172.21.15.62:0/580634723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559c38cd2350 con 0x559c38c4eb40 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d527fc640 1 -- 172.21.15.62:0/580634723 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5401d8d0 con 0x559c38c4eb40 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d527fc640 1 -- 172.21.15.62:0/580634723 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f4d54039730 con 0x559c38c4eb40 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d73a1d740 10 client.17163 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d73a1d740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d73a1d740 20 client.17163 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559c38cd2a40 0x559c38cd4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d73a1d740 4 client.17163 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d73a1d740 1 -- 172.21.15.62:0/580634723 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c38bbad70 con 0x559c38cd2a40 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.825+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559c38cd2a40 0x559c38cd4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d6a7fc640 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559c38cd2a40 0x559c38cd4e60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389eee09b1fcad server_cookie=f9db148b54daa20 in_seq=0 out_seq=0 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d527fc640 10 client.17163 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:02.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d527fc640 1 -- 172.21.15.62:0/580634723 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c38bbad70 con 0x559c38cd2a40 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d527fc640 10 client.17163 handle_command_reply: tid=0 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 client.17163 shutdown 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 -- 172.21.15.62:0/580634723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4eb40 msgr2=0x559c38c60b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4eb40 0x559c38c60b40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4d54000c00 tx=0x7f4d540096a0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 -- 172.21.15.62:0/580634723 shutdown_connections 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559c38cd2a40 0x559c38cd4e60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c38c58370 0x559c38c518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c38c4eb40 0x559c38c60b40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d527fc640 0 client.17163 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 --2- 172.21.15.62:0/580634723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c38c4d5d0 0x559c38c60600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 -- 172.21.15.62:0/580634723 >> 172.21.15.62:0/580634723 conn(0x559c38c16310 msgr2=0x559c38c54130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 -- 172.21.15.62:0/580634723 shutdown_connections 2024-09-12T00:02:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 1 -- 172.21.15.62:0/580634723 wait complete. 2024-09-12T00:02:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.826+0000 7f4d73a1d740 20 client.17163 trim_cache size 0 max 16384 2024-09-12T00:02:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:02:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 -- 172.21.15.62:0/1125493299 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4d580a2cf0 msgr2=0x7f4d580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4d580a2cf0 0x7f4d580a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4d640bbf20 tx=0x7f4d640dc040 comp rx=0 tx=0).stop 2024-09-12T00:02:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 -- 172.21.15.62:0/1125493299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c10c170 msgr2=0x7f4d6c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c10c170 0x7f4d6c101420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009230 tx=0x7f4d640042f0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 -- 172.21.15.62:0/1125493299 shutdown_connections 2024-09-12T00:02:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4d580a2cf0 0x7f4d580a51b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c10c170 0x7f4d6c101420 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c103660 0x7f4d6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 --2- 172.21.15.62:0/1125493299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d6c102d40 0x7f4d6c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 -- 172.21.15.62:0/1125493299 >> 172.21.15.62:0/1125493299 conn(0x7f4d6c0fc7a0 msgr2=0x7f4d6c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 -- 172.21.15.62:0/1125493299 shutdown_connections 2024-09-12T00:02:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:02.827+0000 7f4d71644640 1 -- 172.21.15.62:0/1125493299 wait complete. 2024-09-12T00:02:03.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:03 smithi120 ceph-mon[35182]: pgmap v307: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 28 MiB/s wr, 1.73k op/s 2024-09-12T00:02:03.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:03 smithi083 ceph-mon[34805]: pgmap v307: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 28 MiB/s wr, 1.73k op/s 2024-09-12T00:02:03.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:03 smithi062 ceph-mon[30958]: pgmap v307: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 28 MiB/s wr, 1.73k op/s 2024-09-12T00:02:04.484 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:02:04.484 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:02:04.756 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:05.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.230+0000 7f6e5d757640 1 -- 172.21.15.62:0/1234325906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e58100ca0 msgr2=0x7f6e58106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.230+0000 7f6e5d757640 1 --2- 172.21.15.62:0/1234325906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e58100ca0 0x7f6e58106840 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009930 tx=0x7f6e4002a420 comp rx=0 tx=0).stop 2024-09-12T00:02:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.230+0000 7f6e5d757640 1 -- 172.21.15.62:0/1234325906 shutdown_connections 2024-09-12T00:02:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.230+0000 7f6e5d757640 1 --2- 172.21.15.62:0/1234325906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e58106f50 0x7f6e5810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.230+0000 7f6e5d757640 1 --2- 172.21.15.62:0/1234325906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e58100ca0 0x7f6e58106840 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.230+0000 7f6e5d757640 1 --2- 172.21.15.62:0/1234325906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e581002f0 0x7f6e581006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.230+0000 7f6e5d757640 1 -- 172.21.15.62:0/1234325906 >> 172.21.15.62:0/1234325906 conn(0x7f6e58077210 msgr2=0x7f6e580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.230+0000 7f6e5d757640 1 -- 172.21.15.62:0/1234325906 shutdown_connections 2024-09-12T00:02:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.230+0000 7f6e5d757640 1 -- 172.21.15.62:0/1234325906 wait complete. 2024-09-12T00:02:05.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.230+0000 7f6e5d757640 1 Processor -- start 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e5d757640 1 -- start start 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e5d757640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e581002f0 0x7f6e58198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e5d757640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e58100ca0 0x7f6e58198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e5d757640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e58106f50 0x7f6e5819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e5d757640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58111fe0 con 0x7f6e58106f50 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e5d757640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58111e60 con 0x7f6e58100ca0 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e5d757640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58112160 con 0x7f6e581002f0 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e577fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e58106f50 0x7f6e5819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e577fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e58106f50 0x7f6e5819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51078/0 (socket says 172.21.15.62:51078) 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e577fe640 1 -- 172.21.15.62:0/2354411739 learned_addr learned my addr 172.21.15.62:0/2354411739 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e567fc640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e58100ca0 0x7f6e58198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e581002f0 0x7f6e58198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e577fe640 1 -- 172.21.15.62:0/2354411739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e581002f0 msgr2=0x7f6e58198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e577fe640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e581002f0 0x7f6e58198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e577fe640 1 -- 172.21.15.62:0/2354411739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e58100ca0 msgr2=0x7f6e58198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e577fe640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e58100ca0 0x7f6e58198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.231+0000 7f6e577fe640 1 -- 172.21.15.62:0/2354411739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5819d4b0 con 0x7f6e58106f50 2024-09-12T00:02:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.232+0000 7f6e577fe640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e58106f50 0x7f6e5819cdd0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000eaa0 tx=0x7f6e5000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.232+0000 7f6e4ffff640 1 -- 172.21.15.62:0/2354411739 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50004d20 con 0x7f6e58106f50 2024-09-12T00:02:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.232+0000 7f6e5d757640 1 -- 172.21.15.62:0/2354411739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e5819d7a0 con 0x7f6e58106f50 2024-09-12T00:02:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.232+0000 7f6e4ffff640 1 -- 172.21.15.62:0/2354411739 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e50005020 con 0x7f6e58106f50 2024-09-12T00:02:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.232+0000 7f6e4ffff640 1 -- 172.21.15.62:0/2354411739 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e500116d0 con 0x7f6e58106f50 2024-09-12T00:02:05.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.232+0000 7f6e5d757640 1 -- 172.21.15.62:0/2354411739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e581a5080 con 0x7f6e58106f50 2024-09-12T00:02:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.234+0000 7f6e4ffff640 1 -- 172.21.15.62:0/2354411739 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6e500051d0 con 0x7f6e58106f50 2024-09-12T00:02:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.234+0000 7f6e5fb30740 1 Processor -- start 2024-09-12T00:02:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.234+0000 7f6e5fb30740 1 -- start start 2024-09-12T00:02:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 0x560f336e2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e3360 0x560f336f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336f2150 0x560f336f4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560f33648280 con 0x560f336e3360 2024-09-12T00:02:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f335efb10 con 0x560f336e2a40 2024-09-12T00:02:05.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560f33644240 con 0x560f336f2150 2024-09-12T00:02:05.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e56ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e3360 0x560f336f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e56ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e3360 0x560f336f1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51084/0 (socket says 172.21.15.62:51084) 2024-09-12T00:02:05.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e56ffd640 1 -- 172.21.15.62:0/3576303002 learned_addr learned my addr 172.21.15.62:0/3576303002 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:05.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e567fc640 1 --2- 172.21.15.62:0/3576303002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336f2150 0x560f336f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e567fc640 1 --2- 172.21.15.62:0/3576303002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 0x560f336e2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e56ffd640 1 -- 172.21.15.62:0/3576303002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336f2150 msgr2=0x560f336f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:05.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/3576303002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336f2150 0x560f336f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e56ffd640 1 -- 172.21.15.62:0/3576303002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 msgr2=0x560f336e2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:05.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e4ffff640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6e440a2cf0 0x7f6e440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/3576303002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 0x560f336e2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e56ffd640 1 -- 172.21.15.62:0/3576303002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f335fc1f0 con 0x560f336e3360 2024-09-12T00:02:05.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e567fc640 1 --2- 172.21.15.62:0/3576303002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 0x560f336e2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:05.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e577fe640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6e440a2cf0 0x7f6e440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/3576303002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e3360 0x560f336f1c10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009a70 tx=0x7f6e48009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d75af085caa62e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:05.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.235+0000 7f6e4ffff640 1 -- 172.21.15.62:0/2354411739 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f6e500cf630 con 0x7f6e58106f50 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e4d7fa640 1 -- 172.21.15.62:0/3576303002 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48017070 con 0x560f336e3360 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e4d7fa640 1 -- 172.21.15.62:0/3576303002 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e48013470 con 0x560f336e3360 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e4d7fa640 1 -- 172.21.15.62:0/3576303002 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e480125d0 con 0x560f336e3360 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3576303002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e3360 msgr2=0x560f336f1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e577fe640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6e440a2cf0 0x7f6e440a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e50097fb0 tx=0x7f6e5006b9b0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/3576303002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e3360 0x560f336f1c10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009a70 tx=0x7f6e48009f30 comp rx=0 tx=0).stop 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3576303002 shutdown_connections 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/3576303002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336f2150 0x560f336f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/3576303002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e3360 0x560f336f1c10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/3576303002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 0x560f336e2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.236+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3576303002 >> 172.21.15.62:0/3576303002 conn(0x560f336df0f0 msgr2=0x560f336df4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3576303002 shutdown_connections 2024-09-12T00:02:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3576303002 wait complete. 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 1 -- start start 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 1 Processor -- start 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336ef030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f336ef570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336f2150 0x560f336efab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560f336f4d20 con 0x560f336e2a40 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f33602f10 con 0x560f336f2150 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560f336f4ea0 con 0x560f336e3360 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e5fb30740 10 client.0 _ll_register_callbacks cb 0x560f335cba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e567fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e567fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336ef030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51086/0 (socket says 172.21.15.62:51086) 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e567fc640 1 -- 172.21.15.62:0/866288951 learned_addr learned my addr 172.21.15.62:0/866288951 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.237+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f336ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e567fc640 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336f2150 0x560f336efab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e567fc640 1 -- 172.21.15.62:0/866288951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 msgr2=0x560f336ef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e567fc640 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f336ef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e567fc640 1 -- 172.21.15.62:0/866288951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336f2150 msgr2=0x560f336efab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e567fc640 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336f2150 0x560f336efab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e567fc640 1 -- 172.21.15.62:0/866288951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f333b96a0 con 0x560f336e2a40 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e567fc640 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336ef030 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6e400096c0 tx=0x7f6e40009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80899e6a93da99ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/866288951 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40035070 con 0x560f336e2a40 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/866288951 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f6e40004450 con 0x560f336e2a40 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e5fb30740 1 -- 172.21.15.62:0/866288951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e48013470 con 0x560f336e2a40 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/866288951 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6e40051050 con 0x560f336e2a40 2024-09-12T00:02:05.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e5fb30740 10 client.17175 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e5fb30740 1 -- 172.21.15.62:0/866288951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560f337008d0 con 0x560f336e2a40 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.238+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/866288951 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e48013470 con 0x560f336e2a40 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.239+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/866288951 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f6e40004c80 con 0x560f336e2a40 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.239+0000 7f6e5fb30740 10 client.17175 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.239+0000 7f6e5fb30740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.239+0000 7f6e5fb30740 20 client.17175 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.239+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560f33701ce0 0x560f337040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.239+0000 7f6e5fb30740 4 client.17175 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.239+0000 7f6e5fb30740 1 -- 172.21.15.62:0/866288951 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f336557f0 con 0x560f33701ce0 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.239+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560f33701ce0 0x560f337040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e3e7fc640 10 client.17175 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560f33701ce0 0x560f337040f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52033eadde048305 server_cookie=d6c4005ae0da1817 in_seq=0 out_seq=0 2024-09-12T00:02:05.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/866288951 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f336557f0 con 0x560f33701ce0 2024-09-12T00:02:05.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e3e7fc640 10 client.17175 handle_command_reply: tid=0 2024-09-12T00:02:05.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e5fb30740 1 client.17175 shutdown 2024-09-12T00:02:05.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e5fb30740 1 -- 172.21.15.62:0/866288951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 msgr2=0x560f336ef030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:05.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336ef030 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6e400096c0 tx=0x7f6e40009260 comp rx=0 tx=0).stop 2024-09-12T00:02:05.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e5fb30740 1 -- 172.21.15.62:0/866288951 shutdown_connections 2024-09-12T00:02:05.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560f33701ce0 0x560f337040f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336f2150 0x560f336efab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f336ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/866288951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336ef030 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e3e7fc640 0 client.17175 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:05.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.240+0000 7f6e5fb30740 1 -- 172.21.15.62:0/866288951 >> 172.21.15.62:0/866288951 conn(0x560f336df0f0 msgr2=0x560f336e4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:05.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.241+0000 7f6e5fb30740 1 -- 172.21.15.62:0/866288951 shutdown_connections 2024-09-12T00:02:05.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.241+0000 7f6e5fb30740 1 -- 172.21.15.62:0/866288951 wait complete. 2024-09-12T00:02:05.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.241+0000 7f6e5fb30740 20 client.17175 trim_cache size 0 max 16384 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.258+0000 7f6e5fb30740 1 Processor -- start 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.258+0000 7f6e5fb30740 1 -- start start 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f337099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336f2150 0x560f33709f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581adf70 con 0x560f336e2a40 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581ab810 con 0x560f336f2150 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581a9680 con 0x560f336e3360 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e567fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e567fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336f0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51098/0 (socket says 172.21.15.62:51098) 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e567fc640 1 -- 172.21.15.62:0/2483190634 learned_addr learned my addr 172.21.15.62:0/2483190634 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:05.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e567fc640 1 -- 172.21.15.62:0/2483190634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 msgr2=0x560f337099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e567fc640 1 --2- 172.21.15.62:0/2483190634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f337099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e567fc640 1 -- 172.21.15.62:0/2483190634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336f2150 msgr2=0x560f33709f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e567fc640 1 --2- 172.21.15.62:0/2483190634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336f2150 0x560f33709f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e567fc640 1 -- 172.21.15.62:0/2483190634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f336def20 con 0x560f336e2a40 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.259+0000 7f6e567fc640 1 --2- 172.21.15.62:0/2483190634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336f0e90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6e40006010 tx=0x7f6e40032f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687b0502a7216da3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e4cff9640 1 -- 172.21.15.62:0/2483190634 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40035050 con 0x560f336e2a40 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e4cff9640 1 -- 172.21.15.62:0/2483190634 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4004e070 con 0x560f336e2a40 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e4cff9640 1 -- 172.21.15.62:0/2483190634 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e400044b0 con 0x560f336e2a40 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e5fb30740 1 -- 172.21.15.62:0/2483190634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 msgr2=0x560f336f0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/2483190634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336f0e90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6e40006010 tx=0x7f6e40032f50 comp rx=0 tx=0).stop 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e5fb30740 1 -- 172.21.15.62:0/2483190634 shutdown_connections 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/2483190634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336f2150 0x560f33709f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/2483190634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f337099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/2483190634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336e2a40 0x560f336f0e90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e5fb30740 1 -- 172.21.15.62:0/2483190634 >> 172.21.15.62:0/2483190634 conn(0x560f336df0f0 msgr2=0x560f336f0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:05.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e5fb30740 1 -- 172.21.15.62:0/2483190634 shutdown_connections 2024-09-12T00:02:05.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.260+0000 7f6e5fb30740 1 -- 172.21.15.62:0/2483190634 wait complete. 2024-09-12T00:02:05.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e5fb30740 1 -- start start 2024-09-12T00:02:05.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e5fb30740 1 Processor -- start 2024-09-12T00:02:05.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 0x560f336f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f336f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e5fb30740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336f2150 0x560f3376b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581a9cd0 con 0x560f336f2150 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581ab140 con 0x560f336e2a40 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e5fb30740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581aa3a0 con 0x560f336e3360 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e5fb30740 10 client.0 _ll_register_callbacks cb 0x7f6e40009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e567fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336f2150 0x560f3376b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e567fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336f2150 0x560f3376b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51102/0 (socket says 172.21.15.62:51102) 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e567fc640 1 -- 172.21.15.62:0/3739153436 learned_addr learned my addr 172.21.15.62:0/3739153436 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f336f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.261+0000 7f6e567fc640 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 0x560f336f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e567fc640 1 -- 172.21.15.62:0/3739153436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 msgr2=0x560f336f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e567fc640 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f336f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e567fc640 1 -- 172.21.15.62:0/3739153436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 msgr2=0x560f336f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e567fc640 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 0x560f336f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e567fc640 1 -- 172.21.15.62:0/3739153436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f336da830 con 0x560f336f2150 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f336f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e567fc640 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336f2150 0x560f3376b990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6e40004720 tx=0x7f6e40003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2360af5933ade250 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/3739153436 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40035070 con 0x560f336f2150 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/3739153436 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8513+0+0 (secure 0 0 0) 0x7f6e40004460 con 0x560f336f2150 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3739153436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e4004e070 con 0x560f336f2150 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/3739153436 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6e400319b0 con 0x560f336f2150 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e5fb30740 10 client.17181 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3739153436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560f3376cec0 con 0x560f336f2150 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/3739153436 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4004e070 con 0x560f336f2150 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/3739153436 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f6e40004800 con 0x560f336f2150 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e5fb30740 10 client.17181 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e5fb30740 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e5fb30740 20 client.17181 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.262+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560f3376d4d0 0x560f3376f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.263+0000 7f6e5fb30740 4 client.17181 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.263+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3739153436 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560f3364fd00 con 0x560f3376d4d0 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.263+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560f3376d4d0 0x560f3376f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.263+0000 7f6e3e7fc640 10 client.17181 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:05.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.263+0000 7f6e56ffd640 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560f3376d4d0 0x560f3376f8c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a681936471fcff server_cookie=234bffe417ea4b69 in_seq=0 out_seq=0 2024-09-12T00:02:05.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e3e7fc640 1 -- 172.21.15.62:0/3739153436 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560f3364fd00 con 0x560f3376d4d0 2024-09-12T00:02:05.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e3e7fc640 10 client.17181 handle_command_reply: tid=0 2024-09-12T00:02:05.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 client.17181 shutdown 2024-09-12T00:02:05.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3739153436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336f2150 msgr2=0x560f3376b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:05.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336f2150 0x560f3376b990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6e40004720 tx=0x7f6e40003a50 comp rx=0 tx=0).stop 2024-09-12T00:02:05.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3739153436 shutdown_connections 2024-09-12T00:02:05.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560f3376d4d0 0x560f3376f8c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f336f2150 0x560f3376b990 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f336e3360 0x560f336f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 --2- 172.21.15.62:0/3739153436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f336e2a40 0x560f336f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3739153436 >> 172.21.15.62:0/3739153436 conn(0x560f336df0f0 msgr2=0x560f336e4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:05.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e3e7fc640 0 client.17181 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:05.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3739153436 shutdown_connections 2024-09-12T00:02:05.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 1 -- 172.21.15.62:0/3739153436 wait complete. 2024-09-12T00:02:05.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.265+0000 7f6e5fb30740 20 client.17181 trim_cache size 0 max 16384 2024-09-12T00:02:05.276 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:02:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-12T00:02:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "38515281-bab6-4294-b5a6-c6f20e89f7ad", 2024-09-12T00:02:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-12T00:02:05.276 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:02:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 -- 172.21.15.62:0/2354411739 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6e440a2cf0 msgr2=0x7f6e440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6e440a2cf0 0x7f6e440a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e50097fb0 tx=0x7f6e5006b9b0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 -- 172.21.15.62:0/2354411739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e58106f50 msgr2=0x7f6e5819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e58106f50 0x7f6e5819cdd0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000eaa0 tx=0x7f6e5000eea0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 -- 172.21.15.62:0/2354411739 shutdown_connections 2024-09-12T00:02:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6e440a2cf0 0x7f6e440a51b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e58106f50 0x7f6e5819cdd0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e58100ca0 0x7f6e58198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 --2- 172.21.15.62:0/2354411739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e581002f0 0x7f6e58198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 -- 172.21.15.62:0/2354411739 >> 172.21.15.62:0/2354411739 conn(0x7f6e58077210 msgr2=0x7f6e58105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:05.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 -- 172.21.15.62:0/2354411739 shutdown_connections 2024-09-12T00:02:05.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:05.266+0000 7f6e5d757640 1 -- 172.21.15.62:0/2354411739 wait complete. 2024-09-12T00:02:05.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:05 smithi062 ceph-mon[30958]: pgmap v308: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 31 MiB/s wr, 1.86k op/s 2024-09-12T00:02:05.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:05 smithi120 ceph-mon[35182]: pgmap v308: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 31 MiB/s wr, 1.86k op/s 2024-09-12T00:02:05.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:05 smithi083 ceph-mon[34805]: pgmap v308: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 31 MiB/s wr, 1.86k op/s 2024-09-12T00:02:06.258 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:02:06.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:06 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-12T00:02:06.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:06 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:06 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:02:06.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:06 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:02:06.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:06 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:06 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:02:06.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:06 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:06 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.666 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:06.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:06 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-12T00:02:06.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:06 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:06 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:02:06.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:06 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:02:06.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:06 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:06 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:02:06.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:06 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:06 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:06 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-12T00:02:06.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:06 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:06 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:02:06.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:06 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:02:06.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:06 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:06 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:02:06.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:06 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:06.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:06 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.162+0000 7faeadcc7640 1 -- 172.21.15.62:0/3870237530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faea8102790 msgr2=0x7faea81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.162+0000 7faeadcc7640 1 --2- 172.21.15.62:0/3870237530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faea8102790 0x7faea81067e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fae90009900 tx=0x7fae9002a410 comp rx=0 tx=0).stop 2024-09-12T00:02:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.163+0000 7faeadcc7640 1 -- 172.21.15.62:0/3870237530 shutdown_connections 2024-09-12T00:02:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.163+0000 7faeadcc7640 1 --2- 172.21.15.62:0/3870237530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea8106f10 0x7faea810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.163+0000 7faeadcc7640 1 --2- 172.21.15.62:0/3870237530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faea8102790 0x7faea81067e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.163+0000 7faeadcc7640 1 --2- 172.21.15.62:0/3870237530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faea8101de0 0x7faea81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.163+0000 7faeadcc7640 1 -- 172.21.15.62:0/3870237530 >> 172.21.15.62:0/3870237530 conn(0x7faea80fd8c0 msgr2=0x7faea80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.163+0000 7faeadcc7640 1 -- 172.21.15.62:0/3870237530 shutdown_connections 2024-09-12T00:02:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.163+0000 7faeadcc7640 1 -- 172.21.15.62:0/3870237530 wait complete. 2024-09-12T00:02:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faeadcc7640 1 Processor -- start 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faeadcc7640 1 -- start start 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faeadcc7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faea8101de0 0x7faea8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faeadcc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faea8102790 0x7faea8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faeadcc7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea8106f10 0x7faea819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faeadcc7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faea8111f30 con 0x7faea8101de0 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faeadcc7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faea8111db0 con 0x7faea8106f10 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faeadcc7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faea81120b0 con 0x7faea8102790 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faea77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faea8101de0 0x7faea8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faea77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faea8101de0 0x7faea8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51116/0 (socket says 172.21.15.62:51116) 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faea77fe640 1 -- 172.21.15.62:0/3302233547 learned_addr learned my addr 172.21.15.62:0/3302233547 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faea7fff640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea8106f10 0x7faea819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faea6ffd640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faea8102790 0x7faea8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faea77fe640 1 -- 172.21.15.62:0/3302233547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faea8102790 msgr2=0x7faea8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.164+0000 7faea77fe640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faea8102790 0x7faea8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.165+0000 7faea77fe640 1 -- 172.21.15.62:0/3302233547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea8106f10 msgr2=0x7faea819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.165+0000 7faea77fe640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea8106f10 0x7faea819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.165+0000 7faea77fe640 1 -- 172.21.15.62:0/3302233547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea819d3c0 con 0x7faea8101de0 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.165+0000 7faea77fe640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faea8101de0 0x7faea8198410 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c920 tx=0x7fae9800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.165+0000 7faea4ff9640 1 -- 172.21.15.62:0/3302233547 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98007c70 con 0x7faea8101de0 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.165+0000 7faea4ff9640 1 -- 172.21.15.62:0/3302233547 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae98007e10 con 0x7faea8101de0 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.165+0000 7faeadcc7640 1 -- 172.21.15.62:0/3302233547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea819d6b0 con 0x7faea8101de0 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.165+0000 7faea4ff9640 1 -- 172.21.15.62:0/3302233547 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98018890 con 0x7faea8101de0 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.165+0000 7faeadcc7640 1 -- 172.21.15.62:0/3302233547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea81a4f90 con 0x7faea8101de0 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.167+0000 7faea4ff9640 1 -- 172.21.15.62:0/3302233547 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fae98018a70 con 0x7faea8101de0 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faeb00a0740 1 Processor -- start 2024-09-12T00:02:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faeb00a0740 1 -- start start 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd796880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd796dc0 0x55bebd7a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd7a1680 0x55bebd7a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faeb00a0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bebd6fc2e0 con 0x55bebd7964a0 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faeb00a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bebd6a3ce0 con 0x55bebd7a1680 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faeb00a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bebd6f82a0 con 0x55bebd796dc0 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faea7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd796880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faea7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd796880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51124/0 (socket says 172.21.15.62:51124) 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faea7fff640 1 -- 172.21.15.62:0/3988954026 learned_addr learned my addr 172.21.15.62:0/3988954026 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.168+0000 7faea77fe640 1 --2- 172.21.15.62:0/3988954026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd796dc0 0x55bebd7a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7faea6ffd640 1 --2- 172.21.15.62:0/3988954026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd7a1680 0x55bebd7a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7faea7fff640 1 -- 172.21.15.62:0/3988954026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd796dc0 msgr2=0x55bebd7a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7faea7fff640 1 --2- 172.21.15.62:0/3988954026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd796dc0 0x55bebd7a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7faea7fff640 1 -- 172.21.15.62:0/3988954026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd7a1680 msgr2=0x55bebd7a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7faea7fff640 1 --2- 172.21.15.62:0/3988954026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd7a1680 0x55bebd7a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7faea7fff640 1 -- 172.21.15.62:0/3988954026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bebd688440 con 0x55bebd7964a0 2024-09-12T00:02:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7faea4ff9640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fae940a2d70 0x7fae940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7faea77fe640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fae940a2d70 0x7fae940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7faea7fff640 1 --2- 172.21.15.62:0/3988954026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd796880 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7faea0009f20 tx=0x7faea000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70184aed555eec61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7faea4ff9640 1 -- 172.21.15.62:0/3302233547 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8557+0+0 (secure 0 0 0) 0x7fae980cf630 con 0x7faea8101de0 2024-09-12T00:02:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7fae9dffb640 1 -- 172.21.15.62:0/3988954026 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0015070 con 0x55bebd7964a0 2024-09-12T00:02:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7fae9dffb640 1 -- 172.21.15.62:0/3988954026 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea001ab40 con 0x55bebd7964a0 2024-09-12T00:02:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.169+0000 7fae9dffb640 1 -- 172.21.15.62:0/3988954026 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0011c40 con 0x55bebd7964a0 2024-09-12T00:02:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.170+0000 7faeb00a0740 1 -- 172.21.15.62:0/3988954026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 msgr2=0x55bebd796880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.170+0000 7faea77fe640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fae940a2d70 0x7fae940a5230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55bebd793890 tx=0x7fae98000bc0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.170+0000 7faeb00a0740 1 --2- 172.21.15.62:0/3988954026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd796880 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7faea0009f20 tx=0x7faea000f7f0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.170+0000 7faeb00a0740 1 -- 172.21.15.62:0/3988954026 shutdown_connections 2024-09-12T00:02:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.170+0000 7faeb00a0740 1 --2- 172.21.15.62:0/3988954026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd7a1680 0x55bebd7a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.170+0000 7faeb00a0740 1 --2- 172.21.15.62:0/3988954026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd796dc0 0x55bebd7a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.170+0000 7faeb00a0740 1 --2- 172.21.15.62:0/3988954026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd796880 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.170+0000 7faeb00a0740 1 -- 172.21.15.62:0/3988954026 >> 172.21.15.62:0/3988954026 conn(0x55bebd792a30 msgr2=0x55bebd792e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:07.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.171+0000 7faeb00a0740 1 -- 172.21.15.62:0/3988954026 shutdown_connections 2024-09-12T00:02:07.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.171+0000 7faeb00a0740 1 -- 172.21.15.62:0/3988954026 wait complete. 2024-09-12T00:02:07.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.171+0000 7faeb00a0740 1 -- start start 2024-09-12T00:02:07.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.171+0000 7faeb00a0740 1 Processor -- start 2024-09-12T00:02:07.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.171+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7964a0 0x55bebd7a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.171+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd796dc0 0x55bebd79e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.171+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd7a1680 0x55bebd79e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.171+0000 7faeb00a0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bebd7a4250 con 0x55bebd796dc0 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.171+0000 7faeb00a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bebd6b7050 con 0x55bebd7a1680 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faeb00a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bebd7a43d0 con 0x55bebd7964a0 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faeb00a0740 10 client.0 _ll_register_callbacks cb 0x55bebd6f7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7964a0 0x55bebd7a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd796dc0 0x55bebd79e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7964a0 0x55bebd7a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:37930/0 (socket says 172.21.15.62:37930) 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea7fff640 1 -- 172.21.15.62:0/1605604148 learned_addr learned my addr 172.21.15.62:0/1605604148 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd796dc0 0x55bebd79e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51128/0 (socket says 172.21.15.62:51128) 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea7fff640 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd7a1680 0x55bebd79e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea7fff640 1 -- 172.21.15.62:0/1605604148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd7a1680 msgr2=0x55bebd79e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea7fff640 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd7a1680 0x55bebd79e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea7fff640 1 -- 172.21.15.62:0/1605604148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd796dc0 msgr2=0x55bebd79e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea7fff640 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd796dc0 0x55bebd79e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea7fff640 1 -- 172.21.15.62:0/1605604148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bebd46d3c0 con 0x55bebd7964a0 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea6ffd640 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd796dc0 0x55bebd79e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7faea7fff640 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7964a0 0x55bebd7a0fa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faea0011780 tx=0x7faea0011920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c785b0a9960f2aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7fae86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.172+0000 7fae86ffd640 1 -- 172.21.15.62:0/1605604148 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0015070 con 0x55bebd7964a0 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7faeb00a0740 1 -- 172.21.15.62:0/1605604148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea001ab40 con 0x55bebd7964a0 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7fae86ffd640 1 -- 172.21.15.62:0/1605604148 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8557+0+0 (secure 0 0 0) 0x7faea0003970 con 0x55bebd7964a0 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7fae86ffd640 1 -- 172.21.15.62:0/1605604148 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faea0011e10 con 0x55bebd7964a0 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7faeb00a0740 10 client.26668 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7faeb00a0740 1 -- 172.21.15.62:0/1605604148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bebd7afbd0 con 0x55bebd7964a0 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7fae86ffd640 1 -- 172.21.15.62:0/1605604148 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea001ab40 con 0x55bebd7964a0 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7fae86ffd640 1 -- 172.21.15.62:0/1605604148 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7faea0028c40 con 0x55bebd7964a0 2024-09-12T00:02:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7faeb00a0740 10 client.26668 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:07.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7faeb00a0740 10 client.26668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:07.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7faeb00a0740 20 client.26668 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:07.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7faeb00a0740 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bebd7b0fe0 0x55bebd7b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7faeb00a0740 4 client.26668 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:07.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7faeb00a0740 1 -- 172.21.15.62:0/1605604148 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bebd709850 con 0x55bebd7b0fe0 2024-09-12T00:02:07.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.173+0000 7faea6ffd640 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bebd7b0fe0 0x55bebd7b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.174+0000 7faea6ffd640 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bebd7b0fe0 0x55bebd7b33d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd9b00fb207e281 server_cookie=e2b86b53e83e7b54 in_seq=0 out_seq=0 2024-09-12T00:02:07.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.174+0000 7fae86ffd640 10 client.26668 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:07.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7fae86ffd640 1 -- 172.21.15.62:0/1605604148 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bebd709850 con 0x55bebd7b0fe0 2024-09-12T00:02:07.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7fae86ffd640 10 client.26668 handle_command_reply: tid=0 2024-09-12T00:02:07.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 client.26668 shutdown 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 -- 172.21.15.62:0/1605604148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7964a0 msgr2=0x55bebd7a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7964a0 0x55bebd7a0fa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faea0011780 tx=0x7faea0011920 comp rx=0 tx=0).stop 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 -- 172.21.15.62:0/1605604148 shutdown_connections 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bebd7b0fe0 0x55bebd7b33d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd7a1680 0x55bebd79e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd796dc0 0x55bebd79e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7fae86ffd640 0 client.26668 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 --2- 172.21.15.62:0/1605604148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7964a0 0x55bebd7a0fa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 -- 172.21.15.62:0/1605604148 >> 172.21.15.62:0/1605604148 conn(0x55bebd792a30 msgr2=0x55bebd791b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 -- 172.21.15.62:0/1605604148 shutdown_connections 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 1 -- 172.21.15.62:0/1605604148 wait complete. 2024-09-12T00:02:07.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.175+0000 7faeb00a0740 20 client.26668 trim_cache size 0 max 16384 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.192+0000 7faeb00a0740 1 Processor -- start 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.192+0000 7faeb00a0740 1 -- start start 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.192+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd7ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.192+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd796dc0 0x55bebd7b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.192+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7a1680 0x55bebd7b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.192+0000 7faeb00a0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faea81adfb0 con 0x55bebd7964a0 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.192+0000 7faeb00a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ab850 con 0x55bebd796dc0 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.192+0000 7faeb00a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faea81a96c0 con 0x55bebd7a1680 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7faea7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd7ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7faea7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd7ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51130/0 (socket says 172.21.15.62:51130) 2024-09-12T00:02:07.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7faea7fff640 1 -- 172.21.15.62:0/2792600300 learned_addr learned my addr 172.21.15.62:0/2792600300 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:07.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7faea6ffd640 1 --2- 172.21.15.62:0/2792600300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd796dc0 0x55bebd7b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7faea7fff640 1 -- 172.21.15.62:0/2792600300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7a1680 msgr2=0x55bebd7b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:07.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7faea7fff640 1 --2- 172.21.15.62:0/2792600300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7a1680 0x55bebd7b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7faea7fff640 1 -- 172.21.15.62:0/2792600300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd796dc0 msgr2=0x55bebd7b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:07.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7faea7fff640 1 --2- 172.21.15.62:0/2792600300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd796dc0 0x55bebd7b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7faea7fff640 1 -- 172.21.15.62:0/2792600300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bebd7962d0 con 0x55bebd7964a0 2024-09-12T00:02:07.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7faea7fff640 1 --2- 172.21.15.62:0/2792600300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd7ad4b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7faea0030480 tx=0x7faea0009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c66c717cd983a51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:07.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7fae9d7fa640 1 -- 172.21.15.62:0/2792600300 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0015070 con 0x55bebd7964a0 2024-09-12T00:02:07.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7fae9d7fa640 1 -- 172.21.15.62:0/2792600300 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea001e030 con 0x55bebd7964a0 2024-09-12T00:02:07.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.193+0000 7fae9d7fa640 1 -- 172.21.15.62:0/2792600300 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0003cd0 con 0x55bebd7964a0 2024-09-12T00:02:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 -- 172.21.15.62:0/2792600300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 msgr2=0x55bebd7ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 --2- 172.21.15.62:0/2792600300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd7ad4b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7faea0030480 tx=0x7faea0009a60 comp rx=0 tx=0).stop 2024-09-12T00:02:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 -- 172.21.15.62:0/2792600300 shutdown_connections 2024-09-12T00:02:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 --2- 172.21.15.62:0/2792600300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7a1680 0x55bebd7b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 --2- 172.21.15.62:0/2792600300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd796dc0 0x55bebd7b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 --2- 172.21.15.62:0/2792600300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd7ad4b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 -- 172.21.15.62:0/2792600300 >> 172.21.15.62:0/2792600300 conn(0x55bebd792a30 msgr2=0x55bebd798710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:07.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 -- 172.21.15.62:0/2792600300 shutdown_connections 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 -- 172.21.15.62:0/2792600300 wait complete. 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 -- start start 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.194+0000 7faeb00a0740 1 Processor -- start 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd813cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd796dc0 0x55bebd814200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faeb00a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7a1680 0x55bebd7a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faeb00a0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faea81a9d10 con 0x55bebd7964a0 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faeb00a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ab180 con 0x55bebd796dc0 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faeb00a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faea81aa3e0 con 0x55bebd7a1680 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faeb00a0740 10 client.0 _ll_register_callbacks cb 0x7faea0009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd813cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd813cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51134/0 (socket says 172.21.15.62:51134) 2024-09-12T00:02:07.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea7fff640 1 -- 172.21.15.62:0/331161726 learned_addr learned my addr 172.21.15.62:0/331161726 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea6ffd640 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd796dc0 0x55bebd814200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea7fff640 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7a1680 0x55bebd7a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea7fff640 1 -- 172.21.15.62:0/331161726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7a1680 msgr2=0x55bebd7a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea7fff640 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7a1680 0x55bebd7a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea7fff640 1 -- 172.21.15.62:0/331161726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd796dc0 msgr2=0x55bebd814200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea7fff640 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd796dc0 0x55bebd814200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea7fff640 1 -- 172.21.15.62:0/331161726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bebd7a53d0 con 0x55bebd7964a0 2024-09-12T00:02:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7faea7fff640 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd813cc0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7faea001d040 tx=0x7faea0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8caf1e9a474a025d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7fae86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.195+0000 7fae86ffd640 1 -- 172.21.15.62:0/331161726 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0015050 con 0x55bebd7964a0 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7fae86ffd640 1 -- 172.21.15.62:0/331161726 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8557+0+0 (secure 0 0 0) 0x7faea0030d10 con 0x55bebd7964a0 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7faeb00a0740 1 -- 172.21.15.62:0/331161726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea001e030 con 0x55bebd7964a0 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7fae86ffd640 1 -- 172.21.15.62:0/331161726 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faea0049050 con 0x55bebd7964a0 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7faeb00a0740 10 client.17199 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7faeb00a0740 1 -- 172.21.15.62:0/331161726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bebd81b410 con 0x55bebd7964a0 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7fae86ffd640 1 -- 172.21.15.62:0/331161726 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea001e030 con 0x55bebd7964a0 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7fae86ffd640 1 -- 172.21.15.62:0/331161726 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7faea0037020 con 0x55bebd7964a0 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7faeb00a0740 10 client.17199 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7faeb00a0740 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:07.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7faeb00a0740 20 client.17199 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7faeb00a0740 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bebd81baf0 0x55bebd81df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7faeb00a0740 4 client.17199 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7faeb00a0740 1 -- 172.21.15.62:0/331161726 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bebd703d60 con 0x55bebd81baf0 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.196+0000 7faea6ffd640 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bebd81baf0 0x55bebd81df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7fae86ffd640 10 client.17199 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faea6ffd640 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bebd81baf0 0x55bebd81df10 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c297cc3f6d689f4d server_cookie=9e6170c1a4e04c15 in_seq=0 out_seq=0 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7fae86ffd640 1 -- 172.21.15.62:0/331161726 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bebd703d60 con 0x55bebd81baf0 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7fae86ffd640 10 client.17199 handle_command_reply: tid=0 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 client.17199 shutdown 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 -- 172.21.15.62:0/331161726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 msgr2=0x55bebd813cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd813cc0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7faea001d040 tx=0x7faea0003970 comp rx=0 tx=0).stop 2024-09-12T00:02:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 -- 172.21.15.62:0/331161726 shutdown_connections 2024-09-12T00:02:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bebd81baf0 0x55bebd81df10 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bebd7a1680 0x55bebd7a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bebd796dc0 0x55bebd814200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 --2- 172.21.15.62:0/331161726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bebd7964a0 0x55bebd813cc0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 -- 172.21.15.62:0/331161726 >> 172.21.15.62:0/331161726 conn(0x55bebd792a30 msgr2=0x55bebd798f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7fae86ffd640 0 client.17199 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 -- 172.21.15.62:0/331161726 shutdown_connections 2024-09-12T00:02:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.197+0000 7faeb00a0740 1 -- 172.21.15.62:0/331161726 wait complete. 2024-09-12T00:02:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeb00a0740 20 client.17199 trim_cache size 0 max 16384 2024-09-12T00:02:07.203 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:02:07.203 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:02:07.203 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-12T00:02:07.203 INFO:teuthology.orchestra.run.smithi062.stdout: "38515281-bab6-4294-b5a6-c6f20e89f7ad": { 2024-09-12T00:02:07.203 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-12T00:02:07.203 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "38515281-bab6-4294-b5a6-c6f20e89f7ad", 2024-09-12T00:02:07.203 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-12T00:02:07.203 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:07.204 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:07.204 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 -- 172.21.15.62:0/3302233547 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fae940a2d70 msgr2=0x7fae940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fae940a2d70 0x7fae940a5230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55bebd793890 tx=0x7fae98000bc0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 -- 172.21.15.62:0/3302233547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faea8101de0 msgr2=0x7faea8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faea8101de0 0x7faea8198410 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c920 tx=0x7fae9800cde0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 -- 172.21.15.62:0/3302233547 shutdown_connections 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fae940a2d70 0x7fae940a5230 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea8106f10 0x7faea819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faea8102790 0x7faea8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 --2- 172.21.15.62:0/3302233547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faea8101de0 0x7faea8198410 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 -- 172.21.15.62:0/3302233547 >> 172.21.15.62:0/3302233547 conn(0x7faea80fd8c0 msgr2=0x7faea80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:07.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 -- 172.21.15.62:0/3302233547 shutdown_connections 2024-09-12T00:02:07.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:07.198+0000 7faeadcc7640 1 -- 172.21.15.62:0/3302233547 wait complete. 2024-09-12T00:02:07.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 565.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:02:07.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:07 smithi120 ceph-mon[35182]: pgmap v309: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 24 MiB/s wr, 1.57k op/s 2024-09-12T00:02:07.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:07 smithi120 ceph-mon[35182]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T00:02:07.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:07 smithi083 ceph-mon[34805]: pgmap v309: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 24 MiB/s wr, 1.57k op/s 2024-09-12T00:02:07.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:07 smithi083 ceph-mon[34805]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T00:02:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:07 smithi062 ceph-mon[30958]: pgmap v309: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 24 MiB/s wr, 1.57k op/s 2024-09-12T00:02:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:07 smithi062 ceph-mon[30958]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T00:02:07.903 INFO:tasks.cephfs.filesystem:scrub status for tag:38515281-bab6-4294-b5a6-c6f20e89f7ad - {'path': '/', 'tag': '38515281-bab6-4294-b5a6-c6f20e89f7ad', 'options': 'recursive,force'} 2024-09-12T00:02:08.215 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:02:08 smithi062 sudo[106120]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:08.215 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:02:08 smithi062 sudo[106120]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:08.216 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:02:08 smithi062 sudo[106120]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:08.216 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:02:08 smithi062 sudo[106120]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:08.965 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:02:08 smithi062 sudo[106123]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:08.965 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:02:08 smithi062 sudo[106123]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:08.965 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:02:08 smithi062 sudo[106123]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:08.965 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:02:08 smithi062 sudo[106123]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:09.074 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:02:08 smithi083 sudo[68816]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:09.075 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:02:08 smithi083 sudo[68816]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:09.075 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:02:08 smithi083 sudo[68816]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:09.075 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:02:08 smithi083 sudo[68816]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:09.371 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:02:09 smithi083 sudo[68819]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:09.372 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:02:09 smithi083 sudo[68819]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:09.372 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:02:09 smithi083 sudo[68819]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:09.372 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:02:09 smithi083 sudo[68819]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:09.620 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:02:09 smithi120 sudo[70428]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:09.620 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:02:09 smithi120 sudo[70428]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:09.621 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:02:09 smithi120 sudo[70428]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:09.621 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:02:09 smithi120 sudo[70428]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:09.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:09 smithi120 ceph-mon[35182]: pgmap v311: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 9.0 MiB/s wr, 538 op/s 2024-09-12T00:02:09.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:09.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:09 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:09.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:09 smithi083 ceph-mon[34805]: pgmap v311: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 9.0 MiB/s wr, 538 op/s 2024-09-12T00:02:09.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:09.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:09 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:09.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:09 smithi062 ceph-mon[30958]: pgmap v311: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 9.0 MiB/s wr, 538 op/s 2024-09-12T00:02:09.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:09.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:09 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:10.120 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:02:09 smithi120 sudo[70431]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:10.120 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:02:09 smithi120 sudo[70431]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:10.120 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:02:09 smithi120 sudo[70431]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:10.120 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:02:09 smithi120 sudo[70431]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:10.465 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:02:10 smithi062 sudo[106126]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:10.466 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:02:10 smithi062 sudo[106126]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:10.466 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:02:10 smithi062 sudo[106126]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:10.466 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:02:10 smithi062 sudo[106126]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:10.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:10 smithi062 ceph-mon[30958]: pgmap v312: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-09-12T00:02:10.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:10 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:10.965 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:02:10 smithi062 sudo[106129]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:10.966 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:02:10 smithi062 sudo[106129]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:10.966 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:02:10 smithi062 sudo[106129]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:10.966 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:02:10 smithi062 sudo[106129]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:11.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:10 smithi120 ceph-mon[35182]: pgmap v312: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-09-12T00:02:11.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:10 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:11.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:10 smithi083 ceph-mon[34805]: pgmap v312: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-09-12T00:02:11.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:10 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:11.167 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:02:10 smithi083 sudo[68822]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:11.167 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:02:10 smithi083 sudo[68822]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:11.167 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:02:10 smithi083 sudo[68822]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:11.167 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:02:10 smithi083 sudo[68822]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:11.667 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:02:11 smithi083 sudo[68825]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:11.667 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:02:11 smithi083 sudo[68825]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:11.667 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:02:11 smithi083 sudo[68825]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:11.667 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:02:11 smithi083 sudo[68825]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:11.723 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:02:11 smithi120 sudo[70435]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:11.723 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:02:11 smithi120 sudo[70435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:11.723 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:02:11 smithi120 sudo[70435]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:11.723 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:02:11 smithi120 sudo[70435]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:11.724 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:11 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:02:11.724 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:11 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:02:11.724 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:11 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:02:11.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:11 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:02:11.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:11 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:02:11.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:11 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:02:12.120 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:02:11 smithi120 sudo[70438]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:12.121 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:02:11 smithi120 sudo[70438]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:12.121 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:02:11 smithi120 sudo[70438]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:12.121 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:02:11 smithi120 sudo[70438]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:12.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:11 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:02:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:11 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:02:12.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:11 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:02:12.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 570.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:02:12.465 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:02:12 smithi062 sudo[106132]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:12.465 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:02:12 smithi062 sudo[106132]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:12.465 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:02:12 smithi062 sudo[106132]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:12.465 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:02:12 smithi062 sudo[106132]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:12.947 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:12 smithi083 ceph-mon[34805]: pgmap v313: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.0 MiB/s wr, 174 op/s 2024-09-12T00:02:12.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:12 smithi062 ceph-mon[30958]: pgmap v313: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.0 MiB/s wr, 174 op/s 2024-09-12T00:02:12.965 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:02:12 smithi062 sudo[106136]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:12.966 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:02:12 smithi062 sudo[106136]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:12.966 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:02:12 smithi062 sudo[106136]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:12.966 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:02:12 smithi062 sudo[106136]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:13.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:12 smithi120 ceph-mon[35182]: pgmap v313: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.0 MiB/s wr, 174 op/s 2024-09-12T00:02:13.223 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:02:12 smithi083 sudo[68830]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:13.223 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:02:12 smithi083 sudo[68830]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:13.223 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:02:12 smithi083 sudo[68830]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:13.223 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:02:12 smithi083 sudo[68830]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:13.667 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:02:13 smithi083 sudo[68833]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:13.667 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:02:13 smithi083 sudo[68833]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:13.667 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:02:13 smithi083 sudo[68833]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:13.667 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:02:13 smithi083 sudo[68833]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:13.764 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:02:13 smithi120 sudo[70441]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:13.764 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:02:13 smithi120 sudo[70441]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:13.764 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:02:13 smithi120 sudo[70441]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:13.764 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:02:13 smithi120 sudo[70441]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:14.121 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:02:13 smithi120 sudo[70444]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:14.121 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:02:13 smithi120 sudo[70444]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:14.121 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:02:13 smithi120 sudo[70444]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:14.121 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:02:13 smithi120 sudo[70444]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:14.690 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:02:14 smithi062 sudo[106139]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:14.690 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:02:14 smithi062 sudo[106139]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:14.691 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:02:14 smithi062 sudo[106139]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:14.691 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:02:14 smithi062 sudo[106139]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:14 smithi062 ceph-mon[30958]: pgmap v314: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 177 KiB/s wr, 20 op/s 2024-09-12T00:02:14.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:02:14 smithi062 sudo[106142]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:14.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:02:14 smithi062 sudo[106142]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:14.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:02:14 smithi062 sudo[106142]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:14.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:02:14 smithi062 sudo[106142]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:14.977 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:14 smithi083 ceph-mon[34805]: pgmap v314: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 177 KiB/s wr, 20 op/s 2024-09-12T00:02:15.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:14 smithi120 ceph-mon[35182]: pgmap v314: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 177 KiB/s wr, 20 op/s 2024-09-12T00:02:15.255 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:02:14 smithi083 sudo[68836]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:15.256 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:02:14 smithi083 sudo[68836]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:15.256 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:02:14 smithi083 sudo[68836]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:15.256 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:02:14 smithi083 sudo[68836]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:15.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:02:15 smithi083 sudo[68839]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:15.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:02:15 smithi083 sudo[68839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:15.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:02:15 smithi083 sudo[68839]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:15.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:02:15 smithi083 sudo[68839]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:15.804 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:02:15 smithi120 sudo[70447]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:15.804 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:02:15 smithi120 sudo[70447]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:15.804 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:02:15 smithi120 sudo[70447]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:15.805 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:02:15 smithi120 sudo[70447]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:15.864 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:02:16.120 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:02:15 smithi120 sudo[70450]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:16.120 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:02:15 smithi120 sudo[70450]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:16.120 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:02:15 smithi120 sudo[70450]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:16.121 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:02:15 smithi120 sudo[70450]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:16.130 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:16.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:16 smithi062 sudo[106216]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:16.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:16 smithi062 sudo[106216]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:16.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:16 smithi062 sudo[106216]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:16.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:16 smithi062 sudo[106216]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:16.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 -- 172.21.15.62:0/864717496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda88102570 msgr2=0x7fda881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:16.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 --2- 172.21.15.62:0/864717496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda88102570 0x7fda881029b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fda70009900 tx=0x7fda7002a480 comp rx=0 tx=0).stop 2024-09-12T00:02:16.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 -- 172.21.15.62:0/864717496 shutdown_connections 2024-09-12T00:02:16.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 --2- 172.21.15.62:0/864717496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda88102ef0 0x7fda8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:16.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 --2- 172.21.15.62:0/864717496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda88102570 0x7fda881029b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:16.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 --2- 172.21.15.62:0/864717496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda88100e20 0x7fda88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:16.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 -- 172.21.15.62:0/864717496 >> 172.21.15.62:0/864717496 conn(0x7fda880fc7d0 msgr2=0x7fda880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 -- 172.21.15.62:0/864717496 shutdown_connections 2024-09-12T00:02:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 -- 172.21.15.62:0/864717496 wait complete. 2024-09-12T00:02:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 Processor -- start 2024-09-12T00:02:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.741+0000 7fda8e0ec640 1 -- start start 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda8e0ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda88100e20 0x7fda880fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda8e0ec640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda88102570 0x7fda880fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda8e0ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda88102ef0 0x7fda880fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda8e0ec640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda88110f80 con 0x7fda88102570 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda8e0ec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda88110e00 con 0x7fda88100e20 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda8e0ec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda88111100 con 0x7fda88102ef0 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda7ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda88102570 0x7fda880fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda88100e20 0x7fda880fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda7ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda88102570 0x7fda880fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43070/0 (socket says 172.21.15.62:43070) 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda7ffff640 1 -- 172.21.15.62:0/261330360 learned_addr learned my addr 172.21.15.62:0/261330360 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda7ffff640 1 -- 172.21.15.62:0/261330360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda88102ef0 msgr2=0x7fda880fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda7ffff640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda88102ef0 0x7fda880fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda7ffff640 1 -- 172.21.15.62:0/261330360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda88100e20 msgr2=0x7fda880fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda7ffff640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda88100e20 0x7fda880fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda7ffff640 1 -- 172.21.15.62:0/261330360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda881aaee0 con 0x7fda88102570 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.742+0000 7fda877fe640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda88100e20 0x7fda880fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.743+0000 7fda7ffff640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda88102570 0x7fda880fd1c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fda70006010 tx=0x7fda70003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.743+0000 7fda857fa640 1 -- 172.21.15.62:0/261330360 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7003f070 con 0x7fda88102570 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.743+0000 7fda857fa640 1 -- 172.21.15.62:0/261330360 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda70031c30 con 0x7fda88102570 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.743+0000 7fda8e0ec640 1 -- 172.21.15.62:0/261330360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda881ab0b0 con 0x7fda88102570 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.743+0000 7fda857fa640 1 -- 172.21.15.62:0/261330360 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70035030 con 0x7fda88102570 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.743+0000 7fda8e0ec640 1 -- 172.21.15.62:0/261330360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda881ab640 con 0x7fda88102570 2024-09-12T00:02:16.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.744+0000 7fda8e0ec640 1 -- 172.21.15.62:0/261330360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda881021b0 con 0x7fda88102570 2024-09-12T00:02:16.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.746+0000 7fda857fa640 1 -- 172.21.15.62:0/261330360 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fda70042070 con 0x7fda88102570 2024-09-12T00:02:16.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.746+0000 7fda857fa640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fda740a2d70 0x7fda740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:16.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.746+0000 7fda877fe640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fda740a2d70 0x7fda740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:16.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.746+0000 7fda857fa640 1 -- 172.21.15.62:0/261330360 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fda700f0040 con 0x7fda88102570 2024-09-12T00:02:16.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.747+0000 7fda877fe640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fda740a2d70 0x7fda740a5230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fda780095f0 tx=0x7fda78005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:16.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.750+0000 7fda857fa640 1 -- 172.21.15.62:0/261330360 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda700bcc60 con 0x7fda88102570 2024-09-12T00:02:16.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:16 smithi062 sudo[106236]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:16.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:16 smithi062 sudo[106236]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:16.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:16 smithi062 sudo[106236]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:16.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:16 smithi062 sudo[106236]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:16.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.967+0000 7fda8e0ec640 1 -- 172.21.15.62:0/261330360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda88101200 con 0x7fda88102570 2024-09-12T00:02:16.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.967+0000 7fda857fa640 1 -- 172.21.15.62:0/261330360 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11911 (secure 0 0 0) 0x7fda88101200 con 0x7fda88102570 2024-09-12T00:02:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:02:16.967 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":33,"btime":"2024-09-12T00:01:52:327104+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:01:52.081596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:02:16.968 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 33 2024-09-12T00:02:16.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.969+0000 7fda8e0ec640 1 -- 172.21.15.62:0/261330360 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fda740a2d70 msgr2=0x7fda740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.969+0000 7fda8e0ec640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fda740a2d70 0x7fda740a5230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fda780095f0 tx=0x7fda78005f90 comp rx=0 tx=0).stop 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.969+0000 7fda8e0ec640 1 -- 172.21.15.62:0/261330360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda88102570 msgr2=0x7fda880fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.969+0000 7fda8e0ec640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda88102570 0x7fda880fd1c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fda70006010 tx=0x7fda70003c10 comp rx=0 tx=0).stop 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.969+0000 7fda8e0ec640 1 -- 172.21.15.62:0/261330360 shutdown_connections 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.969+0000 7fda8e0ec640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fda740a2d70 0x7fda740a5230 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.969+0000 7fda8e0ec640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda88102ef0 0x7fda880fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.969+0000 7fda8e0ec640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda88102570 0x7fda880fd1c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.969+0000 7fda8e0ec640 1 --2- 172.21.15.62:0/261330360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda88100e20 0x7fda880fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.969+0000 7fda8e0ec640 1 -- 172.21.15.62:0/261330360 >> 172.21.15.62:0/261330360 conn(0x7fda880fc7d0 msgr2=0x7fda8806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.970+0000 7fda8e0ec640 1 -- 172.21.15.62:0/261330360 shutdown_connections 2024-09-12T00:02:16.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:16.970+0000 7fda8e0ec640 1 -- 172.21.15.62:0/261330360 wait complete. 2024-09-12T00:02:17.247 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:16 smithi120 sudo[70454]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:17.247 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:16 smithi120 sudo[70454]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:17.247 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:16 smithi120 sudo[70454]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:17.247 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 sudo[70454]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 575.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:02:17.512 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 ceph-mon[34805]: pgmap v315: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 177 KiB/s wr, 20 op/s 2024-09-12T00:02:17.512 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:02:17.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:02:17.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:02:17.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 ceph-mon[34805]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T00:02:17.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:02:17.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:02:17.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:02:17.514 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:02:17.514 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:02:17.514 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 sudo[68843]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:02:17.514 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 sudo[68843]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:17.514 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 sudo[68843]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:17.514 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 sudo[68843]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:17.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 sudo[70457]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 sudo[70457]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 sudo[70457]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 sudo[70457]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 ceph-mon[35182]: pgmap v315: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 177 KiB/s wr, 20 op/s 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 ceph-mon[35182]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:02:17.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:02:17.622 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:17 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:02:17.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:17 smithi062 ceph-mon[30958]: pgmap v315: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 177 KiB/s wr, 20 op/s 2024-09-12T00:02:17.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:17 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:02:17.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:17 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:02:17.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:02:17.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:17 smithi062 ceph-mon[30958]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T00:02:17.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:02:17.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:02:17.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:02:17.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:02:17.694 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:17 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:02:17.726 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 494e38e3 --release --await' 2024-09-12T00:02:17.726 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 494e38e3 --release --await 2024-09-12T00:02:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 sudo[68847]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:02:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 sudo[68847]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:02:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 sudo[68847]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:02:17.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:17 smithi083 sudo[68847]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:02:17.991 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:18.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:18 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:02:18.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:18 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/261330360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:18.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:18 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:02:18.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:18 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:02:18.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:18 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:02:18.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:18 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:02:18.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:18 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:02:18.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:18 smithi062 ceph-mon[30958]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:02:18.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.480+0000 7fdafc8af640 1 -- 172.21.15.62:0/630588684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf8102790 msgr2=0x7fdaf81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:18.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.480+0000 7fdafc8af640 1 --2- 172.21.15.62:0/630588684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf8102790 0x7fdaf81067e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009900 tx=0x7fdaec02a410 comp rx=0 tx=0).stop 2024-09-12T00:02:18.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.480+0000 7fdafc8af640 1 -- 172.21.15.62:0/630588684 shutdown_connections 2024-09-12T00:02:18.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.480+0000 7fdafc8af640 1 --2- 172.21.15.62:0/630588684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf8106f10 0x7fdaf810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.480+0000 7fdafc8af640 1 --2- 172.21.15.62:0/630588684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf8102790 0x7fdaf81067e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.480+0000 7fdafc8af640 1 --2- 172.21.15.62:0/630588684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf8101de0 0x7fdaf81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.480+0000 7fdafc8af640 1 -- 172.21.15.62:0/630588684 >> 172.21.15.62:0/630588684 conn(0x7fdaf80fd8c0 msgr2=0x7fdaf80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.480+0000 7fdafc8af640 1 -- 172.21.15.62:0/630588684 shutdown_connections 2024-09-12T00:02:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.480+0000 7fdafc8af640 1 -- 172.21.15.62:0/630588684 wait complete. 2024-09-12T00:02:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.481+0000 7fdafc8af640 1 Processor -- start 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.481+0000 7fdafc8af640 1 -- start start 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.481+0000 7fdafc8af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf8101de0 0x7fdaf8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.481+0000 7fdafc8af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf8102790 0x7fdaf81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.481+0000 7fdafc8af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf8106f10 0x7fdaf819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.481+0000 7fdafc8af640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8111f30 con 0x7fdaf8102790 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.481+0000 7fdafc8af640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8111db0 con 0x7fdaf8106f10 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.481+0000 7fdafc8af640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81120b0 con 0x7fdaf8101de0 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.481+0000 7fdaf5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf8102790 0x7fdaf81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.481+0000 7fdaf5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf8102790 0x7fdaf81988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43092/0 (socket says 172.21.15.62:43092) 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdaf5d9b640 1 -- 172.21.15.62:0/504983687 learned_addr learned my addr 172.21.15.62:0/504983687 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf8106f10 0x7fdaf819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdaf659c640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf8101de0 0x7fdaf8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdaf5d9b640 1 -- 172.21.15.62:0/504983687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf8101de0 msgr2=0x7fdaf8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf8101de0 0x7fdaf8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdaf5d9b640 1 -- 172.21.15.62:0/504983687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf8106f10 msgr2=0x7fdaf819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf8106f10 0x7fdaf819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdaf5d9b640 1 -- 172.21.15.62:0/504983687 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf819d340 con 0x7fdaf8102790 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdaf659c640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf8101de0 0x7fdaf8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf8102790 0x7fdaf81988d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdaec006010 tx=0x7fdaec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdae77fe640 1 -- 172.21.15.62:0/504983687 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec03f030 con 0x7fdaf8102790 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdafc8af640 1 -- 172.21.15.62:0/504983687 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf819d5d0 con 0x7fdaf8102790 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdae77fe640 1 -- 172.21.15.62:0/504983687 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaec003ce0 con 0x7fdaf8102790 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdae77fe640 1 -- 172.21.15.62:0/504983687 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec035070 con 0x7fdaf8102790 2024-09-12T00:02:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.482+0000 7fdafc8af640 1 -- 172.21.15.62:0/504983687 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81a4f10 con 0x7fdaf8102790 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.485+0000 7fdae77fe640 1 -- 172.21.15.62:0/504983687 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdaec042030 con 0x7fdaf8102790 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.485+0000 7fdafec88740 1 Processor -- start 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.485+0000 7fdafec88740 1 -- start start 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.485+0000 7fdafec88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a77753340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdafec88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a77762110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdafec88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 0x555a77764a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdafec88740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555a776b8620 con 0x555a77752f60 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdafec88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a7765fe50 con 0x555a77762650 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdafec88740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a776b45e0 con 0x555a77753880 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a77753340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a77753340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43096/0 (socket says 172.21.15.62:43096) 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/2919042747 learned_addr learned my addr 172.21.15.62:0/2919042747 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf659c640 1 --2- 172.21.15.62:0/2919042747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a77762110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdae77fe640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdae00a2d70 0x7fdae00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/2919042747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 msgr2=0x555a77762110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/2919042747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a77762110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/2919042747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 msgr2=0x555a77764a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdae77fe640 1 -- 172.21.15.62:0/504983687 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fdaec0efb70 con 0x7fdaf8102790 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/2919042747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 0x555a77764a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/2919042747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 0x555a77764a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/2919042747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a77650d40 con 0x555a77752f60 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/2919042747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 0x555a77764a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdae00a2d70 0x7fdae00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.486+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/2919042747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a77753340 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0010440 tx=0x7fdaf0010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629ac539b3ad29c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdae4ff9640 1 -- 172.21.15.62:0/2919042747 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf001d400 con 0x555a77752f60 2024-09-12T00:02:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdae4ff9640 1 -- 172.21.15.62:0/2919042747 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf001da80 con 0x555a77752f60 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdae4ff9640 1 -- 172.21.15.62:0/2919042747 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf001c8f0 con 0x555a77752f60 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdafec88740 1 -- 172.21.15.62:0/2919042747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 msgr2=0x555a77753340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdae00a2d70 0x7fdae00a5230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0021000 tx=0x7fdaf001e000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdafec88740 1 --2- 172.21.15.62:0/2919042747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a77753340 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0010440 tx=0x7fdaf0010900 comp rx=0 tx=0).stop 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdafec88740 1 -- 172.21.15.62:0/2919042747 shutdown_connections 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdafec88740 1 --2- 172.21.15.62:0/2919042747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 0x555a77764a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdafec88740 1 --2- 172.21.15.62:0/2919042747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a77762110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdafec88740 1 --2- 172.21.15.62:0/2919042747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a77753340 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.487+0000 7fdafec88740 1 -- 172.21.15.62:0/2919042747 >> 172.21.15.62:0/2919042747 conn(0x555a7774f5f0 msgr2=0x555a7774f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 1 -- 172.21.15.62:0/2919042747 shutdown_connections 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 1 -- 172.21.15.62:0/2919042747 wait complete. 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 1 -- start start 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 1 Processor -- start 2024-09-12T00:02:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a7775f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a7775fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 0x555a77760080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555a77765280 con 0x555a77752f60 2024-09-12T00:02:18.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a77640f10 con 0x555a77762650 2024-09-12T00:02:18.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a77765400 con 0x555a77753880 2024-09-12T00:02:18.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdafec88740 10 client.0 _ll_register_callbacks cb 0x555a77664db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:18.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdaf659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a7775f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdaf659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a7775f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43108/0 (socket says 172.21.15.62:43108) 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.488+0000 7fdaf659c640 1 -- 172.21.15.62:0/1652564584 learned_addr learned my addr 172.21.15.62:0/1652564584 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 0x555a77760080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a7775fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaf659c640 1 -- 172.21.15.62:0/1652564584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 msgr2=0x555a7775fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaf659c640 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a7775fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaf659c640 1 -- 172.21.15.62:0/1652564584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 msgr2=0x555a77760080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaf659c640 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 0x555a77760080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaf659c640 1 -- 172.21.15.62:0/1652564584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a773c7580 con 0x555a77752f60 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a7775fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaf659c640 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a7775f600 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fdae80117c0 tx=0x7fdae8011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d74adcfb8c1988 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaddffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaddffb640 1 -- 172.21.15.62:0/1652564584 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800f9a0 con 0x555a77752f60 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaddffb640 1 -- 172.21.15.62:0/1652564584 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fdae800fb40 con 0x555a77752f60 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdafec88740 1 -- 172.21.15.62:0/1652564584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf001da80 con 0x555a77752f60 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdaddffb640 1 -- 172.21.15.62:0/1652564584 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdae800fd40 con 0x555a77752f60 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdafec88740 10 client.17220 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.489+0000 7fdafec88740 1 -- 172.21.15.62:0/1652564584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555a77491960 con 0x555a77752f60 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdaddffb640 1 -- 172.21.15.62:0/1652564584 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf001da80 con 0x555a77752f60 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdaddffb640 1 -- 172.21.15.62:0/1652564584 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fdae8018030 con 0x555a77752f60 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdafec88740 10 client.17220 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdafec88740 10 client.17220 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdafec88740 20 client.17220 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdafec88740 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a777722f0 0x555a77774780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdafec88740 4 client.17220 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdafec88740 1 -- 172.21.15.62:0/1652564584 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a776c5b90 con 0x555a777722f0 2024-09-12T00:02:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a777722f0 0x555a77774780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdaddffb640 10 client.17220 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.490+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a777722f0 0x555a77774780 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05596f464ffd44e server_cookie=9d3823282c3a5005 in_seq=0 out_seq=0 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdaddffb640 1 -- 172.21.15.62:0/1652564584 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a776c5b90 con 0x555a777722f0 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdaddffb640 10 client.17220 handle_command_reply: tid=0 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 client.17220 shutdown 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 -- 172.21.15.62:0/1652564584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 msgr2=0x555a7775f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a7775f600 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fdae80117c0 tx=0x7fdae8011c80 comp rx=0 tx=0).stop 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 -- 172.21.15.62:0/1652564584 shutdown_connections 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a777722f0 0x555a77774780 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77762650 0x555a77760080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a7775fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdaddffb640 0 client.17220 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 --2- 172.21.15.62:0/1652564584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77752f60 0x555a7775f600 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 -- 172.21.15.62:0/1652564584 >> 172.21.15.62:0/1652564584 conn(0x555a7774f5f0 msgr2=0x555a77755320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:18.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 -- 172.21.15.62:0/1652564584 shutdown_connections 2024-09-12T00:02:18.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 1 -- 172.21.15.62:0/1652564584 wait complete. 2024-09-12T00:02:18.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.491+0000 7fdafec88740 20 client.17220 trim_cache size 0 max 16384 2024-09-12T00:02:18.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.508+0000 7fdafec88740 1 Processor -- start 2024-09-12T00:02:18.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.508+0000 7fdafec88740 1 -- start start 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdafec88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77752f60 0x555a7777a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdafec88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77753880 0x555a7777a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdafec88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a7775c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdafec88740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81adea0 con 0x555a77762650 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdafec88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81ab740 con 0x555a77753880 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdafec88740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81a95b0 con 0x555a77752f60 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a7775c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a7775c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43114/0 (socket says 172.21.15.62:43114) 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/1448703403 learned_addr learned my addr 172.21.15.62:0/1448703403 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf659c640 1 --2- 172.21.15.62:0/1448703403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77752f60 0x555a7777a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/1448703403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77752f60 msgr2=0x555a7777a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/1448703403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77753880 0x555a7777a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/1448703403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77752f60 0x555a7777a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/1448703403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77753880 msgr2=0x555a7777a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/1448703403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77753880 0x555a7777a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/1448703403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a777542e0 con 0x555a77762650 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/1448703403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77753880 0x555a7777a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.509+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/1448703403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a7775c860 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0010440 tx=0x7fdaf0028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b96708bd5375a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdadffff640 1 -- 172.21.15.62:0/1448703403 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf001cbd0 con 0x555a77762650 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdadffff640 1 -- 172.21.15.62:0/1448703403 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf0036070 con 0x555a77762650 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdadffff640 1 -- 172.21.15.62:0/1448703403 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00050c0 con 0x555a77762650 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdafec88740 1 -- 172.21.15.62:0/1448703403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 msgr2=0x555a7775c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdafec88740 1 --2- 172.21.15.62:0/1448703403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a7775c860 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0010440 tx=0x7fdaf0028000 comp rx=0 tx=0).stop 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdafec88740 1 -- 172.21.15.62:0/1448703403 shutdown_connections 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdafec88740 1 --2- 172.21.15.62:0/1448703403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a7775c860 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdafec88740 1 --2- 172.21.15.62:0/1448703403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77753880 0x555a7777a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdafec88740 1 --2- 172.21.15.62:0/1448703403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77752f60 0x555a7777a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdafec88740 1 -- 172.21.15.62:0/1448703403 >> 172.21.15.62:0/1448703403 conn(0x555a7774f5f0 msgr2=0x555a7775ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdafec88740 1 -- 172.21.15.62:0/1448703403 shutdown_connections 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.510+0000 7fdafec88740 1 -- 172.21.15.62:0/1448703403 wait complete. 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdafec88740 1 -- start start 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdafec88740 1 Processor -- start 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdafec88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77752f60 0x555a77766b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdafec88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a77767050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdafec88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a777dc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdafec88740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81a9c00 con 0x555a77762650 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdafec88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81ab070 con 0x555a77752f60 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdafec88740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81aa2d0 con 0x555a77753880 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdafec88740 10 client.0 _ll_register_callbacks cb 0x7fdae8008c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:18.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a777dc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a777dc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43116/0 (socket says 172.21.15.62:43116) 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/22294222 learned_addr learned my addr 172.21.15.62:0/22294222 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a77767050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.511+0000 7fdaf659c640 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77752f60 0x555a77766b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/22294222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 msgr2=0x555a77767050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a77767050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/22294222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77752f60 msgr2=0x555a77766b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77752f60 0x555a77766b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaf6d9d640 1 -- 172.21.15.62:0/22294222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a777dc560 con 0x555a77762650 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaf5d9b640 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a77767050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaf659c640 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77752f60 0x555a77766b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaf6d9d640 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a777dc020 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0010410 tx=0x7fdaf0009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f06371a2628ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:18.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaddffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaddffb640 1 -- 172.21.15.62:0/22294222 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0003b80 con 0x555a77762650 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaddffb640 1 -- 172.21.15.62:0/22294222 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fdaf0003d20 con 0x555a77762650 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdafec88740 1 -- 172.21.15.62:0/22294222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf0036070 con 0x555a77762650 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaddffb640 1 -- 172.21.15.62:0/22294222 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdaf0005080 con 0x555a77762650 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdafec88740 10 client.17226 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdafec88740 1 -- 172.21.15.62:0/22294222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555a777dd790 con 0x555a77762650 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.512+0000 7fdaddffb640 1 -- 172.21.15.62:0/22294222 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf0036070 con 0x555a77762650 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.513+0000 7fdaddffb640 1 -- 172.21.15.62:0/22294222 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fdaf0030040 con 0x555a77762650 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.513+0000 7fdafec88740 10 client.17226 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.513+0000 7fdafec88740 10 client.17226 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.513+0000 7fdafec88740 20 client.17226 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.513+0000 7fdafec88740 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a777ddca0 0x555a777e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.513+0000 7fdafec88740 4 client.17226 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "494e38e3", "release": true, "await": true}] 2024-09-12T00:02:18.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.513+0000 7fdafec88740 1 -- 172.21.15.62:0/22294222 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "494e38e3", "release": true, "await": true}) -- 0x555a77760410 con 0x555a777ddca0 2024-09-12T00:02:18.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.513+0000 7fdaf659c640 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a777ddca0 0x555a777e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:18.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.513+0000 7fdaf659c640 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a777ddca0 0x555a777e0090 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b5e6e639bcb2dc3 server_cookie=36d533e9f3327b2f in_seq=0 out_seq=0 2024-09-12T00:02:18.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.513+0000 7fdaddffb640 10 client.17226 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:18.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.560+0000 7fdaddffb640 1 -- 172.21.15.62:0/22294222 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555a77760410 con 0x555a777ddca0 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.560+0000 7fdaddffb640 10 client.17226 handle_command_reply: tid=0 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.560+0000 7fdafec88740 1 client.17226 shutdown 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.560+0000 7fdafec88740 1 -- 172.21.15.62:0/22294222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 msgr2=0x555a777dc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.560+0000 7fdafec88740 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a777dc020 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0010410 tx=0x7fdaf0009ea0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.561+0000 7fdafec88740 1 -- 172.21.15.62:0/22294222 shutdown_connections 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.561+0000 7fdafec88740 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a777ddca0 0x555a777e0090 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.561+0000 7fdafec88740 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a77762650 0x555a777dc020 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.561+0000 7fdafec88740 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a77753880 0x555a77767050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.561+0000 7fdafec88740 1 --2- 172.21.15.62:0/22294222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a77752f60 0x555a77766b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.561+0000 7fdafec88740 1 -- 172.21.15.62:0/22294222 >> 172.21.15.62:0/22294222 conn(0x555a7774f5f0 msgr2=0x555a77754ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:18.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.561+0000 7fdaddffb640 0 client.17226 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:18.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.568+0000 7fdafec88740 1 -- 172.21.15.62:0/22294222 shutdown_connections 2024-09-12T00:02:18.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.569+0000 7fdafec88740 1 -- 172.21.15.62:0/22294222 wait complete. 2024-09-12T00:02:18.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.569+0000 7fdafec88740 20 client.17226 trim_cache size 0 max 16384 2024-09-12T00:02:18.569 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:02:18.569 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 33, 2024-09-12T00:02:18.569 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:02:18.569 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 40, 2024-09-12T00:02:18.569 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:02:18.569 INFO:teuthology.orchestra.run.smithi062.stdout: "494e38e3": { 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 40, 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 142.6, 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:02:18.570 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:02:18.571 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:18.571 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:18.571 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:18.571 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:18.571 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:18.571 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:02:18.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.570+0000 7fdafc8af640 1 -- 172.21.15.62:0/504983687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdae00a2d70 msgr2=0x7fdae00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:18.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.570+0000 7fdafc8af640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdae00a2d70 0x7fdae00a5230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0021000 tx=0x7fdaf001e000 comp rx=0 tx=0).stop 2024-09-12T00:02:18.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.570+0000 7fdafc8af640 1 -- 172.21.15.62:0/504983687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf8102790 msgr2=0x7fdaf81988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:18.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.570+0000 7fdafc8af640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf8102790 0x7fdaf81988d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdaec006010 tx=0x7fdaec0042f0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.570+0000 7fdafc8af640 1 -- 172.21.15.62:0/504983687 shutdown_connections 2024-09-12T00:02:18.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.570+0000 7fdafc8af640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdae00a2d70 0x7fdae00a5230 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.571+0000 7fdafc8af640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf8106f10 0x7fdaf819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.571+0000 7fdafc8af640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaf8102790 0x7fdaf81988d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.571+0000 7fdafc8af640 1 --2- 172.21.15.62:0/504983687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf8101de0 0x7fdaf8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:18.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.571+0000 7fdafc8af640 1 -- 172.21.15.62:0/504983687 >> 172.21.15.62:0/504983687 conn(0x7fdaf80fd8c0 msgr2=0x7fdaf80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:18.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.575+0000 7fdafc8af640 1 -- 172.21.15.62:0/504983687 shutdown_connections 2024-09-12T00:02:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:18.577+0000 7fdafc8af640 1 -- 172.21.15.62:0/504983687 wait complete. 2024-09-12T00:02:18.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:18 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:02:18.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:18 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/261330360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:18.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:18 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:02:18.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:18 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:02:18.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:18 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:02:18.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:18 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:02:18.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:18 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:02:18.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:18 smithi120 ceph-mon[35182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:02:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:18 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:02:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:18 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/261330360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:18 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:02:18.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:18 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:02:18.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:18 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:02:18.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:18 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:02:18.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:18 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:02:18.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:18 smithi083 ceph-mon[34805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:02:19.470 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '494e38e3', total seconds elapsed: 26.1 2024-09-12T00:02:19.470 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T00:02:19.470 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:02:19.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:19 smithi120 ceph-mon[35182]: pgmap v317: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 147 KiB/s wr, 21 op/s 2024-09-12T00:02:19.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:19 smithi083 ceph-mon[34805]: pgmap v317: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 147 KiB/s wr, 21 op/s 2024-09-12T00:02:19.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:19 smithi062 ceph-mon[30958]: pgmap v317: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 147 KiB/s wr, 21 op/s 2024-09-12T00:02:19.864 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:20.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.434+0000 7fc01b02f640 1 -- 172.21.15.62:0/550635108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0141036e0 msgr2=0x7fc014107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:20.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.434+0000 7fc01b02f640 1 --2- 172.21.15.62:0/550635108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0141036e0 0x7fc014107b90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc000009930 tx=0x7fc00002a420 comp rx=0 tx=0).stop 2024-09-12T00:02:20.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.434+0000 7fc01b02f640 1 -- 172.21.15.62:0/550635108 shutdown_connections 2024-09-12T00:02:20.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.434+0000 7fc01b02f640 1 --2- 172.21.15.62:0/550635108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141082a0 0x7fc01410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:20.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.434+0000 7fc01b02f640 1 --2- 172.21.15.62:0/550635108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0141036e0 0x7fc014107b90 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:20.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.434+0000 7fc01b02f640 1 --2- 172.21.15.62:0/550635108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc014102d30 0x7fc014103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:20.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.434+0000 7fc01b02f640 1 -- 172.21.15.62:0/550635108 >> 172.21.15.62:0/550635108 conn(0x7fc0140fc7d0 msgr2=0x7fc0140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:20.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.434+0000 7fc01b02f640 1 -- 172.21.15.62:0/550635108 shutdown_connections 2024-09-12T00:02:20.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.434+0000 7fc01b02f640 1 -- 172.21.15.62:0/550635108 wait complete. 2024-09-12T00:02:20.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc01b02f640 1 Processor -- start 2024-09-12T00:02:20.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc01b02f640 1 -- start start 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc01b02f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc014102d30 0x7fc0141985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc01b02f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0141036e0 0x7fc014198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc01b02f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141082a0 0x7fc01419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc01b02f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc014112040 con 0x7fc014102d30 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc01b02f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc014111ec0 con 0x7fc0141036e0 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc01b02f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141121c0 con 0x7fc0141082a0 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc013fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0141036e0 0x7fc014198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc013fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0141036e0 0x7fc014198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:41096/0 (socket says 172.21.15.62:41096) 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc018dcb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc014102d30 0x7fc0141985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc013fff640 1 -- 172.21.15.62:0/889509863 learned_addr learned my addr 172.21.15.62:0/889509863 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.435+0000 7fc0195cc640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141082a0 0x7fc01419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.436+0000 7fc013fff640 1 -- 172.21.15.62:0/889509863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141082a0 msgr2=0x7fc01419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.436+0000 7fc013fff640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141082a0 0x7fc01419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.436+0000 7fc013fff640 1 -- 172.21.15.62:0/889509863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc014102d30 msgr2=0x7fc0141985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.436+0000 7fc013fff640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc014102d30 0x7fc0141985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.436+0000 7fc013fff640 1 -- 172.21.15.62:0/889509863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01419d550 con 0x7fc0141036e0 2024-09-12T00:02:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.436+0000 7fc013fff640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0141036e0 0x7fc014198ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc000005940 tx=0x7fc000031b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.436+0000 7fc011ffb640 1 -- 172.21.15.62:0/889509863 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00003f030 con 0x7fc0141036e0 2024-09-12T00:02:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.436+0000 7fc01b02f640 1 -- 172.21.15.62:0/889509863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01419d7e0 con 0x7fc0141036e0 2024-09-12T00:02:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.436+0000 7fc011ffb640 1 -- 172.21.15.62:0/889509863 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0000043d0 con 0x7fc0141036e0 2024-09-12T00:02:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.436+0000 7fc011ffb640 1 -- 172.21.15.62:0/889509863 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000035070 con 0x7fc0141036e0 2024-09-12T00:02:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.437+0000 7fc01b02f640 1 -- 172.21.15.62:0/889509863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0141008c0 con 0x7fc0141036e0 2024-09-12T00:02:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.437+0000 7fc01b02f640 1 -- 172.21.15.62:0/889509863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01406a850 con 0x7fc0141036e0 2024-09-12T00:02:20.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.440+0000 7fc011ffb640 1 -- 172.21.15.62:0/889509863 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc000042030 con 0x7fc0141036e0 2024-09-12T00:02:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.440+0000 7fc011ffb640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc0040a2d70 0x7fc0040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.441+0000 7fc018dcb640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc0040a2d70 0x7fc0040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.441+0000 7fc011ffb640 1 -- 172.21.15.62:0/889509863 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fc0000ef890 con 0x7fc0141036e0 2024-09-12T00:02:20.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.441+0000 7fc018dcb640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc0040a2d70 0x7fc0040a5230 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc008009bd0 tx=0x7fc008005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.443+0000 7fc011ffb640 1 -- 172.21.15.62:0/889509863 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0000bbe60 con 0x7fc0141036e0 2024-09-12T00:02:20.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.814+0000 7fc01b02f640 1 -- 172.21.15.62:0/889509863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc014103110 con 0x7fc0141036e0 2024-09-12T00:02:20.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.814+0000 7fc011ffb640 1 -- 172.21.15.62:0/889509863 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11911 (secure 0 0 0) 0x7fc014103110 con 0x7fc0141036e0 2024-09-12T00:02:20.814 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:02:20.814 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":33,"btime":"2024-09-12T00:01:52:327104+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:01:52.081596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:02:20.815 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 33 2024-09-12T00:02:20.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.817+0000 7fc01b02f640 1 -- 172.21.15.62:0/889509863 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc0040a2d70 msgr2=0x7fc0040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:20.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.817+0000 7fc01b02f640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc0040a2d70 0x7fc0040a5230 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc008009bd0 tx=0x7fc008005f90 comp rx=0 tx=0).stop 2024-09-12T00:02:20.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.817+0000 7fc01b02f640 1 -- 172.21.15.62:0/889509863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0141036e0 msgr2=0x7fc014198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:20.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.817+0000 7fc01b02f640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0141036e0 0x7fc014198ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc000005940 tx=0x7fc000031b50 comp rx=0 tx=0).stop 2024-09-12T00:02:20.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.818+0000 7fc01b02f640 1 -- 172.21.15.62:0/889509863 shutdown_connections 2024-09-12T00:02:20.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.818+0000 7fc01b02f640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc0040a2d70 0x7fc0040a5230 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:20.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.818+0000 7fc01b02f640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141082a0 0x7fc01419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:20.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.818+0000 7fc01b02f640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0141036e0 0x7fc014198ae0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:20.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.818+0000 7fc01b02f640 1 --2- 172.21.15.62:0/889509863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc014102d30 0x7fc0141985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:20.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.818+0000 7fc01b02f640 1 -- 172.21.15.62:0/889509863 >> 172.21.15.62:0/889509863 conn(0x7fc0140fc7d0 msgr2=0x7fc014105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:20.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.818+0000 7fc01b02f640 1 -- 172.21.15.62:0/889509863 shutdown_connections 2024-09-12T00:02:20.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:20.818+0000 7fc01b02f640 1 -- 172.21.15.62:0/889509863 wait complete. 2024-09-12T00:02:21.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:21 smithi062 ceph-mon[30958]: pgmap v318: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 173 KiB/s wr, 23 op/s 2024-09-12T00:02:21.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:21 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/889509863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:21 smithi120 ceph-mon[35182]: pgmap v318: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 173 KiB/s wr, 23 op/s 2024-09-12T00:02:21.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:21 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/889509863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:21.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:21 smithi083 ceph-mon[34805]: pgmap v318: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 173 KiB/s wr, 23 op/s 2024-09-12T00:02:21.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:21 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/889509863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:21.903 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T00:02:21.903 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T00:02:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 580.0 s, 1968.8 tps, lat 246.628 ms stddev 2050.203 2024-09-12T00:02:22.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.944+0000 7fa682c60640 1 -- 172.21.15.62:0/1587468179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c109b10 msgr2=0x7fa67c116960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:22.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.944+0000 7fa682c60640 1 --2- 172.21.15.62:0/1587468179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c109b10 0x7fa67c116960 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa670009900 tx=0x7fa67002a410 comp rx=0 tx=0).stop 2024-09-12T00:02:22.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.945+0000 7fa682c60640 1 -- 172.21.15.62:0/1587468179 shutdown_connections 2024-09-12T00:02:22.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.945+0000 7fa682c60640 1 --2- 172.21.15.62:0/1587468179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa67c116ea0 0x7fa67c119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.945+0000 7fa682c60640 1 --2- 172.21.15.62:0/1587468179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c109b10 0x7fa67c116960 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.945+0000 7fa682c60640 1 --2- 172.21.15.62:0/1587468179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa67c1091f0 0x7fa67c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.945+0000 7fa682c60640 1 -- 172.21.15.62:0/1587468179 >> 172.21.15.62:0/1587468179 conn(0x7fa67c06e140 msgr2=0x7fa67c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:22.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.946+0000 7fa682c60640 1 -- 172.21.15.62:0/1587468179 shutdown_connections 2024-09-12T00:02:22.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.946+0000 7fa682c60640 1 -- 172.21.15.62:0/1587468179 wait complete. 2024-09-12T00:02:22.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.947+0000 7fa682c60640 1 Processor -- start 2024-09-12T00:02:22.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.947+0000 7fa682c60640 1 -- start start 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa682c60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c1091f0 0x7fa67c10c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa682c60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa67c109b10 0x7fa67c10cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa682c60640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa67c116ea0 0x7fa67c10d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa682c60640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c11c9b0 con 0x7fa67c1091f0 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa682c60640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c11c830 con 0x7fa67c109b10 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa6809fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c1091f0 0x7fa67c10c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa682c60640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c11cb30 con 0x7fa67c116ea0 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa6811fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa67c116ea0 0x7fa67c10d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa6809fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c1091f0 0x7fa67c10c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44128/0 (socket says 172.21.15.62:44128) 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa67c109b10 0x7fa67c10cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa6809fc640 1 -- 172.21.15.62:0/2448179616 learned_addr learned my addr 172.21.15.62:0/2448179616 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:22.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa6811fd640 1 -- 172.21.15.62:0/2448179616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa67c109b10 msgr2=0x7fa67c10cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:22.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa6811fd640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa67c109b10 0x7fa67c10cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa6811fd640 1 -- 172.21.15.62:0/2448179616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c1091f0 msgr2=0x7fa67c10c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:22.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa6811fd640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c1091f0 0x7fa67c10c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.948+0000 7fa6811fd640 1 -- 172.21.15.62:0/2448179616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c1b3530 con 0x7fa67c116ea0 2024-09-12T00:02:22.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.949+0000 7fa6811fd640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa67c116ea0 0x7fa67c10d060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa674009fd0 tx=0x7fa67400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:22.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.949+0000 7fa679ffb640 1 -- 172.21.15.62:0/2448179616 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67400ce20 con 0x7fa67c116ea0 2024-09-12T00:02:22.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.949+0000 7fa679ffb640 1 -- 172.21.15.62:0/2448179616 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa674005020 con 0x7fa67c116ea0 2024-09-12T00:02:22.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.949+0000 7fa682c60640 1 -- 172.21.15.62:0/2448179616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c1b3820 con 0x7fa67c116ea0 2024-09-12T00:02:22.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.949+0000 7fa679ffb640 1 -- 172.21.15.62:0/2448179616 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6740106d0 con 0x7fa67c116ea0 2024-09-12T00:02:22.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.949+0000 7fa682c60640 1 -- 172.21.15.62:0/2448179616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c1b3d30 con 0x7fa67c116ea0 2024-09-12T00:02:22.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.952+0000 7fa679ffb640 1 -- 172.21.15.62:0/2448179616 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa674004b20 con 0x7fa67c116ea0 2024-09-12T00:02:22.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.953+0000 7fa679ffb640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6640a2d70 0x7fa6640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.953+0000 7fa685039740 1 Processor -- start 2024-09-12T00:02:22.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa685039740 1 -- start start 2024-09-12T00:02:22.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa679ffb640 1 -- 172.21.15.62:0/2448179616 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fa6740cfa10 con 0x7fa67c116ea0 2024-09-12T00:02:22.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa6809fc640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6640a2d70 0x7fa6640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa685039740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6579850 0x5626d6579c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa685039740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d65844f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa685039740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6584a30 0x5626d6586e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa685039740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5626d64de160 con 0x5626d6579850 2024-09-12T00:02:22.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa685039740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5626d6485720 con 0x5626d657a170 2024-09-12T00:02:22.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa685039740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5626d64da120 con 0x5626d6584a30 2024-09-12T00:02:22.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d65844f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d65844f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:43170/0 (socket says 172.21.15.62:43170) 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa67bfff640 1 -- 172.21.15.62:0/3836773800 learned_addr learned my addr 172.21.15.62:0/3836773800 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa6811fd640 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6584a30 0x5626d6586e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa6809fc640 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6579850 0x5626d6579c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa67bfff640 1 -- 172.21.15.62:0/3836773800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6584a30 msgr2=0x5626d6586e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa67bfff640 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6584a30 0x5626d6586e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa67bfff640 1 -- 172.21.15.62:0/3836773800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6579850 msgr2=0x5626d6579c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa67bfff640 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6579850 0x5626d6579c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.954+0000 7fa67bfff640 1 -- 172.21.15.62:0/3836773800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626d624fba0 con 0x5626d657a170 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.955+0000 7fa6811fd640 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6584a30 0x5626d6586e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.955+0000 7fa6809fc640 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6579850 0x5626d6579c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.955+0000 7fa67bfff640 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d65844f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa670009900 tx=0x7fa670005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e54e54155daeca6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.955+0000 7fa6809fc640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6640a2d70 0x7fa6640a5230 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa66c005fd0 tx=0x7fa66c00b890 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:22.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.955+0000 7fa662ffd640 1 -- 172.21.15.62:0/3836773800 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670040070 con 0x5626d657a170 2024-09-12T00:02:22.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.955+0000 7fa662ffd640 1 -- 172.21.15.62:0/3836773800 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6700318f0 con 0x5626d657a170 2024-09-12T00:02:22.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.955+0000 7fa662ffd640 1 -- 172.21.15.62:0/3836773800 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670035040 con 0x5626d657a170 2024-09-12T00:02:22.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.956+0000 7fa685039740 1 -- 172.21.15.62:0/3836773800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 msgr2=0x5626d65844f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:22.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.956+0000 7fa685039740 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d65844f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa670009900 tx=0x7fa670005e00 comp rx=0 tx=0).stop 2024-09-12T00:02:22.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.957+0000 7fa685039740 1 -- 172.21.15.62:0/3836773800 shutdown_connections 2024-09-12T00:02:22.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.957+0000 7fa685039740 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6584a30 0x5626d6586e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.957+0000 7fa685039740 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d65844f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.957+0000 7fa685039740 1 --2- 172.21.15.62:0/3836773800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6579850 0x5626d6579c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.957+0000 7fa685039740 1 -- 172.21.15.62:0/3836773800 >> 172.21.15.62:0/3836773800 conn(0x5626d6575de0 msgr2=0x5626d65761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:22.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.957+0000 7fa685039740 1 -- 172.21.15.62:0/3836773800 shutdown_connections 2024-09-12T00:02:22.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.957+0000 7fa685039740 1 -- 172.21.15.62:0/3836773800 wait complete. 2024-09-12T00:02:22.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.958+0000 7fa685039740 1 -- start start 2024-09-12T00:02:22.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.958+0000 7fa685039740 1 Processor -- start 2024-09-12T00:02:22.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa685039740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 0x5626d65818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa685039740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d6581de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa685039740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6584a30 0x5626d6582320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d6581de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d6581de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:43176/0 (socket says 172.21.15.62:43176) 2024-09-12T00:02:22.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa6809fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 0x5626d65818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa6809fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 0x5626d65818a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:41490/0 (socket says 172.21.15.62:41490) 2024-09-12T00:02:22.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa685039740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5626d6587600 con 0x5626d6584a30 2024-09-12T00:02:22.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa685039740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5626d6466900 con 0x5626d657a170 2024-09-12T00:02:22.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa685039740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5626d6587780 con 0x5626d6579850 2024-09-12T00:02:22.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.959+0000 7fa685039740 10 client.0 _ll_register_callbacks cb 0x5626d6464780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:22.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.960+0000 7fa67bfff640 1 -- 172.21.15.62:0/1728359371 learned_addr learned my addr 172.21.15.62:0/1728359371 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:22.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.960+0000 7fa6811fd640 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6584a30 0x5626d6582320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.960+0000 7fa67bfff640 1 -- 172.21.15.62:0/1728359371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 msgr2=0x5626d65818a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:22.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.960+0000 7fa67bfff640 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 0x5626d65818a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.960+0000 7fa67bfff640 1 -- 172.21.15.62:0/1728359371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6584a30 msgr2=0x5626d6582320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.960+0000 7fa67bfff640 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6584a30 0x5626d6582320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.960+0000 7fa67bfff640 1 -- 172.21.15.62:0/1728359371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626d64a6c50 con 0x5626d657a170 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.960+0000 7fa6809fc640 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 0x5626d65818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.960+0000 7fa67bfff640 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d6581de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa6700053a0 tx=0x7fa670000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69a4afb2cf84d5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.960+0000 7fa65bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.961+0000 7fa65bfff640 1 -- 172.21.15.62:0/1728359371 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670040070 con 0x5626d657a170 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.961+0000 7fa65bfff640 1 -- 172.21.15.62:0/1728359371 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fa6700316f0 con 0x5626d657a170 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.961+0000 7fa685039740 1 -- 172.21.15.62:0/1728359371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6700318f0 con 0x5626d657a170 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.961+0000 7fa65bfff640 1 -- 172.21.15.62:0/1728359371 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa670055050 con 0x5626d657a170 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.961+0000 7fa685039740 10 client.26795 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.961+0000 7fa685039740 1 -- 172.21.15.62:0/1728359371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5626d6593230 con 0x5626d657a170 2024-09-12T00:02:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.962+0000 7fa65bfff640 1 -- 172.21.15.62:0/1728359371 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6700048d0 con 0x5626d657a170 2024-09-12T00:02:22.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.962+0000 7fa65bfff640 1 -- 172.21.15.62:0/1728359371 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa670003920 con 0x5626d657a170 2024-09-12T00:02:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.962+0000 7fa685039740 10 client.26795 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.962+0000 7fa685039740 10 client.26795 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:02:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.962+0000 7fa685039740 20 client.26795 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.962+0000 7fa685039740 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5626d6594580 0x5626d6596970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.962+0000 7fa685039740 4 client.26795 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.962+0000 7fa685039740 1 -- 172.21.15.62:0/1728359371 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa6700318f0 con 0x5626d6594580 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.962+0000 7fa6809fc640 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5626d6594580 0x5626d6596970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.963+0000 7fa65bfff640 10 client.26795 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.963+0000 7fa6809fc640 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5626d6594580 0x5626d6596970 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef6bd90765794ef server_cookie=20067bfe13537592 in_seq=0 out_seq=0 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa65bfff640 1 -- 172.21.15.62:0/1728359371 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa6700318f0 con 0x5626d6594580 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa65bfff640 10 client.26795 handle_command_reply: tid=0 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa685039740 1 client.26795 shutdown 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa685039740 1 -- 172.21.15.62:0/1728359371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 msgr2=0x5626d6581de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa685039740 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d6581de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa6700053a0 tx=0x7fa670000bc0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa685039740 1 -- 172.21.15.62:0/1728359371 shutdown_connections 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa685039740 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5626d6594580 0x5626d6596970 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa685039740 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6584a30 0x5626d6582320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa65bfff640 0 client.26795 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa685039740 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d657a170 0x5626d6581de0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa685039740 1 --2- 172.21.15.62:0/1728359371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 0x5626d65818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa685039740 1 -- 172.21.15.62:0/1728359371 >> 172.21.15.62:0/1728359371 conn(0x5626d6575de0 msgr2=0x5626d657bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.964+0000 7fa685039740 1 -- 172.21.15.62:0/1728359371 shutdown_connections 2024-09-12T00:02:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.965+0000 7fa685039740 1 -- 172.21.15.62:0/1728359371 wait complete. 2024-09-12T00:02:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.965+0000 7fa685039740 20 client.26795 trim_cache size 0 max 16384 2024-09-12T00:02:22.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.991+0000 7fa685039740 1 Processor -- start 2024-09-12T00:02:22.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.992+0000 7fa685039740 1 -- start start 2024-09-12T00:02:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.992+0000 7fa685039740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 0x5626d657c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.992+0000 7fa685039740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d657a170 0x5626d659bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.992+0000 7fa685039740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d6584a30 0x5626d659c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.993+0000 7fa685039740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1bf0b0 con 0x5626d657a170 2024-09-12T00:02:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.993+0000 7fa685039740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1bc950 con 0x5626d6584a30 2024-09-12T00:02:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.993+0000 7fa685039740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1ba7c0 con 0x5626d6579850 2024-09-12T00:02:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.993+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d657a170 0x5626d659bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.993+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d657a170 0x5626d659bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44150/0 (socket says 172.21.15.62:44150) 2024-09-12T00:02:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.993+0000 7fa67bfff640 1 -- 172.21.15.62:0/127088377 learned_addr learned my addr 172.21.15.62:0/127088377 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:22.994 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:22 smithi062 ceph-mon[30958]: pgmap v319: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 288 KiB/s wr, 37 op/s 2024-09-12T00:02:22.994 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:22 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:02:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.994+0000 7fa67bfff640 1 -- 172.21.15.62:0/127088377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 msgr2=0x5626d657c2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:02:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.994+0000 7fa67bfff640 1 --2- 172.21.15.62:0/127088377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 0x5626d657c2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.994+0000 7fa67bfff640 1 -- 172.21.15.62:0/127088377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d6584a30 msgr2=0x5626d659c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:22.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.994+0000 7fa67bfff640 1 --2- 172.21.15.62:0/127088377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d6584a30 0x5626d659c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:22.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.994+0000 7fa67bfff640 1 -- 172.21.15.62:0/127088377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626d6575880 con 0x5626d657a170 2024-09-12T00:02:22.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.996+0000 7fa67bfff640 1 --2- 172.21.15.62:0/127088377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d657a170 0x5626d659bfa0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa670003d20 tx=0x7fa670009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a9161f3d339210 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.997+0000 7fa6627fc640 1 -- 172.21.15.62:0/127088377 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67005e070 con 0x5626d657a170 2024-09-12T00:02:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.997+0000 7fa6627fc640 1 -- 172.21.15.62:0/127088377 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa670004020 con 0x5626d657a170 2024-09-12T00:02:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.997+0000 7fa6627fc640 1 -- 172.21.15.62:0/127088377 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670040040 con 0x5626d657a170 2024-09-12T00:02:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.998+0000 7fa685039740 1 -- 172.21.15.62:0/127088377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d657a170 msgr2=0x5626d659bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.998+0000 7fa685039740 1 --2- 172.21.15.62:0/127088377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d657a170 0x5626d659bfa0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa670003d20 tx=0x7fa670009660 comp rx=0 tx=0).stop 2024-09-12T00:02:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.998+0000 7fa685039740 1 -- 172.21.15.62:0/127088377 shutdown_connections 2024-09-12T00:02:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.998+0000 7fa685039740 1 --2- 172.21.15.62:0/127088377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d6584a30 0x5626d659c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.998+0000 7fa685039740 1 --2- 172.21.15.62:0/127088377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d657a170 0x5626d659bfa0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.998+0000 7fa685039740 1 --2- 172.21.15.62:0/127088377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d6579850 0x5626d657c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:22.998+0000 7fa685039740 1 -- 172.21.15.62:0/127088377 >> 172.21.15.62:0/127088377 conn(0x5626d6575de0 msgr2=0x5626d6574ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.008+0000 7fa685039740 1 -- 172.21.15.62:0/127088377 shutdown_connections 2024-09-12T00:02:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.012+0000 7fa685039740 1 -- 172.21.15.62:0/127088377 wait complete. 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.012+0000 7fa685039740 1 -- start start 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.012+0000 7fa685039740 1 Processor -- start 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.013+0000 7fa685039740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d6579850 0x5626d658c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.013+0000 7fa685039740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d657a170 0x5626d658cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.013+0000 7fa685039740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6584a30 0x5626d6588e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.013+0000 7fa685039740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1bae10 con 0x5626d6584a30 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.013+0000 7fa685039740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1bc280 con 0x5626d6579850 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.013+0000 7fa685039740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1bb4e0 con 0x5626d657a170 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.013+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d657a170 0x5626d658cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.013+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d657a170 0x5626d658cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:41508/0 (socket says 172.21.15.62:41508) 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.013+0000 7fa67bfff640 1 -- 172.21.15.62:0/2732248256 learned_addr learned my addr 172.21.15.62:0/2732248256 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.013+0000 7fa685039740 10 client.0 _ll_register_callbacks cb 0x7fa674005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.014+0000 7fa6811fd640 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6584a30 0x5626d6588e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.014+0000 7fa67bfff640 1 -- 172.21.15.62:0/2732248256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d6579850 msgr2=0x5626d658c9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:02:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.014+0000 7fa67bfff640 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d6579850 0x5626d658c9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.014+0000 7fa67bfff640 1 -- 172.21.15.62:0/2732248256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6584a30 msgr2=0x5626d6588e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.014+0000 7fa67bfff640 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6584a30 0x5626d6588e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.014+0000 7fa67bfff640 1 -- 172.21.15.62:0/2732248256 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626d6589690 con 0x5626d657a170 2024-09-12T00:02:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.015+0000 7fa67bfff640 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d657a170 0x5626d658cf20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa670003e10 tx=0x7fa670004400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96837014c492ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.016+0000 7fa65bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:02:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.016+0000 7fa65bfff640 1 -- 172.21.15.62:0/2732248256 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670040040 con 0x5626d657a170 2024-09-12T00:02:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.016+0000 7fa685039740 1 -- 172.21.15.62:0/2732248256 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa670004020 con 0x5626d657a170 2024-09-12T00:02:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.016+0000 7fa65bfff640 1 -- 172.21.15.62:0/2732248256 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fa6700053a0 con 0x5626d657a170 2024-09-12T00:02:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.016+0000 7fa65bfff640 1 -- 172.21.15.62:0/2732248256 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa67008c050 con 0x5626d657a170 2024-09-12T00:02:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.019+0000 7fa685039740 10 client.26713 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.019+0000 7fa685039740 1 -- 172.21.15.62:0/2732248256 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5626d65f7920 con 0x5626d657a170 2024-09-12T00:02:23.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.021+0000 7fa65bfff640 1 -- 172.21.15.62:0/2732248256 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa670004020 con 0x5626d657a170 2024-09-12T00:02:23.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.021+0000 7fa65bfff640 1 -- 172.21.15.62:0/2732248256 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa670004b80 con 0x5626d657a170 2024-09-12T00:02:23.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.021+0000 7fa685039740 10 client.26713 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:23.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.021+0000 7fa685039740 10 client.26713 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:02:23.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.021+0000 7fa685039740 20 client.26713 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:23.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.021+0000 7fa685039740 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5626d6594580 0x5626d65fa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:23.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.022+0000 7fa685039740 4 client.26713 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T00:02:23.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.022+0000 7fa685039740 1 -- 172.21.15.62:0/2732248256 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x5626d64eb890 con 0x5626d6594580 2024-09-12T00:02:23.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.027+0000 7fa6809fc640 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5626d6594580 0x5626d65fa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.028+0000 7fa6809fc640 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5626d6594580 0x5626d65fa0d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed68675f8d1785c3 server_cookie=6ecd22bf8015c3c1 in_seq=0 out_seq=0 2024-09-12T00:02:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.028+0000 7fa65bfff640 10 client.26713 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:23.046 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:22 smithi120 ceph-mon[35182]: pgmap v319: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 288 KiB/s wr, 37 op/s 2024-09-12T00:02:23.046 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:22 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:02:23.072 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:22 smithi083 ceph-mon[34805]: pgmap v319: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 288 KiB/s wr, 37 op/s 2024-09-12T00:02:23.073 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:22 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:02:23.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa65bfff640 1 -- 172.21.15.62:0/2732248256 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5626d64eb890 con 0x5626d6594580 2024-09-12T00:02:23.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa65bfff640 10 client.26713 handle_command_reply: tid=0 2024-09-12T00:02:23.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 client.26713 shutdown 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 -- 172.21.15.62:0/2732248256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d657a170 msgr2=0x5626d658cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d657a170 0x5626d658cf20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa670003e10 tx=0x7fa670004400 comp rx=0 tx=0).stop 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 -- 172.21.15.62:0/2732248256 shutdown_connections 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5626d6594580 0x5626d65fa0d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626d6584a30 0x5626d6588e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626d657a170 0x5626d658cf20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 --2- 172.21.15.62:0/2732248256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626d6579850 0x5626d658c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 -- 172.21.15.62:0/2732248256 >> 172.21.15.62:0/2732248256 conn(0x5626d6575de0 msgr2=0x5626d6583100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa65bfff640 0 client.26713 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 -- 172.21.15.62:0/2732248256 shutdown_connections 2024-09-12T00:02:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.324+0000 7fa685039740 1 -- 172.21.15.62:0/2732248256 wait complete. 2024-09-12T00:02:23.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.325+0000 7fa685039740 20 client.26713 trim_cache size 0 max 16384 2024-09-12T00:02:23.326 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:02:23.326 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 33, 2024-09-12T00:02:23.326 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:02:23.326 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 42, 2024-09-12T00:02:23.326 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:02:23.326 INFO:teuthology.orchestra.run.smithi062.stdout: "93f82611": { 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 42, 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 142.6, 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:02:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:02:23.328 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:23.328 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:23.328 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:23.328 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:23.328 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:23.328 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:02:23.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.325+0000 7fa682c60640 1 -- 172.21.15.62:0/2448179616 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6640a2d70 msgr2=0x7fa6640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:23.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.325+0000 7fa682c60640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6640a2d70 0x7fa6640a5230 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa66c005fd0 tx=0x7fa66c00b890 comp rx=0 tx=0).stop 2024-09-12T00:02:23.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.325+0000 7fa682c60640 1 -- 172.21.15.62:0/2448179616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa67c116ea0 msgr2=0x7fa67c10d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:23.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.325+0000 7fa682c60640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa67c116ea0 0x7fa67c10d060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa674009fd0 tx=0x7fa67400ef10 comp rx=0 tx=0).stop 2024-09-12T00:02:23.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.326+0000 7fa682c60640 1 -- 172.21.15.62:0/2448179616 shutdown_connections 2024-09-12T00:02:23.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.326+0000 7fa682c60640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa6640a2d70 0x7fa6640a5230 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.326+0000 7fa682c60640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa67c116ea0 0x7fa67c10d060 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.326+0000 7fa682c60640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa67c109b10 0x7fa67c10cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.326+0000 7fa682c60640 1 --2- 172.21.15.62:0/2448179616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c1091f0 0x7fa67c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:23.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.326+0000 7fa682c60640 1 -- 172.21.15.62:0/2448179616 >> 172.21.15.62:0/2448179616 conn(0x7fa67c06e140 msgr2=0x7fa67c117420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:23.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.326+0000 7fa682c60640 1 -- 172.21.15.62:0/2448179616 shutdown_connections 2024-09-12T00:02:23.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:23.326+0000 7fa682c60640 1 -- 172.21.15.62:0/2448179616 wait complete. 2024-09-12T00:02:24.750 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '93f82611', quiesce took 5.3 seconds. Will release after: 17.25 2024-09-12T00:02:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: pgmap v320: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 393 op/s 2024-09-12T00:02:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-12T00:02:25.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: Unable to set osd_memory_target on smithi120 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-12T00:02:25.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.122 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:24 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: pgmap v320: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 393 op/s 2024-09-12T00:02:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-12T00:02:25.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: Unable to set osd_memory_target on smithi120 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-12T00:02:25.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:24 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: pgmap v320: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 393 op/s 2024-09-12T00:02:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: Unable to set osd_memory_target on smithi120 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:25.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:24 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:26.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:25 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-12T00:02:26.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:25 smithi120 ceph-mon[35182]: Unable to set osd_memory_target on smithi083 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-12T00:02:26.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:25 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:26.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:25 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:26.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:25 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-12T00:02:26.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:25 smithi083 ceph-mon[34805]: Unable to set osd_memory_target on smithi083 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-12T00:02:26.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:25 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:26.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:25 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:26.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:25 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-12T00:02:26.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:25 smithi062 ceph-mon[30958]: Unable to set osd_memory_target on smithi083 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-12T00:02:26.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:25 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:26.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:25 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:27.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:26 smithi120 ceph-mon[35182]: pgmap v321: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 393 op/s 2024-09-12T00:02:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:26 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:26 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:26 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:26 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:26 smithi120 ceph-mon[35182]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-09-12T00:02:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:26 smithi120 ceph-mon[35182]: Unable to set osd_memory_target on smithi062 to 184586035: error parsing value: Value '184586035' is below minimum 939524096 2024-09-12T00:02:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:26 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:02:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:26 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:02:27.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:26 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:26 smithi083 ceph-mon[34805]: pgmap v321: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 393 op/s 2024-09-12T00:02:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:26 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:26 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:26 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:26 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:26 smithi083 ceph-mon[34805]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-09-12T00:02:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:26 smithi083 ceph-mon[34805]: Unable to set osd_memory_target on smithi062 to 184586035: error parsing value: Value '184586035' is below minimum 939524096 2024-09-12T00:02:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:26 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:02:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:26 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:02:27.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:26 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:27.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:26 smithi062 ceph-mon[30958]: pgmap v321: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 393 op/s 2024-09-12T00:02:27.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:26 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:26 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:26 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:26 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:02:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:26 smithi062 ceph-mon[30958]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-09-12T00:02:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:26 smithi062 ceph-mon[30958]: Unable to set osd_memory_target on smithi062 to 184586035: error parsing value: Value '184586035' is below minimum 939524096 2024-09-12T00:02:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:26 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:02:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:26 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:02:27.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:26 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:02:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 585.0 s, 318.6 tps, lat 60.830 ms stddev 65.028 2024-09-12T00:02:29.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:28 smithi120 ceph-mon[35182]: pgmap v322: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.7 MiB/s wr, 536 op/s 2024-09-12T00:02:29.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:28 smithi083 ceph-mon[34805]: pgmap v322: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.7 MiB/s wr, 536 op/s 2024-09-12T00:02:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:28 smithi062 ceph-mon[30958]: pgmap v322: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.7 MiB/s wr, 536 op/s 2024-09-12T00:02:31.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:30 smithi120 ceph-mon[35182]: pgmap v323: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 517 op/s 2024-09-12T00:02:31.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:30 smithi083 ceph-mon[34805]: pgmap v323: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 517 op/s 2024-09-12T00:02:31.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:30 smithi062 ceph-mon[30958]: pgmap v323: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 517 op/s 2024-09-12T00:02:32.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 590.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:02:33.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:32 smithi120 ceph-mon[35182]: pgmap v324: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 515 op/s 2024-09-12T00:02:33.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:32 smithi083 ceph-mon[34805]: pgmap v324: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 515 op/s 2024-09-12T00:02:33.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:32 smithi062 ceph-mon[30958]: pgmap v324: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 515 op/s 2024-09-12T00:02:35.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:34 smithi120 ceph-mon[35182]: pgmap v325: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.3 MiB/s wr, 505 op/s 2024-09-12T00:02:35.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:34 smithi083 ceph-mon[34805]: pgmap v325: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.3 MiB/s wr, 505 op/s 2024-09-12T00:02:35.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:34 smithi062 ceph-mon[30958]: pgmap v325: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.3 MiB/s wr, 505 op/s 2024-09-12T00:02:37.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:36 smithi120 ceph-mon[35182]: pgmap v326: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-09-12T00:02:37.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:36 smithi083 ceph-mon[34805]: pgmap v326: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-09-12T00:02:37.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:36 smithi062 ceph-mon[30958]: pgmap v326: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-09-12T00:02:37.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 595.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:02:37.904 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:02:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:38.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.655+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/1611060717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc10c170 msgr2=0x7fa9dc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.655+0000 7fa9e0ffc640 1 --2- 172.21.15.62:0/1611060717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc10c170 0x7fa9dc10e610 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009ca0 tx=0x7fa9d402a950 comp rx=0 tx=0).stop 2024-09-12T00:02:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.655+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/1611060717 shutdown_connections 2024-09-12T00:02:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.655+0000 7fa9e0ffc640 1 --2- 172.21.15.62:0/1611060717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc10c170 0x7fa9dc10e610 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.655+0000 7fa9e0ffc640 1 --2- 172.21.15.62:0/1611060717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9dc103660 0x7fa9dc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.655+0000 7fa9e0ffc640 1 --2- 172.21.15.62:0/1611060717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9dc102d40 0x7fa9dc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.655+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/1611060717 >> 172.21.15.62:0/1611060717 conn(0x7fa9dc0fc7a0 msgr2=0x7fa9dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.655+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/1611060717 shutdown_connections 2024-09-12T00:02:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.655+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/1611060717 wait complete. 2024-09-12T00:02:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9e0ffc640 1 Processor -- start 2024-09-12T00:02:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9e0ffc640 1 -- start start 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9e0ffc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9dc102d40 0x7fa9dc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9e0ffc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9dc103660 0x7fa9dc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9e0ffc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc10c170 0x7fa9dc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9e0ffc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc111f70 con 0x7fa9dc10c170 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9e0ffc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc111df0 con 0x7fa9dc102d40 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9e0ffc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1120f0 con 0x7fa9dc103660 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9da59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9dc102d40 0x7fa9dc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9dad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc10c170 0x7fa9dc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9da59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9dc102d40 0x7fa9dc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35412/0 (socket says 172.21.15.62:35412) 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9da59c640 1 -- 172.21.15.62:0/3936096366 learned_addr learned my addr 172.21.15.62:0/3936096366 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.656+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9dc103660 0x7fa9dc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9dad9d640 1 -- 172.21.15.62:0/3936096366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9dc103660 msgr2=0x7fa9dc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9dad9d640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9dc103660 0x7fa9dc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9dad9d640 1 -- 172.21.15.62:0/3936096366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9dc102d40 msgr2=0x7fa9dc1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9dad9d640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9dc102d40 0x7fa9dc1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9dad9d640 1 -- 172.21.15.62:0/3936096366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc19d3a0 con 0x7fa9dc10c170 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9da59c640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9dc102d40 0x7fa9dc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9dad9d640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc10c170 0x7fa9dc19ccc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005cc0 tx=0x7fa9d402f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9d37fe640 1 -- 172.21.15.62:0/3936096366 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4034040 con 0x7fa9dc10c170 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9d37fe640 1 -- 172.21.15.62:0/3936096366 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d4030920 con 0x7fa9dc10c170 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/3936096366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9dc19d630 con 0x7fa9dc10c170 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9d37fe640 1 -- 172.21.15.62:0/3936096366 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4004610 con 0x7fa9dc10c170 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.657+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/3936096366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9dc1a4f70 con 0x7fa9dc10c170 2024-09-12T00:02:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.660+0000 7fa9d37fe640 1 -- 172.21.15.62:0/3936096366 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa9d4037020 con 0x7fa9dc10c170 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.660+0000 7fa9e33d5740 1 Processor -- start 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.660+0000 7fa9e33d5740 1 -- start start 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.660+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb65584a0 0x564cb6558880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.660+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6563140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.660+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6563680 0x564cb6565a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.660+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564cb64be2e0 con 0x564cb6558dc0 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.660+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564cb6465ce0 con 0x564cb65584a0 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.660+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564cb64ba2a0 con 0x564cb6563680 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.660+0000 7fa9da59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6563140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9da59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6563140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34436/0 (socket says 172.21.15.62:34436) 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9da59c640 1 -- 172.21.15.62:0/3573217853 learned_addr learned my addr 172.21.15.62:0/3573217853 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9d37fe640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9c80a2cf0 0x7fa9c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/3573217853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6563680 0x564cb6565a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/3573217853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb65584a0 0x564cb6558880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9dad9d640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9c80a2cf0 0x7fa9c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9da59c640 1 -- 172.21.15.62:0/3573217853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6563680 msgr2=0x564cb6565a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9da59c640 1 --2- 172.21.15.62:0/3573217853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6563680 0x564cb6565a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9da59c640 1 -- 172.21.15.62:0/3573217853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb65584a0 msgr2=0x564cb6558880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9da59c640 1 --2- 172.21.15.62:0/3573217853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb65584a0 0x564cb6558880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9da59c640 1 -- 172.21.15.62:0/3573217853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb644a440 con 0x564cb6558dc0 2024-09-12T00:02:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9d37fe640 1 -- 172.21.15.62:0/3936096366 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fa9d40f00f0 con 0x7fa9dc10c170 2024-09-12T00:02:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9da59c640 1 --2- 172.21.15.62:0/3573217853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6563140 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc011650 tx=0x7fa9cc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478e8ff922e358f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9d0ff9640 1 -- 172.21.15.62:0/3573217853 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc01e070 con 0x564cb6558dc0 2024-09-12T00:02:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9dad9d640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9c80a2cf0 0x7fa9c80a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40bc6b0 tx=0x7fa9d408e6e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9d0ff9640 1 -- 172.21.15.62:0/3573217853 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9cc00fba0 con 0x564cb6558dc0 2024-09-12T00:02:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.661+0000 7fa9d0ff9640 1 -- 172.21.15.62:0/3573217853 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc019d40 con 0x564cb6558dc0 2024-09-12T00:02:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.662+0000 7fa9e33d5740 1 -- 172.21.15.62:0/3573217853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 msgr2=0x564cb6563140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.662+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/3573217853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6563140 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc011650 tx=0x7fa9cc009f90 comp rx=0 tx=0).stop 2024-09-12T00:02:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.662+0000 7fa9e33d5740 1 -- 172.21.15.62:0/3573217853 shutdown_connections 2024-09-12T00:02:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.662+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/3573217853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6563680 0x564cb6565a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.662+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/3573217853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6563140 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.662+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/3573217853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb65584a0 0x564cb6558880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.662+0000 7fa9e33d5740 1 -- 172.21.15.62:0/3573217853 >> 172.21.15.62:0/3573217853 conn(0x564cb6554a30 msgr2=0x564cb6554e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.662+0000 7fa9e33d5740 1 -- 172.21.15.62:0/3573217853 shutdown_connections 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.662+0000 7fa9e33d5740 1 -- 172.21.15.62:0/3573217853 wait complete. 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9e33d5740 1 -- start start 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9e33d5740 1 Processor -- start 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb65584a0 0x564cb6562f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6560220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6563680 0x564cb6560760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564cb6566250 con 0x564cb6558dc0 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564cb6479050 con 0x564cb6563680 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564cb65663d0 con 0x564cb65584a0 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9e33d5740 10 client.0 _ll_register_callbacks cb 0x564cb64b9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9da59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6560220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9da59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6560220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34452/0 (socket says 172.21.15.62:34452) 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9da59c640 1 -- 172.21.15.62:0/46913956 learned_addr learned my addr 172.21.15.62:0/46913956 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6563680 0x564cb6560760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9da59c640 1 -- 172.21.15.62:0/46913956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb65584a0 msgr2=0x564cb6562f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9da59c640 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb65584a0 0x564cb6562f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9da59c640 1 -- 172.21.15.62:0/46913956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6563680 msgr2=0x564cb6560760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9da59c640 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6563680 0x564cb6560760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9da59c640 1 -- 172.21.15.62:0/46913956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb622f3c0 con 0x564cb6558dc0 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6563680 0x564cb6560760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9da59c640 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6560220 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc01c040 tx=0x7fa9cc009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccf75679d0d9e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.663+0000 7fa9c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/46913956 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc01e040 con 0x564cb6558dc0 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/46913956 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fa9cc00fd50 con 0x564cb6558dc0 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9e33d5740 1 -- 172.21.15.62:0/46913956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9cc00fba0 con 0x564cb6558dc0 2024-09-12T00:02:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/46913956 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa9cc030050 con 0x564cb6558dc0 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9e33d5740 10 client.17262 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9e33d5740 1 -- 172.21.15.62:0/46913956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564cb6571bc0 con 0x564cb6558dc0 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/46913956 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9cc00fba0 con 0x564cb6558dc0 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/46913956 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa9cc021070 con 0x564cb6558dc0 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9e33d5740 10 client.17262 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9e33d5740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9e33d5740 20 client.17262 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564cb6572fc0 0x564cb65753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9e33d5740 4 client.17262 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.664+0000 7fa9e33d5740 1 -- 172.21.15.62:0/46913956 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cb64cb850 con 0x564cb6572fc0 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.665+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564cb6572fc0 0x564cb65753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.665+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564cb6572fc0 0x564cb65753b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e11f5efc226e98 server_cookie=2ba3a769dbfa52b4 in_seq=0 out_seq=0 2024-09-12T00:02:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.665+0000 7fa9c1ffb640 10 client.17262 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.665+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/46913956 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564cb64cb850 con 0x564cb6572fc0 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.665+0000 7fa9c1ffb640 10 client.17262 handle_command_reply: tid=0 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 client.17262 shutdown 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 -- 172.21.15.62:0/46913956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 msgr2=0x564cb6560220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6560220 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc01c040 tx=0x7fa9cc009ed0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 -- 172.21.15.62:0/46913956 shutdown_connections 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564cb6572fc0 0x564cb65753b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6563680 0x564cb6560760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6558dc0 0x564cb6560220 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/46913956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb65584a0 0x564cb6562f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 -- 172.21.15.62:0/46913956 >> 172.21.15.62:0/46913956 conn(0x564cb6554a30 msgr2=0x564cb6553b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9c1ffb640 0 client.17262 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 -- 172.21.15.62:0/46913956 shutdown_connections 2024-09-12T00:02:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 1 -- 172.21.15.62:0/46913956 wait complete. 2024-09-12T00:02:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.666+0000 7fa9e33d5740 20 client.17262 trim_cache size 0 max 16384 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9e33d5740 1 Processor -- start 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9e33d5740 1 -- start start 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb65584a0 0x564cb657abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6558dc0 0x564cb657b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6563680 0x564cb655d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1b0110 con 0x564cb65584a0 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1ad9b0 con 0x564cb6558dc0 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1ab820 con 0x564cb6563680 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb65584a0 0x564cb657abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb65584a0 0x564cb657abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34466/0 (socket says 172.21.15.62:34466) 2024-09-12T00:02:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9d3fff640 1 -- 172.21.15.62:0/82411225 learned_addr learned my addr 172.21.15.62:0/82411225 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.683+0000 7fa9da59c640 1 --2- 172.21.15.62:0/82411225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6558dc0 0x564cb657b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.684+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/82411225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6563680 0x564cb655d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.684+0000 7fa9d3fff640 1 -- 172.21.15.62:0/82411225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6563680 msgr2=0x564cb655d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.684+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/82411225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6563680 0x564cb655d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.684+0000 7fa9d3fff640 1 -- 172.21.15.62:0/82411225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6558dc0 msgr2=0x564cb657b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.684+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/82411225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6558dc0 0x564cb657b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.684+0000 7fa9d3fff640 1 -- 172.21.15.62:0/82411225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb6559a10 con 0x564cb65584a0 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.684+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/82411225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb65584a0 0x564cb657abb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0fed30 tx=0x7fa9c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47a0c56bf9b2e87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.684+0000 7fa9c3fff640 1 -- 172.21.15.62:0/82411225 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4005020 con 0x564cb65584a0 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.684+0000 7fa9c3fff640 1 -- 172.21.15.62:0/82411225 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c4026040 con 0x564cb65584a0 2024-09-12T00:02:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.684+0000 7fa9c3fff640 1 -- 172.21.15.62:0/82411225 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4020410 con 0x564cb65584a0 2024-09-12T00:02:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 -- 172.21.15.62:0/82411225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb65584a0 msgr2=0x564cb657abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/82411225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb65584a0 0x564cb657abb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0fed30 tx=0x7fa9c4004730 comp rx=0 tx=0).stop 2024-09-12T00:02:38.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 -- 172.21.15.62:0/82411225 shutdown_connections 2024-09-12T00:02:38.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/82411225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6563680 0x564cb655d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/82411225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb6558dc0 0x564cb657b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/82411225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb65584a0 0x564cb657abb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 -- 172.21.15.62:0/82411225 >> 172.21.15.62:0/82411225 conn(0x564cb6554a30 msgr2=0x564cb6553b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:38.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 -- 172.21.15.62:0/82411225 shutdown_connections 2024-09-12T00:02:38.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 -- 172.21.15.62:0/82411225 wait complete. 2024-09-12T00:02:38.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 -- start start 2024-09-12T00:02:38.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.685+0000 7fa9e33d5740 1 Processor -- start 2024-09-12T00:02:38.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb65584a0 0x564cb65d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6558dc0 0x564cb65d5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9e33d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6563680 0x564cb6568bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1abe70 con 0x564cb6563680 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1ad2e0 con 0x564cb65584a0 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9e33d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1ac540 con 0x564cb6558dc0 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9e33d5740 10 client.0 _ll_register_callbacks cb 0x7fa9cc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6563680 0x564cb6568bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6563680 0x564cb6568bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34472/0 (socket says 172.21.15.62:34472) 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9d3fff640 1 -- 172.21.15.62:0/2893369836 learned_addr learned my addr 172.21.15.62:0/2893369836 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9da59c640 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6558dc0 0x564cb65d5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb65584a0 0x564cb65d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9d3fff640 1 -- 172.21.15.62:0/2893369836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6558dc0 msgr2=0x564cb65d5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6558dc0 0x564cb65d5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9d3fff640 1 -- 172.21.15.62:0/2893369836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb65584a0 msgr2=0x564cb65d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:38.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb65584a0 0x564cb65d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9d3fff640 1 -- 172.21.15.62:0/2893369836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb6569400 con 0x564cb6563680 2024-09-12T00:02:38.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9da59c640 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6558dc0 0x564cb65d5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:38.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9d3fff640 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6563680 0x564cb6568bd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4004c90 tx=0x7fa9c4004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1187d0ff91357bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:38.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:38.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/2893369836 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4004020 con 0x564cb6563680 2024-09-12T00:02:38.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9e33d5740 1 -- 172.21.15.62:0/2893369836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c4026040 con 0x564cb6563680 2024-09-12T00:02:38.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.686+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/2893369836 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fa9c40041c0 con 0x564cb6563680 2024-09-12T00:02:38.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/2893369836 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa9c40043c0 con 0x564cb6563680 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9e33d5740 10 client.17268 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9e33d5740 1 -- 172.21.15.62:0/2893369836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564cb65dd6d0 con 0x564cb6563680 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/2893369836 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c4026040 con 0x564cb6563680 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/2893369836 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa9c4029020 con 0x564cb6563680 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9e33d5740 10 client.17268 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9e33d5740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9e33d5740 20 client.17268 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564cb65ddda0 0x564cb65e01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9e33d5740 4 client.17268 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9e33d5740 1 -- 172.21.15.62:0/2893369836 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564cb64c5d60 con 0x564cb65ddda0 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.687+0000 7fa9da59c640 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564cb65ddda0 0x564cb65e01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9c1ffb640 10 client.17268 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:38.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9da59c640 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564cb65ddda0 0x564cb65e01c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea624da08e275473 server_cookie=60ab2a27880e1152 in_seq=0 out_seq=0 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9c1ffb640 1 -- 172.21.15.62:0/2893369836 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564cb64c5d60 con 0x564cb65ddda0 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9c1ffb640 10 client.17268 handle_command_reply: tid=0 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 client.17268 shutdown 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 -- 172.21.15.62:0/2893369836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6563680 msgr2=0x564cb6568bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6563680 0x564cb6568bd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4004c90 tx=0x7fa9c4004d90 comp rx=0 tx=0).stop 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 -- 172.21.15.62:0/2893369836 shutdown_connections 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564cb65ddda0 0x564cb65e01c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cb6563680 0x564cb6568bd0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564cb6558dc0 0x564cb65d5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 --2- 172.21.15.62:0/2893369836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cb65584a0 0x564cb65d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 -- 172.21.15.62:0/2893369836 >> 172.21.15.62:0/2893369836 conn(0x564cb6554a30 msgr2=0x564cb6561a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9c1ffb640 0 client.17268 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 -- 172.21.15.62:0/2893369836 shutdown_connections 2024-09-12T00:02:38.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 1 -- 172.21.15.62:0/2893369836 wait complete. 2024-09-12T00:02:38.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.688+0000 7fa9e33d5740 20 client.17268 trim_cache size 0 max 16384 2024-09-12T00:02:38.694 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:02:38.694 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:02:38.694 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:02:38.694 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:02:38.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/3936096366 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9c80a2cf0 msgr2=0x7fa9c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:38.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9c80a2cf0 0x7fa9c80a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40bc6b0 tx=0x7fa9d408e6e0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/3936096366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc10c170 msgr2=0x7fa9dc19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:38.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc10c170 0x7fa9dc19ccc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005cc0 tx=0x7fa9d402f420 comp rx=0 tx=0).stop 2024-09-12T00:02:38.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/3936096366 shutdown_connections 2024-09-12T00:02:38.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa9c80a2cf0 0x7fa9c80a51b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9dc10c170 0x7fa9dc19ccc0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9dc103660 0x7fa9dc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 --2- 172.21.15.62:0/3936096366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9dc102d40 0x7fa9dc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:38.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/3936096366 >> 172.21.15.62:0/3936096366 conn(0x7fa9dc0fc7a0 msgr2=0x7fa9dc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:38.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/3936096366 shutdown_connections 2024-09-12T00:02:38.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:38.689+0000 7fa9e0ffc640 1 -- 172.21.15.62:0/3936096366 wait complete. 2024-09-12T00:02:38.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:38 smithi062 ceph-mon[30958]: pgmap v327: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.3 MiB/s wr, 554 op/s 2024-09-12T00:02:39.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:38 smithi120 ceph-mon[35182]: pgmap v327: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.3 MiB/s wr, 554 op/s 2024-09-12T00:02:39.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:38 smithi083 ceph-mon[34805]: pgmap v327: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.3 MiB/s wr, 554 op/s 2024-09-12T00:02:39.489 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:02:39.489 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:02:39.748 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:40.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.237+0000 7f7991f39640 1 -- 172.21.15.62:0/2282389096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f798c1036e0 msgr2=0x7f798c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:40.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.237+0000 7f7991f39640 1 --2- 172.21.15.62:0/2282389096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f798c1036e0 0x7f798c107b90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7980009930 tx=0x7f798002a420 comp rx=0 tx=0).stop 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.237+0000 7f7991f39640 1 -- 172.21.15.62:0/2282389096 shutdown_connections 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.237+0000 7f7991f39640 1 --2- 172.21.15.62:0/2282389096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f798c1082a0 0x7f798c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.237+0000 7f7991f39640 1 --2- 172.21.15.62:0/2282389096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f798c1036e0 0x7f798c107b90 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.237+0000 7f7991f39640 1 --2- 172.21.15.62:0/2282389096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f798c102d30 0x7f798c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.237+0000 7f7991f39640 1 -- 172.21.15.62:0/2282389096 >> 172.21.15.62:0/2282389096 conn(0x7f798c0fc7d0 msgr2=0x7f798c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.237+0000 7f7991f39640 1 -- 172.21.15.62:0/2282389096 shutdown_connections 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.237+0000 7f7991f39640 1 -- 172.21.15.62:0/2282389096 wait complete. 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.237+0000 7f7991f39640 1 Processor -- start 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f7991f39640 1 -- start start 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f7991f39640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f798c102d30 0x7f798c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f7991f39640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f798c1036e0 0x7f798c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f7991f39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f798c1082a0 0x7f798c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f7991f39640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1120a0 con 0x7f798c1036e0 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f7991f39640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f798c111f20 con 0x7f798c1082a0 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f7991f39640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f798c112220 con 0x7f798c102d30 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f798affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f798c1036e0 0x7f798c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f798affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f798c1036e0 0x7f798c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34486/0 (socket says 172.21.15.62:34486) 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f798affd640 1 -- 172.21.15.62:0/3605702597 learned_addr learned my addr 172.21.15.62:0/3605702597 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f798b7fe640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f798c102d30 0x7f798c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f798bfff640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f798c1082a0 0x7f798c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f798affd640 1 -- 172.21.15.62:0/3605702597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f798c102d30 msgr2=0x7f798c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f798affd640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f798c102d30 0x7f798c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f798affd640 1 -- 172.21.15.62:0/3605702597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f798c1082a0 msgr2=0x7f798c19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.238+0000 7f798affd640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f798c1082a0 0x7f798c19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.239+0000 7f798affd640 1 -- 172.21.15.62:0/3605702597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c19d530 con 0x7f798c1036e0 2024-09-12T00:02:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.239+0000 7f798affd640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f798c1036e0 0x7f798c198ac0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7980005940 tx=0x7f79800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.239+0000 7f7988ff9640 1 -- 172.21.15.62:0/3605702597 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798003f030 con 0x7f798c1036e0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.239+0000 7f7991f39640 1 -- 172.21.15.62:0/3605702597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c19d7c0 con 0x7f798c1036e0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.239+0000 7f7988ff9640 1 -- 172.21.15.62:0/3605702597 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7980003ce0 con 0x7f798c1036e0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.239+0000 7f7988ff9640 1 -- 172.21.15.62:0/3605702597 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980035070 con 0x7f798c1036e0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.239+0000 7f7991f39640 1 -- 172.21.15.62:0/3605702597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c19dc70 con 0x7f798c1036e0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.240+0000 7f7991f39640 1 -- 172.21.15.62:0/3605702597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f798c103110 con 0x7f798c1036e0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.241+0000 7f7988ff9640 1 -- 172.21.15.62:0/3605702597 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7980042030 con 0x7f798c1036e0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.242+0000 7f7988ff9640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79740a2cf0 0x7f79740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.242+0000 7f798b7fe640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79740a2cf0 0x7f79740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.242+0000 7f7988ff9640 1 -- 172.21.15.62:0/3605702597 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f79800efac0 con 0x7f798c1036e0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.242+0000 7f798b7fe640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79740a2cf0 0x7f79740a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f797c005fd0 tx=0x7f797c00b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.245+0000 7f7988ff9640 1 -- 172.21.15.62:0/3605702597 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79800bc090 con 0x7f798c1036e0 2024-09-12T00:02:40.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.457+0000 7f7991f39640 1 -- 172.21.15.62:0/3605702597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f798c1998e0 con 0x7f798c1036e0 2024-09-12T00:02:40.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.458+0000 7f7988ff9640 1 -- 172.21.15.62:0/3605702597 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11911 (secure 0 0 0) 0x7f798c1998e0 con 0x7f798c1036e0 2024-09-12T00:02:40.458 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:02:40.458 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":33,"btime":"2024-09-12T00:01:52:327104+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:01:52.081596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:02:40.458 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 33 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 -- 172.21.15.62:0/3605702597 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79740a2cf0 msgr2=0x7f79740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79740a2cf0 0x7f79740a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f797c005fd0 tx=0x7f797c00b390 comp rx=0 tx=0).stop 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 -- 172.21.15.62:0/3605702597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f798c1036e0 msgr2=0x7f798c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f798c1036e0 0x7f798c198ac0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7980005940 tx=0x7f79800042f0 comp rx=0 tx=0).stop 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 -- 172.21.15.62:0/3605702597 shutdown_connections 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f79740a2cf0 0x7f79740a51b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f798c1082a0 0x7f798c19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f798c1036e0 0x7f798c198ac0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 --2- 172.21.15.62:0/3605702597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f798c102d30 0x7f798c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 -- 172.21.15.62:0/3605702597 >> 172.21.15.62:0/3605702597 conn(0x7f798c0fc7d0 msgr2=0x7f798c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:40.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 -- 172.21.15.62:0/3605702597 shutdown_connections 2024-09-12T00:02:40.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:40.460+0000 7f7991f39640 1 -- 172.21.15.62:0/3605702597 wait complete. 2024-09-12T00:02:41.043 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:40 smithi062 ceph-mon[30958]: pgmap v328: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 460 op/s 2024-09-12T00:02:41.043 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:40 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3605702597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:41.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:02:41.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:40 smithi120 ceph-mon[35182]: pgmap v328: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 460 op/s 2024-09-12T00:02:41.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:40 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3605702597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:41.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:40 smithi083 ceph-mon[34805]: pgmap v328: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 460 op/s 2024-09-12T00:02:41.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:40 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3605702597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:41.328 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.824+0000 7f7299196640 1 -- 172.21.15.62:0/2469078210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7294101de0 msgr2=0x7f72941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.824+0000 7f7299196640 1 --2- 172.21.15.62:0/2469078210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7294101de0 0x7f72941021c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7284009ca0 tx=0x7f728402a8c0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.824+0000 7f7299196640 1 -- 172.21.15.62:0/2469078210 shutdown_connections 2024-09-12T00:02:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.824+0000 7f7299196640 1 --2- 172.21.15.62:0/2469078210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294106f10 0x7f729410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.824+0000 7f7299196640 1 --2- 172.21.15.62:0/2469078210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7294102790 0x7f72941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.824+0000 7f7299196640 1 --2- 172.21.15.62:0/2469078210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7294101de0 0x7f72941021c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.824+0000 7f7299196640 1 -- 172.21.15.62:0/2469078210 >> 172.21.15.62:0/2469078210 conn(0x7f72940fd8c0 msgr2=0x7f72940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.824+0000 7f7299196640 1 -- 172.21.15.62:0/2469078210 shutdown_connections 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.824+0000 7f7299196640 1 -- 172.21.15.62:0/2469078210 wait complete. 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f7299196640 1 Processor -- start 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f7299196640 1 -- start start 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f7299196640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7294101de0 0x7f7294198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f7299196640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294102790 0x7f7294198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f7299196640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7294106f10 0x7f729419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f7299196640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7294111f30 con 0x7f7294106f10 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f7299196640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7294111db0 con 0x7f7294102790 2024-09-12T00:02:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f7299196640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72941120b0 con 0x7f7294101de0 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f729359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7294106f10 0x7f729419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f729359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7294106f10 0x7f729419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42828/0 (socket says 172.21.15.62:42828) 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f729359e640 1 -- 172.21.15.62:0/123922873 learned_addr learned my addr 172.21.15.62:0/123922873 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f729259c640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294102790 0x7f7294198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.825+0000 7f7292d9d640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7294101de0 0x7f7294198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f729359e640 1 -- 172.21.15.62:0/123922873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7294101de0 msgr2=0x7f7294198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f729359e640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7294101de0 0x7f7294198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f729359e640 1 -- 172.21.15.62:0/123922873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294102790 msgr2=0x7f7294198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f729359e640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294102790 0x7f7294198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f729359e640 1 -- 172.21.15.62:0/123922873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729419d3b0 con 0x7f7294106f10 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f729359e640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7294106f10 0x7f729419ccd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c970 tx=0x7f728c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:41.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f728bfff640 1 -- 172.21.15.62:0/123922873 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c007bb0 con 0x7f7294106f10 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f728bfff640 1 -- 172.21.15.62:0/123922873 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f728c007d50 con 0x7f7294106f10 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f7299196640 1 -- 172.21.15.62:0/123922873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729419d6a0 con 0x7f7294106f10 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f728bfff640 1 -- 172.21.15.62:0/123922873 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c00f450 con 0x7f7294106f10 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.826+0000 7f7299196640 1 -- 172.21.15.62:0/123922873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72941a4f80 con 0x7f7294106f10 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.828+0000 7f728bfff640 1 -- 172.21.15.62:0/123922873 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f728c004f60 con 0x7f7294106f10 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729b56f740 1 Processor -- start 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729b56f740 1 -- start start 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729b56f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d78b40 0x55fe50d77090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729b56f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 0x55fe50d77a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729b56f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d82370 0x55fe50d84760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729b56f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fe50cdd2f0 con 0x55fe50d82370 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729b56f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe50c84cf0 con 0x55fe50d78b40 2024-09-12T00:02:41.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729b56f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fe50cd92b0 con 0x55fe50d775d0 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d82370 0x55fe50d84760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d82370 0x55fe50d84760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42840/0 (socket says 172.21.15.62:42840) 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729259c640 1 -- 172.21.15.62:0/2045045997 learned_addr learned my addr 172.21.15.62:0/2045045997 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f728bfff640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f72800a2d70 0x7f72800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f7292d9d640 1 --2- 172.21.15.62:0/2045045997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 0x55fe50d77a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.829+0000 7f729259c640 1 --2- 172.21.15.62:0/2045045997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d78b40 0x55fe50d77090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f729359e640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f72800a2d70 0x7f72800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f729259c640 1 -- 172.21.15.62:0/2045045997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 msgr2=0x55fe50d77a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f729259c640 1 --2- 172.21.15.62:0/2045045997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 0x55fe50d77a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f729259c640 1 -- 172.21.15.62:0/2045045997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d78b40 msgr2=0x55fe50d77090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f729259c640 1 --2- 172.21.15.62:0/2045045997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d78b40 0x55fe50d77090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f729259c640 1 -- 172.21.15.62:0/2045045997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe50c693a0 con 0x55fe50d82370 2024-09-12T00:02:41.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f728bfff640 1 -- 172.21.15.62:0/123922873 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f728c0cefd0 con 0x7f7294106f10 2024-09-12T00:02:41.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f729259c640 1 --2- 172.21.15.62:0/2045045997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d82370 0x55fe50d84760 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f727c0127c0 tx=0x7f727c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819e53d7fcae14f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:41.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f72897fa640 1 -- 172.21.15.62:0/2045045997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c012e00 con 0x55fe50d82370 2024-09-12T00:02:41.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f729359e640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f72800a2d70 0x7f72800a5230 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f728c0cc7d0 tx=0x7f728c050d90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:41.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f72897fa640 1 -- 172.21.15.62:0/2045045997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f727c010c50 con 0x55fe50d82370 2024-09-12T00:02:41.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.830+0000 7f72897fa640 1 -- 172.21.15.62:0/2045045997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c01eb30 con 0x55fe50d82370 2024-09-12T00:02:41.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.831+0000 7f729b56f740 1 -- 172.21.15.62:0/2045045997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d82370 msgr2=0x55fe50d84760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.831+0000 7f729b56f740 1 --2- 172.21.15.62:0/2045045997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d82370 0x55fe50d84760 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f727c0127c0 tx=0x7f727c00b7c0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.831+0000 7f729b56f740 1 -- 172.21.15.62:0/2045045997 shutdown_connections 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.831+0000 7f729b56f740 1 --2- 172.21.15.62:0/2045045997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d82370 0x55fe50d84760 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.831+0000 7f729b56f740 1 --2- 172.21.15.62:0/2045045997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 0x55fe50d77a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.831+0000 7f729b56f740 1 --2- 172.21.15.62:0/2045045997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d78b40 0x55fe50d77090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.831+0000 7f729b56f740 1 -- 172.21.15.62:0/2045045997 >> 172.21.15.62:0/2045045997 conn(0x55fe50d40310 msgr2=0x55fe50d73c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.831+0000 7f729b56f740 1 -- 172.21.15.62:0/2045045997 shutdown_connections 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.831+0000 7f729b56f740 1 -- 172.21.15.62:0/2045045997 wait complete. 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729b56f740 1 -- start start 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729b56f740 1 Processor -- start 2024-09-12T00:02:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729b56f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 0x55fe50d81d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729b56f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729b56f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d82370 0x55fe50d7f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729b56f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fe50d85070 con 0x55fe50d78b40 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729b56f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe50c98060 con 0x55fe50d82370 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729b56f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fe50d851f0 con 0x55fe50d775d0 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729b56f740 10 client.0 _ll_register_callbacks cb 0x55fe509eb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f7292d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f7292d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d7eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42856/0 (socket says 172.21.15.62:42856) 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f7292d9d640 1 -- 172.21.15.62:0/2432601027 learned_addr learned my addr 172.21.15.62:0/2432601027 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729259c640 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d82370 0x55fe50d7f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f729259c640 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 0x55fe50d81d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f7292d9d640 1 -- 172.21.15.62:0/2432601027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 msgr2=0x55fe50d81d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:41.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f7292d9d640 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 0x55fe50d81d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f7292d9d640 1 -- 172.21.15.62:0/2432601027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d82370 msgr2=0x55fe50d7f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f7292d9d640 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d82370 0x55fe50d7f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.832+0000 7f7292d9d640 1 -- 172.21.15.62:0/2432601027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe50a4e3c0 con 0x55fe50d78b40 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f729259c640 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 0x55fe50d81d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f7292d9d640 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d7eff0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7284009ca0 tx=0x7f7284031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ada3e387108f51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f727a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f727a7fc640 1 -- 172.21.15.62:0/2432601027 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728402f810 con 0x55fe50d78b40 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f727a7fc640 1 -- 172.21.15.62:0/2432601027 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f728402f9b0 con 0x55fe50d78b40 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f729b56f740 1 -- 172.21.15.62:0/2432601027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f727c010c50 con 0x55fe50d78b40 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f727a7fc640 1 -- 172.21.15.62:0/2432601027 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7284053050 con 0x55fe50d78b40 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f729b56f740 10 client.17286 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:41.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f729b56f740 1 -- 172.21.15.62:0/2432601027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fe50d90ae0 con 0x55fe50d78b40 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f727a7fc640 1 -- 172.21.15.62:0/2432601027 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f727c010c50 con 0x55fe50d78b40 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f727a7fc640 1 -- 172.21.15.62:0/2432601027 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f728404e030 con 0x55fe50d78b40 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f729b56f740 10 client.17286 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.833+0000 7f729b56f740 10 client.17286 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.834+0000 7f729b56f740 20 client.17286 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.834+0000 7f729b56f740 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55fe50d92040 0x55fe50d94460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.834+0000 7f729b56f740 4 client.17286 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.834+0000 7f729b56f740 1 -- 172.21.15.62:0/2432601027 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe50cea860 con 0x55fe50d92040 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.834+0000 7f729259c640 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55fe50d92040 0x55fe50d94460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.834+0000 7f729259c640 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55fe50d92040 0x55fe50d94460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aca101c74a7616c9 server_cookie=49b8dd93e8d41b00 in_seq=0 out_seq=0 2024-09-12T00:02:41.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.834+0000 7f727a7fc640 10 client.17286 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:02:41.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.835+0000 7f727a7fc640 1 -- 172.21.15.62:0/2432601027 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe50cea860 con 0x55fe50d92040 2024-09-12T00:02:41.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.835+0000 7f727a7fc640 10 client.17286 handle_command_reply: tid=0 2024-09-12T00:02:41.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.835+0000 7f729b56f740 1 client.17286 shutdown 2024-09-12T00:02:41.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.835+0000 7f729b56f740 1 -- 172.21.15.62:0/2432601027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 msgr2=0x55fe50d7eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:41.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.835+0000 7f729b56f740 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d7eff0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7284009ca0 tx=0x7f7284031680 comp rx=0 tx=0).stop 2024-09-12T00:02:41.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.836+0000 7f729b56f740 1 -- 172.21.15.62:0/2432601027 shutdown_connections 2024-09-12T00:02:41.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.836+0000 7f729b56f740 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55fe50d92040 0x55fe50d94460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.836+0000 7f729b56f740 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d82370 0x55fe50d7f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.836+0000 7f729b56f740 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d7eff0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.836+0000 7f727a7fc640 0 client.17286 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:02:41.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.836+0000 7f729b56f740 1 --2- 172.21.15.62:0/2432601027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d775d0 0x55fe50d81d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.836+0000 7f729b56f740 1 -- 172.21.15.62:0/2432601027 >> 172.21.15.62:0/2432601027 conn(0x55fe50d40310 msgr2=0x55fe50d79960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:41.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.836+0000 7f729b56f740 1 -- 172.21.15.62:0/2432601027 shutdown_connections 2024-09-12T00:02:41.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.836+0000 7f729b56f740 1 -- 172.21.15.62:0/2432601027 wait complete. 2024-09-12T00:02:41.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.836+0000 7f729b56f740 20 client.17286 trim_cache size 0 max 16384 2024-09-12T00:02:41.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729b56f740 1 Processor -- start 2024-09-12T00:02:41.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729b56f740 1 -- start start 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729b56f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d775d0 0x55fe50d73d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729b56f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d78b40 0x55fe50d8e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729b56f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d82370 0x55fe50d7dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729b56f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72941b0130 con 0x55fe50d775d0 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729b56f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72941ad9d0 con 0x55fe50d78b40 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729b56f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72941ab840 con 0x55fe50d82370 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d775d0 0x55fe50d73d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d775d0 0x55fe50d73d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42866/0 (socket says 172.21.15.62:42866) 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f729259c640 1 -- 172.21.15.62:0/2624762978 learned_addr learned my addr 172.21.15.62:0/2624762978 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.849+0000 7f7292d9d640 1 --2- 172.21.15.62:0/2624762978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d78b40 0x55fe50d8e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.850+0000 7f729259c640 1 -- 172.21.15.62:0/2624762978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d82370 msgr2=0x55fe50d7dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:41.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.850+0000 7f729259c640 1 --2- 172.21.15.62:0/2624762978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d82370 0x55fe50d7dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.850+0000 7f729259c640 1 -- 172.21.15.62:0/2624762978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d78b40 msgr2=0x55fe50d8e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:41.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.850+0000 7f729259c640 1 --2- 172.21.15.62:0/2624762978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d78b40 0x55fe50d8e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.850+0000 7f729259c640 1 -- 172.21.15.62:0/2624762978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe50d94ae0 con 0x55fe50d775d0 2024-09-12T00:02:41.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.850+0000 7f729259c640 1 --2- 172.21.15.62:0/2624762978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d775d0 0x55fe50d73d70 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f727c0127c0 tx=0x7f727c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddebb939c616243 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:41.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.850+0000 7f7288ff9640 1 -- 172.21.15.62:0/2624762978 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c024040 con 0x55fe50d775d0 2024-09-12T00:02:41.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.850+0000 7f7288ff9640 1 -- 172.21.15.62:0/2624762978 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f727c005530 con 0x55fe50d775d0 2024-09-12T00:02:41.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.850+0000 7f7288ff9640 1 -- 172.21.15.62:0/2624762978 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c01edc0 con 0x55fe50d775d0 2024-09-12T00:02:41.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 -- 172.21.15.62:0/2624762978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d775d0 msgr2=0x55fe50d73d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:41.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 --2- 172.21.15.62:0/2624762978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d775d0 0x55fe50d73d70 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f727c0127c0 tx=0x7f727c00b7c0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 -- 172.21.15.62:0/2624762978 shutdown_connections 2024-09-12T00:02:41.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 --2- 172.21.15.62:0/2624762978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d82370 0x55fe50d7dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 --2- 172.21.15.62:0/2624762978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d78b40 0x55fe50d8e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 --2- 172.21.15.62:0/2624762978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d775d0 0x55fe50d73d70 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 -- 172.21.15.62:0/2624762978 >> 172.21.15.62:0/2624762978 conn(0x55fe50d40310 msgr2=0x55fe50d7d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:41.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 -- 172.21.15.62:0/2624762978 shutdown_connections 2024-09-12T00:02:41.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 -- 172.21.15.62:0/2624762978 wait complete. 2024-09-12T00:02:41.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 -- start start 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 Processor -- start 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d775d0 0x55fe50d8a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d8ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d82370 0x55fe50d7b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72941abe90 con 0x55fe50d78b40 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72941ad300 con 0x55fe50d775d0 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.851+0000 7f729b56f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72941ac560 con 0x55fe50d82370 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f729b56f740 10 client.0 _ll_register_callbacks cb 0x7f728402f3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f7292d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d8ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f7292d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d8ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42872/0 (socket says 172.21.15.62:42872) 2024-09-12T00:02:41.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f7292d9d640 1 -- 172.21.15.62:0/737027301 learned_addr learned my addr 172.21.15.62:0/737027301 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:41.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f729259c640 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d775d0 0x55fe50d8a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f729259c640 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d82370 0x55fe50d7b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f7292d9d640 1 -- 172.21.15.62:0/737027301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d82370 msgr2=0x55fe50d7b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:41.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f7292d9d640 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d82370 0x55fe50d7b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f7292d9d640 1 -- 172.21.15.62:0/737027301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d775d0 msgr2=0x55fe50d8a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:41.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f7292d9d640 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d775d0 0x55fe50d8a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f7292d9d640 1 -- 172.21.15.62:0/737027301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe50d7c200 con 0x55fe50d78b40 2024-09-12T00:02:41.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f729259c640 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d775d0 0x55fe50d8a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:41.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f7292d9d640 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d8ac80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f728402a900 tx=0x7f728403b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2382f5e2c7aea2b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:41.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f727a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f727a7fc640 1 -- 172.21.15.62:0/737027301 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728403bc60 con 0x55fe50d78b40 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f727a7fc640 1 -- 172.21.15.62:0/737027301 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f728403be00 con 0x55fe50d78b40 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f729b56f740 1 -- 172.21.15.62:0/737027301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f727c005530 con 0x55fe50d78b40 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f727a7fc640 1 -- 172.21.15.62:0/737027301 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7284064050 con 0x55fe50d78b40 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f729b56f740 10 client.17292 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.852+0000 7f729b56f740 1 -- 172.21.15.62:0/737027301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fe50dfc8e0 con 0x55fe50d78b40 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.853+0000 7f727a7fc640 1 -- 172.21.15.62:0/737027301 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f727c005530 con 0x55fe50d78b40 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.853+0000 7f727a7fc640 1 -- 172.21.15.62:0/737027301 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f728405f030 con 0x55fe50d78b40 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.853+0000 7f729b56f740 10 client.17292 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.853+0000 7f729b56f740 10 client.17292 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.853+0000 7f729b56f740 20 client.17292 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.853+0000 7f729b56f740 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55fe50dfd050 0x55fe50dff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:41.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.853+0000 7f729b56f740 4 client.17292 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.853+0000 7f729b56f740 1 -- 172.21.15.62:0/737027301 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe50ce4d70 con 0x55fe50dfd050 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.853+0000 7f729259c640 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55fe50dfd050 0x55fe50dff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f729259c640 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55fe50dfd050 0x55fe50dff440 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d86ce3b64803a4be server_cookie=d0830c2a551eaa28 in_seq=0 out_seq=0 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f727a7fc640 10 client.17292 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f727a7fc640 1 -- 172.21.15.62:0/737027301 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe50ce4d70 con 0x55fe50dfd050 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f727a7fc640 10 client.17292 handle_command_reply: tid=0 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f729b56f740 1 client.17292 shutdown 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f729b56f740 1 -- 172.21.15.62:0/737027301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 msgr2=0x55fe50d8ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f729b56f740 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d8ac80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f728402a900 tx=0x7f728403b630 comp rx=0 tx=0).stop 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f729b56f740 1 -- 172.21.15.62:0/737027301 shutdown_connections 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f729b56f740 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55fe50dfd050 0x55fe50dff440 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f729b56f740 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fe50d82370 0x55fe50d7b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f727a7fc640 0 client.17292 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:02:41.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f729b56f740 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe50d78b40 0x55fe50d8ac80 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f729b56f740 1 --2- 172.21.15.62:0/737027301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe50d775d0 0x55fe50d8a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.854+0000 7f729b56f740 1 -- 172.21.15.62:0/737027301 >> 172.21.15.62:0/737027301 conn(0x55fe50d40310 msgr2=0x55fe50d7e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:41.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f729b56f740 1 -- 172.21.15.62:0/737027301 shutdown_connections 2024-09-12T00:02:41.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f729b56f740 1 -- 172.21.15.62:0/737027301 wait complete. 2024-09-12T00:02:41.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f729b56f740 20 client.17292 trim_cache size 0 max 16384 2024-09-12T00:02:41.861 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:02:41.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 -- 172.21.15.62:0/123922873 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f72800a2d70 msgr2=0x7f72800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:41.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f72800a2d70 0x7f72800a5230 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f728c0cc7d0 tx=0x7f728c050d90 comp rx=0 tx=0).stop 2024-09-12T00:02:41.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 -- 172.21.15.62:0/123922873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7294106f10 msgr2=0x7f729419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:41.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7294106f10 0x7f729419ccd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c970 tx=0x7f728c00ce30 comp rx=0 tx=0).stop 2024-09-12T00:02:41.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 -- 172.21.15.62:0/123922873 shutdown_connections 2024-09-12T00:02:41.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f72800a2d70 0x7f72800a5230 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7294106f10 0x7f729419ccd0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294102790 0x7f7294198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 --2- 172.21.15.62:0/123922873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7294101de0 0x7f7294198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:41.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 -- 172.21.15.62:0/123922873 >> 172.21.15.62:0/123922873 conn(0x7f72940fd8c0 msgr2=0x7f72940fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:41.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 -- 172.21.15.62:0/123922873 shutdown_connections 2024-09-12T00:02:41.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:41.855+0000 7f7299196640 1 -- 172.21.15.62:0/123922873 wait complete. 2024-09-12T00:02:42.001 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:02:42.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 600.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:02:42.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:02:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:43.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:42 smithi120 ceph-mon[35182]: pgmap v329: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-09-12T00:02:43.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:42 smithi062 ceph-mon[30958]: pgmap v329: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-09-12T00:02:43.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.121+0000 7f130e11f640 1 -- 172.21.15.62:0/4096021731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f130810e500 msgr2=0x7f1308110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.121+0000 7f130e11f640 1 --2- 172.21.15.62:0/4096021731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f130810e500 0x7f1308110910 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1300009ca0 tx=0x7f130002a8c0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.121+0000 7f130e11f640 1 -- 172.21.15.62:0/4096021731 shutdown_connections 2024-09-12T00:02:43.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.121+0000 7f130e11f640 1 --2- 172.21.15.62:0/4096021731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f130810e500 0x7f1308110910 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.121+0000 7f130e11f640 1 --2- 172.21.15.62:0/4096021731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1308101440 0x7f130810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.121+0000 7f130e11f640 1 --2- 172.21.15.62:0/4096021731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1308100b20 0x7f1308100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.121+0000 7f130e11f640 1 -- 172.21.15.62:0/4096021731 >> 172.21.15.62:0/4096021731 conn(0x7f13080fc7d0 msgr2=0x7f13080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.121+0000 7f130e11f640 1 -- 172.21.15.62:0/4096021731 shutdown_connections 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.122+0000 7f130e11f640 1 -- 172.21.15.62:0/4096021731 wait complete. 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.122+0000 7f130e11f640 1 Processor -- start 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f130e11f640 1 -- start start 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f130e11f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1308100b20 0x7f13081a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f130e11f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308101440 0x7f13081a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f130e11f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130810e500 0x7f130819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f130e11f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1308114250 con 0x7f1308101440 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f130e11f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13081140d0 con 0x7f130810e500 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f130e11f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f13081143d0 con 0x7f1308100b20 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f1306ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308101440 0x7f13081a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f1306ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308101440 0x7f13081a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42898/0 (socket says 172.21.15.62:42898) 2024-09-12T00:02:43.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f1306ffd640 1 -- 172.21.15.62:0/310615734 learned_addr learned my addr 172.21.15.62:0/310615734 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f13077fe640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1308100b20 0x7f13081a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f1307fff640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130810e500 0x7f130819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f1306ffd640 1 -- 172.21.15.62:0/310615734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1308100b20 msgr2=0x7f13081a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f1306ffd640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1308100b20 0x7f13081a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f1306ffd640 1 -- 172.21.15.62:0/310615734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130810e500 msgr2=0x7f130819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f1306ffd640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130810e500 0x7f130819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f1306ffd640 1 -- 172.21.15.62:0/310615734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130819acd0 con 0x7f1308101440 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f1307fff640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130810e500 0x7f130819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.123+0000 7f13077fe640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1308100b20 0x7f13081a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.124+0000 7f1306ffd640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308101440 0x7f13081a0920 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f12f800d770 tx=0x7f12f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.124+0000 7f1304ff9640 1 -- 172.21.15.62:0/310615734 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8017070 con 0x7f1308101440 2024-09-12T00:02:43.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.124+0000 7f130e11f640 1 -- 172.21.15.62:0/310615734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130819afc0 con 0x7f1308101440 2024-09-12T00:02:43.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.124+0000 7f1304ff9640 1 -- 172.21.15.62:0/310615734 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f8009c30 con 0x7f1308101440 2024-09-12T00:02:43.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.124+0000 7f1304ff9640 1 -- 172.21.15.62:0/310615734 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8003dd0 con 0x7f1308101440 2024-09-12T00:02:43.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.124+0000 7f130e11f640 1 -- 172.21.15.62:0/310615734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13081a72b0 con 0x7f1308101440 2024-09-12T00:02:43.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.125+0000 7f130e11f640 1 -- 172.21.15.62:0/310615734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130806a850 con 0x7f1308101440 2024-09-12T00:02:43.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.126+0000 7f1304ff9640 1 -- 172.21.15.62:0/310615734 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f12f8003f70 con 0x7f1308101440 2024-09-12T00:02:43.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.127+0000 7f1304ff9640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f12f40a2d70 0x7f12f40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.127+0000 7f13077fe640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f12f40a2d70 0x7f12f40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.127+0000 7f1304ff9640 1 -- 172.21.15.62:0/310615734 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f12f80cf230 con 0x7f1308101440 2024-09-12T00:02:43.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.127+0000 7f13077fe640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f12f40a2d70 0x7f12f40a5230 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f12f00058c0 tx=0x7f12f0005850 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:43.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.130+0000 7f1304ff9640 1 -- 172.21.15.62:0/310615734 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12f8009de0 con 0x7f1308101440 2024-09-12T00:02:43.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:42 smithi083 ceph-mon[34805]: pgmap v329: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-09-12T00:02:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.352+0000 7f130e11f640 1 -- 172.21.15.62:0/310615734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1308100f00 con 0x7f1308101440 2024-09-12T00:02:43.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.352+0000 7f1304ff9640 1 -- 172.21.15.62:0/310615734 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11911 (secure 0 0 0) 0x7f1308100f00 con 0x7f1308101440 2024-09-12T00:02:43.352 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:02:43.352 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":33,"btime":"2024-09-12T00:01:52:327104+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:01:52.081596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:02:43.353 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 33 2024-09-12T00:02:43.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.354+0000 7f130e11f640 1 -- 172.21.15.62:0/310615734 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f12f40a2d70 msgr2=0x7f12f40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.354+0000 7f130e11f640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f12f40a2d70 0x7f12f40a5230 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f12f00058c0 tx=0x7f12f0005850 comp rx=0 tx=0).stop 2024-09-12T00:02:43.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.354+0000 7f130e11f640 1 -- 172.21.15.62:0/310615734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308101440 msgr2=0x7f13081a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.354+0000 7f130e11f640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308101440 0x7f13081a0920 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f12f800d770 tx=0x7f12f800dc30 comp rx=0 tx=0).stop 2024-09-12T00:02:43.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.354+0000 7f130e11f640 1 -- 172.21.15.62:0/310615734 shutdown_connections 2024-09-12T00:02:43.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.354+0000 7f130e11f640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f12f40a2d70 0x7f12f40a5230 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.354+0000 7f130e11f640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130810e500 0x7f130819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.354+0000 7f130e11f640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308101440 0x7f13081a0920 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.354+0000 7f130e11f640 1 --2- 172.21.15.62:0/310615734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1308100b20 0x7f13081a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.354+0000 7f130e11f640 1 -- 172.21.15.62:0/310615734 >> 172.21.15.62:0/310615734 conn(0x7f13080fc7d0 msgr2=0x7f13080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:43.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.355+0000 7f130e11f640 1 -- 172.21.15.62:0/310615734 shutdown_connections 2024-09-12T00:02:43.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.355+0000 7f130e11f640 1 -- 172.21.15.62:0/310615734 wait complete. 2024-09-12T00:02:43.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.888+0000 7f109ed1b640 1 -- 172.21.15.62:0/2414446147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098101df0 msgr2=0x7f10981021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.888+0000 7f109ed1b640 1 --2- 172.21.15.62:0/2414446147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098101df0 0x7f10981021d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f108c009900 tx=0x7f108c02a410 comp rx=0 tx=0).stop 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.888+0000 7f109ed1b640 1 -- 172.21.15.62:0/2414446147 shutdown_connections 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.888+0000 7f109ed1b640 1 --2- 172.21.15.62:0/2414446147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098106f20 0x7f109810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.888+0000 7f109ed1b640 1 --2- 172.21.15.62:0/2414446147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10981027a0 0x7f10981067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.888+0000 7f109ed1b640 1 --2- 172.21.15.62:0/2414446147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098101df0 0x7f10981021d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.888+0000 7f109ed1b640 1 -- 172.21.15.62:0/2414446147 >> 172.21.15.62:0/2414446147 conn(0x7f10980fd8b0 msgr2=0x7f10980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.888+0000 7f109ed1b640 1 -- 172.21.15.62:0/2414446147 shutdown_connections 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.888+0000 7f109ed1b640 1 -- 172.21.15.62:0/2414446147 wait complete. 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f109ed1b640 1 Processor -- start 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f109ed1b640 1 -- start start 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f109ed1b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1098101df0 0x7f10980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f109ed1b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10981027a0 0x7f1098074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f109ed1b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098106f20 0x7f1098074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f109ed1b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1098111fc0 con 0x7f10981027a0 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f109ed1b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1098111e40 con 0x7f1098106f20 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f109ed1b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1098112140 con 0x7f1098101df0 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f1097fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10981027a0 0x7f1098074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f1097fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10981027a0 0x7f1098074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42904/0 (socket says 172.21.15.62:42904) 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f1097fff640 1 -- 172.21.15.62:0/2824544261 learned_addr learned my addr 172.21.15.62:0/2824544261 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f109cab7640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1098101df0 0x7f10980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.889+0000 7f109d2b8640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098106f20 0x7f1098074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f1097fff640 1 -- 172.21.15.62:0/2824544261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1098101df0 msgr2=0x7f10980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f1097fff640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1098101df0 0x7f10980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f1097fff640 1 -- 172.21.15.62:0/2824544261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098106f20 msgr2=0x7f1098074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f1097fff640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098106f20 0x7f1098074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f1097fff640 1 -- 172.21.15.62:0/2824544261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10981a6950 con 0x7f10981027a0 2024-09-12T00:02:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f1097fff640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10981027a0 0x7f1098074a30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f108400d770 tx=0x7f108400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f1095ffb640 1 -- 172.21.15.62:0/2824544261 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084017070 con 0x7f10981027a0 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f109ed1b640 1 -- 172.21.15.62:0/2824544261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981a6c40 con 0x7f10981027a0 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f1095ffb640 1 -- 172.21.15.62:0/2824544261 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1084005500 con 0x7f10981027a0 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f1095ffb640 1 -- 172.21.15.62:0/2824544261 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084004e00 con 0x7f10981027a0 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.890+0000 7f109ed1b640 1 -- 172.21.15.62:0/2824544261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981a7180 con 0x7f10981027a0 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.892+0000 7f1095ffb640 1 -- 172.21.15.62:0/2824544261 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1084004fa0 con 0x7f10981027a0 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f10a10f4740 1 Processor -- start 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f10a10f4740 1 -- start start 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f1095ffb640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10880a2cf0 0x7f10880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 0x563d5eeb0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eebb370 0x563d5eebd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f10a10f4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563d5ee162f0 con 0x563d5eeb1b40 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f10a10f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563d5edbdcf0 con 0x563d5eebb370 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f10a10f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d5ee122b0 con 0x563d5eeb05d0 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f109d2b8640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10880a2cf0 0x7f10880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f109cab7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f1095ffb640 1 -- 172.21.15.62:0/2824544261 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f10840ceed0 con 0x7f10981027a0 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f109cab7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42912/0 (socket says 172.21.15.62:42912) 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.893+0000 7f109cab7640 1 -- 172.21.15.62:0/3637804102 learned_addr learned my addr 172.21.15.62:0/3637804102 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f1097fff640 1 --2- 172.21.15.62:0/3637804102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 0x563d5eeb0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f109d2b8640 1 --2- 172.21.15.62:0/3637804102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eebb370 0x563d5eebd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f109cab7640 1 -- 172.21.15.62:0/3637804102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 msgr2=0x563d5eeb0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f109cab7640 1 --2- 172.21.15.62:0/3637804102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 0x563d5eeb0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f109cab7640 1 -- 172.21.15.62:0/3637804102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eebb370 msgr2=0x563d5eebd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f109cab7640 1 --2- 172.21.15.62:0/3637804102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eebb370 0x563d5eebd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f109cab7640 1 -- 172.21.15.62:0/3637804102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d5eda23a0 con 0x563d5eeb1b40 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f109d2b8640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10880a2cf0 0x7f10880a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f10900070b0 tx=0x7f109000c900 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f109cab7640 1 --2- 172.21.15.62:0/3637804102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb0090 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f108c009900 tx=0x7f108c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a434809655a3d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f1082ffd640 1 -- 172.21.15.62:0/3637804102 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c040070 con 0x563d5eeb1b40 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f1082ffd640 1 -- 172.21.15.62:0/3637804102 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108c005020 con 0x563d5eeb1b40 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.894+0000 7f1082ffd640 1 -- 172.21.15.62:0/3637804102 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c035040 con 0x563d5eeb1b40 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.895+0000 7f10a10f4740 1 -- 172.21.15.62:0/3637804102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 msgr2=0x563d5eeb0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.895+0000 7f10a10f4740 1 --2- 172.21.15.62:0/3637804102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb0090 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f108c009900 tx=0x7f108c031630 comp rx=0 tx=0).stop 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.895+0000 7f10a10f4740 1 -- 172.21.15.62:0/3637804102 shutdown_connections 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.895+0000 7f10a10f4740 1 --2- 172.21.15.62:0/3637804102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eebb370 0x563d5eebd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.895+0000 7f10a10f4740 1 --2- 172.21.15.62:0/3637804102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 0x563d5eeb0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.895+0000 7f10a10f4740 1 --2- 172.21.15.62:0/3637804102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb0090 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.895+0000 7f10a10f4740 1 -- 172.21.15.62:0/3637804102 >> 172.21.15.62:0/3637804102 conn(0x563d5ee79310 msgr2=0x563d5eeacc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.895+0000 7f10a10f4740 1 -- 172.21.15.62:0/3637804102 shutdown_connections 2024-09-12T00:02:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.895+0000 7f10a10f4740 1 -- 172.21.15.62:0/3637804102 wait complete. 2024-09-12T00:02:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.895+0000 7f10a10f4740 1 -- start start 2024-09-12T00:02:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f10a10f4740 1 Processor -- start 2024-09-12T00:02:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb05d0 0x563d5eebacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eebb370 0x563d5eeb84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f10a10f4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563d5eebe010 con 0x563d5eeb1b40 2024-09-12T00:02:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f10a10f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563d5edd1060 con 0x563d5eeb05d0 2024-09-12T00:02:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f10a10f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d5eebe190 con 0x563d5eebb370 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f10a10f4740 10 client.0 _ll_register_callbacks cb 0x563d5eb248c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f1097fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f1097fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42924/0 (socket says 172.21.15.62:42924) 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f1097fff640 1 -- 172.21.15.62:0/4285623620 learned_addr learned my addr 172.21.15.62:0/4285623620 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f109cab7640 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb05d0 0x563d5eebacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f109d2b8640 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eebb370 0x563d5eeb84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f1097fff640 1 -- 172.21.15.62:0/4285623620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eebb370 msgr2=0x563d5eeb84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f1097fff640 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eebb370 0x563d5eeb84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f1097fff640 1 -- 172.21.15.62:0/4285623620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb05d0 msgr2=0x563d5eebacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f1097fff640 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb05d0 0x563d5eebacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f1097fff640 1 -- 172.21.15.62:0/4285623620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5eb873c0 con 0x563d5eeb1b40 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.896+0000 7f109d2b8640 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eebb370 0x563d5eeb84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f1097fff640 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7f80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f10840cc680 tx=0x7f10840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26cdad30774a253 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f1077fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f1077fff640 1 -- 172.21.15.62:0/4285623620 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084020040 con 0x563d5eeb1b40 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f1077fff640 1 -- 172.21.15.62:0/4285623620 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f1084009cc0 con 0x563d5eeb1b40 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f10a10f4740 1 -- 172.21.15.62:0/4285623620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f108c005020 con 0x563d5eeb1b40 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f1077fff640 1 -- 172.21.15.62:0/4285623620 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f108409f7a0 con 0x563d5eeb1b40 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f10a10f4740 10 client.17310 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f10a10f4740 1 -- 172.21.15.62:0/4285623620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563d5eec9a80 con 0x563d5eeb1b40 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f1077fff640 1 -- 172.21.15.62:0/4285623620 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108c005020 con 0x563d5eeb1b40 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f1077fff640 1 -- 172.21.15.62:0/4285623620 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f1084097a60 con 0x563d5eeb1b40 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f10a10f4740 10 client.17310 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f10a10f4740 10 client.17310 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f10a10f4740 20 client.17310 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f10a10f4740 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563d5eecaee0 0x563d5eecd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f10a10f4740 4 client.17310 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.897+0000 7f10a10f4740 1 -- 172.21.15.62:0/4285623620 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d5ee23860 con 0x563d5eecaee0 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.898+0000 7f109cab7640 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563d5eecaee0 0x563d5eecd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.898+0000 7f1077fff640 10 client.17310 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.898+0000 7f109cab7640 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563d5eecaee0 0x563d5eecd310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84aed27a503f98e8 server_cookie=1cc476cf5996e5a4 in_seq=0 out_seq=0 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f1077fff640 1 -- 172.21.15.62:0/4285623620 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d5ee23860 con 0x563d5eecaee0 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f1077fff640 10 client.17310 handle_command_reply: tid=0 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 client.17310 shutdown 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 -- 172.21.15.62:0/4285623620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 msgr2=0x563d5eeb7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7f80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f10840cc680 tx=0x7f10840046c0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 -- 172.21.15.62:0/4285623620 shutdown_connections 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563d5eecaee0 0x563d5eecd310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eebb370 0x563d5eeb84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7f80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f1077fff640 0 client.17310 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:02:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 --2- 172.21.15.62:0/4285623620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb05d0 0x563d5eebacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 -- 172.21.15.62:0/4285623620 >> 172.21.15.62:0/4285623620 conn(0x563d5ee79310 msgr2=0x563d5eeb2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 -- 172.21.15.62:0/4285623620 shutdown_connections 2024-09-12T00:02:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 1 -- 172.21.15.62:0/4285623620 wait complete. 2024-09-12T00:02:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.899+0000 7f10a10f4740 20 client.17310 trim_cache size 0 max 16384 2024-09-12T00:02:43.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.912+0000 7f10a10f4740 1 Processor -- start 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.912+0000 7f10a10f4740 1 -- start start 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 0x563d5eec72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eebb370 0x563d5eeb3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f10a10f4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b21d0 con 0x563d5eebb370 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f10a10f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10981afa70 con 0x563d5eeb1b40 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f10a10f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10981ad8e0 con 0x563d5eeb05d0 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109d2b8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eebb370 0x563d5eeb3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109d2b8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eebb370 0x563d5eeb3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42940/0 (socket says 172.21.15.62:42940) 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109d2b8640 1 -- 172.21.15.62:0/2425541504 learned_addr learned my addr 172.21.15.62:0/2425541504 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f1097fff640 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109cab7640 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 0x563d5eec72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109d2b8640 1 -- 172.21.15.62:0/2425541504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 msgr2=0x563d5eec72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109d2b8640 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 0x563d5eec72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109d2b8640 1 -- 172.21.15.62:0/2425541504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb1b40 msgr2=0x563d5eeb7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109d2b8640 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109d2b8640 1 -- 172.21.15.62:0/2425541504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d5eead960 con 0x563d5eebb370 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109cab7640 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 0x563d5eec72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f1097fff640 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.913+0000 7f109d2b8640 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eebb370 0x563d5eeb3120 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f10900075e0 tx=0x7f1090009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0222b0ee60f866 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10827fc640 1 -- 172.21.15.62:0/2425541504 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090029070 con 0x563d5eebb370 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10827fc640 1 -- 172.21.15.62:0/2425541504 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1090024910 con 0x563d5eebb370 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10827fc640 1 -- 172.21.15.62:0/2425541504 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109001a850 con 0x563d5eebb370 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10a10f4740 1 -- 172.21.15.62:0/2425541504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eebb370 msgr2=0x563d5eeb3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10a10f4740 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eebb370 0x563d5eeb3120 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f10900075e0 tx=0x7f1090009470 comp rx=0 tx=0).stop 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10a10f4740 1 -- 172.21.15.62:0/2425541504 shutdown_connections 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10a10f4740 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eebb370 0x563d5eeb3120 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10a10f4740 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eeb1b40 0x563d5eeb7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10a10f4740 1 --2- 172.21.15.62:0/2425541504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb05d0 0x563d5eec72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10a10f4740 1 -- 172.21.15.62:0/2425541504 >> 172.21.15.62:0/2425541504 conn(0x563d5ee79310 msgr2=0x563d5eea7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10a10f4740 1 -- 172.21.15.62:0/2425541504 shutdown_connections 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.914+0000 7f10a10f4740 1 -- 172.21.15.62:0/2425541504 wait complete. 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f10a10f4740 1 -- start start 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f10a10f4740 1 Processor -- start 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb05d0 0x563d5eec3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb1b40 0x563d5eebef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f10a10f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eebb370 0x563d5eebf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f10a10f4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f10981adf30 con 0x563d5eeb05d0 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f10a10f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10981af3a0 con 0x563d5eebb370 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f10a10f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10981ae600 con 0x563d5eeb1b40 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f10a10f4740 10 client.0 _ll_register_callbacks cb 0x7f10840056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f109cab7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb05d0 0x563d5eec3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f109cab7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb05d0 0x563d5eec3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42950/0 (socket says 172.21.15.62:42950) 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f109cab7640 1 -- 172.21.15.62:0/249397659 learned_addr learned my addr 172.21.15.62:0/249397659 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f109d2b8640 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eebb370 0x563d5eebf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.915+0000 7f1097fff640 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb1b40 0x563d5eebef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f109cab7640 1 -- 172.21.15.62:0/249397659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb1b40 msgr2=0x563d5eebef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f109cab7640 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb1b40 0x563d5eebef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f109cab7640 1 -- 172.21.15.62:0/249397659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eebb370 msgr2=0x563d5eebf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f109cab7640 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eebb370 0x563d5eebf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f109cab7640 1 -- 172.21.15.62:0/249397659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5eebfca0 con 0x563d5eeb05d0 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f109cab7640 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb05d0 0x563d5eec3be0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f108c005940 tx=0x7f108c031ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec9d1df8cbe671d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f1077fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f1077fff640 1 -- 172.21.15.62:0/249397659 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c040050 con 0x563d5eeb05d0 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f1077fff640 1 -- 172.21.15.62:0/249397659 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f108c004450 con 0x563d5eeb05d0 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f10a10f4740 1 -- 172.21.15.62:0/249397659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1090024910 con 0x563d5eeb05d0 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f1077fff640 1 -- 172.21.15.62:0/249397659 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f108c05e050 con 0x563d5eeb05d0 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f10a10f4740 10 client.17316 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f10a10f4740 1 -- 172.21.15.62:0/249397659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563d5ef35bb0 con 0x563d5eeb05d0 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.916+0000 7f1077fff640 1 -- 172.21.15.62:0/249397659 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1090024910 con 0x563d5eeb05d0 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f1077fff640 1 -- 172.21.15.62:0/249397659 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f108c0048c0 con 0x563d5eeb05d0 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f10a10f4740 10 client.17316 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f10a10f4740 10 client.17316 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f10a10f4740 20 client.17316 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:43.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f10a10f4740 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563d5ef35f10 0x563d5ef38300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f10a10f4740 4 client.17316 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f10a10f4740 1 -- 172.21.15.62:0/249397659 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d5ee1dd70 con 0x563d5ef35f10 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f1097fff640 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563d5ef35f10 0x563d5ef38300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f1097fff640 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563d5ef35f10 0x563d5ef38300 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8689fd5f808d71a7 server_cookie=e6c9171f9cf98c78 in_seq=0 out_seq=0 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f1077fff640 10 client.17316 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f1077fff640 1 -- 172.21.15.62:0/249397659 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d5ee1dd70 con 0x563d5ef35f10 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.917+0000 7f1077fff640 10 client.17316 handle_command_reply: tid=0 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 client.17316 shutdown 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 -- 172.21.15.62:0/249397659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb05d0 msgr2=0x563d5eec3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb05d0 0x563d5eec3be0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f108c005940 tx=0x7f108c031ae0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 -- 172.21.15.62:0/249397659 shutdown_connections 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x563d5ef35f10 0x563d5ef38300 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d5eebb370 0x563d5eebf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f1077fff640 0 client.17316 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:02:43.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d5eeb1b40 0x563d5eebef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 --2- 172.21.15.62:0/249397659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d5eeb05d0 0x563d5eec3be0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 -- 172.21.15.62:0/249397659 >> 172.21.15.62:0/249397659 conn(0x563d5ee79310 msgr2=0x563d5eeb97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:43.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 -- 172.21.15.62:0/249397659 shutdown_connections 2024-09-12T00:02:43.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 1 -- 172.21.15.62:0/249397659 wait complete. 2024-09-12T00:02:43.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f10a10f4740 20 client.17316 trim_cache size 0 max 16384 2024-09-12T00:02:43.925 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:02:43.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f109ed1b640 1 -- 172.21.15.62:0/2824544261 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10880a2cf0 msgr2=0x7f10880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f109ed1b640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10880a2cf0 0x7f10880a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f10900070b0 tx=0x7f109000c900 comp rx=0 tx=0).stop 2024-09-12T00:02:43.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f109ed1b640 1 -- 172.21.15.62:0/2824544261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10981027a0 msgr2=0x7f1098074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:43.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.918+0000 7f109ed1b640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10981027a0 0x7f1098074a30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f108400d770 tx=0x7f108400dc30 comp rx=0 tx=0).stop 2024-09-12T00:02:43.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.919+0000 7f109ed1b640 1 -- 172.21.15.62:0/2824544261 shutdown_connections 2024-09-12T00:02:43.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.919+0000 7f109ed1b640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f10880a2cf0 0x7f10880a51b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.919+0000 7f109ed1b640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1098106f20 0x7f1098074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.919+0000 7f109ed1b640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10981027a0 0x7f1098074a30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.919+0000 7f109ed1b640 1 --2- 172.21.15.62:0/2824544261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1098101df0 0x7f10980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:43.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.919+0000 7f109ed1b640 1 -- 172.21.15.62:0/2824544261 >> 172.21.15.62:0/2824544261 conn(0x7f10980fd8b0 msgr2=0x7f10980feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:43.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.919+0000 7f109ed1b640 1 -- 172.21.15.62:0/2824544261 shutdown_connections 2024-09-12T00:02:43.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:43.919+0000 7f109ed1b640 1 -- 172.21.15.62:0/2824544261 wait complete. 2024-09-12T00:02:44.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:43 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/310615734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:44.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:43 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/310615734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:44.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:43 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/310615734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:02:44.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:02:44.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:44 smithi062 ceph-mon[30958]: pgmap v330: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-12T00:02:45.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:44 smithi120 ceph-mon[35182]: pgmap v330: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-12T00:02:45.141 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 93f82611 --release --await' 2024-09-12T00:02:45.142 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 93f82611 --release --await 2024-09-12T00:02:45.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:44 smithi083 ceph-mon[34805]: pgmap v330: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-12T00:02:45.236 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:45.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.726+0000 7f34a5760640 1 -- 172.21.15.62:0/3958855278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a0102570 msgr2=0x7f34a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:45.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.726+0000 7f34a5760640 1 --2- 172.21.15.62:0/3958855278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a0102570 0x7f34a01029b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3494009900 tx=0x7f349402a480 comp rx=0 tx=0).stop 2024-09-12T00:02:45.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.726+0000 7f34a5760640 1 -- 172.21.15.62:0/3958855278 shutdown_connections 2024-09-12T00:02:45.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.726+0000 7f34a5760640 1 --2- 172.21.15.62:0/3958855278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0102ef0 0x7f34a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.726+0000 7f34a5760640 1 --2- 172.21.15.62:0/3958855278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a0102570 0x7f34a01029b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.726+0000 7f34a5760640 1 --2- 172.21.15.62:0/3958855278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0100e20 0x7f34a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.726+0000 7f34a5760640 1 -- 172.21.15.62:0/3958855278 >> 172.21.15.62:0/3958855278 conn(0x7f34a00fc7d0 msgr2=0x7f34a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.726+0000 7f34a5760640 1 -- 172.21.15.62:0/3958855278 shutdown_connections 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.726+0000 7f34a5760640 1 -- 172.21.15.62:0/3958855278 wait complete. 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f34a5760640 1 Processor -- start 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f34a5760640 1 -- start start 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f34a5760640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0100e20 0x7f34a0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f34a5760640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0102570 0x7f34a0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f34a5760640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a0102ef0 0x7f34a019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f34a5760640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0110f80 con 0x7f34a0102ef0 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f34a5760640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0110e00 con 0x7f34a0102570 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f34a5760640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0111100 con 0x7f34a0100e20 2024-09-12T00:02:45.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f349f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a0102ef0 0x7f34a019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f349f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a0102ef0 0x7f34a019ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42976/0 (socket says 172.21.15.62:42976) 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f349f7fe640 1 -- 172.21.15.62:0/3317946623 learned_addr learned my addr 172.21.15.62:0/3317946623 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.727+0000 7f349e7fc640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0102570 0x7f34a0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f349effd640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0100e20 0x7f34a0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f349f7fe640 1 -- 172.21.15.62:0/3317946623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0100e20 msgr2=0x7f34a0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f349f7fe640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0100e20 0x7f34a0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f349f7fe640 1 -- 172.21.15.62:0/3317946623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0102570 msgr2=0x7f34a0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f349f7fe640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0102570 0x7f34a0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f349f7fe640 1 -- 172.21.15.62:0/3317946623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a019d510 con 0x7f34a0102ef0 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f349f7fe640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a0102ef0 0x7f34a019ce30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f349800cad0 tx=0x7f349800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f3487fff640 1 -- 172.21.15.62:0/3317946623 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349800ae20 con 0x7f34a0102ef0 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f3487fff640 1 -- 172.21.15.62:0/3317946623 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3498005020 con 0x7f34a0102ef0 2024-09-12T00:02:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f34a5760640 1 -- 172.21.15.62:0/3317946623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a019d800 con 0x7f34a0102ef0 2024-09-12T00:02:45.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f3487fff640 1 -- 172.21.15.62:0/3317946623 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349800f6f0 con 0x7f34a0102ef0 2024-09-12T00:02:45.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.728+0000 7f34a5760640 1 -- 172.21.15.62:0/3317946623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01a50e0 con 0x7f34a0102ef0 2024-09-12T00:02:45.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f3487fff640 1 -- 172.21.15.62:0/3317946623 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f34980051d0 con 0x7f34a0102ef0 2024-09-12T00:02:45.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f34a7b39740 1 Processor -- start 2024-09-12T00:02:45.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f34a7b39740 1 -- start start 2024-09-12T00:02:45.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c11742f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11742f5d0 0x55c11742fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11743a370 0x55c11743c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f34a7b39740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c1173952f0 con 0x55c11743a370 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f34a7b39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c11733ccf0 con 0x55c117430b40 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f34a7b39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c1173912b0 con 0x55c11742f5d0 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f349e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11743a370 0x55c11743c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f349e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11743a370 0x55c11743c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42990/0 (socket says 172.21.15.62:42990) 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f349e7fc640 1 -- 172.21.15.62:0/1051540314 learned_addr learned my addr 172.21.15.62:0/1051540314 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f3487fff640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f34880a2cf0 0x7f34880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f349e7fc640 1 --2- 172.21.15.62:0/1051540314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c11742f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.731+0000 7f349effd640 1 --2- 172.21.15.62:0/1051540314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11742f5d0 0x55c11742fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f349e7fc640 1 -- 172.21.15.62:0/1051540314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11742f5d0 msgr2=0x55c11742fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f349e7fc640 1 --2- 172.21.15.62:0/1051540314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11742f5d0 0x55c11742fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f349e7fc640 1 -- 172.21.15.62:0/1051540314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 msgr2=0x55c11742f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:45.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f349e7fc640 1 --2- 172.21.15.62:0/1051540314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c11742f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f349e7fc640 1 -- 172.21.15.62:0/1051540314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1173213a0 con 0x55c11743a370 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f3487fff640 1 -- 172.21.15.62:0/3317946623 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f34980cf790 con 0x7f34a0102ef0 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f349f7fe640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f34880a2cf0 0x7f34880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f349e7fc640 1 --2- 172.21.15.62:0/1051540314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11743a370 0x55c11743c760 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f34940314e0 tx=0x7f3494000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dda95d73bdd598a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f34857fa640 1 -- 172.21.15.62:0/1051540314 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494035040 con 0x55c11743a370 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f34857fa640 1 -- 172.21.15.62:0/1051540314 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3494003cd0 con 0x55c11743a370 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.732+0000 7f34857fa640 1 -- 172.21.15.62:0/1051540314 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349403d480 con 0x55c11743a370 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 -- 172.21.15.62:0/1051540314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11743a370 msgr2=0x55c11743c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f349f7fe640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f34880a2cf0 0x7f34880a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f349809bbe0 tx=0x7f3498005b40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 --2- 172.21.15.62:0/1051540314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11743a370 0x55c11743c760 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f34940314e0 tx=0x7f3494000bc0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 -- 172.21.15.62:0/1051540314 shutdown_connections 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 --2- 172.21.15.62:0/1051540314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11743a370 0x55c11743c760 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 --2- 172.21.15.62:0/1051540314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11742f5d0 0x55c11742fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 --2- 172.21.15.62:0/1051540314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c11742f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 -- 172.21.15.62:0/1051540314 >> 172.21.15.62:0/1051540314 conn(0x55c1173f8310 msgr2=0x55c11742bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 -- 172.21.15.62:0/1051540314 shutdown_connections 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 -- 172.21.15.62:0/1051540314 wait complete. 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 -- start start 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 Processor -- start 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117439d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.733+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c117437020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117437610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f34a7b39740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c11743d010 con 0x55c11742f5d0 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f34a7b39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c117350060 con 0x55c117430b40 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f34a7b39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c11743d190 con 0x55c11743a370 2024-09-12T00:02:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f34a7b39740 10 client.0 _ll_register_callbacks cb 0x55c1170a38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117439d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117439d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43006/0 (socket says 172.21.15.62:43006) 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349e7fc640 1 -- 172.21.15.62:0/536496746 learned_addr learned my addr 172.21.15.62:0/536496746 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349effd640 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c117437020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349e7fc640 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117437610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349e7fc640 1 -- 172.21.15.62:0/536496746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 msgr2=0x55c117437610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349e7fc640 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117437610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349e7fc640 1 -- 172.21.15.62:0/536496746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 msgr2=0x55c117437020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349e7fc640 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c117437020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349e7fc640 1 -- 172.21.15.62:0/536496746 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1171063c0 con 0x55c11742f5d0 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f349e7fc640 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117439d70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3494004260 tx=0x7f3494031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcd53d96f465fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f3482ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.734+0000 7f3482ff5640 1 -- 172.21.15.62:0/536496746 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494035070 con 0x55c11742f5d0 2024-09-12T00:02:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f3482ff5640 1 -- 172.21.15.62:0/536496746 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f34940035f0 con 0x55c11742f5d0 2024-09-12T00:02:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f34a7b39740 1 -- 172.21.15.62:0/536496746 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3494003cd0 con 0x55c11742f5d0 2024-09-12T00:02:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f3482ff5640 1 -- 172.21.15.62:0/536496746 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f34940037f0 con 0x55c11742f5d0 2024-09-12T00:02:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f34a7b39740 10 client.17328 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f34a7b39740 1 -- 172.21.15.62:0/536496746 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c11743d010 con 0x55c11742f5d0 2024-09-12T00:02:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f3482ff5640 1 -- 172.21.15.62:0/536496746 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3494003cd0 con 0x55c11742f5d0 2024-09-12T00:02:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f3482ff5640 1 -- 172.21.15.62:0/536496746 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f3494004850 con 0x55c11742f5d0 2024-09-12T00:02:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f34a7b39740 10 client.17328 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f34a7b39740 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f34a7b39740 20 client.17328 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f34a7b39740 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c11744e040 0x55c117450430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f34a7b39740 4 client.17328 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.735+0000 7f34a7b39740 1 -- 172.21.15.62:0/536496746 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1173a2860 con 0x55c11744e040 2024-09-12T00:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.736+0000 7f349effd640 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c11744e040 0x55c117450430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.736+0000 7f349effd640 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c11744e040 0x55c117450430 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76979355d8375ac0 server_cookie=9f99e9d2e526ff3f in_seq=0 out_seq=0 2024-09-12T00:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.736+0000 7f3482ff5640 10 client.17328 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f3482ff5640 1 -- 172.21.15.62:0/536496746 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1173a2860 con 0x55c11744e040 2024-09-12T00:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f3482ff5640 10 client.17328 handle_command_reply: tid=0 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 client.17328 shutdown 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 -- 172.21.15.62:0/536496746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 msgr2=0x55c117439d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117439d70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3494004260 tx=0x7f3494031a90 comp rx=0 tx=0).stop 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 -- 172.21.15.62:0/536496746 shutdown_connections 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c11744e040 0x55c117450430 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117437610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c117437020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 --2- 172.21.15.62:0/536496746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117439d70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 -- 172.21.15.62:0/536496746 >> 172.21.15.62:0/536496746 conn(0x55c1173f8310 msgr2=0x55c117431960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f3482ff5640 0 client.17328 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 -- 172.21.15.62:0/536496746 shutdown_connections 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 1 -- 172.21.15.62:0/536496746 wait complete. 2024-09-12T00:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.737+0000 7f34a7b39740 20 client.17328 trim_cache size 0 max 16384 2024-09-12T00:02:45.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.752+0000 7f34a7b39740 1 Processor -- start 2024-09-12T00:02:45.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.752+0000 7f34a7b39740 1 -- start start 2024-09-12T00:02:45.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117432120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c117432660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117455a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f34a7b39740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01adfa0 con 0x55c11742f5d0 2024-09-12T00:02:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f34a7b39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01ab840 con 0x55c117430b40 2024-09-12T00:02:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f34a7b39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01a96b0 con 0x55c11743a370 2024-09-12T00:02:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117432120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117432120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43008/0 (socket says 172.21.15.62:43008) 2024-09-12T00:02:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349e7fc640 1 -- 172.21.15.62:0/229149961 learned_addr learned my addr 172.21.15.62:0/229149961 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:45.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349effd640 1 --2- 172.21.15.62:0/229149961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c117432660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349e7fc640 1 -- 172.21.15.62:0/229149961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 msgr2=0x55c117455a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349e7fc640 1 --2- 172.21.15.62:0/229149961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117455a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349e7fc640 1 -- 172.21.15.62:0/229149961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 msgr2=0x55c117432660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349e7fc640 1 --2- 172.21.15.62:0/229149961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c117432660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349e7fc640 1 -- 172.21.15.62:0/229149961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c117439990 con 0x55c11742f5d0 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349effd640 1 --2- 172.21.15.62:0/229149961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c117432660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.753+0000 7f349e7fc640 1 --2- 172.21.15.62:0/229149961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117432120 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f34940059d0 tx=0x7f349402ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67327be0aa37571 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.754+0000 7f3484ff9640 1 -- 172.21.15.62:0/229149961 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494035070 con 0x55c11742f5d0 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.754+0000 7f3484ff9640 1 -- 172.21.15.62:0/229149961 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f349403d480 con 0x55c11742f5d0 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.754+0000 7f3484ff9640 1 -- 172.21.15.62:0/229149961 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494003990 con 0x55c11742f5d0 2024-09-12T00:02:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.754+0000 7f34a7b39740 1 -- 172.21.15.62:0/229149961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 msgr2=0x55c117432120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.754+0000 7f34a7b39740 1 --2- 172.21.15.62:0/229149961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117432120 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f34940059d0 tx=0x7f349402ac00 comp rx=0 tx=0).stop 2024-09-12T00:02:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.754+0000 7f34a7b39740 1 -- 172.21.15.62:0/229149961 shutdown_connections 2024-09-12T00:02:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.754+0000 7f34a7b39740 1 --2- 172.21.15.62:0/229149961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117455a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.754+0000 7f34a7b39740 1 --2- 172.21.15.62:0/229149961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117430b40 0x55c117432660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.754+0000 7f34a7b39740 1 --2- 172.21.15.62:0/229149961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c11742f5d0 0x55c117432120 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.754+0000 7f34a7b39740 1 -- 172.21.15.62:0/229149961 >> 172.21.15.62:0/229149961 conn(0x55c1173f8310 msgr2=0x55c117436130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.755+0000 7f34a7b39740 1 -- 172.21.15.62:0/229149961 shutdown_connections 2024-09-12T00:02:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.755+0000 7f34a7b39740 1 -- 172.21.15.62:0/229149961 wait complete. 2024-09-12T00:02:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.755+0000 7f34a7b39740 1 -- start start 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.755+0000 7f34a7b39740 1 Processor -- start 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c117430b40 0x55c117442bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117433d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f34a7b39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117434270 0x55c1174346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f34a7b39740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01a9d00 con 0x55c117430b40 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f34a7b39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01ab170 con 0x55c117434270 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f34a7b39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01aa3d0 con 0x55c11743a370 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f34a7b39740 10 client.0 _ll_register_callbacks cb 0x7f34940043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f349e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c117430b40 0x55c117442bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f349e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c117430b40 0x55c117442bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43024/0 (socket says 172.21.15.62:43024) 2024-09-12T00:02:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f349e7fc640 1 -- 172.21.15.62:0/3294046187 learned_addr learned my addr 172.21.15.62:0/3294046187 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f349effd640 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117433d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f349e7fc640 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117434270 0x55c1174346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f349e7fc640 1 -- 172.21.15.62:0/3294046187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 msgr2=0x55c117433d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f349e7fc640 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117433d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f349e7fc640 1 -- 172.21.15.62:0/3294046187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117434270 msgr2=0x55c1174346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f349e7fc640 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117434270 0x55c1174346d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.756+0000 7f349e7fc640 1 -- 172.21.15.62:0/3294046187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1174b7c30 con 0x55c117430b40 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f349e7fc640 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c117430b40 0x55c117442bd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3494005ec0 tx=0x7f34940035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cde13b47c497c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f3482ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f3482ff5640 1 -- 172.21.15.62:0/3294046187 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494035070 con 0x55c117430b40 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f3482ff5640 1 -- 172.21.15.62:0/3294046187 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f349403dbc0 con 0x55c117430b40 2024-09-12T00:02:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f34a7b39740 1 -- 172.21.15.62:0/3294046187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f349403d480 con 0x55c117430b40 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f3482ff5640 1 -- 172.21.15.62:0/3294046187 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f349406a050 con 0x55c117430b40 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f34a7b39740 10 client.17334 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f34a7b39740 1 -- 172.21.15.62:0/3294046187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c1174b8ea0 con 0x55c117430b40 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f3482ff5640 1 -- 172.21.15.62:0/3294046187 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f349403d480 con 0x55c117430b40 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f3482ff5640 1 -- 172.21.15.62:0/3294046187 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f3494004ed0 con 0x55c117430b40 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f34a7b39740 10 client.17334 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f34a7b39740 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f34a7b39740 20 client.17334 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f34a7b39740 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c1174b93d0 0x55c1174bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f34a7b39740 4 client.17334 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.757+0000 7f34a7b39740 1 -- 172.21.15.62:0/3294046187 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c11739cd70 con 0x55c1174b93d0 2024-09-12T00:02:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.758+0000 7f349effd640 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c1174b93d0 0x55c1174bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.758+0000 7f349effd640 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c1174b93d0 0x55c1174bb7c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983ed9b9ab97857a server_cookie=438149c3ac1fa8da in_seq=0 out_seq=0 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.758+0000 7f3482ff5640 10 client.17334 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.758+0000 7f3482ff5640 1 -- 172.21.15.62:0/3294046187 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c11739cd70 con 0x55c1174b93d0 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.758+0000 7f3482ff5640 10 client.17334 handle_command_reply: tid=0 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.758+0000 7f34a7b39740 1 client.17334 shutdown 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.758+0000 7f34a7b39740 1 -- 172.21.15.62:0/3294046187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c117430b40 msgr2=0x55c117442bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.758+0000 7f34a7b39740 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c117430b40 0x55c117442bd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3494005ec0 tx=0x7f34940035f0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.759+0000 7f34a7b39740 1 -- 172.21.15.62:0/3294046187 shutdown_connections 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.759+0000 7f34a7b39740 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c1174b93d0 0x55c1174bb7c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.759+0000 7f34a7b39740 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c117434270 0x55c1174346d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.759+0000 7f34a7b39740 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c11743a370 0x55c117433d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.759+0000 7f34a7b39740 1 --2- 172.21.15.62:0/3294046187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c117430b40 0x55c117442bd0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.759+0000 7f3482ff5640 0 client.17334 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.759+0000 7f34a7b39740 1 -- 172.21.15.62:0/3294046187 >> 172.21.15.62:0/3294046187 conn(0x55c1173f8310 msgr2=0x55c117438880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.759+0000 7f34a7b39740 1 -- 172.21.15.62:0/3294046187 shutdown_connections 2024-09-12T00:02:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.759+0000 7f34a7b39740 1 -- 172.21.15.62:0/3294046187 wait complete. 2024-09-12T00:02:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.759+0000 7f34a7b39740 20 client.17334 trim_cache size 0 max 16384 2024-09-12T00:02:45.764 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:02:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 -- 172.21.15.62:0/3317946623 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f34880a2cf0 msgr2=0x7f34880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f34880a2cf0 0x7f34880a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f349809bbe0 tx=0x7f3498005b40 comp rx=0 tx=0).stop 2024-09-12T00:02:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 -- 172.21.15.62:0/3317946623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a0102ef0 msgr2=0x7f34a019ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a0102ef0 0x7f34a019ce30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f349800cad0 tx=0x7f349800cf90 comp rx=0 tx=0).stop 2024-09-12T00:02:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 -- 172.21.15.62:0/3317946623 shutdown_connections 2024-09-12T00:02:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f34880a2cf0 0x7f34880a51b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a0102ef0 0x7f34a019ce30 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0102570 0x7f34a0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 --2- 172.21.15.62:0/3317946623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a0100e20 0x7f34a0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 -- 172.21.15.62:0/3317946623 >> 172.21.15.62:0/3317946623 conn(0x7f34a00fc7d0 msgr2=0x7f34a006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 -- 172.21.15.62:0/3317946623 shutdown_connections 2024-09-12T00:02:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:45.760+0000 7f34a5760640 1 -- 172.21.15.62:0/3317946623 wait complete. 2024-09-12T00:02:46.004 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:46.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.939+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3531873521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbff8102570 msgr2=0x7fbff81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:46.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.939+0000 7fbffd6b3640 1 --2- 172.21.15.62:0/3531873521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbff8102570 0x7fbff81029b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009900 tx=0x7fbfe002a480 comp rx=0 tx=0).stop 2024-09-12T00:02:46.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.939+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3531873521 shutdown_connections 2024-09-12T00:02:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.939+0000 7fbffd6b3640 1 --2- 172.21.15.62:0/3531873521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff8102ef0 0x7fbff810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.939+0000 7fbffd6b3640 1 --2- 172.21.15.62:0/3531873521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbff8102570 0x7fbff81029b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.939+0000 7fbffd6b3640 1 --2- 172.21.15.62:0/3531873521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8100e20 0x7fbff8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.940+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3531873521 >> 172.21.15.62:0/3531873521 conn(0x7fbff80fc7d0 msgr2=0x7fbff80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.940+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3531873521 shutdown_connections 2024-09-12T00:02:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.940+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3531873521 wait complete. 2024-09-12T00:02:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.940+0000 7fbffd6b3640 1 Processor -- start 2024-09-12T00:02:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.940+0000 7fbffd6b3640 1 -- start start 2024-09-12T00:02:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbffd6b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8100e20 0x7fbff81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbffd6b3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff8102570 0x7fbff8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbffd6b3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbff8102ef0 0x7fbff819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbffd6b3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8110f80 con 0x7fbff8102ef0 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbffd6b3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8110e00 con 0x7fbff8100e20 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbffd6b3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8111100 con 0x7fbff8102570 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbff8102ef0 0x7fbff819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbff8102ef0 0x7fbff819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43050/0 (socket says 172.21.15.62:43050) 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff77fe640 1 -- 172.21.15.62:0/3857009544 learned_addr learned my addr 172.21.15.62:0/3857009544 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8100e20 0x7fbff81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff67fc640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff8102570 0x7fbff8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff77fe640 1 -- 172.21.15.62:0/3857009544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff8102570 msgr2=0x7fbff8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff77fe640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff8102570 0x7fbff8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff77fe640 1 -- 172.21.15.62:0/3857009544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8100e20 msgr2=0x7fbff81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:46.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff77fe640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8100e20 0x7fbff81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff77fe640 1 -- 172.21.15.62:0/3857009544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff819d480 con 0x7fbff8102ef0 2024-09-12T00:02:46.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbff77fe640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbff8102ef0 0x7fbff819cda0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d750 tx=0x7fbff000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:46.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbfeffff640 1 -- 172.21.15.62:0/3857009544 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0017070 con 0x7fbff8102ef0 2024-09-12T00:02:46.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbfeffff640 1 -- 172.21.15.62:0/3857009544 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff0005500 con 0x7fbff8102ef0 2024-09-12T00:02:46.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.941+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3857009544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff819d770 con 0x7fbff8102ef0 2024-09-12T00:02:46.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.942+0000 7fbfeffff640 1 -- 172.21.15.62:0/3857009544 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0004d90 con 0x7fbff8102ef0 2024-09-12T00:02:46.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.942+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3857009544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff81a5050 con 0x7fbff8102ef0 2024-09-12T00:02:46.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.944+0000 7fbfeffff640 1 -- 172.21.15.62:0/3857009544 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbff0004f30 con 0x7fbff8102ef0 2024-09-12T00:02:46.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.944+0000 7fbfffa8c740 1 Processor -- start 2024-09-12T00:02:46.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.944+0000 7fbfffa8c740 1 -- start start 2024-09-12T00:02:46.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0da110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0dca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55743d030620 con 0x55743d0cb860 2024-09-12T00:02:46.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55743cfd7e50 con 0x55743d0da650 2024-09-12T00:02:46.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55743d02c5e0 con 0x55743d0caf40 2024-09-12T00:02:46.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0da110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0da110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43060/0 (socket says 172.21.15.62:43060) 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff6ffd640 1 -- 172.21.15.62:0/1869111099 learned_addr learned my addr 172.21.15.62:0/1869111099 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff67fc640 1 --2- 172.21.15.62:0/1869111099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff67fc640 1 --2- 172.21.15.62:0/1869111099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0dca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbfeffff640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 0x7fbfe40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff6ffd640 1 -- 172.21.15.62:0/1869111099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 msgr2=0x55743d0cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/1869111099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff6ffd640 1 -- 172.21.15.62:0/1869111099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 msgr2=0x55743d0dca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/1869111099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0dca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff6ffd640 1 -- 172.21.15.62:0/1869111099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55743cfc8d40 con 0x55743d0cb860 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff67fc640 1 --2- 172.21.15.62:0/1869111099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:46.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.945+0000 7fbff77fe640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 0x7fbfe40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.946+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/1869111099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0da110 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8007ed0 tx=0x7fbfe800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc7e656a6a8e36a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:46.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.946+0000 7fbfeffff640 1 -- 172.21.15.62:0/3857009544 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fbff00cee50 con 0x7fbff8102ef0 2024-09-12T00:02:46.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.946+0000 7fbfed7fa640 1 -- 172.21.15.62:0/1869111099 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8014070 con 0x55743d0cb860 2024-09-12T00:02:46.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.946+0000 7fbfed7fa640 1 -- 172.21.15.62:0/1869111099 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe8004d20 con 0x55743d0cb860 2024-09-12T00:02:46.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.946+0000 7fbfed7fa640 1 -- 172.21.15.62:0/1869111099 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe80054c0 con 0x55743d0cb860 2024-09-12T00:02:46.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.946+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1869111099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 msgr2=0x55743d0da110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:46.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/1869111099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0da110 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8007ed0 tx=0x7fbfe800e720 comp rx=0 tx=0).stop 2024-09-12T00:02:46.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbff77fe640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 0x7fbfe40a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbff009b300 tx=0x7fbff006e3e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1869111099 shutdown_connections 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/1869111099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0dca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/1869111099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0da110 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/1869111099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1869111099 >> 172.21.15.62:0/1869111099 conn(0x55743d0c75f0 msgr2=0x55743d0c79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1869111099 shutdown_connections 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1869111099 wait complete. 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 -- start start 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 Processor -- start 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0d75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0cb860 0x55743d0d7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0da650 0x55743d0d8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55743d0dd220 con 0x55743d0da650 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55743cfb8f10 con 0x55743d0cb860 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55743d0dd3a0 con 0x55743d0caf40 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.947+0000 7fbfffa8c740 10 client.0 _ll_register_callbacks cb 0x55743cfdcdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0da650 0x55743d0d8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0da650 0x55743d0d8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43074/0 (socket says 172.21.15.62:43074) 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff67fc640 1 -- 172.21.15.62:0/763083532 learned_addr learned my addr 172.21.15.62:0/763083532 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff67fc640 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0d75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0cb860 0x55743d0d7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff67fc640 1 -- 172.21.15.62:0/763083532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 msgr2=0x55743d0d75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:46.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff67fc640 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0d75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff67fc640 1 -- 172.21.15.62:0/763083532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0cb860 msgr2=0x55743d0d7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff67fc640 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0cb860 0x55743d0d7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff67fc640 1 -- 172.21.15.62:0/763083532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55743cd3f580 con 0x55743d0da650 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0cb860 0x55743d0d7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbff67fc640 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0da650 0x55743d0d8030 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00096c0 tx=0x7fbfe0005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d4fbdbb47426cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbfde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbfde7fc640 1 -- 172.21.15.62:0/763083532 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0035040 con 0x55743d0da650 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbfffa8c740 1 -- 172.21.15.62:0/763083532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfe8004d20 con 0x55743d0da650 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.948+0000 7fbfde7fc640 1 -- 172.21.15.62:0/763083532 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fbfe0003e40 con 0x55743d0da650 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfde7fc640 1 -- 172.21.15.62:0/763083532 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbfe0031750 con 0x55743d0da650 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfffa8c740 10 client.17346 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfffa8c740 1 -- 172.21.15.62:0/763083532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55743ce09960 con 0x55743d0da650 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfde7fc640 1 -- 172.21.15.62:0/763083532 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe8004d20 con 0x55743d0da650 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfde7fc640 1 -- 172.21.15.62:0/763083532 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fbfe003a040 con 0x55743d0da650 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfffa8c740 10 client.17346 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfffa8c740 10 client.17346 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfffa8c740 20 client.17346 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55743d0ea220 0x55743d0ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfffa8c740 4 client.17346 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbfffa8c740 1 -- 172.21.15.62:0/763083532 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55743d03db90 con 0x55743d0ea220 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.949+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55743d0ea220 0x55743d0ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.950+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55743d0ea220 0x55743d0ec610 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c63b4e402632ba53 server_cookie=95588fb8fd279b36 in_seq=0 out_seq=0 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.950+0000 7fbfde7fc640 10 client.17346 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:46.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfde7fc640 1 -- 172.21.15.62:0/763083532 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55743d03db90 con 0x55743d0ea220 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfde7fc640 10 client.17346 handle_command_reply: tid=0 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 client.17346 shutdown 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 -- 172.21.15.62:0/763083532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0da650 msgr2=0x55743d0d8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0da650 0x55743d0d8030 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00096c0 tx=0x7fbfe0005be0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 -- 172.21.15.62:0/763083532 shutdown_connections 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55743d0ea220 0x55743d0ec610 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0da650 0x55743d0d8030 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0cb860 0x55743d0d7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfde7fc640 0 client.17346 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/763083532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0d75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 -- 172.21.15.62:0/763083532 >> 172.21.15.62:0/763083532 conn(0x55743d0c75f0 msgr2=0x55743d0cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 -- 172.21.15.62:0/763083532 shutdown_connections 2024-09-12T00:02:46.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 1 -- 172.21.15.62:0/763083532 wait complete. 2024-09-12T00:02:46.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.951+0000 7fbfffa8c740 20 client.17346 trim_cache size 0 max 16384 2024-09-12T00:02:46.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:46 smithi062 ceph-mon[30958]: pgmap v331: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-09-12T00:02:46.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.971+0000 7fbfffa8c740 1 Processor -- start 2024-09-12T00:02:46.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.971+0000 7fbfffa8c740 1 -- start start 2024-09-12T00:02:46.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0caf40 0x55743d0f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0cb860 0x55743d0f2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbff81adf90 con 0x55743d0caf40 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbff81ab830 con 0x55743d0da650 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbff81a9680 con 0x55743d0cb860 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0caf40 0x55743d0f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0caf40 0x55743d0f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43088/0 (socket says 172.21.15.62:43088) 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff67fc640 1 -- 172.21.15.62:0/3322958877 learned_addr learned my addr 172.21.15.62:0/3322958877 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/3322958877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0cb860 0x55743d0f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff67fc640 1 -- 172.21.15.62:0/3322958877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0cb860 msgr2=0x55743d0f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff67fc640 1 --2- 172.21.15.62:0/3322958877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0cb860 0x55743d0f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff67fc640 1 -- 172.21.15.62:0/3322958877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 msgr2=0x55743d0d4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff67fc640 1 --2- 172.21.15.62:0/3322958877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff67fc640 1 -- 172.21.15.62:0/3322958877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55743d0cb690 con 0x55743d0caf40 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/3322958877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0cb860 0x55743d0f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.972+0000 7fbff67fc640 1 --2- 172.21.15.62:0/3322958877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0caf40 0x55743d0f1ec0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0005860 tx=0x7fbfe00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dea5306ce3a5e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.973+0000 7fbfecff9640 1 -- 172.21.15.62:0/3322958877 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0035020 con 0x55743d0caf40 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.973+0000 7fbfecff9640 1 -- 172.21.15.62:0/3322958877 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe0003e20 con 0x55743d0caf40 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.973+0000 7fbfecff9640 1 -- 172.21.15.62:0/3322958877 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0031b80 con 0x55743d0caf40 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 -- 172.21.15.62:0/3322958877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0caf40 msgr2=0x55743d0f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/3322958877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0caf40 0x55743d0f1ec0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0005860 tx=0x7fbfe00058b0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 -- 172.21.15.62:0/3322958877 shutdown_connections 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/3322958877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0d4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/3322958877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0cb860 0x55743d0f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/3322958877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0caf40 0x55743d0f1ec0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 -- 172.21.15.62:0/3322958877 >> 172.21.15.62:0/3322958877 conn(0x55743d0c75f0 msgr2=0x55743d0d5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 -- 172.21.15.62:0/3322958877 shutdown_connections 2024-09-12T00:02:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 -- 172.21.15.62:0/3322958877 wait complete. 2024-09-12T00:02:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 -- start start 2024-09-12T00:02:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 Processor -- start 2024-09-12T00:02:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0e2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0deb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbff81a9cf0 con 0x55743d0cb860 2024-09-12T00:02:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbff81ab160 con 0x55743d0da650 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbff81aa3c0 con 0x55743d0caf40 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.974+0000 7fbfffa8c740 10 client.0 _ll_register_callbacks cb 0x7fbfe0003ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0e2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43100/0 (socket says 172.21.15.62:43100) 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff6ffd640 1 -- 172.21.15.62:0/1160344195 learned_addr learned my addr 172.21.15.62:0/1160344195 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff67fc640 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0e2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff67fc640 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0deb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff6ffd640 1 -- 172.21.15.62:0/1160344195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 msgr2=0x55743d0e2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0e2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff6ffd640 1 -- 172.21.15.62:0/1160344195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 msgr2=0x55743d0deb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0deb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff6ffd640 1 -- 172.21.15.62:0/1160344195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55743d0df370 con 0x55743d0cb860 2024-09-12T00:02:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff67fc640 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbff6ffd640 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0e2c40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x55743d0cdc50 tx=0x7fbfe8007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c017ea07c3b6c55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbfde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbfde7fc640 1 -- 172.21.15.62:0/1160344195 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8014040 con 0x55743d0cb860 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbfde7fc640 1 -- 172.21.15.62:0/1160344195 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fbfe8003b70 con 0x55743d0cb860 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1160344195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfe0003e20 con 0x55743d0cb860 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbfde7fc640 1 -- 172.21.15.62:0/1160344195 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbfe8003d70 con 0x55743d0cb860 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbfffa8c740 10 client.17352 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.975+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1160344195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55743d1558b0 con 0x55743d0cb860 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbfde7fc640 1 -- 172.21.15.62:0/1160344195 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe0003e20 con 0x55743d0cb860 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbfde7fc640 1 -- 172.21.15.62:0/1160344195 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fbfe8004290 con 0x55743d0cb860 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbfffa8c740 10 client.17352 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbfffa8c740 10 client.17352 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:02:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbfffa8c740 20 client.17352 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55743d0ea220 0x55743d157d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbfffa8c740 4 client.17352 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "93f82611", "release": true, "await": true}] 2024-09-12T00:02:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1160344195 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "93f82611", "release": true, "await": true}) -- 0x55743d0380a0 con 0x55743d0ea220 2024-09-12T00:02:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbff67fc640 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55743d0ea220 0x55743d157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbff67fc640 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55743d0ea220 0x55743d157d90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f659b2591c34e78e server_cookie=f93cb6d89d21b785 in_seq=0 out_seq=0 2024-09-12T00:02:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:46.976+0000 7fbfde7fc640 10 client.17352 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.015+0000 7fbfde7fc640 1 -- 172.21.15.62:0/1160344195 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55743d0380a0 con 0x55743d0ea220 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.015+0000 7fbfde7fc640 10 client.17352 handle_command_reply: tid=0 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.015+0000 7fbfffa8c740 1 client.17352 shutdown 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.015+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1160344195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 msgr2=0x55743d0e2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.015+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0e2c40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x55743d0cdc50 tx=0x7fbfe8007b90 comp rx=0 tx=0).stop 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.016+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1160344195 shutdown_connections 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.016+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55743d0ea220 0x55743d157d90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.016+0000 7fbfde7fc640 0 client.17352 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.016+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55743d0da650 0x55743d0deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.016+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55743d0cb860 0x55743d0e2c40 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.016+0000 7fbfffa8c740 1 --2- 172.21.15.62:0/1160344195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55743d0caf40 0x55743d0e2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.016+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1160344195 >> 172.21.15.62:0/1160344195 conn(0x55743d0c75f0 msgr2=0x55743d0cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:47.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.016+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1160344195 shutdown_connections 2024-09-12T00:02:47.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.016+0000 7fbfffa8c740 1 -- 172.21.15.62:0/1160344195 wait complete. 2024-09-12T00:02:47.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.017+0000 7fbfffa8c740 20 client.17352 trim_cache size 0 max 16384 2024-09-12T00:02:47.017 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 33, 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 44, 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "93f82611": { 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 44, 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:02:47.018 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 142.6, 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:47.019 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:02:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.018+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3857009544 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 msgr2=0x7fbfe40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.018+0000 7fbffd6b3640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 0x7fbfe40a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbff009b300 tx=0x7fbff006e3e0 comp rx=0 tx=0).stop 2024-09-12T00:02:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.018+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3857009544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbff8102ef0 msgr2=0x7fbff819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.018+0000 7fbffd6b3640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbff8102ef0 0x7fbff819cda0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d750 tx=0x7fbff000dc10 comp rx=0 tx=0).stop 2024-09-12T00:02:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.018+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3857009544 shutdown_connections 2024-09-12T00:02:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.018+0000 7fbffd6b3640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbfe40a2cf0 0x7fbfe40a51b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.018+0000 7fbffd6b3640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbff8102ef0 0x7fbff819cda0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.018+0000 7fbffd6b3640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff8102570 0x7fbff8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.018+0000 7fbffd6b3640 1 --2- 172.21.15.62:0/3857009544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8100e20 0x7fbff81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.018+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3857009544 >> 172.21.15.62:0/3857009544 conn(0x7fbff80fc7d0 msgr2=0x7fbff806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.019+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3857009544 shutdown_connections 2024-09-12T00:02:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:47.019+0000 7fbffd6b3640 1 -- 172.21.15.62:0/3857009544 wait complete. 2024-09-12T00:02:47.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:46 smithi120 ceph-mon[35182]: pgmap v331: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-09-12T00:02:47.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:46 smithi083 ceph-mon[34805]: pgmap v331: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-09-12T00:02:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 605.0 s, 206.8 tps, lat 2326.891 ms stddev 7000.371 2024-09-12T00:02:47.460 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:02:47.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:02:47.854 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '93f82611', total seconds elapsed: 28.4 2024-09-12T00:02:47.923 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:48.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.493+0000 7fa74dffb640 1 -- 172.21.15.62:0/3240018185 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74802c040 con 0x7fa750102e00 2024-09-12T00:02:48.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.494+0000 7fa7554d6640 1 -- 172.21.15.62:0/3240018185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa750102e00 msgr2=0x7fa7501031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:48.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.494+0000 7fa7554d6640 1 --2- 172.21.15.62:0/3240018185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa750102e00 0x7fa7501031e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa748009ca0 tx=0x7fa74802a950 comp rx=0 tx=0).stop 2024-09-12T00:02:48.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.495+0000 7fa7554d6640 1 -- 172.21.15.62:0/3240018185 shutdown_connections 2024-09-12T00:02:48.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.495+0000 7fa7554d6640 1 --2- 172.21.15.62:0/3240018185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75010c1a0 0x7fa75010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.495+0000 7fa7554d6640 1 --2- 172.21.15.62:0/3240018185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750103720 0x7fa75010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.495+0000 7fa7554d6640 1 --2- 172.21.15.62:0/3240018185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa750102e00 0x7fa7501031e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.495+0000 7fa7554d6640 1 -- 172.21.15.62:0/3240018185 >> 172.21.15.62:0/3240018185 conn(0x7fa7500fc7a0 msgr2=0x7fa7500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:48.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.496+0000 7fa7554d6640 1 -- 172.21.15.62:0/3240018185 shutdown_connections 2024-09-12T00:02:48.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.496+0000 7fa7554d6640 1 -- 172.21.15.62:0/3240018185 wait complete. 2024-09-12T00:02:48.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa7554d6640 1 Processor -- start 2024-09-12T00:02:48.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa7554d6640 1 -- start start 2024-09-12T00:02:48.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa7554d6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750102e00 0x7fa750198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa7554d6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa750103720 0x7fa750198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa7554d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75010c1a0 0x7fa75019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa7554d6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa750111fc0 con 0x7fa750103720 2024-09-12T00:02:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa7554d6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa750111e40 con 0x7fa75010c1a0 2024-09-12T00:02:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa7554d6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa750112140 con 0x7fa750102e00 2024-09-12T00:02:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa74effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750102e00 0x7fa750198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa74effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750102e00 0x7fa750198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:41176/0 (socket says 172.21.15.62:41176) 2024-09-12T00:02:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.497+0000 7fa74effd640 1 -- 172.21.15.62:0/1843120704 learned_addr learned my addr 172.21.15.62:0/1843120704 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.498+0000 7fa74f7fe640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75010c1a0 0x7fa75019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.498+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa750103720 0x7fa750198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.498+0000 7fa74f7fe640 1 -- 172.21.15.62:0/1843120704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750102e00 msgr2=0x7fa750198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.498+0000 7fa74f7fe640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750102e00 0x7fa750198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.498+0000 7fa74f7fe640 1 -- 172.21.15.62:0/1843120704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa750103720 msgr2=0x7fa750198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.498+0000 7fa74f7fe640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa750103720 0x7fa750198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.498+0000 7fa74f7fe640 1 -- 172.21.15.62:0/1843120704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75019d3d0 con 0x7fa75010c1a0 2024-09-12T00:02:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.498+0000 7fa74effd640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750102e00 0x7fa750198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:02:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.498+0000 7fa74f7fe640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75010c1a0 0x7fa75019ccf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa74400e970 tx=0x7fa74400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.499+0000 7fa737fff640 1 -- 172.21.15.62:0/1843120704 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74400ccc0 con 0x7fa75010c1a0 2024-09-12T00:02:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.499+0000 7fa7554d6640 1 -- 172.21.15.62:0/1843120704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75019d6c0 con 0x7fa75010c1a0 2024-09-12T00:02:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.499+0000 7fa7554d6640 1 -- 172.21.15.62:0/1843120704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa750100990 con 0x7fa75010c1a0 2024-09-12T00:02:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.499+0000 7fa737fff640 1 -- 172.21.15.62:0/1843120704 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa744005020 con 0x7fa75010c1a0 2024-09-12T00:02:48.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.499+0000 7fa737fff640 1 -- 172.21.15.62:0/1843120704 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74400ccc0 con 0x7fa75010c1a0 2024-09-12T00:02:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.504+0000 7fa7578af740 1 Processor -- start 2024-09-12T00:02:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.504+0000 7fa7578af740 1 -- start start 2024-09-12T00:02:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.504+0000 7fa737fff640 1 -- 172.21.15.62:0/1843120704 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa7440047c0 con 0x7fa75010c1a0 2024-09-12T00:02:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.505+0000 7fa7578af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e119ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.505+0000 7fa7578af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11a0480 0x55a1e11aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.505+0000 7fa74e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e119ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.505+0000 7fa74effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11a0480 0x55a1e11aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.505+0000 7fa74e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e119ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40608/0 (socket says 172.21.15.62:40608) 2024-09-12T00:02:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.505+0000 7fa74effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11a0480 0x55a1e11aa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43122/0 (socket says 172.21.15.62:43122) 2024-09-12T00:02:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.506+0000 7fa7578af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11aad40 0x55a1e11ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.506+0000 7fa74e7fc640 1 -- 172.21.15.62:0/2353538174 learned_addr learned my addr 172.21.15.62:0/2353538174 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:48.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.506+0000 7fa7578af740 1 -- 172.21.15.62:0/2353538174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a1e1105280 con 0x55a1e11a0480 2024-09-12T00:02:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.506+0000 7fa7578af740 1 -- 172.21.15.62:0/2353538174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1e10acb10 con 0x55a1e119fb60 2024-09-12T00:02:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.506+0000 7fa7578af740 1 -- 172.21.15.62:0/2353538174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1e1101240 con 0x55a1e11aad40 2024-09-12T00:02:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.507+0000 7fa737fff640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa7380a2f70 0x7fa7380a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.507+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/2353538174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11aad40 0x55a1e11ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.507+0000 7fa74f7fe640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa7380a2f70 0x7fa7380a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.507+0000 7fa74e7fc640 1 -- 172.21.15.62:0/2353538174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11aad40 msgr2=0x55a1e11ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.507+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/2353538174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11aad40 0x55a1e11ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.507+0000 7fa74e7fc640 1 -- 172.21.15.62:0/2353538174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11a0480 msgr2=0x55a1e11aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.507+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/2353538174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11a0480 0x55a1e11aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.507+0000 7fa74e7fc640 1 -- 172.21.15.62:0/2353538174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e10b91f0 con 0x55a1e119fb60 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.507+0000 7fa737fff640 1 -- 172.21.15.62:0/1843120704 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fa7440ceef0 con 0x7fa75010c1a0 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.507+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/2353538174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e119ff40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa740009770 tx=0x7fa740010a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4879e483532df1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.508+0000 7fa7357fa640 1 -- 172.21.15.62:0/2353538174 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740004020 con 0x55a1e119fb60 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.508+0000 7fa7357fa640 1 -- 172.21.15.62:0/2353538174 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74001a040 con 0x55a1e119fb60 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.508+0000 7fa7357fa640 1 -- 172.21.15.62:0/2353538174 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7400039d0 con 0x55a1e119fb60 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.509+0000 7fa7578af740 1 -- 172.21.15.62:0/2353538174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 msgr2=0x55a1e119ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.509+0000 7fa7578af740 1 --2- 172.21.15.62:0/2353538174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e119ff40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa740009770 tx=0x7fa740010a60 comp rx=0 tx=0).stop 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.509+0000 7fa74f7fe640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa7380a2f70 0x7fa7380a5430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa74409b0b0 tx=0x7fa744065f30 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.509+0000 7fa7578af740 1 -- 172.21.15.62:0/2353538174 shutdown_connections 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.509+0000 7fa7578af740 1 --2- 172.21.15.62:0/2353538174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11aad40 0x55a1e11ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.509+0000 7fa7578af740 1 --2- 172.21.15.62:0/2353538174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11a0480 0x55a1e11aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.509+0000 7fa7578af740 1 --2- 172.21.15.62:0/2353538174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e119ff40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.509+0000 7fa7578af740 1 -- 172.21.15.62:0/2353538174 >> 172.21.15.62:0/2353538174 conn(0x55a1e119c0f0 msgr2=0x55a1e119c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.510+0000 7fa7578af740 1 -- 172.21.15.62:0/2353538174 shutdown_connections 2024-09-12T00:02:48.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.510+0000 7fa7578af740 1 -- 172.21.15.62:0/2353538174 wait complete. 2024-09-12T00:02:48.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.510+0000 7fa7578af740 1 -- start start 2024-09-12T00:02:48.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.510+0000 7fa7578af740 1 Processor -- start 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa7578af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e119fb60 0x55a1e11a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa7578af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11a0480 0x55a1e11a81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa7578af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11aad40 0x55a1e11a8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e119fb60 0x55a1e11a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11a0480 0x55a1e11a81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11a0480 0x55a1e11a81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40612/0 (socket says 172.21.15.62:40612) 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74effd640 1 -- 172.21.15.62:0/3237888918 learned_addr learned my addr 172.21.15.62:0/3237888918 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a1e11ad910 con 0x55a1e11aad40 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1e10bff10 con 0x55a1e11a0480 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1e11ada90 con 0x55a1e119fb60 2024-09-12T00:02:48.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa7578af740 10 client.0 _ll_register_callbacks cb 0x55a1e1088a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11aad40 0x55a1e11a8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74effd640 1 -- 172.21.15.62:0/3237888918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e119fb60 msgr2=0x55a1e11a7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74effd640 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e119fb60 0x55a1e11a7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74effd640 1 -- 172.21.15.62:0/3237888918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11aad40 msgr2=0x55a1e11a8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74effd640 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11aad40 0x55a1e11a8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74effd640 1 -- 172.21.15.62:0/3237888918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e0e766a0 con 0x55a1e11a0480 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.511+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11aad40 0x55a1e11a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.512+0000 7fa74effd640 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11a0480 0x55a1e11a81e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa748009dc0 tx=0x7fa748003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b12b65e46bc24c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.512+0000 7fa732ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.512+0000 7fa732ff5640 1 -- 172.21.15.62:0/3237888918 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74804a060 con 0x55a1e11a0480 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.512+0000 7fa732ff5640 1 -- 172.21.15.62:0/3237888918 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fa748004240 con 0x55a1e11a0480 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.512+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa74001a040 con 0x55a1e11a0480 2024-09-12T00:02:48.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.512+0000 7fa732ff5640 1 -- 172.21.15.62:0/3237888918 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa748034040 con 0x55a1e11a0480 2024-09-12T00:02:48.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.512+0000 7fa7578af740 10 client.26885 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:48.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.512+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a1e11b9630 con 0x55a1e11a0480 2024-09-12T00:02:48.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.513+0000 7fa732ff5640 1 -- 172.21.15.62:0/3237888918 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74001a040 con 0x55a1e11a0480 2024-09-12T00:02:48.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.513+0000 7fa732ff5640 1 -- 172.21.15.62:0/3237888918 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa748030970 con 0x55a1e11a0480 2024-09-12T00:02:48.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.513+0000 7fa7578af740 10 client.26885 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:48.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.513+0000 7fa7578af740 10 client.26885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:48.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.513+0000 7fa7578af740 20 client.26885 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:48.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.513+0000 7fa7578af740 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a1e11babc0 0x55a1e11bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.513+0000 7fa7578af740 4 client.26885 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:48.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.513+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1e11127f0 con 0x55a1e11babc0 2024-09-12T00:02:48.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.514+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a1e11babc0 0x55a1e11bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.516+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a1e11babc0 0x55a1e11bd020 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6e4e59fffc6a75 server_cookie=34b52f1ef37d3553 in_seq=0 out_seq=0 2024-09-12T00:02:48.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.516+0000 7fa732ff5640 10 client.26885 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:48.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.516+0000 7fa732ff5640 1 -- 172.21.15.62:0/3237888918 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1e11127f0 con 0x55a1e11babc0 2024-09-12T00:02:48.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.516+0000 7fa732ff5640 10 client.26885 handle_command_reply: tid=0 2024-09-12T00:02:48.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.516+0000 7fa7578af740 1 client.26885 shutdown 2024-09-12T00:02:48.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.517+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11a0480 msgr2=0x55a1e11a81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:48.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.517+0000 7fa7578af740 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11a0480 0x55a1e11a81e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa748009dc0 tx=0x7fa748003f90 comp rx=0 tx=0).stop 2024-09-12T00:02:48.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.517+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 shutdown_connections 2024-09-12T00:02:48.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.517+0000 7fa7578af740 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a1e11babc0 0x55a1e11bd020 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.517+0000 7fa7578af740 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11aad40 0x55a1e11a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.517+0000 7fa7578af740 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11a0480 0x55a1e11a81e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.517+0000 7fa7578af740 1 --2- 172.21.15.62:0/3237888918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e119fb60 0x55a1e11a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.517+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 >> 172.21.15.62:0/3237888918 conn(0x55a1e119c0f0 msgr2=0x55a1e119b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:48.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.517+0000 7fa732ff5640 0 client.26885 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:48.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.518+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 shutdown_connections 2024-09-12T00:02:48.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.518+0000 7fa7578af740 1 -- 172.21.15.62:0/3237888918 wait complete. 2024-09-12T00:02:48.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.518+0000 7fa7578af740 20 client.26885 trim_cache size 0 max 16384 2024-09-12T00:02:48.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.553+0000 7fa7578af740 1 Processor -- start 2024-09-12T00:02:48.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.553+0000 7fa7578af740 1 -- start start 2024-09-12T00:02:48.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.553+0000 7fa7578af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e119fb60 0x55a1e11a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.553+0000 7fa7578af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11a0480 0x55a1e11c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.553+0000 7fa7578af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11aad40 0x55a1e11c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.553+0000 7fa7578af740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c011050 con 0x55a1e119fb60 2024-09-12T00:02:48.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.553+0000 7fa7578af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c00e8f0 con 0x55a1e11aad40 2024-09-12T00:02:48.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.553+0000 7fa7578af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c00c760 con 0x55a1e11a0480 2024-09-12T00:02:48.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.554+0000 7fa74effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11a0480 0x55a1e11c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.554+0000 7fa74e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11aad40 0x55a1e11c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.555+0000 7fa74e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11aad40 0x55a1e11c2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40622/0 (socket says 172.21.15.62:40622) 2024-09-12T00:02:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.555+0000 7fa74e7fc640 1 -- 172.21.15.62:0/203358619 learned_addr learned my addr 172.21.15.62:0/203358619 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.555+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/203358619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e119fb60 0x55a1e11a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.556+0000 7fa74e7fc640 1 -- 172.21.15.62:0/203358619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11a0480 msgr2=0x55a1e11c28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.554+0000 7fa74effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11a0480 0x55a1e11c28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:41194/0 (socket says 172.21.15.62:41194) 2024-09-12T00:02:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.557+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/203358619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11a0480 0x55a1e11c28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.557+0000 7fa74e7fc640 1 -- 172.21.15.62:0/203358619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e119fb60 msgr2=0x55a1e11a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.557+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/203358619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e119fb60 0x55a1e11a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.557+0000 7fa74e7fc640 1 -- 172.21.15.62:0/203358619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e119bb90 con 0x55a1e11aad40 2024-09-12T00:02:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.558+0000 7fa74effd640 1 --2- 172.21.15.62:0/203358619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11a0480 0x55a1e11c28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:02:48.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.559+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/203358619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11aad40 0x55a1e11c2e30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa740000c00 tx=0x7fa740005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cdfd8a78493083f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:48.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.562+0000 7fa734ff9640 1 -- 172.21.15.62:0/203358619 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740005240 con 0x55a1e11aad40 2024-09-12T00:02:48.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.562+0000 7fa734ff9640 1 -- 172.21.15.62:0/203358619 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74000bd20 con 0x55a1e11aad40 2024-09-12T00:02:48.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.563+0000 7fa7578af740 1 -- 172.21.15.62:0/203358619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11aad40 msgr2=0x55a1e11c2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:48.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.563+0000 7fa7578af740 1 --2- 172.21.15.62:0/203358619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11aad40 0x55a1e11c2e30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa740000c00 tx=0x7fa740005850 comp rx=0 tx=0).stop 2024-09-12T00:02:48.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.564+0000 7fa7578af740 1 -- 172.21.15.62:0/203358619 shutdown_connections 2024-09-12T00:02:48.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.564+0000 7fa7578af740 1 --2- 172.21.15.62:0/203358619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e11aad40 0x55a1e11c2e30 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa740000c00 tx=0x7fa740005850 comp rx=0 tx=0).stop 2024-09-12T00:02:48.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.564+0000 7fa7578af740 1 --2- 172.21.15.62:0/203358619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11a0480 0x55a1e11c28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.564+0000 7fa7578af740 1 --2- 172.21.15.62:0/203358619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e119fb60 0x55a1e11a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.564+0000 7fa7578af740 1 -- 172.21.15.62:0/203358619 >> 172.21.15.62:0/203358619 conn(0x55a1e119c0f0 msgr2=0x55a1e11a6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:48.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.566+0000 7fa7578af740 1 -- 172.21.15.62:0/203358619 shutdown_connections 2024-09-12T00:02:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.567+0000 7fa7578af740 1 -- 172.21.15.62:0/203358619 wait complete. 2024-09-12T00:02:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.567+0000 7fa7578af740 1 -- start start 2024-09-12T00:02:48.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.567+0000 7fa7578af740 1 Processor -- start 2024-09-12T00:02:48.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.569+0000 7fa7578af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e11b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.569+0000 7fa7578af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11a0480 0x55a1e11b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.569+0000 7fa7578af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11afd40 0x55a1e11b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.569+0000 7fa7578af740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c00cdb0 con 0x55a1e11a0480 2024-09-12T00:02:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.569+0000 7fa7578af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c00e220 con 0x55a1e119fb60 2024-09-12T00:02:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.569+0000 7fa7578af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c00d480 con 0x55a1e11afd40 2024-09-12T00:02:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.569+0000 7fa7578af740 10 client.0 _ll_register_callbacks cb 0x7fa748009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.569+0000 7fa74e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e11b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.569+0000 7fa74e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e11b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40626/0 (socket says 172.21.15.62:40626) 2024-09-12T00:02:48.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.569+0000 7fa74e7fc640 1 -- 172.21.15.62:0/1833261122 learned_addr learned my addr 172.21.15.62:0/1833261122 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.570+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11afd40 0x55a1e11b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.571+0000 7fa74e7fc640 1 -- 172.21.15.62:0/1833261122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11afd40 msgr2=0x55a1e11b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.571+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11afd40 0x55a1e11b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.571+0000 7fa74e7fc640 1 -- 172.21.15.62:0/1833261122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11a0480 msgr2=0x55a1e11b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.571+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11a0480 0x55a1e11b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.571+0000 7fa74e7fc640 1 -- 172.21.15.62:0/1833261122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e11b09d0 con 0x55a1e119fb60 2024-09-12T00:02:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.571+0000 7fa74e7fc640 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e11b3000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa740000c00 tx=0x7fa740005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9271f350a49ef6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.571+0000 7fa732ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:02:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.571+0000 7fa732ff5640 1 -- 172.21.15.62:0/1833261122 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740034040 con 0x55a1e119fb60 2024-09-12T00:02:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.572+0000 7fa732ff5640 1 -- 172.21.15.62:0/1833261122 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7fa74000ba40 con 0x55a1e119fb60 2024-09-12T00:02:48.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.573+0000 7fa7578af740 1 -- 172.21.15.62:0/1833261122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa74000bd20 con 0x55a1e119fb60 2024-09-12T00:02:48.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.577+0000 7fa732ff5640 1 -- 172.21.15.62:0/1833261122 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa740004fd0 con 0x55a1e119fb60 2024-09-12T00:02:48.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.577+0000 7fa7578af740 10 client.26891 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:48.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.577+0000 7fa7578af740 1 -- 172.21.15.62:0/1833261122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a1e12267f0 con 0x55a1e119fb60 2024-09-12T00:02:48.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.577+0000 7fa732ff5640 1 -- 172.21.15.62:0/1833261122 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74000bd20 con 0x55a1e119fb60 2024-09-12T00:02:48.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.577+0000 7fa732ff5640 1 -- 172.21.15.62:0/1833261122 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa74003a040 con 0x55a1e119fb60 2024-09-12T00:02:48.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.578+0000 7fa7578af740 10 client.26891 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:48.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.578+0000 7fa7578af740 10 client.26891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:48.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.578+0000 7fa7578af740 20 client.26891 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:48.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.578+0000 7fa7578af740 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a1e1226dd0 0x55a1e12291f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:48.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.578+0000 7fa7578af740 4 client.26891 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:02:48.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.578+0000 7fa7578af740 1 -- 172.21.15.62:0/1833261122 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1e110cd00 con 0x55a1e1226dd0 2024-09-12T00:02:48.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.578+0000 7fa74effd640 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a1e1226dd0 0x55a1e12291f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:48.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.580+0000 7fa74effd640 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a1e1226dd0 0x55a1e12291f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccbeda42d8dec2e0 server_cookie=40f1f7ac17afe4ef in_seq=0 out_seq=0 2024-09-12T00:02:48.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.580+0000 7fa732ff5640 10 client.26891 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:48.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.581+0000 7fa732ff5640 1 -- 172.21.15.62:0/1833261122 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1e110cd00 con 0x55a1e1226dd0 2024-09-12T00:02:48.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.581+0000 7fa732ff5640 10 client.26891 handle_command_reply: tid=0 2024-09-12T00:02:48.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.581+0000 7fa7578af740 1 client.26891 shutdown 2024-09-12T00:02:48.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa7578af740 1 -- 172.21.15.62:0/1833261122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 msgr2=0x55a1e11b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:48.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa7578af740 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e11b3000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa740000c00 tx=0x7fa740005780 comp rx=0 tx=0).stop 2024-09-12T00:02:48.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa7578af740 1 -- 172.21.15.62:0/1833261122 shutdown_connections 2024-09-12T00:02:48.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa7578af740 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a1e1226dd0 0x55a1e12291f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa7578af740 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e11afd40 0x55a1e11b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa7578af740 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1e11a0480 0x55a1e11b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa7578af740 1 --2- 172.21.15.62:0/1833261122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1e119fb60 0x55a1e11b3000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa7578af740 1 -- 172.21.15.62:0/1833261122 >> 172.21.15.62:0/1833261122 conn(0x55a1e119c0f0 msgr2=0x55a1e119b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:48.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa732ff5640 0 client.26891 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:48.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa7578af740 1 -- 172.21.15.62:0/1833261122 shutdown_connections 2024-09-12T00:02:48.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.582+0000 7fa7578af740 1 -- 172.21.15.62:0/1833261122 wait complete. 2024-09-12T00:02:48.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.583+0000 7fa7578af740 20 client.26891 trim_cache size 0 max 16384 2024-09-12T00:02:48.584 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:02:48.584 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-12T00:02:48.584 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "948efd9d-c318-4bfa-8336-eb09c777287c", 2024-09-12T00:02:48.584 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-12T00:02:48.584 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:02:48.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.583+0000 7fa7554d6640 1 -- 172.21.15.62:0/1843120704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa7380a2f70 msgr2=0x7fa7380a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:48.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.583+0000 7fa7554d6640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa7380a2f70 0x7fa7380a5430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa74409b0b0 tx=0x7fa744065f30 comp rx=0 tx=0).stop 2024-09-12T00:02:48.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.583+0000 7fa7554d6640 1 -- 172.21.15.62:0/1843120704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75010c1a0 msgr2=0x7fa75019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:48.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.583+0000 7fa7554d6640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75010c1a0 0x7fa75019ccf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa74400e970 tx=0x7fa74400ee30 comp rx=0 tx=0).stop 2024-09-12T00:02:48.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.584+0000 7fa7554d6640 1 -- 172.21.15.62:0/1843120704 shutdown_connections 2024-09-12T00:02:48.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.584+0000 7fa7554d6640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa7380a2f70 0x7fa7380a5430 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.584+0000 7fa7554d6640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75010c1a0 0x7fa75019ccf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.584+0000 7fa7554d6640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa750103720 0x7fa750198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.584+0000 7fa7554d6640 1 --2- 172.21.15.62:0/1843120704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750102e00 0x7fa750198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:48.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.584+0000 7fa7554d6640 1 -- 172.21.15.62:0/1843120704 >> 172.21.15.62:0/1843120704 conn(0x7fa7500fc7a0 msgr2=0x7fa7500fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:48.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.584+0000 7fa7554d6640 1 -- 172.21.15.62:0/1843120704 shutdown_connections 2024-09-12T00:02:48.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:48.584+0000 7fa7554d6640 1 -- 172.21.15.62:0/1843120704 wait complete. 2024-09-12T00:02:49.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:48 smithi120 ceph-mon[35182]: pgmap v332: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 435 op/s 2024-09-12T00:02:49.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:48 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-12T00:02:49.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:48 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:49.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:48 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:02:49.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:48 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:49.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:48 smithi083 ceph-mon[34805]: pgmap v332: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 435 op/s 2024-09-12T00:02:49.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:48 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-12T00:02:49.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:48 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:49.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:48 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:02:49.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:48 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:48 smithi062 ceph-mon[30958]: pgmap v332: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 435 op/s 2024-09-12T00:02:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:48 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-12T00:02:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:48 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:48 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:02:49.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:48 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:49.336 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:02:49.764 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:02:50.075 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:49 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:02:50.075 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:49 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:50.075 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:49 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:02:50.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:49 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:50.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:49 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:02:50.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:49 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:50.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:49 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:02:50.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:49 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:50.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:49 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:02:50.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:49 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:50.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:49 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:02:50.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:49 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:02:50.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.322+0000 7f5b75cc6640 1 -- 172.21.15.62:0/2434409889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70106f40 msgr2=0x7f5b7010eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:50.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.322+0000 7f5b75cc6640 1 --2- 172.21.15.62:0/2434409889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70106f40 0x7f5b7010eac0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009be0 tx=0x7f5b6802a6d0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.322+0000 7f5b75cc6640 1 -- 172.21.15.62:0/2434409889 shutdown_connections 2024-09-12T00:02:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.322+0000 7f5b75cc6640 1 --2- 172.21.15.62:0/2434409889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70106f40 0x7f5b7010eac0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.322+0000 7f5b75cc6640 1 --2- 172.21.15.62:0/2434409889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b701027c0 0x7f5b70106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.322+0000 7f5b75cc6640 1 --2- 172.21.15.62:0/2434409889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b70101e10 0x7f5b701021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.322+0000 7f5b75cc6640 1 -- 172.21.15.62:0/2434409889 >> 172.21.15.62:0/2434409889 conn(0x7f5b700fd8d0 msgr2=0x7f5b700ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.322+0000 7f5b75cc6640 1 -- 172.21.15.62:0/2434409889 shutdown_connections 2024-09-12T00:02:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.322+0000 7f5b75cc6640 1 -- 172.21.15.62:0/2434409889 wait complete. 2024-09-12T00:02:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.323+0000 7f5b75cc6640 1 Processor -- start 2024-09-12T00:02:50.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.323+0000 7f5b75cc6640 1 -- start start 2024-09-12T00:02:50.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.323+0000 7f5b75cc6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b70101e10 0x7f5b701983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.323+0000 7f5b75cc6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b701027c0 0x7f5b70198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.323+0000 7f5b75cc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70106f40 0x7f5b7019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.323+0000 7f5b75cc6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70111f50 con 0x7f5b70101e10 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.323+0000 7f5b75cc6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70111dd0 con 0x7f5b701027c0 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.323+0000 7f5b75cc6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701120d0 con 0x7f5b70106f40 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b701027c0 0x7f5b70198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70106f40 0x7f5b7019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b701027c0 0x7f5b70198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40640/0 (socket says 172.21.15.62:40640) 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6effd640 1 -- 172.21.15.62:0/426361932 learned_addr learned my addr 172.21.15.62:0/426361932 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b70101e10 0x7f5b701983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6effd640 1 -- 172.21.15.62:0/426361932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70106f40 msgr2=0x7f5b7019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6effd640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70106f40 0x7f5b7019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6effd640 1 -- 172.21.15.62:0/426361932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b70101e10 msgr2=0x7f5b701983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6effd640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b70101e10 0x7f5b701983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6effd640 1 -- 172.21.15.62:0/426361932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7019d380 con 0x7f5b701027c0 2024-09-12T00:02:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6effd640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b701027c0 0x7f5b70198910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400e9f0 tx=0x7f5b6400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6cff9640 1 -- 172.21.15.62:0/426361932 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400ccc0 con 0x7f5b701027c0 2024-09-12T00:02:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6cff9640 1 -- 172.21.15.62:0/426361932 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b64010910 con 0x7f5b701027c0 2024-09-12T00:02:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b75cc6640 1 -- 172.21.15.62:0/426361932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7019d670 con 0x7f5b701027c0 2024-09-12T00:02:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.324+0000 7f5b6cff9640 1 -- 172.21.15.62:0/426361932 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b640188a0 con 0x7f5b701027c0 2024-09-12T00:02:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.325+0000 7f5b75cc6640 1 -- 172.21.15.62:0/426361932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b701a4f50 con 0x7f5b701027c0 2024-09-12T00:02:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.329+0000 7f5b6cff9640 1 -- 172.21.15.62:0/426361932 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5b64010ac0 con 0x7f5b701027c0 2024-09-12T00:02:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.329+0000 7f5b7809f740 1 Processor -- start 2024-09-12T00:02:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.329+0000 7f5b7809f740 1 -- start start 2024-09-12T00:02:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 0x564155486880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155491140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 0x564155493a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b7809f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5641553ec2e0 con 0x564155491680 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b7809f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564155393ce0 con 0x564155486dc0 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b7809f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5641553e82a0 con 0x5641554864a0 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 0x564155493a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 0x564155486880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155491140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 0x564155486880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:41220/0 (socket says 172.21.15.62:41220) 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6ffff640 1 -- 172.21.15.62:0/1397462449 learned_addr learned my addr 172.21.15.62:0/1397462449 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155491140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40654/0 (socket says 172.21.15.62:40654) 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6f7fe640 1 -- 172.21.15.62:0/1397462449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 msgr2=0x564155486880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/1397462449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 0x564155486880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6f7fe640 1 -- 172.21.15.62:0/1397462449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 msgr2=0x564155493a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/1397462449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 0x564155493a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.330+0000 7f5b6f7fe640 1 -- 172.21.15.62:0/1397462449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564155378440 con 0x564155486dc0 2024-09-12T00:02:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.331+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/1397462449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155491140 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009c30 tx=0x7f5b60011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1afded73bde95cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.331+0000 7f5b6cff9640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5b580a2d70 0x7f5b580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.331+0000 7f5b55ffb640 1 -- 172.21.15.62:0/1397462449 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6001e070 con 0x564155486dc0 2024-09-12T00:02:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.331+0000 7f5b6ffff640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5b580a2d70 0x7f5b580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.331+0000 7f5b6cff9640 1 -- 172.21.15.62:0/426361932 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f5b640cf890 con 0x7f5b701027c0 2024-09-12T00:02:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.331+0000 7f5b55ffb640 1 -- 172.21.15.62:0/1397462449 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60009de0 con 0x564155486dc0 2024-09-12T00:02:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.331+0000 7f5b55ffb640 1 -- 172.21.15.62:0/1397462449 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60019410 con 0x564155486dc0 2024-09-12T00:02:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.332+0000 7f5b7809f740 1 -- 172.21.15.62:0/1397462449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 msgr2=0x564155491140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.332+0000 7f5b6ffff640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5b580a2d70 0x7f5b580a5230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b680099a0 tx=0x7f5b68009280 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.332+0000 7f5b7809f740 1 --2- 172.21.15.62:0/1397462449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155491140 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009c30 tx=0x7f5b60011810 comp rx=0 tx=0).stop 2024-09-12T00:02:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.332+0000 7f5b7809f740 1 -- 172.21.15.62:0/1397462449 shutdown_connections 2024-09-12T00:02:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.332+0000 7f5b7809f740 1 --2- 172.21.15.62:0/1397462449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 0x564155493a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.332+0000 7f5b7809f740 1 --2- 172.21.15.62:0/1397462449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155491140 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.332+0000 7f5b7809f740 1 --2- 172.21.15.62:0/1397462449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 0x564155486880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.332+0000 7f5b7809f740 1 -- 172.21.15.62:0/1397462449 >> 172.21.15.62:0/1397462449 conn(0x564155482a30 msgr2=0x564155482e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.332+0000 7f5b7809f740 1 -- 172.21.15.62:0/1397462449 shutdown_connections 2024-09-12T00:02:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.332+0000 7f5b7809f740 1 -- 172.21.15.62:0/1397462449 wait complete. 2024-09-12T00:02:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b7809f740 1 -- start start 2024-09-12T00:02:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b7809f740 1 Processor -- start 2024-09-12T00:02:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641554864a0 0x564155490f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564155486dc0 0x56415548e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155491680 0x56415548e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b7809f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564155498700 con 0x5641554864a0 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b7809f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641553a7050 con 0x564155491680 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b7809f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564155498880 con 0x564155486dc0 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b7809f740 10 client.0 _ll_register_callbacks cb 0x5641553e7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b6f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641554864a0 0x564155490f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b6f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641554864a0 0x564155490f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43194/0 (socket says 172.21.15.62:43194) 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b6f7fe640 1 -- 172.21.15.62:0/958265285 learned_addr learned my addr 172.21.15.62:0/958265285 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b6ffff640 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155491680 0x56415548e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.333+0000 7f5b6effd640 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564155486dc0 0x56415548e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.334+0000 7f5b6ffff640 1 -- 172.21.15.62:0/958265285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564155486dc0 msgr2=0x56415548e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.334+0000 7f5b6ffff640 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564155486dc0 0x56415548e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.334+0000 7f5b6ffff640 1 -- 172.21.15.62:0/958265285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641554864a0 msgr2=0x564155490f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.334+0000 7f5b6ffff640 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641554864a0 0x564155490f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.334+0000 7f5b6ffff640 1 -- 172.21.15.62:0/958265285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56415515d3c0 con 0x564155491680 2024-09-12T00:02:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.334+0000 7f5b6ffff640 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155491680 0x56415548e760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b680092f0 tx=0x7f5b68009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45fdd5d09fcace29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.334+0000 7f5b4effd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:02:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.335+0000 7f5b4effd640 1 -- 172.21.15.62:0/958265285 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68036050 con 0x564155491680 2024-09-12T00:02:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.335+0000 7f5b4effd640 1 -- 172.21.15.62:0/958265285 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f5b68003d80 con 0x564155491680 2024-09-12T00:02:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.335+0000 7f5b7809f740 1 -- 172.21.15.62:0/958265285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b60009de0 con 0x564155491680 2024-09-12T00:02:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.335+0000 7f5b4effd640 1 -- 172.21.15.62:0/958265285 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5b68004040 con 0x564155491680 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.335+0000 7f5b7809f740 10 client.26903 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.335+0000 7f5b7809f740 1 -- 172.21.15.62:0/958265285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641554a4070 con 0x564155491680 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.336+0000 7f5b4effd640 1 -- 172.21.15.62:0/958265285 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60009de0 con 0x564155491680 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.336+0000 7f5b4effd640 1 -- 172.21.15.62:0/958265285 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5b6803c040 con 0x564155491680 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.336+0000 7f5b7809f740 10 client.26903 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.336+0000 7f5b7809f740 10 client.26903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.336+0000 7f5b7809f740 20 client.26903 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.336+0000 7f5b7809f740 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5641554a5530 0x5641554a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.336+0000 7f5b7809f740 4 client.26903 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.336+0000 7f5b7809f740 1 -- 172.21.15.62:0/958265285 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641553f9850 con 0x5641554a5530 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.336+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5641554a5530 0x5641554a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.337+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5641554a5530 0x5641554a7920 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3572e70bbf72a798 server_cookie=ddf885dd565a7eb4 in_seq=0 out_seq=0 2024-09-12T00:02:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.337+0000 7f5b4effd640 10 client.26903 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.337+0000 7f5b4effd640 1 -- 172.21.15.62:0/958265285 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641553f9850 con 0x5641554a5530 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.337+0000 7f5b4effd640 10 client.26903 handle_command_reply: tid=0 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.337+0000 7f5b7809f740 1 client.26903 shutdown 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.337+0000 7f5b7809f740 1 -- 172.21.15.62:0/958265285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155491680 msgr2=0x56415548e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.337+0000 7f5b7809f740 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155491680 0x56415548e760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b680092f0 tx=0x7f5b68009500 comp rx=0 tx=0).stop 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.337+0000 7f5b7809f740 1 -- 172.21.15.62:0/958265285 shutdown_connections 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.337+0000 7f5b7809f740 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5641554a5530 0x5641554a7920 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.338+0000 7f5b7809f740 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155491680 0x56415548e760 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.338+0000 7f5b7809f740 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564155486dc0 0x56415548e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.338+0000 7f5b4effd640 0 client.26903 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.338+0000 7f5b7809f740 1 --2- 172.21.15.62:0/958265285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641554864a0 0x564155490f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.338+0000 7f5b7809f740 1 -- 172.21.15.62:0/958265285 >> 172.21.15.62:0/958265285 conn(0x564155482a30 msgr2=0x564155488710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.338+0000 7f5b7809f740 1 -- 172.21.15.62:0/958265285 shutdown_connections 2024-09-12T00:02:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.338+0000 7f5b7809f740 1 -- 172.21.15.62:0/958265285 wait complete. 2024-09-12T00:02:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.338+0000 7f5b7809f740 20 client.26903 trim_cache size 0 max 16384 2024-09-12T00:02:50.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.362+0000 7f5b7809f740 1 Processor -- start 2024-09-12T00:02:50.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.362+0000 7f5b7809f740 1 -- start start 2024-09-12T00:02:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.362+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641554864a0 0x56415548d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.362+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155486dc0 0x5641554ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.362+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564155491680 0x5641554ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b7809f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ae0a0 con 0x564155486dc0 2024-09-12T00:02:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b7809f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ab940 con 0x5641554864a0 2024-09-12T00:02:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b7809f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701a97b0 con 0x564155491680 2024-09-12T00:02:50.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641554864a0 0x56415548d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641554864a0 0x56415548d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40668/0 (socket says 172.21.15.62:40668) 2024-09-12T00:02:50.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6f7fe640 1 -- 172.21.15.62:0/3479551528 learned_addr learned my addr 172.21.15.62:0/3479551528 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:50.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6ffff640 1 --2- 172.21.15.62:0/3479551528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564155491680 0x5641554ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6effd640 1 --2- 172.21.15.62:0/3479551528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155486dc0 0x5641554ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6f7fe640 1 -- 172.21.15.62:0/3479551528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564155491680 msgr2=0x5641554ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:50.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/3479551528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564155491680 0x5641554ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6f7fe640 1 -- 172.21.15.62:0/3479551528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155486dc0 msgr2=0x5641554ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/3479551528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155486dc0 0x5641554ad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6f7fe640 1 -- 172.21.15.62:0/3479551528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56415548f5a0 con 0x5641554864a0 2024-09-12T00:02:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6effd640 1 --2- 172.21.15.62:0/3479551528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155486dc0 0x5641554ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/3479551528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641554864a0 0x56415548d160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009b80 tx=0x7f5b60009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c87936d5fa36e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.363+0000 7f5b557fa640 1 -- 172.21.15.62:0/3479551528 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6001e040 con 0x5641554864a0 2024-09-12T00:02:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.364+0000 7f5b557fa640 1 -- 172.21.15.62:0/3479551528 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60005520 con 0x5641554864a0 2024-09-12T00:02:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.364+0000 7f5b557fa640 1 -- 172.21.15.62:0/3479551528 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6002b7e0 con 0x5641554864a0 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.364+0000 7f5b7809f740 1 -- 172.21.15.62:0/3479551528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641554864a0 msgr2=0x56415548d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.364+0000 7f5b7809f740 1 --2- 172.21.15.62:0/3479551528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641554864a0 0x56415548d160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009b80 tx=0x7f5b60009640 comp rx=0 tx=0).stop 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.365+0000 7f5b7809f740 1 -- 172.21.15.62:0/3479551528 shutdown_connections 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.365+0000 7f5b7809f740 1 --2- 172.21.15.62:0/3479551528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564155491680 0x5641554ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.365+0000 7f5b7809f740 1 --2- 172.21.15.62:0/3479551528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155486dc0 0x5641554ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.365+0000 7f5b7809f740 1 --2- 172.21.15.62:0/3479551528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641554864a0 0x56415548d160 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.365+0000 7f5b7809f740 1 -- 172.21.15.62:0/3479551528 >> 172.21.15.62:0/3479551528 conn(0x564155482a30 msgr2=0x5641554902a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.365+0000 7f5b7809f740 1 -- 172.21.15.62:0/3479551528 shutdown_connections 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.365+0000 7f5b7809f740 1 -- 172.21.15.62:0/3479551528 wait complete. 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.365+0000 7f5b7809f740 1 -- start start 2024-09-12T00:02:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.365+0000 7f5b7809f740 1 Processor -- start 2024-09-12T00:02:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 0x56415549e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155496e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b7809f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 0x5641554973b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b7809f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701a9e00 con 0x564155491680 2024-09-12T00:02:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b7809f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ab270 con 0x564155486dc0 2024-09-12T00:02:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b7809f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701aa4d0 con 0x5641554864a0 2024-09-12T00:02:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b7809f740 10 client.0 _ll_register_callbacks cb 0x7f5b64005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:02:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 0x5641554973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 0x5641554973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43208/0 (socket says 172.21.15.62:43208) 2024-09-12T00:02:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6ffff640 1 -- 172.21.15.62:0/256458790 learned_addr learned my addr 172.21.15.62:0/256458790 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:02:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 0x56415549e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6effd640 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155496e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6ffff640 1 -- 172.21.15.62:0/256458790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 msgr2=0x56415549e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6ffff640 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 0x56415549e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6ffff640 1 -- 172.21.15.62:0/256458790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 msgr2=0x564155496e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:02:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6ffff640 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155496e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6ffff640 1 -- 172.21.15.62:0/256458790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564155497be0 con 0x564155491680 2024-09-12T00:02:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.366+0000 7f5b6effd640 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155496e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:02:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.367+0000 7f5b6ffff640 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 0x5641554973b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5b68031040 tx=0x7f5b68003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7646adac5bd518 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:02:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.367+0000 7f5b4effd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:02:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.367+0000 7f5b4effd640 1 -- 172.21.15.62:0/256458790 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68036040 con 0x564155491680 2024-09-12T00:02:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.367+0000 7f5b4effd640 1 -- 172.21.15.62:0/256458790 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8513+0+0 (secure 0 0 0) 0x7f5b680041e0 con 0x564155491680 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.367+0000 7f5b7809f740 1 -- 172.21.15.62:0/256458790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b60005520 con 0x564155491680 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.367+0000 7f5b4effd640 1 -- 172.21.15.62:0/256458790 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5b6806c050 con 0x564155491680 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.367+0000 7f5b7809f740 10 client.17382 fetch_fsmap learned FSMap version 33 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.367+0000 7f5b7809f740 1 -- 172.21.15.62:0/256458790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56415550c1a0 con 0x564155491680 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.368+0000 7f5b4effd640 1 -- 172.21.15.62:0/256458790 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60005520 con 0x564155491680 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.368+0000 7f5b4effd640 1 -- 172.21.15.62:0/256458790 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5b6803fa10 con 0x564155491680 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.368+0000 7f5b7809f740 10 client.17382 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.368+0000 7f5b7809f740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.368+0000 7f5b7809f740 20 client.17382 populate_metadata read hostname 'smithi062' 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.368+0000 7f5b7809f740 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56415550c9a0 0x56415550edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:02:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.368+0000 7f5b7809f740 4 client.17382 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.368+0000 7f5b7809f740 1 -- 172.21.15.62:0/256458790 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641553f3d60 con 0x56415550c9a0 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.368+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56415550c9a0 0x56415550edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b4effd640 10 client.17382 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b6f7fe640 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56415550c9a0 0x56415550edc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5ce2f064fd0791 server_cookie=e085a4fd4a12e092 in_seq=0 out_seq=0 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b4effd640 1 -- 172.21.15.62:0/256458790 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5641553f3d60 con 0x56415550c9a0 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b4effd640 10 client.17382 handle_command_reply: tid=0 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b7809f740 1 client.17382 shutdown 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b7809f740 1 -- 172.21.15.62:0/256458790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 msgr2=0x5641554973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b7809f740 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 0x5641554973b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5b68031040 tx=0x7f5b68003db0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b7809f740 1 -- 172.21.15.62:0/256458790 shutdown_connections 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b7809f740 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56415550c9a0 0x56415550edc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b7809f740 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564155491680 0x5641554973b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b7809f740 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564155486dc0 0x564155496e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b7809f740 1 --2- 172.21.15.62:0/256458790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5641554864a0 0x56415549e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b7809f740 1 -- 172.21.15.62:0/256458790 >> 172.21.15.62:0/256458790 conn(0x564155482a30 msgr2=0x564155481350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.369+0000 7f5b4effd640 0 client.17382 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:02:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b7809f740 1 -- 172.21.15.62:0/256458790 shutdown_connections 2024-09-12T00:02:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b7809f740 1 -- 172.21.15.62:0/256458790 wait complete. 2024-09-12T00:02:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b7809f740 20 client.17382 trim_cache size 0 max 16384 2024-09-12T00:02:50.381 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:02:50.381 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:02:50.381 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-12T00:02:50.381 INFO:teuthology.orchestra.run.smithi062.stdout: "948efd9d-c318-4bfa-8336-eb09c777287c": { 2024-09-12T00:02:50.381 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-12T00:02:50.381 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "948efd9d-c318-4bfa-8336-eb09c777287c", 2024-09-12T00:02:50.381 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-12T00:02:50.381 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:50.381 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:02:50.382 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:02:50.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b75cc6640 1 -- 172.21.15.62:0/426361932 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5b580a2d70 msgr2=0x7f5b580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:50.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b75cc6640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5b580a2d70 0x7f5b580a5230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b680099a0 tx=0x7f5b68009280 comp rx=0 tx=0).stop 2024-09-12T00:02:50.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b75cc6640 1 -- 172.21.15.62:0/426361932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b701027c0 msgr2=0x7f5b70198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:02:50.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b75cc6640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b701027c0 0x7f5b70198910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400e9f0 tx=0x7f5b6400edf0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b75cc6640 1 -- 172.21.15.62:0/426361932 shutdown_connections 2024-09-12T00:02:50.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b75cc6640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5b580a2d70 0x7f5b580a5230 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b75cc6640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70106f40 0x7f5b7019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b75cc6640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b701027c0 0x7f5b70198910 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b75cc6640 1 --2- 172.21.15.62:0/426361932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b70101e10 0x7f5b701983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:02:50.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.370+0000 7f5b75cc6640 1 -- 172.21.15.62:0/426361932 >> 172.21.15.62:0/426361932 conn(0x7f5b700fd8d0 msgr2=0x7f5b700fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:02:50.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.371+0000 7f5b75cc6640 1 -- 172.21.15.62:0/426361932 shutdown_connections 2024-09-12T00:02:50.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:02:50.371+0000 7f5b75cc6640 1 -- 172.21.15.62:0/426361932 wait complete. 2024-09-12T00:02:51.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:50 smithi062 ceph-mon[30958]: pgmap v333: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 835 KiB/s wr, 84 op/s 2024-09-12T00:02:51.091 INFO:tasks.cephfs.filesystem:scrub status for tag:948efd9d-c318-4bfa-8336-eb09c777287c - {'path': '/', 'tag': '948efd9d-c318-4bfa-8336-eb09c777287c', 'options': 'recursive,force'} 2024-09-12T00:02:51.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:50 smithi120 ceph-mon[35182]: pgmap v333: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 835 KiB/s wr, 84 op/s 2024-09-12T00:02:51.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:50 smithi083 ceph-mon[34805]: pgmap v333: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 835 KiB/s wr, 84 op/s 2024-09-12T00:02:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 610.0 s, 2347.0 tps, lat 42.569 ms stddev 50.038 2024-09-12T00:02:53.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:52 smithi120 ceph-mon[35182]: pgmap v334: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 275 KiB/s wr, 26 op/s 2024-09-12T00:02:53.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:52 smithi083 ceph-mon[34805]: pgmap v334: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 275 KiB/s wr, 26 op/s 2024-09-12T00:02:53.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:52 smithi062 ceph-mon[30958]: pgmap v334: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 275 KiB/s wr, 26 op/s 2024-09-12T00:02:55.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:54 smithi120 ceph-mon[35182]: pgmap v335: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.6 MiB/s wr, 295 op/s 2024-09-12T00:02:55.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:54 smithi083 ceph-mon[34805]: pgmap v335: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.6 MiB/s wr, 295 op/s 2024-09-12T00:02:55.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:54 smithi062 ceph-mon[30958]: pgmap v335: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.6 MiB/s wr, 295 op/s 2024-09-12T00:02:56.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:55 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:02:56.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:55 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:02:56.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:55 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:02:57.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:56 smithi120 ceph-mon[35182]: pgmap v336: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.6 MiB/s wr, 293 op/s 2024-09-12T00:02:57.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:56 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:02:57.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:56 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:02:57.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:56 smithi083 ceph-mon[34805]: pgmap v336: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.6 MiB/s wr, 293 op/s 2024-09-12T00:02:57.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:56 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:02:57.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:56 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:02:57.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:56 smithi062 ceph-mon[30958]: pgmap v336: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.6 MiB/s wr, 293 op/s 2024-09-12T00:02:57.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:56 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:02:57.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:56 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:02:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 615.0 s, 2574.2 tps, lat 38.347 ms stddev 45.474 2024-09-12T00:02:59.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:02:58 smithi120 ceph-mon[35182]: pgmap v337: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.8 MiB/s wr, 541 op/s 2024-09-12T00:02:59.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:02:58 smithi083 ceph-mon[34805]: pgmap v337: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.8 MiB/s wr, 541 op/s 2024-09-12T00:02:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:02:58 smithi062 ceph-mon[30958]: pgmap v337: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.8 MiB/s wr, 541 op/s 2024-09-12T00:03:01.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:00 smithi120 ceph-mon[35182]: pgmap v338: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 599 op/s 2024-09-12T00:03:01.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:00 smithi083 ceph-mon[34805]: pgmap v338: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 599 op/s 2024-09-12T00:03:01.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:00 smithi062 ceph-mon[30958]: pgmap v338: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 599 op/s 2024-09-12T00:03:02.452 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 620.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:03:03.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:02 smithi120 ceph-mon[35182]: pgmap v339: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.9 MiB/s wr, 594 op/s 2024-09-12T00:03:03.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:02 smithi083 ceph-mon[34805]: pgmap v339: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.9 MiB/s wr, 594 op/s 2024-09-12T00:03:03.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:02 smithi062 ceph-mon[30958]: pgmap v339: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.9 MiB/s wr, 594 op/s 2024-09-12T00:03:05.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:04 smithi120 ceph-mon[35182]: pgmap v340: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 31 MiB/s wr, 1.69k op/s 2024-09-12T00:03:05.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:04 smithi083 ceph-mon[34805]: pgmap v340: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 31 MiB/s wr, 1.69k op/s 2024-09-12T00:03:05.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:04 smithi062 ceph-mon[30958]: pgmap v340: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 31 MiB/s wr, 1.69k op/s 2024-09-12T00:03:07.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:06 smithi120 ceph-mon[35182]: pgmap v341: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 27 MiB/s wr, 1.42k op/s 2024-09-12T00:03:07.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:06 smithi083 ceph-mon[34805]: pgmap v341: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 27 MiB/s wr, 1.42k op/s 2024-09-12T00:03:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:06 smithi062 ceph-mon[30958]: pgmap v341: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 27 MiB/s wr, 1.42k op/s 2024-09-12T00:03:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 625.0 s, 209.6 tps, lat 962.226 ms stddev 2883.613 2024-09-12T00:03:09.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:08 smithi120 ceph-mon[35182]: pgmap v342: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 29 MiB/s wr, 1.53k op/s 2024-09-12T00:03:09.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:08 smithi083 ceph-mon[34805]: pgmap v342: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 29 MiB/s wr, 1.53k op/s 2024-09-12T00:03:09.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:08 smithi062 ceph-mon[30958]: pgmap v342: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 29 MiB/s wr, 1.53k op/s 2024-09-12T00:03:11.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:10 smithi083 ceph-mon[34805]: pgmap v343: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 24 MiB/s wr, 1.30k op/s 2024-09-12T00:03:11.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:10 smithi062 ceph-mon[30958]: pgmap v343: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 24 MiB/s wr, 1.30k op/s 2024-09-12T00:03:11.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:10 smithi120 ceph-mon[35182]: pgmap v343: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 24 MiB/s wr, 1.30k op/s 2024-09-12T00:03:12.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:11 smithi083 ceph-mon[34805]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T00:03:12.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:11 smithi062 ceph-mon[30958]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T00:03:12.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:11 smithi120 ceph-mon[35182]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T00:03:12.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 630.0 s, 1872.4 tps, lat 52.934 ms stddev 69.525 2024-09-12T00:03:13.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:12 smithi083 ceph-mon[34805]: pgmap v345: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 1.48k op/s 2024-09-12T00:03:13.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:12 smithi062 ceph-mon[30958]: pgmap v345: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 1.48k op/s 2024-09-12T00:03:13.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:12 smithi120 ceph-mon[35182]: pgmap v345: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 1.48k op/s 2024-09-12T00:03:14.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:13 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:03:14.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:13 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:03:14.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:13 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:03:15.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:14 smithi083 ceph-mon[34805]: pgmap v346: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 484 op/s 2024-09-12T00:03:15.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:14 smithi062 ceph-mon[30958]: pgmap v346: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 484 op/s 2024-09-12T00:03:15.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:14 smithi120 ceph-mon[35182]: pgmap v346: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 484 op/s 2024-09-12T00:03:17.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:16 smithi083 ceph-mon[34805]: pgmap v347: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 484 op/s 2024-09-12T00:03:17.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:16 smithi083 ceph-mon[34805]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T00:03:17.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:16 smithi062 ceph-mon[30958]: pgmap v347: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 484 op/s 2024-09-12T00:03:17.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:16 smithi062 ceph-mon[30958]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T00:03:17.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:16 smithi120 ceph-mon[35182]: pgmap v347: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 484 op/s 2024-09-12T00:03:17.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:16 smithi120 ceph-mon[35182]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T00:03:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 635.0 s, 2480.6 tps, lat 40.704 ms stddev 46.150 2024-09-12T00:03:19.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:18 smithi083 ceph-mon[34805]: pgmap v349: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.6 MiB/s wr, 758 op/s 2024-09-12T00:03:19.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:18 smithi062 ceph-mon[30958]: pgmap v349: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.6 MiB/s wr, 758 op/s 2024-09-12T00:03:19.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:18 smithi120 ceph-mon[35182]: pgmap v349: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.6 MiB/s wr, 758 op/s 2024-09-12T00:03:21.093 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:03:21.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:20 smithi083 ceph-mon[34805]: pgmap v350: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.2 MiB/s wr, 739 op/s 2024-09-12T00:03:21.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:20 smithi062 ceph-mon[30958]: pgmap v350: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.2 MiB/s wr, 739 op/s 2024-09-12T00:03:21.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:20 smithi120 ceph-mon[35182]: pgmap v350: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.2 MiB/s wr, 739 op/s 2024-09-12T00:03:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:03:22.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.086+0000 7ffa189f9640 1 -- 172.21.15.62:0/2673219067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa14100e20 msgr2=0x7ffa14101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:22.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.086+0000 7ffa189f9640 1 --2- 172.21.15.62:0/2673219067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa14100e20 0x7ffa14101200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009930 tx=0x7ffa0402a4d0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.086+0000 7ffa189f9640 1 -- 172.21.15.62:0/2673219067 shutdown_connections 2024-09-12T00:03:22.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.086+0000 7ffa189f9640 1 --2- 172.21.15.62:0/2673219067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa14102ef0 0x7ffa1410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.086+0000 7ffa189f9640 1 --2- 172.21.15.62:0/2673219067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa14102570 0x7ffa141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.086+0000 7ffa189f9640 1 --2- 172.21.15.62:0/2673219067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa14100e20 0x7ffa14101200 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.086+0000 7ffa189f9640 1 -- 172.21.15.62:0/2673219067 >> 172.21.15.62:0/2673219067 conn(0x7ffa140fc7d0 msgr2=0x7ffa140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:22.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.087+0000 7ffa189f9640 1 -- 172.21.15.62:0/2673219067 shutdown_connections 2024-09-12T00:03:22.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.087+0000 7ffa189f9640 1 -- 172.21.15.62:0/2673219067 wait complete. 2024-09-12T00:03:22.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.088+0000 7ffa189f9640 1 Processor -- start 2024-09-12T00:03:22.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa189f9640 1 -- start start 2024-09-12T00:03:22.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa189f9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa14100e20 0x7ffa141993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa189f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa14102570 0x7ffa141998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa189f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa14102ef0 0x7ffa141a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa189f9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14110f50 con 0x7ffa14100e20 2024-09-12T00:03:22.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa189f9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14110dd0 con 0x7ffa14102570 2024-09-12T00:03:22.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa189f9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141110d0 con 0x7ffa14102ef0 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa14100e20 0x7ffa141993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa14100e20 0x7ffa141993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54588/0 (socket says 172.21.15.62:54588) 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa1259c640 1 -- 172.21.15.62:0/277463186 learned_addr learned my addr 172.21.15.62:0/277463186 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.089+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa14102570 0x7ffa141998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa12d9d640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa14102ef0 0x7ffa141a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa0bfff640 1 -- 172.21.15.62:0/277463186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa14102ef0 msgr2=0x7ffa141a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa14102ef0 0x7ffa141a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa0bfff640 1 -- 172.21.15.62:0/277463186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa14100e20 msgr2=0x7ffa141993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa14100e20 0x7ffa141993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa0bfff640 1 -- 172.21.15.62:0/277463186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa141a4960 con 0x7ffa14102570 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa1259c640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa14100e20 0x7ffa141993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa14102570 0x7ffa141998e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00e9f0 tx=0x7ff9fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:22.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa0b7fe640 1 -- 172.21.15.62:0/277463186 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00ccc0 con 0x7ffa14102570 2024-09-12T00:03:22.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa0b7fe640 1 -- 172.21.15.62:0/277463186 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc010910 con 0x7ffa14102570 2024-09-12T00:03:22.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa189f9640 1 -- 172.21.15.62:0/277463186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa141a4b30 con 0x7ffa14102570 2024-09-12T00:03:22.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa0b7fe640 1 -- 172.21.15.62:0/277463186 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc018960 con 0x7ffa14102570 2024-09-12T00:03:22.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.090+0000 7ffa189f9640 1 -- 172.21.15.62:0/277463186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa141a4fb0 con 0x7ffa14102570 2024-09-12T00:03:22.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.095+0000 7ffa1add2740 1 Processor -- start 2024-09-12T00:03:22.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.096+0000 7ffa1add2740 1 -- start start 2024-09-12T00:03:22.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.096+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080b9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.096+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080b9880 0x5610080b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.097+0000 7ffa0b7fe640 1 -- 172.21.15.62:0/277463186 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff9fc010430 con 0x7ffa14102570 2024-09-12T00:03:22.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.097+0000 7ffa12d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.097+0000 7ffa12d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080b9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33824/0 (socket says 172.21.15.62:33824) 2024-09-12T00:03:22.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.097+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080b9880 0x5610080b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.097+0000 7ffa12d9d640 1 -- 172.21.15.62:0/1902355314 learned_addr learned my addr 172.21.15.62:0/1902355314 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:22.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.097+0000 7ffa1add2740 1 --2- 172.21.15.62:0/1902355314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080ccd20 0x5610080cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.097+0000 7ffa1add2740 1 -- 172.21.15.62:0/1902355314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56100801f2e0 con 0x5610080b9880 2024-09-12T00:03:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.097+0000 7ffa1add2740 1 -- 172.21.15.62:0/1902355314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561007fc6ce0 con 0x5610080ccd20 2024-09-12T00:03:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.097+0000 7ffa1add2740 1 -- 172.21.15.62:0/1902355314 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56100801b2a0 con 0x5610080baa20 2024-09-12T00:03:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.098+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/1902355314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080ccd20 0x5610080cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.098+0000 7ffa1259c640 1 -- 172.21.15.62:0/1902355314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 msgr2=0x5610080b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.098+0000 7ffa1259c640 1 --2- 172.21.15.62:0/1902355314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.098+0000 7ffa1259c640 1 -- 172.21.15.62:0/1902355314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080ccd20 msgr2=0x5610080cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.098+0000 7ffa1259c640 1 --2- 172.21.15.62:0/1902355314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080ccd20 0x5610080cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.098+0000 7ffa1259c640 1 -- 172.21.15.62:0/1902355314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561007fab440 con 0x5610080b9880 2024-09-12T00:03:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.099+0000 7ffa1259c640 1 --2- 172.21.15.62:0/1902355314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080b9880 0x5610080b9cc0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009900 tx=0x7ffa04032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2278a253b22e9de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:22.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.099+0000 7ffa08ff9640 1 -- 172.21.15.62:0/1902355314 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0403f070 con 0x5610080b9880 2024-09-12T00:03:22.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.099+0000 7ffa08ff9640 1 -- 172.21.15.62:0/1902355314 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa04032e20 con 0x5610080b9880 2024-09-12T00:03:22.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.100+0000 7ffa0b7fe640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa000a2df0 0x7ffa000a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.100+0000 7ffa12d9d640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa000a2df0 0x7ffa000a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.100+0000 7ffa0b7fe640 1 -- 172.21.15.62:0/277463186 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7ff9fc0cf990 con 0x7ffa14102570 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.100+0000 7ffa1add2740 1 -- 172.21.15.62:0/1902355314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080b9880 msgr2=0x5610080b9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.100+0000 7ffa1add2740 1 --2- 172.21.15.62:0/1902355314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080b9880 0x5610080b9cc0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009900 tx=0x7ffa04032c30 comp rx=0 tx=0).stop 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.100+0000 7ffa08ff9640 1 -- 172.21.15.62:0/1902355314 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04035070 con 0x5610080b9880 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.101+0000 7ffa12d9d640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa000a2df0 0x7ffa000a52b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009fd0 tx=0x7ffa0c009500 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.101+0000 7ffa1add2740 1 -- 172.21.15.62:0/1902355314 shutdown_connections 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.101+0000 7ffa1add2740 1 --2- 172.21.15.62:0/1902355314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080ccd20 0x5610080cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.101+0000 7ffa1add2740 1 --2- 172.21.15.62:0/1902355314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080b9880 0x5610080b9cc0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.101+0000 7ffa1add2740 1 --2- 172.21.15.62:0/1902355314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.101+0000 7ffa1add2740 1 -- 172.21.15.62:0/1902355314 >> 172.21.15.62:0/1902355314 conn(0x5610080b5a30 msgr2=0x5610080b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.102+0000 7ffa1add2740 1 -- 172.21.15.62:0/1902355314 shutdown_connections 2024-09-12T00:03:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.102+0000 7ffa1add2740 1 -- 172.21.15.62:0/1902355314 wait complete. 2024-09-12T00:03:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.102+0000 7ffa1add2740 1 -- start start 2024-09-12T00:03:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.103+0000 7ffa1add2740 1 Processor -- start 2024-09-12T00:03:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.103+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 0x5610080bfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.103+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080bcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.103+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 0x5610080bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.103+0000 7ffa1add2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5610080cfa90 con 0x5610080ccd20 2024-09-12T00:03:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.104+0000 7ffa0bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080bcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.104+0000 7ffa0bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080bcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33826/0 (socket says 172.21.15.62:33826) 2024-09-12T00:03:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.104+0000 7ffa0bfff640 1 -- 172.21.15.62:0/352092203 learned_addr learned my addr 172.21.15.62:0/352092203 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.104+0000 7ffa12d9d640 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 0x5610080bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.104+0000 7ffa1259c640 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 0x5610080bfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.104+0000 7ffa1add2740 1 -- 172.21.15.62:0/352092203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561007fda050 con 0x5610080b9880 2024-09-12T00:03:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.104+0000 7ffa1add2740 1 -- 172.21.15.62:0/352092203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610080cfc10 con 0x5610080baa20 2024-09-12T00:03:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.104+0000 7ffa1add2740 10 client.0 _ll_register_callbacks cb 0x56100801a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.105+0000 7ffa0bfff640 1 -- 172.21.15.62:0/352092203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 msgr2=0x5610080bfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.105+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 0x5610080bfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.105+0000 7ffa0bfff640 1 -- 172.21.15.62:0/352092203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 msgr2=0x5610080bd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.105+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 0x5610080bd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.105+0000 7ffa0bfff640 1 -- 172.21.15.62:0/352092203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561007d903c0 con 0x5610080baa20 2024-09-12T00:03:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.105+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080bcdd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc09bb50 tx=0x7ff9fc00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b5d613f11ee870 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.105+0000 7ff9f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:03:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.105+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/352092203 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc09b990 con 0x5610080baa20 2024-09-12T00:03:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.105+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/352092203 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7ff9fc09b2a0 con 0x5610080baa20 2024-09-12T00:03:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.106+0000 7ffa1add2740 1 -- 172.21.15.62:0/352092203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa04032e20 con 0x5610080baa20 2024-09-12T00:03:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.106+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/352092203 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc09b7b0 con 0x5610080baa20 2024-09-12T00:03:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.107+0000 7ffa1add2740 10 client.26833 fetch_fsmap learned FSMap version 34 2024-09-12T00:03:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.107+0000 7ffa1add2740 1 -- 172.21.15.62:0/352092203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=34}) -- 0x5610080db540 con 0x5610080baa20 2024-09-12T00:03:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.107+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/352092203 <== mon.1 v2:172.21.15.120:3300/0 4 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7ff9fc018b00 con 0x5610080baa20 2024-09-12T00:03:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.107+0000 7ffa1add2740 10 client.26833 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T00:03:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.107+0000 7ffa1add2740 10 client.26833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:03:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.107+0000 7ffa1add2740 20 client.26833 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.107+0000 7ffa1add2740 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5610080dc890 0x5610080dec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.108+0000 7ffa1add2740 4 client.26833 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:03:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.108+0000 7ffa1add2740 1 -- 172.21.15.62:0/352092203 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56100802c850 con 0x5610080dc890 2024-09-12T00:03:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.108+0000 7ffa1259c640 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5610080dc890 0x5610080dec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.109+0000 7ffa1259c640 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5610080dc890 0x5610080dec80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f17823e8477e91 server_cookie=aa96771ece75c67c in_seq=0 out_seq=0 2024-09-12T00:03:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.109+0000 7ff9f9ffb640 10 client.26833 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.109+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/352092203 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56100802c850 con 0x5610080dc890 2024-09-12T00:03:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.109+0000 7ff9f9ffb640 10 client.26833 handle_command_reply: tid=0 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.109+0000 7ffa1add2740 1 client.26833 shutdown 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 1 -- 172.21.15.62:0/352092203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 msgr2=0x5610080bcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080bcdd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc09bb50 tx=0x7ff9fc00cf60 comp rx=0 tx=0).stop 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 1 -- 172.21.15.62:0/352092203 shutdown_connections 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5610080dc890 0x5610080dec80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ff9f9ffb640 0 client.26833 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 0x5610080bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080bcdd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 1 --2- 172.21.15.62:0/352092203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 0x5610080bfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 1 -- 172.21.15.62:0/352092203 >> 172.21.15.62:0/352092203 conn(0x5610080b5a30 msgr2=0x5610080bb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 1 -- 172.21.15.62:0/352092203 shutdown_connections 2024-09-12T00:03:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 1 -- 172.21.15.62:0/352092203 wait complete. 2024-09-12T00:03:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.110+0000 7ffa1add2740 20 client.26833 trim_cache size 0 max 16384 2024-09-12T00:03:22.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.141+0000 7ffa1add2740 1 Processor -- start 2024-09-12T00:03:22.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.141+0000 7ffa1add2740 1 -- start start 2024-09-12T00:03:22.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.141+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080b9880 0x5610080d8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.141+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080baa20 0x5610080b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080ccd20 0x5610080bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1add2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141ae080 con 0x5610080baa20 2024-09-12T00:03:22.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1add2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141ab8f0 con 0x5610080ccd20 2024-09-12T00:03:22.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1add2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141a9760 con 0x5610080b9880 2024-09-12T00:03:22.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080b9880 0x5610080d8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080b9880 0x5610080d8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33832/0 (socket says 172.21.15.62:33832) 2024-09-12T00:03:22.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1259c640 1 -- 172.21.15.62:0/1481175622 learned_addr learned my addr 172.21.15.62:0/1481175622 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:22.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/1481175622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080baa20 0x5610080b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1259c640 1 -- 172.21.15.62:0/1481175622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080ccd20 msgr2=0x5610080bc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:22.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1259c640 1 --2- 172.21.15.62:0/1481175622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080ccd20 0x5610080bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1259c640 1 -- 172.21.15.62:0/1481175622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080baa20 msgr2=0x5610080b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:22.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1259c640 1 --2- 172.21.15.62:0/1481175622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080baa20 0x5610080b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1259c640 1 -- 172.21.15.62:0/1481175622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610080ba290 con 0x5610080b9880 2024-09-12T00:03:22.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/1481175622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080baa20 0x5610080b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:22.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.142+0000 7ffa1259c640 1 --2- 172.21.15.62:0/1481175622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080b9880 0x5610080d8de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa04032fd0 tx=0x7ffa0402a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd5a7d20aa333006 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:22.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.143+0000 7ff9fbfff640 1 -- 172.21.15.62:0/1481175622 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0403f040 con 0x5610080b9880 2024-09-12T00:03:22.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.143+0000 7ff9fbfff640 1 -- 172.21.15.62:0/1481175622 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa040317b0 con 0x5610080b9880 2024-09-12T00:03:22.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.143+0000 7ff9fbfff640 1 -- 172.21.15.62:0/1481175622 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04035040 con 0x5610080b9880 2024-09-12T00:03:22.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.144+0000 7ffa1add2740 1 -- 172.21.15.62:0/1481175622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080b9880 msgr2=0x5610080d8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:22.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.144+0000 7ffa1add2740 1 --2- 172.21.15.62:0/1481175622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080b9880 0x5610080d8de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa04032fd0 tx=0x7ffa0402a4d0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.145+0000 7ffa1add2740 1 -- 172.21.15.62:0/1481175622 shutdown_connections 2024-09-12T00:03:22.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.145+0000 7ffa1add2740 1 --2- 172.21.15.62:0/1481175622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080ccd20 0x5610080bc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.145+0000 7ffa1add2740 1 --2- 172.21.15.62:0/1481175622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080baa20 0x5610080b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.145+0000 7ffa1add2740 1 --2- 172.21.15.62:0/1481175622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080b9880 0x5610080d8de0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.145+0000 7ffa1add2740 1 -- 172.21.15.62:0/1481175622 >> 172.21.15.62:0/1481175622 conn(0x5610080b5a30 msgr2=0x5610080b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:22.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.146+0000 7ffa1add2740 1 -- 172.21.15.62:0/1481175622 shutdown_connections 2024-09-12T00:03:22.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.146+0000 7ffa1add2740 1 -- 172.21.15.62:0/1481175622 wait complete. 2024-09-12T00:03:22.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.146+0000 7ffa1add2740 1 -- start start 2024-09-12T00:03:22.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.147+0000 7ffa1add2740 1 Processor -- start 2024-09-12T00:03:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.147+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 0x5610080d53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.147+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.147+0000 7ffa1add2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 0x5610080e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.147+0000 7ffa1add2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141a9db0 con 0x5610080ccd20 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.147+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 0x5610080d53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.147+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 0x5610080d53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:54608/0 (socket says 172.21.15.62:54608) 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.147+0000 7ffa1259c640 1 -- 172.21.15.62:0/3417409341 learned_addr learned my addr 172.21.15.62:0/3417409341 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.147+0000 7ffa1add2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141ab220 con 0x5610080b9880 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa1add2740 1 -- 172.21.15.62:0/3417409341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141aa480 con 0x5610080baa20 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa12d9d640 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 0x5610080e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa1add2740 10 client.0 _ll_register_callbacks cb 0x7ff9fc00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa1259c640 1 -- 172.21.15.62:0/3417409341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 msgr2=0x5610080e3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa1259c640 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080e3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa1259c640 1 -- 172.21.15.62:0/3417409341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 msgr2=0x5610080e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa1259c640 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 0x5610080e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa1259c640 1 -- 172.21.15.62:0/3417409341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610080e4610 con 0x5610080b9880 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa12d9d640 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 0x5610080e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ffa1259c640 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 0x5610080d53a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffa04032c70 tx=0x7ffa0404f6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731b7ee5d7e41ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.148+0000 7ff9f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.149+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/3417409341 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0403f070 con 0x5610080b9880 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.149+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/3417409341 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7ffa0404fae0 con 0x5610080b9880 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.149+0000 7ffa1add2740 1 -- 172.21.15.62:0/3417409341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa040317b0 con 0x5610080b9880 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.149+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/3417409341 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffa04066050 con 0x5610080b9880 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.149+0000 7ffa1add2740 10 client.26924 fetch_fsmap learned FSMap version 34 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.150+0000 7ffa1add2740 1 -- 172.21.15.62:0/3417409341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5610080d3c60 con 0x5610080b9880 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.150+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/3417409341 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa0404fce0 con 0x5610080b9880 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.150+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/3417409341 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7ffa04056030 con 0x5610080b9880 2024-09-12T00:03:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.150+0000 7ffa1add2740 10 client.26924 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T00:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.150+0000 7ffa1add2740 10 client.26924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.150+0000 7ffa1add2740 20 client.26924 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.150+0000 7ffa1add2740 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5610080dc890 0x561008149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.150+0000 7ffa1add2740 4 client.26924 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.150+0000 7ffa1add2740 1 -- 172.21.15.62:0/3417409341 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ffa040317b0 con 0x5610080dc890 2024-09-12T00:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.151+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5610080dc890 0x561008149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.151+0000 7ffa0bfff640 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5610080dc890 0x561008149500 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7fb95a4cc4f288c server_cookie=c254d1299390cf5 in_seq=0 out_seq=0 2024-09-12T00:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.152+0000 7ff9f9ffb640 10 client.26924 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.152+0000 7ff9f9ffb640 1 -- 172.21.15.62:0/3417409341 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7ffa040317b0 con 0x5610080dc890 2024-09-12T00:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.152+0000 7ff9f9ffb640 10 client.26924 handle_command_reply: tid=0 2024-09-12T00:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.152+0000 7ffa1add2740 1 client.26924 shutdown 2024-09-12T00:03:22.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.152+0000 7ffa1add2740 1 -- 172.21.15.62:0/3417409341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 msgr2=0x5610080d53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:22.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.152+0000 7ffa1add2740 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 0x5610080d53a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffa04032c70 tx=0x7ffa0404f6b0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.154+0000 7ffa1add2740 1 -- 172.21.15.62:0/3417409341 shutdown_connections 2024-09-12T00:03:22.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.154+0000 7ffa1add2740 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5610080dc890 0x561008149500 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.154+0000 7ff9f9ffb640 0 client.26924 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:22.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.154+0000 7ffa1add2740 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610080ccd20 0x5610080e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.154+0000 7ffa1add2740 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610080baa20 0x5610080e3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.154+0000 7ffa1add2740 1 --2- 172.21.15.62:0/3417409341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610080b9880 0x5610080d53a0 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffa04032c70 tx=0x7ffa0404f6b0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.154+0000 7ffa1add2740 1 -- 172.21.15.62:0/3417409341 >> 172.21.15.62:0/3417409341 conn(0x5610080b5a30 msgr2=0x5610080ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.159+0000 7ffa1add2740 1 -- 172.21.15.62:0/3417409341 shutdown_connections 2024-09-12T00:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.162+0000 7ffa1add2740 1 -- 172.21.15.62:0/3417409341 wait complete. 2024-09-12T00:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.162+0000 7ffa1add2740 20 client.26924 trim_cache size 0 max 16384 2024-09-12T00:03:22.166 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:03:22.166 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:03:22.166 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:03:22.166 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:03:22.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.170+0000 7ffa189f9640 1 -- 172.21.15.62:0/277463186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa000a2df0 msgr2=0x7ffa000a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:22.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.170+0000 7ffa189f9640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa000a2df0 0x7ffa000a52b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009fd0 tx=0x7ffa0c009500 comp rx=0 tx=0).stop 2024-09-12T00:03:22.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.170+0000 7ffa189f9640 1 -- 172.21.15.62:0/277463186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa14102570 msgr2=0x7ffa141998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:22.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.170+0000 7ffa189f9640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa14102570 0x7ffa141998e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00e9f0 tx=0x7ff9fc00eeb0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.175+0000 7ffa189f9640 1 -- 172.21.15.62:0/277463186 shutdown_connections 2024-09-12T00:03:22.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.175+0000 7ffa189f9640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa000a2df0 0x7ffa000a52b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.175+0000 7ffa189f9640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa14102ef0 0x7ffa141a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.175+0000 7ffa189f9640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa14102570 0x7ffa141998e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.175+0000 7ffa189f9640 1 --2- 172.21.15.62:0/277463186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa14100e20 0x7ffa141993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:22.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.175+0000 7ffa189f9640 1 -- 172.21.15.62:0/277463186 >> 172.21.15.62:0/277463186 conn(0x7ffa140fc7d0 msgr2=0x7ffa14104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:22.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.176+0000 7ffa189f9640 1 -- 172.21.15.62:0/277463186 shutdown_connections 2024-09-12T00:03:22.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:22.176+0000 7ffa189f9640 1 -- 172.21.15.62:0/277463186 wait complete. 2024-09-12T00:03:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 640.0 s, 2726.4 tps, lat 36.484 ms stddev 40.903 2024-09-12T00:03:23.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:22 smithi083 ceph-mon[34805]: pgmap v351: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.0 MiB/s wr, 649 op/s 2024-09-12T00:03:23.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:22 smithi062 ceph-mon[30958]: pgmap v351: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.0 MiB/s wr, 649 op/s 2024-09-12T00:03:23.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:22 smithi120 ceph-mon[35182]: pgmap v351: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.0 MiB/s wr, 649 op/s 2024-09-12T00:03:23.579 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:03:23.579 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:03:23.982 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:03:24.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.824+0000 7f9132375640 1 -- 172.21.15.62:0/1845883757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f912c100e20 msgr2=0x7f912c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:24.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.824+0000 7f9132375640 1 --2- 172.21.15.62:0/1845883757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f912c100e20 0x7f912c101200 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f911c009930 tx=0x7f911c02a4f0 comp rx=0 tx=0).stop 2024-09-12T00:03:24.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.824+0000 7f9132375640 1 -- 172.21.15.62:0/1845883757 shutdown_connections 2024-09-12T00:03:24.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.824+0000 7f9132375640 1 --2- 172.21.15.62:0/1845883757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912c102ef0 0x7f912c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:24.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.824+0000 7f9132375640 1 --2- 172.21.15.62:0/1845883757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f912c102570 0x7f912c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:24.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.824+0000 7f9132375640 1 --2- 172.21.15.62:0/1845883757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f912c100e20 0x7f912c101200 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:24.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.824+0000 7f9132375640 1 -- 172.21.15.62:0/1845883757 >> 172.21.15.62:0/1845883757 conn(0x7f912c0fc7d0 msgr2=0x7f912c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:24.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.824+0000 7f9132375640 1 -- 172.21.15.62:0/1845883757 shutdown_connections 2024-09-12T00:03:24.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.824+0000 7f9132375640 1 -- 172.21.15.62:0/1845883757 wait complete. 2024-09-12T00:03:24.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9132375640 1 Processor -- start 2024-09-12T00:03:24.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9132375640 1 -- start start 2024-09-12T00:03:24.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9132375640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f912c100e20 0x7f912c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:24.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9132375640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912c102570 0x7f912c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:24.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9132375640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f912c102ef0 0x7f912c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:24.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9132375640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f912c111010 con 0x7f912c102570 2024-09-12T00:03:24.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9132375640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f912c110e90 con 0x7f912c100e20 2024-09-12T00:03:24.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9132375640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f912c111190 con 0x7f912c102ef0 2024-09-12T00:03:24.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f912bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f912c100e20 0x7f912c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:24.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9130912640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f912c102ef0 0x7f912c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:24.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9123fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912c102570 0x7f912c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:24.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f912bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f912c100e20 0x7f912c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:54628/0 (socket says 172.21.15.62:54628) 2024-09-12T00:03:24.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f9130912640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f912c102ef0 0x7f912c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33848/0 (socket says 172.21.15.62:33848) 2024-09-12T00:03:24.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.825+0000 7f912bfff640 1 -- 172.21.15.62:0/3550490908 learned_addr learned my addr 172.21.15.62:0/3550490908 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:24.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f912bfff640 1 -- 172.21.15.62:0/3550490908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f912c102ef0 msgr2=0x7f912c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:24.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f912bfff640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f912c102ef0 0x7f912c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:24.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f912bfff640 1 -- 172.21.15.62:0/3550490908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912c102570 msgr2=0x7f912c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:24.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f912bfff640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912c102570 0x7f912c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:24.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f912bfff640 1 -- 172.21.15.62:0/3550490908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c19d570 con 0x7f912c100e20 2024-09-12T00:03:24.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f912bfff640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f912c100e20 0x7f912c1985c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f911c02aa10 tx=0x7f911c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:24.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f9129ffb640 1 -- 172.21.15.62:0/3550490908 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c03f070 con 0x7f912c100e20 2024-09-12T00:03:24.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f9132375640 1 -- 172.21.15.62:0/3550490908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912c19d800 con 0x7f912c100e20 2024-09-12T00:03:24.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f9132375640 1 -- 172.21.15.62:0/3550490908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912c0fcbb0 con 0x7f912c100e20 2024-09-12T00:03:24.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f9129ffb640 1 -- 172.21.15.62:0/3550490908 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f911c031c90 con 0x7f912c100e20 2024-09-12T00:03:24.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.826+0000 7f9129ffb640 1 -- 172.21.15.62:0/3550490908 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c035030 con 0x7f912c100e20 2024-09-12T00:03:24.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.828+0000 7f9132375640 1 -- 172.21.15.62:0/3550490908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c005290 con 0x7f912c100e20 2024-09-12T00:03:24.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.829+0000 7f9129ffb640 1 -- 172.21.15.62:0/3550490908 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f911c042070 con 0x7f912c100e20 2024-09-12T00:03:24.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.830+0000 7f9129ffb640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91180a2d70 0x7f91180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:24.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.830+0000 7f9129ffb640 1 -- 172.21.15.62:0/3550490908 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f911c0efd70 con 0x7f912c100e20 2024-09-12T00:03:24.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.831+0000 7f9123fff640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91180a2d70 0x7f91180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:24.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.831+0000 7f9123fff640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91180a2d70 0x7f91180a5230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f912c1073d0 tx=0x7f911400f040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:24.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:24.835+0000 7f9129ffb640 1 -- 172.21.15.62:0/3550490908 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f911c0bc340 con 0x7f912c100e20 2024-09-12T00:03:24.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:24 smithi062 ceph-mon[30958]: pgmap v352: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.9 MiB/s wr, 764 op/s 2024-09-12T00:03:25.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:24 smithi083 ceph-mon[34805]: pgmap v352: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.9 MiB/s wr, 764 op/s 2024-09-12T00:03:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.193+0000 7f9132375640 1 -- 172.21.15.62:0/3550490908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f910c005850 con 0x7f912c100e20 2024-09-12T00:03:25.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.193+0000 7f9129ffb640 1 -- 172.21.15.62:0/3550490908 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11913 (secure 0 0 0) 0x7f911c0c0640 con 0x7f912c100e20 2024-09-12T00:03:25.193 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:03:25.194 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":34,"btime":"2024-09-12T00:03:12:880967+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:03:12.082806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:03:25.194 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 34 2024-09-12T00:03:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 -- 172.21.15.62:0/3550490908 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91180a2d70 msgr2=0x7f91180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91180a2d70 0x7f91180a5230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f912c1073d0 tx=0x7f911400f040 comp rx=0 tx=0).stop 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 -- 172.21.15.62:0/3550490908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f912c100e20 msgr2=0x7f912c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f912c100e20 0x7f912c1985c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f911c02aa10 tx=0x7f911c003b90 comp rx=0 tx=0).stop 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 -- 172.21.15.62:0/3550490908 shutdown_connections 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f91180a2d70 0x7f91180a5230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f912c102ef0 0x7f912c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912c102570 0x7f912c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 --2- 172.21.15.62:0/3550490908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f912c100e20 0x7f912c1985c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 -- 172.21.15.62:0/3550490908 >> 172.21.15.62:0/3550490908 conn(0x7f912c0fc7d0 msgr2=0x7f912c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 -- 172.21.15.62:0/3550490908 shutdown_connections 2024-09-12T00:03:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:25.198+0000 7f9132375640 1 -- 172.21.15.62:0/3550490908 wait complete. 2024-09-12T00:03:25.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:24 smithi120 ceph-mon[35182]: pgmap v352: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.9 MiB/s wr, 764 op/s 2024-09-12T00:03:26.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:03:26.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:25 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3550490908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:03:26.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:25 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3550490908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:03:26.516 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:25 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3550490908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:03:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:03:26.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.966+0000 7fbdc9c17640 1 -- 172.21.15.62:0/3843904857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc410a560 msgr2=0x7fbdc410a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:26.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.966+0000 7fbdc9c17640 1 --2- 172.21.15.62:0/3843904857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc410a560 0x7fbdc410a940 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009900 tx=0x7fbdb402a410 comp rx=0 tx=0).stop 2024-09-12T00:03:26.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.966+0000 7fbdc9c17640 1 -- 172.21.15.62:0/3843904857 shutdown_connections 2024-09-12T00:03:26.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.966+0000 7fbdc9c17640 1 --2- 172.21.15.62:0/3843904857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc41139e0 0x7fbdc4115e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.966+0000 7fbdc9c17640 1 --2- 172.21.15.62:0/3843904857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdc410ae80 0x7fbdc4113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.966+0000 7fbdc9c17640 1 --2- 172.21.15.62:0/3843904857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc410a560 0x7fbdc410a940 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.966+0000 7fbdc9c17640 1 -- 172.21.15.62:0/3843904857 >> 172.21.15.62:0/3843904857 conn(0x7fbdc406e2d0 msgr2=0x7fbdc406e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.967+0000 7fbdc9c17640 1 -- 172.21.15.62:0/3843904857 shutdown_connections 2024-09-12T00:03:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.967+0000 7fbdc9c17640 1 -- 172.21.15.62:0/3843904857 wait complete. 2024-09-12T00:03:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.968+0000 7fbdc9c17640 1 Processor -- start 2024-09-12T00:03:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.968+0000 7fbdc9c17640 1 -- start start 2024-09-12T00:03:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.968+0000 7fbdc9c17640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc410a560 0x7fbdc41a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.968+0000 7fbdc9c17640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc410ae80 0x7fbdc41a6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.968+0000 7fbdc9c17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdc41139e0 0x7fbdc41a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.968+0000 7fbdc9c17640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4119720 con 0x7fbdc410a560 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.968+0000 7fbdc9c17640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41195a0 con 0x7fbdc410ae80 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.968+0000 7fbdc9c17640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41198a0 con 0x7fbdc41139e0 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc410a560 0x7fbdc41a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc410a560 0x7fbdc41a6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54632/0 (socket says 172.21.15.62:54632) 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc37fe640 1 -- 172.21.15.62:0/1870142511 learned_addr learned my addr 172.21.15.62:0/1870142511 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc3fff640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdc41139e0 0x7fbdc41a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc410ae80 0x7fbdc41a6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc37fe640 1 -- 172.21.15.62:0/1870142511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdc41139e0 msgr2=0x7fbdc41a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc37fe640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdc41139e0 0x7fbdc41a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc37fe640 1 -- 172.21.15.62:0/1870142511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc410ae80 msgr2=0x7fbdc41a6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc37fe640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc410ae80 0x7fbdc41a6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc37fe640 1 -- 172.21.15.62:0/1870142511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc41a16c0 con 0x7fbdc410a560 2024-09-12T00:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc3fff640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdc41139e0 0x7fbdc41a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc37fe640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc410a560 0x7fbdc41a6a90 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4004770 tx=0x7fbdb4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc0ff9640 1 -- 172.21.15.62:0/1870142511 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb403f070 con 0x7fbdc410a560 2024-09-12T00:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc0ff9640 1 -- 172.21.15.62:0/1870142511 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb4031ca0 con 0x7fbdc410a560 2024-09-12T00:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.969+0000 7fbdc0ff9640 1 -- 172.21.15.62:0/1870142511 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4035030 con 0x7fbdc410a560 2024-09-12T00:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.970+0000 7fbdc9c17640 1 -- 172.21.15.62:0/1870142511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc41a1950 con 0x7fbdc410a560 2024-09-12T00:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.970+0000 7fbdc9c17640 1 -- 172.21.15.62:0/1870142511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc41ad7f0 con 0x7fbdc410a560 2024-09-12T00:03:26.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.973+0000 7fbdcbff0740 1 Processor -- start 2024-09-12T00:03:26.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.973+0000 7fbdcbff0740 1 -- start start 2024-09-12T00:03:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6440 0x55b0864e6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6df0 0x55b0864eb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b0864422f0 con 0x55b0864e6df0 2024-09-12T00:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b0863e9cf0 con 0x55b0864e6440 2024-09-12T00:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b08643e2b0 con 0x55b0864eba40 2024-09-12T00:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6440 0x55b0864e6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc0ff9640 1 -- 172.21.15.62:0/1870142511 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbdb4042070 con 0x7fbdc410a560 2024-09-12T00:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6440 0x55b0864e6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:54662/0 (socket says 172.21.15.62:54662) 2024-09-12T00:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc3fff640 1 -- 172.21.15.62:0/835771784 learned_addr learned my addr 172.21.15.62:0/835771784 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:26.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc37fe640 1 --2- 172.21.15.62:0/835771784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6df0 0x55b0864eb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc2ffd640 1 -- 172.21.15.62:0/835771784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6440 msgr2=0x55b0864e6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/835771784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6440 0x55b0864e6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc2ffd640 1 -- 172.21.15.62:0/835771784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6df0 msgr2=0x55b0864eb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/835771784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6df0 0x55b0864eb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc2ffd640 1 -- 172.21.15.62:0/835771784 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0863ce3a0 con 0x55b0864eba40 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc3fff640 1 --2- 172.21.15.62:0/835771784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6440 0x55b0864e6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.975+0000 7fbdc37fe640 1 --2- 172.21.15.62:0/835771784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6df0 0x55b0864eb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.976+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/835771784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864ebef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbdc41135f0 tx=0x7fbdb8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f482c1081696b141 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.976+0000 7fbda9ffb640 1 -- 172.21.15.62:0/835771784 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8004230 con 0x55b0864eba40 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.976+0000 7fbda9ffb640 1 -- 172.21.15.62:0/835771784 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb8009e10 con 0x55b0864eba40 2024-09-12T00:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.976+0000 7fbda9ffb640 1 -- 172.21.15.62:0/835771784 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80197a0 con 0x55b0864eba40 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.976+0000 7fbdc0ff9640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbdac0a2cf0 0x7fbdac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.976+0000 7fbdc3fff640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbdac0a2cf0 0x7fbdac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.976+0000 7fbdc0ff9640 1 -- 172.21.15.62:0/1870142511 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fbdb40efa20 con 0x7fbdc410a560 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdcbff0740 1 -- 172.21.15.62:0/835771784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 msgr2=0x55b0864ebef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/835771784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864ebef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbdc41135f0 tx=0x7fbdb8011810 comp rx=0 tx=0).stop 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdcbff0740 1 -- 172.21.15.62:0/835771784 shutdown_connections 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/835771784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864ebef0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/835771784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6df0 0x55b0864eb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/835771784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6440 0x55b0864e6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdcbff0740 1 -- 172.21.15.62:0/835771784 >> 172.21.15.62:0/835771784 conn(0x55b0864a5310 msgr2=0x55b0864d86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdcbff0740 1 -- 172.21.15.62:0/835771784 shutdown_connections 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdc3fff640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbdac0a2cf0 0x7fbdac0a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc012c30 tx=0x7fbdbc009500 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdcbff0740 1 -- 172.21.15.62:0/835771784 wait complete. 2024-09-12T00:03:26.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.977+0000 7fbdcbff0740 1 -- start start 2024-09-12T00:03:26.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.978+0000 7fbdcbff0740 1 Processor -- start 2024-09-12T00:03:26.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.978+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.978+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6df0 0x55b0864ef060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.978+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864ef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.978+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b0864f28d0 con 0x55b0864e6440 2024-09-12T00:03:26.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.978+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b0863fd060 con 0x55b0864e6df0 2024-09-12T00:03:26.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.978+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b0864f2a50 con 0x55b0864eba40 2024-09-12T00:03:26.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.978+0000 7fbdcbff0740 10 client.0 _ll_register_callbacks cb 0x55b0861508c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6df0 0x55b0864ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54662/0 (socket says 172.21.15.62:54662) 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc37fe640 1 -- 172.21.15.62:0/3906657529 learned_addr learned my addr 172.21.15.62:0/3906657529 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc3fff640 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864ef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc2ffd640 1 -- 172.21.15.62:0/3906657529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 msgr2=0x55b0864ef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864ef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc2ffd640 1 -- 172.21.15.62:0/3906657529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 msgr2=0x55b0864f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc2ffd640 1 -- 172.21.15.62:0/3906657529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0861b33c0 con 0x55b0864e6df0 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc3fff640 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864ef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:03:26.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6df0 0x55b0864ef060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbdc41135f0 tx=0x7fbdb8004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d46703ce9cbd2f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.979+0000 7fbda2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.980+0000 7fbda2ffd640 1 -- 172.21.15.62:0/3906657529 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8027070 con 0x55b0864e6df0 2024-09-12T00:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.980+0000 7fbdcbff0740 1 -- 172.21.15.62:0/3906657529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb8009e10 con 0x55b0864e6df0 2024-09-12T00:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.980+0000 7fbda2ffd640 1 -- 172.21.15.62:0/3906657529 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fbdb8003a00 con 0x55b0864e6df0 2024-09-12T00:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.980+0000 7fbda2ffd640 1 -- 172.21.15.62:0/3906657529 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbdb8033050 con 0x55b0864e6df0 2024-09-12T00:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.981+0000 7fbdcbff0740 10 client.26942 fetch_fsmap learned FSMap version 34 2024-09-12T00:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.981+0000 7fbdcbff0740 1 -- 172.21.15.62:0/3906657529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b0864fe190 con 0x55b0864e6df0 2024-09-12T00:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.982+0000 7fbda2ffd640 1 -- 172.21.15.62:0/3906657529 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb8009e10 con 0x55b0864e6df0 2024-09-12T00:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.982+0000 7fbda2ffd640 1 -- 172.21.15.62:0/3906657529 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fbdb8004b20 con 0x55b0864e6df0 2024-09-12T00:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.982+0000 7fbdcbff0740 10 client.26942 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T00:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.982+0000 7fbdcbff0740 10 client.26942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.982+0000 7fbdcbff0740 20 client.26942 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.982+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b0864ffa50 0x55b086501eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.982+0000 7fbdcbff0740 4 client.26942 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.982+0000 7fbdcbff0740 1 -- 172.21.15.62:0/3906657529 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b08644f860 con 0x55b0864ffa50 2024-09-12T00:03:26.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.984+0000 7fbdc37fe640 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b0864ffa50 0x55b086501eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:26.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.985+0000 7fbdc37fe640 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b0864ffa50 0x55b086501eb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7b7f7cc3eff16d9 server_cookie=b270402d854c99e0 in_seq=0 out_seq=0 2024-09-12T00:03:26.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.985+0000 7fbda2ffd640 10 client.26942 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:03:26.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.985+0000 7fbda2ffd640 1 -- 172.21.15.62:0/3906657529 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b08644f860 con 0x55b0864ffa50 2024-09-12T00:03:26.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.985+0000 7fbda2ffd640 10 client.26942 handle_command_reply: tid=0 2024-09-12T00:03:26.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.986+0000 7fbdcbff0740 1 client.26942 shutdown 2024-09-12T00:03:26.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.986+0000 7fbdcbff0740 1 -- 172.21.15.62:0/3906657529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6df0 msgr2=0x55b0864ef060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:26.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.986+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6df0 0x55b0864ef060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbdc41135f0 tx=0x7fbdb8004560 comp rx=0 tx=0).stop 2024-09-12T00:03:26.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.986+0000 7fbdcbff0740 1 -- 172.21.15.62:0/3906657529 shutdown_connections 2024-09-12T00:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.986+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b0864ffa50 0x55b086501eb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.986+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864ef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.986+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864e6df0 0x55b0864ef060 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbdc41135f0 tx=0x7fbdb8004560 comp rx=0 tx=0).stop 2024-09-12T00:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.986+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/3906657529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.986+0000 7fbdcbff0740 1 -- 172.21.15.62:0/3906657529 >> 172.21.15.62:0/3906657529 conn(0x55b0864a5310 msgr2=0x55b0864e5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.986+0000 7fbda2ffd640 0 client.26942 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:03:26.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.987+0000 7fbdcbff0740 1 -- 172.21.15.62:0/3906657529 shutdown_connections 2024-09-12T00:03:26.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.987+0000 7fbdcbff0740 1 -- 172.21.15.62:0/3906657529 wait complete. 2024-09-12T00:03:26.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:26.988+0000 7fbdcbff0740 20 client.26942 trim_cache size 0 max 16384 2024-09-12T00:03:27.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.016+0000 7fbdcbff0740 1 Processor -- start 2024-09-12T00:03:27.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.016+0000 7fbdcbff0740 1 -- start start 2024-09-12T00:03:27.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.017+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:27.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.017+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864eba40 0x55b0864ea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:27.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.017+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864ef690 0x55b0864f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:27.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.017+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41b6860 con 0x55b0864e6440 2024-09-12T00:03:27.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.017+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41b40d0 con 0x55b0864eba40 2024-09-12T00:03:27.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.017+0000 7fbdc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864ef690 0x55b0864f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:27.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.017+0000 7fbdc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864ef690 0x55b0864f08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33886/0 (socket says 172.21.15.62:33886) 2024-09-12T00:03:27.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.017+0000 7fbdc3fff640 1 -- 172.21.15.62:0/94871404 learned_addr learned my addr 172.21.15.62:0/94871404 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:27.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.018+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/94871404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864eba40 0x55b0864ea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:27.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.019+0000 7fbdc37fe640 1 --2- 172.21.15.62:0/94871404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.022+0000 7fbdcbff0740 1 -- 172.21.15.62:0/94871404 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41b1f40 con 0x55b0864ef690 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.023+0000 7fbdc2ffd640 1 -- 172.21.15.62:0/94871404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864ef690 msgr2=0x55b0864f08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.023+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/94871404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864ef690 0x55b0864f08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.023+0000 7fbdc2ffd640 1 -- 172.21.15.62:0/94871404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 msgr2=0x55b0864fbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.023+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/94871404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864fbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.023+0000 7fbdc2ffd640 1 -- 172.21.15.62:0/94871404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0864d8c20 con 0x55b0864eba40 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.023+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/94871404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864eba40 0x55b0864ea1f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55b0864f0610 tx=0x7fbdb8005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=913400ed95842da3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.025+0000 7fbda97fa640 1 -- 172.21.15.62:0/94871404 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8027070 con 0x55b0864eba40 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.025+0000 7fbda97fa640 1 -- 172.21.15.62:0/94871404 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb8019a70 con 0x55b0864eba40 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.025+0000 7fbda97fa640 1 -- 172.21.15.62:0/94871404 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8005260 con 0x55b0864eba40 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.026+0000 7fbdcbff0740 1 -- 172.21.15.62:0/94871404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864eba40 msgr2=0x55b0864ea1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:27.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.026+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/94871404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864eba40 0x55b0864ea1f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55b0864f0610 tx=0x7fbdb8005990 comp rx=0 tx=0).stop 2024-09-12T00:03:27.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.027+0000 7fbdcbff0740 1 -- 172.21.15.62:0/94871404 shutdown_connections 2024-09-12T00:03:27.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.027+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/94871404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864ef690 0x55b0864f08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.027+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/94871404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864eba40 0x55b0864ea1f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.027+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/94871404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.028+0000 7fbdcbff0740 1 -- 172.21.15.62:0/94871404 >> 172.21.15.62:0/94871404 conn(0x55b0864a5310 msgr2=0x55b0864ed930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:27.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.031+0000 7fbdcbff0740 1 -- 172.21.15.62:0/94871404 shutdown_connections 2024-09-12T00:03:27.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.031+0000 7fbdcbff0740 1 -- 172.21.15.62:0/94871404 wait complete. 2024-09-12T00:03:27.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.032+0000 7fbdcbff0740 1 -- start start 2024-09-12T00:03:27.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.032+0000 7fbdcbff0740 1 Processor -- start 2024-09-12T00:03:27.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:27.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:27.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdcbff0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864ef690 0x55b0865067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:27.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41b2590 con 0x55b0864e6440 2024-09-12T00:03:27.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41b3a00 con 0x55b0864ef690 2024-09-12T00:03:27.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdcbff0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41b2c60 con 0x55b0864eba40 2024-09-12T00:03:27.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdcbff0740 10 client.0 _ll_register_callbacks cb 0x7fbdb8011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:27.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdc37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:27.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdc3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864ef690 0x55b0865067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:27.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdc37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864f7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54680/0 (socket says 172.21.15.62:54680) 2024-09-12T00:03:27.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.033+0000 7fbdc37fe640 1 -- 172.21.15.62:0/2690546265 learned_addr learned my addr 172.21.15.62:0/2690546265 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.034+0000 7fbdc2ffd640 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.034+0000 7fbdc3fff640 1 -- 172.21.15.62:0/2690546265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 msgr2=0x55b0864f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.034+0000 7fbdc3fff640 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.034+0000 7fbdc3fff640 1 -- 172.21.15.62:0/2690546265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 msgr2=0x55b0864f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.034+0000 7fbdc3fff640 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.034+0000 7fbdc3fff640 1 -- 172.21.15.62:0/2690546265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b086506cf0 con 0x55b0864ef690 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.034+0000 7fbdc3fff640 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864ef690 0x55b0865067b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00fc20 tx=0x7fbdbc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=475c3f0354ea0fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.035+0000 7fbda2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.035+0000 7fbda2ffd640 1 -- 172.21.15.62:0/2690546265 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc02f060 con 0x55b0864ef690 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.035+0000 7fbdcbff0740 1 -- 172.21.15.62:0/2690546265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb8019a70 con 0x55b0864ef690 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.035+0000 7fbda2ffd640 1 -- 172.21.15.62:0/2690546265 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fbdbc003970 con 0x55b0864ef690 2024-09-12T00:03:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.036+0000 7fbda2ffd640 1 -- 172.21.15.62:0/2690546265 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbdbc011e20 con 0x55b0864ef690 2024-09-12T00:03:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.036+0000 7fbdcbff0740 10 client.26948 fetch_fsmap learned FSMap version 34 2024-09-12T00:03:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.036+0000 7fbdcbff0740 1 -- 172.21.15.62:0/2690546265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b086569e70 con 0x55b0864ef690 2024-09-12T00:03:27.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.039+0000 7fbda2ffd640 1 -- 172.21.15.62:0/2690546265 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb8019a70 con 0x55b0864ef690 2024-09-12T00:03:27.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.039+0000 7fbda2ffd640 1 -- 172.21.15.62:0/2690546265 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fbdbc029030 con 0x55b0864ef690 2024-09-12T00:03:27.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.040+0000 7fbdcbff0740 10 client.26948 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T00:03:27.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.040+0000 7fbdcbff0740 10 client.26948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:03:27.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.040+0000 7fbdcbff0740 20 client.26948 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:27.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.040+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b08656a570 0x55b08656c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:27.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.040+0000 7fbdcbff0740 4 client.26948 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:03:27.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.040+0000 7fbdcbff0740 1 -- 172.21.15.62:0/2690546265 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b086449d70 con 0x55b08656a570 2024-09-12T00:03:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.041+0000 7fbdc37fe640 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b08656a570 0x55b08656c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.042+0000 7fbdc37fe640 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b08656a570 0x55b08656c990 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37bd9a66a2d21340 server_cookie=267e698bd08d8f9b in_seq=0 out_seq=0 2024-09-12T00:03:27.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.043+0000 7fbda2ffd640 10 client.26948 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:03:27.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.043+0000 7fbda2ffd640 1 -- 172.21.15.62:0/2690546265 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b086449d70 con 0x55b08656a570 2024-09-12T00:03:27.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.043+0000 7fbda2ffd640 10 client.26948 handle_command_reply: tid=0 2024-09-12T00:03:27.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.043+0000 7fbdcbff0740 1 client.26948 shutdown 2024-09-12T00:03:27.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.044+0000 7fbdcbff0740 1 -- 172.21.15.62:0/2690546265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864ef690 msgr2=0x55b0865067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:27.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.044+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864ef690 0x55b0865067b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00fc20 tx=0x7fbdbc004ab0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.044+0000 7fbdcbff0740 1 -- 172.21.15.62:0/2690546265 shutdown_connections 2024-09-12T00:03:27.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.044+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55b08656a570 0x55b08656c990 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.044+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0864ef690 0x55b0865067b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.044+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0864eba40 0x55b0864f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.044+0000 7fbdcbff0740 1 --2- 172.21.15.62:0/2690546265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0864e6440 0x55b0864f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.044+0000 7fbdcbff0740 1 -- 172.21.15.62:0/2690546265 >> 172.21.15.62:0/2690546265 conn(0x55b0864a5310 msgr2=0x55b0864f08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:27.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.044+0000 7fbda2ffd640 0 client.26948 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:03:27.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.045+0000 7fbdcbff0740 1 -- 172.21.15.62:0/2690546265 shutdown_connections 2024-09-12T00:03:27.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.045+0000 7fbdcbff0740 1 -- 172.21.15.62:0/2690546265 wait complete. 2024-09-12T00:03:27.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.048+0000 7fbdcbff0740 20 client.26948 trim_cache size 0 max 16384 2024-09-12T00:03:27.050 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:03:27.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.051+0000 7fbdc9c17640 1 -- 172.21.15.62:0/1870142511 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbdac0a2cf0 msgr2=0x7fbdac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:27.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.051+0000 7fbdc9c17640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbdac0a2cf0 0x7fbdac0a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc012c30 tx=0x7fbdbc009500 comp rx=0 tx=0).stop 2024-09-12T00:03:27.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.051+0000 7fbdc9c17640 1 -- 172.21.15.62:0/1870142511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc410a560 msgr2=0x7fbdc41a6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:27.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.051+0000 7fbdc9c17640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc410a560 0x7fbdc41a6a90 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4004770 tx=0x7fbdb4003c10 comp rx=0 tx=0).stop 2024-09-12T00:03:27.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.051+0000 7fbdc9c17640 1 -- 172.21.15.62:0/1870142511 shutdown_connections 2024-09-12T00:03:27.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.051+0000 7fbdc9c17640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbdac0a2cf0 0x7fbdac0a51b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.051+0000 7fbdc9c17640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdc41139e0 0x7fbdc41a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.051+0000 7fbdc9c17640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc410ae80 0x7fbdc41a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.051+0000 7fbdc9c17640 1 --2- 172.21.15.62:0/1870142511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc410a560 0x7fbdc41a6a90 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.051+0000 7fbdc9c17640 1 -- 172.21.15.62:0/1870142511 >> 172.21.15.62:0/1870142511 conn(0x7fbdc406e2d0 msgr2=0x7fbdc4114d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.052+0000 7fbdc9c17640 1 -- 172.21.15.62:0/1870142511 shutdown_connections 2024-09-12T00:03:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:27.052+0000 7fbdc9c17640 1 -- 172.21.15.62:0/1870142511 wait complete. 2024-09-12T00:03:27.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:26 smithi062 ceph-mon[30958]: pgmap v353: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.9 MiB/s wr, 764 op/s 2024-09-12T00:03:27.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:26 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:03:27.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:26 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:03:27.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:26 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:03:27.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:26 smithi120 ceph-mon[35182]: pgmap v353: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.9 MiB/s wr, 764 op/s 2024-09-12T00:03:27.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:26 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:03:27.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:26 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:03:27.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:26 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:03:27.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:26 smithi083 ceph-mon[34805]: pgmap v353: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.9 MiB/s wr, 764 op/s 2024-09-12T00:03:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:26 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:03:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:26 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:03:27.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:26 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:03:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 645.0 s, 2865.2 tps, lat 35.029 ms stddev 40.568 2024-09-12T00:03:27.875 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:03:28.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:27 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:03:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:27 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:03:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:27 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:03:28.254 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:03:28.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:27 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:03:28.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:27 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:03:28.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:27 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:03:28.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:27 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:03:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:27 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:03:28.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:27 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:03:29.569 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:29 smithi062 ceph-mon[30958]: pgmap v354: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 12 MiB/s wr, 990 op/s 2024-09-12T00:03:29.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:29 smithi120 ceph-mon[35182]: pgmap v354: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 12 MiB/s wr, 990 op/s 2024-09-12T00:03:29.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:29 smithi083 ceph-mon[34805]: pgmap v354: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 12 MiB/s wr, 990 op/s 2024-09-12T00:03:29.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.695+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/1039723675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc1049b0 msgr2=0x7fdcbc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:29.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.695+0000 7fdcc3a4a640 1 --2- 172.21.15.62:0/1039723675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc1049b0 0x7fdcbc108a00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009900 tx=0x7fdcac02a410 comp rx=0 tx=0).stop 2024-09-12T00:03:29.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.695+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/1039723675 shutdown_connections 2024-09-12T00:03:29.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.695+0000 7fdcc3a4a640 1 --2- 172.21.15.62:0/1039723675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcbc109130 0x7fdcbc110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.695+0000 7fdcc3a4a640 1 --2- 172.21.15.62:0/1039723675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc1049b0 0x7fdcbc108a00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.696+0000 7fdcc3a4a640 1 --2- 172.21.15.62:0/1039723675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcbc104000 0x7fdcbc1043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.696+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/1039723675 >> 172.21.15.62:0/1039723675 conn(0x7fdcbc0ffa20 msgr2=0x7fdcbc101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:29.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.696+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/1039723675 shutdown_connections 2024-09-12T00:03:29.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.696+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/1039723675 wait complete. 2024-09-12T00:03:29.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.696+0000 7fdcc3a4a640 1 Processor -- start 2024-09-12T00:03:29.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.696+0000 7fdcc3a4a640 1 -- start start 2024-09-12T00:03:29.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc3a4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcbc104000 0x7fdcbc073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc3a4a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc1049b0 0x7fdcbc06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc3a4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcbc109130 0x7fdcbc06e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc3a4a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc06c640 con 0x7fdcbc1049b0 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc3a4a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc06c4c0 con 0x7fdcbc109130 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc3a4a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc06c7c0 con 0x7fdcbc104000 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc0fe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc1049b0 0x7fdcbc06dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc0fe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc1049b0 0x7fdcbc06dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54712/0 (socket says 172.21.15.62:54712) 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/232141225 learned_addr learned my addr 172.21.15.62:0/232141225 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcbc104000 0x7fdcbc073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc1fe7640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcbc109130 0x7fdcbc06e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc17e6640 1 -- 172.21.15.62:0/232141225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcbc109130 msgr2=0x7fdcbc06e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcbc109130 0x7fdcbc06e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc17e6640 1 -- 172.21.15.62:0/232141225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc1049b0 msgr2=0x7fdcbc06dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc1049b0 0x7fdcbc06dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc17e6640 1 -- 172.21.15.62:0/232141225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc06e960 con 0x7fdcbc104000 2024-09-12T00:03:29.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.697+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc1049b0 0x7fdcbc06dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:29.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.698+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcbc104000 0x7fdcbc073bd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800e9f0 tx=0x7fdcb800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:29.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.698+0000 7fdcb27fc640 1 -- 172.21.15.62:0/232141225 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800ccc0 con 0x7fdcbc104000 2024-09-12T00:03:29.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.698+0000 7fdcb27fc640 1 -- 172.21.15.62:0/232141225 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb8010910 con 0x7fdcbc104000 2024-09-12T00:03:29.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.698+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/232141225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc1af4d0 con 0x7fdcbc104000 2024-09-12T00:03:29.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.698+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/232141225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc1af9e0 con 0x7fdcbc104000 2024-09-12T00:03:29.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.699+0000 7fdcb27fc640 1 -- 172.21.15.62:0/232141225 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800ccc0 con 0x7fdcbc104000 2024-09-12T00:03:29.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.703+0000 7fdcc5e23740 1 Processor -- start 2024-09-12T00:03:29.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.703+0000 7fdcc5e23740 1 -- start start 2024-09-12T00:03:29.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.703+0000 7fdcb27fc640 1 -- 172.21.15.62:0/232141225 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdcb8010ac0 con 0x7fdcbc104000 2024-09-12T00:03:29.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e849090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e8495d0 0x56328e849a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e85cb80 0x56328e85ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56328e7af2f0 con 0x56328e84ab40 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56328e756cf0 con 0x56328e8495d0 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56328e7ab2b0 con 0x56328e85cb80 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc0fe5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e85cb80 0x56328e85ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc0fe5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e85cb80 0x56328e85ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:33916/0 (socket says 172.21.15.62:33916) 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/101003330 learned_addr learned my addr 172.21.15.62:0/101003330 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc1fe7640 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e849090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e8495d0 0x56328e849a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/101003330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e8495d0 msgr2=0x56328e849a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e8495d0 0x56328e849a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/101003330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 msgr2=0x56328e849090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:29.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e849090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/101003330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328e73b3a0 con 0x56328e85cb80 2024-09-12T00:03:29.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.704+0000 7fdcc1fe7640 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e849090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:29.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.705+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e8495d0 0x56328e849a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:03:29.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.705+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e85cb80 0x56328e85ef70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009a20 tx=0x7fdcac005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b807197d541b60bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.705+0000 7fdcb27fc640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdca80a2ef0 0x7fdca80a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.705+0000 7fdca77fe640 1 -- 172.21.15.62:0/101003330 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac040070 con 0x56328e85cb80 2024-09-12T00:03:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.705+0000 7fdca77fe640 1 -- 172.21.15.62:0/101003330 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcac005520 con 0x56328e85cb80 2024-09-12T00:03:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.705+0000 7fdcc1fe7640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdca80a2ef0 0x7fdca80a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.705+0000 7fdcb27fc640 1 -- 172.21.15.62:0/232141225 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fdcb80cf430 con 0x7fdcbc104000 2024-09-12T00:03:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.706+0000 7fdca77fe640 1 -- 172.21.15.62:0/101003330 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac035040 con 0x56328e85cb80 2024-09-12T00:03:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.706+0000 7fdcc1fe7640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdca80a2ef0 0x7fdca80a53b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b6a0 tx=0x7fdcb400e040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.706+0000 7fdcc5e23740 1 -- 172.21.15.62:0/101003330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e85cb80 msgr2=0x56328e85ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.706+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e85cb80 0x56328e85ef70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009a20 tx=0x7fdcac005e00 comp rx=0 tx=0).stop 2024-09-12T00:03:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.706+0000 7fdcc5e23740 1 -- 172.21.15.62:0/101003330 shutdown_connections 2024-09-12T00:03:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.706+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e85cb80 0x56328e85ef70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.706+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e8495d0 0x56328e849a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.706+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/101003330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e849090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.706+0000 7fdcc5e23740 1 -- 172.21.15.62:0/101003330 >> 172.21.15.62:0/101003330 conn(0x56328e812310 msgr2=0x56328e845c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.707+0000 7fdcc5e23740 1 -- 172.21.15.62:0/101003330 shutdown_connections 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.707+0000 7fdcc5e23740 1 -- 172.21.15.62:0/101003330 wait complete. 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.707+0000 7fdcc5e23740 1 -- start start 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.707+0000 7fdcc5e23740 1 Processor -- start 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.707+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e85c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e8597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e85cb80 0x56328e859de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56328e85f870 con 0x56328e84ab40 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56328e76a060 con 0x56328e85cb80 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56328e85f9f0 con 0x56328e8495d0 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc0fe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e8597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc5e23740 10 client.0 _ll_register_callbacks cb 0x56328e4bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc0fe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e8597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54724/0 (socket says 172.21.15.62:54724) 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/3351711820 learned_addr learned my addr 172.21.15.62:0/3351711820 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e85c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc1fe7640 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e85cb80 0x56328e859de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/3351711820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 msgr2=0x56328e85c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e85c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/3351711820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e85cb80 msgr2=0x56328e859de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e85cb80 0x56328e859de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/3351711820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328e5203c0 con 0x56328e84ab40 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.708+0000 7fdcc1fe7640 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e85cb80 0x56328e859de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.709+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e8597f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdcac031550 tx=0x7fdcac0314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc989c999c43e7c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.709+0000 7fdca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.710+0000 7fdca4ff9640 1 -- 172.21.15.62:0/3351711820 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac040070 con 0x56328e84ab40 2024-09-12T00:03:29.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.710+0000 7fdca4ff9640 1 -- 172.21.15.62:0/3351711820 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fdcac0037f0 con 0x56328e84ab40 2024-09-12T00:03:29.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.710+0000 7fdcc5e23740 1 -- 172.21.15.62:0/3351711820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcac005520 con 0x56328e84ab40 2024-09-12T00:03:29.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.710+0000 7fdca4ff9640 1 -- 172.21.15.62:0/3351711820 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdcac035040 con 0x56328e84ab40 2024-09-12T00:03:29.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.711+0000 7fdcc5e23740 10 client.17433 fetch_fsmap learned FSMap version 34 2024-09-12T00:03:29.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.711+0000 7fdcc5e23740 1 -- 172.21.15.62:0/3351711820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56328e86b290 con 0x56328e84ab40 2024-09-12T00:03:29.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.711+0000 7fdca4ff9640 1 -- 172.21.15.62:0/3351711820 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcac005520 con 0x56328e84ab40 2024-09-12T00:03:29.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.711+0000 7fdca4ff9640 1 -- 172.21.15.62:0/3351711820 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fdcac005070 con 0x56328e84ab40 2024-09-12T00:03:29.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.711+0000 7fdcc5e23740 10 client.17433 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T00:03:29.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.711+0000 7fdcc5e23740 10 client.17433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:03:29.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.711+0000 7fdcc5e23740 20 client.17433 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:29.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.711+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56328e86c5e0 0x56328e86e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.711+0000 7fdcc5e23740 4 client.17433 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:03:29.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.711+0000 7fdcc5e23740 1 -- 172.21.15.62:0/3351711820 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56328e7bc860 con 0x56328e86c5e0 2024-09-12T00:03:29.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.712+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56328e86c5e0 0x56328e86e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.712+0000 7fdca4ff9640 10 client.17433 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:03:29.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.712+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56328e86c5e0 0x56328e86e9d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79e9fbb3f971a354 server_cookie=95d676da0ed24e41 in_seq=0 out_seq=0 2024-09-12T00:03:29.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdca4ff9640 1 -- 172.21.15.62:0/3351711820 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56328e7bc860 con 0x56328e86c5e0 2024-09-12T00:03:29.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdca4ff9640 10 client.17433 handle_command_reply: tid=0 2024-09-12T00:03:29.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdcc5e23740 1 client.17433 shutdown 2024-09-12T00:03:29.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdcc5e23740 1 -- 172.21.15.62:0/3351711820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 msgr2=0x56328e8597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:29.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e8597f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdcac031550 tx=0x7fdcac0314b0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdcc5e23740 1 -- 172.21.15.62:0/3351711820 shutdown_connections 2024-09-12T00:03:29.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56328e86c5e0 0x56328e86e9d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e85cb80 0x56328e859de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e84ab40 0x56328e8597f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdca4ff9640 0 client.17433 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:03:29.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/3351711820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e85c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.713+0000 7fdcc5e23740 1 -- 172.21.15.62:0/3351711820 >> 172.21.15.62:0/3351711820 conn(0x56328e812310 msgr2=0x56328e84b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:29.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.714+0000 7fdcc5e23740 1 -- 172.21.15.62:0/3351711820 shutdown_connections 2024-09-12T00:03:29.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.714+0000 7fdcc5e23740 1 -- 172.21.15.62:0/3351711820 wait complete. 2024-09-12T00:03:29.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.714+0000 7fdcc5e23740 20 client.17433 trim_cache size 0 max 16384 2024-09-12T00:03:29.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc5e23740 1 Processor -- start 2024-09-12T00:03:29.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc5e23740 1 -- start start 2024-09-12T00:03:29.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e868b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e873fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 0x56328e8744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc1b6840 con 0x56328e85cb80 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc1b40b0 con 0x56328e84ab40 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc1b1f20 con 0x56328e8495d0 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc0fe5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e873fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc1fe7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 0x56328e8744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc0fe5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e873fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:54752/0 (socket says 172.21.15.62:54752) 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/739470264 learned_addr learned my addr 172.21.15.62:0/739470264 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.734+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/739470264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e868b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.735+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/739470264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 msgr2=0x56328e868b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.735+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/739470264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e868b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.735+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/739470264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 msgr2=0x56328e8744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:29.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.735+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/739470264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 0x56328e8744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.735+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/739470264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328e85c8b0 con 0x56328e84ab40 2024-09-12T00:03:29.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.735+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/739470264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e873fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdcac031520 tx=0x7fdcac005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=981507b8b90a4e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:29.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.736+0000 7fdca6ffd640 1 -- 172.21.15.62:0/739470264 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac05b020 con 0x56328e84ab40 2024-09-12T00:03:29.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.736+0000 7fdca6ffd640 1 -- 172.21.15.62:0/739470264 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcac04e070 con 0x56328e84ab40 2024-09-12T00:03:29.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.736+0000 7fdca6ffd640 1 -- 172.21.15.62:0/739470264 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac040040 con 0x56328e84ab40 2024-09-12T00:03:29.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.737+0000 7fdcc5e23740 1 -- 172.21.15.62:0/739470264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 msgr2=0x56328e873fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:29.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.737+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/739470264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e873fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdcac031520 tx=0x7fdcac005d50 comp rx=0 tx=0).stop 2024-09-12T00:03:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.738+0000 7fdcc5e23740 1 -- 172.21.15.62:0/739470264 shutdown_connections 2024-09-12T00:03:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.738+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/739470264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 0x56328e8744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.738+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/739470264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e873fa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.738+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/739470264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e868b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.738+0000 7fdcc5e23740 1 -- 172.21.15.62:0/739470264 >> 172.21.15.62:0/739470264 conn(0x56328e812310 msgr2=0x56328e8580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.738+0000 7fdcc5e23740 1 -- 172.21.15.62:0/739470264 shutdown_connections 2024-09-12T00:03:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.738+0000 7fdcc5e23740 1 -- 172.21.15.62:0/739470264 wait complete. 2024-09-12T00:03:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.739+0000 7fdcc5e23740 1 -- start start 2024-09-12T00:03:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.739+0000 7fdcc5e23740 1 Processor -- start 2024-09-12T00:03:29.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.739+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e860b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.739+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e8610a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.739+0000 7fdcc5e23740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 0x56328e8615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.739+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc1b2570 con 0x56328e85cb80 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.739+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc1b39e0 con 0x56328e84ab40 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.739+0000 7fdcc5e23740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc1b2c40 con 0x56328e8495d0 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.739+0000 7fdcc5e23740 10 client.0 _ll_register_callbacks cb 0x7fdcb4009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc0fe5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e8610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc0fe5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e8610a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:54762/0 (socket says 172.21.15.62:54762) 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/243267191 learned_addr learned my addr 172.21.15.62:0/243267191 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc1fe7640 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 0x56328e8615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/243267191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 msgr2=0x56328e860b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e860b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/243267191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 msgr2=0x56328e8615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 0x56328e8615e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc0fe5640 1 -- 172.21.15.62:0/243267191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328e8d60d0 con 0x56328e84ab40 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc1fe7640 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 0x56328e8615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdcc0fe5640 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e8610a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdcac031980 tx=0x7fdcac003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6bc4ede0ecd1e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.740+0000 7fdca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdca4ff9640 1 -- 172.21.15.62:0/243267191 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac05b030 con 0x56328e84ab40 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdca4ff9640 1 -- 172.21.15.62:0/243267191 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fdcac004950 con 0x56328e84ab40 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdcc5e23740 1 -- 172.21.15.62:0/243267191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcac04e070 con 0x56328e84ab40 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdca4ff9640 1 -- 172.21.15.62:0/243267191 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdcac06f050 con 0x56328e84ab40 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdcc5e23740 10 client.26963 fetch_fsmap learned FSMap version 34 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdcc5e23740 1 -- 172.21.15.62:0/243267191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56328e8d7370 con 0x56328e84ab40 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdca4ff9640 1 -- 172.21.15.62:0/243267191 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcac04e070 con 0x56328e84ab40 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdca4ff9640 1 -- 172.21.15.62:0/243267191 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fdcac045880 con 0x56328e84ab40 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdcc5e23740 10 client.26963 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T00:03:29.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdcc5e23740 10 client.26963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdcc5e23740 20 client.26963 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56328e8d7a30 0x56328e8d9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdcc5e23740 4 client.26963 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.741+0000 7fdcc5e23740 1 -- 172.21.15.62:0/243267191 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56328e7b6d70 con 0x56328e8d7a30 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.742+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56328e8d7a30 0x56328e8d9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.742+0000 7fdca4ff9640 10 client.26963 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.742+0000 7fdcc17e6640 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56328e8d7a30 0x56328e8d9e50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c808e99c6221fa6a server_cookie=d435efd5f1554006 in_seq=0 out_seq=0 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.742+0000 7fdca4ff9640 1 -- 172.21.15.62:0/243267191 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56328e7b6d70 con 0x56328e8d7a30 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.742+0000 7fdca4ff9640 10 client.26963 handle_command_reply: tid=0 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.742+0000 7fdcc5e23740 1 client.26963 shutdown 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.742+0000 7fdcc5e23740 1 -- 172.21.15.62:0/243267191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 msgr2=0x56328e8610a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.742+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e8610a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdcac031980 tx=0x7fdcac003820 comp rx=0 tx=0).stop 2024-09-12T00:03:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.742+0000 7fdcc5e23740 1 -- 172.21.15.62:0/243267191 shutdown_connections 2024-09-12T00:03:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.742+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56328e8d7a30 0x56328e8d9e50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56328e85cb80 0x56328e8615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56328e84ab40 0x56328e8610a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc5e23740 1 --2- 172.21.15.62:0/243267191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56328e8495d0 0x56328e860b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc5e23740 1 -- 172.21.15.62:0/243267191 >> 172.21.15.62:0/243267191 conn(0x56328e812310 msgr2=0x56328e8444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdca4ff9640 0 client.26963 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:03:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc5e23740 1 -- 172.21.15.62:0/243267191 shutdown_connections 2024-09-12T00:03:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc5e23740 1 -- 172.21.15.62:0/243267191 wait complete. 2024-09-12T00:03:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc5e23740 20 client.26963 trim_cache size 0 max 16384 2024-09-12T00:03:29.746 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:03:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/232141225 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdca80a2ef0 msgr2=0x7fdca80a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc3a4a640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdca80a2ef0 0x7fdca80a53b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b6a0 tx=0x7fdcb400e040 comp rx=0 tx=0).stop 2024-09-12T00:03:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/232141225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcbc104000 msgr2=0x7fdcbc073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.743+0000 7fdcc3a4a640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcbc104000 0x7fdcbc073bd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800e9f0 tx=0x7fdcb800eeb0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.744+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/232141225 shutdown_connections 2024-09-12T00:03:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.744+0000 7fdcc3a4a640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdca80a2ef0 0x7fdca80a53b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.744+0000 7fdcc3a4a640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcbc109130 0x7fdcbc06e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.744+0000 7fdcc3a4a640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc1049b0 0x7fdcbc06dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.744+0000 7fdcc3a4a640 1 --2- 172.21.15.62:0/232141225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcbc104000 0x7fdcbc073bd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.744+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/232141225 >> 172.21.15.62:0/232141225 conn(0x7fdcbc0ffa20 msgr2=0x7fdcbc100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.744+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/232141225 shutdown_connections 2024-09-12T00:03:29.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:29.744+0000 7fdcc3a4a640 1 -- 172.21.15.62:0/232141225 wait complete. 2024-09-12T00:03:30.456 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:03:30.885 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:03:31.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.402+0000 7f6b61b3e640 1 -- 172.21.15.62:0/3445978297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b5c077040 msgr2=0x7f6b5c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:31.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.402+0000 7f6b61b3e640 1 --2- 172.21.15.62:0/3445978297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b5c077040 0x7f6b5c1124f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009900 tx=0x7f6b5002a410 comp rx=0 tx=0).stop 2024-09-12T00:03:31.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.402+0000 7f6b61b3e640 1 -- 172.21.15.62:0/3445978297 shutdown_connections 2024-09-12T00:03:31.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.402+0000 7f6b61b3e640 1 --2- 172.21.15.62:0/3445978297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c112a30 0x7f6b5c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.402+0000 7f6b61b3e640 1 --2- 172.21.15.62:0/3445978297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b5c077040 0x7f6b5c1124f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.402+0000 7f6b61b3e640 1 --2- 172.21.15.62:0/3445978297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b5c076720 0x7f6b5c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.402+0000 7f6b61b3e640 1 -- 172.21.15.62:0/3445978297 >> 172.21.15.62:0/3445978297 conn(0x7f6b5c0ffab0 msgr2=0x7f6b5c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:31.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.402+0000 7f6b61b3e640 1 -- 172.21.15.62:0/3445978297 shutdown_connections 2024-09-12T00:03:31.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.402+0000 7f6b61b3e640 1 -- 172.21.15.62:0/3445978297 wait complete. 2024-09-12T00:03:31.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.403+0000 7f6b61b3e640 1 Processor -- start 2024-09-12T00:03:31.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.405+0000 7f6b61b3e640 1 -- start start 2024-09-12T00:03:31.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.406+0000 7f6b61b3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c076720 0x7f6b5c1a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.406+0000 7f6b61b3e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b5c077040 0x7f6b5c1a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.406+0000 7f6b61b3e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b5c112a30 0x7f6b5c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.406+0000 7f6b61b3e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c1185b0 con 0x7f6b5c112a30 2024-09-12T00:03:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.406+0000 7f6b61b3e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c118430 con 0x7f6b5c077040 2024-09-12T00:03:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.406+0000 7f6b61b3e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c118730 con 0x7f6b5c076720 2024-09-12T00:03:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.406+0000 7f6b5b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c076720 0x7f6b5c1a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.406+0000 7f6b5b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c076720 0x7f6b5c1a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:56436/0 (socket says 172.21.15.62:56436) 2024-09-12T00:03:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.406+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/246876730 learned_addr learned my addr 172.21.15.62:0/246876730 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.406+0000 7f6b5affd640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b5c077040 0x7f6b5c1a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.407+0000 7f6b5bfff640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b5c112a30 0x7f6b5c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.407+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/246876730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b5c077040 msgr2=0x7f6b5c1a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.407+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b5c077040 0x7f6b5c1a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.407+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b5c112a30 msgr2=0x7f6b5c19e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.407+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b5c112a30 0x7f6b5c19e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.407+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/246876730 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c19f110 con 0x7f6b5c076720 2024-09-12T00:03:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.407+0000 7f6b5bfff640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b5c112a30 0x7f6b5c19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:31.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.408+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c076720 0x7f6b5c1a4820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00e9f0 tx=0x7f6b4c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:31.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.408+0000 7f6b58ff9640 1 -- 172.21.15.62:0/246876730 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c00cd80 con 0x7f6b5c076720 2024-09-12T00:03:31.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.408+0000 7f6b58ff9640 1 -- 172.21.15.62:0/246876730 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b4c010910 con 0x7f6b5c076720 2024-09-12T00:03:31.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.408+0000 7f6b58ff9640 1 -- 172.21.15.62:0/246876730 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c018990 con 0x7f6b5c076720 2024-09-12T00:03:31.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.408+0000 7f6b61b3e640 1 -- 172.21.15.62:0/246876730 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5c19f400 con 0x7f6b5c076720 2024-09-12T00:03:31.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.408+0000 7f6b61b3e640 1 -- 172.21.15.62:0/246876730 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5c1ab6f0 con 0x7f6b5c076720 2024-09-12T00:03:31.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.411+0000 7f6b58ff9640 1 -- 172.21.15.62:0/246876730 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6b4c010ac0 con 0x7f6b5c076720 2024-09-12T00:03:31.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.411+0000 7f6b63f17740 1 Processor -- start 2024-09-12T00:03:31.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.412+0000 7f6b58ff9640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b440a2e40 0x7f6b440a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.412+0000 7f6b58ff9640 1 -- 172.21.15.62:0/246876730 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f6b4c0cf690 con 0x7f6b5c076720 2024-09-12T00:03:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.412+0000 7f6b63f17740 1 -- start start 2024-09-12T00:03:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.413+0000 7f6b5bfff640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b440a2e40 0x7f6b440a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.413+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 0x5576e27a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.413+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a65d0 0x5576e27a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.413+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 0x5576e27b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.414+0000 7f6b5affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a65d0 0x5576e27a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.414+0000 7f6b5affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a65d0 0x5576e27a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55694/0 (socket says 172.21.15.62:55694) 2024-09-12T00:03:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.414+0000 7f6b5affd640 1 -- 172.21.15.62:0/2294356472 learned_addr learned my addr 172.21.15.62:0/2294356472 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.414+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/2294356472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 0x5576e27a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.414+0000 7f6b63f17740 1 -- 172.21.15.62:0/2294356472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5576e270c2f0 con 0x5576e27a65d0 2024-09-12T00:03:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.414+0000 7f6b63f17740 1 -- 172.21.15.62:0/2294356472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5576e26b3cf0 con 0x5576e27b1370 2024-09-12T00:03:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.414+0000 7f6b63f17740 1 -- 172.21.15.62:0/2294356472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576e27082b0 con 0x5576e27a7b40 2024-09-12T00:03:31.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.415+0000 7f6b5bfff640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b440a2e40 0x7f6b440a5300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c06aa00 tx=0x7f6b5400a4d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:31.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.415+0000 7f6b5bfff640 1 --2- 172.21.15.62:0/2294356472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 0x5576e27b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.415+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/2294356472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 msgr2=0x5576e27b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.415+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/2294356472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 0x5576e27b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.415+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/2294356472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a65d0 msgr2=0x5576e27a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.415+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/2294356472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a65d0 0x5576e27a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.415+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/2294356472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e26983a0 con 0x5576e27a7b40 2024-09-12T00:03:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.415+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/2294356472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 0x5576e27a6090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0ccd80 tx=0x7f6b4c09bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b2df817a16a573 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.416+0000 7f6b41ffb640 1 -- 172.21.15.62:0/2294356472 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c0cd000 con 0x5576e27a7b40 2024-09-12T00:03:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.416+0000 7f6b41ffb640 1 -- 172.21.15.62:0/2294356472 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b4c098270 con 0x5576e27a7b40 2024-09-12T00:03:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.416+0000 7f6b41ffb640 1 -- 172.21.15.62:0/2294356472 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c09b2b0 con 0x5576e27a7b40 2024-09-12T00:03:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.417+0000 7f6b63f17740 1 -- 172.21.15.62:0/2294356472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 msgr2=0x5576e27a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.417+0000 7f6b63f17740 1 --2- 172.21.15.62:0/2294356472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 0x5576e27a6090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0ccd80 tx=0x7f6b4c09bcc0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.417+0000 7f6b63f17740 1 -- 172.21.15.62:0/2294356472 shutdown_connections 2024-09-12T00:03:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.417+0000 7f6b63f17740 1 --2- 172.21.15.62:0/2294356472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 0x5576e27b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.417+0000 7f6b63f17740 1 --2- 172.21.15.62:0/2294356472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a65d0 0x5576e27a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.417+0000 7f6b63f17740 1 --2- 172.21.15.62:0/2294356472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 0x5576e27a6090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.417+0000 7f6b63f17740 1 -- 172.21.15.62:0/2294356472 >> 172.21.15.62:0/2294356472 conn(0x5576e276f310 msgr2=0x5576e27a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.417+0000 7f6b63f17740 1 -- 172.21.15.62:0/2294356472 shutdown_connections 2024-09-12T00:03:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.417+0000 7f6b63f17740 1 -- 172.21.15.62:0/2294356472 wait complete. 2024-09-12T00:03:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.418+0000 7f6b63f17740 1 -- start start 2024-09-12T00:03:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.418+0000 7f6b63f17740 1 Processor -- start 2024-09-12T00:03:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.418+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a65d0 0x5576e27b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.418+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.418+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 0x5576e27ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.418+0000 7f6b63f17740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5576e27b4070 con 0x5576e27a7b40 2024-09-12T00:03:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.418+0000 7f6b63f17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5576e26c7060 con 0x5576e27b1370 2024-09-12T00:03:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.418+0000 7f6b63f17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576e27b41f0 con 0x5576e27a65d0 2024-09-12T00:03:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 0x5576e27ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55710/0 (socket says 172.21.15.62:55710) 2024-09-12T00:03:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5affd640 1 -- 172.21.15.62:0/3746519487 learned_addr learned my addr 172.21.15.62:0/3746519487 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a65d0 0x5576e27b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b63f17740 10 client.0 _ll_register_callbacks cb 0x5576e241a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5bfff640 1 -- 172.21.15.62:0/3746519487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a65d0 msgr2=0x5576e27b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5bfff640 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a65d0 0x5576e27b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5bfff640 1 -- 172.21.15.62:0/3746519487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 msgr2=0x5576e27ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5bfff640 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5bfff640 1 -- 172.21.15.62:0/3746519487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e247d3c0 con 0x5576e27b1370 2024-09-12T00:03:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a65d0 0x5576e27b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5affd640 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b5bfff640 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 0x5576e27ae850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5576e27a7070 tx=0x7f6b54004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362e3e60d34a65ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.419+0000 7f6b3affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:03:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.420+0000 7f6b3affd640 1 -- 172.21.15.62:0/3746519487 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54016070 con 0x5576e27b1370 2024-09-12T00:03:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.420+0000 7f6b63f17740 1 -- 172.21.15.62:0/3746519487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b4c098270 con 0x5576e27b1370 2024-09-12T00:03:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.420+0000 7f6b3affd640 1 -- 172.21.15.62:0/3746519487 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f6b5401d840 con 0x5576e27b1370 2024-09-12T00:03:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.420+0000 7f6b3affd640 1 -- 172.21.15.62:0/3746519487 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6b5401da40 con 0x5576e27b1370 2024-09-12T00:03:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.420+0000 7f6b63f17740 10 client.26975 fetch_fsmap learned FSMap version 35 2024-09-12T00:03:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.420+0000 7f6b63f17740 1 -- 172.21.15.62:0/3746519487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5576e27bfc30 con 0x5576e27b1370 2024-09-12T00:03:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.421+0000 7f6b3affd640 1 -- 172.21.15.62:0/3746519487 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b4c098270 con 0x5576e27b1370 2024-09-12T00:03:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.422+0000 7f6b3affd640 1 -- 172.21.15.62:0/3746519487 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f6b540243a0 con 0x5576e27b1370 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.422+0000 7f6b63f17740 10 client.26975 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.422+0000 7f6b63f17740 10 client.26975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.422+0000 7f6b63f17740 20 client.26975 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.422+0000 7f6b63f17740 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5576e27c1100 0x5576e27c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.422+0000 7f6b63f17740 4 client.26975 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.422+0000 7f6b63f17740 1 -- 172.21.15.62:0/3746519487 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576e2719860 con 0x5576e27c1100 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.423+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5576e27c1100 0x5576e27c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.423+0000 7f6b3affd640 10 client.26975 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.423+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5576e27c1100 0x5576e27c3580 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c82689c3973c07 server_cookie=1d2980a32ad33954 in_seq=0 out_seq=0 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.424+0000 7f6b3affd640 1 -- 172.21.15.62:0/3746519487 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576e2719860 con 0x5576e27c1100 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.424+0000 7f6b3affd640 10 client.26975 handle_command_reply: tid=0 2024-09-12T00:03:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.424+0000 7f6b63f17740 1 client.26975 shutdown 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.424+0000 7f6b63f17740 1 -- 172.21.15.62:0/3746519487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 msgr2=0x5576e27ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.424+0000 7f6b63f17740 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 0x5576e27ae850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5576e27a7070 tx=0x7f6b54004ab0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.425+0000 7f6b63f17740 1 -- 172.21.15.62:0/3746519487 shutdown_connections 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.425+0000 7f6b63f17740 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5576e27c1100 0x5576e27c3580 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.425+0000 7f6b63f17740 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27b1370 0x5576e27ae850 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.425+0000 7f6b63f17740 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.425+0000 7f6b3affd640 0 client.26975 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.425+0000 7f6b63f17740 1 --2- 172.21.15.62:0/3746519487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a65d0 0x5576e27b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.425+0000 7f6b63f17740 1 -- 172.21.15.62:0/3746519487 >> 172.21.15.62:0/3746519487 conn(0x5576e276f310 msgr2=0x5576e27a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.425+0000 7f6b63f17740 1 -- 172.21.15.62:0/3746519487 shutdown_connections 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.425+0000 7f6b63f17740 1 -- 172.21.15.62:0/3746519487 wait complete. 2024-09-12T00:03:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.426+0000 7f6b63f17740 20 client.26975 trim_cache size 0 max 16384 2024-09-12T00:03:31.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.446+0000 7f6b63f17740 1 Processor -- start 2024-09-12T00:03:31.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.446+0000 7f6b63f17740 1 -- start start 2024-09-12T00:03:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 0x5576e279da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27b1370 0x5576e27a9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b5b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 0x5576e279da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b5b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 0x5576e279da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40122/0 (socket says 172.21.15.62:40122) 2024-09-12T00:03:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/3903673523 learned_addr learned my addr 172.21.15.62:0/3903673523 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b5affd640 1 --2- 172.21.15.62:0/3903673523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b63f17740 1 -- 172.21.15.62:0/3903673523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c1b67a0 con 0x5576e27a7b40 2024-09-12T00:03:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b63f17740 1 -- 172.21.15.62:0/3903673523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c1b4010 con 0x5576e27a65d0 2024-09-12T00:03:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b63f17740 1 -- 172.21.15.62:0/3903673523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c1b1f10 con 0x5576e27b1370 2024-09-12T00:03:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b5bfff640 1 --2- 172.21.15.62:0/3903673523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27b1370 0x5576e27a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b5affd640 1 -- 172.21.15.62:0/3903673523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27b1370 msgr2=0x5576e27a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b5affd640 1 --2- 172.21.15.62:0/3903673523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27b1370 0x5576e27a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b5affd640 1 -- 172.21.15.62:0/3903673523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 msgr2=0x5576e279da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b5affd640 1 --2- 172.21.15.62:0/3903673523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 0x5576e279da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.447+0000 7f6b5affd640 1 -- 172.21.15.62:0/3903673523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e27b1180 con 0x5576e27a7b40 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.448+0000 7f6b5affd640 1 --2- 172.21.15.62:0/3903673523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27a9120 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6b500057c0 tx=0x7f6b500318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc08ef94ed6d4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.448+0000 7f6b417fa640 1 -- 172.21.15.62:0/3903673523 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50035040 con 0x5576e27a7b40 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.448+0000 7f6b417fa640 1 -- 172.21.15.62:0/3903673523 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b50031930 con 0x5576e27a7b40 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.448+0000 7f6b417fa640 1 -- 172.21.15.62:0/3903673523 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b500049b0 con 0x5576e27a7b40 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.449+0000 7f6b63f17740 1 -- 172.21.15.62:0/3903673523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 msgr2=0x5576e27a9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.449+0000 7f6b63f17740 1 --2- 172.21.15.62:0/3903673523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27a9120 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6b500057c0 tx=0x7f6b500318c0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.449+0000 7f6b63f17740 1 -- 172.21.15.62:0/3903673523 shutdown_connections 2024-09-12T00:03:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.449+0000 7f6b63f17740 1 --2- 172.21.15.62:0/3903673523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27b1370 0x5576e27a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.449+0000 7f6b63f17740 1 --2- 172.21.15.62:0/3903673523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27a7b40 0x5576e27a9120 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.449+0000 7f6b63f17740 1 --2- 172.21.15.62:0/3903673523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 0x5576e279da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.449+0000 7f6b63f17740 1 -- 172.21.15.62:0/3903673523 >> 172.21.15.62:0/3903673523 conn(0x5576e276f310 msgr2=0x5576e27ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.449+0000 7f6b63f17740 1 -- 172.21.15.62:0/3903673523 shutdown_connections 2024-09-12T00:03:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.449+0000 7f6b63f17740 1 -- 172.21.15.62:0/3903673523 wait complete. 2024-09-12T00:03:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.449+0000 7f6b63f17740 1 -- start start 2024-09-12T00:03:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b63f17740 1 Processor -- start 2024-09-12T00:03:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 0x5576e27b9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 0x5576e27b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b63f17740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27b1370 0x5576e27aa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b63f17740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c1b24d0 con 0x5576e27b1370 2024-09-12T00:03:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b63f17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c1b3940 con 0x5576e27a65d0 2024-09-12T00:03:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b63f17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c1b2ba0 con 0x5576e27a7b40 2024-09-12T00:03:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27b1370 0x5576e27aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 0x5576e27b9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b63f17740 10 client.0 _ll_register_callbacks cb 0x7f6b54010550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27b1370 0x5576e27aa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55736/0 (socket says 172.21.15.62:55736) 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5bfff640 1 -- 172.21.15.62:0/1672840179 learned_addr learned my addr 172.21.15.62:0/1672840179 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5affd640 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 0x5576e27b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/1672840179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 msgr2=0x5576e27b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 0x5576e27b99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/1672840179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27b1370 msgr2=0x5576e27aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27b1370 0x5576e27aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5b7fe640 1 -- 172.21.15.62:0/1672840179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e27aac50 con 0x5576e27a65d0 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.450+0000 7f6b5bfff640 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27b1370 0x5576e27aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.451+0000 7f6b5affd640 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 0x5576e27b99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.451+0000 7f6b5b7fe640 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 0x5576e27b9480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0ccdb0 tx=0x7f6b4c0a4940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daee94635ee94052 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.451+0000 7f6b3affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:03:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.451+0000 7f6b3affd640 1 -- 172.21.15.62:0/1672840179 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c018d40 con 0x5576e27a65d0 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.451+0000 7f6b3affd640 1 -- 172.21.15.62:0/1672840179 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f6b4c09b050 con 0x5576e27a65d0 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.451+0000 7f6b63f17740 1 -- 172.21.15.62:0/1672840179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b50031930 con 0x5576e27a65d0 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b3affd640 1 -- 172.21.15.62:0/1672840179 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6b4c09b250 con 0x5576e27a65d0 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b63f17740 10 client.26981 fetch_fsmap learned FSMap version 35 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b63f17740 1 -- 172.21.15.62:0/1672840179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5576e282b600 con 0x5576e27a65d0 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b3affd640 1 -- 172.21.15.62:0/1672840179 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b4c0ae070 con 0x5576e27a65d0 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b3affd640 1 -- 172.21.15.62:0/1672840179 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f6b4c097950 con 0x5576e27a65d0 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b63f17740 10 client.26981 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b63f17740 10 client.26981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b63f17740 20 client.26981 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b63f17740 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5576e27c1100 0x5576e282d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b63f17740 4 client.26981 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.452+0000 7f6b63f17740 1 -- 172.21.15.62:0/1672840179 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6b50031930 con 0x5576e27c1100 2024-09-12T00:03:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.453+0000 7f6b5affd640 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5576e27c1100 0x5576e282d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.454+0000 7f6b3affd640 10 client.26981 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.454+0000 7f6b5affd640 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5576e27c1100 0x5576e282d950 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d17b23ce90f0bb7 server_cookie=e5e1342f30d0e085 in_seq=0 out_seq=0 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.454+0000 7f6b3affd640 1 -- 172.21.15.62:0/1672840179 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6b50031930 con 0x5576e27c1100 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b3affd640 10 client.26981 handle_command_reply: tid=0 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b63f17740 1 client.26981 shutdown 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b63f17740 1 -- 172.21.15.62:0/1672840179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 msgr2=0x5576e27b9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b63f17740 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 0x5576e27b9480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0ccdb0 tx=0x7f6b4c0a4940 comp rx=0 tx=0).stop 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b63f17740 1 -- 172.21.15.62:0/1672840179 shutdown_connections 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b63f17740 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5576e27c1100 0x5576e282d950 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b63f17740 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576e27b1370 0x5576e27aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b63f17740 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e27a7b40 0x5576e27b99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b63f17740 1 --2- 172.21.15.62:0/1672840179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576e27a65d0 0x5576e27b9480 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0ccdb0 tx=0x7f6b4c0a4940 comp rx=0 tx=0).stop 2024-09-12T00:03:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b63f17740 1 -- 172.21.15.62:0/1672840179 >> 172.21.15.62:0/1672840179 conn(0x5576e276f310 msgr2=0x5576e27ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b3affd640 0 client.26981 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.455+0000 7f6b63f17740 1 -- 172.21.15.62:0/1672840179 shutdown_connections 2024-09-12T00:03:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.459+0000 7f6b63f17740 1 -- 172.21.15.62:0/1672840179 wait complete. 2024-09-12T00:03:31.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.463+0000 7f6b63f17740 20 client.26981 trim_cache size 0 max 16384 2024-09-12T00:03:31.469 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:03:31.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.478+0000 7f6b61b3e640 1 -- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b440a2e40 msgr2=0x7f6b440a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:31.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.478+0000 7f6b61b3e640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b440a2e40 0x7f6b440a5300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c06aa00 tx=0x7f6b5400a4d0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.478+0000 7f6b61b3e640 1 -- 172.21.15.62:0/246876730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c076720 msgr2=0x7f6b5c1a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:31.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.478+0000 7f6b61b3e640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c076720 0x7f6b5c1a4820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00e9f0 tx=0x7f6b4c00eeb0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.481+0000 7f6b61b3e640 1 -- 172.21.15.62:0/246876730 shutdown_connections 2024-09-12T00:03:31.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.481+0000 7f6b61b3e640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6b440a2e40 0x7f6b440a5300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.481+0000 7f6b61b3e640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b5c112a30 0x7f6b5c19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.481+0000 7f6b61b3e640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b5c077040 0x7f6b5c1a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.481+0000 7f6b61b3e640 1 --2- 172.21.15.62:0/246876730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5c076720 0x7f6b5c1a4820 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.481+0000 7f6b61b3e640 1 -- 172.21.15.62:0/246876730 >> 172.21.15.62:0/246876730 conn(0x7f6b5c0ffab0 msgr2=0x7f6b5c101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.482+0000 7f6b61b3e640 1 -- 172.21.15.62:0/246876730 shutdown_connections 2024-09-12T00:03:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:31.482+0000 7f6b61b3e640 1 -- 172.21.15.62:0/246876730 wait complete. 2024-09-12T00:03:31.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:31 smithi120 ceph-mon[35182]: pgmap v355: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.7 MiB/s wr, 743 op/s 2024-09-12T00:03:31.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:31 smithi083 ceph-mon[34805]: pgmap v355: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.7 MiB/s wr, 743 op/s 2024-09-12T00:03:31.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:31 smithi062 ceph-mon[30958]: pgmap v355: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.7 MiB/s wr, 743 op/s 2024-09-12T00:03:32.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 650.0 s, 2621.4 tps, lat 38.218 ms stddev 41.267 2024-09-12T00:03:32.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:32 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:03:32.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:32 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:03:32.685 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:32 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:03:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:33 smithi120 ceph-mon[35182]: pgmap v356: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.4 MiB/s wr, 707 op/s 2024-09-12T00:03:33.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:33 smithi083 ceph-mon[34805]: pgmap v356: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.4 MiB/s wr, 707 op/s 2024-09-12T00:03:33.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:33 smithi062 ceph-mon[30958]: pgmap v356: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.4 MiB/s wr, 707 op/s 2024-09-12T00:03:34.058 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:03:34.058 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:03:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:03:35.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.084+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1801640316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad58100e20 msgr2=0x7fad58101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:35.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.084+0000 7fad5f9c9640 1 --2- 172.21.15.62:0/1801640316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad58100e20 0x7fad58101200 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fad54009900 tx=0x7fad5402a490 comp rx=0 tx=0).stop 2024-09-12T00:03:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.087+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1801640316 shutdown_connections 2024-09-12T00:03:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.087+0000 7fad5f9c9640 1 --2- 172.21.15.62:0/1801640316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad58102ef0 0x7fad5810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.087+0000 7fad5f9c9640 1 --2- 172.21.15.62:0/1801640316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad58102570 0x7fad581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.087+0000 7fad5f9c9640 1 --2- 172.21.15.62:0/1801640316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad58100e20 0x7fad58101200 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.087+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1801640316 >> 172.21.15.62:0/1801640316 conn(0x7fad580fc7d0 msgr2=0x7fad580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.090+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1801640316 shutdown_connections 2024-09-12T00:03:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.093+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1801640316 wait complete. 2024-09-12T00:03:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.094+0000 7fad5f9c9640 1 Processor -- start 2024-09-12T00:03:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.095+0000 7fad5f9c9640 1 -- start start 2024-09-12T00:03:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.095+0000 7fad5f9c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad58100e20 0x7fad58116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.095+0000 7fad5f9c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad58102570 0x7fad58116880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.096+0000 7fad5f9c9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad58102ef0 0x7fad581105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.096+0000 7fad5f9c9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad5810f040 con 0x7fad58102ef0 2024-09-12T00:03:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.096+0000 7fad5f9c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad5810eec0 con 0x7fad58100e20 2024-09-12T00:03:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.096+0000 7fad5f9c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad5810f1c0 con 0x7fad58102570 2024-09-12T00:03:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.096+0000 7fad5d765640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad58100e20 0x7fad58116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.096+0000 7fad5d765640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad58100e20 0x7fad58116340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40134/0 (socket says 172.21.15.62:40134) 2024-09-12T00:03:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.096+0000 7fad5d765640 1 -- 172.21.15.62:0/1088701740 learned_addr learned my addr 172.21.15.62:0/1088701740 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.096+0000 7fad5df66640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad58102ef0 0x7fad581105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.097+0000 7fad5cf64640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad58102570 0x7fad58116880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.098+0000 7fad5cf64640 1 -- 172.21.15.62:0/1088701740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad58100e20 msgr2=0x7fad58116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.098+0000 7fad5cf64640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad58100e20 0x7fad58116340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.098+0000 7fad5cf64640 1 -- 172.21.15.62:0/1088701740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad58102ef0 msgr2=0x7fad581105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.098+0000 7fad5cf64640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad58102ef0 0x7fad581105f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.098+0000 7fad5cf64640 1 -- 172.21.15.62:0/1088701740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad58110eb0 con 0x7fad58102570 2024-09-12T00:03:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.098+0000 7fad5cf64640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad58102570 0x7fad58116880 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fad4800e9f0 tx=0x7fad4800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:35.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.100+0000 7fad4e7fc640 1 -- 172.21.15.62:0/1088701740 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4800ccc0 con 0x7fad58102570 2024-09-12T00:03:35.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.100+0000 7fad4e7fc640 1 -- 172.21.15.62:0/1088701740 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad48010910 con 0x7fad58102570 2024-09-12T00:03:35.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.100+0000 7fad4e7fc640 1 -- 172.21.15.62:0/1088701740 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48018980 con 0x7fad58102570 2024-09-12T00:03:35.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.100+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1088701740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad581111a0 con 0x7fad58102570 2024-09-12T00:03:35.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.100+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1088701740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad58071e70 con 0x7fad58102570 2024-09-12T00:03:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.112+0000 7fad61da2740 1 Processor -- start 2024-09-12T00:03:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.114+0000 7fad61da2740 1 -- start start 2024-09-12T00:03:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.115+0000 7fad61da2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc88b60 0x55c9ddc88f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.115+0000 7fad61da2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc89480 0x55c9ddc93800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.115+0000 7fad61da2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc93d40 0x55c9ddc96130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.115+0000 7fad61da2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c9ddbee280 con 0x55c9ddc89480 2024-09-12T00:03:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.115+0000 7fad61da2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c9ddb95b10 con 0x55c9ddc88b60 2024-09-12T00:03:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.115+0000 7fad61da2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9ddbea240 con 0x55c9ddc93d40 2024-09-12T00:03:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.116+0000 7fad5df66640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc88b60 0x55c9ddc88f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.116+0000 7fad5df66640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc88b60 0x55c9ddc88f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40146/0 (socket says 172.21.15.62:40146) 2024-09-12T00:03:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.116+0000 7fad5df66640 1 -- 172.21.15.62:0/340331902 learned_addr learned my addr 172.21.15.62:0/340331902 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.117+0000 7fad5df66640 1 -- 172.21.15.62:0/340331902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc93d40 msgr2=0x55c9ddc96130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:03:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.117+0000 7fad5df66640 1 --2- 172.21.15.62:0/340331902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc93d40 0x55c9ddc96130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.117+0000 7fad5df66640 1 -- 172.21.15.62:0/340331902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc89480 msgr2=0x55c9ddc93800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.117+0000 7fad5df66640 1 --2- 172.21.15.62:0/340331902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc89480 0x55c9ddc93800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.117+0000 7fad5df66640 1 -- 172.21.15.62:0/340331902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ddba21f0 con 0x55c9ddc88b60 2024-09-12T00:03:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.122+0000 7fad5df66640 1 --2- 172.21.15.62:0/340331902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc88b60 0x55c9ddc88f40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fad50009f50 tx=0x7fad500118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e01e11402baa8fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.123+0000 7fad437fe640 1 -- 172.21.15.62:0/340331902 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50004b20 con 0x55c9ddc88b60 2024-09-12T00:03:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.123+0000 7fad4e7fc640 1 -- 172.21.15.62:0/1088701740 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fad48018b20 con 0x7fad58102570 2024-09-12T00:03:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.123+0000 7fad437fe640 1 -- 172.21.15.62:0/340331902 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad50004cc0 con 0x55c9ddc88b60 2024-09-12T00:03:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.124+0000 7fad4e7fc640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad440a2f00 0x7fad440a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.124+0000 7fad61da2740 1 -- 172.21.15.62:0/340331902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc88b60 msgr2=0x55c9ddc88f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.124+0000 7fad61da2740 1 --2- 172.21.15.62:0/340331902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc88b60 0x55c9ddc88f40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fad50009f50 tx=0x7fad500118d0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.125+0000 7fad4e7fc640 1 -- 172.21.15.62:0/1088701740 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fad480cf660 con 0x7fad58102570 2024-09-12T00:03:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.125+0000 7fad61da2740 1 -- 172.21.15.62:0/340331902 shutdown_connections 2024-09-12T00:03:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.125+0000 7fad61da2740 1 --2- 172.21.15.62:0/340331902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc93d40 0x55c9ddc96130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.125+0000 7fad61da2740 1 --2- 172.21.15.62:0/340331902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc89480 0x55c9ddc93800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.125+0000 7fad61da2740 1 --2- 172.21.15.62:0/340331902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc88b60 0x55c9ddc88f40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.125+0000 7fad61da2740 1 -- 172.21.15.62:0/340331902 >> 172.21.15.62:0/340331902 conn(0x55c9ddc850f0 msgr2=0x55c9ddc854d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.125+0000 7fad61da2740 1 -- 172.21.15.62:0/340331902 shutdown_connections 2024-09-12T00:03:35.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.133+0000 7fad5d765640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad440a2f00 0x7fad440a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.133+0000 7fad61da2740 1 -- 172.21.15.62:0/340331902 wait complete. 2024-09-12T00:03:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.133+0000 7fad61da2740 1 -- start start 2024-09-12T00:03:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.134+0000 7fad61da2740 1 Processor -- start 2024-09-12T00:03:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.137+0000 7fad61da2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc89480 0x55c9ddc936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.137+0000 7fad61da2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc93d40 0x55c9ddc90960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.137+0000 7fad61da2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc912e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.137+0000 7fad61da2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c9ddc96960 con 0x55c9ddc90ea0 2024-09-12T00:03:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.137+0000 7fad61da2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c9ddba8f10 con 0x55c9ddc93d40 2024-09-12T00:03:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.137+0000 7fad61da2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9ddc96ae0 con 0x55c9ddc89480 2024-09-12T00:03:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.137+0000 7fad61da2740 10 client.0 _ll_register_callbacks cb 0x55c9ddb71a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.137+0000 7fad5cf64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc93d40 0x55c9ddc90960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.137+0000 7fad5cf64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc93d40 0x55c9ddc90960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40156/0 (socket says 172.21.15.62:40156) 2024-09-12T00:03:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.137+0000 7fad5cf64640 1 -- 172.21.15.62:0/1376728762 learned_addr learned my addr 172.21.15.62:0/1376728762 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.138+0000 7fad5d765640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad440a2f00 0x7fad440a53c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55c9ddc85f50 tx=0x7fad5400b040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.138+0000 7fad5d765640 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc89480 0x55c9ddc936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.138+0000 7fad5cf64640 1 -- 172.21.15.62:0/1376728762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc89480 msgr2=0x55c9ddc936b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.138+0000 7fad5cf64640 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc89480 0x55c9ddc936b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.138+0000 7fad5cf64640 1 -- 172.21.15.62:0/1376728762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc90ea0 msgr2=0x55c9ddc912e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.138+0000 7fad5cf64640 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc912e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.138+0000 7fad5cf64640 1 -- 172.21.15.62:0/1376728762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9dd95f6a0 con 0x55c9ddc93d40 2024-09-12T00:03:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.139+0000 7fad5cf64640 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc93d40 0x55c9ddc90960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fad480ccdb0 tx=0x7fad4809bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edbb9b2a38a51e21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.139+0000 7fad40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:03:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.139+0000 7fad40ff9640 1 -- 172.21.15.62:0/1376728762 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48010cc0 con 0x55c9ddc93d40 2024-09-12T00:03:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.139+0000 7fad40ff9640 1 -- 172.21.15.62:0/1376728762 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fad4809b020 con 0x55c9ddc93d40 2024-09-12T00:03:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.142+0000 7fad61da2740 1 -- 172.21.15.62:0/1376728762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c9ddbfb7f0 con 0x55c9ddc93d40 2024-09-12T00:03:35.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.144+0000 7fad40ff9640 1 -- 172.21.15.62:0/1376728762 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fad4809b220 con 0x55c9ddc93d40 2024-09-12T00:03:35.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.144+0000 7fad61da2740 10 client.26993 fetch_fsmap learned FSMap version 35 2024-09-12T00:03:35.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.144+0000 7fad61da2740 1 -- 172.21.15.62:0/1376728762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c9ddca28a0 con 0x55c9ddc93d40 2024-09-12T00:03:35.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.145+0000 7fad40ff9640 1 -- 172.21.15.62:0/1376728762 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c9ddbfb7f0 con 0x55c9ddc93d40 2024-09-12T00:03:35.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.145+0000 7fad40ff9640 1 -- 172.21.15.62:0/1376728762 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fad48097ab0 con 0x55c9ddc93d40 2024-09-12T00:03:35.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.145+0000 7fad61da2740 10 client.26993 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T00:03:35.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.145+0000 7fad61da2740 10 client.26993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:03:35.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.145+0000 7fad61da2740 20 client.26993 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:35.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.145+0000 7fad61da2740 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c9ddca3e80 0x55c9ddca62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.145+0000 7fad61da2740 4 client.26993 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:03:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.145+0000 7fad61da2740 1 -- 172.21.15.62:0/1376728762 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9ddbf5d00 con 0x55c9ddca3e80 2024-09-12T00:03:35.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.147+0000 7fad5d765640 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c9ddca3e80 0x55c9ddca62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.157+0000 7fad40ff9640 10 client.26993 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:35.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.157+0000 7fad5d765640 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c9ddca3e80 0x55c9ddca62a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb12ed27630b336 server_cookie=2bd035133558c0bb in_seq=0 out_seq=0 2024-09-12T00:03:35.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.157+0000 7fad40ff9640 1 -- 172.21.15.62:0/1376728762 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9ddbf5d00 con 0x55c9ddca3e80 2024-09-12T00:03:35.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.158+0000 7fad40ff9640 10 client.26993 handle_command_reply: tid=0 2024-09-12T00:03:35.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.158+0000 7fad61da2740 1 client.26993 shutdown 2024-09-12T00:03:35.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.158+0000 7fad61da2740 1 -- 172.21.15.62:0/1376728762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc93d40 msgr2=0x55c9ddc90960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:35.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.158+0000 7fad61da2740 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc93d40 0x55c9ddc90960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fad480ccdb0 tx=0x7fad4809bd10 comp rx=0 tx=0).stop 2024-09-12T00:03:35.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.158+0000 7fad61da2740 1 -- 172.21.15.62:0/1376728762 shutdown_connections 2024-09-12T00:03:35.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.158+0000 7fad61da2740 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c9ddca3e80 0x55c9ddca62a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.158+0000 7fad61da2740 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc912e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.158+0000 7fad61da2740 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc93d40 0x55c9ddc90960 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fad480ccdb0 tx=0x7fad4809bd10 comp rx=0 tx=0).stop 2024-09-12T00:03:35.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.158+0000 7fad61da2740 1 --2- 172.21.15.62:0/1376728762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc89480 0x55c9ddc936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.158+0000 7fad61da2740 1 -- 172.21.15.62:0/1376728762 >> 172.21.15.62:0/1376728762 conn(0x55c9ddc850f0 msgr2=0x55c9ddc83a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:35.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.159+0000 7fad40ff9640 0 client.26993 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:35.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.159+0000 7fad61da2740 1 -- 172.21.15.62:0/1376728762 shutdown_connections 2024-09-12T00:03:35.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.159+0000 7fad61da2740 1 -- 172.21.15.62:0/1376728762 wait complete. 2024-09-12T00:03:35.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.159+0000 7fad61da2740 20 client.26993 trim_cache size 0 max 16384 2024-09-12T00:03:35.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad61da2740 1 Processor -- start 2024-09-12T00:03:35.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad61da2740 1 -- start start 2024-09-12T00:03:35.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad61da2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 0x55c9ddc84160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad61da2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc95130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad61da2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc95670 0x55c9ddcabb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad61da2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c011220 con 0x55c9ddc95670 2024-09-12T00:03:35.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad61da2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c00ea90 con 0x55c9ddc89480 2024-09-12T00:03:35.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad61da2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c00c900 con 0x55c9ddc90ea0 2024-09-12T00:03:35.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad5df66640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc95670 0x55c9ddcabb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad5df66640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc95670 0x55c9ddcabb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55782/0 (socket says 172.21.15.62:55782) 2024-09-12T00:03:35.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad5df66640 1 -- 172.21.15.62:0/1067174752 learned_addr learned my addr 172.21.15.62:0/1067174752 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.191+0000 7fad5cf64640 1 --2- 172.21.15.62:0/1067174752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc95130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad5d765640 1 --2- 172.21.15.62:0/1067174752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 0x55c9ddc84160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad5df66640 1 -- 172.21.15.62:0/1067174752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc90ea0 msgr2=0x55c9ddc95130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad5df66640 1 --2- 172.21.15.62:0/1067174752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc95130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad5df66640 1 -- 172.21.15.62:0/1067174752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 msgr2=0x55c9ddc84160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad5df66640 1 --2- 172.21.15.62:0/1067174752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 0x55c9ddc84160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad5df66640 1 -- 172.21.15.62:0/1067174752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ddc93a20 con 0x55c9ddc95670 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad5cf64640 1 --2- 172.21.15.62:0/1067174752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc95130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad5df66640 1 --2- 172.21.15.62:0/1067174752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc95670 0x55c9ddcabb70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fad50012fd0 tx=0x7fad50005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cae1669ffde63d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad42ffd640 1 -- 172.21.15.62:0/1067174752 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50027020 con 0x55c9ddc95670 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad42ffd640 1 -- 172.21.15.62:0/1067174752 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad50003970 con 0x55c9ddc95670 2024-09-12T00:03:35.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.192+0000 7fad42ffd640 1 -- 172.21.15.62:0/1067174752 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5000fa10 con 0x55c9ddc95670 2024-09-12T00:03:35.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.193+0000 7fad61da2740 1 -- 172.21.15.62:0/1067174752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc95670 msgr2=0x55c9ddcabb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:35.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.193+0000 7fad61da2740 1 --2- 172.21.15.62:0/1067174752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc95670 0x55c9ddcabb70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fad50012fd0 tx=0x7fad50005bd0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.194+0000 7fad61da2740 1 -- 172.21.15.62:0/1067174752 shutdown_connections 2024-09-12T00:03:35.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.194+0000 7fad61da2740 1 --2- 172.21.15.62:0/1067174752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc95670 0x55c9ddcabb70 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.194+0000 7fad61da2740 1 --2- 172.21.15.62:0/1067174752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc95130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.194+0000 7fad61da2740 1 --2- 172.21.15.62:0/1067174752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 0x55c9ddc84160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.194+0000 7fad61da2740 1 -- 172.21.15.62:0/1067174752 >> 172.21.15.62:0/1067174752 conn(0x55c9ddc850f0 msgr2=0x55c9ddc83a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:35.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.194+0000 7fad61da2740 1 -- 172.21.15.62:0/1067174752 shutdown_connections 2024-09-12T00:03:35.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.194+0000 7fad61da2740 1 -- 172.21.15.62:0/1067174752 wait complete. 2024-09-12T00:03:35.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.195+0000 7fad61da2740 1 -- start start 2024-09-12T00:03:35.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.195+0000 7fad61da2740 1 Processor -- start 2024-09-12T00:03:35.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad61da2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 0x55c9ddc9c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad61da2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc9c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad61da2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc95670 0x55c9ddc96dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad61da2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c00cf50 con 0x55c9ddc90ea0 2024-09-12T00:03:35.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad61da2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c00e3c0 con 0x55c9ddc89480 2024-09-12T00:03:35.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad61da2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c00d620 con 0x55c9ddc95670 2024-09-12T00:03:35.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad61da2740 10 client.0 _ll_register_callbacks cb 0x7fad48097ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:35.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad5cf64640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc9c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad5d765640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 0x55c9ddc9c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad5cf64640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc9c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55796/0 (socket says 172.21.15.62:55796) 2024-09-12T00:03:35.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad5cf64640 1 -- 172.21.15.62:0/4271956694 learned_addr learned my addr 172.21.15.62:0/4271956694 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:35.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.196+0000 7fad5df66640 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc95670 0x55c9ddc96dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.197+0000 7fad5d765640 1 -- 172.21.15.62:0/4271956694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc95670 msgr2=0x55c9ddc96dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.197+0000 7fad5d765640 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc95670 0x55c9ddc96dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.197+0000 7fad5d765640 1 -- 172.21.15.62:0/4271956694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc90ea0 msgr2=0x55c9ddc9c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.197+0000 7fad5d765640 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc9c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.197+0000 7fad5d765640 1 -- 172.21.15.62:0/4271956694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ddc975d0 con 0x55c9ddc89480 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.197+0000 7fad5d765640 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 0x55c9ddc9c000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55c9ddca6b10 tx=0x7fad54038000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b71a83857f150b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.197+0000 7fad40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.198+0000 7fad40ff9640 1 -- 172.21.15.62:0/4271956694 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54040070 con 0x55c9ddc89480 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.198+0000 7fad40ff9640 1 -- 172.21.15.62:0/4271956694 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7fad54032750 con 0x55c9ddc89480 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.198+0000 7fad61da2740 1 -- 172.21.15.62:0/4271956694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad50003970 con 0x55c9ddc89480 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.198+0000 7fad40ff9640 1 -- 172.21.15.62:0/4271956694 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fad54035040 con 0x55c9ddc89480 2024-09-12T00:03:35.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.198+0000 7fad61da2740 10 client.26999 fetch_fsmap learned FSMap version 35 2024-09-12T00:03:35.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.198+0000 7fad61da2740 1 -- 172.21.15.62:0/4271956694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c9ddd0f680 con 0x55c9ddc89480 2024-09-12T00:03:35.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.199+0000 7fad40ff9640 1 -- 172.21.15.62:0/4271956694 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad54032e30 con 0x55c9ddc89480 2024-09-12T00:03:35.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.199+0000 7fad40ff9640 1 -- 172.21.15.62:0/4271956694 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fad5403cb10 con 0x55c9ddc89480 2024-09-12T00:03:35.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.199+0000 7fad61da2740 10 client.26999 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T00:03:35.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.199+0000 7fad61da2740 10 client.26999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:03:35.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.199+0000 7fad61da2740 20 client.26999 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:35.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.199+0000 7fad61da2740 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c9ddd0fdd0 0x55c9ddd121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:35.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.199+0000 7fad61da2740 4 client.26999 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:03:35.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.199+0000 7fad61da2740 1 -- 172.21.15.62:0/4271956694 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fad50003970 con 0x55c9ddd0fdd0 2024-09-12T00:03:35.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.199+0000 7fad5cf64640 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c9ddd0fdd0 0x55c9ddd121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.200+0000 7fad40ff9640 10 client.26999 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.200+0000 7fad5cf64640 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c9ddd0fdd0 0x55c9ddd121f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2b8c1ee14cbb56 server_cookie=598d7496458e1976 in_seq=0 out_seq=0 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad40ff9640 1 -- 172.21.15.62:0/4271956694 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fad50003970 con 0x55c9ddd0fdd0 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad40ff9640 10 client.26999 handle_command_reply: tid=0 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad61da2740 1 client.26999 shutdown 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad61da2740 1 -- 172.21.15.62:0/4271956694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 msgr2=0x55c9ddc9c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad61da2740 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 0x55c9ddc9c000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55c9ddca6b10 tx=0x7fad54038000 comp rx=0 tx=0).stop 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad61da2740 1 -- 172.21.15.62:0/4271956694 shutdown_connections 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad61da2740 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55c9ddd0fdd0 0x55c9ddd121f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad61da2740 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9ddc95670 0x55c9ddc96dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad40ff9640 0 client.26999 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:35.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad61da2740 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9ddc90ea0 0x55c9ddc9c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad61da2740 1 --2- 172.21.15.62:0/4271956694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9ddc89480 0x55c9ddc9c000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad61da2740 1 -- 172.21.15.62:0/4271956694 >> 172.21.15.62:0/4271956694 conn(0x55c9ddc850f0 msgr2=0x55c9ddc8f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:35.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.201+0000 7fad61da2740 1 -- 172.21.15.62:0/4271956694 shutdown_connections 2024-09-12T00:03:35.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.202+0000 7fad61da2740 1 -- 172.21.15.62:0/4271956694 wait complete. 2024-09-12T00:03:35.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.202+0000 7fad61da2740 20 client.26999 trim_cache size 0 max 16384 2024-09-12T00:03:35.203 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:03:35.203 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-12T00:03:35.203 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "849c03b9-818d-4920-8a2c-ec6e459557f4", 2024-09-12T00:03:35.203 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-12T00:03:35.204 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:03:35.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1088701740 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad440a2f00 msgr2=0x7fad440a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad440a2f00 0x7fad440a53c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55c9ddc85f50 tx=0x7fad5400b040 comp rx=0 tx=0).stop 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1088701740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad58102570 msgr2=0x7fad58116880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad58102570 0x7fad58116880 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fad4800e9f0 tx=0x7fad4800eeb0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1088701740 shutdown_connections 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fad440a2f00 0x7fad440a53c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad58102ef0 0x7fad581105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad58102570 0x7fad58116880 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 --2- 172.21.15.62:0/1088701740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad58100e20 0x7fad58116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1088701740 >> 172.21.15.62:0/1088701740 conn(0x7fad580fc7d0 msgr2=0x7fad581056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1088701740 shutdown_connections 2024-09-12T00:03:35.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:35.203+0000 7fad5f9c9640 1 -- 172.21.15.62:0/1088701740 wait complete. 2024-09-12T00:03:35.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:35 smithi062 ceph-mon[30958]: pgmap v357: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-09-12T00:03:35.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:35 smithi120 ceph-mon[35182]: pgmap v357: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-09-12T00:03:35.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:35 smithi083 ceph-mon[34805]: pgmap v357: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-09-12T00:03:36.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:03:36.537 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:03:36.567 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:36 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-12T00:03:36.567 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:36 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:36.567 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:36 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:03:36.567 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:36 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:03:36.567 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:36 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:36.567 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:36 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:03:36.568 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:36 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:36.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:36 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-12T00:03:36.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:36 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:36.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:36 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:03:36.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:36 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:03:36.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:36 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:36.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:36 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:03:36.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:36 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:36.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:36 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-12T00:03:36.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:36 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:36.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:36 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:03:36.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:36 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:03:36.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:36 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:36.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:36 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:03:36.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:36 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:37.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.273+0000 7f3827fff640 1 -- 172.21.15.62:0/4267504581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f382009d790 msgr2=0x7f382009db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:37.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.273+0000 7f3827fff640 1 --2- 172.21.15.62:0/4267504581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f382009d790 0x7f382009db70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3818009930 tx=0x7f381802a420 comp rx=0 tx=0).stop 2024-09-12T00:03:37.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.273+0000 7f3827fff640 1 -- 172.21.15.62:0/4267504581 shutdown_connections 2024-09-12T00:03:37.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.273+0000 7f3827fff640 1 --2- 172.21.15.62:0/4267504581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3820098130 0x7f38200a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.273+0000 7f3827fff640 1 --2- 172.21.15.62:0/4267504581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3820097790 0x7f3820097bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.273+0000 7f3827fff640 1 --2- 172.21.15.62:0/4267504581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f382009d790 0x7f382009db70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.273+0000 7f3827fff640 1 -- 172.21.15.62:0/4267504581 >> 172.21.15.62:0/4267504581 conn(0x7f3820093490 msgr2=0x7f38200958b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:37.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.273+0000 7f3827fff640 1 -- 172.21.15.62:0/4267504581 shutdown_connections 2024-09-12T00:03:37.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.273+0000 7f3827fff640 1 -- 172.21.15.62:0/4267504581 wait complete. 2024-09-12T00:03:37.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.274+0000 7f3827fff640 1 Processor -- start 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.274+0000 7f3827fff640 1 -- start start 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f3827fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3820097790 0x7f3820008bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f3827fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3820098130 0x7f38200090f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f3827fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f382009d790 0x7f3820009630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f38267fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3820098130 0x7f38200090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f38267fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3820098130 0x7f38200090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55822/0 (socket says 172.21.15.62:55822) 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3820097790 0x7f3820008bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3820097790 0x7f3820008bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40194/0 (socket says 172.21.15.62:40194) 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f38267fc640 1 -- 172.21.15.62:0/3497111207 learned_addr learned my addr 172.21.15.62:0/3497111207 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f38200a7ba0 con 0x7f3820098130 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f38277fe640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f382009d790 0x7f3820009630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38200a7a20 con 0x7f3820097790 2024-09-12T00:03:37.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.275+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38200a7d20 con 0x7f382009d790 2024-09-12T00:03:37.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.276+0000 7f38267fc640 1 -- 172.21.15.62:0/3497111207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f382009d790 msgr2=0x7f3820009630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:37.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.276+0000 7f38267fc640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f382009d790 0x7f3820009630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.276+0000 7f38267fc640 1 -- 172.21.15.62:0/3497111207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3820097790 msgr2=0x7f3820008bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:37.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.276+0000 7f38267fc640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3820097790 0x7f3820008bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.276+0000 7f38267fc640 1 -- 172.21.15.62:0/3497111207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382013e700 con 0x7f3820098130 2024-09-12T00:03:37.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.276+0000 7f38267fc640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3820098130 0x7f38200090f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f381c00ea90 tx=0x7f381c00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:37.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.276+0000 7f380ffff640 1 -- 172.21.15.62:0/3497111207 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c00ce40 con 0x7f3820098130 2024-09-12T00:03:37.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.276+0000 7f380ffff640 1 -- 172.21.15.62:0/3497111207 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f381c0035f0 con 0x7f3820098130 2024-09-12T00:03:37.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.276+0000 7f380ffff640 1 -- 172.21.15.62:0/3497111207 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c010690 con 0x7f3820098130 2024-09-12T00:03:37.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.276+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382013e9f0 con 0x7f3820098130 2024-09-12T00:03:37.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.277+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f382013eee0 con 0x7f3820098130 2024-09-12T00:03:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.280+0000 7f382f6df740 1 Processor -- start 2024-09-12T00:03:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.280+0000 7f382f6df740 1 -- start start 2024-09-12T00:03:37.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.280+0000 7f382f6df740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8670 0x557fa71c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.280+0000 7f382f6df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.280+0000 7f382f6df740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.281+0000 7f382f6df740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557fa712e2e0 con 0x557fa71d3850 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.281+0000 7f382f6df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557fa70d5ce0 con 0x557fa71c8f90 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.281+0000 7f382f6df740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557fa712a2a0 con 0x557fa71c8670 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.281+0000 7f38277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8670 0x557fa71c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.281+0000 7f380ffff640 1 -- 172.21.15.62:0/3497111207 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f381c0037a0 con 0x7f3820098130 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.281+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.281+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40204/0 (socket says 172.21.15.62:40204) 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.281+0000 7f3826ffd640 1 -- 172.21.15.62:0/700194698 learned_addr learned my addr 172.21.15.62:0/700194698 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.282+0000 7f3826ffd640 1 -- 172.21.15.62:0/700194698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8670 msgr2=0x557fa71c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.281+0000 7f38267fc640 1 --2- 172.21.15.62:0/700194698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.282+0000 7f3826ffd640 1 --2- 172.21.15.62:0/700194698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8670 0x557fa71c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.282+0000 7f3826ffd640 1 -- 172.21.15.62:0/700194698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 msgr2=0x557fa71d5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.282+0000 7f3826ffd640 1 --2- 172.21.15.62:0/700194698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.282+0000 7f3826ffd640 1 -- 172.21.15.62:0/700194698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa70ba440 con 0x557fa71c8f90 2024-09-12T00:03:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.282+0000 7f38267fc640 1 --2- 172.21.15.62:0/700194698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:03:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.282+0000 7f380ffff640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f38100a2d80 0x7f38100a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.282+0000 7f38277fe640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f38100a2d80 0x7f38100a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.283+0000 7f380ffff640 1 -- 172.21.15.62:0/3497111207 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f381c0cf770 con 0x7f3820098130 2024-09-12T00:03:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.283+0000 7f3826ffd640 1 --2- 172.21.15.62:0/700194698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d3310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3818009930 tx=0x7f3818009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9fcf74415aa0136 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.283+0000 7f380d7fa640 1 -- 172.21.15.62:0/700194698 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818035040 con 0x557fa71c8f90 2024-09-12T00:03:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.283+0000 7f380d7fa640 1 -- 172.21.15.62:0/700194698 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3818004480 con 0x557fa71c8f90 2024-09-12T00:03:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.283+0000 7f380d7fa640 1 -- 172.21.15.62:0/700194698 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818004e60 con 0x557fa71c8f90 2024-09-12T00:03:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.283+0000 7f38277fe640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f38100a2d80 0x7f38100a5240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3828067c70 tx=0x7f38280505d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.284+0000 7f382f6df740 1 -- 172.21.15.62:0/700194698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 msgr2=0x557fa71d3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.284+0000 7f382f6df740 1 --2- 172.21.15.62:0/700194698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d3310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3818009930 tx=0x7f3818009650 comp rx=0 tx=0).stop 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.285+0000 7f382f6df740 1 -- 172.21.15.62:0/700194698 shutdown_connections 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.285+0000 7f382f6df740 1 --2- 172.21.15.62:0/700194698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.285+0000 7f382f6df740 1 --2- 172.21.15.62:0/700194698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d3310 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.285+0000 7f382f6df740 1 --2- 172.21.15.62:0/700194698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8670 0x557fa71c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.285+0000 7f382f6df740 1 -- 172.21.15.62:0/700194698 >> 172.21.15.62:0/700194698 conn(0x557fa71c4c00 msgr2=0x557fa71c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.285+0000 7f382f6df740 1 -- 172.21.15.62:0/700194698 shutdown_connections 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.285+0000 7f382f6df740 1 -- 172.21.15.62:0/700194698 wait complete. 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f382f6df740 1 -- start start 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f382f6df740 1 Processor -- start 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f382f6df740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71c8670 0x557fa71d3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f382f6df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f382f6df740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71d3850 0x557fa71d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f382f6df740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557fa71d6420 con 0x557fa71c8670 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f382f6df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557fa70e9050 con 0x557fa71c8f90 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f382f6df740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557fa71d65a0 con 0x557fa71d3850 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f382f6df740 10 client.0 _ll_register_callbacks cb 0x557fa7129160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71c8670 0x557fa71d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71c8670 0x557fa71d3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55828/0 (socket says 172.21.15.62:55828) 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f3826ffd640 1 -- 172.21.15.62:0/3491676702 learned_addr learned my addr 172.21.15.62:0/3491676702 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f38267fc640 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f38277fe640 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71d3850 0x557fa71d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.286+0000 7f38267fc640 1 -- 172.21.15.62:0/3491676702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71d3850 msgr2=0x557fa71d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f38267fc640 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71d3850 0x557fa71d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f38267fc640 1 -- 172.21.15.62:0/3491676702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71c8670 msgr2=0x557fa71d3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f38267fc640 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71c8670 0x557fa71d3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f38267fc640 1 -- 172.21.15.62:0/3491676702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa6e9f3c0 con 0x557fa71c8f90 2024-09-12T00:03:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f38267fc640 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d03c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f381c0ccec0 tx=0x7f381c09bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fc728651976ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f380aff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:03:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f380aff5640 1 -- 172.21.15.62:0/3491676702 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c09bc90 con 0x557fa71c8f90 2024-09-12T00:03:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f380aff5640 1 -- 172.21.15.62:0/3491676702 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f381c098300 con 0x557fa71c8f90 2024-09-12T00:03:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f382f6df740 1 -- 172.21.15.62:0/3491676702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3818004480 con 0x557fa71c8f90 2024-09-12T00:03:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f380aff5640 1 -- 172.21.15.62:0/3491676702 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f381c0a0040 con 0x557fa71c8f90 2024-09-12T00:03:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f382f6df740 10 client.27011 fetch_fsmap learned FSMap version 36 2024-09-12T00:03:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f382f6df740 1 -- 172.21.15.62:0/3491676702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557fa71e1da0 con 0x557fa71c8f90 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f380aff5640 1 -- 172.21.15.62:0/3491676702 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3818004480 con 0x557fa71c8f90 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.287+0000 7f380aff5640 1 -- 172.21.15.62:0/3491676702 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f381c0979e0 con 0x557fa71c8f90 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.288+0000 7f382f6df740 10 client.27011 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.288+0000 7f382f6df740 10 client.27011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.288+0000 7f382f6df740 20 client.27011 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.288+0000 7f382f6df740 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fa71e3270 0x557fa71e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.288+0000 7f382f6df740 4 client.27011 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.288+0000 7f382f6df740 1 -- 172.21.15.62:0/3491676702 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fa713b850 con 0x557fa71e3270 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.288+0000 7f3826ffd640 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fa71e3270 0x557fa71e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.288+0000 7f380aff5640 10 client.27011 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.288+0000 7f3826ffd640 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fa71e3270 0x557fa71e56f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247d036b84e3a84d server_cookie=bfec63dc7171854b in_seq=0 out_seq=0 2024-09-12T00:03:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f380aff5640 1 -- 172.21.15.62:0/3491676702 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fa713b850 con 0x557fa71e3270 2024-09-12T00:03:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f380aff5640 10 client.27011 handle_command_reply: tid=0 2024-09-12T00:03:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 client.27011 shutdown 2024-09-12T00:03:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 -- 172.21.15.62:0/3491676702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 msgr2=0x557fa71d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d03c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f381c0ccec0 tx=0x7f381c09bb20 comp rx=0 tx=0).stop 2024-09-12T00:03:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 -- 172.21.15.62:0/3491676702 shutdown_connections 2024-09-12T00:03:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fa71e3270 0x557fa71e56f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71d3850 0x557fa71d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71d03c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 --2- 172.21.15.62:0/3491676702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71c8670 0x557fa71d3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f380aff5640 0 client.27011 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:37.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 -- 172.21.15.62:0/3491676702 >> 172.21.15.62:0/3491676702 conn(0x557fa71c4c00 msgr2=0x557fa71ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:37.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 -- 172.21.15.62:0/3491676702 shutdown_connections 2024-09-12T00:03:37.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 1 -- 172.21.15.62:0/3491676702 wait complete. 2024-09-12T00:03:37.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.289+0000 7f382f6df740 20 client.27011 trim_cache size 0 max 16384 2024-09-12T00:03:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f382f6df740 1 Processor -- start 2024-09-12T00:03:37.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f382f6df740 1 -- start start 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f382f6df740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8670 0x557fa71df6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f382f6df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f382f6df740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f382f6df740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3820141a40 con 0x557fa71d3850 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f382f6df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f382013f2b0 con 0x557fa71c8f90 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f382f6df740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3820097bf0 con 0x557fa71c8670 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f38277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f38277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71eafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55838/0 (socket says 172.21.15.62:55838) 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8670 0x557fa71df6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f38277fe640 1 -- 172.21.15.62:0/1571106299 learned_addr learned my addr 172.21.15.62:0/1571106299 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.315+0000 7f38267fc640 1 --2- 172.21.15.62:0/1571106299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.316+0000 7f38267fc640 1 -- 172.21.15.62:0/1571106299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8670 msgr2=0x557fa71df6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.316+0000 7f38267fc640 1 --2- 172.21.15.62:0/1571106299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8670 0x557fa71df6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.316+0000 7f38267fc640 1 -- 172.21.15.62:0/1571106299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 msgr2=0x557fa71eafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.316+0000 7f38267fc640 1 --2- 172.21.15.62:0/1571106299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71eafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.316+0000 7f38267fc640 1 -- 172.21.15.62:0/1571106299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa71cf370 con 0x557fa71c8f90 2024-09-12T00:03:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.316+0000 7f38267fc640 1 --2- 172.21.15.62:0/1571106299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71bfb90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c760 tx=0x7f381c09bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b225ac72c631420 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:37.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.317+0000 7f380cff9640 1 -- 172.21.15.62:0/1571106299 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c09bc90 con 0x557fa71c8f90 2024-09-12T00:03:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.317+0000 7f380cff9640 1 -- 172.21.15.62:0/1571106299 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f381c0a4900 con 0x557fa71c8f90 2024-09-12T00:03:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.318+0000 7f380cff9640 1 -- 172.21.15.62:0/1571106299 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c07adb0 con 0x557fa71c8f90 2024-09-12T00:03:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.319+0000 7f382f6df740 1 -- 172.21.15.62:0/1571106299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 msgr2=0x557fa71bfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.319+0000 7f382f6df740 1 --2- 172.21.15.62:0/1571106299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71bfb90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c760 tx=0x7f381c09bb90 comp rx=0 tx=0).stop 2024-09-12T00:03:37.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.319+0000 7f382f6df740 1 -- 172.21.15.62:0/1571106299 shutdown_connections 2024-09-12T00:03:37.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.319+0000 7f382f6df740 1 --2- 172.21.15.62:0/1571106299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.319+0000 7f382f6df740 1 --2- 172.21.15.62:0/1571106299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8f90 0x557fa71bfb90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.319+0000 7f382f6df740 1 --2- 172.21.15.62:0/1571106299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8670 0x557fa71df6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.319+0000 7f382f6df740 1 -- 172.21.15.62:0/1571106299 >> 172.21.15.62:0/1571106299 conn(0x557fa71c4c00 msgr2=0x557fa71cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:37.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.319+0000 7f382f6df740 1 -- 172.21.15.62:0/1571106299 shutdown_connections 2024-09-12T00:03:37.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.319+0000 7f382f6df740 1 -- 172.21.15.62:0/1571106299 wait complete. 2024-09-12T00:03:37.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.320+0000 7f382f6df740 1 -- start start 2024-09-12T00:03:37.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.320+0000 7f382f6df740 1 Processor -- start 2024-09-12T00:03:37.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.320+0000 7f382f6df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8670 0x557fa71d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.320+0000 7f382f6df740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8f90 0x557fa71d7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.320+0000 7f382f6df740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.320+0000 7f382f6df740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3820134c00 con 0x557fa71d3850 2024-09-12T00:03:37.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f382f6df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3820136070 con 0x557fa71c8670 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f382f6df740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38201352d0 con 0x557fa71c8f90 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f382f6df740 10 client.0 _ll_register_callbacks cb 0x7f381c003d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55844/0 (socket says 172.21.15.62:55844) 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38277fe640 1 -- 172.21.15.62:0/2033415042 learned_addr learned my addr 172.21.15.62:0/2033415042 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38267fc640 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8f90 0x557fa71d7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f3826ffd640 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8670 0x557fa71d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38277fe640 1 -- 172.21.15.62:0/2033415042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8f90 msgr2=0x557fa71d7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38277fe640 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8f90 0x557fa71d7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38277fe640 1 -- 172.21.15.62:0/2033415042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8670 msgr2=0x557fa71d75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38277fe640 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8670 0x557fa71d75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38277fe640 1 -- 172.21.15.62:0/2033415042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa724cfc0 con 0x557fa71d3850 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38267fc640 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8f90 0x557fa71d7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f38277fe640 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d8040 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3828050280 tx=0x7f3828068410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f79ce4a6de04b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.321+0000 7f380aff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.322+0000 7f380aff5640 1 -- 172.21.15.62:0/2033415042 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382807fb90 con 0x557fa71d3850 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.322+0000 7f380aff5640 1 -- 172.21.15.62:0/2033415042 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8513+0+0 (secure 0 0 0) 0x7f3828086070 con 0x557fa71d3850 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.322+0000 7f382f6df740 1 -- 172.21.15.62:0/2033415042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f381c0a4900 con 0x557fa71d3850 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.322+0000 7f380aff5640 1 -- 172.21.15.62:0/2033415042 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f382808c420 con 0x557fa71d3850 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.322+0000 7f382f6df740 10 client.17484 fetch_fsmap learned FSMap version 36 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.322+0000 7f382f6df740 1 -- 172.21.15.62:0/2033415042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557fa724e2f0 con 0x557fa71d3850 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.322+0000 7f380aff5640 1 -- 172.21.15.62:0/2033415042 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f381c0a4900 con 0x557fa71d3850 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.322+0000 7f380aff5640 1 -- 172.21.15.62:0/2033415042 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f3828074660 con 0x557fa71d3850 2024-09-12T00:03:37.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.323+0000 7f382f6df740 10 client.17484 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.323+0000 7f382f6df740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.323+0000 7f382f6df740 20 client.17484 populate_metadata read hostname 'smithi062' 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.323+0000 7f382f6df740 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fa724e9f0 0x557fa7250e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.323+0000 7f382f6df740 4 client.17484 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.323+0000 7f382f6df740 1 -- 172.21.15.62:0/2033415042 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557fa7135d60 con 0x557fa724e9f0 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.323+0000 7f3826ffd640 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fa724e9f0 0x557fa7250e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.323+0000 7f380aff5640 10 client.17484 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.323+0000 7f3826ffd640 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fa724e9f0 0x557fa7250e10 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f11d5eb6b9f88db6 server_cookie=a8fbb152a48133df in_seq=0 out_seq=0 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.324+0000 7f380aff5640 1 -- 172.21.15.62:0/2033415042 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557fa7135d60 con 0x557fa724e9f0 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.324+0000 7f380aff5640 10 client.17484 handle_command_reply: tid=0 2024-09-12T00:03:37.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.324+0000 7f382f6df740 1 client.17484 shutdown 2024-09-12T00:03:37.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.326+0000 7f382f6df740 1 -- 172.21.15.62:0/2033415042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 msgr2=0x557fa71d8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:37.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.326+0000 7f382f6df740 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d8040 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3828050280 tx=0x7f3828068410 comp rx=0 tx=0).stop 2024-09-12T00:03:37.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.326+0000 7f382f6df740 1 -- 172.21.15.62:0/2033415042 shutdown_connections 2024-09-12T00:03:37.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.326+0000 7f382f6df740 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557fa724e9f0 0x557fa7250e10 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.326+0000 7f382f6df740 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fa71d3850 0x557fa71d8040 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.326+0000 7f382f6df740 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa71c8f90 0x557fa71d7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.326+0000 7f382f6df740 1 --2- 172.21.15.62:0/2033415042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fa71c8670 0x557fa71d75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.326+0000 7f382f6df740 1 -- 172.21.15.62:0/2033415042 >> 172.21.15.62:0/2033415042 conn(0x557fa71c4c00 msgr2=0x557fa71d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.326+0000 7f380aff5640 0 client.17484 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:03:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.326+0000 7f382f6df740 1 -- 172.21.15.62:0/2033415042 shutdown_connections 2024-09-12T00:03:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.327+0000 7f382f6df740 1 -- 172.21.15.62:0/2033415042 wait complete. 2024-09-12T00:03:37.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.327+0000 7f382f6df740 20 client.17484 trim_cache size 0 max 16384 2024-09-12T00:03:37.330 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:03:37.330 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:03:37.330 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-12T00:03:37.330 INFO:teuthology.orchestra.run.smithi062.stdout: "849c03b9-818d-4920-8a2c-ec6e459557f4": { 2024-09-12T00:03:37.330 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-12T00:03:37.330 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "849c03b9-818d-4920-8a2c-ec6e459557f4", 2024-09-12T00:03:37.330 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-12T00:03:37.330 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:03:37.330 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:03:37.330 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:03:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.327+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f38100a2d80 msgr2=0x7f38100a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.327+0000 7f3827fff640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f38100a2d80 0x7f38100a5240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3828067c70 tx=0x7f38280505d0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.327+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3820098130 msgr2=0x7f38200090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:03:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.327+0000 7f3827fff640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3820098130 0x7f38200090f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f381c00ea90 tx=0x7f381c00ef50 comp rx=0 tx=0).stop 2024-09-12T00:03:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.328+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 shutdown_connections 2024-09-12T00:03:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.328+0000 7f3827fff640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f38100a2d80 0x7f38100a5240 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.328+0000 7f3827fff640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f382009d790 0x7f3820009630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.328+0000 7f3827fff640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3820098130 0x7f38200090f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.328+0000 7f3827fff640 1 --2- 172.21.15.62:0/3497111207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3820097790 0x7f3820008bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:03:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.328+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 >> 172.21.15.62:0/3497111207 conn(0x7f3820093490 msgr2=0x7f3820093c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:03:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.328+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 shutdown_connections 2024-09-12T00:03:37.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:03:37.328+0000 7f3827fff640 1 -- 172.21.15.62:0/3497111207 wait complete. 2024-09-12T00:03:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 655.0 s, 2228.4 tps, lat 44.073 ms stddev 85.428 2024-09-12T00:03:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:37 smithi120 ceph-mon[35182]: pgmap v358: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.2 MiB/s wr, 717 op/s 2024-09-12T00:03:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:37 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:03:37.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:37 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:03:37.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:37 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:03:37.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:37 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:37.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:37 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:37.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:37 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:37.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:37 smithi083 ceph-mon[34805]: pgmap v358: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.2 MiB/s wr, 717 op/s 2024-09-12T00:03:37.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:37 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:03:37.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:37 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:03:37.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:37 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:03:37.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:37 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:37.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:37 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:37.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:37 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:37.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:37 smithi062 ceph-mon[30958]: pgmap v358: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.2 MiB/s wr, 717 op/s 2024-09-12T00:03:37.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:37 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:03:37.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:37 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:03:37.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:37 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:03:37.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:37 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:37.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:37 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:37.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:37 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:03:38.117 INFO:tasks.cephfs.filesystem:scrub status for tag:849c03b9-818d-4920-8a2c-ec6e459557f4 - {'path': '/', 'tag': '849c03b9-818d-4920-8a2c-ec6e459557f4', 'options': 'recursive,force'} 2024-09-12T00:03:39.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:39 smithi120 ceph-mon[35182]: pgmap v359: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 27 MiB/s wr, 1.82k op/s 2024-09-12T00:03:39.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:39 smithi083 ceph-mon[34805]: pgmap v359: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 27 MiB/s wr, 1.82k op/s 2024-09-12T00:03:39.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:39 smithi062 ceph-mon[30958]: pgmap v359: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 27 MiB/s wr, 1.82k op/s 2024-09-12T00:03:41.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:41 smithi120 ceph-mon[35182]: pgmap v360: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2024-09-12T00:03:41.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:41 smithi083 ceph-mon[34805]: pgmap v360: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2024-09-12T00:03:41.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:41 smithi062 ceph-mon[30958]: pgmap v360: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2024-09-12T00:03:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 660.0 s, 1794.8 tps, lat 56.683 ms stddev 63.078 2024-09-12T00:03:43.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:43 smithi120 ceph-mon[35182]: pgmap v361: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 27 MiB/s wr, 1.71k op/s 2024-09-12T00:03:43.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:43 smithi083 ceph-mon[34805]: pgmap v361: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 27 MiB/s wr, 1.71k op/s 2024-09-12T00:03:43.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:43 smithi062 ceph-mon[30958]: pgmap v361: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 27 MiB/s wr, 1.71k op/s 2024-09-12T00:03:45.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:45 smithi120 ceph-mon[35182]: pgmap v362: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 30 MiB/s wr, 1.97k op/s 2024-09-12T00:03:45.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:45 smithi083 ceph-mon[34805]: pgmap v362: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 30 MiB/s wr, 1.97k op/s 2024-09-12T00:03:45.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:45 smithi062 ceph-mon[30958]: pgmap v362: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 30 MiB/s wr, 1.97k op/s 2024-09-12T00:03:46.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:46 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:03:46.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:46 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:03:46.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:46 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:03:46.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:46 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:03:46.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:46 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:03:46.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:46 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:03:46.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:46 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:03:46.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:46 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:03:46.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:46 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:03:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 665.0 s, 2759.8 tps, lat 36.057 ms stddev 43.430 2024-09-12T00:03:47.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:47 smithi120 ceph-mon[35182]: pgmap v363: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 26 MiB/s wr, 1.59k op/s 2024-09-12T00:03:47.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:47 smithi083 ceph-mon[34805]: pgmap v363: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 26 MiB/s wr, 1.59k op/s 2024-09-12T00:03:47.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:47 smithi062 ceph-mon[30958]: pgmap v363: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 26 MiB/s wr, 1.59k op/s 2024-09-12T00:03:49.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:49 smithi120 ceph-mon[35182]: pgmap v364: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 28 MiB/s wr, 1.84k op/s 2024-09-12T00:03:49.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:49 smithi083 ceph-mon[34805]: pgmap v364: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 28 MiB/s wr, 1.84k op/s 2024-09-12T00:03:49.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:49 smithi062 ceph-mon[30958]: pgmap v364: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 28 MiB/s wr, 1.84k op/s 2024-09-12T00:03:51.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:51 smithi120 ceph-mon[35182]: pgmap v365: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.7 MiB/s wr, 772 op/s 2024-09-12T00:03:51.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:51 smithi083 ceph-mon[34805]: pgmap v365: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.7 MiB/s wr, 772 op/s 2024-09-12T00:03:51.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:51 smithi062 ceph-mon[30958]: pgmap v365: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.7 MiB/s wr, 772 op/s 2024-09-12T00:03:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 670.0 s, 2582.2 tps, lat 38.793 ms stddev 45.418 2024-09-12T00:03:53.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:53 smithi120 ceph-mon[35182]: pgmap v366: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.5 MiB/s wr, 545 op/s 2024-09-12T00:03:53.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:53 smithi083 ceph-mon[34805]: pgmap v366: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.5 MiB/s wr, 545 op/s 2024-09-12T00:03:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:53 smithi062 ceph-mon[30958]: pgmap v366: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.5 MiB/s wr, 545 op/s 2024-09-12T00:03:55.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:55 smithi120 ceph-mon[35182]: pgmap v367: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.9 MiB/s wr, 899 op/s 2024-09-12T00:03:55.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:55 smithi083 ceph-mon[34805]: pgmap v367: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.9 MiB/s wr, 899 op/s 2024-09-12T00:03:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:55 smithi062 ceph-mon[30958]: pgmap v367: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.9 MiB/s wr, 899 op/s 2024-09-12T00:03:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 675.0 s, 2327.2 tps, lat 43.101 ms stddev 46.496 2024-09-12T00:03:57.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:57 smithi120 ceph-mon[35182]: pgmap v368: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 641 op/s 2024-09-12T00:03:57.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:57 smithi083 ceph-mon[34805]: pgmap v368: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 641 op/s 2024-09-12T00:03:57.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:57 smithi062 ceph-mon[30958]: pgmap v368: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 641 op/s 2024-09-12T00:03:59.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:03:59 smithi120 ceph-mon[35182]: pgmap v369: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.9 MiB/s wr, 900 op/s 2024-09-12T00:03:59.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:03:59 smithi083 ceph-mon[34805]: pgmap v369: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.9 MiB/s wr, 900 op/s 2024-09-12T00:03:59.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:03:59 smithi062 ceph-mon[30958]: pgmap v369: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.9 MiB/s wr, 900 op/s 2024-09-12T00:04:01.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:01 smithi120 ceph-mon[35182]: pgmap v370: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.8 MiB/s wr, 735 op/s 2024-09-12T00:04:01.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:01 smithi083 ceph-mon[34805]: pgmap v370: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.8 MiB/s wr, 735 op/s 2024-09-12T00:04:01.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:01 smithi062 ceph-mon[30958]: pgmap v370: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.8 MiB/s wr, 735 op/s 2024-09-12T00:04:02.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 680.0 s, 1306.8 tps, lat 68.158 ms stddev 141.620 2024-09-12T00:04:03.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:03 smithi120 ceph-mon[35182]: pgmap v371: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 698 op/s 2024-09-12T00:04:03.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:03 smithi083 ceph-mon[34805]: pgmap v371: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 698 op/s 2024-09-12T00:04:03.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:03 smithi062 ceph-mon[30958]: pgmap v371: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 698 op/s 2024-09-12T00:04:05.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:05 smithi120 ceph-mon[35182]: pgmap v372: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 25 MiB/s wr, 1.87k op/s 2024-09-12T00:04:05.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:05 smithi083 ceph-mon[34805]: pgmap v372: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 25 MiB/s wr, 1.87k op/s 2024-09-12T00:04:05.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:05 smithi062 ceph-mon[30958]: pgmap v372: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 25 MiB/s wr, 1.87k op/s 2024-09-12T00:04:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 685.0 s, 1968.8 tps, lat 56.211 ms stddev 118.019 2024-09-12T00:04:07.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:07 smithi083 ceph-mon[34805]: pgmap v373: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 21 MiB/s wr, 1.52k op/s 2024-09-12T00:04:07.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:07 smithi083 ceph-mon[34805]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T00:04:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:07 smithi062 ceph-mon[30958]: pgmap v373: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 21 MiB/s wr, 1.52k op/s 2024-09-12T00:04:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:07 smithi062 ceph-mon[30958]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T00:04:07.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:07 smithi120 ceph-mon[35182]: pgmap v373: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 21 MiB/s wr, 1.52k op/s 2024-09-12T00:04:07.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:07 smithi120 ceph-mon[35182]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T00:04:08.119 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:04:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:08.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.983+0000 7f1bd395d640 1 -- 172.21.15.62:0/3770491597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc105ec0 msgr2=0x7f1bcc106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:08.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.983+0000 7f1bd395d640 1 --2- 172.21.15.62:0/3770491597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc105ec0 0x7f1bcc106340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009cd0 tx=0x7f1bc402a8f0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.983+0000 7f1bd395d640 1 -- 172.21.15.62:0/3770491597 shutdown_connections 2024-09-12T00:04:08.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.983+0000 7f1bd395d640 1 --2- 172.21.15.62:0/3770491597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc105ec0 0x7f1bcc106340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.983+0000 7f1bd395d640 1 --2- 172.21.15.62:0/3770491597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bcc1011e0 0x7f1bcc105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.983+0000 7f1bd395d640 1 --2- 172.21.15.62:0/3770491597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bcc1008c0 0x7f1bcc100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.983+0000 7f1bd395d640 1 -- 172.21.15.62:0/3770491597 >> 172.21.15.62:0/3770491597 conn(0x7f1bcc0fc7d0 msgr2=0x7f1bcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:08.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.983+0000 7f1bd395d640 1 -- 172.21.15.62:0/3770491597 shutdown_connections 2024-09-12T00:04:08.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.983+0000 7f1bd395d640 1 -- 172.21.15.62:0/3770491597 wait complete. 2024-09-12T00:04:08.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.984+0000 7f1bd395d640 1 Processor -- start 2024-09-12T00:04:08.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.984+0000 7f1bd395d640 1 -- start start 2024-09-12T00:04:08.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.985+0000 7f1bd395d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bcc1008c0 0x7f1bcc1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:08.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.985+0000 7f1bd395d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc1011e0 0x7f1bcc1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:08.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.985+0000 7f1bd395d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bcc105ec0 0x7f1bcc19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:08.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.985+0000 7f1bd395d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc118610 con 0x7f1bcc1008c0 2024-09-12T00:04:08.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.985+0000 7f1bd395d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc118490 con 0x7f1bcc105ec0 2024-09-12T00:04:08.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.985+0000 7f1bd395d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc118790 con 0x7f1bcc1011e0 2024-09-12T00:04:08.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.985+0000 7f1bd0ef8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc1011e0 0x7f1bcc1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.985+0000 7f1bd16f9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bcc1008c0 0x7f1bcc1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.985+0000 7f1bd1efa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bcc105ec0 0x7f1bcc19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd1efa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bcc105ec0 0x7f1bcc19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:54236/0 (socket says 172.21.15.62:54236) 2024-09-12T00:04:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd1efa640 1 -- 172.21.15.62:0/3593152105 learned_addr learned my addr 172.21.15.62:0/3593152105 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd1efa640 1 -- 172.21.15.62:0/3593152105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc1011e0 msgr2=0x7f1bcc1a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd1efa640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc1011e0 0x7f1bcc1a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd1efa640 1 -- 172.21.15.62:0/3593152105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bcc1008c0 msgr2=0x7f1bcc1a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd1efa640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bcc1008c0 0x7f1bcc1a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd1efa640 1 -- 172.21.15.62:0/3593152105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc19ef90 con 0x7f1bcc105ec0 2024-09-12T00:04:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bcc1008c0 0x7f1bcc1a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:04:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc1011e0 0x7f1bcc1a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:04:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd1efa640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bcc105ec0 0x7f1bcc19e860 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4005cc0 tx=0x7f1bc4009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bc27fc640 1 -- 172.21.15.62:0/3593152105 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4003d80 con 0x7f1bcc105ec0 2024-09-12T00:04:08.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bc27fc640 1 -- 172.21.15.62:0/3593152105 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc4040070 con 0x7f1bcc105ec0 2024-09-12T00:04:08.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bd395d640 1 -- 172.21.15.62:0/3593152105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc19f220 con 0x7f1bcc105ec0 2024-09-12T00:04:08.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.986+0000 7f1bc27fc640 1 -- 172.21.15.62:0/3593152105 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4031a30 con 0x7f1bcc105ec0 2024-09-12T00:04:08.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.987+0000 7f1bd395d640 1 -- 172.21.15.62:0/3593152105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc1ab570 con 0x7f1bcc105ec0 2024-09-12T00:04:08.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.989+0000 7f1bc27fc640 1 -- 172.21.15.62:0/3593152105 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1bc40046b0 con 0x7f1bcc105ec0 2024-09-12T00:04:08.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.989+0000 7f1bd5d36740 1 Processor -- start 2024-09-12T00:04:08.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.989+0000 7f1bd5d36740 1 -- start start 2024-09-12T00:04:08.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.989+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7ded880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:08.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.990+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7deddc0 0x55bca7df8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:08.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.990+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7df8680 0x55bca7dfaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:08.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.990+0000 7f1bd5d36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bca7d532e0 con 0x55bca7deddc0 2024-09-12T00:04:08.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.990+0000 7f1bd5d36740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bca7cface0 con 0x55bca7df8680 2024-09-12T00:04:08.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.990+0000 7f1bd5d36740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bca7d4f2a0 con 0x55bca7ded4a0 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.990+0000 7f1bc27fc640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1bb80a2cf0 0x7f1bb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.990+0000 7f1bd16f9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7deddc0 0x55bca7df8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.990+0000 7f1bd0ef8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7ded880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.990+0000 7f1bd0ef8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7ded880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:35034/0 (socket says 172.21.15.62:35034) 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.991+0000 7f1bd0ef8640 1 -- 172.21.15.62:0/80998121 learned_addr learned my addr 172.21.15.62:0/80998121 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.991+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/80998121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7df8680 0x55bca7dfaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.991+0000 7f1bc27fc640 1 -- 172.21.15.62:0/3593152105 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f1bc40f0500 con 0x7f1bcc105ec0 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.991+0000 7f1bd1efa640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1bb80a2cf0 0x7f1bb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.991+0000 7f1bd0ef8640 1 -- 172.21.15.62:0/80998121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7df8680 msgr2=0x55bca7dfaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.991+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/80998121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7df8680 0x55bca7dfaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.991+0000 7f1bd0ef8640 1 -- 172.21.15.62:0/80998121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7deddc0 msgr2=0x55bca7df8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:08.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.991+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/80998121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7deddc0 0x55bca7df8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.991+0000 7f1bd0ef8640 1 -- 172.21.15.62:0/80998121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bca7cdf440 con 0x55bca7ded4a0 2024-09-12T00:04:08.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.991+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/80998121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7deddc0 0x55bca7df8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:04:08.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.992+0000 7f1bd1efa640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1bb80a2cf0 0x7f1bb80a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40bc8a0 tx=0x7f1bc4032f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:08.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.992+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/80998121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7ded880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc011ad0 tx=0x7f1bbc00aa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b88fd2d4f293e9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:08.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.992+0000 7f1bb77fe640 1 -- 172.21.15.62:0/80998121 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00fa90 con 0x55bca7ded4a0 2024-09-12T00:04:08.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.992+0000 7f1bb77fe640 1 -- 172.21.15.62:0/80998121 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc005500 con 0x55bca7ded4a0 2024-09-12T00:04:08.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.992+0000 7f1bb77fe640 1 -- 172.21.15.62:0/80998121 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc004d70 con 0x55bca7ded4a0 2024-09-12T00:04:08.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.993+0000 7f1bd5d36740 1 -- 172.21.15.62:0/80998121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 msgr2=0x55bca7ded880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:08.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.993+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/80998121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7ded880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc011ad0 tx=0x7f1bbc00aa70 comp rx=0 tx=0).stop 2024-09-12T00:04:08.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.993+0000 7f1bd5d36740 1 -- 172.21.15.62:0/80998121 shutdown_connections 2024-09-12T00:04:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.993+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/80998121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7df8680 0x55bca7dfaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.993+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/80998121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7deddc0 0x55bca7df8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.993+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/80998121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7ded880 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.993+0000 7f1bd5d36740 1 -- 172.21.15.62:0/80998121 >> 172.21.15.62:0/80998121 conn(0x55bca7de9a30 msgr2=0x55bca7de9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.993+0000 7f1bd5d36740 1 -- 172.21.15.62:0/80998121 shutdown_connections 2024-09-12T00:04:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.993+0000 7f1bd5d36740 1 -- 172.21.15.62:0/80998121 wait complete. 2024-09-12T00:04:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.993+0000 7f1bd5d36740 1 -- start start 2024-09-12T00:04:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.994+0000 7f1bd5d36740 1 Processor -- start 2024-09-12T00:04:08.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.995+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7ded4a0 0x55bca7df80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:08.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.995+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:08.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.995+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7df8680 0x55bca7df5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:08.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.995+0000 7f1bd5d36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bca7dfb250 con 0x55bca7ded4a0 2024-09-12T00:04:08.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.995+0000 7f1bd5d36740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bca7d0e050 con 0x55bca7deddc0 2024-09-12T00:04:08.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.995+0000 7f1bd5d36740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bca7dfb3d0 con 0x55bca7df8680 2024-09-12T00:04:08.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.995+0000 7f1bd16f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:08.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.995+0000 7f1bd5d36740 10 client.0 _ll_register_callbacks cb 0x55bca7d4e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:08.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd0ef8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7ded4a0 0x55bca7df80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:08.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd16f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:54260/0 (socket says 172.21.15.62:54260) 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd16f9640 1 -- 172.21.15.62:0/4217965378 learned_addr learned my addr 172.21.15.62:0/4217965378 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7df8680 0x55bca7df5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd16f9640 1 -- 172.21.15.62:0/4217965378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7df8680 msgr2=0x55bca7df5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7df8680 0x55bca7df5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd16f9640 1 -- 172.21.15.62:0/4217965378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7ded4a0 msgr2=0x55bca7df80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7ded4a0 0x55bca7df80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd16f9640 1 -- 172.21.15.62:0/4217965378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bca7ac43c0 con 0x55bca7deddc0 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df5360 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8013710 tx=0x7f1bc8013bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8337f99c12eb4d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/4217965378 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8011d50 con 0x55bca7deddc0 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.996+0000 7f1bd5d36740 1 -- 172.21.15.62:0/4217965378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bbc005500 con 0x55bca7deddc0 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/4217965378 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f1bc8004020 con 0x55bca7deddc0 2024-09-12T00:04:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/4217965378 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1bc8004220 con 0x55bca7deddc0 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bd5d36740 10 client.27029 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bd5d36740 1 -- 172.21.15.62:0/4217965378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bca7e06d80 con 0x55bca7deddc0 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/4217965378 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc005500 con 0x55bca7deddc0 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/4217965378 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f1bc801a030 con 0x55bca7deddc0 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bd5d36740 10 client.27029 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bd5d36740 10 client.27029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bd5d36740 20 client.27029 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bca7e08230 0x55bca7e0a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bd5d36740 4 client.27029 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bd5d36740 1 -- 172.21.15.62:0/4217965378 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bca7d60850 con 0x55bca7e08230 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.997+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bca7e08230 0x55bca7e0a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.998+0000 7f1bb4ff9640 10 client.27029 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.998+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bca7e08230 0x55bca7e0a640 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d0d157f57dd799 server_cookie=4d043159e878a79d in_seq=0 out_seq=0 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.998+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/4217965378 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bca7d60850 con 0x55bca7e08230 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.998+0000 7f1bb4ff9640 10 client.27029 handle_command_reply: tid=0 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.998+0000 7f1bd5d36740 1 client.27029 shutdown 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 1 -- 172.21.15.62:0/4217965378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 msgr2=0x55bca7df5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df5360 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8013710 tx=0x7f1bc8013bd0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 1 -- 172.21.15.62:0/4217965378 shutdown_connections 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bca7e08230 0x55bca7e0a640 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7df8680 0x55bca7df5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df5360 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/4217965378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7ded4a0 0x55bca7df80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bb4ff9640 0 client.27029 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 1 -- 172.21.15.62:0/4217965378 >> 172.21.15.62:0/4217965378 conn(0x55bca7de9a30 msgr2=0x55bca7def760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 1 -- 172.21.15.62:0/4217965378 shutdown_connections 2024-09-12T00:04:09.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 1 -- 172.21.15.62:0/4217965378 wait complete. 2024-09-12T00:04:09.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:08.999+0000 7f1bd5d36740 20 client.27029 trim_cache size 0 max 16384 2024-09-12T00:04:09.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.032+0000 7f1bd5d36740 1 Processor -- start 2024-09-12T00:04:09.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.033+0000 7f1bd5d36740 1 -- start start 2024-09-12T00:04:09.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.033+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7ded4a0 0x55bca7df4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:09.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.033+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:09.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.033+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7df8680 0x55bca7e0ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:09.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.033+0000 7f1bd5d36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1b6780 con 0x55bca7ded4a0 2024-09-12T00:04:09.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.033+0000 7f1bd5d36740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1b3ff0 con 0x55bca7deddc0 2024-09-12T00:04:09.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.033+0000 7f1bd5d36740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1b1e60 con 0x55bca7df8680 2024-09-12T00:04:09.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.035+0000 7f1bd16f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:09.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.035+0000 7f1bd16f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:54266/0 (socket says 172.21.15.62:54266) 2024-09-12T00:04:09.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.035+0000 7f1bd16f9640 1 -- 172.21.15.62:0/3020097613 learned_addr learned my addr 172.21.15.62:0/3020097613 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:09.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.035+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/3020097613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7df8680 0x55bca7e0ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:09.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.035+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/3020097613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7ded4a0 0x55bca7df4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:09.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.035+0000 7f1bd16f9640 1 -- 172.21.15.62:0/3020097613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7df8680 msgr2=0x55bca7e0ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:09.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.035+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/3020097613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7df8680 0x55bca7e0ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.035+0000 7f1bd16f9640 1 -- 172.21.15.62:0/3020097613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7ded4a0 msgr2=0x55bca7df4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:09.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.035+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/3020097613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7ded4a0 0x55bca7df4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.035+0000 7f1bd16f9640 1 -- 172.21.15.62:0/3020097613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bca7deaae0 con 0x55bca7deddc0 2024-09-12T00:04:09.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.036+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/3020097613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df73e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80247a0 tx=0x7f1bc8024a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c51d1c72e15f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:09.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.036+0000 7f1bb6ffd640 1 -- 172.21.15.62:0/3020097613 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8005630 con 0x55bca7deddc0 2024-09-12T00:04:09.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.036+0000 7f1bb6ffd640 1 -- 172.21.15.62:0/3020097613 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc8024bd0 con 0x55bca7deddc0 2024-09-12T00:04:09.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.036+0000 7f1bb6ffd640 1 -- 172.21.15.62:0/3020097613 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800f980 con 0x55bca7deddc0 2024-09-12T00:04:09.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.037+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3020097613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 msgr2=0x55bca7df73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:09.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.037+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/3020097613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df73e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80247a0 tx=0x7f1bc8024a50 comp rx=0 tx=0).stop 2024-09-12T00:04:09.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.037+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3020097613 shutdown_connections 2024-09-12T00:04:09.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.037+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/3020097613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7df8680 0x55bca7e0ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.037+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/3020097613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7df73e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.037+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/3020097613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7ded4a0 0x55bca7df4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.037+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3020097613 >> 172.21.15.62:0/3020097613 conn(0x55bca7de9a30 msgr2=0x55bca7de9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:09.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.037+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3020097613 shutdown_connections 2024-09-12T00:04:09.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.037+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3020097613 wait complete. 2024-09-12T00:04:09.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.038+0000 7f1bd5d36740 1 -- start start 2024-09-12T00:04:09.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.038+0000 7f1bd5d36740 1 Processor -- start 2024-09-12T00:04:09.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.038+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7dfe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:09.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.038+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7dfec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:09.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.038+0000 7f1bd5d36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7df8680 0x55bca7e82f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:09.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd0ef8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7dfe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:09.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd0ef8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7dfe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:35080/0 (socket says 172.21.15.62:35080) 2024-09-12T00:04:09.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd0ef8640 1 -- 172.21.15.62:0/3576471213 learned_addr learned my addr 172.21.15.62:0/3576471213 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:09.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1b24b0 con 0x55bca7df8680 2024-09-12T00:04:09.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1b3920 con 0x55bca7deddc0 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1b2b80 con 0x55bca7ded4a0 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7dfec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd5d36740 10 client.0 _ll_register_callbacks cb 0x7f1bc80096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7df8680 0x55bca7e82f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd0ef8640 1 -- 172.21.15.62:0/3576471213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 msgr2=0x55bca7dfec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7dfec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd0ef8640 1 -- 172.21.15.62:0/3576471213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7df8680 msgr2=0x55bca7e82f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7df8680 0x55bca7e82f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd0ef8640 1 -- 172.21.15.62:0/3576471213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bca7dff570 con 0x55bca7ded4a0 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bc8024bd0 con 0x55bca7ded4a0 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bd0ef8640 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7dfe730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc004960 tx=0x7f1bbc021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74381912cd0db7f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.039+0000 7f1bb4ff9640 10 client.26938 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.040+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/3576471213 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc011260 con 0x55bca7ded4a0 2024-09-12T00:04:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.040+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/3576471213 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f1bbc02de80 con 0x55bca7ded4a0 2024-09-12T00:04:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.040+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/3576471213 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1bbc02d350 con 0x55bca7ded4a0 2024-09-12T00:04:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.040+0000 7f1bd5d36740 10 client.26938 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.041+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55bca7e84420 con 0x55bca7ded4a0 2024-09-12T00:04:09.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.041+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/3576471213 <== mon.1 v2:172.21.15.120:3300/0 4 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f1bbc036020 con 0x55bca7ded4a0 2024-09-12T00:04:09.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.041+0000 7f1bd5d36740 10 client.26938 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:09.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.041+0000 7f1bd5d36740 10 client.26938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.041+0000 7f1bd5d36740 20 client.26938 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.041+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bca7e84790 0x55bca7e86bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.041+0000 7f1bd5d36740 4 client.26938 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.041+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bca7d5ad60 con 0x55bca7e84790 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.042+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bca7e84790 0x55bca7e86bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.042+0000 7f1bb4ff9640 10 client.26938 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.042+0000 7f1bd16f9640 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bca7e84790 0x55bca7e86bb0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c817cbbc0440a4d server_cookie=1cad4fff7a093d6c in_seq=0 out_seq=0 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.042+0000 7f1bb4ff9640 1 -- 172.21.15.62:0/3576471213 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bca7d5ad60 con 0x55bca7e84790 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.042+0000 7f1bb4ff9640 10 client.26938 handle_command_reply: tid=0 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.042+0000 7f1bd5d36740 1 client.26938 shutdown 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 msgr2=0x55bca7dfe730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:09.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7dfe730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc004960 tx=0x7f1bbc021040 comp rx=0 tx=0).stop 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 shutdown_connections 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55bca7e84790 0x55bca7e86bb0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca7df8680 0x55bca7e82f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bb4ff9640 0 client.26938 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca7deddc0 0x55bca7dfec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 1 --2- 172.21.15.62:0/3576471213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bca7ded4a0 0x55bca7dfe730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 >> 172.21.15.62:0/3576471213 conn(0x55bca7de9a30 msgr2=0x55bca7df4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 shutdown_connections 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 1 -- 172.21.15.62:0/3576471213 wait complete. 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.043+0000 7f1bd5d36740 20 client.26938 trim_cache size 0 max 16384 2024-09-12T00:04:09.044 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:04:09.045 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:04:09.045 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:04:09.045 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:04:09.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 -- 172.21.15.62:0/3593152105 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1bb80a2cf0 msgr2=0x7f1bb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:09.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1bb80a2cf0 0x7f1bb80a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40bc8a0 tx=0x7f1bc4032f90 comp rx=0 tx=0).stop 2024-09-12T00:04:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 -- 172.21.15.62:0/3593152105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bcc105ec0 msgr2=0x7f1bcc19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bcc105ec0 0x7f1bcc19e860 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4005cc0 tx=0x7f1bc4009800 comp rx=0 tx=0).stop 2024-09-12T00:04:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 -- 172.21.15.62:0/3593152105 shutdown_connections 2024-09-12T00:04:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1bb80a2cf0 0x7f1bb80a51b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bcc105ec0 0x7f1bcc19e860 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc1011e0 0x7f1bcc1a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 --2- 172.21.15.62:0/3593152105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bcc1008c0 0x7f1bcc1a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 -- 172.21.15.62:0/3593152105 >> 172.21.15.62:0/3593152105 conn(0x7f1bcc0fc7d0 msgr2=0x7f1bcc103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 -- 172.21.15.62:0/3593152105 shutdown_connections 2024-09-12T00:04:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:09.044+0000 7f1bd395d640 1 -- 172.21.15.62:0/3593152105 wait complete. 2024-09-12T00:04:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:09 smithi062 ceph-mon[30958]: pgmap v375: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 26 MiB/s wr, 1.92k op/s 2024-09-12T00:04:09.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:09 smithi083 ceph-mon[34805]: pgmap v375: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 26 MiB/s wr, 1.92k op/s 2024-09-12T00:04:09.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:09 smithi120 ceph-mon[35182]: pgmap v375: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 26 MiB/s wr, 1.92k op/s 2024-09-12T00:04:09.899 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:04:09.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:04:10.250 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:10.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.832+0000 7f9dee49f640 1 -- 172.21.15.62:0/437337716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de8109d70 msgr2=0x7f9de810e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:10.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.832+0000 7f9dee49f640 1 --2- 172.21.15.62:0/437337716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de8109d70 0x7f9de810e350 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009900 tx=0x7f9ddc02a400 comp rx=0 tx=0).stop 2024-09-12T00:04:10.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.833+0000 7f9dee49f640 1 -- 172.21.15.62:0/437337716 shutdown_connections 2024-09-12T00:04:10.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.833+0000 7f9dee49f640 1 --2- 172.21.15.62:0/437337716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de810e890 0x7f9de810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:10.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.833+0000 7f9dee49f640 1 --2- 172.21.15.62:0/437337716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de8109d70 0x7f9de810e350 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:10.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.833+0000 7f9dee49f640 1 --2- 172.21.15.62:0/437337716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9de8109450 0x7f9de8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:10.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.833+0000 7f9dee49f640 1 -- 172.21.15.62:0/437337716 >> 172.21.15.62:0/437337716 conn(0x7f9de806e2d0 msgr2=0x7f9de806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:10.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.835+0000 7f9dee49f640 1 -- 172.21.15.62:0/437337716 shutdown_connections 2024-09-12T00:04:10.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.835+0000 7f9dee49f640 1 -- 172.21.15.62:0/437337716 wait complete. 2024-09-12T00:04:10.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.835+0000 7f9dee49f640 1 Processor -- start 2024-09-12T00:04:10.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9dee49f640 1 -- start start 2024-09-12T00:04:10.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9dee49f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9de8109450 0x7f9de819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:10.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9dee49f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de8109d70 0x7f9de819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:10.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9dee49f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de810e890 0x7f9de81a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:10.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9dee49f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8119690 con 0x7f9de810e890 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9dee49f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8119510 con 0x7f9de8109d70 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9dee49f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8119810 con 0x7f9de8109450 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9deca3c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de810e890 0x7f9de81a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9deca3c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de810e890 0x7f9de81a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38550/0 (socket says 172.21.15.62:38550) 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9deca3c640 1 -- 172.21.15.62:0/585378297 learned_addr learned my addr 172.21.15.62:0/585378297 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9de77fe640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de8109d70 0x7f9de819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.836+0000 7f9de7fff640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9de8109450 0x7f9de819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9deca3c640 1 -- 172.21.15.62:0/585378297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9de8109450 msgr2=0x7f9de819f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9deca3c640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9de8109450 0x7f9de819f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9deca3c640 1 -- 172.21.15.62:0/585378297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de8109d70 msgr2=0x7f9de819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9deca3c640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de8109d70 0x7f9de819fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9deca3c640 1 -- 172.21.15.62:0/585378297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de81af1b0 con 0x7f9de810e890 2024-09-12T00:04:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9deca3c640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de810e890 0x7f9de81a03c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9de000eaa0 tx=0x7f9de000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9de57fa640 1 -- 172.21.15.62:0/585378297 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de000cdb0 con 0x7f9de810e890 2024-09-12T00:04:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9dee49f640 1 -- 172.21.15.62:0/585378297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de81af4a0 con 0x7f9de810e890 2024-09-12T00:04:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9de57fa640 1 -- 172.21.15.62:0/585378297 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de0005020 con 0x7f9de810e890 2024-09-12T00:04:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9de57fa640 1 -- 172.21.15.62:0/585378297 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de00116d0 con 0x7f9de810e890 2024-09-12T00:04:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.837+0000 7f9dee49f640 1 -- 172.21.15.62:0/585378297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de81af960 con 0x7f9de810e890 2024-09-12T00:04:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.839+0000 7f9dee49f640 1 -- 172.21.15.62:0/585378297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de810e350 con 0x7f9de810e890 2024-09-12T00:04:10.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.841+0000 7f9de57fa640 1 -- 172.21.15.62:0/585378297 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9de0005540 con 0x7f9de810e890 2024-09-12T00:04:10.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.842+0000 7f9de57fa640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9dd00a2cf0 0x7f9dd00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:10.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.842+0000 7f9de57fa640 1 -- 172.21.15.62:0/585378297 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f9de00cf620 con 0x7f9de810e890 2024-09-12T00:04:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.843+0000 7f9de7fff640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9dd00a2cf0 0x7f9dd00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.843+0000 7f9de7fff640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9dd00a2cf0 0x7f9dd00a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005d00 tx=0x7f9dd800b410 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:10.849+0000 7f9de57fa640 1 -- 172.21.15.62:0/585378297 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9de0016030 con 0x7f9de810e890 2024-09-12T00:04:11.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.174+0000 7f9dee49f640 1 -- 172.21.15.62:0/585378297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9de81afdc0 con 0x7f9de810e890 2024-09-12T00:04:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.175+0000 7f9de57fa640 1 -- 172.21.15.62:0/585378297 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11913 (secure 0 0 0) 0x7f9de009bb70 con 0x7f9de810e890 2024-09-12T00:04:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:04:11.175 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":36,"btime":"2024-09-12T00:03:36:303242+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:03:35.719292+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:04:11.175 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 36 2024-09-12T00:04:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 -- 172.21.15.62:0/585378297 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9dd00a2cf0 msgr2=0x7f9dd00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9dd00a2cf0 0x7f9dd00a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005d00 tx=0x7f9dd800b410 comp rx=0 tx=0).stop 2024-09-12T00:04:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 -- 172.21.15.62:0/585378297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de810e890 msgr2=0x7f9de81a03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:11.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de810e890 0x7f9de81a03c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9de000eaa0 tx=0x7f9de000ef60 comp rx=0 tx=0).stop 2024-09-12T00:04:11.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 -- 172.21.15.62:0/585378297 shutdown_connections 2024-09-12T00:04:11.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f9dd00a2cf0 0x7f9dd00a51b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:11.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de810e890 0x7f9de81a03c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:11.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de8109d70 0x7f9de819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:11.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 --2- 172.21.15.62:0/585378297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9de8109450 0x7f9de819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:11.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 -- 172.21.15.62:0/585378297 >> 172.21.15.62:0/585378297 conn(0x7f9de806e2d0 msgr2=0x7f9de810c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:11.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 -- 172.21.15.62:0/585378297 shutdown_connections 2024-09-12T00:04:11.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:11.179+0000 7f9dee49f640 1 -- 172.21.15.62:0/585378297 wait complete. 2024-09-12T00:04:11.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:11 smithi062 ceph-mon[30958]: pgmap v376: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 25 MiB/s wr, 1.92k op/s 2024-09-12T00:04:11.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:11 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/585378297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:11.613 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-12T00:04:11.614 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:04:11.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:11 smithi083 ceph-mon[34805]: pgmap v376: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 25 MiB/s wr, 1.92k op/s 2024-09-12T00:04:11.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:11 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/585378297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:11.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:11 smithi120 ceph-mon[35182]: pgmap v376: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 25 MiB/s wr, 1.92k op/s 2024-09-12T00:04:11.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:11 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/585378297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:12.120 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:04:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:12.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 690.0 s, 2293.0 tps, lat 43.536 ms stddev 48.168 2024-09-12T00:04:12.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.865+0000 7f6dc31eb640 1 -- 172.21.15.62:0/3485729106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6dbc11b140 msgr2=0x7f6dbc11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.865+0000 7f6dc31eb640 1 --2- 172.21.15.62:0/3485729106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6dbc11b140 0x7f6dbc11d530 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6db401acb0 tx=0x7f6db403b920 comp rx=0 tx=0).stop 2024-09-12T00:04:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.868+0000 7f6dc31eb640 1 -- 172.21.15.62:0/3485729106 shutdown_connections 2024-09-12T00:04:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.868+0000 7f6dc31eb640 1 --2- 172.21.15.62:0/3485729106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6dbc11b140 0x7f6dbc11d530 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.868+0000 7f6dc31eb640 1 --2- 172.21.15.62:0/3485729106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dbc109830 0x7f6dbc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.868+0000 7f6dc31eb640 1 --2- 172.21.15.62:0/3485729106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dbc109450 0x7f6dbc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.868+0000 7f6dc31eb640 1 -- 172.21.15.62:0/3485729106 >> 172.21.15.62:0/3485729106 conn(0x7f6dbc06e140 msgr2=0x7f6dbc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:12.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.876+0000 7f6dc31eb640 1 -- 172.21.15.62:0/3485729106 shutdown_connections 2024-09-12T00:04:12.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.877+0000 7f6dc31eb640 1 -- 172.21.15.62:0/3485729106 wait complete. 2024-09-12T00:04:12.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.877+0000 7f6dc31eb640 1 Processor -- start 2024-09-12T00:04:12.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.877+0000 7f6dc31eb640 1 -- start start 2024-09-12T00:04:12.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.878+0000 7f6dc31eb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dbc109450 0x7f6dbc11a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:12.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.878+0000 7f6dc31eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6dbc109830 0x7f6dbc1167a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:12.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.878+0000 7f6dc31eb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dbc11b140 0x7f6dbc116ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:12.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.878+0000 7f6dc31eb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc113300 con 0x7f6dbc109450 2024-09-12T00:04:12.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.878+0000 7f6dc31eb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc113180 con 0x7f6dbc109830 2024-09-12T00:04:12.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.878+0000 7f6dc31eb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc113480 con 0x7f6dbc11b140 2024-09-12T00:04:12.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.878+0000 7f6dc1788640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dbc11b140 0x7f6dbc116ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:12.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.878+0000 7f6dc1788640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dbc11b140 0x7f6dbc116ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:47150/0 (socket says 172.21.15.62:47150) 2024-09-12T00:04:12.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.878+0000 7f6dc1788640 1 -- 172.21.15.62:0/4058003858 learned_addr learned my addr 172.21.15.62:0/4058003858 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:12.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.878+0000 7f6dbbfff640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6dbc109830 0x7f6dbc1167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:12.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.879+0000 7f6dc1788640 1 -- 172.21.15.62:0/4058003858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6dbc109830 msgr2=0x7f6dbc1167a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:12.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.879+0000 7f6dc1788640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6dbc109830 0x7f6dbc1167a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:12.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.879+0000 7f6dc1788640 1 -- 172.21.15.62:0/4058003858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dbc109450 msgr2=0x7f6dbc11a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:12.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.879+0000 7f6dc1788640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dbc109450 0x7f6dbc11a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:12.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.879+0000 7f6dc1788640 1 -- 172.21.15.62:0/4058003858 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc117220 con 0x7f6dbc11b140 2024-09-12T00:04:12.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.880+0000 7f6dc1788640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dbc11b140 0x7f6dbc116ce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6db403be90 tx=0x7f6db40474f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:12.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.880+0000 7f6db9ffb640 1 -- 172.21.15.62:0/4058003858 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4011070 con 0x7f6dbc11b140 2024-09-12T00:04:12.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.880+0000 7f6dc31eb640 1 -- 172.21.15.62:0/4058003858 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc1174b0 con 0x7f6dbc11b140 2024-09-12T00:04:12.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.880+0000 7f6dc31eb640 1 -- 172.21.15.62:0/4058003858 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dbc1cd3b0 con 0x7f6dbc11b140 2024-09-12T00:04:12.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.880+0000 7f6db9ffb640 1 -- 172.21.15.62:0/4058003858 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db400d420 con 0x7f6dbc11b140 2024-09-12T00:04:12.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.880+0000 7f6db9ffb640 1 -- 172.21.15.62:0/4058003858 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4045070 con 0x7f6dbc11b140 2024-09-12T00:04:12.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.888+0000 7f6db9ffb640 1 -- 172.21.15.62:0/4058003858 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6db4014070 con 0x7f6dbc11b140 2024-09-12T00:04:12.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.888+0000 7f6db37fe640 1 -- 172.21.15.62:0/4058003858 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da0005290 con 0x7f6dbc11b140 2024-09-12T00:04:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.891+0000 7f6db9ffb640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6da80a2e30 0x7f6da80a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.891+0000 7f6dc0f87640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6da80a2e30 0x7f6da80a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.892+0000 7f6db9ffb640 1 -- 172.21.15.62:0/4058003858 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f6db40f02d0 con 0x7f6dbc11b140 2024-09-12T00:04:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.892+0000 7f6dc0f87640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6da80a2e30 0x7f6da80a52f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6dac005fd0 tx=0x7f6dac005850 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:12.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:12.897+0000 7f6db9ffb640 1 -- 172.21.15.62:0/4058003858 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6db403e070 con 0x7f6dbc11b140 2024-09-12T00:04:12.912 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:13.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.220+0000 7f6db37fe640 1 -- 172.21.15.62:0/4058003858 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6da0005850 con 0x7f6dbc11b140 2024-09-12T00:04:13.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.220+0000 7f6db9ffb640 1 -- 172.21.15.62:0/4058003858 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11913 (secure 0 0 0) 0x7f6db40c53e0 con 0x7f6dbc11b140 2024-09-12T00:04:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:04:13.219 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":36,"btime":"2024-09-12T00:03:36:303242+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:03:35.719292+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:04:13.220 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 36 2024-09-12T00:04:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 -- 172.21.15.62:0/4058003858 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6da80a2e30 msgr2=0x7f6da80a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6da80a2e30 0x7f6da80a52f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6dac005fd0 tx=0x7f6dac005850 comp rx=0 tx=0).stop 2024-09-12T00:04:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 -- 172.21.15.62:0/4058003858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dbc11b140 msgr2=0x7f6dbc116ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dbc11b140 0x7f6dbc116ce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6db403be90 tx=0x7f6db40474f0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 -- 172.21.15.62:0/4058003858 shutdown_connections 2024-09-12T00:04:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6da80a2e30 0x7f6da80a52f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dbc11b140 0x7f6dbc116ce0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6dbc109830 0x7f6dbc1167a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 --2- 172.21.15.62:0/4058003858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dbc109450 0x7f6dbc11a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 -- 172.21.15.62:0/4058003858 >> 172.21.15.62:0/4058003858 conn(0x7f6dbc06e140 msgr2=0x7f6dbc070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 -- 172.21.15.62:0/4058003858 shutdown_connections 2024-09-12T00:04:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.223+0000 7f6db37fe640 1 -- 172.21.15.62:0/4058003858 wait complete. 2024-09-12T00:04:13.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:13 smithi083 ceph-mon[34805]: pgmap v377: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 25 MiB/s wr, 1.92k op/s 2024-09-12T00:04:13.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:13 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/4058003858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:13.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.461+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/76580979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f38101440 msgr2=0x7f6f3810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.461+0000 7f6f3dfa9640 1 --2- 172.21.15.62:0/76580979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f38101440 0x7f6f3810df30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009be0 tx=0x7f6f3002a6c0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.461+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/76580979 shutdown_connections 2024-09-12T00:04:13.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.461+0000 7f6f3dfa9640 1 --2- 172.21.15.62:0/76580979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f3810e500 0x7f6f38110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.461+0000 7f6f3dfa9640 1 --2- 172.21.15.62:0/76580979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f38101440 0x7f6f3810df30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.461+0000 7f6f3dfa9640 1 --2- 172.21.15.62:0/76580979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f38100b20 0x7f6f38100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.461+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/76580979 >> 172.21.15.62:0/76580979 conn(0x7f6f380fc7d0 msgr2=0x7f6f380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:13.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.461+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/76580979 shutdown_connections 2024-09-12T00:04:13.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.461+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/76580979 wait complete. 2024-09-12T00:04:13.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.462+0000 7f6f3dfa9640 1 Processor -- start 2024-09-12T00:04:13.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.462+0000 7f6f3dfa9640 1 -- start start 2024-09-12T00:04:13.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.462+0000 7f6f3dfa9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f38100b20 0x7f6f381a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.462+0000 7f6f3dfa9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f38101440 0x7f6f381a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f3dfa9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3810e500 0x7f6f3819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f3dfa9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6f38114220 con 0x7f6f38100b20 2024-09-12T00:04:13.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f36ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f38101440 0x7f6f381a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f3dfa9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381140a0 con 0x7f6f3810e500 2024-09-12T00:04:13.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f3dfa9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381143a0 con 0x7f6f38101440 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f36ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f38101440 0x7f6f381a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:47174/0 (socket says 172.21.15.62:47174) 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3292692574 learned_addr learned my addr 172.21.15.62:0/3292692574 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f37fff640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3810e500 0x7f6f3819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3292692574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3810e500 msgr2=0x7f6f3819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f36ffd640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3810e500 0x7f6f3819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3292692574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f38100b20 msgr2=0x7f6f381a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f36ffd640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f38100b20 0x7f6f381a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.463+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3292692574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3819ad00 con 0x7f6f38101440 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.464+0000 7f6f36ffd640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f38101440 0x7f6f381a0950 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006010 tx=0x7f6f300305b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.464+0000 7f6f34ff9640 1 -- 172.21.15.62:0/3292692574 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30041020 con 0x7f6f38101440 2024-09-12T00:04:13.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.464+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/3292692574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3819af90 con 0x7f6f38101440 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.464+0000 7f6f34ff9640 1 -- 172.21.15.62:0/3292692574 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f30037040 con 0x7f6f38101440 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.464+0000 7f6f34ff9640 1 -- 172.21.15.62:0/3292692574 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30036070 con 0x7f6f38101440 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.464+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/3292692574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f381a72e0 con 0x7f6f38101440 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.466+0000 7f6f34ff9640 1 -- 172.21.15.62:0/3292692574 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6f30044020 con 0x7f6f38101440 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f40382740 1 Processor -- start 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f40382740 1 -- start start 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f40382740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec73b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f40382740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73b5d0 0x55ecec73ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f40382740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 0x55ecec748760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f34ff9640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f200a2cf0 0x7f6f200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f40382740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ecec6a12f0 con 0x55ecec73b5d0 2024-09-12T00:04:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f40382740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ecec648cf0 con 0x55ecec746370 2024-09-12T00:04:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f40382740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ecec69d2b0 con 0x55ecec73cb40 2024-09-12T00:04:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f377fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73b5d0 0x55ecec73ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f36ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 0x55ecec748760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f377fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73b5d0 0x55ecec73ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47144/0 (socket says 172.21.15.62:47144) 2024-09-12T00:04:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f37fff640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f200a2cf0 0x7f6f200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f377fe640 1 -- 172.21.15.62:0/628691069 learned_addr learned my addr 172.21.15.62:0/628691069 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f34ff9640 1 -- 172.21.15.62:0/3292692574 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f6f300f0060 con 0x7f6f38101440 2024-09-12T00:04:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f37fff640 1 --2- 172.21.15.62:0/628691069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec73b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f36ffd640 1 -- 172.21.15.62:0/628691069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 msgr2=0x55ecec73b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f36ffd640 1 --2- 172.21.15.62:0/628691069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec73b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f36ffd640 1 -- 172.21.15.62:0/628691069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73b5d0 msgr2=0x55ecec73ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f36ffd640 1 --2- 172.21.15.62:0/628691069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73b5d0 0x55ecec73ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f36ffd640 1 -- 172.21.15.62:0/628691069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecec62d3a0 con 0x55ecec746370 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.467+0000 7f6f377fe640 1 --2- 172.21.15.62:0/628691069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73b5d0 0x55ecec73ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.468+0000 7f6f37fff640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f200a2cf0 0x7f6f200a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f38069760 tx=0x7f6f28012820 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.468+0000 7f6f36ffd640 1 --2- 172.21.15.62:0/628691069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 0x55ecec748760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f300ed790 tx=0x7f6f30004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d05f57870712c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.468+0000 7f6f1dffb640 1 -- 172.21.15.62:0/628691069 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f300bab20 con 0x55ecec746370 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.468+0000 7f6f1dffb640 1 -- 172.21.15.62:0/628691069 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f300bcc60 con 0x55ecec746370 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.468+0000 7f6f1dffb640 1 -- 172.21.15.62:0/628691069 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30049120 con 0x55ecec746370 2024-09-12T00:04:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.469+0000 7f6f40382740 1 -- 172.21.15.62:0/628691069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 msgr2=0x55ecec748760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.469+0000 7f6f40382740 1 --2- 172.21.15.62:0/628691069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 0x55ecec748760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f300ed790 tx=0x7f6f30004040 comp rx=0 tx=0).stop 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.469+0000 7f6f40382740 1 -- 172.21.15.62:0/628691069 shutdown_connections 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.469+0000 7f6f40382740 1 --2- 172.21.15.62:0/628691069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 0x55ecec748760 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.469+0000 7f6f40382740 1 --2- 172.21.15.62:0/628691069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73b5d0 0x55ecec73ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.469+0000 7f6f40382740 1 --2- 172.21.15.62:0/628691069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec73b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.469+0000 7f6f40382740 1 -- 172.21.15.62:0/628691069 >> 172.21.15.62:0/628691069 conn(0x55ecec704310 msgr2=0x55ecec737c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.471+0000 7f6f40382740 1 -- 172.21.15.62:0/628691069 shutdown_connections 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.471+0000 7f6f40382740 1 -- 172.21.15.62:0/628691069 wait complete. 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.471+0000 7f6f40382740 1 -- start start 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.472+0000 7f6f40382740 1 Processor -- start 2024-09-12T00:04:13.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.472+0000 7f6f40382740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec73b5d0 0x55ecec745dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.472+0000 7f6f40382740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec743080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.472+0000 7f6f40382740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec743810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.472+0000 7f6f40382740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ecec748e30 con 0x55ecec746370 2024-09-12T00:04:13.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.472+0000 7f6f40382740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ecec65c060 con 0x55ecec73b5d0 2024-09-12T00:04:13.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.472+0000 7f6f40382740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ecec748fb0 con 0x55ecec73cb40 2024-09-12T00:04:13.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.473+0000 7f6f40382740 10 client.0 _ll_register_callbacks cb 0x55ecec3af8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:13.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.473+0000 7f6f37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec743810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.473+0000 7f6f37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec743810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47152/0 (socket says 172.21.15.62:47152) 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.473+0000 7f6f37fff640 1 -- 172.21.15.62:0/3627445818 learned_addr learned my addr 172.21.15.62:0/3627445818 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.476+0000 7f6f37fff640 1 -- 172.21.15.62:0/3627445818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 msgr2=0x55ecec743080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.476+0000 7f6f37fff640 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec743080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.476+0000 7f6f37fff640 1 -- 172.21.15.62:0/3627445818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec73b5d0 msgr2=0x55ecec745dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.477+0000 7f6f37fff640 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec73b5d0 0x55ecec745dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.477+0000 7f6f37fff640 1 -- 172.21.15.62:0/3627445818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecec4123c0 con 0x55ecec746370 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.491+0000 7f6f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.491+0000 7f6f37fff640 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec743810 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009fd0 tx=0x7f6f28005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b46ad582d2cd37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.492+0000 7f6f16ffd640 1 -- 172.21.15.62:0/3627445818 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28004020 con 0x55ecec746370 2024-09-12T00:04:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.493+0000 7f6f40382740 1 -- 172.21.15.62:0/3627445818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f300bcc60 con 0x55ecec746370 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.493+0000 7f6f16ffd640 1 -- 172.21.15.62:0/3627445818 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f6f2800fd90 con 0x55ecec746370 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.497+0000 7f6f16ffd640 1 -- 172.21.15.62:0/3627445818 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6f2802c050 con 0x55ecec746370 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.497+0000 7f6f40382740 10 client.17517 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.497+0000 7f6f40382740 1 -- 172.21.15.62:0/3627445818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ecec75d0a0 con 0x55ecec746370 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.499+0000 7f6f16ffd640 1 -- 172.21.15.62:0/3627445818 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f300bcc60 con 0x55ecec746370 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.500+0000 7f6f16ffd640 1 -- 172.21.15.62:0/3627445818 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f6f280035f0 con 0x55ecec746370 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.500+0000 7f6f40382740 10 client.17517 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.500+0000 7f6f40382740 10 client.17517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.500+0000 7f6f40382740 20 client.17517 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.501+0000 7f6f40382740 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ecec75e3f0 0x55ecec760820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.501+0000 7f6f40382740 4 client.17517 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:04:13.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.501+0000 7f6f40382740 1 -- 172.21.15.62:0/3627445818 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecec6ae860 con 0x55ecec75e3f0 2024-09-12T00:04:13.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.501+0000 7f6f377fe640 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ecec75e3f0 0x55ecec760820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.502+0000 7f6f377fe640 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ecec75e3f0 0x55ecec760820 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcb7fb5e72d377b5 server_cookie=d09769f146fe9ac3 in_seq=0 out_seq=0 2024-09-12T00:04:13.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.502+0000 7f6f16ffd640 10 client.17517 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:04:13.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.503+0000 7f6f16ffd640 1 -- 172.21.15.62:0/3627445818 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ecec6ae860 con 0x55ecec75e3f0 2024-09-12T00:04:13.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.504+0000 7f6f16ffd640 10 client.17517 handle_command_reply: tid=0 2024-09-12T00:04:13.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.504+0000 7f6f40382740 1 client.17517 shutdown 2024-09-12T00:04:13.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.506+0000 7f6f40382740 1 -- 172.21.15.62:0/3627445818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 msgr2=0x55ecec743810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.506+0000 7f6f40382740 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec743810 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009fd0 tx=0x7f6f28005f90 comp rx=0 tx=0).stop 2024-09-12T00:04:13.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.506+0000 7f6f40382740 1 -- 172.21.15.62:0/3627445818 shutdown_connections 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.506+0000 7f6f40382740 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ecec75e3f0 0x55ecec760820 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.506+0000 7f6f40382740 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec743810 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.506+0000 7f6f40382740 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec743080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.506+0000 7f6f40382740 1 --2- 172.21.15.62:0/3627445818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec73b5d0 0x55ecec745dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.506+0000 7f6f40382740 1 -- 172.21.15.62:0/3627445818 >> 172.21.15.62:0/3627445818 conn(0x55ecec704310 msgr2=0x55ecec73d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.506+0000 7f6f16ffd640 0 client.17517 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.507+0000 7f6f40382740 1 -- 172.21.15.62:0/3627445818 shutdown_connections 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.507+0000 7f6f40382740 1 -- 172.21.15.62:0/3627445818 wait complete. 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.507+0000 7f6f40382740 20 client.17517 trim_cache size 0 max 16384 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f40382740 1 Processor -- start 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f40382740 1 -- start start 2024-09-12T00:04:13.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f40382740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73b5d0 0x55ecec737e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f40382740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73cb40 0x55ecec73e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f40382740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 0x55ecec73e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f40382740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381ae380 con 0x55ecec73cb40 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f40382740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381abbf0 con 0x55ecec746370 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f40382740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381a99e0 con 0x55ecec73b5d0 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f36ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73cb40 0x55ecec73e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f36ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73cb40 0x55ecec73e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47156/0 (socket says 172.21.15.62:47156) 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3791765942 learned_addr learned my addr 172.21.15.62:0/3791765942 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f37fff640 1 --2- 172.21.15.62:0/3791765942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 0x55ecec73e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.521+0000 7f6f377fe640 1 --2- 172.21.15.62:0/3791765942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73b5d0 0x55ecec737e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.522+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3791765942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73b5d0 msgr2=0x55ecec737e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.522+0000 7f6f36ffd640 1 --2- 172.21.15.62:0/3791765942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73b5d0 0x55ecec737e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.522+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3791765942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 msgr2=0x55ecec73e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.522+0000 7f6f36ffd640 1 --2- 172.21.15.62:0/3791765942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 0x55ecec73e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.522+0000 7f6f36ffd640 1 -- 172.21.15.62:0/3791765942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecec73eba0 con 0x55ecec73cb40 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.522+0000 7f6f36ffd640 1 --2- 172.21.15.62:0/3791765942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73cb40 0x55ecec73e120 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6f300bce10 tx=0x7f6f3002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5333a582faa13ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.522+0000 7f6f1d7fa640 1 -- 172.21.15.62:0/3791765942 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f300ba900 con 0x55ecec73cb40 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.522+0000 7f6f1d7fa640 1 -- 172.21.15.62:0/3791765942 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f300bab20 con 0x55ecec73cb40 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.522+0000 7f6f1d7fa640 1 -- 172.21.15.62:0/3791765942 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f300eda10 con 0x55ecec73cb40 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 -- 172.21.15.62:0/3791765942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73cb40 msgr2=0x55ecec73e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 --2- 172.21.15.62:0/3791765942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73cb40 0x55ecec73e120 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6f300bce10 tx=0x7f6f3002a6c0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 -- 172.21.15.62:0/3791765942 shutdown_connections 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 --2- 172.21.15.62:0/3791765942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec746370 0x55ecec73e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 --2- 172.21.15.62:0/3791765942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec73cb40 0x55ecec73e120 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 --2- 172.21.15.62:0/3791765942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73b5d0 0x55ecec737e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 -- 172.21.15.62:0/3791765942 >> 172.21.15.62:0/3791765942 conn(0x55ecec704310 msgr2=0x55ecec737700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 -- 172.21.15.62:0/3791765942 shutdown_connections 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 -- 172.21.15.62:0/3791765942 wait complete. 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 -- start start 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.523+0000 7f6f40382740 1 Processor -- start 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f40382740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec73b5d0 0x55ecec752810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f40382740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec752d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f40382740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec753290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f40382740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381aa0b0 con 0x55ecec746370 2024-09-12T00:04:13.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f40382740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381ab520 con 0x55ecec73b5d0 2024-09-12T00:04:13.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f40382740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381aa780 con 0x55ecec73cb40 2024-09-12T00:04:13.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f40382740 10 client.0 _ll_register_callbacks cb 0x7f6f280095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:13.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec753290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec753290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47158/0 (socket says 172.21.15.62:47158) 2024-09-12T00:04:13.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f37fff640 1 -- 172.21.15.62:0/48789099 learned_addr learned my addr 172.21.15.62:0/48789099 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:13.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f377fe640 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec73b5d0 0x55ecec752810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f36ffd640 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec752d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f37fff640 1 -- 172.21.15.62:0/48789099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 msgr2=0x55ecec752d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:13.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f37fff640 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec752d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f37fff640 1 -- 172.21.15.62:0/48789099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec73b5d0 msgr2=0x55ecec752810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f37fff640 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec73b5d0 0x55ecec752810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f37fff640 1 -- 172.21.15.62:0/48789099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecec7c81c0 con 0x55ecec746370 2024-09-12T00:04:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f37fff640 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec753290 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6f28003b40 tx=0x7f6f28003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2655b1676e8de90a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f16ffd640 1 -- 172.21.15.62:0/48789099 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2802c050 con 0x55ecec746370 2024-09-12T00:04:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f40382740 1 -- 172.21.15.62:0/48789099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f300bab20 con 0x55ecec746370 2024-09-12T00:04:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.524+0000 7f6f16ffd640 1 -- 172.21.15.62:0/48789099 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f6f28025d30 con 0x55ecec746370 2024-09-12T00:04:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f16ffd640 1 -- 172.21.15.62:0/48789099 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6f2803e050 con 0x55ecec746370 2024-09-12T00:04:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f40382740 10 client.17523 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f40382740 1 -- 172.21.15.62:0/48789099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ecec7c94a0 con 0x55ecec746370 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f16ffd640 1 -- 172.21.15.62:0/48789099 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f300bab20 con 0x55ecec746370 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f16ffd640 1 -- 172.21.15.62:0/48789099 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f6f280035f0 con 0x55ecec746370 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f40382740 10 client.17523 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f40382740 10 client.17523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f40382740 20 client.17523 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f40382740 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ecec7c9b60 0x55ecec7cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f40382740 4 client.17523 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.525+0000 7f6f40382740 1 -- 172.21.15.62:0/48789099 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ecec6a8d70 con 0x55ecec7c9b60 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f377fe640 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ecec7c9b60 0x55ecec7cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f377fe640 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ecec7c9b60 0x55ecec7cbf50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9278319758213e14 server_cookie=185cd1928a12e67d in_seq=0 out_seq=0 2024-09-12T00:04:13.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f16ffd640 10 client.17523 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f16ffd640 1 -- 172.21.15.62:0/48789099 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ecec6a8d70 con 0x55ecec7c9b60 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f16ffd640 10 client.17523 handle_command_reply: tid=0 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f40382740 1 client.17523 shutdown 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f40382740 1 -- 172.21.15.62:0/48789099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 msgr2=0x55ecec753290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f40382740 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec753290 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6f28003b40 tx=0x7f6f28003c10 comp rx=0 tx=0).stop 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f40382740 1 -- 172.21.15.62:0/48789099 shutdown_connections 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f40382740 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55ecec7c9b60 0x55ecec7cbf50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f16ffd640 0 client.17523 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f40382740 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecec746370 0x55ecec753290 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f40382740 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ecec73cb40 0x55ecec752d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f40382740 1 --2- 172.21.15.62:0/48789099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecec73b5d0 0x55ecec752810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f40382740 1 -- 172.21.15.62:0/48789099 >> 172.21.15.62:0/48789099 conn(0x55ecec704310 msgr2=0x55ecec7448e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.526+0000 7f6f40382740 1 -- 172.21.15.62:0/48789099 shutdown_connections 2024-09-12T00:04:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f40382740 1 -- 172.21.15.62:0/48789099 wait complete. 2024-09-12T00:04:13.697 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:04:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f40382740 20 client.17523 trim_cache size 0 max 16384 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/3292692574 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f200a2cf0 msgr2=0x7f6f200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f200a2cf0 0x7f6f200a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f38069760 tx=0x7f6f28012820 comp rx=0 tx=0).stop 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/3292692574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f38101440 msgr2=0x7f6f381a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f38101440 0x7f6f381a0950 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006010 tx=0x7f6f300305b0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/3292692574 shutdown_connections 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6f200a2cf0 0x7f6f200a51b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3810e500 0x7f6f3819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f38101440 0x7f6f381a0950 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 --2- 172.21.15.62:0/3292692574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f38100b20 0x7f6f381a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/3292692574 >> 172.21.15.62:0/3292692574 conn(0x7f6f380fc7d0 msgr2=0x7f6f380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/3292692574 shutdown_connections 2024-09-12T00:04:13.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:13.527+0000 7f6f3dfa9640 1 -- 172.21.15.62:0/3292692574 wait complete. 2024-09-12T00:04:13.699 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:13 smithi062 ceph-mon[30958]: pgmap v377: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 25 MiB/s wr, 1.92k op/s 2024-09-12T00:04:13.699 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:13 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/4058003858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:13.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:13 smithi120 ceph-mon[35182]: pgmap v377: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 25 MiB/s wr, 1.92k op/s 2024-09-12T00:04:13.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:13 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/4058003858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:14.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:04:15.045 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-12T00:04:15.045 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-12T00:04:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:15.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:15 smithi062 ceph-mon[30958]: pgmap v378: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 831 op/s 2024-09-12T00:04:15.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:15 smithi083 ceph-mon[34805]: pgmap v378: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 831 op/s 2024-09-12T00:04:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.738+0000 7f5e619a4640 1 -- 172.21.15.62:0/2638674975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e5c102ef0 msgr2=0x7f5e5c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.738+0000 7f5e619a4640 1 --2- 172.21.15.62:0/2638674975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e5c102ef0 0x7f5e5c10da40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009c10 tx=0x7f5e5402a770 comp rx=0 tx=0).stop 2024-09-12T00:04:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.738+0000 7f5e619a4640 1 -- 172.21.15.62:0/2638674975 shutdown_connections 2024-09-12T00:04:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.738+0000 7f5e619a4640 1 --2- 172.21.15.62:0/2638674975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e5c102ef0 0x7f5e5c10da40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.738+0000 7f5e619a4640 1 --2- 172.21.15.62:0/2638674975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e5c102570 0x7f5e5c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.738+0000 7f5e619a4640 1 --2- 172.21.15.62:0/2638674975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c100e20 0x7f5e5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.738+0000 7f5e619a4640 1 -- 172.21.15.62:0/2638674975 >> 172.21.15.62:0/2638674975 conn(0x7f5e5c0fc7d0 msgr2=0x7f5e5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.738+0000 7f5e619a4640 1 -- 172.21.15.62:0/2638674975 shutdown_connections 2024-09-12T00:04:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.738+0000 7f5e619a4640 1 -- 172.21.15.62:0/2638674975 wait complete. 2024-09-12T00:04:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.739+0000 7f5e619a4640 1 Processor -- start 2024-09-12T00:04:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.739+0000 7f5e619a4640 1 -- start start 2024-09-12T00:04:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.739+0000 7f5e619a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e5c100e20 0x7f5e5c073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.739+0000 7f5e619a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e5c102570 0x7f5e5c06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e619a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c102ef0 0x7f5e5c06d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e619a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c110f30 con 0x7f5e5c102ef0 2024-09-12T00:04:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e619a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c110db0 con 0x7f5e5c102570 2024-09-12T00:04:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e619a4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1110b0 con 0x7f5e5c100e20 2024-09-12T00:04:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e5c102570 0x7f5e5c06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e5c102570 0x7f5e5c06cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35380/0 (socket says 172.21.15.62:35380) 2024-09-12T00:04:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5a7fc640 1 -- 172.21.15.62:0/256010793 learned_addr learned my addr 172.21.15.62:0/256010793 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:15.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e5c100e20 0x7f5e5c073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5b7fe640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c102ef0 0x7f5e5c06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5a7fc640 1 -- 172.21.15.62:0/256010793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e5c100e20 msgr2=0x7f5e5c073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:15.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e5c100e20 0x7f5e5c073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5a7fc640 1 -- 172.21.15.62:0/256010793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c102ef0 msgr2=0x7f5e5c06d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:15.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c102ef0 0x7f5e5c06d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5a7fc640 1 -- 172.21.15.62:0/256010793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5c06d650 con 0x7f5e5c102570 2024-09-12T00:04:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.740+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e5c102570 0x7f5e5c06cad0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000d950 tx=0x7f5e5000dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.741+0000 7f5e609a2640 1 -- 172.21.15.62:0/256010793 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50017070 con 0x7f5e5c102570 2024-09-12T00:04:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.741+0000 7f5e619a4640 1 -- 172.21.15.62:0/256010793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5c071570 con 0x7f5e5c102570 2024-09-12T00:04:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.741+0000 7f5e619a4640 1 -- 172.21.15.62:0/256010793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5c071b00 con 0x7f5e5c102570 2024-09-12T00:04:15.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.741+0000 7f5e609a2640 1 -- 172.21.15.62:0/256010793 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e5000de80 con 0x7f5e5c102570 2024-09-12T00:04:15.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.741+0000 7f5e609a2640 1 -- 172.21.15.62:0/256010793 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000f620 con 0x7f5e5c102570 2024-09-12T00:04:15.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.743+0000 7f5e609a2640 1 -- 172.21.15.62:0/256010793 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5e50009c30 con 0x7f5e5c102570 2024-09-12T00:04:15.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.744+0000 7f5e63d7d740 1 Processor -- start 2024-09-12T00:04:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.745+0000 7f5e63d7d740 1 -- start start 2024-09-12T00:04:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.745+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 0x55cbb1f50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.745+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f505d0 0x55cbb1f50a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.745+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f5d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.745+0000 7f5e63d7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cbb1eb62f0 con 0x55cbb1f51b40 2024-09-12T00:04:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.745+0000 7f5e63d7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cbb1e5dcf0 con 0x55cbb1f5b370 2024-09-12T00:04:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.745+0000 7f5e63d7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cbb1eb22b0 con 0x55cbb1f505d0 2024-09-12T00:04:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.745+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 0x55cbb1f50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.746+0000 7f5e5a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f5d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.746+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 0x55cbb1f50090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47186/0 (socket says 172.21.15.62:47186) 2024-09-12T00:04:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.746+0000 7f5e5a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f5d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35384/0 (socket says 172.21.15.62:35384) 2024-09-12T00:04:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.746+0000 7f5e5b7fe640 1 -- 172.21.15.62:0/2836460670 learned_addr learned my addr 172.21.15.62:0/2836460670 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.746+0000 7f5e5affd640 1 --2- 172.21.15.62:0/2836460670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f505d0 0x55cbb1f50a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.746+0000 7f5e5a7fc640 1 -- 172.21.15.62:0/2836460670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f505d0 msgr2=0x55cbb1f50a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.746+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/2836460670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f505d0 0x55cbb1f50a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.746+0000 7f5e5a7fc640 1 -- 172.21.15.62:0/2836460670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 msgr2=0x55cbb1f50090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.746+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/2836460670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 0x55cbb1f50090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.746+0000 7f5e5a7fc640 1 -- 172.21.15.62:0/2836460670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb1e423a0 con 0x55cbb1f5b370 2024-09-12T00:04:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.748+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/2836460670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f5d760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5e500cc520 tx=0x7f5e500d7570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42eca0596e752ff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.748+0000 7f5e41ffb640 1 -- 172.21.15.62:0/2836460670 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e500e2070 con 0x55cbb1f5b370 2024-09-12T00:04:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.748+0000 7f5e41ffb640 1 -- 172.21.15.62:0/2836460670 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e500ddae0 con 0x55cbb1f5b370 2024-09-12T00:04:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.748+0000 7f5e41ffb640 1 -- 172.21.15.62:0/2836460670 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e500dca80 con 0x55cbb1f5b370 2024-09-12T00:04:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.749+0000 7f5e609a2640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5e440a2d70 0x7f5e440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.749+0000 7f5e5b7fe640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5e440a2d70 0x7f5e440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.749+0000 7f5e63d7d740 1 -- 172.21.15.62:0/2836460670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 msgr2=0x55cbb1f5d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.749+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/2836460670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f5d760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5e500cc520 tx=0x7f5e500d7570 comp rx=0 tx=0).stop 2024-09-12T00:04:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.750+0000 7f5e609a2640 1 -- 172.21.15.62:0/256010793 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f5e500cedf0 con 0x7f5e5c102570 2024-09-12T00:04:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.750+0000 7f5e5b7fe640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5e440a2d70 0x7f5e440a5230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009be0 tx=0x7f5e540098a0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.750+0000 7f5e63d7d740 1 -- 172.21.15.62:0/2836460670 shutdown_connections 2024-09-12T00:04:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.750+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/2836460670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f5d760 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.750+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/2836460670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f505d0 0x55cbb1f50a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.750+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/2836460670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 0x55cbb1f50090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.750+0000 7f5e63d7d740 1 -- 172.21.15.62:0/2836460670 >> 172.21.15.62:0/2836460670 conn(0x55cbb1f19310 msgr2=0x55cbb1f4cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.750+0000 7f5e63d7d740 1 -- 172.21.15.62:0/2836460670 shutdown_connections 2024-09-12T00:04:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.750+0000 7f5e63d7d740 1 -- 172.21.15.62:0/2836460670 wait complete. 2024-09-12T00:04:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.750+0000 7f5e63d7d740 1 -- start start 2024-09-12T00:04:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.751+0000 7f5e63d7d740 1 Processor -- start 2024-09-12T00:04:15.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.751+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f505d0 0x55cbb1f5ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.751+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 0x55cbb1f57fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.751+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f58570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.751+0000 7f5e63d7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cbb1f5e070 con 0x55cbb1f51b40 2024-09-12T00:04:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.751+0000 7f5e63d7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cbb1e71060 con 0x55cbb1f5b370 2024-09-12T00:04:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.751+0000 7f5e63d7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cbb1f5e1f0 con 0x55cbb1f505d0 2024-09-12T00:04:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.751+0000 7f5e63d7d740 10 client.0 _ll_register_callbacks cb 0x55cbb1bc48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:15.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f58570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f505d0 0x55cbb1f5ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f58570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35396/0 (socket says 172.21.15.62:35396) 2024-09-12T00:04:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5b7fe640 1 -- 172.21.15.62:0/4064835640 learned_addr learned my addr 172.21.15.62:0/4064835640 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 0x55cbb1f57fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5b7fe640 1 -- 172.21.15.62:0/4064835640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f505d0 msgr2=0x55cbb1f5ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5b7fe640 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f505d0 0x55cbb1f5ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5b7fe640 1 -- 172.21.15.62:0/4064835640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 msgr2=0x55cbb1f57fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5b7fe640 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 0x55cbb1f57fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5b7fe640 1 -- 172.21.15.62:0/4064835640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb1c273c0 con 0x55cbb1f5b370 2024-09-12T00:04:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e5b7fe640 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f58570 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009370 tx=0x7f5e54009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=583a3c26cd9eb95f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.752+0000 7f5e3affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:04:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.753+0000 7f5e3affd640 1 -- 172.21.15.62:0/4064835640 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54036050 con 0x55cbb1f5b370 2024-09-12T00:04:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.753+0000 7f5e63d7d740 1 -- 172.21.15.62:0/4064835640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e500ddae0 con 0x55cbb1f5b370 2024-09-12T00:04:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.753+0000 7f5e3affd640 1 -- 172.21.15.62:0/4064835640 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f5e54004040 con 0x55cbb1f5b370 2024-09-12T00:04:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.754+0000 7f5e3affd640 1 -- 172.21.15.62:0/4064835640 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5e54003e40 con 0x55cbb1f5b370 2024-09-12T00:04:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.754+0000 7f5e63d7d740 10 client.27074 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.754+0000 7f5e63d7d740 1 -- 172.21.15.62:0/4064835640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cbb1f69b50 con 0x55cbb1f5b370 2024-09-12T00:04:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.755+0000 7f5e3affd640 1 -- 172.21.15.62:0/4064835640 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e500ddae0 con 0x55cbb1f5b370 2024-09-12T00:04:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.755+0000 7f5e3affd640 1 -- 172.21.15.62:0/4064835640 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f5e5403c040 con 0x55cbb1f5b370 2024-09-12T00:04:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.755+0000 7f5e63d7d740 10 client.27074 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.755+0000 7f5e63d7d740 10 client.27074 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:04:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.755+0000 7f5e63d7d740 20 client.27074 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.755+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cbb1f6b060 0x55cbb1f6d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.755+0000 7f5e63d7d740 4 client.27074 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:04:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.755+0000 7f5e63d7d740 1 -- 172.21.15.62:0/4064835640 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbb1ec3860 con 0x55cbb1f6b060 2024-09-12T00:04:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.755+0000 7f5e5affd640 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cbb1f6b060 0x55cbb1f6d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.756+0000 7f5e5affd640 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cbb1f6b060 0x55cbb1f6d470 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f98ef71a4bd517d server_cookie=14c56c350770bd47 in_seq=0 out_seq=0 2024-09-12T00:04:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.756+0000 7f5e3affd640 10 client.27074 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:04:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e3affd640 1 -- 172.21.15.62:0/4064835640 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbb1ec3860 con 0x55cbb1f6b060 2024-09-12T00:04:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e3affd640 10 client.27074 handle_command_reply: tid=0 2024-09-12T00:04:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e63d7d740 1 client.27074 shutdown 2024-09-12T00:04:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e63d7d740 1 -- 172.21.15.62:0/4064835640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 msgr2=0x55cbb1f58570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f58570 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009370 tx=0x7f5e54009500 comp rx=0 tx=0).stop 2024-09-12T00:04:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e63d7d740 1 -- 172.21.15.62:0/4064835640 shutdown_connections 2024-09-12T00:04:15.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cbb1f6b060 0x55cbb1f6d470 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f58570 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f51b40 0x55cbb1f57fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/4064835640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f505d0 0x55cbb1f5ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e63d7d740 1 -- 172.21.15.62:0/4064835640 >> 172.21.15.62:0/4064835640 conn(0x55cbb1f19310 msgr2=0x55cbb1f52930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e3affd640 0 client.27074 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:04:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.757+0000 7f5e63d7d740 1 -- 172.21.15.62:0/4064835640 shutdown_connections 2024-09-12T00:04:15.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.758+0000 7f5e63d7d740 1 -- 172.21.15.62:0/4064835640 wait complete. 2024-09-12T00:04:15.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.758+0000 7f5e63d7d740 20 client.27074 trim_cache size 0 max 16384 2024-09-12T00:04:15.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.779+0000 7f5e63d7d740 1 Processor -- start 2024-09-12T00:04:15.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.779+0000 7f5e63d7d740 1 -- start start 2024-09-12T00:04:15.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f4ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f51b40 0x55cbb1f67370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f5b370 0x55cbb1f72d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e63d7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5e480111d0 con 0x55cbb1f505d0 2024-09-12T00:04:15.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e5affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f4ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e5affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f4ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47210/0 (socket says 172.21.15.62:47210) 2024-09-12T00:04:15.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e5affd640 1 -- 172.21.15.62:0/354846184 learned_addr learned my addr 172.21.15.62:0/354846184 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:15.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e63d7d740 1 -- 172.21.15.62:0/354846184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4800ea40 con 0x55cbb1f51b40 2024-09-12T00:04:15.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e63d7d740 1 -- 172.21.15.62:0/354846184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4800c8b0 con 0x55cbb1f5b370 2024-09-12T00:04:15.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/354846184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f51b40 0x55cbb1f67370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e5b7fe640 1 --2- 172.21.15.62:0/354846184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f5b370 0x55cbb1f72d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e5a7fc640 1 -- 172.21.15.62:0/354846184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f5b370 msgr2=0x55cbb1f72d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:15.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/354846184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f5b370 0x55cbb1f72d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e5a7fc640 1 -- 172.21.15.62:0/354846184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 msgr2=0x55cbb1f4ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:15.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/354846184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f4ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.780+0000 7f5e5a7fc640 1 -- 172.21.15.62:0/354846184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb1f588a0 con 0x55cbb1f51b40 2024-09-12T00:04:15.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.781+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/354846184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f51b40 0x55cbb1f67370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55cbb1f59610 tx=0x7f5e50000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e341da2c63d15e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:15.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.781+0000 7f5e417fa640 1 -- 172.21.15.62:0/354846184 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50020070 con 0x55cbb1f51b40 2024-09-12T00:04:15.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.781+0000 7f5e417fa640 1 -- 172.21.15.62:0/354846184 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e5009b9f0 con 0x55cbb1f51b40 2024-09-12T00:04:15.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.781+0000 7f5e417fa640 1 -- 172.21.15.62:0/354846184 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50095e60 con 0x55cbb1f51b40 2024-09-12T00:04:15.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 -- 172.21.15.62:0/354846184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f51b40 msgr2=0x55cbb1f67370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:15.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/354846184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f51b40 0x55cbb1f67370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55cbb1f59610 tx=0x7f5e50000bc0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 -- 172.21.15.62:0/354846184 shutdown_connections 2024-09-12T00:04:15.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/354846184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f5b370 0x55cbb1f72d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/354846184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f51b40 0x55cbb1f67370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/354846184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f4ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 -- 172.21.15.62:0/354846184 >> 172.21.15.62:0/354846184 conn(0x55cbb1f19310 msgr2=0x55cbb1f4c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:15.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 -- 172.21.15.62:0/354846184 shutdown_connections 2024-09-12T00:04:15.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 -- 172.21.15.62:0/354846184 wait complete. 2024-09-12T00:04:15.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 -- start start 2024-09-12T00:04:15.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.782+0000 7f5e63d7d740 1 Processor -- start 2024-09-12T00:04:15.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.783+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.783+0000 7f5e63d7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f51b40 0x55cbb1f54970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.783+0000 7f5e5affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.783+0000 7f5e5affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f638e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47216/0 (socket says 172.21.15.62:47216) 2024-09-12T00:04:15.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.783+0000 7f5e5affd640 1 -- 172.21.15.62:0/3511017281 learned_addr learned my addr 172.21.15.62:0/3511017281 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:15.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.783+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f51b40 0x55cbb1f54970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.783+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f54eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.784+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4800cf00 con 0x55cbb1f505d0 2024-09-12T00:04:15.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.784+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4800e370 con 0x55cbb1f5b370 2024-09-12T00:04:15.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.784+0000 7f5e5b7fe640 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f54eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.784+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4800d5d0 con 0x55cbb1f51b40 2024-09-12T00:04:15.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.784+0000 7f5e63d7d740 10 client.0 _ll_register_callbacks cb 0x7f5e4c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:15.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.785+0000 7f5e5affd640 1 -- 172.21.15.62:0/3511017281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f51b40 msgr2=0x55cbb1f54970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:15.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.785+0000 7f5e5affd640 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f51b40 0x55cbb1f54970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.785+0000 7f5e5affd640 1 -- 172.21.15.62:0/3511017281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 msgr2=0x55cbb1f54eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:15.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.785+0000 7f5e5affd640 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f54eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.785+0000 7f5e5affd640 1 -- 172.21.15.62:0/3511017281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb1f553f0 con 0x55cbb1f505d0 2024-09-12T00:04:15.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.785+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f51b40 0x55cbb1f54970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:04:15.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.785+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e5009b9f0 con 0x55cbb1f505d0 2024-09-12T00:04:15.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.785+0000 7f5e5affd640 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f638e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009b80 tx=0x7f5e4c0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80403a0870452d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:15.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.785+0000 7f5e3affd640 10 client.17538 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:15.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.785+0000 7f5e3affd640 1 -- 172.21.15.62:0/3511017281 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c005020 con 0x55cbb1f505d0 2024-09-12T00:04:15.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.786+0000 7f5e3affd640 1 -- 172.21.15.62:0/3511017281 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8557+0+0 (secure 0 0 0) 0x7f5e4c0051c0 con 0x55cbb1f505d0 2024-09-12T00:04:15.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.786+0000 7f5e3affd640 1 -- 172.21.15.62:0/3511017281 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5e4c0053e0 con 0x55cbb1f505d0 2024-09-12T00:04:15.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.786+0000 7f5e63d7d740 10 client.17538 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:15.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.786+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55cbb1fd5960 con 0x55cbb1f505d0 2024-09-12T00:04:15.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.787+0000 7f5e3affd640 1 -- 172.21.15.62:0/3511017281 <== mon.0 v2:172.21.15.62:3300/0 4 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f5e4c0047a0 con 0x55cbb1f505d0 2024-09-12T00:04:15.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.787+0000 7f5e63d7d740 10 client.17538 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:15.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.787+0000 7f5e63d7d740 10 client.17538 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:04:15.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.787+0000 7f5e63d7d740 20 client.17538 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:15.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.787+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cbb1fd6030 0x55cbb1fd8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:15.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.787+0000 7f5e63d7d740 4 client.17538 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:04:15.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.787+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5e5009b9f0 con 0x55cbb1fd6030 2024-09-12T00:04:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.787+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cbb1fd6030 0x55cbb1fd8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.788+0000 7f5e3affd640 10 client.17538 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:04:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.788+0000 7f5e5a7fc640 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cbb1fd6030 0x55cbb1fd8450 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32fd89ab7abd45ad server_cookie=79fd0acb3fad06b3 in_seq=0 out_seq=0 2024-09-12T00:04:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.788+0000 7f5e3affd640 1 -- 172.21.15.62:0/3511017281 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5e5009b9f0 con 0x55cbb1fd6030 2024-09-12T00:04:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.788+0000 7f5e3affd640 10 client.17538 handle_command_reply: tid=0 2024-09-12T00:04:15.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.788+0000 7f5e63d7d740 1 client.17538 shutdown 2024-09-12T00:04:15.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.789+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 msgr2=0x55cbb1f638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:15.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.789+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f638e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009b80 tx=0x7f5e4c0044d0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.789+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 shutdown_connections 2024-09-12T00:04:15.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.789+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55cbb1fd6030 0x55cbb1fd8450 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.789+0000 7f5e3affd640 0 client.17538 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:04:15.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.789+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbb1f5b370 0x55cbb1f54eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.789+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbb1f51b40 0x55cbb1f54970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.789+0000 7f5e63d7d740 1 --2- 172.21.15.62:0/3511017281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbb1f505d0 0x55cbb1f638e0 secure :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009b80 tx=0x7f5e4c0044d0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.789+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 >> 172.21.15.62:0/3511017281 conn(0x55cbb1f19310 msgr2=0x55cbb1f52110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.790+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 shutdown_connections 2024-09-12T00:04:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.790+0000 7f5e63d7d740 1 -- 172.21.15.62:0/3511017281 wait complete. 2024-09-12T00:04:15.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.790+0000 7f5e63d7d740 20 client.17538 trim_cache size 0 max 16384 2024-09-12T00:04:15.812 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:04:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 -- 172.21.15.62:0/256010793 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5e440a2d70 msgr2=0x7f5e440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:15.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5e440a2d70 0x7f5e440a5230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009be0 tx=0x7f5e540098a0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 -- 172.21.15.62:0/256010793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e5c102570 msgr2=0x7f5e5c06cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:15.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e5c102570 0x7f5e5c06cad0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000d950 tx=0x7f5e5000dd50 comp rx=0 tx=0).stop 2024-09-12T00:04:15.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 -- 172.21.15.62:0/256010793 shutdown_connections 2024-09-12T00:04:15.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5e440a2d70 0x7f5e440a5230 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c102ef0 0x7f5e5c06d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e5c102570 0x7f5e5c06cad0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 --2- 172.21.15.62:0/256010793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e5c100e20 0x7f5e5c073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:15.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 -- 172.21.15.62:0/256010793 >> 172.21.15.62:0/256010793 conn(0x7f5e5c0fc7d0 msgr2=0x7f5e5c1048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:15.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 -- 172.21.15.62:0/256010793 shutdown_connections 2024-09-12T00:04:15.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:15.791+0000 7f5e619a4640 1 -- 172.21.15.62:0/256010793 wait complete. 2024-09-12T00:04:15.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:15 smithi120 ceph-mon[35182]: pgmap v378: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 831 op/s 2024-09-12T00:04:16.502 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:16.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:04:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.091+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3485655104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab38109d70 msgr2=0x7fab3810e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.091+0000 7fab3c8dd640 1 --2- 172.21.15.62:0/3485655104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab38109d70 0x7fab3810e350 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009860 tx=0x7fab2c02a340 comp rx=0 tx=0).stop 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.091+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3485655104 shutdown_connections 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.091+0000 7fab3c8dd640 1 --2- 172.21.15.62:0/3485655104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3810e890 0x7fab3810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.091+0000 7fab3c8dd640 1 --2- 172.21.15.62:0/3485655104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab38109d70 0x7fab3810e350 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.091+0000 7fab3c8dd640 1 --2- 172.21.15.62:0/3485655104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab38109450 0x7fab38109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.091+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3485655104 >> 172.21.15.62:0/3485655104 conn(0x7fab3806e2d0 msgr2=0x7fab3806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.091+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3485655104 shutdown_connections 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.091+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3485655104 wait complete. 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.092+0000 7fab3c8dd640 1 Processor -- start 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.092+0000 7fab3c8dd640 1 -- start start 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab3c8dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab38109450 0x7fab381a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab3c8dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab38109d70 0x7fab381a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab3c8dd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3810e890 0x7fab381a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab3c8dd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab3811a7a0 con 0x7fab3810e890 2024-09-12T00:04:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab3c8dd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab3811a620 con 0x7fab38109450 2024-09-12T00:04:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab3c8dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fab3811a920 con 0x7fab38109d70 2024-09-12T00:04:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab35d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab38109d70 0x7fab381a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab3659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab38109450 0x7fab381a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab3659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab38109450 0x7fab381a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35432/0 (socket says 172.21.15.62:35432) 2024-09-12T00:04:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab35d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab38109d70 0x7fab381a6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:47296/0 (socket says 172.21.15.62:47296) 2024-09-12T00:04:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.093+0000 7fab3659c640 1 -- 172.21.15.62:0/3499206058 learned_addr learned my addr 172.21.15.62:0/3499206058 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.094+0000 7fab36d9d640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3810e890 0x7fab381a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.094+0000 7fab35d9b640 1 -- 172.21.15.62:0/3499206058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab38109450 msgr2=0x7fab381a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.094+0000 7fab35d9b640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab38109450 0x7fab381a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.094+0000 7fab35d9b640 1 -- 172.21.15.62:0/3499206058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3810e890 msgr2=0x7fab381a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:17.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.094+0000 7fab35d9b640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3810e890 0x7fab381a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.094+0000 7fab35d9b640 1 -- 172.21.15.62:0/3499206058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab381a11e0 con 0x7fab38109d70 2024-09-12T00:04:17.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.095+0000 7fab35d9b640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab38109d70 0x7fab381a6e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab2c035040 tx=0x7fab2c033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.099+0000 7fab277fe640 1 -- 172.21.15.62:0/3499206058 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c03f020 con 0x7fab38109d70 2024-09-12T00:04:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.099+0000 7fab277fe640 1 -- 172.21.15.62:0/3499206058 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2c030550 con 0x7fab38109d70 2024-09-12T00:04:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.099+0000 7fab277fe640 1 -- 172.21.15.62:0/3499206058 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c037040 con 0x7fab38109d70 2024-09-12T00:04:17.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.099+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3499206058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab381a1470 con 0x7fab38109d70 2024-09-12T00:04:17.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.099+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3499206058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab381ad7c0 con 0x7fab38109d70 2024-09-12T00:04:17.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.103+0000 7fab277fe640 1 -- 172.21.15.62:0/3499206058 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fab2c042020 con 0x7fab38109d70 2024-09-12T00:04:17.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.104+0000 7fab277fe640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fab200a2ec0 0x7fab200a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.104+0000 7fab277fe640 1 -- 172.21.15.62:0/3499206058 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fab2c0eec40 con 0x7fab38109d70 2024-09-12T00:04:17.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.111+0000 7fab3ecb6740 1 Processor -- start 2024-09-12T00:04:17.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.111+0000 7fab3ecb6740 1 -- start start 2024-09-12T00:04:17.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.112+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 0x5581b3360d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.112+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 0x5581b335b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.112+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b335bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.112+0000 7fab3ecb6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5581b32b72a0 con 0x5581b335b960 2024-09-12T00:04:17.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.112+0000 7fab3ecb6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581b325eb10 con 0x5581b33612c0 2024-09-12T00:04:17.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.112+0000 7fab3659c640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fab200a2ec0 0x7fab200a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.112+0000 7fab3ecb6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5581b32b3260 con 0x5581b33609a0 2024-09-12T00:04:17.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.114+0000 7fab3659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 0x5581b3360d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.114+0000 7fab3659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 0x5581b3360d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:47310/0 (socket says 172.21.15.62:47310) 2024-09-12T00:04:17.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.114+0000 7fab3659c640 1 -- 172.21.15.62:0/1178038023 learned_addr learned my addr 172.21.15.62:0/1178038023 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:17.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.115+0000 7fab35d9b640 1 --2- 172.21.15.62:0/1178038023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 0x5581b335b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.116+0000 7fab35d9b640 1 -- 172.21.15.62:0/1178038023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 msgr2=0x5581b3360d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:17.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.116+0000 7fab35d9b640 1 --2- 172.21.15.62:0/1178038023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 0x5581b3360d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.116+0000 7fab35d9b640 1 -- 172.21.15.62:0/1178038023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 msgr2=0x5581b335bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:17.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.116+0000 7fab35d9b640 1 --2- 172.21.15.62:0/1178038023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b335bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.116+0000 7fab35d9b640 1 -- 172.21.15.62:0/1178038023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b3028b90 con 0x5581b33612c0 2024-09-12T00:04:17.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.116+0000 7fab35d9b640 1 --2- 172.21.15.62:0/1178038023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 0x5581b335b2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0ec440 tx=0x7fab2c0bb0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3909963866e2bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:17.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.119+0000 7fab24ff9640 1 -- 172.21.15.62:0/1178038023 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0bb210 con 0x5581b33612c0 2024-09-12T00:04:17.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.119+0000 7fab24ff9640 1 -- 172.21.15.62:0/1178038023 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2c0d9e50 con 0x5581b33612c0 2024-09-12T00:04:17.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.119+0000 7fab24ff9640 1 -- 172.21.15.62:0/1178038023 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0db0e0 con 0x5581b33612c0 2024-09-12T00:04:17.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.122+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1178038023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 msgr2=0x5581b335b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:17.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.122+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/1178038023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 0x5581b335b2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0ec440 tx=0x7fab2c0bb0a0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.123+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1178038023 shutdown_connections 2024-09-12T00:04:17.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.123+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/1178038023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b335bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.123+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/1178038023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 0x5581b335b2f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.123+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/1178038023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 0x5581b3360d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.123+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1178038023 >> 172.21.15.62:0/1178038023 conn(0x5581b334d7a0 msgr2=0x5581b334db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.125+0000 7fab3659c640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fab200a2ec0 0x7fab200a5380 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab28013a00 tx=0x7fab280098a0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.125+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1178038023 shutdown_connections 2024-09-12T00:04:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.125+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1178038023 wait complete. 2024-09-12T00:04:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.126+0000 7fab3ecb6740 1 -- start start 2024-09-12T00:04:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.126+0000 7fab3ecb6740 1 Processor -- start 2024-09-12T00:04:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.127+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b3364870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.127+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 0x5581b3364db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.127+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 0x5581b33652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.127+0000 7fab3659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b3364870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.128+0000 7fab3659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b3364870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47240/0 (socket says 172.21.15.62:47240) 2024-09-12T00:04:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.128+0000 7fab35d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 0x5581b3364db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.128+0000 7fab35d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 0x5581b3364db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:47326/0 (socket says 172.21.15.62:47326) 2024-09-12T00:04:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.128+0000 7fab3ecb6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5581b325eb10 con 0x5581b335b960 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.128+0000 7fab3659c640 1 -- 172.21.15.62:0/271585952 learned_addr learned my addr 172.21.15.62:0/271585952 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.128+0000 7fab36d9d640 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 0x5581b33652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.128+0000 7fab3ecb6740 1 -- 172.21.15.62:0/271585952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581b323f6e0 con 0x5581b33612c0 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.128+0000 7fab3ecb6740 1 -- 172.21.15.62:0/271585952 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5581b3367aa0 con 0x5581b33609a0 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.128+0000 7fab3ecb6740 10 client.0 _ll_register_callbacks cb 0x5581b3263a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.129+0000 7fab3659c640 1 -- 172.21.15.62:0/271585952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 msgr2=0x5581b3364db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.129+0000 7fab3659c640 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 0x5581b3364db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.129+0000 7fab3659c640 1 -- 172.21.15.62:0/271585952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 msgr2=0x5581b33652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.129+0000 7fab3659c640 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 0x5581b33652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.130+0000 7fab3659c640 1 -- 172.21.15.62:0/271585952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b3370f50 con 0x5581b335b960 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.130+0000 7fab36d9d640 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 0x5581b33652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.130+0000 7fab3659c640 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b3364870 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fab28009e60 tx=0x7fab28005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f3135f6330e828 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.130+0000 7fab1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.131+0000 7fab1dffb640 1 -- 172.21.15.62:0/271585952 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28027070 con 0x5581b335b960 2024-09-12T00:04:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.131+0000 7fab3ecb6740 1 -- 172.21.15.62:0/271585952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab2c0d9e50 con 0x5581b335b960 2024-09-12T00:04:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.131+0000 7fab1dffb640 1 -- 172.21.15.62:0/271585952 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fab28011b90 con 0x5581b335b960 2024-09-12T00:04:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.131+0000 7fab1dffb640 1 -- 172.21.15.62:0/271585952 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fab28031050 con 0x5581b335b960 2024-09-12T00:04:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.134+0000 7fab3ecb6740 10 client.17547 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.134+0000 7fab3ecb6740 1 -- 172.21.15.62:0/271585952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5581b33738d0 con 0x5581b335b960 2024-09-12T00:04:17.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.142+0000 7fab1dffb640 1 -- 172.21.15.62:0/271585952 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2c0d9e50 con 0x5581b335b960 2024-09-12T00:04:17.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.143+0000 7fab1dffb640 1 -- 172.21.15.62:0/271585952 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fab2801f040 con 0x5581b335b960 2024-09-12T00:04:17.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.143+0000 7fab3ecb6740 10 client.17547 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:17.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.143+0000 7fab3ecb6740 10 client.17547 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:04:17.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.143+0000 7fab3ecb6740 20 client.17547 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:17.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.143+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581b3375180 0x5581b3377600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.143+0000 7fab3ecb6740 4 client.17547 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:04:17.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.143+0000 7fab3ecb6740 1 -- 172.21.15.62:0/271585952 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581b32c49d0 con 0x5581b3375180 2024-09-12T00:04:17.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.144+0000 7fab35d9b640 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581b3375180 0x5581b3377600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.150+0000 7fab35d9b640 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581b3375180 0x5581b3377600 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8109335a01f235eb server_cookie=c653f27bc95ca61f in_seq=0 out_seq=0 2024-09-12T00:04:17.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.150+0000 7fab1dffb640 10 client.17547 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:17.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.152+0000 7fab1dffb640 1 -- 172.21.15.62:0/271585952 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581b32c49d0 con 0x5581b3375180 2024-09-12T00:04:17.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.152+0000 7fab1dffb640 10 client.17547 handle_command_reply: tid=0 2024-09-12T00:04:17.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.152+0000 7fab3ecb6740 1 client.17547 shutdown 2024-09-12T00:04:17.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.153+0000 7fab3ecb6740 1 -- 172.21.15.62:0/271585952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 msgr2=0x5581b3364870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:17.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.153+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b3364870 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fab28009e60 tx=0x7fab28005f90 comp rx=0 tx=0).stop 2024-09-12T00:04:17.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.153+0000 7fab3ecb6740 1 -- 172.21.15.62:0/271585952 shutdown_connections 2024-09-12T00:04:17.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.153+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581b3375180 0x5581b3377600 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.153+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33612c0 0x5581b33652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.153+0000 7fab1dffb640 0 client.17547 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:17.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.153+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33609a0 0x5581b3364db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.153+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/271585952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b3364870 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.153+0000 7fab3ecb6740 1 -- 172.21.15.62:0/271585952 >> 172.21.15.62:0/271585952 conn(0x5581b334d7a0 msgr2=0x5581b334ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.154+0000 7fab3ecb6740 1 -- 172.21.15.62:0/271585952 shutdown_connections 2024-09-12T00:04:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.154+0000 7fab3ecb6740 1 -- 172.21.15.62:0/271585952 wait complete. 2024-09-12T00:04:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.154+0000 7fab3ecb6740 20 client.17547 trim_cache size 0 max 16384 2024-09-12T00:04:17.161 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:17.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.194+0000 7fab3ecb6740 1 Processor -- start 2024-09-12T00:04:17.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab3ecb6740 1 -- start start 2024-09-12T00:04:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b335b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33609a0 0x5581b337cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33612c0 0x5581b337d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab3ecb6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab18010f60 con 0x5581b335b960 2024-09-12T00:04:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab3ecb6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab1800e7d0 con 0x5581b33609a0 2024-09-12T00:04:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab3ecb6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fab1800c6d0 con 0x5581b33612c0 2024-09-12T00:04:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab35d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33609a0 0x5581b337cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab35d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33609a0 0x5581b337cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35480/0 (socket says 172.21.15.62:35480) 2024-09-12T00:04:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab35d9b640 1 -- 172.21.15.62:0/19326295 learned_addr learned my addr 172.21.15.62:0/19326295 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.195+0000 7fab36d9d640 1 --2- 172.21.15.62:0/19326295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33612c0 0x5581b337d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.196+0000 7fab35d9b640 1 -- 172.21.15.62:0/19326295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33612c0 msgr2=0x5581b337d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:17.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.196+0000 7fab35d9b640 1 --2- 172.21.15.62:0/19326295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33612c0 0x5581b337d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.196+0000 7fab35d9b640 1 -- 172.21.15.62:0/19326295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 msgr2=0x5581b335b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:17.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.196+0000 7fab35d9b640 1 --2- 172.21.15.62:0/19326295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b335b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.196+0000 7fab35d9b640 1 -- 172.21.15.62:0/19326295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b33604d0 con 0x5581b33609a0 2024-09-12T00:04:17.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.196+0000 7fab35d9b640 1 --2- 172.21.15.62:0/19326295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33609a0 0x5581b337cf10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0304f0 tx=0x7fab2c0bb110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f5fac4326cec842 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:17.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.196+0000 7fab1ffff640 1 -- 172.21.15.62:0/19326295 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0bf5c0 con 0x5581b33609a0 2024-09-12T00:04:17.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.197+0000 7fab1ffff640 1 -- 172.21.15.62:0/19326295 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2c0b7880 con 0x5581b33609a0 2024-09-12T00:04:17.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.197+0000 7fab1ffff640 1 -- 172.21.15.62:0/19326295 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0ec7c0 con 0x5581b33609a0 2024-09-12T00:04:17.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.198+0000 7fab3ecb6740 1 -- 172.21.15.62:0/19326295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33609a0 msgr2=0x5581b337cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:17.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.198+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/19326295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33609a0 0x5581b337cf10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0304f0 tx=0x7fab2c0bb110 comp rx=0 tx=0).stop 2024-09-12T00:04:17.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.198+0000 7fab3ecb6740 1 -- 172.21.15.62:0/19326295 shutdown_connections 2024-09-12T00:04:17.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.198+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/19326295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33612c0 0x5581b337d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.198+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/19326295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b33609a0 0x5581b337cf10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.198+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/19326295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b335b960 0x5581b335b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.198+0000 7fab3ecb6740 1 -- 172.21.15.62:0/19326295 >> 172.21.15.62:0/19326295 conn(0x5581b334d7a0 msgr2=0x5581b33668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:17.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.198+0000 7fab3ecb6740 1 -- 172.21.15.62:0/19326295 shutdown_connections 2024-09-12T00:04:17.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.198+0000 7fab3ecb6740 1 -- 172.21.15.62:0/19326295 wait complete. 2024-09-12T00:04:17.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.199+0000 7fab3ecb6740 1 -- start start 2024-09-12T00:04:17.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.199+0000 7fab3ecb6740 1 Processor -- start 2024-09-12T00:04:17.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b335b960 0x5581b336ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b33609a0 0x5581b336d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab3ecb6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33612c0 0x5581b3369cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab3ecb6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab1800cc90 con 0x5581b33609a0 2024-09-12T00:04:17.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab3ecb6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab1800e100 con 0x5581b335b960 2024-09-12T00:04:17.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab3ecb6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fab1800d360 con 0x5581b33612c0 2024-09-12T00:04:17.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab3659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b335b960 0x5581b336ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab3659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b335b960 0x5581b336ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:35488/0 (socket says 172.21.15.62:35488) 2024-09-12T00:04:17.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab3659c640 1 -- 172.21.15.62:0/1823429700 learned_addr learned my addr 172.21.15.62:0/1823429700 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:17.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab3ecb6740 10 client.0 _ll_register_callbacks cb 0x7fab28009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:17.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab36d9d640 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33612c0 0x5581b3369cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.200+0000 7fab35d9b640 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b33609a0 0x5581b336d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.201+0000 7fab3659c640 1 -- 172.21.15.62:0/1823429700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33612c0 msgr2=0x5581b3369cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:17.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.201+0000 7fab3659c640 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33612c0 0x5581b3369cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.201+0000 7fab3659c640 1 -- 172.21.15.62:0/1823429700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b33609a0 msgr2=0x5581b336d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:17.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.201+0000 7fab3659c640 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b33609a0 0x5581b336d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.201+0000 7fab3659c640 1 -- 172.21.15.62:0/1823429700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b336a400 con 0x5581b335b960 2024-09-12T00:04:17.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.201+0000 7fab3659c640 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b335b960 0x5581b336ced0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab2800f7d0 tx=0x7fab28013490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=267e93ac6628a026 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:17.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.201+0000 7fab1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:04:17.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.201+0000 7fab1dffb640 1 -- 172.21.15.62:0/1823429700 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28027070 con 0x5581b335b960 2024-09-12T00:04:17.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.202+0000 7fab1dffb640 1 -- 172.21.15.62:0/1823429700 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fab28013740 con 0x5581b335b960 2024-09-12T00:04:17.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.202+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1823429700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab2c0b7880 con 0x5581b335b960 2024-09-12T00:04:17.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.203+0000 7fab1dffb640 1 -- 172.21.15.62:0/1823429700 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fab280092a0 con 0x5581b335b960 2024-09-12T00:04:17.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.203+0000 7fab3ecb6740 10 client.27098 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:17.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.203+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1823429700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5581b33e08d0 con 0x5581b335b960 2024-09-12T00:04:17.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.203+0000 7fab1dffb640 1 -- 172.21.15.62:0/1823429700 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2c0b7880 con 0x5581b335b960 2024-09-12T00:04:17.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.204+0000 7fab1dffb640 1 -- 172.21.15.62:0/1823429700 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fab28004660 con 0x5581b335b960 2024-09-12T00:04:17.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.204+0000 7fab3ecb6740 10 client.27098 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:17.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.204+0000 7fab3ecb6740 10 client.27098 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:04:17.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.204+0000 7fab3ecb6740 20 client.27098 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:17.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.204+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581b3375180 0x5581b33e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:17.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.204+0000 7fab3ecb6740 4 client.27098 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-12T00:04:17.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.204+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1823429700 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x5581b32bed20 con 0x5581b3375180 2024-09-12T00:04:17.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.204+0000 7fab35d9b640 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581b3375180 0x5581b33e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:17.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.205+0000 7fab35d9b640 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581b3375180 0x5581b33e3100 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=783e7194b62dd7a8 server_cookie=63cdf2a0bcb1f283 in_seq=0 out_seq=0 2024-09-12T00:04:17.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.205+0000 7fab1dffb640 10 client.27098 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab1dffb640 1 -- 172.21.15.62:0/1823429700 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5581b32bed20 con 0x5581b3375180 2024-09-12T00:04:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab1dffb640 10 client.27098 handle_command_reply: tid=0 2024-09-12T00:04:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 client.27098 shutdown 2024-09-12T00:04:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1823429700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b335b960 msgr2=0x5581b336ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b335b960 0x5581b336ced0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab2800f7d0 tx=0x7fab28013490 comp rx=0 tx=0).stop 2024-09-12T00:04:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1823429700 shutdown_connections 2024-09-12T00:04:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5581b3375180 0x5581b33e3100 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581b33612c0 0x5581b3369cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581b33609a0 0x5581b336d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 --2- 172.21.15.62:0/1823429700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581b335b960 0x5581b336ced0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab1dffb640 0 client.27098 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1823429700 >> 172.21.15.62:0/1823429700 conn(0x5581b334d7a0 msgr2=0x5581b334c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1823429700 shutdown_connections 2024-09-12T00:04:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 1 -- 172.21.15.62:0/1823429700 wait complete. 2024-09-12T00:04:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.427+0000 7fab3ecb6740 20 client.27098 trim_cache size 0 max 16384 2024-09-12T00:04:17.427 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 36, 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 46, 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "51961a7": { 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 46, 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:04:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 147.2, 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:17.429 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:04:17.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3499206058 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fab200a2ec0 msgr2=0x7fab200a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:17.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fab200a2ec0 0x7fab200a5380 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab28013a00 tx=0x7fab280098a0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3499206058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab38109d70 msgr2=0x7fab381a6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:17.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab38109d70 0x7fab381a6e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab2c035040 tx=0x7fab2c033b60 comp rx=0 tx=0).stop 2024-09-12T00:04:17.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3499206058 shutdown_connections 2024-09-12T00:04:17.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fab200a2ec0 0x7fab200a5380 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab3810e890 0x7fab381a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab38109d70 0x7fab381a6e30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 --2- 172.21.15.62:0/3499206058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab38109450 0x7fab381a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:17.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3499206058 >> 172.21.15.62:0/3499206058 conn(0x7fab3806e2d0 msgr2=0x7fab3810a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:17.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3499206058 shutdown_connections 2024-09-12T00:04:17.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:17.428+0000 7fab3c8dd640 1 -- 172.21.15.62:0/3499206058 wait complete. 2024-09-12T00:04:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 695.0 s, 2319.6 tps, lat 41.481 ms stddev 49.933 2024-09-12T00:04:17.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:17 smithi083 ceph-mon[34805]: pgmap v379: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 831 op/s 2024-09-12T00:04:17.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:17 smithi083 ceph-mon[34805]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T00:04:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:17 smithi062 ceph-mon[30958]: pgmap v379: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 831 op/s 2024-09-12T00:04:17.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:17 smithi062 ceph-mon[30958]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T00:04:17.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:17 smithi120 ceph-mon[35182]: pgmap v379: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 831 op/s 2024-09-12T00:04:17.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:17 smithi120 ceph-mon[35182]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T00:04:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.605+0000 7f044e3fb640 1 -- 172.21.15.62:0/377585522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448100350 msgr2=0x7f0448100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.605+0000 7f044e3fb640 1 --2- 172.21.15.62:0/377585522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448100350 0x7f0448100730 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0438009930 tx=0x7f043802a420 comp rx=0 tx=0).stop 2024-09-12T00:04:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.605+0000 7f044e3fb640 1 -- 172.21.15.62:0/377585522 shutdown_connections 2024-09-12T00:04:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.605+0000 7f044e3fb640 1 --2- 172.21.15.62:0/377585522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0448106f50 0x7f044810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.605+0000 7f044e3fb640 1 --2- 172.21.15.62:0/377585522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448100d00 0x7f0448106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.605+0000 7f044e3fb640 1 --2- 172.21.15.62:0/377585522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448100350 0x7f0448100730 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.605+0000 7f044e3fb640 1 -- 172.21.15.62:0/377585522 >> 172.21.15.62:0/377585522 conn(0x7f0448077210 msgr2=0x7f04480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.605+0000 7f044e3fb640 1 -- 172.21.15.62:0/377585522 shutdown_connections 2024-09-12T00:04:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.606+0000 7f044e3fb640 1 -- 172.21.15.62:0/377585522 wait complete. 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.606+0000 7f044e3fb640 1 Processor -- start 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.606+0000 7f044e3fb640 1 -- start start 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044e3fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0448100350 0x7f0448198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044e3fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448100d00 0x7f04481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044e3fb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448106f50 0x7f044819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044e3fb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0448111ff0 con 0x7f0448106f50 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044e3fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0448111e70 con 0x7f0448100d00 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044e3fb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0448112170 con 0x7f0448100350 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044c998640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448106f50 0x7f044819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044c998640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448106f50 0x7f044819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47272/0 (socket says 172.21.15.62:47272) 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044c998640 1 -- 172.21.15.62:0/443545377 learned_addr learned my addr 172.21.15.62:0/443545377 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:18.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f043ffff640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448100d00 0x7f04481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f0447fff640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0448100350 0x7f0448198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044c998640 1 -- 172.21.15.62:0/443545377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0448100350 msgr2=0x7f0448198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044c998640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0448100350 0x7f0448198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044c998640 1 -- 172.21.15.62:0/443545377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448100d00 msgr2=0x7f04481989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044c998640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448100d00 0x7f04481989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044c998640 1 -- 172.21.15.62:0/443545377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044819d420 con 0x7f0448106f50 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044c998640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448106f50 0x7f044819cd40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f044000cce0 tx=0x7f044000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f0445ffb640 1 -- 172.21.15.62:0/443545377 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044000f450 con 0x7f0448106f50 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f0445ffb640 1 -- 172.21.15.62:0/443545377 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044000fad0 con 0x7f0448106f50 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.607+0000 7f044e3fb640 1 -- 172.21.15.62:0/443545377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044819d710 con 0x7f0448106f50 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.608+0000 7f0445ffb640 1 -- 172.21.15.62:0/443545377 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440018c10 con 0x7f0448106f50 2024-09-12T00:04:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.608+0000 7f044e3fb640 1 -- 172.21.15.62:0/443545377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04481a4ff0 con 0x7f0448106f50 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.610+0000 7f0445ffb640 1 -- 172.21.15.62:0/443545377 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f044000f5f0 con 0x7f0448106f50 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.610+0000 7f04507d4740 1 Processor -- start 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.610+0000 7f04507d4740 1 -- start start 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f04507d4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068279090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f04507d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068279a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f04507d4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a068288780 0x55a06828ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f04507d4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a0681df2f0 con 0x55a06827ab40 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f04507d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a068186cf0 con 0x55a0682795d0 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f04507d4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a0681db2b0 con 0x55a068288780 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f0445ffb640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f04340a2d70 0x7f04340a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f043ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068279090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f043ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068279090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47282/0 (socket says 172.21.15.62:47282) 2024-09-12T00:04:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f043ffff640 1 -- 172.21.15.62:0/1728428780 learned_addr learned my addr 172.21.15.62:0/1728428780 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f044c998640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f04340a2d70 0x7f04340a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f0447fff640 1 --2- 172.21.15.62:0/1728428780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068279a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f0445ffb640 1 -- 172.21.15.62:0/443545377 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f04400cf970 con 0x7f0448106f50 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f043ffff640 1 --2- 172.21.15.62:0/1728428780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a068288780 0x55a06828ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f043ffff640 1 -- 172.21.15.62:0/1728428780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a068288780 msgr2=0x55a06828ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f043ffff640 1 --2- 172.21.15.62:0/1728428780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a068288780 0x55a06828ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f043ffff640 1 -- 172.21.15.62:0/1728428780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 msgr2=0x55a068279a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.611+0000 7f043ffff640 1 --2- 172.21.15.62:0/1728428780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068279a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.612+0000 7f043ffff640 1 -- 172.21.15.62:0/1728428780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a06816b3a0 con 0x55a06827ab40 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.612+0000 7f0447fff640 1 --2- 172.21.15.62:0/1728428780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068279a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:04:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.612+0000 7f044c998640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f04340a2d70 0x7f04340a5230 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f04400982f0 tx=0x7f044006de50 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.612+0000 7f043ffff640 1 --2- 172.21.15.62:0/1728428780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068279090 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f043000bfd0 tx=0x7f043000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dadc0adddd8f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.612+0000 7f043e7fc640 1 -- 172.21.15.62:0/1728428780 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430004030 con 0x55a06827ab40 2024-09-12T00:04:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.612+0000 7f043e7fc640 1 -- 172.21.15.62:0/1728428780 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04300041d0 con 0x55a06827ab40 2024-09-12T00:04:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.612+0000 7f043e7fc640 1 -- 172.21.15.62:0/1728428780 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043000f6e0 con 0x55a06827ab40 2024-09-12T00:04:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 -- 172.21.15.62:0/1728428780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 msgr2=0x55a068279090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 --2- 172.21.15.62:0/1728428780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068279090 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f043000bfd0 tx=0x7f043000b820 comp rx=0 tx=0).stop 2024-09-12T00:04:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 -- 172.21.15.62:0/1728428780 shutdown_connections 2024-09-12T00:04:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 --2- 172.21.15.62:0/1728428780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a068288780 0x55a06828ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 --2- 172.21.15.62:0/1728428780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068279a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 --2- 172.21.15.62:0/1728428780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068279090 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 -- 172.21.15.62:0/1728428780 >> 172.21.15.62:0/1728428780 conn(0x55a068242310 msgr2=0x55a068275c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 -- 172.21.15.62:0/1728428780 shutdown_connections 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 -- 172.21.15.62:0/1728428780 wait complete. 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 -- start start 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.613+0000 7f04507d4740 1 Processor -- start 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f04507d4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a0682795d0 0x55a068288200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f04507d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06827ab40 0x55a0682854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f04507d4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068285bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f04507d4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a06828b480 con 0x55a068288780 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f04507d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a06819a060 con 0x55a06827ab40 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f04507d4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a06828b600 con 0x55a0682795d0 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f04507d4740 10 client.0 _ll_register_callbacks cb 0x55a067eed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f043ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068285bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f043ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068285bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47296/0 (socket says 172.21.15.62:47296) 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f043ffff640 1 -- 172.21.15.62:0/1558095347 learned_addr learned my addr 172.21.15.62:0/1558095347 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f0447fff640 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06827ab40 0x55a0682854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f043ffff640 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a0682795d0 0x55a068288200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f043ffff640 1 -- 172.21.15.62:0/1558095347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a0682795d0 msgr2=0x55a068288200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f043ffff640 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a0682795d0 0x55a068288200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f043ffff640 1 -- 172.21.15.62:0/1558095347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06827ab40 msgr2=0x55a0682854b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f043ffff640 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06827ab40 0x55a0682854b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f043ffff640 1 -- 172.21.15.62:0/1558095347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a067f503c0 con 0x55a068288780 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f043ffff640 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068285bf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0430003cd0 tx=0x7f0430003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99d6f9817a0cbb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.614+0000 7f04277fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04277fe640 1 -- 172.21.15.62:0/1558095347 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430023020 con 0x55a068288780 2024-09-12T00:04:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04277fe640 1 -- 172.21.15.62:0/1558095347 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f04300037f0 con 0x55a068288780 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04507d4740 1 -- 172.21.15.62:0/1558095347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04300041d0 con 0x55a068288780 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04277fe640 1 -- 172.21.15.62:0/1558095347 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f04300039f0 con 0x55a068288780 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04507d4740 10 client.17562 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04507d4740 1 -- 172.21.15.62:0/1558095347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a068296fb0 con 0x55a068288780 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04277fe640 1 -- 172.21.15.62:0/1558095347 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04300041d0 con 0x55a068288780 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04277fe640 1 -- 172.21.15.62:0/1558095347 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f043001c070 con 0x55a068288780 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04507d4740 10 client.17562 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04507d4740 10 client.17562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04507d4740 20 client.17562 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04507d4740 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a068298300 0x55a06829a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04507d4740 4 client.17562 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.615+0000 7f04507d4740 1 -- 172.21.15.62:0/1558095347 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0681ec860 con 0x55a068298300 2024-09-12T00:04:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.616+0000 7f0447fff640 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a068298300 0x55a06829a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.616+0000 7f04277fe640 10 client.17562 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.616+0000 7f0447fff640 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a068298300 0x55a06829a6f0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46960ecd312b5e1 server_cookie=d686dd5606426966 in_seq=0 out_seq=0 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04277fe640 1 -- 172.21.15.62:0/1558095347 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0681ec860 con 0x55a068298300 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04277fe640 10 client.17562 handle_command_reply: tid=0 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 client.17562 shutdown 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 -- 172.21.15.62:0/1558095347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 msgr2=0x55a068285bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068285bf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0430003cd0 tx=0x7f0430003ec0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 -- 172.21.15.62:0/1558095347 shutdown_connections 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a068298300 0x55a06829a6f0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068285bf0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06827ab40 0x55a0682854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 --2- 172.21.15.62:0/1558095347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a0682795d0 0x55a068288200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04277fe640 0 client.17562 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 -- 172.21.15.62:0/1558095347 >> 172.21.15.62:0/1558095347 conn(0x55a068242310 msgr2=0x55a06827b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 -- 172.21.15.62:0/1558095347 shutdown_connections 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 1 -- 172.21.15.62:0/1558095347 wait complete. 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.617+0000 7f04507d4740 20 client.17562 trim_cache size 0 max 16384 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.630+0000 7f04507d4740 1 Processor -- start 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f04507d4740 1 -- start start 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f04507d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068284a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f04507d4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068275510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f04507d4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a068288780 0x55a068275a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f04507d4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f04481adf70 con 0x55a06827ab40 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f04507d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04481ab7e0 con 0x55a0682795d0 2024-09-12T00:04:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f04507d4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04481a9650 con 0x55a068288780 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f0447fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068275510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f0447fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068275510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47312/0 (socket says 172.21.15.62:47312) 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f0447fff640 1 -- 172.21.15.62:0/3989268509 learned_addr learned my addr 172.21.15.62:0/3989268509 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f0447fff640 1 -- 172.21.15.62:0/3989268509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a068288780 msgr2=0x55a068275a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f0447fff640 1 --2- 172.21.15.62:0/3989268509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a068288780 0x55a068275a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f0447fff640 1 -- 172.21.15.62:0/3989268509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 msgr2=0x55a068284a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f0447fff640 1 --2- 172.21.15.62:0/3989268509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068284a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f0447fff640 1 -- 172.21.15.62:0/3989268509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a06827a760 con 0x55a06827ab40 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f0447fff640 1 --2- 172.21.15.62:0/3989268509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068275510 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0438031550 tx=0x7f0438000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3945b7e34007d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.631+0000 7f043dffb640 1 -- 172.21.15.62:0/3989268509 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438035040 con 0x55a06827ab40 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f043dffb640 1 -- 172.21.15.62:0/3989268509 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043804b070 con 0x55a06827ab40 2024-09-12T00:04:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f043dffb640 1 -- 172.21.15.62:0/3989268509 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438031950 con 0x55a06827ab40 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f04507d4740 1 -- 172.21.15.62:0/3989268509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 msgr2=0x55a068275510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f04507d4740 1 --2- 172.21.15.62:0/3989268509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068275510 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0438031550 tx=0x7f0438000bc0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f04507d4740 1 -- 172.21.15.62:0/3989268509 shutdown_connections 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f04507d4740 1 --2- 172.21.15.62:0/3989268509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a068288780 0x55a068275a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f04507d4740 1 --2- 172.21.15.62:0/3989268509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a06827ab40 0x55a068275510 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f04507d4740 1 --2- 172.21.15.62:0/3989268509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068284a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f04507d4740 1 -- 172.21.15.62:0/3989268509 >> 172.21.15.62:0/3989268509 conn(0x55a068242310 msgr2=0x55a068287530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f04507d4740 1 -- 172.21.15.62:0/3989268509 shutdown_connections 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f04507d4740 1 -- 172.21.15.62:0/3989268509 wait complete. 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.632+0000 7f04507d4740 1 -- start start 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f04507d4740 1 Processor -- start 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f04507d4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068290a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f04507d4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a06827ab40 0x55a068290fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f04507d4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068281ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f04507d4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f04481a9ca0 con 0x55a068288780 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f04507d4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04481ab110 con 0x55a0682795d0 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f04507d4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04481aa370 con 0x55a06827ab40 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f04507d4740 10 client.0 _ll_register_callbacks cb 0x7f0438009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f043ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068281ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f043ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068281ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47328/0 (socket says 172.21.15.62:47328) 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f043ffff640 1 -- 172.21.15.62:0/3337692460 learned_addr learned my addr 172.21.15.62:0/3337692460 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f043ffff640 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068290a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.633+0000 7f0447fff640 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a06827ab40 0x55a068290fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f043ffff640 1 -- 172.21.15.62:0/3337692460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a06827ab40 msgr2=0x55a068290fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f043ffff640 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a06827ab40 0x55a068290fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f043ffff640 1 -- 172.21.15.62:0/3337692460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 msgr2=0x55a068290a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f043ffff640 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068290a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f043ffff640 1 -- 172.21.15.62:0/3337692460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0682824c0 con 0x55a068288780 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f043ffff640 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068281ce0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f043000c010 tx=0x7f043000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e43ed50024840c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f04277fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f04277fe640 1 -- 172.21.15.62:0/3337692460 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430036060 con 0x55a068288780 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f04277fe640 1 -- 172.21.15.62:0/3337692460 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f04300120d0 con 0x55a068288780 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f04507d4740 1 -- 172.21.15.62:0/3337692460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f043804b070 con 0x55a068288780 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f04277fe640 1 -- 172.21.15.62:0/3337692460 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0430023070 con 0x55a068288780 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f04507d4740 10 client.17568 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f04507d4740 1 -- 172.21.15.62:0/3337692460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a068302860 con 0x55a068288780 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.634+0000 7f04277fe640 1 -- 172.21.15.62:0/3337692460 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043804b070 con 0x55a068288780 2024-09-12T00:04:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f04277fe640 1 -- 172.21.15.62:0/3337692460 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f043001c070 con 0x55a068288780 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f04507d4740 10 client.17568 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f04507d4740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f04507d4740 20 client.17568 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f04507d4740 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a068302f80 0x55a0683053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f04507d4740 4 client.17568 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f04507d4740 1 -- 172.21.15.62:0/3337692460 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0681e6d70 con 0x55a068302f80 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f0447fff640 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a068302f80 0x55a0683053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f0447fff640 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a068302f80 0x55a0683053a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0668c71d786659f server_cookie=7330c2a63fb53608 in_seq=0 out_seq=0 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f04277fe640 10 client.17568 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f04277fe640 1 -- 172.21.15.62:0/3337692460 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0681e6d70 con 0x55a068302f80 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.635+0000 7f04277fe640 10 client.17568 handle_command_reply: tid=0 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 client.17568 shutdown 2024-09-12T00:04:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 -- 172.21.15.62:0/3337692460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 msgr2=0x55a068281ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068281ce0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f043000c010 tx=0x7f043000bf60 comp rx=0 tx=0).stop 2024-09-12T00:04:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 -- 172.21.15.62:0/3337692460 shutdown_connections 2024-09-12T00:04:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a068302f80 0x55a0683053a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04277fe640 0 client.17568 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a068288780 0x55a068281ce0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a06827ab40 0x55a068290fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 --2- 172.21.15.62:0/3337692460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0682795d0 0x55a068290a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 -- 172.21.15.62:0/3337692460 >> 172.21.15.62:0/3337692460 conn(0x55a068242310 msgr2=0x55a068286d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 -- 172.21.15.62:0/3337692460 shutdown_connections 2024-09-12T00:04:18.827 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:04:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 1 -- 172.21.15.62:0/3337692460 wait complete. 2024-09-12T00:04:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f04507d4740 20 client.17568 trim_cache size 0 max 16384 2024-09-12T00:04:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f044e3fb640 1 -- 172.21.15.62:0/443545377 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f04340a2d70 msgr2=0x7f04340a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f044e3fb640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f04340a2d70 0x7f04340a5230 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f04400982f0 tx=0x7f044006de50 comp rx=0 tx=0).stop 2024-09-12T00:04:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f044e3fb640 1 -- 172.21.15.62:0/443545377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448106f50 msgr2=0x7f044819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f044e3fb640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448106f50 0x7f044819cd40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f044000cce0 tx=0x7f044000a620 comp rx=0 tx=0).stop 2024-09-12T00:04:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.636+0000 7f044e3fb640 1 -- 172.21.15.62:0/443545377 shutdown_connections 2024-09-12T00:04:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.637+0000 7f044e3fb640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f04340a2d70 0x7f04340a5230 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.637+0000 7f044e3fb640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448106f50 0x7f044819cd40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.637+0000 7f044e3fb640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448100d00 0x7f04481989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.637+0000 7f044e3fb640 1 --2- 172.21.15.62:0/443545377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0448100350 0x7f0448198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.637+0000 7f044e3fb640 1 -- 172.21.15.62:0/443545377 >> 172.21.15.62:0/443545377 conn(0x7f0448077210 msgr2=0x7f0448105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.637+0000 7f044e3fb640 1 -- 172.21.15.62:0/443545377 shutdown_connections 2024-09-12T00:04:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:18.637+0000 7f044e3fb640 1 -- 172.21.15.62:0/443545377 wait complete. 2024-09-12T00:04:19.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:19 smithi062 ceph-mon[30958]: pgmap v381: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.3 MiB/s wr, 723 op/s 2024-09-12T00:04:19.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:19 smithi083 ceph-mon[34805]: pgmap v381: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.3 MiB/s wr, 723 op/s 2024-09-12T00:04:19.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:19 smithi120 ceph-mon[35182]: pgmap v381: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.3 MiB/s wr, 723 op/s 2024-09-12T00:04:20.303 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '51961a7', quiesce took 8.7 seconds. Will release after: 18.5 2024-09-12T00:04:20.491 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:04:20.492 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:04:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:21.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 -- 172.21.15.62:0/1645918715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1501036e0 msgr2=0x7fd150107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 --2- 172.21.15.62:0/1645918715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1501036e0 0x7fd150107b90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd140009930 tx=0x7fd14002a420 comp rx=0 tx=0).stop 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 -- 172.21.15.62:0/1645918715 shutdown_connections 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 --2- 172.21.15.62:0/1645918715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1501082a0 0x7fd15010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 --2- 172.21.15.62:0/1645918715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1501036e0 0x7fd150107b90 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 --2- 172.21.15.62:0/1645918715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd150102d30 0x7fd150103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 -- 172.21.15.62:0/1645918715 >> 172.21.15.62:0/1645918715 conn(0x7fd1500fc7d0 msgr2=0x7fd1500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 -- 172.21.15.62:0/1645918715 shutdown_connections 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 -- 172.21.15.62:0/1645918715 wait complete. 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 Processor -- start 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.221+0000 7fd156f51640 1 -- start start 2024-09-12T00:04:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd156f51640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd150102d30 0x7fd1501a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd156f51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1501036e0 0x7fd1501a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd156f51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1501082a0 0x7fd15019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd156f51640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501120c0 con 0x7fd150102d30 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd156f51640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd150111f40 con 0x7fd1501082a0 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd156f51640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd150112240 con 0x7fd1501036e0 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd154ced640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd150102d30 0x7fd1501a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd154ced640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd150102d30 0x7fd1501a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36996/0 (socket says 172.21.15.62:36996) 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd154ced640 1 -- 172.21.15.62:0/3152216337 learned_addr learned my addr 172.21.15.62:0/3152216337 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd147fff640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1501036e0 0x7fd1501a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd154ced640 1 -- 172.21.15.62:0/3152216337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1501036e0 msgr2=0x7fd1501a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd154ced640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1501036e0 0x7fd1501a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd154ced640 1 -- 172.21.15.62:0/3152216337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1501082a0 msgr2=0x7fd15019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd154ced640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1501082a0 0x7fd15019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd154ced640 1 -- 172.21.15.62:0/3152216337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15019af00 con 0x7fd150102d30 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd147fff640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1501036e0 0x7fd1501a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd154ced640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd150102d30 0x7fd1501a05c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00b4a0 tx=0x7fd14c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.222+0000 7fd145ffb640 1 -- 172.21.15.62:0/3152216337 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c014070 con 0x7fd150102d30 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.223+0000 7fd145ffb640 1 -- 172.21.15.62:0/3152216337 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd14c010af0 con 0x7fd150102d30 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.223+0000 7fd156f51640 1 -- 172.21.15.62:0/3152216337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15019b1f0 con 0x7fd150102d30 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.223+0000 7fd145ffb640 1 -- 172.21.15.62:0/3152216337 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c017d10 con 0x7fd150102d30 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.223+0000 7fd156f51640 1 -- 172.21.15.62:0/3152216337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1501a7370 con 0x7fd150102d30 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.225+0000 7fd145ffb640 1 -- 172.21.15.62:0/3152216337 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd14c017070 con 0x7fd150102d30 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.225+0000 7fd15932a740 1 Processor -- start 2024-09-12T00:04:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.225+0000 7fd15932a740 1 -- start start 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.225+0000 7fd15932a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438cb60 0x56422438cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.225+0000 7fd15932a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x564224397800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.225+0000 7fd15932a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564224397d40 0x56422439a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.225+0000 7fd15932a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5642242f2280 con 0x56422438cb60 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.225+0000 7fd15932a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564224299b10 con 0x56422438d480 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.225+0000 7fd15932a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642242ee240 con 0x564224397d40 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd145ffb640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1380a2cf0 0x7fd1380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd1554ee640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438cb60 0x56422438cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd1554ee640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438cb60 0x56422438cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37002/0 (socket says 172.21.15.62:37002) 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd1554ee640 1 -- 172.21.15.62:0/2614153719 learned_addr learned my addr 172.21.15.62:0/2614153719 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd147fff640 1 --2- 172.21.15.62:0/2614153719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564224397d40 0x56422439a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd145ffb640 1 -- 172.21.15.62:0/3152216337 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fd14c0ce830 con 0x7fd150102d30 2024-09-12T00:04:21.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd154ced640 1 --2- 172.21.15.62:0/2614153719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x564224397800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd1554ee640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1380a2cf0 0x7fd1380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd1554ee640 1 -- 172.21.15.62:0/2614153719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564224397d40 msgr2=0x56422439a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:21.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd1554ee640 1 --2- 172.21.15.62:0/2614153719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564224397d40 0x56422439a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd1554ee640 1 -- 172.21.15.62:0/2614153719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 msgr2=0x564224397800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:21.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd1554ee640 1 --2- 172.21.15.62:0/2614153719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x564224397800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd1554ee640 1 -- 172.21.15.62:0/2614153719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642242a61f0 con 0x56422438cb60 2024-09-12T00:04:21.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd154ced640 1 --2- 172.21.15.62:0/2614153719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x564224397800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:04:21.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd1554ee640 1 --2- 172.21.15.62:0/2614153719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438cb60 0x56422438cf40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd148014970 tx=0x7fd148014e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0511508554fc47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd136ffd640 1 -- 172.21.15.62:0/2614153719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148004b20 con 0x56422438cb60 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd136ffd640 1 -- 172.21.15.62:0/2614153719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd148004cc0 con 0x56422438cb60 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.226+0000 7fd136ffd640 1 -- 172.21.15.62:0/2614153719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148005290 con 0x56422438cb60 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.227+0000 7fd15932a740 1 -- 172.21.15.62:0/2614153719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438cb60 msgr2=0x56422438cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.227+0000 7fd1554ee640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1380a2cf0 0x7fd1380a51b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd14801e000 tx=0x7fd14801d000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.227+0000 7fd15932a740 1 --2- 172.21.15.62:0/2614153719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438cb60 0x56422438cf40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd148014970 tx=0x7fd148014e30 comp rx=0 tx=0).stop 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.227+0000 7fd15932a740 1 -- 172.21.15.62:0/2614153719 shutdown_connections 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.227+0000 7fd15932a740 1 --2- 172.21.15.62:0/2614153719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564224397d40 0x56422439a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.227+0000 7fd15932a740 1 --2- 172.21.15.62:0/2614153719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x564224397800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.227+0000 7fd15932a740 1 --2- 172.21.15.62:0/2614153719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438cb60 0x56422438cf40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.227+0000 7fd15932a740 1 -- 172.21.15.62:0/2614153719 >> 172.21.15.62:0/2614153719 conn(0x5642243890f0 msgr2=0x5642243894d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:21.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.227+0000 7fd15932a740 1 -- 172.21.15.62:0/2614153719 shutdown_connections 2024-09-12T00:04:21.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.227+0000 7fd15932a740 1 -- 172.21.15.62:0/2614153719 wait complete. 2024-09-12T00:04:21.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd15932a740 1 -- start start 2024-09-12T00:04:21.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd15932a740 1 Processor -- start 2024-09-12T00:04:21.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd15932a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x564224394bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd15932a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x564224395110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd15932a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x564224395650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd15932a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56422439a910 con 0x564224397d40 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd15932a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642242acf10 con 0x56422438d480 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd15932a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56422439aa90 con 0x56422438cb60 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd15932a740 10 client.0 _ll_register_callbacks cb 0x564224275a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd1554ee640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x564224395650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd1554ee640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x564224395650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37008/0 (socket says 172.21.15.62:37008) 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd1554ee640 1 -- 172.21.15.62:0/3342099021 learned_addr learned my addr 172.21.15.62:0/3342099021 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd147fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x564224395110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.228+0000 7fd154ced640 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x564224394bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd1554ee640 1 -- 172.21.15.62:0/3342099021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 msgr2=0x564224394bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:21.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd1554ee640 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x564224394bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd1554ee640 1 -- 172.21.15.62:0/3342099021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 msgr2=0x564224395110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd1554ee640 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x564224395110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd1554ee640 1 -- 172.21.15.62:0/3342099021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642240636a0 con 0x564224397d40 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd1554ee640 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x564224395650 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd1480047d0 tx=0x7fd148011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6f01de700f2b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd12ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd12ffff640 1 -- 172.21.15.62:0/3342099021 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148035060 con 0x564224397d40 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd12ffff640 1 -- 172.21.15.62:0/3342099021 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fd148005050 con 0x564224397d40 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd15932a740 1 -- 172.21.15.62:0/3342099021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd148004cc0 con 0x564224397d40 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd12ffff640 1 -- 172.21.15.62:0/3342099021 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd148004b20 con 0x564224397d40 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd15932a740 10 client.17580 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd15932a740 1 -- 172.21.15.62:0/3342099021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5642243a64c0 con 0x564224397d40 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd12ffff640 1 -- 172.21.15.62:0/3342099021 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd148004cc0 con 0x564224397d40 2024-09-12T00:04:21.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.229+0000 7fd12ffff640 1 -- 172.21.15.62:0/3342099021 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fd14800f050 con 0x564224397d40 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.230+0000 7fd15932a740 10 client.17580 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.230+0000 7fd15932a740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.230+0000 7fd15932a740 20 client.17580 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.230+0000 7fd15932a740 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5642243a7870 0x5642243a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.230+0000 7fd15932a740 4 client.17580 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.230+0000 7fd15932a740 1 -- 172.21.15.62:0/3342099021 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642242ff7f0 con 0x5642243a7870 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.230+0000 7fd154ced640 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5642243a7870 0x5642243a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.230+0000 7fd12ffff640 10 client.17580 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.230+0000 7fd154ced640 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5642243a7870 0x5642243a9c60 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1630be5bdb807302 server_cookie=44f76a810b0619a1 in_seq=0 out_seq=0 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd12ffff640 1 -- 172.21.15.62:0/3342099021 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd14c06ce80 con 0x5642243a7870 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd12ffff640 10 client.17580 handle_command_reply: tid=0 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 client.17580 shutdown 2024-09-12T00:04:21.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 -- 172.21.15.62:0/3342099021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 msgr2=0x564224395650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x564224395650 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd1480047d0 tx=0x7fd148011f70 comp rx=0 tx=0).stop 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 -- 172.21.15.62:0/3342099021 shutdown_connections 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5642243a7870 0x5642243a9c60 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x564224395650 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x564224395110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 --2- 172.21.15.62:0/3342099021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x564224394bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 -- 172.21.15.62:0/3342099021 >> 172.21.15.62:0/3342099021 conn(0x5642243890f0 msgr2=0x564224388270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd12ffff640 0 client.17580 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 -- 172.21.15.62:0/3342099021 shutdown_connections 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 1 -- 172.21.15.62:0/3342099021 wait complete. 2024-09-12T00:04:21.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.231+0000 7fd15932a740 20 client.17580 trim_cache size 0 max 16384 2024-09-12T00:04:21.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd15932a740 1 Processor -- start 2024-09-12T00:04:21.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd15932a740 1 -- start start 2024-09-12T00:04:21.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd15932a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x56422438f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd15932a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438d480 0x564224393a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd15932a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564224397d40 0x5642243af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd15932a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501b2550 con 0x56422438d480 2024-09-12T00:04:21.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd15932a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501afdc0 con 0x564224397d40 2024-09-12T00:04:21.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd15932a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501adc30 con 0x56422438cb60 2024-09-12T00:04:21.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd147fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438d480 0x564224393a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd147fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438d480 0x564224393a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37016/0 (socket says 172.21.15.62:37016) 2024-09-12T00:04:21.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.249+0000 7fd147fff640 1 -- 172.21.15.62:0/2274613555 learned_addr learned my addr 172.21.15.62:0/2274613555 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:21.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd154ced640 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x56422438f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd1554ee640 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564224397d40 0x5642243af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd147fff640 1 -- 172.21.15.62:0/2274613555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 msgr2=0x56422438f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:21.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd147fff640 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x56422438f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd147fff640 1 -- 172.21.15.62:0/2274613555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564224397d40 msgr2=0x5642243af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:21.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd147fff640 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564224397d40 0x5642243af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd147fff640 1 -- 172.21.15.62:0/2274613555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56422438d2b0 con 0x56422438d480 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd1554ee640 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564224397d40 0x5642243af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd154ced640 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x56422438f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd147fff640 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438d480 0x564224393a80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd140005ec0 tx=0x7fd140032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79bf79d62cdf58e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd1367fc640 1 -- 172.21.15.62:0/2274613555 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140035040 con 0x56422438d480 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd1367fc640 1 -- 172.21.15.62:0/2274613555 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1400318f0 con 0x56422438d480 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.250+0000 7fd1367fc640 1 -- 172.21.15.62:0/2274613555 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140004920 con 0x56422438d480 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 -- 172.21.15.62:0/2274613555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438d480 msgr2=0x564224393a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438d480 0x564224393a80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd140005ec0 tx=0x7fd140032f00 comp rx=0 tx=0).stop 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 -- 172.21.15.62:0/2274613555 shutdown_connections 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564224397d40 0x5642243af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56422438d480 0x564224393a80 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 --2- 172.21.15.62:0/2274613555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x56422438f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 -- 172.21.15.62:0/2274613555 >> 172.21.15.62:0/2274613555 conn(0x5642243890f0 msgr2=0x564224393370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 -- 172.21.15.62:0/2274613555 shutdown_connections 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 -- 172.21.15.62:0/2274613555 wait complete. 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 -- start start 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.251+0000 7fd15932a740 1 Processor -- start 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd15932a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x5642243a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd15932a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x56422439b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd15932a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x56422439c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd15932a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501ae280 con 0x564224397d40 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd15932a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501af6f0 con 0x56422438d480 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd15932a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501ae950 con 0x56422438cb60 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd15932a740 10 client.0 _ll_register_callbacks cb 0x7fd140009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd1554ee640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x56422439c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd1554ee640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x56422439c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37022/0 (socket says 172.21.15.62:37022) 2024-09-12T00:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd1554ee640 1 -- 172.21.15.62:0/3925920020 learned_addr learned my addr 172.21.15.62:0/3925920020 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd147fff640 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x56422439b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd154ced640 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x5642243a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd1554ee640 1 -- 172.21.15.62:0/3925920020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 msgr2=0x5642243a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd1554ee640 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x5642243a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd1554ee640 1 -- 172.21.15.62:0/3925920020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 msgr2=0x56422439b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd1554ee640 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x56422439b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd1554ee640 1 -- 172.21.15.62:0/3925920020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642244113f0 con 0x564224397d40 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd147fff640 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x56422439b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd154ced640 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x5642243a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd1554ee640 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x56422439c030 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd148005fd0 tx=0x7fd148005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75aaa18ed400a147 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.252+0000 7fd12ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd12ffff640 1 -- 172.21.15.62:0/3925920020 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148035040 con 0x564224397d40 2024-09-12T00:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd12ffff640 1 -- 172.21.15.62:0/3925920020 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fd148004350 con 0x564224397d40 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd15932a740 1 -- 172.21.15.62:0/3925920020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1400318f0 con 0x564224397d40 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd12ffff640 1 -- 172.21.15.62:0/3925920020 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd148046050 con 0x564224397d40 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd15932a740 10 client.17586 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd15932a740 1 -- 172.21.15.62:0/3925920020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564224412540 con 0x564224397d40 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd12ffff640 1 -- 172.21.15.62:0/3925920020 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1400318f0 con 0x564224397d40 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd12ffff640 1 -- 172.21.15.62:0/3925920020 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fd148030470 con 0x564224397d40 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd15932a740 10 client.17586 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd15932a740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd15932a740 20 client.17586 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd15932a740 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564224412c00 0x564224415010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd15932a740 4 client.17586 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd15932a740 1 -- 172.21.15.62:0/3925920020 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5642242f9d00 con 0x564224412c00 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.253+0000 7fd154ced640 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564224412c00 0x564224415010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:21.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.254+0000 7fd154ced640 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564224412c00 0x564224415010 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a41b68cff69171a6 server_cookie=3698f5b10df15377 in_seq=0 out_seq=0 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.254+0000 7fd12ffff640 10 client.17586 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd12ffff640 1 -- 172.21.15.62:0/3925920020 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5642242f9d00 con 0x564224412c00 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd12ffff640 10 client.17586 handle_command_reply: tid=0 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 client.17586 shutdown 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 -- 172.21.15.62:0/3925920020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 msgr2=0x56422439c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x56422439c030 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd148005fd0 tx=0x7fd148005c10 comp rx=0 tx=0).stop 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 -- 172.21.15.62:0/3925920020 shutdown_connections 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x564224412c00 0x564224415010 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564224397d40 0x56422439c030 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56422438d480 0x56422439b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd12ffff640 0 client.17586 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 --2- 172.21.15.62:0/3925920020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56422438cb60 0x5642243a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 -- 172.21.15.62:0/3925920020 >> 172.21.15.62:0/3925920020 conn(0x5642243890f0 msgr2=0x564224395360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:21.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 -- 172.21.15.62:0/3925920020 shutdown_connections 2024-09-12T00:04:21.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 1 -- 172.21.15.62:0/3925920020 wait complete. 2024-09-12T00:04:21.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.255+0000 7fd15932a740 20 client.17586 trim_cache size 0 max 16384 2024-09-12T00:04:21.265 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:04:21.265 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-12T00:04:21.265 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "90617977-6c53-47b9-a8d9-018bee329a6b", 2024-09-12T00:04:21.265 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-12T00:04:21.265 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:04:21.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 -- 172.21.15.62:0/3152216337 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1380a2cf0 msgr2=0x7fd1380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:21.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1380a2cf0 0x7fd1380a51b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd14801e000 tx=0x7fd14801d000 comp rx=0 tx=0).stop 2024-09-12T00:04:21.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 -- 172.21.15.62:0/3152216337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd150102d30 msgr2=0x7fd1501a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:21.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd150102d30 0x7fd1501a05c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00b4a0 tx=0x7fd14c00b960 comp rx=0 tx=0).stop 2024-09-12T00:04:21.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 -- 172.21.15.62:0/3152216337 shutdown_connections 2024-09-12T00:04:21.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd1380a2cf0 0x7fd1380a51b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1501082a0 0x7fd15019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1501036e0 0x7fd1501a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 --2- 172.21.15.62:0/3152216337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd150102d30 0x7fd1501a05c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:21.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 -- 172.21.15.62:0/3152216337 >> 172.21.15.62:0/3152216337 conn(0x7fd1500fc7d0 msgr2=0x7fd150100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:21.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 -- 172.21.15.62:0/3152216337 shutdown_connections 2024-09-12T00:04:21.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:21.256+0000 7fd156f51640 1 -- 172.21.15.62:0/3152216337 wait complete. 2024-09-12T00:04:21.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:21 smithi083 ceph-mon[34805]: pgmap v382: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-12T00:04:21.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:21 smithi062 ceph-mon[30958]: pgmap v382: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-12T00:04:21.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:21 smithi120 ceph-mon[35182]: pgmap v382: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-12T00:04:21.926 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:04:22.184 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:22.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 700.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:04:22.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.652+0000 7fbca757d640 1 -- 172.21.15.62:0/2196488602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0106a80 msgr2=0x7fbca0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:22.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.652+0000 7fbca757d640 1 --2- 172.21.15.62:0/2196488602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0106a80 0x7fbca0106e60 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009930 tx=0x7fbc9c02a420 comp rx=0 tx=0).stop 2024-09-12T00:04:22.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.652+0000 7fbca757d640 1 -- 172.21.15.62:0/2196488602 shutdown_connections 2024-09-12T00:04:22.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.652+0000 7fbca757d640 1 --2- 172.21.15.62:0/2196488602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0101420 0x7fbca010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.652+0000 7fbca757d640 1 --2- 172.21.15.62:0/2196488602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbca0100a80 0x7fbca0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.652+0000 7fbca757d640 1 --2- 172.21.15.62:0/2196488602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0106a80 0x7fbca0106e60 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.652+0000 7fbca757d640 1 -- 172.21.15.62:0/2196488602 >> 172.21.15.62:0/2196488602 conn(0x7fbca00fc7a0 msgr2=0x7fbca00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:22.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.652+0000 7fbca757d640 1 -- 172.21.15.62:0/2196488602 shutdown_connections 2024-09-12T00:04:22.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.652+0000 7fbca757d640 1 -- 172.21.15.62:0/2196488602 wait complete. 2024-09-12T00:04:22.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca757d640 1 Processor -- start 2024-09-12T00:04:22.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca757d640 1 -- start start 2024-09-12T00:04:22.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca757d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0100a80 0x7fbca01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca757d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbca0101420 0x7fbca0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca757d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0106a80 0x7fbca01a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca757d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0110e40 con 0x7fbca0106a80 2024-09-12T00:04:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca757d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0110cc0 con 0x7fbca0100a80 2024-09-12T00:04:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca757d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0110fc0 con 0x7fbca0101420 2024-09-12T00:04:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca5b1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0106a80 0x7fbca01a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca5319640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0100a80 0x7fbca01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca5b1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0106a80 0x7fbca01a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37042/0 (socket says 172.21.15.62:37042) 2024-09-12T00:04:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca5b1a640 1 -- 172.21.15.62:0/597636480 learned_addr learned my addr 172.21.15.62:0/597636480 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.653+0000 7fbca4b18640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbca0101420 0x7fbca0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbca5b1a640 1 -- 172.21.15.62:0/597636480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbca0101420 msgr2=0x7fbca0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbca5b1a640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbca0101420 0x7fbca0199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbca5b1a640 1 -- 172.21.15.62:0/597636480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0100a80 msgr2=0x7fbca01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbca5b1a640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0100a80 0x7fbca01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbca5b1a640 1 -- 172.21.15.62:0/597636480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca01a48b0 con 0x7fbca0106a80 2024-09-12T00:04:22.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbca5319640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0100a80 0x7fbca01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:04:22.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbca5b1a640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0106a80 0x7fbca01a4370 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800eb10 tx=0x7fbc9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:22.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbc927fc640 1 -- 172.21.15.62:0/597636480 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98004b20 con 0x7fbca0106a80 2024-09-12T00:04:22.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbca757d640 1 -- 172.21.15.62:0/597636480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca01a4a80 con 0x7fbca0106a80 2024-09-12T00:04:22.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbc927fc640 1 -- 172.21.15.62:0/597636480 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc98004cc0 con 0x7fbca0106a80 2024-09-12T00:04:22.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbc927fc640 1 -- 172.21.15.62:0/597636480 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98010760 con 0x7fbca0106a80 2024-09-12T00:04:22.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.654+0000 7fbca757d640 1 -- 172.21.15.62:0/597636480 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca01a4f40 con 0x7fbca0106a80 2024-09-12T00:04:22.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.656+0000 7fbc927fc640 1 -- 172.21.15.62:0/597636480 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbc98005020 con 0x7fbca0106a80 2024-09-12T00:04:22.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca9956740 1 Processor -- start 2024-09-12T00:04:22.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca9956740 1 -- start start 2024-09-12T00:04:22.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca9956740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca9956740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2de140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca9956740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca9956740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5635ff2392e0 con 0x5635ff2de680 2024-09-12T00:04:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca9956740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5635ff1e0ce0 con 0x5635ff2d3dc0 2024-09-12T00:04:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca9956740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5635ff2352a0 con 0x5635ff2d34a0 2024-09-12T00:04:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca4b18640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca4b18640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37058/0 (socket says 172.21.15.62:37058) 2024-09-12T00:04:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca4b18640 1 -- 172.21.15.62:0/2680889994 learned_addr learned my addr 172.21.15.62:0/2680889994 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca5319640 1 --2- 172.21.15.62:0/2680889994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2de140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca4b18640 1 --2- 172.21.15.62:0/2680889994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca4b18640 1 -- 172.21.15.62:0/2680889994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 msgr2=0x5635ff2d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:22.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca4b18640 1 --2- 172.21.15.62:0/2680889994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2d3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbc927fc640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc8c0a2cf0 0x7fbc8c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca4b18640 1 -- 172.21.15.62:0/2680889994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 msgr2=0x5635ff2de140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca4b18640 1 --2- 172.21.15.62:0/2680889994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2de140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.657+0000 7fbca4b18640 1 -- 172.21.15.62:0/2680889994 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ff1c5440 con 0x5635ff2de680 2024-09-12T00:04:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.658+0000 7fbca5b1a640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc8c0a2cf0 0x7fbc8c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.658+0000 7fbca4b18640 1 --2- 172.21.15.62:0/2680889994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e0a70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fbc94012540 tx=0x7fbc9400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05bd312f7907e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.658+0000 7fbc927fc640 1 -- 172.21.15.62:0/597636480 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fbc980cf940 con 0x7fbca0106a80 2024-09-12T00:04:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.658+0000 7fbc8b7fe640 1 -- 172.21.15.62:0/2680889994 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc940106b0 con 0x5635ff2de680 2024-09-12T00:04:22.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.658+0000 7fbc8b7fe640 1 -- 172.21.15.62:0/2680889994 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc94010d30 con 0x5635ff2de680 2024-09-12T00:04:22.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.658+0000 7fbc8b7fe640 1 -- 172.21.15.62:0/2680889994 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9401ddf0 con 0x5635ff2de680 2024-09-12T00:04:22.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 -- 172.21.15.62:0/2680889994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 msgr2=0x5635ff2e0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:22.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 --2- 172.21.15.62:0/2680889994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e0a70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fbc94012540 tx=0x7fbc9400b870 comp rx=0 tx=0).stop 2024-09-12T00:04:22.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca5b1a640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc8c0a2cf0 0x7fbc8c0a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc9809bdf0 tx=0x7fbc98086990 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:22.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 -- 172.21.15.62:0/2680889994 shutdown_connections 2024-09-12T00:04:22.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 --2- 172.21.15.62:0/2680889994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e0a70 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 --2- 172.21.15.62:0/2680889994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2de140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 --2- 172.21.15.62:0/2680889994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2d3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 -- 172.21.15.62:0/2680889994 >> 172.21.15.62:0/2680889994 conn(0x5635ff2cfa30 msgr2=0x5635ff2cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:22.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 -- 172.21.15.62:0/2680889994 shutdown_connections 2024-09-12T00:04:22.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 -- 172.21.15.62:0/2680889994 wait complete. 2024-09-12T00:04:22.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 -- start start 2024-09-12T00:04:22.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.659+0000 7fbca9956740 1 Processor -- start 2024-09-12T00:04:22.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca9956740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2de050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca9956740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d3dc0 0x5635ff2db300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca9956740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2de680 0x5635ff2db8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca9956740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5635ff2e1250 con 0x5635ff2d3dc0 2024-09-12T00:04:22.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca9956740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5635ff1f4050 con 0x5635ff2de680 2024-09-12T00:04:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca9956740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5635ff2e13d0 con 0x5635ff2d34a0 2024-09-12T00:04:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca9956740 10 client.0 _ll_register_callbacks cb 0x5635ff234160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca5319640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d3dc0 0x5635ff2db300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca5319640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d3dc0 0x5635ff2db300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37068/0 (socket says 172.21.15.62:37068) 2024-09-12T00:04:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca5319640 1 -- 172.21.15.62:0/1376468565 learned_addr learned my addr 172.21.15.62:0/1376468565 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca4b18640 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2de680 0x5635ff2db8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca4b18640 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca5319640 1 -- 172.21.15.62:0/1376468565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 msgr2=0x5635ff2de050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca5319640 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2de050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca5319640 1 -- 172.21.15.62:0/1376468565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2de680 msgr2=0x5635ff2db8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca5319640 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2de680 0x5635ff2db8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca5319640 1 -- 172.21.15.62:0/1376468565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fefaa3c0 con 0x5635ff2d3dc0 2024-09-12T00:04:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca4b18640 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2de680 0x5635ff2db8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:04:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbca5319640 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d3dc0 0x5635ff2db300 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c02a940 tx=0x7fbc9c031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fc4a937318bd12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:22.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbc88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:22.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.660+0000 7fbc88ff9640 1 -- 172.21.15.62:0/1376468565 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c035040 con 0x5635ff2d3dc0 2024-09-12T00:04:22.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbca9956740 1 -- 172.21.15.62:0/1376468565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc94010d30 con 0x5635ff2d3dc0 2024-09-12T00:04:22.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbc88ff9640 1 -- 172.21.15.62:0/1376468565 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fbc9c004450 con 0x5635ff2d3dc0 2024-09-12T00:04:22.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbc88ff9640 1 -- 172.21.15.62:0/1376468565 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbc9c0035f0 con 0x5635ff2d3dc0 2024-09-12T00:04:22.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbca9956740 10 client.17598 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:22.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbca9956740 1 -- 172.21.15.62:0/1376468565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5635ff2eccd0 con 0x5635ff2d3dc0 2024-09-12T00:04:22.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbc88ff9640 1 -- 172.21.15.62:0/1376468565 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc94010d30 con 0x5635ff2d3dc0 2024-09-12T00:04:22.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbc88ff9640 1 -- 172.21.15.62:0/1376468565 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fbc9c051950 con 0x5635ff2d3dc0 2024-09-12T00:04:22.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbca9956740 10 client.17598 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:22.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbca9956740 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:04:22.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbca9956740 20 client.17598 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:22.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbca9956740 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5635ff2ee0e0 0x5635ff2f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbca9956740 4 client.17598 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:04:22.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbca9956740 1 -- 172.21.15.62:0/1376468565 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635ff246850 con 0x5635ff2ee0e0 2024-09-12T00:04:22.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.661+0000 7fbca4b18640 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5635ff2ee0e0 0x5635ff2f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.662+0000 7fbca4b18640 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5635ff2ee0e0 0x5635ff2f0500 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5c523b9cc63f1a server_cookie=583aec8e2816adcb in_seq=0 out_seq=0 2024-09-12T00:04:22.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.662+0000 7fbc88ff9640 10 client.17598 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:22.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.662+0000 7fbc88ff9640 1 -- 172.21.15.62:0/1376468565 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635ff246850 con 0x5635ff2ee0e0 2024-09-12T00:04:22.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.662+0000 7fbc88ff9640 10 client.17598 handle_command_reply: tid=0 2024-09-12T00:04:22.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.662+0000 7fbca9956740 1 client.17598 shutdown 2024-09-12T00:04:22.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.662+0000 7fbca9956740 1 -- 172.21.15.62:0/1376468565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d3dc0 msgr2=0x5635ff2db300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:22.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.662+0000 7fbca9956740 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d3dc0 0x5635ff2db300 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c02a940 tx=0x7fbc9c031920 comp rx=0 tx=0).stop 2024-09-12T00:04:22.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.663+0000 7fbca9956740 1 -- 172.21.15.62:0/1376468565 shutdown_connections 2024-09-12T00:04:22.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.663+0000 7fbca9956740 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5635ff2ee0e0 0x5635ff2f0500 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.663+0000 7fbca9956740 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2de680 0x5635ff2db8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.663+0000 7fbca9956740 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d3dc0 0x5635ff2db300 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.663+0000 7fbca9956740 1 --2- 172.21.15.62:0/1376468565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2de050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.663+0000 7fbca9956740 1 -- 172.21.15.62:0/1376468565 >> 172.21.15.62:0/1376468565 conn(0x5635ff2cfa30 msgr2=0x5635ff2d5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:22.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.663+0000 7fbc88ff9640 0 client.17598 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:22.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.663+0000 7fbca9956740 1 -- 172.21.15.62:0/1376468565 shutdown_connections 2024-09-12T00:04:22.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.663+0000 7fbca9956740 1 -- 172.21.15.62:0/1376468565 wait complete. 2024-09-12T00:04:22.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.663+0000 7fbca9956740 20 client.17598 trim_cache size 0 max 16384 2024-09-12T00:04:22.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.680+0000 7fbca9956740 1 Processor -- start 2024-09-12T00:04:22.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.680+0000 7fbca9956740 1 -- start start 2024-09-12T00:04:22.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca9956740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d34a0 0x5635ff2ea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca9956740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2dd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca9956740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2de680 0x5635ff2f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca9956740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01b2430 con 0x5635ff2d34a0 2024-09-12T00:04:22.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca9956740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01afca0 con 0x5635ff2d3dc0 2024-09-12T00:04:22.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca9956740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01adb10 con 0x5635ff2de680 2024-09-12T00:04:22.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca4b18640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d34a0 0x5635ff2ea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca4b18640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d34a0 0x5635ff2ea4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37078/0 (socket says 172.21.15.62:37078) 2024-09-12T00:04:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca4b18640 1 -- 172.21.15.62:0/4229867468 learned_addr learned my addr 172.21.15.62:0/4229867468 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca4b18640 1 -- 172.21.15.62:0/4229867468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2de680 msgr2=0x5635ff2f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca4b18640 1 --2- 172.21.15.62:0/4229867468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2de680 0x5635ff2f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca5319640 1 --2- 172.21.15.62:0/4229867468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca4b18640 1 -- 172.21.15.62:0/4229867468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 msgr2=0x5635ff2dd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca4b18640 1 --2- 172.21.15.62:0/4229867468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2dd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca4b18640 1 -- 172.21.15.62:0/4229867468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ff2e0da0 con 0x5635ff2d34a0 2024-09-12T00:04:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca5319640 1 --2- 172.21.15.62:0/4229867468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2dd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:04:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbca4b18640 1 --2- 172.21.15.62:0/4229867468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d34a0 0x5635ff2ea4b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400b6b0 tx=0x7fbc9400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77467d8f8ce52a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.681+0000 7fbc8affd640 1 -- 172.21.15.62:0/4229867468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94023040 con 0x5635ff2d34a0 2024-09-12T00:04:22.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbc8affd640 1 -- 172.21.15.62:0/4229867468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9402e020 con 0x5635ff2d34a0 2024-09-12T00:04:22.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbc8affd640 1 -- 172.21.15.62:0/4229867468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94005520 con 0x5635ff2d34a0 2024-09-12T00:04:22.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbca9956740 1 -- 172.21.15.62:0/4229867468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d34a0 msgr2=0x5635ff2ea4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:22.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbca9956740 1 --2- 172.21.15.62:0/4229867468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d34a0 0x5635ff2ea4b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400b6b0 tx=0x7fbc9400b870 comp rx=0 tx=0).stop 2024-09-12T00:04:22.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbca9956740 1 -- 172.21.15.62:0/4229867468 shutdown_connections 2024-09-12T00:04:22.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbca9956740 1 --2- 172.21.15.62:0/4229867468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2de680 0x5635ff2f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbca9956740 1 --2- 172.21.15.62:0/4229867468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2dd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbca9956740 1 --2- 172.21.15.62:0/4229867468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2d34a0 0x5635ff2ea4b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbca9956740 1 -- 172.21.15.62:0/4229867468 >> 172.21.15.62:0/4229867468 conn(0x5635ff2cfa30 msgr2=0x5635ff2dcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:22.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbca9956740 1 -- 172.21.15.62:0/4229867468 shutdown_connections 2024-09-12T00:04:22.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.682+0000 7fbca9956740 1 -- 172.21.15.62:0/4229867468 wait complete. 2024-09-12T00:04:22.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca9956740 1 -- start start 2024-09-12T00:04:22.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca9956740 1 Processor -- start 2024-09-12T00:04:22.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca9956740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2e6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca9956740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca9956740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca9956740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01ae160 con 0x5635ff2de680 2024-09-12T00:04:22.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca9956740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01af5d0 con 0x5635ff2d3dc0 2024-09-12T00:04:22.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca9956740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01ae830 con 0x5635ff2d34a0 2024-09-12T00:04:22.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca9956740 10 client.0 _ll_register_callbacks cb 0x7fbc9c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:22.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca4b18640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca4b18640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37090/0 (socket says 172.21.15.62:37090) 2024-09-12T00:04:22.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca4b18640 1 -- 172.21.15.62:0/4151441384 learned_addr learned my addr 172.21.15.62:0/4151441384 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:22.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.683+0000 7fbca5319640 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbca4b18640 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2e6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbca4b18640 1 -- 172.21.15.62:0/4151441384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 msgr2=0x5635ff2e6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:22.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbca4b18640 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2e6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbca4b18640 1 -- 172.21.15.62:0/4151441384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 msgr2=0x5635ff2e6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:22.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbca4b18640 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2e6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbca4b18640 1 -- 172.21.15.62:0/4151441384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ff2e3cb0 con 0x5635ff2de680 2024-09-12T00:04:22.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbca4b18640 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e35f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fbc94012540 tx=0x7fbc9400bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70750552e38ebe97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:22.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbc88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:22.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbc88ff9640 1 -- 172.21.15.62:0/4151441384 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94005450 con 0x5635ff2de680 2024-09-12T00:04:22.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbc88ff9640 1 -- 172.21.15.62:0/4151441384 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fbc9402f040 con 0x5635ff2de680 2024-09-12T00:04:22.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbca9956740 1 -- 172.21.15.62:0/4151441384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9402e020 con 0x5635ff2de680 2024-09-12T00:04:22.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbc88ff9640 1 -- 172.21.15.62:0/4151441384 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbc9401da00 con 0x5635ff2de680 2024-09-12T00:04:22.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbca9956740 10 client.17604 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:22.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.684+0000 7fbca9956740 1 -- 172.21.15.62:0/4151441384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5635ff3596f0 con 0x5635ff2de680 2024-09-12T00:04:22.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.685+0000 7fbc88ff9640 1 -- 172.21.15.62:0/4151441384 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9402e020 con 0x5635ff2de680 2024-09-12T00:04:22.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.685+0000 7fbc88ff9640 1 -- 172.21.15.62:0/4151441384 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fbc9401c070 con 0x5635ff2de680 2024-09-12T00:04:22.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.685+0000 7fbca9956740 10 client.17604 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:22.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.685+0000 7fbca9956740 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:04:22.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.685+0000 7fbca9956740 20 client.17604 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:22.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.685+0000 7fbca9956740 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5635ff359b10 0x5635ff35bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:22.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.685+0000 7fbca9956740 4 client.17604 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:04:22.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.685+0000 7fbca9956740 1 -- 172.21.15.62:0/4151441384 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635ff240d60 con 0x5635ff359b10 2024-09-12T00:04:22.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.685+0000 7fbca5319640 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5635ff359b10 0x5635ff35bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:22.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca5319640 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5635ff359b10 0x5635ff35bf30 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1ed0c51bbdc710 server_cookie=85b17f95f99af80a in_seq=0 out_seq=0 2024-09-12T00:04:22.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbc88ff9640 10 client.17604 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:22.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbc88ff9640 1 -- 172.21.15.62:0/4151441384 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5635ff240d60 con 0x5635ff359b10 2024-09-12T00:04:22.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbc88ff9640 10 client.17604 handle_command_reply: tid=0 2024-09-12T00:04:22.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 client.17604 shutdown 2024-09-12T00:04:22.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 -- 172.21.15.62:0/4151441384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 msgr2=0x5635ff2e35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:22.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e35f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fbc94012540 tx=0x7fbc9400bfb0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 -- 172.21.15.62:0/4151441384 shutdown_connections 2024-09-12T00:04:22.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5635ff359b10 0x5635ff35bf30 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635ff2de680 0x5635ff2e35f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635ff2d3dc0 0x5635ff2e6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 --2- 172.21.15.62:0/4151441384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635ff2d34a0 0x5635ff2e6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 -- 172.21.15.62:0/4151441384 >> 172.21.15.62:0/4151441384 conn(0x5635ff2cfa30 msgr2=0x5635ff2ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbc88ff9640 0 client.17604 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 -- 172.21.15.62:0/4151441384 shutdown_connections 2024-09-12T00:04:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 1 -- 172.21.15.62:0/4151441384 wait complete. 2024-09-12T00:04:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.686+0000 7fbca9956740 20 client.17604 trim_cache size 0 max 16384 2024-09-12T00:04:22.719 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:04:22.720 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:04:22.720 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-12T00:04:22.720 INFO:teuthology.orchestra.run.smithi062.stdout: "90617977-6c53-47b9-a8d9-018bee329a6b": { 2024-09-12T00:04:22.720 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-12T00:04:22.720 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "90617977-6c53-47b9-a8d9-018bee329a6b", 2024-09-12T00:04:22.720 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-12T00:04:22.720 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:22.720 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:22.720 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:04:22.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 -- 172.21.15.62:0/597636480 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc8c0a2cf0 msgr2=0x7fbc8c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:22.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc8c0a2cf0 0x7fbc8c0a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc9809bdf0 tx=0x7fbc98086990 comp rx=0 tx=0).stop 2024-09-12T00:04:22.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 -- 172.21.15.62:0/597636480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0106a80 msgr2=0x7fbca01a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:22.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0106a80 0x7fbca01a4370 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800eb10 tx=0x7fbc9800c2d0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 -- 172.21.15.62:0/597636480 shutdown_connections 2024-09-12T00:04:22.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbc8c0a2cf0 0x7fbc8c0a51b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0106a80 0x7fbca01a4370 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbca0101420 0x7fbca0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 --2- 172.21.15.62:0/597636480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0100a80 0x7fbca01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:22.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 -- 172.21.15.62:0/597636480 >> 172.21.15.62:0/597636480 conn(0x7fbca00fc7a0 msgr2=0x7fbca00fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:22.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 -- 172.21.15.62:0/597636480 shutdown_connections 2024-09-12T00:04:22.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:22.687+0000 7fbca757d640 1 -- 172.21.15.62:0/597636480 wait complete. 2024-09-12T00:04:22.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-12T00:04:22.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:22.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:04:22.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:04:22.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:22.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:04:22.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:04:22.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:22.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:22.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:22.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:22 smithi062 ceph-mon[30958]: pgmap v383: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-12T00:04:23.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-12T00:04:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:04:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:04:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:04:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:04:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:23.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:22 smithi120 ceph-mon[35182]: pgmap v383: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-12T00:04:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-12T00:04:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:04:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:04:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:04:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:04:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:23.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:04:23.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:22 smithi083 ceph-mon[34805]: pgmap v383: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-12T00:04:23.499 INFO:tasks.cephfs.filesystem:scrub status for tag:90617977-6c53-47b9-a8d9-018bee329a6b - {'path': '/', 'tag': '90617977-6c53-47b9-a8d9-018bee329a6b', 'options': 'recursive,force'} 2024-09-12T00:04:25.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:24 smithi120 ceph-mon[35182]: pgmap v384: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-12T00:04:25.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:24 smithi083 ceph-mon[34805]: pgmap v384: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-12T00:04:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:24 smithi062 ceph-mon[30958]: pgmap v384: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-12T00:04:27.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:26 smithi120 ceph-mon[35182]: pgmap v385: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-12T00:04:27.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:26 smithi083 ceph-mon[34805]: pgmap v385: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-12T00:04:27.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:26 smithi062 ceph-mon[30958]: pgmap v385: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-12T00:04:27.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 705.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:04:28.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:27 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:04:28.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:27 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:27 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:27 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:27 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:27 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:04:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:27 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:27 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:27 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:27 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:27 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:04:28.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:27 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:27 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:27 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:28.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:27 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:29.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:28 smithi120 ceph-mon[35182]: pgmap v386: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.9 MiB/s wr, 450 op/s 2024-09-12T00:04:29.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:28 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:04:29.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:28 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:04:29.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:28 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:29.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:28 smithi083 ceph-mon[34805]: pgmap v386: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.9 MiB/s wr, 450 op/s 2024-09-12T00:04:29.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:28 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:04:29.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:28 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:04:29.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:28 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:28 smithi062 ceph-mon[30958]: pgmap v386: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.9 MiB/s wr, 450 op/s 2024-09-12T00:04:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:28 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:04:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:28 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:04:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:28 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:04:31.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:30 smithi120 ceph-mon[35182]: pgmap v387: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-09-12T00:04:31.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:30 smithi083 ceph-mon[34805]: pgmap v387: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-09-12T00:04:31.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:30 smithi062 ceph-mon[30958]: pgmap v387: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-09-12T00:04:32.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:31 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:04:32.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:31 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:04:32.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:31 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:04:32.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:31 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:04:32.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:31 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:04:32.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:31 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:04:32.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:31 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:04:32.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:31 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:04:32.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:31 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:04:32.452 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 710.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:04:33.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:32 smithi120 ceph-mon[35182]: pgmap v388: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-09-12T00:04:33.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:32 smithi083 ceph-mon[34805]: pgmap v388: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-09-12T00:04:33.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:32 smithi062 ceph-mon[30958]: pgmap v388: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-09-12T00:04:35.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:34 smithi120 ceph-mon[35182]: pgmap v389: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.8 MiB/s wr, 812 op/s 2024-09-12T00:04:35.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:34 smithi083 ceph-mon[34805]: pgmap v389: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.8 MiB/s wr, 812 op/s 2024-09-12T00:04:35.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:34 smithi062 ceph-mon[30958]: pgmap v389: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.8 MiB/s wr, 812 op/s 2024-09-12T00:04:37.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:36 smithi120 ceph-mon[35182]: pgmap v390: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.4 MiB/s wr, 752 op/s 2024-09-12T00:04:37.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:36 smithi083 ceph-mon[34805]: pgmap v390: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.4 MiB/s wr, 752 op/s 2024-09-12T00:04:37.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:36 smithi062 ceph-mon[30958]: pgmap v390: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.4 MiB/s wr, 752 op/s 2024-09-12T00:04:37.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 715.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:04:38.805 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:04:39.077 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:39.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:38 smithi120 ceph-mon[35182]: pgmap v391: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-09-12T00:04:39.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:38 smithi083 ceph-mon[34805]: pgmap v391: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-09-12T00:04:39.198 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:38 smithi062 ceph-mon[30958]: pgmap v391: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-09-12T00:04:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.520+0000 7ff40a0bd640 1 -- 172.21.15.62:0/3928769750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff404102570 msgr2=0x7ff4041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.520+0000 7ff40a0bd640 1 --2- 172.21.15.62:0/3928769750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff404102570 0x7ff4041029b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009900 tx=0x7ff3f802a480 comp rx=0 tx=0).stop 2024-09-12T00:04:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.520+0000 7ff40a0bd640 1 -- 172.21.15.62:0/3928769750 shutdown_connections 2024-09-12T00:04:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.520+0000 7ff40a0bd640 1 --2- 172.21.15.62:0/3928769750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff404102ef0 0x7ff40410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.520+0000 7ff40a0bd640 1 --2- 172.21.15.62:0/3928769750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff404102570 0x7ff4041029b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.520+0000 7ff40a0bd640 1 --2- 172.21.15.62:0/3928769750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404100e20 0x7ff404101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.520+0000 7ff40a0bd640 1 -- 172.21.15.62:0/3928769750 >> 172.21.15.62:0/3928769750 conn(0x7ff4040fc7d0 msgr2=0x7ff4040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.520+0000 7ff40a0bd640 1 -- 172.21.15.62:0/3928769750 shutdown_connections 2024-09-12T00:04:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.520+0000 7ff40a0bd640 1 -- 172.21.15.62:0/3928769750 wait complete. 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.520+0000 7ff40a0bd640 1 Processor -- start 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff40a0bd640 1 -- start start 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff40a0bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff404100e20 0x7ff404199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff40a0bd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff404102570 0x7ff404199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff40a0bd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404102ef0 0x7ff4041a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff40a0bd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff404110f80 con 0x7ff404102570 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff40a0bd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff404110e00 con 0x7ff404102ef0 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff40a0bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff404111100 con 0x7ff404100e20 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff402ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff404102570 0x7ff404199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff402ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff404102570 0x7ff404199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60144/0 (socket says 172.21.15.62:60144) 2024-09-12T00:04:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff402ffd640 1 -- 172.21.15.62:0/1576269246 learned_addr learned my addr 172.21.15.62:0/1576269246 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff4037fe640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff404100e20 0x7ff404199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff403fff640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404102ef0 0x7ff4041a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff402ffd640 1 -- 172.21.15.62:0/1576269246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff404100e20 msgr2=0x7ff404199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff402ffd640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff404100e20 0x7ff404199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff402ffd640 1 -- 172.21.15.62:0/1576269246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404102ef0 msgr2=0x7ff4041a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff402ffd640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404102ef0 0x7ff4041a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.521+0000 7ff402ffd640 1 -- 172.21.15.62:0/1576269246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4041a48d0 con 0x7ff404102570 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.522+0000 7ff402ffd640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff404102570 0x7ff404199850 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8006010 tx=0x7ff3f8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.522+0000 7ff400ff9640 1 -- 172.21.15.62:0/1576269246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f803f070 con 0x7ff404102570 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.522+0000 7ff40a0bd640 1 -- 172.21.15.62:0/1576269246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4041a4aa0 con 0x7ff404102570 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.522+0000 7ff400ff9640 1 -- 172.21.15.62:0/1576269246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f8031c30 con 0x7ff404102570 2024-09-12T00:04:39.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.522+0000 7ff400ff9640 1 -- 172.21.15.62:0/1576269246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8035030 con 0x7ff404102570 2024-09-12T00:04:39.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.522+0000 7ff40a0bd640 1 -- 172.21.15.62:0/1576269246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4041a4f80 con 0x7ff404102570 2024-09-12T00:04:39.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.523+0000 7ff40a0bd640 1 -- 172.21.15.62:0/1576269246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff404101200 con 0x7ff404102570 2024-09-12T00:04:39.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.524+0000 7ff400ff9640 1 -- 172.21.15.62:0/1576269246 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff3f8042070 con 0x7ff404102570 2024-09-12T00:04:39.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.525+0000 7ff400ff9640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3ec0a2cf0 0x7ff3ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:39.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.525+0000 7ff4037fe640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3ec0a2cf0 0x7ff3ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:39.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.525+0000 7ff400ff9640 1 -- 172.21.15.62:0/1576269246 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7ff3f80f0060 con 0x7ff404102570 2024-09-12T00:04:39.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.525+0000 7ff4037fe640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3ec0a2cf0 0x7ff3ec0a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400bbe0 tx=0x7ff3f400b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:39.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.528+0000 7ff400ff9640 1 -- 172.21.15.62:0/1576269246 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3f8031de0 con 0x7ff404102570 2024-09-12T00:04:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.742+0000 7ff40a0bd640 1 -- 172.21.15.62:0/1576269246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff404109090 con 0x7ff404102570 2024-09-12T00:04:39.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.743+0000 7ff400ff9640 1 -- 172.21.15.62:0/1576269246 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11913 (secure 0 0 0) 0x7ff3f80bc630 con 0x7ff404102570 2024-09-12T00:04:39.742 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:04:39.743 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":36,"btime":"2024-09-12T00:03:36:303242+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:03:35.719292+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:04:39.743 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 36 2024-09-12T00:04:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 -- 172.21.15.62:0/1576269246 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3ec0a2cf0 msgr2=0x7ff3ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3ec0a2cf0 0x7ff3ec0a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400bbe0 tx=0x7ff3f400b390 comp rx=0 tx=0).stop 2024-09-12T00:04:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 -- 172.21.15.62:0/1576269246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff404102570 msgr2=0x7ff404199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff404102570 0x7ff404199850 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8006010 tx=0x7ff3f8003c10 comp rx=0 tx=0).stop 2024-09-12T00:04:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 -- 172.21.15.62:0/1576269246 shutdown_connections 2024-09-12T00:04:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff3ec0a2cf0 0x7ff3ec0a51b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404102ef0 0x7ff4041a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff404102570 0x7ff404199850 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:39.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 --2- 172.21.15.62:0/1576269246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff404100e20 0x7ff404199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:39.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 -- 172.21.15.62:0/1576269246 >> 172.21.15.62:0/1576269246 conn(0x7ff4040fc7d0 msgr2=0x7ff4040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:39.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 -- 172.21.15.62:0/1576269246 shutdown_connections 2024-09-12T00:04:39.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:39.745+0000 7ff40a0bd640 1 -- 172.21.15.62:0/1576269246 wait complete. 2024-09-12T00:04:40.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:39 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1576269246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:40.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:39 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1576269246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:40.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:39 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1576269246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:40.482 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 51961a7 --release --await' 2024-09-12T00:04:40.482 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 51961a7 --release --await 2024-09-12T00:04:40.739 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:41.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:40 smithi062 ceph-mon[30958]: pgmap v392: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 959 op/s 2024-09-12T00:04:41.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:40 smithi120 ceph-mon[35182]: pgmap v392: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 959 op/s 2024-09-12T00:04:41.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:40 smithi083 ceph-mon[34805]: pgmap v392: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 959 op/s 2024-09-12T00:04:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 -- 172.21.15.62:0/3731667158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd464109120 msgr2=0x7fd464110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 --2- 172.21.15.62:0/3731667158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd464109120 0x7fd464110ca0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd454009ca0 tx=0x7fd45402a8c0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 -- 172.21.15.62:0/3731667158 shutdown_connections 2024-09-12T00:04:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 --2- 172.21.15.62:0/3731667158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd464109120 0x7fd464110ca0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 --2- 172.21.15.62:0/3731667158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4641049a0 0x7fd4641089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 --2- 172.21.15.62:0/3731667158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464103ff0 0x7fd4641043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 -- 172.21.15.62:0/3731667158 >> 172.21.15.62:0/3731667158 conn(0x7fd4640ffa90 msgr2=0x7fd464101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 -- 172.21.15.62:0/3731667158 shutdown_connections 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 -- 172.21.15.62:0/3731667158 wait complete. 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 Processor -- start 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.188+0000 7fd46b5ad640 1 -- start start 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd46b5ad640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd464103ff0 0x7fd46419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd46b5ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4641049a0 0x7fd46419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd46b5ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464109120 0x7fd46419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd46b5ad640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd464114150 con 0x7fd464103ff0 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd46b5ad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd464113fd0 con 0x7fd464109120 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd46b5ad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641142d0 con 0x7fd4641049a0 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd469349640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd464103ff0 0x7fd46419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd469349640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd464103ff0 0x7fd46419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41092/0 (socket says 172.21.15.62:41092) 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd469349640 1 -- 172.21.15.62:0/2386024164 learned_addr learned my addr 172.21.15.62:0/2386024164 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:41.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd468b48640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4641049a0 0x7fd46419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd469b4a640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464109120 0x7fd46419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd469349640 1 -- 172.21.15.62:0/2386024164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4641049a0 msgr2=0x7fd46419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd469349640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4641049a0 0x7fd46419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd469349640 1 -- 172.21.15.62:0/2386024164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464109120 msgr2=0x7fd46419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd469349640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464109120 0x7fd46419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd469349640 1 -- 172.21.15.62:0/2386024164 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46419f5a0 con 0x7fd464103ff0 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd468b48640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4641049a0 0x7fd46419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.189+0000 7fd469349640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd464103ff0 0x7fd46419a5f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00c890 tx=0x7fd45c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.190+0000 7fd4627fc640 1 -- 172.21.15.62:0/2386024164 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c004220 con 0x7fd464103ff0 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.190+0000 7fd46b5ad640 1 -- 172.21.15.62:0/2386024164 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46419f890 con 0x7fd464103ff0 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.190+0000 7fd4627fc640 1 -- 172.21.15.62:0/2386024164 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd45c0043c0 con 0x7fd464103ff0 2024-09-12T00:04:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.190+0000 7fd4627fc640 1 -- 172.21.15.62:0/2386024164 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0049e0 con 0x7fd464103ff0 2024-09-12T00:04:41.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.190+0000 7fd46b5ad640 1 -- 172.21.15.62:0/2386024164 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4641a7170 con 0x7fd464103ff0 2024-09-12T00:04:41.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.192+0000 7fd4627fc640 1 -- 172.21.15.62:0/2386024164 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd45c0079b0 con 0x7fd464103ff0 2024-09-12T00:04:41.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.192+0000 7fd46d986740 1 Processor -- start 2024-09-12T00:04:41.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.192+0000 7fd46d986740 1 -- start start 2024-09-12T00:04:41.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd46d986740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f57440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd46d986740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f61d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd46d986740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f62240 0x55fdd9f64630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd46d986740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9ebc620 con 0x55fdd9f57060 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd46d986740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9e63e50 con 0x55fdd9f62240 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd46d986740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9eb85e0 con 0x55fdd9f57980 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469b4a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f57440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469b4a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f57440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41098/0 (socket says 172.21.15.62:41098) 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469b4a640 1 -- 172.21.15.62:0/1163221848 learned_addr learned my addr 172.21.15.62:0/1163221848 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469349640 1 --2- 172.21.15.62:0/1163221848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f61d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd468b48640 1 --2- 172.21.15.62:0/1163221848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f62240 0x55fdd9f64630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469b4a640 1 -- 172.21.15.62:0/1163221848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 msgr2=0x55fdd9f61d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469b4a640 1 --2- 172.21.15.62:0/1163221848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f61d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469b4a640 1 -- 172.21.15.62:0/1163221848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f62240 msgr2=0x55fdd9f64630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469b4a640 1 --2- 172.21.15.62:0/1163221848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f62240 0x55fdd9f64630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd4627fc640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd44c0a2cf0 0x7fd44c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469b4a640 1 -- 172.21.15.62:0/1163221848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd9e54d40 con 0x55fdd9f57060 2024-09-12T00:04:41.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd468b48640 1 --2- 172.21.15.62:0/1163221848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f62240 0x55fdd9f64630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:04:41.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd4627fc640 1 -- 172.21.15.62:0/2386024164 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fd45c0cf560 con 0x7fd464103ff0 2024-09-12T00:04:41.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469b4a640 1 --2- 172.21.15.62:0/1163221848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f57440 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd454005cc0 tx=0x7fd454005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751fd7f0a6b63d67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:41.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.193+0000 7fd469349640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd44c0a2cf0 0x7fd44c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.194+0000 7fd44b7fe640 1 -- 172.21.15.62:0/1163221848 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454031db0 con 0x55fdd9f57060 2024-09-12T00:04:41.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.194+0000 7fd44b7fe640 1 -- 172.21.15.62:0/1163221848 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd454036070 con 0x55fdd9f57060 2024-09-12T00:04:41.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.194+0000 7fd44b7fe640 1 -- 172.21.15.62:0/1163221848 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4540042a0 con 0x55fdd9f57060 2024-09-12T00:04:41.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.194+0000 7fd46d986740 1 -- 172.21.15.62:0/1163221848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 msgr2=0x55fdd9f57440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:41.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.194+0000 7fd46d986740 1 --2- 172.21.15.62:0/1163221848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f57440 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd454005cc0 tx=0x7fd454005f70 comp rx=0 tx=0).stop 2024-09-12T00:04:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.194+0000 7fd469349640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd44c0a2cf0 0x7fd44c0a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0ccd60 tx=0x7fd45c005300 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 -- 172.21.15.62:0/1163221848 shutdown_connections 2024-09-12T00:04:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 --2- 172.21.15.62:0/1163221848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f62240 0x55fdd9f64630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 --2- 172.21.15.62:0/1163221848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f61d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 --2- 172.21.15.62:0/1163221848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f57440 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 -- 172.21.15.62:0/1163221848 >> 172.21.15.62:0/1163221848 conn(0x55fdd9f535f0 msgr2=0x55fdd9f539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 -- 172.21.15.62:0/1163221848 shutdown_connections 2024-09-12T00:04:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 -- 172.21.15.62:0/1163221848 wait complete. 2024-09-12T00:04:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 -- start start 2024-09-12T00:04:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 Processor -- start 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57060 0x55fdd9f5f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f5f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.195+0000 7fd46d986740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f62240 0x55fdd9f5fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd46d986740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9f64e10 con 0x55fdd9f62240 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd46d986740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9e44f10 con 0x55fdd9f57060 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd46d986740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9f64f90 con 0x55fdd9f57980 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd46d986740 10 client.0 _ll_register_callbacks cb 0x55fdd9e68db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd469b4a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f62240 0x55fdd9f5fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd469b4a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f62240 0x55fdd9f5fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41104/0 (socket says 172.21.15.62:41104) 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd469b4a640 1 -- 172.21.15.62:0/2986943054 learned_addr learned my addr 172.21.15.62:0/2986943054 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd468b48640 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f5f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd469b4a640 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57060 0x55fdd9f5f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd469b4a640 1 -- 172.21.15.62:0/2986943054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 msgr2=0x55fdd9f5f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:41.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd469b4a640 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f5f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd469b4a640 1 -- 172.21.15.62:0/2986943054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57060 msgr2=0x55fdd9f5f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:41.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd469b4a640 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57060 0x55fdd9f5f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd469b4a640 1 -- 172.21.15.62:0/2986943054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdd9bcb580 con 0x55fdd9f62240 2024-09-12T00:04:41.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd468b48640 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f5f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:04:41.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd469b4a640 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f62240 0x55fdd9f5fb50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd454009620 tx=0x7fd454031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d33b7d1a8056160 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd448ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.196+0000 7fd448ff9640 1 -- 172.21.15.62:0/2986943054 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454049040 con 0x55fdd9f62240 2024-09-12T00:04:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd448ff9640 1 -- 172.21.15.62:0/2986943054 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fd454003b70 con 0x55fdd9f62240 2024-09-12T00:04:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd46d986740 1 -- 172.21.15.62:0/2986943054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd454036070 con 0x55fdd9f62240 2024-09-12T00:04:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd448ff9640 1 -- 172.21.15.62:0/2986943054 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd454003d70 con 0x55fdd9f62240 2024-09-12T00:04:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd46d986740 10 client.17622 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd46d986740 1 -- 172.21.15.62:0/2986943054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fdd9c95960 con 0x55fdd9f62240 2024-09-12T00:04:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd448ff9640 1 -- 172.21.15.62:0/2986943054 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd454036070 con 0x55fdd9f62240 2024-09-12T00:04:41.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd448ff9640 1 -- 172.21.15.62:0/2986943054 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fd45404f070 con 0x55fdd9f62240 2024-09-12T00:04:41.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd46d986740 10 client.17622 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:41.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd46d986740 10 client.17622 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:04:41.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd46d986740 20 client.17622 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:41.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd46d986740 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fdd9f71bd0 0x55fdd9f73fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd46d986740 4 client.17622 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:04:41.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.197+0000 7fd46d986740 1 -- 172.21.15.62:0/2986943054 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdd9ec9b90 con 0x55fdd9f71bd0 2024-09-12T00:04:41.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.198+0000 7fd468b48640 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fdd9f71bd0 0x55fdd9f73fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.198+0000 7fd468b48640 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fdd9f71bd0 0x55fdd9f73fc0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4045ddfed4ccaf97 server_cookie=c0efc318f62744f9 in_seq=0 out_seq=0 2024-09-12T00:04:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.198+0000 7fd448ff9640 10 client.17622 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd448ff9640 1 -- 172.21.15.62:0/2986943054 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdd9ec9b90 con 0x55fdd9f71bd0 2024-09-12T00:04:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd448ff9640 10 client.17622 handle_command_reply: tid=0 2024-09-12T00:04:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 client.17622 shutdown 2024-09-12T00:04:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 -- 172.21.15.62:0/2986943054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f62240 msgr2=0x55fdd9f5fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f62240 0x55fdd9f5fb50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd454009620 tx=0x7fd454031880 comp rx=0 tx=0).stop 2024-09-12T00:04:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 -- 172.21.15.62:0/2986943054 shutdown_connections 2024-09-12T00:04:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fdd9f71bd0 0x55fdd9f73fc0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f62240 0x55fdd9f5fb50 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f5f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd448ff9640 0 client.17622 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 --2- 172.21.15.62:0/2986943054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57060 0x55fdd9f5f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 -- 172.21.15.62:0/2986943054 >> 172.21.15.62:0/2986943054 conn(0x55fdd9f535f0 msgr2=0x55fdd9f592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 -- 172.21.15.62:0/2986943054 shutdown_connections 2024-09-12T00:04:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 1 -- 172.21.15.62:0/2986943054 wait complete. 2024-09-12T00:04:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.199+0000 7fd46d986740 20 client.17622 trim_cache size 0 max 16384 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd46d986740 1 Processor -- start 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd46d986740 1 -- start start 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd46d986740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f535f0 0x55fdd9f5a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd46d986740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57060 0x55fdd9f6d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd46d986740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f5c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd46d986740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641b23d0 con 0x55fdd9f535f0 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd46d986740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641afc40 con 0x55fdd9f57060 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd46d986740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641adab0 con 0x55fdd9f57980 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd469b4a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f535f0 0x55fdd9f5a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd469b4a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f535f0 0x55fdd9f5a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41110/0 (socket says 172.21.15.62:41110) 2024-09-12T00:04:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.216+0000 7fd469b4a640 1 -- 172.21.15.62:0/2447773402 learned_addr learned my addr 172.21.15.62:0/2447773402 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd468b48640 1 --2- 172.21.15.62:0/2447773402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57060 0x55fdd9f6d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd469b4a640 1 --2- 172.21.15.62:0/2447773402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f5c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd469b4a640 1 -- 172.21.15.62:0/2447773402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 msgr2=0x55fdd9f5c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd469b4a640 1 --2- 172.21.15.62:0/2447773402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f5c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd469b4a640 1 -- 172.21.15.62:0/2447773402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57060 msgr2=0x55fdd9f6d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd469b4a640 1 --2- 172.21.15.62:0/2447773402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57060 0x55fdd9f6d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd469b4a640 1 -- 172.21.15.62:0/2447773402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd9f61010 con 0x55fdd9f535f0 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd469b4a640 1 --2- 172.21.15.62:0/2447773402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f535f0 0x55fdd9f5a3e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd454031b50 tx=0x7fd45402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35797a0c9ec4ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd44affd640 1 -- 172.21.15.62:0/2447773402 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45405f070 con 0x55fdd9f535f0 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd44affd640 1 -- 172.21.15.62:0/2447773402 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd454036070 con 0x55fdd9f535f0 2024-09-12T00:04:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.217+0000 7fd44affd640 1 -- 172.21.15.62:0/2447773402 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454003970 con 0x55fdd9f535f0 2024-09-12T00:04:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 -- 172.21.15.62:0/2447773402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f535f0 msgr2=0x55fdd9f5a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 --2- 172.21.15.62:0/2447773402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f535f0 0x55fdd9f5a3e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd454031b50 tx=0x7fd45402bfd0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 -- 172.21.15.62:0/2447773402 shutdown_connections 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 --2- 172.21.15.62:0/2447773402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f57980 0x55fdd9f5c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 --2- 172.21.15.62:0/2447773402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57060 0x55fdd9f6d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 --2- 172.21.15.62:0/2447773402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f535f0 0x55fdd9f5a3e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 -- 172.21.15.62:0/2447773402 >> 172.21.15.62:0/2447773402 conn(0x55fdd9f5aac0 msgr2=0x55fdd9f6d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 -- 172.21.15.62:0/2447773402 shutdown_connections 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 -- 172.21.15.62:0/2447773402 wait complete. 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 -- start start 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.218+0000 7fd46d986740 1 Processor -- start 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd46d986740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f535f0 0x55fdd9f75140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd46d986740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f75680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd46d986740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57980 0x55fdd9fdb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd46d986740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641ae100 con 0x55fdd9f57060 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd46d986740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641af570 con 0x55fdd9f57980 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd46d986740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641ae7d0 con 0x55fdd9f535f0 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd46d986740 10 client.0 _ll_register_callbacks cb 0x7fd454031d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd468b48640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f75680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd468b48640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f75680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41112/0 (socket says 172.21.15.62:41112) 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd468b48640 1 -- 172.21.15.62:0/3269735771 learned_addr learned my addr 172.21.15.62:0/3269735771 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd469b4a640 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57980 0x55fdd9fdb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd469b4a640 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f535f0 0x55fdd9f75140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd468b48640 1 -- 172.21.15.62:0/3269735771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f535f0 msgr2=0x55fdd9f75140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd468b48640 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f535f0 0x55fdd9f75140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd468b48640 1 -- 172.21.15.62:0/3269735771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57980 msgr2=0x55fdd9fdb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd468b48640 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57980 0x55fdd9fdb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd468b48640 1 -- 172.21.15.62:0/3269735771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdd9fdba50 con 0x55fdd9f57060 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd468b48640 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f75680 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd458005d80 tx=0x7fd458005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7d33e459c1a99f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.219+0000 7fd448ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd448ff9640 1 -- 172.21.15.62:0/3269735771 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4580137c0 con 0x55fdd9f57060 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd46d986740 1 -- 172.21.15.62:0/3269735771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd454036070 con 0x55fdd9f57060 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd448ff9640 1 -- 172.21.15.62:0/3269735771 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fd458013960 con 0x55fdd9f57060 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd448ff9640 1 -- 172.21.15.62:0/3269735771 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd458013b60 con 0x55fdd9f57060 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd46d986740 10 client.17628 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd46d986740 1 -- 172.21.15.62:0/3269735771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fdd9fdccf0 con 0x55fdd9f57060 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd448ff9640 1 -- 172.21.15.62:0/3269735771 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd454036070 con 0x55fdd9f57060 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd448ff9640 1 -- 172.21.15.62:0/3269735771 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fd458022070 con 0x55fdd9f57060 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd46d986740 10 client.17628 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd46d986740 10 client.17628 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:04:41.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd46d986740 20 client.17628 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:41.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd46d986740 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fdd9f62240 0x55fdd9fdf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:41.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd46d986740 4 client.17628 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "51961a7", "release": true, "await": true}] 2024-09-12T00:04:41.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.220+0000 7fd46d986740 1 -- 172.21.15.62:0/3269735771 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "51961a7", "release": true, "await": true}) -- 0x55fdd9ec40a0 con 0x55fdd9f62240 2024-09-12T00:04:41.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.221+0000 7fd469b4a640 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fdd9f62240 0x55fdd9fdf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:41.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.221+0000 7fd469b4a640 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fdd9f62240 0x55fdd9fdf510 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad2f08e5212e245 server_cookie=b3aafb816e22bd6b in_seq=0 out_seq=0 2024-09-12T00:04:41.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.221+0000 7fd448ff9640 10 client.17628 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd448ff9640 1 -- 172.21.15.62:0/3269735771 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55fdd9ec40a0 con 0x55fdd9f62240 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd448ff9640 10 client.17628 handle_command_reply: tid=0 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 client.17628 shutdown 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 -- 172.21.15.62:0/3269735771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 msgr2=0x55fdd9f75680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f75680 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd458005d80 tx=0x7fd458005630 comp rx=0 tx=0).stop 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 -- 172.21.15.62:0/3269735771 shutdown_connections 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55fdd9f62240 0x55fdd9fdf510 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdd9f57980 0x55fdd9fdb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fdd9f57060 0x55fdd9f75680 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd448ff9640 0 client.17628 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 --2- 172.21.15.62:0/3269735771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fdd9f535f0 0x55fdd9f75140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 -- 172.21.15.62:0/3269735771 >> 172.21.15.62:0/3269735771 conn(0x55fdd9f5aac0 msgr2=0x55fdd9f5f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 -- 172.21.15.62:0/3269735771 shutdown_connections 2024-09-12T00:04:41.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.293+0000 7fd46d986740 1 -- 172.21.15.62:0/3269735771 wait complete. 2024-09-12T00:04:41.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46d986740 20 client.17628 trim_cache size 0 max 16384 2024-09-12T00:04:41.295 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:04:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 36, 2024-09-12T00:04:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:04:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 48, 2024-09-12T00:04:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:04:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "51961a7": { 2024-09-12T00:04:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 48, 2024-09-12T00:04:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:04:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:04:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:04:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:04:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:04:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:04:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 147.2, 2024-09-12T00:04:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:04:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:04:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:04:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:04:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:04:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:04:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:04:41.298 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:04:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 -- 172.21.15.62:0/2386024164 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd44c0a2cf0 msgr2=0x7fd44c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd44c0a2cf0 0x7fd44c0a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0ccd60 tx=0x7fd45c005300 comp rx=0 tx=0).stop 2024-09-12T00:04:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 -- 172.21.15.62:0/2386024164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd464103ff0 msgr2=0x7fd46419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd464103ff0 0x7fd46419a5f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00c890 tx=0x7fd45c00cd50 comp rx=0 tx=0).stop 2024-09-12T00:04:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 -- 172.21.15.62:0/2386024164 shutdown_connections 2024-09-12T00:04:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fd44c0a2cf0 0x7fd44c0a51b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464109120 0x7fd46419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4641049a0 0x7fd46419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 --2- 172.21.15.62:0/2386024164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd464103ff0 0x7fd46419a5f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 -- 172.21.15.62:0/2386024164 >> 172.21.15.62:0/2386024164 conn(0x7fd4640ffa90 msgr2=0x7fd464100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 -- 172.21.15.62:0/2386024164 shutdown_connections 2024-09-12T00:04:41.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:41.294+0000 7fd46b5ad640 1 -- 172.21.15.62:0/2386024164 wait complete. 2024-09-12T00:04:42.165 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '51961a7', total seconds elapsed: 30.6 2024-09-12T00:04:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 720.0 s, 562.6 tps, lat 896.970 ms stddev 4453.532 2024-09-12T00:04:43.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:42 smithi120 ceph-mon[35182]: pgmap v393: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-09-12T00:04:43.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:42 smithi083 ceph-mon[34805]: pgmap v393: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-09-12T00:04:43.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:42 smithi062 ceph-mon[30958]: pgmap v393: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-09-12T00:04:45.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:44 smithi120 ceph-mon[35182]: pgmap v394: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-12T00:04:45.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:44 smithi083 ceph-mon[34805]: pgmap v394: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-12T00:04:45.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:44 smithi062 ceph-mon[30958]: pgmap v394: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-12T00:04:47.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:46 smithi120 ceph-mon[35182]: pgmap v395: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.0 MiB/s wr, 348 op/s 2024-09-12T00:04:47.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:46 smithi083 ceph-mon[34805]: pgmap v395: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.0 MiB/s wr, 348 op/s 2024-09-12T00:04:47.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:46 smithi062 ceph-mon[30958]: pgmap v395: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.0 MiB/s wr, 348 op/s 2024-09-12T00:04:47.444 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 725.0 s, 1635.4 tps, lat 60.178 ms stddev 61.412 2024-09-12T00:04:49.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:48 smithi120 ceph-mon[35182]: pgmap v396: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.0 MiB/s wr, 562 op/s 2024-09-12T00:04:49.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:48 smithi083 ceph-mon[34805]: pgmap v396: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.0 MiB/s wr, 562 op/s 2024-09-12T00:04:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:48 smithi062 ceph-mon[30958]: pgmap v396: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.0 MiB/s wr, 562 op/s 2024-09-12T00:04:51.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:50 smithi120 ceph-mon[35182]: pgmap v397: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-09-12T00:04:51.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:50 smithi083 ceph-mon[34805]: pgmap v397: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-09-12T00:04:51.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:50 smithi062 ceph-mon[30958]: pgmap v397: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-09-12T00:04:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 730.0 s, 1578.8 tps, lat 64.008 ms stddev 73.598 2024-09-12T00:04:53.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:52 smithi120 ceph-mon[35182]: pgmap v398: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.9 MiB/s wr, 323 op/s 2024-09-12T00:04:53.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:52 smithi083 ceph-mon[34805]: pgmap v398: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.9 MiB/s wr, 323 op/s 2024-09-12T00:04:53.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:52 smithi062 ceph-mon[30958]: pgmap v398: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.9 MiB/s wr, 323 op/s 2024-09-12T00:04:53.501 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:04:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:54.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.586+0000 7f5a6d985640 1 -- 172.21.15.62:0/4129168739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a68101380 msgr2=0x7f5a6810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:54.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.586+0000 7f5a6d985640 1 --2- 172.21.15.62:0/4129168739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a68101380 0x7f5a6810dfb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009ca0 tx=0x7f5a6002a8d0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.586+0000 7f5a6d985640 1 -- 172.21.15.62:0/4129168739 shutdown_connections 2024-09-12T00:04:54.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.586+0000 7f5a6d985640 1 --2- 172.21.15.62:0/4129168739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a6810e4f0 0x7f5a68110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.586+0000 7f5a6d985640 1 --2- 172.21.15.62:0/4129168739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a68101380 0x7f5a6810dfb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.586+0000 7f5a6d985640 1 --2- 172.21.15.62:0/4129168739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a681009d0 0x7f5a68100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.586+0000 7f5a6d985640 1 -- 172.21.15.62:0/4129168739 >> 172.21.15.62:0/4129168739 conn(0x7f5a680fc7d0 msgr2=0x7f5a680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:54.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.586+0000 7f5a6d985640 1 -- 172.21.15.62:0/4129168739 shutdown_connections 2024-09-12T00:04:54.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.586+0000 7f5a6d985640 1 -- 172.21.15.62:0/4129168739 wait complete. 2024-09-12T00:04:54.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.587+0000 7f5a6d985640 1 Processor -- start 2024-09-12T00:04:54.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a6d985640 1 -- start start 2024-09-12T00:04:54.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a6d985640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a681009d0 0x7f5a681a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a6d985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a68101380 0x7f5a681a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a6d985640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a6810e4f0 0x7f5a6819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a6d985640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68114310 con 0x7f5a681009d0 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a6d985640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68114190 con 0x7f5a68101380 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a6d985640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68114490 con 0x7f5a6810e4f0 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a68101380 0x7f5a681a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a68101380 0x7f5a681a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40868/0 (socket says 172.21.15.62:40868) 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a667fc640 1 -- 172.21.15.62:0/54290098 learned_addr learned my addr 172.21.15.62:0/54290098 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.588+0000 7f5a677fe640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a6810e4f0 0x7f5a6819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.589+0000 7f5a667fc640 1 -- 172.21.15.62:0/54290098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a6810e4f0 msgr2=0x7f5a6819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.589+0000 7f5a667fc640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a6810e4f0 0x7f5a6819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.589+0000 7f5a667fc640 1 -- 172.21.15.62:0/54290098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a681009d0 msgr2=0x7f5a681a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.589+0000 7f5a667fc640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a681009d0 0x7f5a681a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.589+0000 7f5a667fc640 1 -- 172.21.15.62:0/54290098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6819ace0 con 0x7f5a68101380 2024-09-12T00:04:54.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.589+0000 7f5a677fe640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a6810e4f0 0x7f5a6819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:04:54.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.589+0000 7f5a667fc640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a68101380 0x7f5a681a0930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5a6002f510 tx=0x7f5a60003ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:54.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.589+0000 7f5a6c983640 1 -- 172.21.15.62:0/54290098 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60031760 con 0x7f5a68101380 2024-09-12T00:04:54.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.589+0000 7f5a6c983640 1 -- 172.21.15.62:0/54290098 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60031900 con 0x7f5a68101380 2024-09-12T00:04:54.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.589+0000 7f5a6c983640 1 -- 172.21.15.62:0/54290098 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6002fc10 con 0x7f5a68101380 2024-09-12T00:04:54.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.590+0000 7f5a6d985640 1 -- 172.21.15.62:0/54290098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6819af70 con 0x7f5a68101380 2024-09-12T00:04:54.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.591+0000 7f5a6d985640 1 -- 172.21.15.62:0/54290098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a681a72c0 con 0x7f5a68101380 2024-09-12T00:04:54.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.595+0000 7f5a6c983640 1 -- 172.21.15.62:0/54290098 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5a60004040 con 0x7f5a68101380 2024-09-12T00:04:54.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.595+0000 7f5a6fd5e740 1 Processor -- start 2024-09-12T00:04:54.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.595+0000 7f5a6fd5e740 1 -- start start 2024-09-12T00:04:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.595+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439f4a0 0x556d5439f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.595+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439fdc0 0x556d543aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.595+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a6fd5e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556d543052e0 con 0x556d5439fdc0 2024-09-12T00:04:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a6fd5e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556d542acce0 con 0x556d543aa680 2024-09-12T00:04:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a6fd5e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556d543012a0 con 0x556d5439f4a0 2024-09-12T00:04:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a66ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439fdc0 0x556d543aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a66ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439fdc0 0x556d543aa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46028/0 (socket says 172.21.15.62:46028) 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40884/0 (socket says 172.21.15.62:40884) 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a66ffd640 1 -- 172.21.15.62:0/4082218182 learned_addr learned my addr 172.21.15.62:0/4082218182 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a677fe640 1 --2- 172.21.15.62:0/4082218182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439f4a0 0x556d5439f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a6c983640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5a500a2f00 0x7f5a500a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a667fc640 1 -- 172.21.15.62:0/4082218182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439f4a0 msgr2=0x556d5439f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a667fc640 1 --2- 172.21.15.62:0/4082218182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439f4a0 0x556d5439f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a667fc640 1 -- 172.21.15.62:0/4082218182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439fdc0 msgr2=0x556d543aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a667fc640 1 --2- 172.21.15.62:0/4082218182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439fdc0 0x556d543aa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.596+0000 7f5a667fc640 1 -- 172.21.15.62:0/4082218182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d54291440 con 0x556d543aa680 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.597+0000 7f5a677fe640 1 --2- 172.21.15.62:0/4082218182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439f4a0 0x556d5439f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.597+0000 7f5a667fc640 1 --2- 172.21.15.62:0/4082218182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543aca70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5a600edfe0 tx=0x7f5a600879b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9a02931dd6fba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.597+0000 7f5a4dffb640 1 -- 172.21.15.62:0/4082218182 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600bc8f0 con 0x556d543aa680 2024-09-12T00:04:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.597+0000 7f5a4dffb640 1 -- 172.21.15.62:0/4082218182 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60087ec0 con 0x556d543aa680 2024-09-12T00:04:54.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.597+0000 7f5a4dffb640 1 -- 172.21.15.62:0/4082218182 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600af440 con 0x556d543aa680 2024-09-12T00:04:54.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.597+0000 7f5a6c983640 1 -- 172.21.15.62:0/54290098 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f5a600f07e0 con 0x7f5a68101380 2024-09-12T00:04:54.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.597+0000 7f5a677fe640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5a500a2f00 0x7f5a500a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.598+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/4082218182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 msgr2=0x556d543aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.598+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/4082218182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543aca70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5a600edfe0 tx=0x7f5a600879b0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.598+0000 7f5a677fe640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5a500a2f00 0x7f5a500a53c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5a68069760 tx=0x7f5a5c004730 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.598+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/4082218182 shutdown_connections 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.598+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/4082218182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543aca70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.598+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/4082218182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439fdc0 0x556d543aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.598+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/4082218182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439f4a0 0x556d5439f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.598+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/4082218182 >> 172.21.15.62:0/4082218182 conn(0x556d5439ba30 msgr2=0x556d5439be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.598+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/4082218182 shutdown_connections 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.599+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/4082218182 wait complete. 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.599+0000 7f5a6fd5e740 1 -- start start 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.600+0000 7f5a6fd5e740 1 Processor -- start 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.600+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439f4a0 0x556d543a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.600+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439fdc0 0x556d543a7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.600+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543a7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439fdc0 0x556d543a7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439fdc0 0x556d543a7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:54900/0 (socket says 172.21.15.62:54900) 2024-09-12T00:04:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a667fc640 1 -- 172.21.15.62:0/2450846641 learned_addr learned my addr 172.21.15.62:0/2450846641 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a6fd5e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556d543ad010 con 0x556d5439f4a0 2024-09-12T00:04:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/2450846641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556d542c0050 con 0x556d543aa680 2024-09-12T00:04:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/2450846641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556d543ad190 con 0x556d5439fdc0 2024-09-12T00:04:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a6fd5e740 10 client.0 _ll_register_callbacks cb 0x556d54300160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a66ffd640 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439f4a0 0x556d543a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a677fe640 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a667fc640 1 -- 172.21.15.62:0/2450846641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 msgr2=0x556d543a7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a667fc640 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543a7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a667fc640 1 -- 172.21.15.62:0/2450846641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439f4a0 msgr2=0x556d543a9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a667fc640 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439f4a0 0x556d543a9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a667fc640 1 -- 172.21.15.62:0/2450846641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d540763c0 con 0x556d5439fdc0 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.601+0000 7f5a677fe640 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543a7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.602+0000 7f5a667fc640 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439fdc0 0x556d543a7150 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5a600c54f0 tx=0x7f5a60003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17c76105e680a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.602+0000 7f5a46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.602+0000 7f5a46ffd640 1 -- 172.21.15.62:0/2450846641 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600acd20 con 0x556d5439fdc0 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.602+0000 7f5a46ffd640 1 -- 172.21.15.62:0/2450846641 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f5a6002fdb0 con 0x556d5439fdc0 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.602+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/2450846641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a60087ec0 con 0x556d5439fdc0 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.603+0000 7f5a46ffd640 1 -- 172.21.15.62:0/2450846641 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5a600aa120 con 0x556d5439fdc0 2024-09-12T00:04:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.603+0000 7f5a6fd5e740 10 client.27058 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.603+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/2450846641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556d543ad010 con 0x556d5439fdc0 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.603+0000 7f5a46ffd640 1 -- 172.21.15.62:0/2450846641 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60036020 con 0x556d5439fdc0 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.603+0000 7f5a46ffd640 1 -- 172.21.15.62:0/2450846641 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f5a600b9420 con 0x556d5439fdc0 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.603+0000 7f5a6fd5e740 10 client.27058 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.603+0000 7f5a6fd5e740 10 client.27058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.603+0000 7f5a6fd5e740 20 client.27058 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.603+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d543b9c40 0x556d543bc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.604+0000 7f5a6fd5e740 4 client.27058 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.604+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/2450846641 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5a60087ec0 con 0x556d543b9c40 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.604+0000 7f5a66ffd640 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d543b9c40 0x556d543bc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.604+0000 7f5a66ffd640 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d543b9c40 0x556d543bc100 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e9cc68053310a server_cookie=511d73de04cf71f9 in_seq=0 out_seq=0 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.604+0000 7f5a46ffd640 10 client.27058 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.605+0000 7f5a46ffd640 1 -- 172.21.15.62:0/2450846641 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5a60087ec0 con 0x556d543b9c40 2024-09-12T00:04:54.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.605+0000 7f5a46ffd640 10 client.27058 handle_command_reply: tid=0 2024-09-12T00:04:54.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.605+0000 7f5a6fd5e740 1 client.27058 shutdown 2024-09-12T00:04:54.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.605+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/2450846641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439fdc0 msgr2=0x556d543a7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:54.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.605+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439fdc0 0x556d543a7150 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5a600c54f0 tx=0x7f5a60003f90 comp rx=0 tx=0).stop 2024-09-12T00:04:54.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.605+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/2450846641 shutdown_connections 2024-09-12T00:04:54.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.605+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d543b9c40 0x556d543bc100 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.605+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d543aa680 0x556d543a7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.606+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d5439fdc0 0x556d543a7150 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5a600c54f0 tx=0x7f5a60003f90 comp rx=0 tx=0).stop 2024-09-12T00:04:54.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.606+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/2450846641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d5439f4a0 0x556d543a9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.606+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/2450846641 >> 172.21.15.62:0/2450846641 conn(0x556d5439ba30 msgr2=0x556d543a1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.606+0000 7f5a46ffd640 0 client.27058 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.606+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/2450846641 shutdown_connections 2024-09-12T00:04:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.606+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/2450846641 wait complete. 2024-09-12T00:04:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.606+0000 7f5a6fd5e740 20 client.27058 trim_cache size 0 max 16384 2024-09-12T00:04:54.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.632+0000 7f5a6fd5e740 1 Processor -- start 2024-09-12T00:04:54.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.632+0000 7f5a6fd5e740 1 -- start start 2024-09-12T00:04:54.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 0x556d543a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 0x556d543a90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 0x556d543c19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a6fd5e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681b2520 con 0x556d543a2870 2024-09-12T00:04:54.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a6fd5e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681afd90 con 0x556d5439f4a0 2024-09-12T00:04:54.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a6fd5e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681adc00 con 0x556d543aa680 2024-09-12T00:04:54.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a66ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 0x556d543a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a66ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 0x556d543a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40908/0 (socket says 172.21.15.62:40908) 2024-09-12T00:04:54.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 0x556d543a90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a66ffd640 1 -- 172.21.15.62:0/1691622582 learned_addr learned my addr 172.21.15.62:0/1691622582 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a677fe640 1 --2- 172.21.15.62:0/1691622582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 0x556d543c19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a667fc640 1 -- 172.21.15.62:0/1691622582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 msgr2=0x556d543a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a667fc640 1 --2- 172.21.15.62:0/1691622582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 0x556d543a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a667fc640 1 -- 172.21.15.62:0/1691622582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 msgr2=0x556d543c19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a667fc640 1 --2- 172.21.15.62:0/1691622582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 0x556d543c19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.633+0000 7f5a667fc640 1 -- 172.21.15.62:0/1691622582 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d543a6100 con 0x556d543aa680 2024-09-12T00:04:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a667fc640 1 --2- 172.21.15.62:0/1691622582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 0x556d543a90c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x556d543a8240 tx=0x7f5a600bb2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35aadcdd7269eae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a4d7fa640 1 -- 172.21.15.62:0/1691622582 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600aa320 con 0x556d543aa680 2024-09-12T00:04:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a4d7fa640 1 -- 172.21.15.62:0/1691622582 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60036030 con 0x556d543aa680 2024-09-12T00:04:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a4d7fa640 1 -- 172.21.15.62:0/1691622582 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600a7720 con 0x556d543aa680 2024-09-12T00:04:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1691622582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 msgr2=0x556d543a90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/1691622582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 0x556d543a90c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x556d543a8240 tx=0x7f5a600bb2c0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1691622582 shutdown_connections 2024-09-12T00:04:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/1691622582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 0x556d543c19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/1691622582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 0x556d543a90c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/1691622582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 0x556d543a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.634+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1691622582 >> 172.21.15.62:0/1691622582 conn(0x556d5439ba30 msgr2=0x556d543a89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1691622582 shutdown_connections 2024-09-12T00:04:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1691622582 wait complete. 2024-09-12T00:04:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 1 -- start start 2024-09-12T00:04:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 1 Processor -- start 2024-09-12T00:04:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 0x556d543b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 0x556d543b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 0x556d543ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681ae250 con 0x556d543a2870 2024-09-12T00:04:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681af6c0 con 0x556d5439f4a0 2024-09-12T00:04:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681ae920 con 0x556d543aa680 2024-09-12T00:04:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.635+0000 7f5a6fd5e740 10 client.0 _ll_register_callbacks cb 0x7f5a58009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:54.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.636+0000 7f5a66ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 0x556d543b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.636+0000 7f5a66ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 0x556d543b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40918/0 (socket says 172.21.15.62:40918) 2024-09-12T00:04:54.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.636+0000 7f5a66ffd640 1 -- 172.21.15.62:0/1129639773 learned_addr learned my addr 172.21.15.62:0/1129639773 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:54.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.636+0000 7f5a667fc640 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 0x556d543b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.636+0000 7f5a677fe640 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 0x556d543ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.636+0000 7f5a667fc640 1 -- 172.21.15.62:0/1129639773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 msgr2=0x556d543ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.636+0000 7f5a667fc640 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 0x556d543ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.636+0000 7f5a667fc640 1 -- 172.21.15.62:0/1129639773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 msgr2=0x556d543b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.636+0000 7f5a667fc640 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 0x556d543b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.636+0000 7f5a667fc640 1 -- 172.21.15.62:0/1129639773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d543adc80 con 0x556d543a2870 2024-09-12T00:04:54.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.639+0000 7f5a667fc640 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 0x556d543b2bc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x556d543a8240 tx=0x7f5a6002a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8946dc3ef41fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:54.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.639+0000 7f5a46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:04:54.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.639+0000 7f5a46ffd640 1 -- 172.21.15.62:0/1129639773 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600a7a70 con 0x556d543a2870 2024-09-12T00:04:54.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.639+0000 7f5a46ffd640 1 -- 172.21.15.62:0/1129639773 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f5a600d0f10 con 0x556d543a2870 2024-09-12T00:04:54.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.639+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1129639773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a60036030 con 0x556d543a2870 2024-09-12T00:04:54.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a46ffd640 1 -- 172.21.15.62:0/1129639773 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5a600bc710 con 0x556d543a2870 2024-09-12T00:04:54.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a6fd5e740 10 client.17640 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:54.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1129639773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556d543adc80 con 0x556d543a2870 2024-09-12T00:04:54.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a46ffd640 1 -- 172.21.15.62:0/1129639773 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60036030 con 0x556d543a2870 2024-09-12T00:04:54.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a46ffd640 1 -- 172.21.15.62:0/1129639773 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f5a600b8bc0 con 0x556d543a2870 2024-09-12T00:04:54.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a6fd5e740 10 client.17640 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a6fd5e740 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:04:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a6fd5e740 20 client.17640 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d54425960 0x556d54427d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a6fd5e740 4 client.17640 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:04:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1129639773 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d54312850 con 0x556d54425960 2024-09-12T00:04:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.640+0000 7f5a66ffd640 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d54425960 0x556d54427d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:54.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a46ffd640 10 client.17640 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:54.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a66ffd640 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d54425960 0x556d54427d50 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=164b0524d551c437 server_cookie=de3c35660cbe5794 in_seq=0 out_seq=0 2024-09-12T00:04:54.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a46ffd640 1 -- 172.21.15.62:0/1129639773 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556d54312850 con 0x556d54425960 2024-09-12T00:04:54.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a46ffd640 10 client.17640 handle_command_reply: tid=0 2024-09-12T00:04:54.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a6fd5e740 1 client.17640 shutdown 2024-09-12T00:04:54.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1129639773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 msgr2=0x556d543b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:54.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 0x556d543b2bc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x556d543a8240 tx=0x7f5a6002a8d0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1129639773 shutdown_connections 2024-09-12T00:04:54.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x556d54425960 0x556d54427d50 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d543aa680 0x556d543ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556d543a2870 0x556d543b2bc0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a46ffd640 0 client.17640 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:04:54.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a6fd5e740 1 --2- 172.21.15.62:0/1129639773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d5439f4a0 0x556d543b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.641+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1129639773 >> 172.21.15.62:0/1129639773 conn(0x556d5439ba30 msgr2=0x556d543a89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:54.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1129639773 shutdown_connections 2024-09-12T00:04:54.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6fd5e740 1 -- 172.21.15.62:0/1129639773 wait complete. 2024-09-12T00:04:54.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6fd5e740 20 client.17640 trim_cache size 0 max 16384 2024-09-12T00:04:54.657 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:04:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:04:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:04:54.658 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:04:54.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6d985640 1 -- 172.21.15.62:0/54290098 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5a500a2f00 msgr2=0x7f5a500a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:54.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6d985640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5a500a2f00 0x7f5a500a53c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5a68069760 tx=0x7f5a5c004730 comp rx=0 tx=0).stop 2024-09-12T00:04:54.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6d985640 1 -- 172.21.15.62:0/54290098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a68101380 msgr2=0x7f5a681a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:54.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6d985640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a68101380 0x7f5a681a0930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5a6002f510 tx=0x7f5a60003ef0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6d985640 1 -- 172.21.15.62:0/54290098 shutdown_connections 2024-09-12T00:04:54.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6d985640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f5a500a2f00 0x7f5a500a53c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6d985640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a6810e4f0 0x7f5a6819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6d985640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a68101380 0x7f5a681a0930 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6d985640 1 --2- 172.21.15.62:0/54290098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a681009d0 0x7f5a681a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:54.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.642+0000 7f5a6d985640 1 -- 172.21.15.62:0/54290098 >> 172.21.15.62:0/54290098 conn(0x7f5a680fc7d0 msgr2=0x7f5a680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:54.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.643+0000 7f5a6d985640 1 -- 172.21.15.62:0/54290098 shutdown_connections 2024-09-12T00:04:54.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:54.643+0000 7f5a6d985640 1 -- 172.21.15.62:0/54290098 wait complete. 2024-09-12T00:04:55.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:54 smithi120 ceph-mon[35182]: pgmap v399: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.2 MiB/s wr, 575 op/s 2024-09-12T00:04:55.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:54 smithi083 ceph-mon[34805]: pgmap v399: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.2 MiB/s wr, 575 op/s 2024-09-12T00:04:55.177 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:54 smithi062 ceph-mon[30958]: pgmap v399: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.2 MiB/s wr, 575 op/s 2024-09-12T00:04:55.745 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:04:55.746 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:04:56.131 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.625+0000 7f7d16b22640 1 -- 172.21.15.62:0/1498464721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10074510 msgr2=0x7f7d100748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.625+0000 7f7d16b22640 1 --2- 172.21.15.62:0/1498464721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10074510 0x7f7d100748f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009900 tx=0x7f7d0c02a480 comp rx=0 tx=0).stop 2024-09-12T00:04:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.625+0000 7f7d16b22640 1 -- 172.21.15.62:0/1498464721 shutdown_connections 2024-09-12T00:04:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.625+0000 7f7d16b22640 1 --2- 172.21.15.62:0/1498464721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1010e260 0x7f7d10110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.625+0000 7f7d16b22640 1 --2- 172.21.15.62:0/1498464721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d10074e30 0x7f7d1010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.625+0000 7f7d16b22640 1 --2- 172.21.15.62:0/1498464721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10074510 0x7f7d100748f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.625+0000 7f7d16b22640 1 -- 172.21.15.62:0/1498464721 >> 172.21.15.62:0/1498464721 conn(0x7f7d100fd680 msgr2=0x7f7d100ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.625+0000 7f7d16b22640 1 -- 172.21.15.62:0/1498464721 shutdown_connections 2024-09-12T00:04:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.625+0000 7f7d16b22640 1 -- 172.21.15.62:0/1498464721 wait complete. 2024-09-12T00:04:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d16b22640 1 Processor -- start 2024-09-12T00:04:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d16b22640 1 -- start start 2024-09-12T00:04:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d16b22640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10074510 0x7f7d1019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d16b22640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d10074e30 0x7f7d1019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d16b22640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d1010e260 0x7f7d1019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d16b22640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7d10113f90 con 0x7f7d10074510 2024-09-12T00:04:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d16b22640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d10113e10 con 0x7f7d10074e30 2024-09-12T00:04:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d16b22640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d10114110 con 0x7f7d1010e260 2024-09-12T00:04:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d15b20640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10074510 0x7f7d1019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d15b20640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10074510 0x7f7d1019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46062/0 (socket says 172.21.15.62:46062) 2024-09-12T00:04:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.626+0000 7f7d15b20640 1 -- 172.21.15.62:0/3764629787 learned_addr learned my addr 172.21.15.62:0/3764629787 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.627+0000 7f7d1531f640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d10074e30 0x7f7d1019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.627+0000 7f7d16321640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d1010e260 0x7f7d1019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.627+0000 7f7d15b20640 1 -- 172.21.15.62:0/3764629787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d1010e260 msgr2=0x7f7d1019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.627+0000 7f7d15b20640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d1010e260 0x7f7d1019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.627+0000 7f7d15b20640 1 -- 172.21.15.62:0/3764629787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d10074e30 msgr2=0x7f7d1019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.627+0000 7f7d15b20640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d10074e30 0x7f7d1019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.627+0000 7f7d15b20640 1 -- 172.21.15.62:0/3764629787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1019f530 con 0x7f7d10074510 2024-09-12T00:04:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.628+0000 7f7d15b20640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10074510 0x7f7d1019a580 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c006010 tx=0x7f7d0c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.628+0000 7f7d06ffd640 1 -- 172.21.15.62:0/3764629787 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c03f030 con 0x7f7d10074510 2024-09-12T00:04:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.629+0000 7f7d06ffd640 1 -- 172.21.15.62:0/3764629787 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c004440 con 0x7f7d10074510 2024-09-12T00:04:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.629+0000 7f7d16b22640 1 -- 172.21.15.62:0/3764629787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1019f7c0 con 0x7f7d10074510 2024-09-12T00:04:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.629+0000 7f7d06ffd640 1 -- 172.21.15.62:0/3764629787 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c035070 con 0x7f7d10074510 2024-09-12T00:04:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.630+0000 7f7d16b22640 1 -- 172.21.15.62:0/3764629787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d101a7100 con 0x7f7d10074510 2024-09-12T00:04:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.631+0000 7f7d16b22640 1 -- 172.21.15.62:0/3764629787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d10075c50 con 0x7f7d10074510 2024-09-12T00:04:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.637+0000 7f7d06ffd640 1 -- 172.21.15.62:0/3764629787 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7d0c042030 con 0x7f7d10074510 2024-09-12T00:04:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.638+0000 7f7d06ffd640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7cfc0a2d70 0x7f7cfc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.639+0000 7f7d06ffd640 1 -- 172.21.15.62:0/3764629787 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f7d0c0b9a80 con 0x7f7d10074510 2024-09-12T00:04:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.645+0000 7f7d1531f640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7cfc0a2d70 0x7f7cfc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.651+0000 7f7d1531f640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7cfc0a2d70 0x7f7cfc0a5230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7d100ffd10 tx=0x7f7d00005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.651+0000 7f7d06ffd640 1 -- 172.21.15.62:0/3764629787 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d0c0bb9e0 con 0x7f7d10074510 2024-09-12T00:04:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.977+0000 7f7d16b22640 1 -- 172.21.15.62:0/3764629787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d101093e0 con 0x7f7d10074510 2024-09-12T00:04:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.978+0000 7f7d06ffd640 1 -- 172.21.15.62:0/3764629787 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11913 (secure 0 0 0) 0x7f7d0c0bfce0 con 0x7f7d10074510 2024-09-12T00:04:56.978 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:04:56.978 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":36,"btime":"2024-09-12T00:03:36:303242+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:03:35.719292+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:04:56.979 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 36 2024-09-12T00:04:56.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.981+0000 7f7d16b22640 1 -- 172.21.15.62:0/3764629787 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7cfc0a2d70 msgr2=0x7f7cfc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.981+0000 7f7d16b22640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7cfc0a2d70 0x7f7cfc0a5230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7d100ffd10 tx=0x7f7d00005f90 comp rx=0 tx=0).stop 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.981+0000 7f7d16b22640 1 -- 172.21.15.62:0/3764629787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10074510 msgr2=0x7f7d1019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.982+0000 7f7d16b22640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10074510 0x7f7d1019a580 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c006010 tx=0x7f7d0c004040 comp rx=0 tx=0).stop 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.982+0000 7f7d16b22640 1 -- 172.21.15.62:0/3764629787 shutdown_connections 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.982+0000 7f7d16b22640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7cfc0a2d70 0x7f7cfc0a5230 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.982+0000 7f7d16b22640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d1010e260 0x7f7d1019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.982+0000 7f7d16b22640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d10074e30 0x7f7d1019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.982+0000 7f7d16b22640 1 --2- 172.21.15.62:0/3764629787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10074510 0x7f7d1019a580 secure :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c006010 tx=0x7f7d0c004040 comp rx=0 tx=0).stop 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.982+0000 7f7d16b22640 1 -- 172.21.15.62:0/3764629787 >> 172.21.15.62:0/3764629787 conn(0x7f7d100fd680 msgr2=0x7f7d100feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.982+0000 7f7d16b22640 1 -- 172.21.15.62:0/3764629787 shutdown_connections 2024-09-12T00:04:56.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:56.982+0000 7f7d16b22640 1 -- 172.21.15.62:0/3764629787 wait complete. 2024-09-12T00:04:57.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:56 smithi120 ceph-mon[35182]: pgmap v400: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.6 MiB/s wr, 505 op/s 2024-09-12T00:04:57.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:56 smithi083 ceph-mon[34805]: pgmap v400: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.6 MiB/s wr, 505 op/s 2024-09-12T00:04:57.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:56 smithi062 ceph-mon[30958]: pgmap v400: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.6 MiB/s wr, 505 op/s 2024-09-12T00:04:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 735.0 s, 1546.2 tps, lat 64.617 ms stddev 74.355 2024-09-12T00:04:57.771 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:04:58.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:57 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3764629787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:04:58.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:57 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3764629787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:58.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:57 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3764629787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:04:58.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.663+0000 7fe0686d8640 1 -- 172.21.15.62:0/2908057425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe060106b80 msgr2=0x7fe060106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:58.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.663+0000 7fe0686d8640 1 --2- 172.21.15.62:0/2908057425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe060106b80 0x7fe060106f60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009900 tx=0x7fe05c02a410 comp rx=0 tx=0).stop 2024-09-12T00:04:58.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.663+0000 7fe0686d8640 1 -- 172.21.15.62:0/2908057425 shutdown_connections 2024-09-12T00:04:58.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.663+0000 7fe0686d8640 1 --2- 172.21.15.62:0/2908057425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe060100aa0 0x7fe06010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.663+0000 7fe0686d8640 1 --2- 172.21.15.62:0/2908057425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe060100100 0x7fe060100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.663+0000 7fe0686d8640 1 --2- 172.21.15.62:0/2908057425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe060106b80 0x7fe060106f60 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.663+0000 7fe0686d8640 1 -- 172.21.15.62:0/2908057425 >> 172.21.15.62:0/2908057425 conn(0x7fe060077230 msgr2=0x7fe0600fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:58.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.663+0000 7fe0686d8640 1 -- 172.21.15.62:0/2908057425 shutdown_connections 2024-09-12T00:04:58.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.663+0000 7fe0686d8640 1 -- 172.21.15.62:0/2908057425 wait complete. 2024-09-12T00:04:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe0686d8640 1 Processor -- start 2024-09-12T00:04:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe0686d8640 1 -- start start 2024-09-12T00:04:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe0686d8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe060100100 0x7fe06019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe0686d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe060100aa0 0x7fe06019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe0686d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe060106b80 0x7fe06019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe0686d8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe060101ad0 con 0x7fe060100100 2024-09-12T00:04:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe0686d8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe060101950 con 0x7fe060106b80 2024-09-12T00:04:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe0686d8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe060101c50 con 0x7fe060100aa0 2024-09-12T00:04:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe066474640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe060100100 0x7fe06019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe066474640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe060100100 0x7fe06019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46068/0 (socket says 172.21.15.62:46068) 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe066474640 1 -- 172.21.15.62:0/2672981148 learned_addr learned my addr 172.21.15.62:0/2672981148 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.664+0000 7fe065c73640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe060100aa0 0x7fe06019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe066c75640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe060106b80 0x7fe06019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe066474640 1 -- 172.21.15.62:0/2672981148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe060100aa0 msgr2=0x7fe06019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe066474640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe060100aa0 0x7fe06019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe066474640 1 -- 172.21.15.62:0/2672981148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe060106b80 msgr2=0x7fe06019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe066474640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe060106b80 0x7fe06019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe066474640 1 -- 172.21.15.62:0/2672981148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06019f620 con 0x7fe060100100 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe066474640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe060100100 0x7fe06019aa80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe05c006010 tx=0x7fe05c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe0577fe640 1 -- 172.21.15.62:0/2672981148 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c03f030 con 0x7fe060100100 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe0686d8640 1 -- 172.21.15.62:0/2672981148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06019f8b0 con 0x7fe060100100 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe0577fe640 1 -- 172.21.15.62:0/2672981148 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05c004440 con 0x7fe060100100 2024-09-12T00:04:58.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe0686d8640 1 -- 172.21.15.62:0/2672981148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0601a71f0 con 0x7fe060100100 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.665+0000 7fe0577fe640 1 -- 172.21.15.62:0/2672981148 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c035070 con 0x7fe060100100 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.668+0000 7fe06aab1740 1 Processor -- start 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.669+0000 7fe0577fe640 1 -- 172.21.15.62:0/2672981148 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe05c042030 con 0x7fe060100100 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.670+0000 7fe06aab1740 1 -- start start 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.670+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744238090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.670+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744238a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.670+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x56274424df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.670+0000 7fe06aab1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56274419e2f0 con 0x5627442385d0 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.670+0000 7fe06aab1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562744145cf0 con 0x562744239b40 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.670+0000 7fe06aab1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56274419a2b0 con 0x56274424bb80 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe066474640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744238a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe065c73640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x56274424df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe065c73640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x56274424df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:54966/0 (socket says 172.21.15.62:54966) 2024-09-12T00:04:58.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe065c73640 1 -- 172.21.15.62:0/2224359539 learned_addr learned my addr 172.21.15.62:0/2224359539 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe066474640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744238a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46078/0 (socket says 172.21.15.62:46078) 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe066c75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744238090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe066c75640 1 -- 172.21.15.62:0/2224359539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 msgr2=0x56274424df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe066c75640 1 --2- 172.21.15.62:0/2224359539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x56274424df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe066c75640 1 -- 172.21.15.62:0/2224359539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 msgr2=0x562744238a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe066c75640 1 --2- 172.21.15.62:0/2224359539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744238a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.671+0000 7fe066c75640 1 -- 172.21.15.62:0/2224359539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56274412a3a0 con 0x562744239b40 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.672+0000 7fe066c75640 1 --2- 172.21.15.62:0/2224359539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744238090 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe058005fd0 tx=0x7fe058011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de52312c52eeac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.672+0000 7fe054ff9640 1 -- 172.21.15.62:0/2224359539 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058004d20 con 0x562744239b40 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.672+0000 7fe054ff9640 1 -- 172.21.15.62:0/2224359539 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe058004030 con 0x562744239b40 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.672+0000 7fe054ff9640 1 -- 172.21.15.62:0/2224359539 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058005280 con 0x562744239b40 2024-09-12T00:04:58.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.672+0000 7fe0577fe640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe04c0a2d70 0x7fe04c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.672+0000 7fe066474640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe04c0a2d70 0x7fe04c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.672+0000 7fe0577fe640 1 -- 172.21.15.62:0/2672981148 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fe05c0ef720 con 0x7fe060100100 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe066474640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe04c0a2d70 0x7fe04c0a5230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0ecf20 tx=0x7fe05c0f5550 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe06aab1740 1 -- 172.21.15.62:0/2224359539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 msgr2=0x562744238090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe06aab1740 1 --2- 172.21.15.62:0/2224359539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744238090 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe058005fd0 tx=0x7fe058011810 comp rx=0 tx=0).stop 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe06aab1740 1 -- 172.21.15.62:0/2224359539 shutdown_connections 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe06aab1740 1 --2- 172.21.15.62:0/2224359539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x56274424df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe06aab1740 1 --2- 172.21.15.62:0/2224359539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744238a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe06aab1740 1 --2- 172.21.15.62:0/2224359539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744238090 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe06aab1740 1 -- 172.21.15.62:0/2224359539 >> 172.21.15.62:0/2224359539 conn(0x562744201310 msgr2=0x562744234c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe06aab1740 1 -- 172.21.15.62:0/2224359539 shutdown_connections 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe06aab1740 1 -- 172.21.15.62:0/2224359539 wait complete. 2024-09-12T00:04:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.673+0000 7fe06aab1740 1 -- start start 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe06aab1740 1 Processor -- start 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744245520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744245a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x56274424b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe06aab1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56274423b6e0 con 0x5627442385d0 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe06aab1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562744159060 con 0x562744239b40 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe06aab1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56274423b860 con 0x56274424bb80 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe06aab1740 10 client.0 _ll_register_callbacks cb 0x562743eac8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe066c75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744245520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe065c73640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744245a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe066c75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744245520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46088/0 (socket says 172.21.15.62:46088) 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe065c73640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744245a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:40978/0 (socket says 172.21.15.62:40978) 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe066c75640 1 -- 172.21.15.62:0/3982408186 learned_addr learned my addr 172.21.15.62:0/3982408186 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe065c73640 1 -- 172.21.15.62:0/3982408186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 msgr2=0x56274424b230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:04:58.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe065c73640 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x56274424b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe065c73640 1 -- 172.21.15.62:0/3982408186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 msgr2=0x562744245520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe065c73640 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744245520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe065c73640 1 -- 172.21.15.62:0/3982408186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562743f0f3c0 con 0x562744239b40 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.674+0000 7fe066c75640 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744245520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.675+0000 7fe065c73640 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744245a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe0500126b0 tx=0x7fe050012ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e3c1f1d618b349 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.675+0000 7fe049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.675+0000 7fe049ffb640 1 -- 172.21.15.62:0/3982408186 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050004050 con 0x562744239b40 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.675+0000 7fe049ffb640 1 -- 172.21.15.62:0/3982408186 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fe0500041f0 con 0x562744239b40 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.675+0000 7fe06aab1740 1 -- 172.21.15.62:0/3982408186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe058004030 con 0x562744239b40 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.675+0000 7fe049ffb640 1 -- 172.21.15.62:0/3982408186 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe0500043f0 con 0x562744239b40 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.675+0000 7fe06aab1740 10 client.27188 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.675+0000 7fe06aab1740 1 -- 172.21.15.62:0/3982408186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562744262bd0 con 0x562744239b40 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.675+0000 7fe049ffb640 1 -- 172.21.15.62:0/3982408186 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe058004030 con 0x562744239b40 2024-09-12T00:04:58.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.676+0000 7fe049ffb640 1 -- 172.21.15.62:0/3982408186 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fe050003980 con 0x562744239b40 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.676+0000 7fe06aab1740 10 client.27188 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.676+0000 7fe06aab1740 10 client.27188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.676+0000 7fe06aab1740 20 client.27188 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.676+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5627442640e0 0x562744266540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.676+0000 7fe06aab1740 4 client.27188 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.676+0000 7fe06aab1740 1 -- 172.21.15.62:0/3982408186 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627441ab860 con 0x5627442640e0 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.676+0000 7fe066c75640 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5627442640e0 0x562744266540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.677+0000 7fe066c75640 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5627442640e0 0x562744266540 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a45e72fcd8272847 server_cookie=85f9fc5ea669a3c9 in_seq=0 out_seq=0 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.677+0000 7fe049ffb640 10 client.27188 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe049ffb640 1 -- 172.21.15.62:0/3982408186 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627441ab860 con 0x5627442640e0 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe049ffb640 10 client.27188 handle_command_reply: tid=0 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 client.27188 shutdown 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 -- 172.21.15.62:0/3982408186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 msgr2=0x562744245a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:58.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744245a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe0500126b0 tx=0x7fe050012ab0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 -- 172.21.15.62:0/3982408186 shutdown_connections 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5627442640e0 0x562744266540 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x56274424b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x562744245a60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3982408186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744245520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe049ffb640 0 client.27188 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 -- 172.21.15.62:0/3982408186 >> 172.21.15.62:0/3982408186 conn(0x562744201310 msgr2=0x56274423a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 -- 172.21.15.62:0/3982408186 shutdown_connections 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 1 -- 172.21.15.62:0/3982408186 wait complete. 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.678+0000 7fe06aab1740 20 client.27188 trim_cache size 0 max 16384 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe06aab1740 1 Processor -- start 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe06aab1740 1 -- start start 2024-09-12T00:04:58.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x56274424b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562744239b40 0x56274426be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56274424bb80 0x56274426c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe06aab1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601adf20 con 0x5627442385d0 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe06aab1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601ab790 con 0x56274424bb80 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe06aab1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601a96a0 con 0x562744239b40 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe066c75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x56274424b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe066c75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x56274424b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46100/0 (socket says 172.21.15.62:46100) 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.697+0000 7fe066c75640 1 -- 172.21.15.62:0/2636791771 learned_addr learned my addr 172.21.15.62:0/2636791771 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.698+0000 7fe065c73640 1 --2- 172.21.15.62:0/2636791771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562744239b40 0x56274426be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.698+0000 7fe066c75640 1 --2- 172.21.15.62:0/2636791771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56274424bb80 0x56274426c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.698+0000 7fe066c75640 1 -- 172.21.15.62:0/2636791771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562744239b40 msgr2=0x56274426be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:58.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.698+0000 7fe066c75640 1 --2- 172.21.15.62:0/2636791771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562744239b40 0x56274426be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.698+0000 7fe066c75640 1 -- 172.21.15.62:0/2636791771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56274424bb80 msgr2=0x56274426c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.698+0000 7fe066c75640 1 --2- 172.21.15.62:0/2636791771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56274424bb80 0x56274426c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.698+0000 7fe066c75640 1 -- 172.21.15.62:0/2636791771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562744241790 con 0x5627442385d0 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.698+0000 7fe065c73640 1 --2- 172.21.15.62:0/2636791771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562744239b40 0x56274426be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.699+0000 7fe066c75640 1 --2- 172.21.15.62:0/2636791771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x56274424b210 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe058003f50 tx=0x7fe058005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c9320db8c8138b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.699+0000 7fe04bfff640 1 -- 172.21.15.62:0/2636791771 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05800f540 con 0x5627442385d0 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.699+0000 7fe04bfff640 1 -- 172.21.15.62:0/2636791771 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05800fbc0 con 0x5627442385d0 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.699+0000 7fe04bfff640 1 -- 172.21.15.62:0/2636791771 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058004860 con 0x5627442385d0 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.700+0000 7fe06aab1740 1 -- 172.21.15.62:0/2636791771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 msgr2=0x56274424b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.700+0000 7fe06aab1740 1 --2- 172.21.15.62:0/2636791771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x56274424b210 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe058003f50 tx=0x7fe058005c40 comp rx=0 tx=0).stop 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.700+0000 7fe06aab1740 1 -- 172.21.15.62:0/2636791771 shutdown_connections 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.700+0000 7fe06aab1740 1 --2- 172.21.15.62:0/2636791771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56274424bb80 0x56274426c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.700+0000 7fe06aab1740 1 --2- 172.21.15.62:0/2636791771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562744239b40 0x56274426be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.700+0000 7fe06aab1740 1 --2- 172.21.15.62:0/2636791771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x56274424b210 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.700+0000 7fe06aab1740 1 -- 172.21.15.62:0/2636791771 >> 172.21.15.62:0/2636791771 conn(0x562744201310 msgr2=0x56274423f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.700+0000 7fe06aab1740 1 -- 172.21.15.62:0/2636791771 shutdown_connections 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.700+0000 7fe06aab1740 1 -- 172.21.15.62:0/2636791771 wait complete. 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe06aab1740 1 -- start start 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe06aab1740 1 Processor -- start 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744273b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x5627442740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe06aab1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x562744274620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe06aab1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601a9cf0 con 0x5627442385d0 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe06aab1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601ab0c0 con 0x562744239b40 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe06aab1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601aa340 con 0x56274424bb80 2024-09-12T00:04:58.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe066c75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744273b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe065c73640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x5627442740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe066c75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744273b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46104/0 (socket says 172.21.15.62:46104) 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe066c75640 1 -- 172.21.15.62:0/3449656591 learned_addr learned my addr 172.21.15.62:0/3449656591 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.701+0000 7fe06aab1740 10 client.0 _ll_register_callbacks cb 0x7fe050009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe066c75640 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x562744274620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe065c73640 1 -- 172.21.15.62:0/3449656591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 msgr2=0x562744274620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe065c73640 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x562744274620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe065c73640 1 -- 172.21.15.62:0/3449656591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 msgr2=0x562744273b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe065c73640 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744273b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe065c73640 1 -- 172.21.15.62:0/3449656591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627442cdf30 con 0x562744239b40 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe065c73640 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x5627442740c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe050003da0 tx=0x7fe050004240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18883c094e1a9210 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe049ffb640 1 -- 172.21.15.62:0/3449656591 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050029040 con 0x562744239b40 2024-09-12T00:04:58.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe049ffb640 1 -- 172.21.15.62:0/3449656591 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fe0500036b0 con 0x562744239b40 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.702+0000 7fe06aab1740 1 -- 172.21.15.62:0/3449656591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05800fbc0 con 0x562744239b40 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.703+0000 7fe049ffb640 1 -- 172.21.15.62:0/3449656591 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe0500038b0 con 0x562744239b40 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.703+0000 7fe06aab1740 10 client.27191 fetch_fsmap learned FSMap version 36 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.703+0000 7fe06aab1740 1 -- 172.21.15.62:0/3449656591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5627442cf270 con 0x562744239b40 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.703+0000 7fe049ffb640 1 -- 172.21.15.62:0/3449656591 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe050005520 con 0x562744239b40 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.704+0000 7fe049ffb640 1 -- 172.21.15.62:0/3449656591 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fe05002c070 con 0x562744239b40 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.705+0000 7fe06aab1740 10 client.27191 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.705+0000 7fe06aab1740 10 client.27191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.705+0000 7fe06aab1740 20 client.27191 populate_metadata read hostname 'smithi062' 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.705+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5627442cf990 0x5627442d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.706+0000 7fe06aab1740 4 client.27191 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.706+0000 7fe06aab1740 1 -- 172.21.15.62:0/3449656591 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe05800fbc0 con 0x5627442cf990 2024-09-12T00:04:58.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.706+0000 7fe066c75640 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5627442cf990 0x5627442d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.706+0000 7fe049ffb640 10 client.27191 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.706+0000 7fe066c75640 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5627442cf990 0x5627442d1d80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33a7b1943e590ab6 server_cookie=d5c284a7541f7aac in_seq=0 out_seq=0 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe049ffb640 1 -- 172.21.15.62:0/3449656591 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe05800fbc0 con 0x5627442cf990 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe049ffb640 10 client.27191 handle_command_reply: tid=0 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 client.27191 shutdown 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 -- 172.21.15.62:0/3449656591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 msgr2=0x5627442740c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x5627442740c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe050003da0 tx=0x7fe050004240 comp rx=0 tx=0).stop 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 -- 172.21.15.62:0/3449656591 shutdown_connections 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5627442cf990 0x5627442d1d80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56274424bb80 0x562744274620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe049ffb640 0 client.27191 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562744239b40 0x5627442740c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 --2- 172.21.15.62:0/3449656591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627442385d0 0x562744273b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 -- 172.21.15.62:0/3449656591 >> 172.21.15.62:0/3449656591 conn(0x562744201310 msgr2=0x5627442334e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:58.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 -- 172.21.15.62:0/3449656591 shutdown_connections 2024-09-12T00:04:58.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 1 -- 172.21.15.62:0/3449656591 wait complete. 2024-09-12T00:04:58.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.707+0000 7fe06aab1740 20 client.27191 trim_cache size 0 max 16384 2024-09-12T00:04:58.747 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 -- 172.21.15.62:0/2672981148 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe04c0a2d70 msgr2=0x7fe04c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe04c0a2d70 0x7fe04c0a5230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0ecf20 tx=0x7fe05c0f5550 comp rx=0 tx=0).stop 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 -- 172.21.15.62:0/2672981148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe060100100 msgr2=0x7fe06019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe060100100 0x7fe06019aa80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe05c006010 tx=0x7fe05c004040 comp rx=0 tx=0).stop 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 -- 172.21.15.62:0/2672981148 shutdown_connections 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe04c0a2d70 0x7fe04c0a5230 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe060106b80 0x7fe06019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe060100aa0 0x7fe06019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 --2- 172.21.15.62:0/2672981148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe060100100 0x7fe06019aa80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 -- 172.21.15.62:0/2672981148 >> 172.21.15.62:0/2672981148 conn(0x7fe060077230 msgr2=0x7fe0601051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:04:58.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 -- 172.21.15.62:0/2672981148 shutdown_connections 2024-09-12T00:04:58.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:04:58.708+0000 7fe0686d8640 1 -- 172.21.15.62:0/2672981148 wait complete. 2024-09-12T00:04:59.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:04:58 smithi120 ceph-mon[35182]: pgmap v401: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.2 MiB/s wr, 691 op/s 2024-09-12T00:04:59.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:04:58 smithi083 ceph-mon[34805]: pgmap v401: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.2 MiB/s wr, 691 op/s 2024-09-12T00:04:59.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:04:58 smithi062 ceph-mon[30958]: pgmap v401: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.2 MiB/s wr, 691 op/s 2024-09-12T00:04:59.935 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:05:00.289 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:00.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.872+0000 7fc68ee2d640 1 -- 172.21.15.62:0/1936744187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc688109450 msgr2=0x7fc688110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:00.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.872+0000 7fc68ee2d640 1 --2- 172.21.15.62:0/1936744187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc688109450 0x7fc688110250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc674009860 tx=0x7fc67402a360 comp rx=0 tx=0).stop 2024-09-12T00:05:00.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.872+0000 7fc68ee2d640 1 -- 172.21.15.62:0/1936744187 shutdown_connections 2024-09-12T00:05:00.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.872+0000 7fc68ee2d640 1 --2- 172.21.15.62:0/1936744187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc68811b140 0x7fc68811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.872+0000 7fc68ee2d640 1 --2- 172.21.15.62:0/1936744187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc688109830 0x7fc688109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.872+0000 7fc68ee2d640 1 --2- 172.21.15.62:0/1936744187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc688109450 0x7fc688110250 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc674009860 tx=0x7fc67402a360 comp rx=0 tx=0).stop 2024-09-12T00:05:00.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.872+0000 7fc68ee2d640 1 -- 172.21.15.62:0/1936744187 >> 172.21.15.62:0/1936744187 conn(0x7fc68806e140 msgr2=0x7fc68806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:00.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.875+0000 7fc68ee2d640 1 -- 172.21.15.62:0/1936744187 shutdown_connections 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.876+0000 7fc68ee2d640 1 -- 172.21.15.62:0/1936744187 wait complete. 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.877+0000 7fc68ee2d640 1 Processor -- start 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.877+0000 7fc68ee2d640 1 -- start start 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.877+0000 7fc68ee2d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc688109830 0x7fc6881a7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.877+0000 7fc68ee2d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc68811b140 0x7fc6881a79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.877+0000 7fc68ee2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6881abd50 0x7fc6881ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.877+0000 7fc68ee2d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc688120df0 con 0x7fc68811b140 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.877+0000 7fc68ee2d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc688120c70 con 0x7fc688109830 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.877+0000 7fc68ee2d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc688120f70 con 0x7fc6881abd50 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.878+0000 7fc68d3ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6881abd50 0x7fc6881ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.878+0000 7fc68d3ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6881abd50 0x7fc6881ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:55006/0 (socket says 172.21.15.62:55006) 2024-09-12T00:05:00.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.878+0000 7fc68d3ca640 1 -- 172.21.15.62:0/3542201428 learned_addr learned my addr 172.21.15.62:0/3542201428 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:00.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.878+0000 7fc687fff640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc68811b140 0x7fc6881a79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.879+0000 7fc687fff640 1 -- 172.21.15.62:0/3542201428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6881abd50 msgr2=0x7fc6881ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:00.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.879+0000 7fc687fff640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6881abd50 0x7fc6881ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.879+0000 7fc687fff640 1 -- 172.21.15.62:0/3542201428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc688109830 msgr2=0x7fc6881a7480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T00:05:00.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.879+0000 7fc687fff640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc688109830 0x7fc6881a7480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.879+0000 7fc687fff640 1 -- 172.21.15.62:0/3542201428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6881ac810 con 0x7fc68811b140 2024-09-12T00:05:00.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.880+0000 7fc687fff640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc68811b140 0x7fc6881a79c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc67c005fd0 tx=0x7fc67c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.881+0000 7fc685ffb640 1 -- 172.21.15.62:0/3542201428 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c00cd00 con 0x7fc68811b140 2024-09-12T00:05:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.881+0000 7fc68ee2d640 1 -- 172.21.15.62:0/3542201428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6881acb00 con 0x7fc68811b140 2024-09-12T00:05:00.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.881+0000 7fc68ee2d640 1 -- 172.21.15.62:0/3542201428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6881b4330 con 0x7fc68811b140 2024-09-12T00:05:00.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.883+0000 7fc685ffb640 1 -- 172.21.15.62:0/3542201428 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c010910 con 0x7fc68811b140 2024-09-12T00:05:00.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.883+0000 7fc685ffb640 1 -- 172.21.15.62:0/3542201428 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c017790 con 0x7fc68811b140 2024-09-12T00:05:00.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.886+0000 7fc691206740 1 Processor -- start 2024-09-12T00:05:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.886+0000 7fc691206740 1 -- start start 2024-09-12T00:05:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.887+0000 7fc691206740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a8fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.887+0000 7fc691206740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8fe5d0 0x55765a8fea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.887+0000 7fc691206740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a909370 0x55765a90b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.887+0000 7fc691206740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55765a8642f0 con 0x55765a8fe5d0 2024-09-12T00:05:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.887+0000 7fc691206740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55765a80bcf0 con 0x55765a8ffb40 2024-09-12T00:05:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.887+0000 7fc691206740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55765a8602b0 con 0x55765a909370 2024-09-12T00:05:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.887+0000 7fc68d3ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a8fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.887+0000 7fc68d3ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a8fe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:41032/0 (socket says 172.21.15.62:41032) 2024-09-12T00:05:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.887+0000 7fc68d3ca640 1 -- 172.21.15.62:0/2845486731 learned_addr learned my addr 172.21.15.62:0/2845486731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.888+0000 7fc68d3ca640 1 -- 172.21.15.62:0/2845486731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a909370 msgr2=0x55765a90b760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:05:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.888+0000 7fc68d3ca640 1 --2- 172.21.15.62:0/2845486731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a909370 0x55765a90b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.888+0000 7fc68d3ca640 1 -- 172.21.15.62:0/2845486731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8fe5d0 msgr2=0x55765a8fea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.888+0000 7fc68d3ca640 1 --2- 172.21.15.62:0/2845486731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8fe5d0 0x55765a8fea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.888+0000 7fc68d3ca640 1 -- 172.21.15.62:0/2845486731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55765a7f03a0 con 0x55765a8ffb40 2024-09-12T00:05:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.890+0000 7fc685ffb640 1 -- 172.21.15.62:0/3542201428 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc67c010430 con 0x7fc68811b140 2024-09-12T00:05:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.891+0000 7fc68d3ca640 1 --2- 172.21.15.62:0/2845486731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a8fe090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc680009f20 tx=0x7fc680010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=794caa224b3a9a24 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.892+0000 7fc672ffd640 1 -- 172.21.15.62:0/2845486731 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680016070 con 0x55765a8ffb40 2024-09-12T00:05:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.892+0000 7fc672ffd640 1 -- 172.21.15.62:0/2845486731 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc680012930 con 0x55765a8ffb40 2024-09-12T00:05:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.892+0000 7fc672ffd640 1 -- 172.21.15.62:0/2845486731 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6800119d0 con 0x55765a8ffb40 2024-09-12T00:05:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.893+0000 7fc691206740 1 -- 172.21.15.62:0/2845486731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 msgr2=0x55765a8fe090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.894+0000 7fc691206740 1 --2- 172.21.15.62:0/2845486731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a8fe090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc680009f20 tx=0x7fc680010800 comp rx=0 tx=0).stop 2024-09-12T00:05:00.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.894+0000 7fc685ffb640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc6780a2db0 0x7fc6780a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.895+0000 7fc685ffb640 1 -- 172.21.15.62:0/3542201428 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fc67c0cf6b0 con 0x7fc68811b140 2024-09-12T00:05:00.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.899+0000 7fc68d3ca640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc6780a2db0 0x7fc6780a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.899+0000 7fc691206740 1 -- 172.21.15.62:0/2845486731 shutdown_connections 2024-09-12T00:05:00.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.899+0000 7fc691206740 1 --2- 172.21.15.62:0/2845486731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a909370 0x55765a90b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.899+0000 7fc691206740 1 --2- 172.21.15.62:0/2845486731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8fe5d0 0x55765a8fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.899+0000 7fc691206740 1 --2- 172.21.15.62:0/2845486731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a8fe090 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.899+0000 7fc691206740 1 -- 172.21.15.62:0/2845486731 >> 172.21.15.62:0/2845486731 conn(0x55765a8c7310 msgr2=0x55765a8fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:00.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.901+0000 7fc68d3ca640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc6780a2db0 0x7fc6780a5270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc680010d00 tx=0x7fc680009a30 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:00.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.901+0000 7fc691206740 1 -- 172.21.15.62:0/2845486731 shutdown_connections 2024-09-12T00:05:00.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.901+0000 7fc691206740 1 -- 172.21.15.62:0/2845486731 wait complete. 2024-09-12T00:05:00.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.902+0000 7fc691206740 1 -- start start 2024-09-12T00:05:00.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.902+0000 7fc691206740 1 Processor -- start 2024-09-12T00:05:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.906+0000 7fc691206740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8fe5d0 0x55765a908d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.906+0000 7fc691206740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8ffb40 0x55765a906010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.906+0000 7fc691206740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 0x55765a906600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.906+0000 7fc691206740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55765a90c070 con 0x55765a8fe5d0 2024-09-12T00:05:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.906+0000 7fc691206740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55765a81f060 con 0x55765a909370 2024-09-12T00:05:00.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.906+0000 7fc691206740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55765a90c1f0 con 0x55765a8ffb40 2024-09-12T00:05:00.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.906+0000 7fc68d3ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 0x55765a906600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.906+0000 7fc691206740 10 client.0 _ll_register_callbacks cb 0x55765a5728c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.907+0000 7fc68d3ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 0x55765a906600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:41038/0 (socket says 172.21.15.62:41038) 2024-09-12T00:05:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.907+0000 7fc68d3ca640 1 -- 172.21.15.62:0/3581436228 learned_addr learned my addr 172.21.15.62:0/3581436228 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.909+0000 7fc68cbc9640 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8fe5d0 0x55765a908d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.910+0000 7fc687fff640 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8ffb40 0x55765a906010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.910+0000 7fc68d3ca640 1 -- 172.21.15.62:0/3581436228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8ffb40 msgr2=0x55765a906010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.910+0000 7fc68d3ca640 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8ffb40 0x55765a906010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.910+0000 7fc68d3ca640 1 -- 172.21.15.62:0/3581436228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8fe5d0 msgr2=0x55765a908d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.910+0000 7fc68d3ca640 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8fe5d0 0x55765a908d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.910+0000 7fc68d3ca640 1 -- 172.21.15.62:0/3581436228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55765a5d53c0 con 0x55765a909370 2024-09-12T00:05:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.911+0000 7fc687fff640 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8ffb40 0x55765a906010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:05:00.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.913+0000 7fc68d3ca640 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 0x55765a906600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc68001a000 tx=0x7fc6800115b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3035b30aea2516ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:00.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.914+0000 7fc667fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:05:00.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.915+0000 7fc667fff640 1 -- 172.21.15.62:0/3581436228 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680016070 con 0x55765a909370 2024-09-12T00:05:00.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.915+0000 7fc667fff640 1 -- 172.21.15.62:0/3581436228 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fc680004c90 con 0x55765a909370 2024-09-12T00:05:00.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.918+0000 7fc691206740 1 -- 172.21.15.62:0/3581436228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc680012930 con 0x55765a909370 2024-09-12T00:05:00.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.920+0000 7fc667fff640 1 -- 172.21.15.62:0/3581436228 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc680005040 con 0x55765a909370 2024-09-12T00:05:00.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.922+0000 7fc691206740 10 client.27200 fetch_fsmap learned FSMap version 36 2024-09-12T00:05:00.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.922+0000 7fc691206740 1 -- 172.21.15.62:0/3581436228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55765a917ba0 con 0x55765a909370 2024-09-12T00:05:00.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.924+0000 7fc667fff640 1 -- 172.21.15.62:0/3581436228 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc680012930 con 0x55765a909370 2024-09-12T00:05:00.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.924+0000 7fc667fff640 1 -- 172.21.15.62:0/3581436228 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fc68000f040 con 0x55765a909370 2024-09-12T00:05:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.924+0000 7fc691206740 10 client.27200 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:05:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.924+0000 7fc691206740 10 client.27200 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:05:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.924+0000 7fc691206740 20 client.27200 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.924+0000 7fc691206740 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55765a918fa0 0x55765a91b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.924+0000 7fc691206740 4 client.27200 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.924+0000 7fc691206740 1 -- 172.21.15.62:0/3581436228 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55765a871860 con 0x55765a918fa0 2024-09-12T00:05:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.926+0000 7fc68cbc9640 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55765a918fa0 0x55765a91b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.935+0000 7fc68cbc9640 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55765a918fa0 0x55765a91b3b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3cc7f26de332d4 server_cookie=802f65354e7faf59 in_seq=0 out_seq=0 2024-09-12T00:05:00.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.936+0000 7fc667fff640 10 client.27200 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:05:00.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.940+0000 7fc667fff640 1 -- 172.21.15.62:0/3581436228 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55765a871860 con 0x55765a918fa0 2024-09-12T00:05:00.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.940+0000 7fc667fff640 10 client.27200 handle_command_reply: tid=0 2024-09-12T00:05:00.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.940+0000 7fc691206740 1 client.27200 shutdown 2024-09-12T00:05:00.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.943+0000 7fc691206740 1 -- 172.21.15.62:0/3581436228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 msgr2=0x55765a906600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:00.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.943+0000 7fc691206740 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 0x55765a906600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc68001a000 tx=0x7fc6800115b0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.944+0000 7fc691206740 1 -- 172.21.15.62:0/3581436228 shutdown_connections 2024-09-12T00:05:00.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.944+0000 7fc691206740 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55765a918fa0 0x55765a91b3b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.944+0000 7fc691206740 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 0x55765a906600 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.944+0000 7fc691206740 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8ffb40 0x55765a906010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.944+0000 7fc691206740 1 --2- 172.21.15.62:0/3581436228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8fe5d0 0x55765a908d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.944+0000 7fc667fff640 0 client.27200 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:05:00.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.945+0000 7fc691206740 1 -- 172.21.15.62:0/3581436228 >> 172.21.15.62:0/3581436228 conn(0x55765a8c7310 msgr2=0x55765a900930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:00.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.947+0000 7fc691206740 1 -- 172.21.15.62:0/3581436228 shutdown_connections 2024-09-12T00:05:00.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.949+0000 7fc691206740 1 -- 172.21.15.62:0/3581436228 wait complete. 2024-09-12T00:05:00.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.949+0000 7fc691206740 20 client.27200 trim_cache size 0 max 16384 2024-09-12T00:05:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:00 smithi062 ceph-mon[30958]: pgmap v402: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 554 op/s 2024-09-12T00:05:00.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.976+0000 7fc691206740 1 Processor -- start 2024-09-12T00:05:00.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.977+0000 7fc691206740 1 -- start start 2024-09-12T00:05:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.981+0000 7fc691206740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 0x55765a907f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.981+0000 7fc691206740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a901120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.981+0000 7fc691206740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a909370 0x55765a901660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.981+0000 7fc691206740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c011190 con 0x55765a909370 2024-09-12T00:05:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.981+0000 7fc691206740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c00ea00 con 0x55765a8ffb40 2024-09-12T00:05:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.981+0000 7fc691206740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c00c870 con 0x55765a8fe5d0 2024-09-12T00:05:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.981+0000 7fc68cbc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 0x55765a907f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.982+0000 7fc68cbc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 0x55765a907f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:55012/0 (socket says 172.21.15.62:55012) 2024-09-12T00:05:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.982+0000 7fc68cbc9640 1 -- 172.21.15.62:0/1363961402 learned_addr learned my addr 172.21.15.62:0/1363961402 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.982+0000 7fc687fff640 1 --2- 172.21.15.62:0/1363961402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a901120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.982+0000 7fc68d3ca640 1 --2- 172.21.15.62:0/1363961402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a909370 0x55765a901660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.984+0000 7fc687fff640 1 -- 172.21.15.62:0/1363961402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 msgr2=0x55765a907f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.984+0000 7fc687fff640 1 --2- 172.21.15.62:0/1363961402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 0x55765a907f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.984+0000 7fc687fff640 1 -- 172.21.15.62:0/1363961402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a909370 msgr2=0x55765a901660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.984+0000 7fc687fff640 1 --2- 172.21.15.62:0/1363961402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a909370 0x55765a901660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.984+0000 7fc687fff640 1 -- 172.21.15.62:0/1363961402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55765a8fee30 con 0x55765a8ffb40 2024-09-12T00:05:00.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.987+0000 7fc687fff640 1 --2- 172.21.15.62:0/1363961402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a901120 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0ccda0 tx=0x7fc67c09b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15a30524acc1c38b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:00.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.993+0000 7fc6727fc640 1 -- 172.21.15.62:0/1363961402 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c09bb70 con 0x55765a8ffb40 2024-09-12T00:05:00.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.993+0000 7fc6727fc640 1 -- 172.21.15.62:0/1363961402 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c09ff20 con 0x55765a8ffb40 2024-09-12T00:05:00.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.993+0000 7fc6727fc640 1 -- 172.21.15.62:0/1363961402 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0a4350 con 0x55765a8ffb40 2024-09-12T00:05:00.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.994+0000 7fc691206740 1 -- 172.21.15.62:0/1363961402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 msgr2=0x55765a901120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:00.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.994+0000 7fc691206740 1 --2- 172.21.15.62:0/1363961402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a901120 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0ccda0 tx=0x7fc67c09b7d0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.996+0000 7fc691206740 1 -- 172.21.15.62:0/1363961402 shutdown_connections 2024-09-12T00:05:00.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.996+0000 7fc691206740 1 --2- 172.21.15.62:0/1363961402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a909370 0x55765a901660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.996+0000 7fc691206740 1 --2- 172.21.15.62:0/1363961402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a8ffb40 0x55765a901120 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.996+0000 7fc691206740 1 --2- 172.21.15.62:0/1363961402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 0x55765a907f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:00.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:00.996+0000 7fc691206740 1 -- 172.21.15.62:0/1363961402 >> 172.21.15.62:0/1363961402 conn(0x55765a8c7310 msgr2=0x55765a907870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:01.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.000+0000 7fc691206740 1 -- 172.21.15.62:0/1363961402 shutdown_connections 2024-09-12T00:05:01.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.000+0000 7fc691206740 1 -- 172.21.15.62:0/1363961402 wait complete. 2024-09-12T00:05:01.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.000+0000 7fc691206740 1 -- start start 2024-09-12T00:05:01.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.000+0000 7fc691206740 1 Processor -- start 2024-09-12T00:05:01.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.001+0000 7fc691206740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 0x55765a911c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:01.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.001+0000 7fc691206740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8ffb40 0x55765a90cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:01.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.001+0000 7fc68cbc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 0x55765a911c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:01.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.002+0000 7fc687fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8ffb40 0x55765a90cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:01.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.002+0000 7fc687fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8ffb40 0x55765a90cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46156/0 (socket says 172.21.15.62:46156) 2024-09-12T00:05:01.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.002+0000 7fc68cbc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 0x55765a911c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:55024/0 (socket says 172.21.15.62:55024) 2024-09-12T00:05:01.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.002+0000 7fc687fff640 1 -- 172.21.15.62:0/1334935675 learned_addr learned my addr 172.21.15.62:0/1334935675 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.002+0000 7fc691206740 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 0x55765a90d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.002+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c00cec0 con 0x55765a8ffb40 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.002+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c00e330 con 0x55765a909370 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.002+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c00d590 con 0x55765a8fe5d0 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.002+0000 7fc68d3ca640 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 0x55765a90d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.003+0000 7fc687fff640 1 -- 172.21.15.62:0/1334935675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 msgr2=0x55765a911c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.003+0000 7fc687fff640 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 0x55765a911c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.003+0000 7fc687fff640 1 -- 172.21.15.62:0/1334935675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 msgr2=0x55765a90d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.003+0000 7fc687fff640 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 0x55765a90d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.003+0000 7fc687fff640 1 -- 172.21.15.62:0/1334935675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55765a90dc60 con 0x55765a8ffb40 2024-09-12T00:05:01.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.003+0000 7fc691206740 10 client.0 _ll_register_callbacks cb 0x7fc680009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:01.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.003+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc67c09ff20 con 0x55765a8ffb40 2024-09-12T00:05:01.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.003+0000 7fc687fff640 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8ffb40 0x55765a90cf40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00c6a0 tx=0x7fc67c09bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5292b7a6f2a0e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:01.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.003+0000 7fc667fff640 10 client.17679 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:01.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.004+0000 7fc667fff640 1 -- 172.21.15.62:0/1334935675 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c064020 con 0x55765a8ffb40 2024-09-12T00:05:01.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.004+0000 7fc667fff640 1 -- 172.21.15.62:0/1334935675 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fc67c0a4350 con 0x55765a8ffb40 2024-09-12T00:05:01.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.004+0000 7fc667fff640 1 -- 172.21.15.62:0/1334935675 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc67c0b9070 con 0x55765a8ffb40 2024-09-12T00:05:01.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.006+0000 7fc691206740 10 client.17679 fetch_fsmap learned FSMap version 36 2024-09-12T00:05:01.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.006+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55765a983a40 con 0x55765a8ffb40 2024-09-12T00:05:01.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.010+0000 7fc667fff640 1 -- 172.21.15.62:0/1334935675 <== mon.0 v2:172.21.15.62:3300/0 4 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fc67c0ba260 con 0x55765a8ffb40 2024-09-12T00:05:01.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.010+0000 7fc691206740 10 client.17679 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:05:01.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.010+0000 7fc691206740 10 client.17679 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:05:01.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.010+0000 7fc691206740 20 client.17679 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:01.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.010+0000 7fc691206740 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55765a983f40 0x55765a986380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:01.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.010+0000 7fc691206740 4 client.17679 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:05:01.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.010+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55765a86bd70 con 0x55765a983f40 2024-09-12T00:05:01.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.011+0000 7fc68cbc9640 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55765a983f40 0x55765a986380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:01.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.019+0000 7fc68cbc9640 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55765a983f40 0x55765a986380 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1644ee3635d317c0 server_cookie=ac6dbfa600ce160b in_seq=0 out_seq=0 2024-09-12T00:05:01.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.019+0000 7fc667fff640 10 client.17679 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:05:01.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc667fff640 1 -- 172.21.15.62:0/1334935675 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55765a86bd70 con 0x55765a983f40 2024-09-12T00:05:01.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc667fff640 10 client.17679 handle_command_reply: tid=0 2024-09-12T00:05:01.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc691206740 1 client.17679 shutdown 2024-09-12T00:05:01.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8ffb40 msgr2=0x55765a90cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:01.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc691206740 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8ffb40 0x55765a90cf40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00c6a0 tx=0x7fc67c09bbe0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 shutdown_connections 2024-09-12T00:05:01.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc691206740 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55765a983f40 0x55765a986380 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc691206740 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55765a909370 0x55765a90d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc691206740 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55765a8ffb40 0x55765a90cf40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc691206740 1 --2- 172.21.15.62:0/1334935675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55765a8fe5d0 0x55765a911c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.020+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 >> 172.21.15.62:0/1334935675 conn(0x55765a8c7310 msgr2=0x55765a908090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:01.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.021+0000 7fc667fff640 0 client.17679 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:05:01.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.021+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 shutdown_connections 2024-09-12T00:05:01.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.021+0000 7fc691206740 1 -- 172.21.15.62:0/1334935675 wait complete. 2024-09-12T00:05:01.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.021+0000 7fc691206740 20 client.17679 trim_cache size 0 max 16384 2024-09-12T00:05:01.024 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:05:01.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.022+0000 7fc68ee2d640 1 -- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc6780a2db0 msgr2=0x7fc6780a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:01.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.022+0000 7fc68ee2d640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc6780a2db0 0x7fc6780a5270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc680010d00 tx=0x7fc680009a30 comp rx=0 tx=0).stop 2024-09-12T00:05:01.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.023+0000 7fc68ee2d640 1 -- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc68811b140 msgr2=0x7fc6881a79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:01.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.023+0000 7fc68ee2d640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc68811b140 0x7fc6881a79c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc67c005fd0 tx=0x7fc67c00ee50 comp rx=0 tx=0).stop 2024-09-12T00:05:01.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.023+0000 7fc68ee2d640 1 -- 172.21.15.62:0/3542201428 shutdown_connections 2024-09-12T00:05:01.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.023+0000 7fc68ee2d640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc6780a2db0 0x7fc6780a5270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.023+0000 7fc68ee2d640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6881abd50 0x7fc6881ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.023+0000 7fc68ee2d640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc68811b140 0x7fc6881a79c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.023+0000 7fc68ee2d640 1 --2- 172.21.15.62:0/3542201428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc688109830 0x7fc6881a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:01.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.023+0000 7fc68ee2d640 1 -- 172.21.15.62:0/3542201428 >> 172.21.15.62:0/3542201428 conn(0x7fc68806e140 msgr2=0x7fc68811c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:01.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.024+0000 7fc68ee2d640 1 -- 172.21.15.62:0/3542201428 shutdown_connections 2024-09-12T00:05:01.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:01.024+0000 7fc68ee2d640 1 -- 172.21.15.62:0/3542201428 wait complete. 2024-09-12T00:05:01.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:00 smithi120 ceph-mon[35182]: pgmap v402: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 554 op/s 2024-09-12T00:05:01.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:00 smithi083 ceph-mon[34805]: pgmap v402: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 554 op/s 2024-09-12T00:05:01.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:05:02.147 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:02.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 740.0 s, 1087.6 tps, lat 91.280 ms stddev 138.985 2024-09-12T00:05:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.705+0000 7f781ab81640 1 -- 172.21.15.62:0/4204366106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7814103650 msgr2=0x7f781410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.705+0000 7f781ab81640 1 --2- 172.21.15.62:0/4204366106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7814103650 0x7f781410bb40 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7800009900 tx=0x7f780002a480 comp rx=0 tx=0).stop 2024-09-12T00:05:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.706+0000 7f781ab81640 1 -- 172.21.15.62:0/4204366106 shutdown_connections 2024-09-12T00:05:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.706+0000 7f781ab81640 1 --2- 172.21.15.62:0/4204366106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f781410c160 0x7f781410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.706+0000 7f781ab81640 1 --2- 172.21.15.62:0/4204366106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7814103650 0x7f781410bb40 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.706+0000 7f781ab81640 1 --2- 172.21.15.62:0/4204366106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814102d30 0x7f7814103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.706+0000 7f781ab81640 1 -- 172.21.15.62:0/4204366106 >> 172.21.15.62:0/4204366106 conn(0x7f78140fc790 msgr2=0x7f78140febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.706+0000 7f781ab81640 1 -- 172.21.15.62:0/4204366106 shutdown_connections 2024-09-12T00:05:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.706+0000 7f781ab81640 1 -- 172.21.15.62:0/4204366106 wait complete. 2024-09-12T00:05:02.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.706+0000 7f781ab81640 1 Processor -- start 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.706+0000 7f781ab81640 1 -- start start 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781ab81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7814102d30 0x7f7814073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781ab81640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7814103650 0x7f781406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781ab81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781410c160 0x7f781406c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781ab81640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7814111f60 con 0x7f7814103650 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781ab81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7814111de0 con 0x7f7814102d30 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781ab81640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78141120e0 con 0x7f781410c160 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781911e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781410c160 0x7f781406c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781911e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781410c160 0x7f781406c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:58348/0 (socket says 172.21.15.62:58348) 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781911e640 1 -- 172.21.15.62:0/1800340082 learned_addr learned my addr 172.21.15.62:0/1800340082 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781891d640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7814102d30 0x7f7814073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781911e640 1 -- 172.21.15.62:0/1800340082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7814102d30 msgr2=0x7f7814073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781911e640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7814102d30 0x7f7814073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781911e640 1 -- 172.21.15.62:0/1800340082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7814103650 msgr2=0x7f781406bce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781911e640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7814103650 0x7f781406bce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.707+0000 7f781911e640 1 -- 172.21.15.62:0/1800340082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781406ca40 con 0x7f781410c160 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.708+0000 7f781891d640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7814102d30 0x7f7814073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.708+0000 7f781911e640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781410c160 0x7f781406c220 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f780c030b20 tx=0x7f780c02e2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.708+0000 7f7811ffb640 1 -- 172.21.15.62:0/1800340082 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c004b20 con 0x7f781410c160 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.708+0000 7f781ab81640 1 -- 172.21.15.62:0/1800340082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7814100920 con 0x7f781410c160 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.709+0000 7f7811ffb640 1 -- 172.21.15.62:0/1800340082 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c004cc0 con 0x7f781410c160 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.709+0000 7f7811ffb640 1 -- 172.21.15.62:0/1800340082 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c0056e0 con 0x7f781410c160 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.709+0000 7f781ab81640 1 -- 172.21.15.62:0/1800340082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7814100e60 con 0x7f781410c160 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.711+0000 7f7811ffb640 1 -- 172.21.15.62:0/1800340082 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f780c005880 con 0x7f781410c160 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f781cf5a740 1 Processor -- start 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f781cf5a740 1 -- start start 2024-09-12T00:05:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e20a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f7811ffb640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f78040ab650 0x7f78040adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e2b370 0x555590e2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f781cf5a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555590d862f0 con 0x555590e21b40 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f781cf5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555590d2dcf0 con 0x555590e2b370 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f781cf5a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555590d822b0 con 0x555590e205d0 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f7813fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.712+0000 7f7813fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e20090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38770/0 (socket says 172.21.15.62:38770) 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.713+0000 7f7813fff640 1 -- 172.21.15.62:0/50175862 learned_addr learned my addr 172.21.15.62:0/50175862 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.713+0000 7f7813fff640 1 --2- 172.21.15.62:0/50175862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e20a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.713+0000 7f7811ffb640 1 -- 172.21.15.62:0/1800340082 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f780c0f19e0 con 0x7f781410c160 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.713+0000 7f781911e640 1 --2- 172.21.15.62:0/50175862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e2b370 0x555590e2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.713+0000 7f7813fff640 1 -- 172.21.15.62:0/50175862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e2b370 msgr2=0x555590e2d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.713+0000 7f7813fff640 1 --2- 172.21.15.62:0/50175862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e2b370 0x555590e2d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.713+0000 7f7813fff640 1 -- 172.21.15.62:0/50175862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 msgr2=0x555590e20090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.713+0000 7f781891d640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f78040ab650 0x7f78040adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.713+0000 7f7813fff640 1 --2- 172.21.15.62:0/50175862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e20090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.713+0000 7f7813fff640 1 -- 172.21.15.62:0/50175862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555590d123a0 con 0x555590e205d0 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.714+0000 7f7813fff640 1 --2- 172.21.15.62:0/50175862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e20a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f78000096c0 tx=0x7f7800005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634254b7a28b257 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.714+0000 7f77feffd640 1 -- 172.21.15.62:0/50175862 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800035030 con 0x555590e205d0 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.715+0000 7f77feffd640 1 -- 172.21.15.62:0/50175862 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7800031880 con 0x555590e205d0 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.715+0000 7f77feffd640 1 -- 172.21.15.62:0/50175862 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800003df0 con 0x555590e205d0 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.715+0000 7f781cf5a740 1 -- 172.21.15.62:0/50175862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 msgr2=0x555590e20a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.715+0000 7f781891d640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f78040ab650 0x7f78040adb10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7808009b50 tx=0x7f7808005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.715+0000 7f781cf5a740 1 --2- 172.21.15.62:0/50175862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e20a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f78000096c0 tx=0x7f7800005fd0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.715+0000 7f781cf5a740 1 -- 172.21.15.62:0/50175862 shutdown_connections 2024-09-12T00:05:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.715+0000 7f781cf5a740 1 --2- 172.21.15.62:0/50175862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e2b370 0x555590e2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.715+0000 7f781cf5a740 1 --2- 172.21.15.62:0/50175862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e20a10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.715+0000 7f781cf5a740 1 --2- 172.21.15.62:0/50175862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e20090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.715+0000 7f781cf5a740 1 -- 172.21.15.62:0/50175862 >> 172.21.15.62:0/50175862 conn(0x555590de9310 msgr2=0x555590e1cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.716+0000 7f781cf5a740 1 -- 172.21.15.62:0/50175862 shutdown_connections 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.716+0000 7f781cf5a740 1 -- 172.21.15.62:0/50175862 wait complete. 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.716+0000 7f781cf5a740 1 -- start start 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.716+0000 7f781cf5a740 1 Processor -- start 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.716+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e2ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.716+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 0x555590e28040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e2b370 0x555590e28650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781891d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e2ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781cf5a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555590e2e010 con 0x555590e2b370 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781cf5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555590d41060 con 0x555590e21b40 2024-09-12T00:05:02.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781cf5a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555590e2e190 con 0x555590e205d0 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781891d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e2ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:58372/0 (socket says 172.21.15.62:58372) 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781891d640 1 -- 172.21.15.62:0/1229382048 learned_addr learned my addr 172.21.15.62:0/1229382048 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781911e640 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e2b370 0x555590e28650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f7813fff640 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 0x555590e28040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781cf5a740 10 client.0 _ll_register_callbacks cb 0x555590a948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781891d640 1 -- 172.21.15.62:0/1229382048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 msgr2=0x555590e28040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781891d640 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 0x555590e28040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781891d640 1 -- 172.21.15.62:0/1229382048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e2b370 msgr2=0x555590e28650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781891d640 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e2b370 0x555590e28650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781891d640 1 -- 172.21.15.62:0/1229382048 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555590af73c0 con 0x555590e205d0 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f781891d640 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e2ad90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7808010fd0 tx=0x7f78080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30e432bee7ef1da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.717+0000 7f77f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:05:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f77f3fff640 1 -- 172.21.15.62:0/1229382048 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808023070 con 0x555590e205d0 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f77f3fff640 1 -- 172.21.15.62:0/1229382048 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f780801f650 con 0x555590e205d0 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f781cf5a740 1 -- 172.21.15.62:0/1229382048 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7800031880 con 0x555590e205d0 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f77f3fff640 1 -- 172.21.15.62:0/1229382048 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f780802f050 con 0x555590e205d0 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f781cf5a740 10 client.27109 fetch_fsmap learned FSMap version 36 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f781cf5a740 1 -- 172.21.15.62:0/1229382048 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555590e39a70 con 0x555590e205d0 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f77f3fff640 1 -- 172.21.15.62:0/1229382048 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7800031880 con 0x555590e205d0 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f77f3fff640 1 -- 172.21.15.62:0/1229382048 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f780800f040 con 0x555590e205d0 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f781cf5a740 10 client.27109 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f781cf5a740 10 client.27109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f781cf5a740 20 client.27109 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555590e3aed0 0x555590e3d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f781cf5a740 4 client.27109 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f781cf5a740 1 -- 172.21.15.62:0/1229382048 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555590d93860 con 0x555590e3aed0 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.718+0000 7f7813fff640 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555590e3aed0 0x555590e3d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.719+0000 7f7813fff640 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555590e3aed0 0x555590e3d340 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a39893905e4b5a server_cookie=7cb2105ba9d996bf in_seq=0 out_seq=0 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.719+0000 7f77f3fff640 10 client.27109 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f77f3fff640 1 -- 172.21.15.62:0/1229382048 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555590d93860 con 0x555590e3aed0 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f77f3fff640 10 client.27109 handle_command_reply: tid=0 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f781cf5a740 1 client.27109 shutdown 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f781cf5a740 1 -- 172.21.15.62:0/1229382048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 msgr2=0x555590e2ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e2ad90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7808010fd0 tx=0x7f78080046c0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f781cf5a740 1 -- 172.21.15.62:0/1229382048 shutdown_connections 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555590e3aed0 0x555590e3d340 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f77f3fff640 0 client.27109 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e2b370 0x555590e28650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 0x555590e28040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1229382048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e2ad90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.721+0000 7f781cf5a740 1 -- 172.21.15.62:0/1229382048 >> 172.21.15.62:0/1229382048 conn(0x555590de9310 msgr2=0x555590e22960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.722+0000 7f781cf5a740 1 -- 172.21.15.62:0/1229382048 shutdown_connections 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.722+0000 7f781cf5a740 1 -- 172.21.15.62:0/1229382048 wait complete. 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.722+0000 7f781cf5a740 20 client.27109 trim_cache size 0 max 16384 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781cf5a740 1 Processor -- start 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781cf5a740 1 -- start start 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e205d0 0x555590e1bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 0x555590e1c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e2b370 0x555590e23120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781cf5a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f78141a1730 con 0x555590e205d0 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781cf5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f781419efa0 con 0x555590e21b40 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781cf5a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f781419ce10 con 0x555590e2b370 2024-09-12T00:05:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781891d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e205d0 0x555590e1bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f7813fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 0x555590e1c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781891d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e205d0 0x555590e1bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38786/0 (socket says 172.21.15.62:38786) 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781891d640 1 -- 172.21.15.62:0/4179428106 learned_addr learned my addr 172.21.15.62:0/4179428106 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f7813fff640 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 0x555590e1c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:51244/0 (socket says 172.21.15.62:51244) 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.739+0000 7f781911e640 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e2b370 0x555590e23120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f781891d640 1 -- 172.21.15.62:0/4179428106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e2b370 msgr2=0x555590e23120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f781891d640 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e2b370 0x555590e23120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f781891d640 1 -- 172.21.15.62:0/4179428106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 msgr2=0x555590e1c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f781891d640 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 0x555590e1c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f781891d640 1 -- 172.21.15.62:0/4179428106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555590e28fc0 con 0x555590e205d0 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f781911e640 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e2b370 0x555590e23120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f7813fff640 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 0x555590e1c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f781891d640 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e205d0 0x555590e1bbf0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7808010800 tx=0x7f7808009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3af5d26ad8a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f77fe7fc640 1 -- 172.21.15.62:0/4179428106 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808037020 con 0x555590e205d0 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f77fe7fc640 1 -- 172.21.15.62:0/4179428106 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780801f650 con 0x555590e205d0 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.740+0000 7f77fe7fc640 1 -- 172.21.15.62:0/4179428106 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808023070 con 0x555590e205d0 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.741+0000 7f781cf5a740 1 -- 172.21.15.62:0/4179428106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e205d0 msgr2=0x555590e1bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.741+0000 7f781cf5a740 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e205d0 0x555590e1bbf0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7808010800 tx=0x7f7808009920 comp rx=0 tx=0).stop 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.741+0000 7f781cf5a740 1 -- 172.21.15.62:0/4179428106 shutdown_connections 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.741+0000 7f781cf5a740 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e2b370 0x555590e23120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.741+0000 7f781cf5a740 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e21b40 0x555590e1c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.741+0000 7f781cf5a740 1 --2- 172.21.15.62:0/4179428106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e205d0 0x555590e1bbf0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.741+0000 7f781cf5a740 1 -- 172.21.15.62:0/4179428106 >> 172.21.15.62:0/4179428106 conn(0x555590de9310 msgr2=0x555590e1b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.741+0000 7f781cf5a740 1 -- 172.21.15.62:0/4179428106 shutdown_connections 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.741+0000 7f781cf5a740 1 -- 172.21.15.62:0/4179428106 wait complete. 2024-09-12T00:05:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f781cf5a740 1 -- start start 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f781cf5a740 1 Processor -- start 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e250c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e25600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f781cf5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e2b370 0x555590e25b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f781cf5a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f781419d460 con 0x555590e21b40 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f781cf5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f781419e8d0 con 0x555590e2b370 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f781cf5a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f781419db30 con 0x555590e205d0 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f7813fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f7813fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e25600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38800/0 (socket says 172.21.15.62:38800) 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f7813fff640 1 -- 172.21.15.62:0/1858273001 learned_addr learned my addr 172.21.15.62:0/1858273001 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.742+0000 7f781cf5a740 10 client.0 _ll_register_callbacks cb 0x7f7808016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f781891d640 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e250c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f781911e640 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e2b370 0x555590e25b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f7813fff640 1 -- 172.21.15.62:0/1858273001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 msgr2=0x555590e250c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f7813fff640 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e250c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f7813fff640 1 -- 172.21.15.62:0/1858273001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e2b370 msgr2=0x555590e25b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f7813fff640 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e2b370 0x555590e25b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f7813fff640 1 -- 172.21.15.62:0/1858273001 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555590ea4c30 con 0x555590e21b40 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f7813fff640 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e25600 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f780002a9a0 tx=0x7f7800003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2081ebc861362474 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f77f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f77f3fff640 1 -- 172.21.15.62:0/1858273001 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800057060 con 0x555590e21b40 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f781cf5a740 1 -- 172.21.15.62:0/1858273001 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780801f650 con 0x555590e21b40 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f77f3fff640 1 -- 172.21.15.62:0/1858273001 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f7800052420 con 0x555590e21b40 2024-09-12T00:05:02.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f77f3fff640 1 -- 172.21.15.62:0/1858273001 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7800035040 con 0x555590e21b40 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f781cf5a740 10 client.17694 fetch_fsmap learned FSMap version 36 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f781cf5a740 1 -- 172.21.15.62:0/1858273001 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555590ea5f10 con 0x555590e21b40 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.743+0000 7f77f3fff640 1 -- 172.21.15.62:0/1858273001 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780801f650 con 0x555590e21b40 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.744+0000 7f77f3fff640 1 -- 172.21.15.62:0/1858273001 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f780003a040 con 0x555590e21b40 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.744+0000 7f781cf5a740 10 client.17694 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.744+0000 7f781cf5a740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.744+0000 7f781cf5a740 20 client.17694 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.744+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555590e3aed0 0x555590ea8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.744+0000 7f781cf5a740 4 client.17694 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.744+0000 7f781cf5a740 1 -- 172.21.15.62:0/1858273001 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555590d8dd70 con 0x555590e3aed0 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.745+0000 7f781891d640 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555590e3aed0 0x555590ea8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.745+0000 7f781891d640 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555590e3aed0 0x555590ea8200 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8fc398e8a82fed server_cookie=97b559532723f3cc in_seq=0 out_seq=0 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.745+0000 7f77f3fff640 10 client.17694 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.745+0000 7f77f3fff640 1 -- 172.21.15.62:0/1858273001 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555590d8dd70 con 0x555590e3aed0 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.746+0000 7f77f3fff640 10 client.17694 handle_command_reply: tid=0 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.746+0000 7f781cf5a740 1 client.17694 shutdown 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.746+0000 7f781cf5a740 1 -- 172.21.15.62:0/1858273001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 msgr2=0x555590e25600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.746+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e25600 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f780002a9a0 tx=0x7f7800003930 comp rx=0 tx=0).stop 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.746+0000 7f781cf5a740 1 -- 172.21.15.62:0/1858273001 shutdown_connections 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.746+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555590e3aed0 0x555590ea8200 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.746+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555590e2b370 0x555590e25b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.746+0000 7f77f3fff640 0 client.17694 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.746+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555590e21b40 0x555590e25600 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.746+0000 7f781cf5a740 1 --2- 172.21.15.62:0/1858273001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555590e205d0 0x555590e250c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.747+0000 7f781cf5a740 1 -- 172.21.15.62:0/1858273001 >> 172.21.15.62:0/1858273001 conn(0x555590de9310 msgr2=0x555590e1bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:02.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.747+0000 7f781cf5a740 1 -- 172.21.15.62:0/1858273001 shutdown_connections 2024-09-12T00:05:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:05:02.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.747+0000 7f781cf5a740 1 -- 172.21.15.62:0/1858273001 wait complete. 2024-09-12T00:05:02.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.747+0000 7f781cf5a740 20 client.17694 trim_cache size 0 max 16384 2024-09-12T00:05:02.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 -- 172.21.15.62:0/1800340082 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f78040ab650 msgr2=0x7f78040adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:02.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f78040ab650 0x7f78040adb10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7808009b50 tx=0x7f7808005f90 comp rx=0 tx=0).stop 2024-09-12T00:05:02.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 -- 172.21.15.62:0/1800340082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781410c160 msgr2=0x7f781406c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:02.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781410c160 0x7f781406c220 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f780c030b20 tx=0x7f780c02e2e0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 -- 172.21.15.62:0/1800340082 shutdown_connections 2024-09-12T00:05:02.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f78040ab650 0x7f78040adb10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781410c160 0x7f781406c220 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7814103650 0x7f781406bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 --2- 172.21.15.62:0/1800340082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7814102d30 0x7f7814073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:02.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 -- 172.21.15.62:0/1800340082 >> 172.21.15.62:0/1800340082 conn(0x7f78140fc790 msgr2=0x7f78140fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:02.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 -- 172.21.15.62:0/1800340082 shutdown_connections 2024-09-12T00:05:02.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:02.748+0000 7f781ab81640 1 -- 172.21.15.62:0/1800340082 wait complete. 2024-09-12T00:05:03.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:02 smithi120 ceph-mon[35182]: pgmap v403: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.7 MiB/s wr, 514 op/s 2024-09-12T00:05:03.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:02 smithi083 ceph-mon[34805]: pgmap v403: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.7 MiB/s wr, 514 op/s 2024-09-12T00:05:03.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:02 smithi062 ceph-mon[30958]: pgmap v403: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.7 MiB/s wr, 514 op/s 2024-09-12T00:05:04.765 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:05:04.765 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:05:05.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:04 smithi120 ceph-mon[35182]: pgmap v404: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-09-12T00:05:05.134 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:05.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:04 smithi083 ceph-mon[34805]: pgmap v404: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-09-12T00:05:05.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:04 smithi062 ceph-mon[30958]: pgmap v404: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-09-12T00:05:06.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.957+0000 7f0d899cf640 1 -- 172.21.15.62:0/638184784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84100a70 msgr2=0x7f0d84100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:06.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.957+0000 7f0d899cf640 1 --2- 172.21.15.62:0/638184784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84100a70 0x7f0d84100ed0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009930 tx=0x7f0d7802a420 comp rx=0 tx=0).stop 2024-09-12T00:05:06.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.957+0000 7f0d899cf640 1 -- 172.21.15.62:0/638184784 shutdown_connections 2024-09-12T00:05:06.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.957+0000 7f0d899cf640 1 --2- 172.21.15.62:0/638184784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84101410 0x7f0d8410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.957+0000 7f0d899cf640 1 --2- 172.21.15.62:0/638184784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84100a70 0x7f0d84100ed0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.957+0000 7f0d899cf640 1 --2- 172.21.15.62:0/638184784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d84106a70 0x7f0d84106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.957+0000 7f0d899cf640 1 -- 172.21.15.62:0/638184784 >> 172.21.15.62:0/638184784 conn(0x7f0d840fc7d0 msgr2=0x7f0d840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:06.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.958+0000 7f0d899cf640 1 -- 172.21.15.62:0/638184784 shutdown_connections 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.958+0000 7f0d899cf640 1 -- 172.21.15.62:0/638184784 wait complete. 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.958+0000 7f0d899cf640 1 Processor -- start 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d899cf640 1 -- start start 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d899cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84100a70 0x7f0d84198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d899cf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d84101410 0x7f0d841989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d899cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84106a70 0x7f0d8419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d899cf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84110e30 con 0x7f0d84106a70 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d899cf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84110cb0 con 0x7f0d84100a70 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d899cf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84110fb0 con 0x7f0d84101410 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84106a70 0x7f0d8419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d82ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84100a70 0x7f0d84198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d827fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d84101410 0x7f0d841989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84106a70 0x7f0d8419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38804/0 (socket says 172.21.15.62:38804) 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d82ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84100a70 0x7f0d84198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:51270/0 (socket says 172.21.15.62:51270) 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d837fe640 1 -- 172.21.15.62:0/2147930014 learned_addr learned my addr 172.21.15.62:0/2147930014 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d82ffd640 1 -- 172.21.15.62:0/2147930014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d84101410 msgr2=0x7f0d841989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d82ffd640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d84101410 0x7f0d841989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d82ffd640 1 -- 172.21.15.62:0/2147930014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84106a70 msgr2=0x7f0d8419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d82ffd640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84106a70 0x7f0d8419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d82ffd640 1 -- 172.21.15.62:0/2147930014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8419d440 con 0x7f0d84100a70 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.959+0000 7f0d837fe640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84106a70 0x7f0d8419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.960+0000 7f0d82ffd640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84100a70 0x7f0d84198490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400d6d0 tx=0x7f0d7400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.960+0000 7f0d889cd640 1 -- 172.21.15.62:0/2147930014 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74017070 con 0x7f0d84100a70 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.960+0000 7f0d899cf640 1 -- 172.21.15.62:0/2147930014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8419d730 con 0x7f0d84100a70 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.960+0000 7f0d889cd640 1 -- 172.21.15.62:0/2147930014 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74009820 con 0x7f0d84100a70 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.960+0000 7f0d889cd640 1 -- 172.21.15.62:0/2147930014 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74004cd0 con 0x7f0d84100a70 2024-09-12T00:05:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.960+0000 7f0d899cf640 1 -- 172.21.15.62:0/2147930014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d841a5010 con 0x7f0d84100a70 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.963+0000 7f0d8bda8740 1 Processor -- start 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.963+0000 7f0d8bda8740 1 -- start start 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.963+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x561839056f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.963+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839057480 0x561839061800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.963+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x561839064130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.963+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561838fbc280 con 0x561839061d40 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.963+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561838f63b10 con 0x561839056b60 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.963+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561838fb8240 con 0x561839057480 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.964+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x561839056f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.964+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x561839056f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:51272/0 (socket says 172.21.15.62:51272) 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.964+0000 7f0d837fe640 1 -- 172.21.15.62:0/3119531498 learned_addr learned my addr 172.21.15.62:0/3119531498 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.964+0000 7f0d837fe640 1 -- 172.21.15.62:0/3119531498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839057480 msgr2=0x561839061800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:05:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.964+0000 7f0d837fe640 1 --2- 172.21.15.62:0/3119531498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839057480 0x561839061800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.964+0000 7f0d837fe640 1 -- 172.21.15.62:0/3119531498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 msgr2=0x561839064130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.964+0000 7f0d837fe640 1 --2- 172.21.15.62:0/3119531498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x561839064130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.964+0000 7f0d837fe640 1 -- 172.21.15.62:0/3119531498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561838f701f0 con 0x561839056b60 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.964+0000 7f0d837fe640 1 --2- 172.21.15.62:0/3119531498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x561839056f40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0118e0 tx=0x7f0d7c011da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd079a9888528f43 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.965+0000 7f0d69ffb640 1 -- 172.21.15.62:0/3119531498 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c01e070 con 0x561839056b60 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.965+0000 7f0d889cd640 1 -- 172.21.15.62:0/2147930014 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0d740099d0 con 0x7f0d84100a70 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.966+0000 7f0d69ffb640 1 -- 172.21.15.62:0/3119531498 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c019410 con 0x561839056b60 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.966+0000 7f0d69ffb640 1 -- 172.21.15.62:0/3119531498 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c018610 con 0x561839056b60 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d889cd640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0d6c0a2cf0 0x7f0d6c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d82ffd640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0d6c0a2cf0 0x7f0d6c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d889cd640 1 -- 172.21.15.62:0/2147930014 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f0d740ceef0 con 0x7f0d84100a70 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d8bda8740 1 -- 172.21.15.62:0/3119531498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 msgr2=0x561839056f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/3119531498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x561839056f40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0118e0 tx=0x7f0d7c011da0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d8bda8740 1 -- 172.21.15.62:0/3119531498 shutdown_connections 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/3119531498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x561839064130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/3119531498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839057480 0x561839061800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/3119531498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x561839056f40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d8bda8740 1 -- 172.21.15.62:0/3119531498 >> 172.21.15.62:0/3119531498 conn(0x5618390530f0 msgr2=0x5618390534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d8bda8740 1 -- 172.21.15.62:0/3119531498 shutdown_connections 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.967+0000 7f0d82ffd640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0d6c0a2cf0 0x7f0d6c0a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x561839053f50 tx=0x7f0d74016b40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.968+0000 7f0d8bda8740 1 -- 172.21.15.62:0/3119531498 wait complete. 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.968+0000 7f0d8bda8740 1 -- start start 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.968+0000 7f0d8bda8740 1 Processor -- start 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.968+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x56183905ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.968+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839057480 0x56183905f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.968+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839061d40 0x56183905f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.968+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561839068d20 con 0x561839057480 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.968+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561838f76f10 con 0x561839061d40 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.968+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561839068ea0 con 0x561839056b60 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.969+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x56183905ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.969+0000 7f0d827fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839057480 0x56183905f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.969+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x56183905ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:58424/0 (socket says 172.21.15.62:58424) 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.969+0000 7f0d837fe640 1 -- 172.21.15.62:0/1638741948 learned_addr learned my addr 172.21.15.62:0/1638741948 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.968+0000 7f0d8bda8740 10 client.0 _ll_register_callbacks cb 0x561838f3fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.969+0000 7f0d837fe640 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839061d40 0x56183905f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.969+0000 7f0d837fe640 1 -- 172.21.15.62:0/1638741948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839061d40 msgr2=0x56183905f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.969+0000 7f0d837fe640 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839061d40 0x56183905f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.969+0000 7f0d837fe640 1 -- 172.21.15.62:0/1638741948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839057480 msgr2=0x56183905f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.969+0000 7f0d837fe640 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839057480 0x56183905f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.969+0000 7f0d837fe640 1 -- 172.21.15.62:0/1638741948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561838d2d6a0 con 0x561839056b60 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.970+0000 7f0d837fe640 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x56183905ec40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00f7b0 tx=0x7f0d7c019e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a23c4f1541e94eac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.970+0000 7f0d62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.970+0000 7f0d62ffd640 1 -- 172.21.15.62:0/1638741948 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c01e030 con 0x561839056b60 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.970+0000 7f0d62ffd640 1 -- 172.21.15.62:0/1638741948 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f0d7c00fc90 con 0x561839056b60 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.970+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1638741948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d7c019410 con 0x561839056b60 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.970+0000 7f0d62ffd640 1 -- 172.21.15.62:0/1638741948 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0d7c036050 con 0x561839056b60 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.970+0000 7f0d8bda8740 10 client.27124 fetch_fsmap learned FSMap version 36 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.970+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1638741948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561839074910 con 0x561839056b60 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.971+0000 7f0d62ffd640 1 -- 172.21.15.62:0/1638741948 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c019410 con 0x561839056b60 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.971+0000 7f0d62ffd640 1 -- 172.21.15.62:0/1638741948 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f0d7c0313b0 con 0x561839056b60 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.971+0000 7f0d8bda8740 10 client.27124 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.971+0000 7f0d8bda8740 10 client.27124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.971+0000 7f0d8bda8740 20 client.27124 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.971+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561839076080 0x5618390784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.971+0000 7f0d8bda8740 4 client.27124 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.971+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1638741948 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561838fc97f0 con 0x561839076080 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.971+0000 7f0d827fc640 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561839076080 0x5618390784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.972+0000 7f0d827fc640 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561839076080 0x5618390784e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a97e918ac5bfa0 server_cookie=a091e3de91c96bb6 in_seq=0 out_seq=0 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.972+0000 7f0d62ffd640 10 client.27124 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d62ffd640 1 -- 172.21.15.62:0/1638741948 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561838fc97f0 con 0x561839076080 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d62ffd640 10 client.27124 handle_command_reply: tid=0 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 client.27124 shutdown 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1638741948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 msgr2=0x56183905ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x56183905ec40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00f7b0 tx=0x7f0d7c019e50 comp rx=0 tx=0).stop 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1638741948 shutdown_connections 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561839076080 0x5618390784e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839061d40 0x56183905f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839057480 0x56183905f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/1638741948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x56183905ec40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1638741948 >> 172.21.15.62:0/1638741948 conn(0x5618390530f0 msgr2=0x561839052270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d62ffd640 0 client.27124 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1638741948 shutdown_connections 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1638741948 wait complete. 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.973+0000 7f0d8bda8740 20 client.27124 trim_cache size 0 max 16384 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d8bda8740 1 Processor -- start 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d8bda8740 1 -- start start 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x5618390595c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839057480 0x56183907ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x56183907e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841ae150 con 0x561839061d40 2024-09-12T00:05:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841ab9c0 con 0x561839057480 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841a9830 con 0x561839056b60 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x5618390595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839057480 0x56183907ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839057480 0x56183907ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:51278/0 (socket says 172.21.15.62:51278) 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x5618390595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:58436/0 (socket says 172.21.15.62:58436) 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d837fe640 1 -- 172.21.15.62:0/1303430362 learned_addr learned my addr 172.21.15.62:0/1303430362 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.996+0000 7f0d837fe640 1 --2- 172.21.15.62:0/1303430362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x56183907e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.997+0000 7f0d837fe640 1 -- 172.21.15.62:0/1303430362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839057480 msgr2=0x56183907ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.997+0000 7f0d837fe640 1 --2- 172.21.15.62:0/1303430362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839057480 0x56183907ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.997+0000 7f0d837fe640 1 -- 172.21.15.62:0/1303430362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 msgr2=0x56183907e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.997+0000 7f0d837fe640 1 --2- 172.21.15.62:0/1303430362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x56183907e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.997+0000 7f0d837fe640 1 -- 172.21.15.62:0/1303430362 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56183905d190 con 0x561839056b60 2024-09-12T00:05:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.997+0000 7f0d837fe640 1 --2- 172.21.15.62:0/1303430362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x5618390595c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009560 tx=0x7f0d7c014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f13e0a0e7b181d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.997+0000 7f0d697fa640 1 -- 172.21.15.62:0/1303430362 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c01e070 con 0x561839056b60 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.997+0000 7f0d697fa640 1 -- 172.21.15.62:0/1303430362 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c02d070 con 0x561839056b60 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.997+0000 7f0d697fa640 1 -- 172.21.15.62:0/1303430362 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c03a910 con 0x561839056b60 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1303430362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 msgr2=0x5618390595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/1303430362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x5618390595c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009560 tx=0x7f0d7c014040 comp rx=0 tx=0).stop 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1303430362 shutdown_connections 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/1303430362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x56183907e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/1303430362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839057480 0x56183907ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/1303430362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839056b60 0x5618390595c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1303430362 >> 172.21.15.62:0/1303430362 conn(0x5618390530f0 msgr2=0x561839060bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1303430362 shutdown_connections 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 -- 172.21.15.62:0/1303430362 wait complete. 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 -- start start 2024-09-12T00:05:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.998+0000 7f0d8bda8740 1 Processor -- start 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x56183906a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839057480 0x56183906a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d8bda8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x5618390dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841a9e80 con 0x561839061d40 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841ab2f0 con 0x561839056b60 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d8bda8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841aa550 con 0x561839057480 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d8bda8740 10 client.0 _ll_register_callbacks cb 0x7f0d7c004ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x5618390dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x56183906a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d827fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839057480 0x56183906a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x56183906a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:51282/0 (socket says 172.21.15.62:51282) 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d837fe640 1 -- 172.21.15.62:0/2323030929 learned_addr learned my addr 172.21.15.62:0/2323030929 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:06.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d837fe640 1 -- 172.21.15.62:0/2323030929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839057480 msgr2=0x56183906a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d837fe640 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839057480 0x56183906a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d837fe640 1 -- 172.21.15.62:0/2323030929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 msgr2=0x5618390dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d837fe640 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x5618390dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d837fe640 1 -- 172.21.15.62:0/2323030929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56183906b020 con 0x561839056b60 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d827fc640 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839057480 0x56183906a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d837fe640 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x56183906a1e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009fa0 tx=0x7f0d7c011da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10e31f751af8f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:05.999+0000 7f0d62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.000+0000 7f0d62ffd640 1 -- 172.21.15.62:0/2323030929 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c04d060 con 0x561839056b60 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.000+0000 7f0d62ffd640 1 -- 172.21.15.62:0/2323030929 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f0d7c031420 con 0x561839056b60 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.000+0000 7f0d8bda8740 1 -- 172.21.15.62:0/2323030929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d7c02d070 con 0x561839056b60 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.000+0000 7f0d62ffd640 1 -- 172.21.15.62:0/2323030929 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0d7c01e070 con 0x561839056b60 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.000+0000 7f0d8bda8740 10 client.27239 fetch_fsmap learned FSMap version 36 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.000+0000 7f0d8bda8740 1 -- 172.21.15.62:0/2323030929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5618390e1470 con 0x561839056b60 2024-09-12T00:05:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.000+0000 7f0d62ffd640 1 -- 172.21.15.62:0/2323030929 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c02d070 con 0x561839056b60 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.001+0000 7f0d62ffd640 1 -- 172.21.15.62:0/2323030929 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f0d7c047040 con 0x561839056b60 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.001+0000 7f0d8bda8740 10 client.27239 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.001+0000 7f0d8bda8740 10 client.27239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.001+0000 7f0d8bda8740 20 client.27239 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.001+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5618390e1a20 0x5618390e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.001+0000 7f0d8bda8740 4 client.27239 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.001+0000 7f0d8bda8740 1 -- 172.21.15.62:0/2323030929 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561838fc3d00 con 0x5618390e1a20 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.006+0000 7f0d827fc640 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5618390e1a20 0x5618390e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.025+0000 7f0d827fc640 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5618390e1a20 0x5618390e3e40 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e69a731fb1d762 server_cookie=f1e3019b9574d3b3 in_seq=0 out_seq=0 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.025+0000 7f0d62ffd640 10 client.27239 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:06.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.029+0000 7f0d62ffd640 1 -- 172.21.15.62:0/2323030929 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561838fc3d00 con 0x5618390e1a20 2024-09-12T00:05:06.985 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:05:06.985 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-12T00:05:06.985 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "fe27fd46-3577-4469-b0a6-2200665e52c4", 2024-09-12T00:05:06.985 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-12T00:05:06.985 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:05:06.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.029+0000 7f0d62ffd640 10 client.27239 handle_command_reply: tid=0 2024-09-12T00:05:06.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.030+0000 7f0d8bda8740 1 client.27239 shutdown 2024-09-12T00:05:06.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.032+0000 7f0d8bda8740 1 -- 172.21.15.62:0/2323030929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 msgr2=0x56183906a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.032+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x56183906a1e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009fa0 tx=0x7f0d7c011da0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.032+0000 7f0d8bda8740 1 -- 172.21.15.62:0/2323030929 shutdown_connections 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.032+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5618390e1a20 0x5618390e3e40 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.032+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561839061d40 0x5618390dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.032+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561839057480 0x56183906a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.032+0000 7f0d8bda8740 1 --2- 172.21.15.62:0/2323030929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561839056b60 0x56183906a1e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.032+0000 7f0d8bda8740 1 -- 172.21.15.62:0/2323030929 >> 172.21.15.62:0/2323030929 conn(0x5618390530f0 msgr2=0x561839051a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.032+0000 7f0d62ffd640 0 client.27239 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.033+0000 7f0d8bda8740 1 -- 172.21.15.62:0/2323030929 shutdown_connections 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.034+0000 7f0d8bda8740 1 -- 172.21.15.62:0/2323030929 wait complete. 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.034+0000 7f0d8bda8740 20 client.27239 trim_cache size 0 max 16384 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.035+0000 7f0d899cf640 1 -- 172.21.15.62:0/2147930014 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0d6c0a2cf0 msgr2=0x7f0d6c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:06.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.035+0000 7f0d899cf640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0d6c0a2cf0 0x7f0d6c0a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x561839053f50 tx=0x7f0d74016b40 comp rx=0 tx=0).stop 2024-09-12T00:05:06.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.035+0000 7f0d899cf640 1 -- 172.21.15.62:0/2147930014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84100a70 msgr2=0x7f0d84198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:06.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.035+0000 7f0d899cf640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84100a70 0x7f0d84198490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400d6d0 tx=0x7f0d7400db90 comp rx=0 tx=0).stop 2024-09-12T00:05:06.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.036+0000 7f0d899cf640 1 -- 172.21.15.62:0/2147930014 shutdown_connections 2024-09-12T00:05:06.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.036+0000 7f0d899cf640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f0d6c0a2cf0 0x7f0d6c0a51b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.036+0000 7f0d899cf640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d84106a70 0x7f0d8419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.036+0000 7f0d899cf640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d84101410 0x7f0d841989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.036+0000 7f0d899cf640 1 --2- 172.21.15.62:0/2147930014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84100a70 0x7f0d84198490 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:06.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.036+0000 7f0d899cf640 1 -- 172.21.15.62:0/2147930014 >> 172.21.15.62:0/2147930014 conn(0x7f0d840fc7d0 msgr2=0x7f0d8406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:06.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.036+0000 7f0d899cf640 1 -- 172.21.15.62:0/2147930014 shutdown_connections 2024-09-12T00:05:06.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:06.036+0000 7f0d899cf640 1 -- 172.21.15.62:0/2147930014 wait complete. 2024-09-12T00:05:07.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:06 smithi083 ceph-mon[34805]: pgmap v405: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-09-12T00:05:07.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:06 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-12T00:05:07.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:06 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:07.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:06 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:05:07.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:06 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:05:07.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:06 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:07.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:06 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:06 smithi062 ceph-mon[30958]: pgmap v405: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-09-12T00:05:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:06 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-12T00:05:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:06 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:06 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:05:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:06 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:05:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:06 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:06 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:07.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:06 smithi120 ceph-mon[35182]: pgmap v405: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-09-12T00:05:07.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:06 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-12T00:05:07.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:06 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:07.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:06 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:05:07.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:06 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:05:07.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:06 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:07.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:06 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 745.0 s, 1735.4 tps, lat 58.430 ms stddev 66.075 2024-09-12T00:05:08.009 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:07 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:05:08.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:07 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:08.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:07 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:05:08.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:07 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:08.046 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:05:08.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:07 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:05:08.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:07 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:08.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:07 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:05:08.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:07 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:08.271 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:07 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:05:08.271 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:07 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:08.271 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:07 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:05:08.271 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:07 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:08.440 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:09.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:08 smithi083 ceph-mon[34805]: pgmap v406: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 1.63k op/s 2024-09-12T00:05:09.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:08 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:05:09.347 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:08 smithi062 ceph-mon[30958]: pgmap v406: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 1.63k op/s 2024-09-12T00:05:09.347 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:08 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:05:09.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.340+0000 7f515e020640 1 -- 172.21.15.62:0/1488443191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51581027a0 msgr2=0x7f51581067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:09.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.340+0000 7f515e020640 1 --2- 172.21.15.62:0/1488443191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51581027a0 0x7f51581067f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f514c009900 tx=0x7f514c02a410 comp rx=0 tx=0).stop 2024-09-12T00:05:09.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.340+0000 7f515e020640 1 -- 172.21.15.62:0/1488443191 shutdown_connections 2024-09-12T00:05:09.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.340+0000 7f515e020640 1 --2- 172.21.15.62:0/1488443191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5158106f20 0x7f515810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.340+0000 7f515e020640 1 --2- 172.21.15.62:0/1488443191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51581027a0 0x7f51581067f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.340+0000 7f515e020640 1 --2- 172.21.15.62:0/1488443191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158101df0 0x7f51581021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.340+0000 7f515e020640 1 -- 172.21.15.62:0/1488443191 >> 172.21.15.62:0/1488443191 conn(0x7f51580fd8f0 msgr2=0x7f51580ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:09.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.340+0000 7f515e020640 1 -- 172.21.15.62:0/1488443191 shutdown_connections 2024-09-12T00:05:09.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.340+0000 7f515e020640 1 -- 172.21.15.62:0/1488443191 wait complete. 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.341+0000 7f515e020640 1 Processor -- start 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.341+0000 7f515e020640 1 -- start start 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f515e020640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158101df0 0x7f51580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f515e020640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51581027a0 0x7f5158074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f515e020640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5158106f20 0x7f5158074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f515e020640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5158111f40 con 0x7f51581027a0 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f515e020640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5158111dc0 con 0x7f5158106f20 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f515e020640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51581120c0 con 0x7f5158101df0 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f5156ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51581027a0 0x7f5158074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f51577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158101df0 0x7f51580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f5156ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51581027a0 0x7f5158074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38858/0 (socket says 172.21.15.62:38858) 2024-09-12T00:05:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f5156ffd640 1 -- 172.21.15.62:0/2847743441 learned_addr learned my addr 172.21.15.62:0/2847743441 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f5157fff640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5158106f20 0x7f5158074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f51577fe640 1 -- 172.21.15.62:0/2847743441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5158106f20 msgr2=0x7f5158074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f51577fe640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5158106f20 0x7f5158074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f51577fe640 1 -- 172.21.15.62:0/2847743441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51581027a0 msgr2=0x7f5158074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f51577fe640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51581027a0 0x7f5158074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.342+0000 7f51577fe640 1 -- 172.21.15.62:0/2847743441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51581a68c0 con 0x7f5158101df0 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.343+0000 7f5156ffd640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51581027a0 0x7f5158074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.343+0000 7f51577fe640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158101df0 0x7f51580744f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f514800e9f0 tx=0x7f514800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.343+0000 7f5154ff9640 1 -- 172.21.15.62:0/2847743441 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800ccc0 con 0x7f5158101df0 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.343+0000 7f5154ff9640 1 -- 172.21.15.62:0/2847743441 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5148010910 con 0x7f5158101df0 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.343+0000 7f5154ff9640 1 -- 172.21.15.62:0/2847743441 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148018960 con 0x7f5158101df0 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.343+0000 7f515e020640 1 -- 172.21.15.62:0/2847743441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51581a6bb0 con 0x7f5158101df0 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.344+0000 7f515e020640 1 -- 172.21.15.62:0/2847743441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51581a6fe0 con 0x7f5158101df0 2024-09-12T00:05:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.346+0000 7f5154ff9640 1 -- 172.21.15.62:0/2847743441 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5148010ac0 con 0x7f5158101df0 2024-09-12T00:05:09.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f5154ff9640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f51400a2ec0 0x7f51400a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f5154ff9640 1 -- 172.21.15.62:0/2847743441 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f51480cf710 con 0x7f5158101df0 2024-09-12T00:05:09.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51603f9740 1 Processor -- start 2024-09-12T00:05:09.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f5156ffd640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f51400a2ec0 0x7f51400a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51603f9740 1 -- start start 2024-09-12T00:05:09.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51603f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb69880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51603f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb69dc0 0x55608fb74140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51603f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb74680 0x55608fb76a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51603f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55608facf2e0 con 0x55608fb694a0 2024-09-12T00:05:09.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51603f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55608fa76ce0 con 0x55608fb69dc0 2024-09-12T00:05:09.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51603f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55608facb2a0 con 0x55608fb74680 2024-09-12T00:05:09.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51577fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb69880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f5156ffd640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f51400a2ec0 0x7f51400a5380 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f514c005c20 tx=0x7f514c005b70 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:09.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51577fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb69880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38872/0 (socket says 172.21.15.62:38872) 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.347+0000 7f51577fe640 1 -- 172.21.15.62:0/370711537 learned_addr learned my addr 172.21.15.62:0/370711537 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f5156ffd640 1 --2- 172.21.15.62:0/370711537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb69dc0 0x55608fb74140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f5157fff640 1 --2- 172.21.15.62:0/370711537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb74680 0x55608fb76a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f5156ffd640 1 -- 172.21.15.62:0/370711537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb74680 msgr2=0x55608fb76a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f5156ffd640 1 --2- 172.21.15.62:0/370711537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb74680 0x55608fb76a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f5156ffd640 1 -- 172.21.15.62:0/370711537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 msgr2=0x55608fb69880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f5156ffd640 1 --2- 172.21.15.62:0/370711537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb69880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f5156ffd640 1 -- 172.21.15.62:0/370711537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55608fa5b440 con 0x55608fb69dc0 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f51577fe640 1 --2- 172.21.15.62:0/370711537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb69880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f5156ffd640 1 --2- 172.21.15.62:0/370711537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb69dc0 0x55608fb74140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f514c000c00 tx=0x7f514c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4907931d7c79f089 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f513dffb640 1 -- 172.21.15.62:0/370711537 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c035020 con 0x55608fb69dc0 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f513dffb640 1 -- 172.21.15.62:0/370711537 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514c031c80 con 0x55608fb69dc0 2024-09-12T00:05:09.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.348+0000 7f513dffb640 1 -- 172.21.15.62:0/370711537 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c0035f0 con 0x55608fb69dc0 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.349+0000 7f51603f9740 1 -- 172.21.15.62:0/370711537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb69dc0 msgr2=0x55608fb74140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.349+0000 7f51603f9740 1 --2- 172.21.15.62:0/370711537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb69dc0 0x55608fb74140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f514c000c00 tx=0x7f514c03f000 comp rx=0 tx=0).stop 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.349+0000 7f51603f9740 1 -- 172.21.15.62:0/370711537 shutdown_connections 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.349+0000 7f51603f9740 1 --2- 172.21.15.62:0/370711537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb74680 0x55608fb76a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.349+0000 7f51603f9740 1 --2- 172.21.15.62:0/370711537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb69dc0 0x55608fb74140 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f514c000c00 tx=0x7f514c03f000 comp rx=0 tx=0).stop 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.349+0000 7f51603f9740 1 --2- 172.21.15.62:0/370711537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb69880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.349+0000 7f51603f9740 1 -- 172.21.15.62:0/370711537 >> 172.21.15.62:0/370711537 conn(0x55608fb65a30 msgr2=0x55608fb65e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.350+0000 7f51603f9740 1 -- 172.21.15.62:0/370711537 shutdown_connections 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.350+0000 7f51603f9740 1 -- 172.21.15.62:0/370711537 wait complete. 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.350+0000 7f51603f9740 1 -- start start 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51603f9740 1 Processor -- start 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51603f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51603f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb71a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51603f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb88ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51603f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55608fb77250 con 0x55608fb74680 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51603f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55608fa8a050 con 0x55608fb71f40 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51603f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55608fb773d0 con 0x55608fb694a0 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51603f9740 10 client.0 _ll_register_callbacks cb 0x55608faca160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f5157fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb88ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f5156ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb71a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:58476/0 (socket says 172.21.15.62:58476) 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.351+0000 7f51577fe640 1 -- 172.21.15.62:0/3389318408 learned_addr learned my addr 172.21.15.62:0/3389318408 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f51577fe640 1 -- 172.21.15.62:0/3389318408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 msgr2=0x55608fb88ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f51577fe640 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb88ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f51577fe640 1 -- 172.21.15.62:0/3389318408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 msgr2=0x55608fb71a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f51577fe640 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb71a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f51577fe640 1 -- 172.21.15.62:0/3389318408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55608f8403c0 con 0x55608fb694a0 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f5157fff640 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb88ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f5156ffd640 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb71a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f51577fe640 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb714c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51480cce60 tx=0x7f514809bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2898433fa69975b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f5136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f5136ffd640 1 -- 172.21.15.62:0/3389318408 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51480cd290 con 0x55608fb694a0 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f51603f9740 1 -- 172.21.15.62:0/3389318408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f514c031c80 con 0x55608fb694a0 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f5136ffd640 1 -- 172.21.15.62:0/3389318408 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f514809b2d0 con 0x55608fb694a0 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f5136ffd640 1 -- 172.21.15.62:0/3389318408 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f51480a4840 con 0x55608fb694a0 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f51603f9740 10 client.27142 fetch_fsmap learned FSMap version 37 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.352+0000 7f51603f9740 1 -- 172.21.15.62:0/3389318408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55608fb8ba70 con 0x55608fb694a0 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.353+0000 7f5136ffd640 1 -- 172.21.15.62:0/3389318408 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514c031c80 con 0x55608fb694a0 2024-09-12T00:05:09.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.354+0000 7f5136ffd640 1 -- 172.21.15.62:0/3389318408 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f51480979d0 con 0x55608fb694a0 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.354+0000 7f51603f9740 10 client.27142 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.354+0000 7f51603f9740 10 client.27142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.354+0000 7f51603f9740 20 client.27142 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.354+0000 7f51603f9740 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55608fb8ce80 0x55608fb8f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.354+0000 7f51603f9740 4 client.27142 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.354+0000 7f51603f9740 1 -- 172.21.15.62:0/3389318408 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55608fadc850 con 0x55608fb8ce80 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.355+0000 7f5156ffd640 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55608fb8ce80 0x55608fb8f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.355+0000 7f5136ffd640 10 client.27142 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.355+0000 7f5156ffd640 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55608fb8ce80 0x55608fb8f2b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db7158d46f3bdc82 server_cookie=cf60e7dc08abfecd in_seq=0 out_seq=0 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.356+0000 7f5136ffd640 1 -- 172.21.15.62:0/3389318408 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55608fadc850 con 0x55608fb8ce80 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.356+0000 7f5136ffd640 10 client.27142 handle_command_reply: tid=0 2024-09-12T00:05:09.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.356+0000 7f51603f9740 1 client.27142 shutdown 2024-09-12T00:05:09.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 1 -- 172.21.15.62:0/3389318408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 msgr2=0x55608fb714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:09.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb714c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51480cce60 tx=0x7f514809bc80 comp rx=0 tx=0).stop 2024-09-12T00:05:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 1 -- 172.21.15.62:0/3389318408 shutdown_connections 2024-09-12T00:05:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55608fb8ce80 0x55608fb8f2b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb88ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb71a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 1 --2- 172.21.15.62:0/3389318408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb714c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f5136ffd640 0 client.27142 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 1 -- 172.21.15.62:0/3389318408 >> 172.21.15.62:0/3389318408 conn(0x55608fb65a30 msgr2=0x55608fb64b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 1 -- 172.21.15.62:0/3389318408 shutdown_connections 2024-09-12T00:05:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 1 -- 172.21.15.62:0/3389318408 wait complete. 2024-09-12T00:05:09.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.357+0000 7f51603f9740 20 client.27142 trim_cache size 0 max 16384 2024-09-12T00:05:09.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:08 smithi120 ceph-mon[35182]: pgmap v406: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 1.63k op/s 2024-09-12T00:05:09.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:08 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:05:09.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f51603f9740 1 Processor -- start 2024-09-12T00:05:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f51603f9740 1 -- start start 2024-09-12T00:05:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f51603f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f51603f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb88d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f51603f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb89280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f51603f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51581adff0 con 0x55608fb74680 2024-09-12T00:05:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f51603f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51581ab890 con 0x55608fb71f40 2024-09-12T00:05:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f51603f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51581a9720 con 0x55608fb694a0 2024-09-12T00:05:09.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f5157fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb89280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f51577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f5157fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb89280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38878/0 (socket says 172.21.15.62:38878) 2024-09-12T00:05:09.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f5157fff640 1 -- 172.21.15.62:0/3026147690 learned_addr learned my addr 172.21.15.62:0/3026147690 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.380+0000 7f5156ffd640 1 --2- 172.21.15.62:0/3026147690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb88d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.381+0000 7f51577fe640 1 -- 172.21.15.62:0/3026147690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 msgr2=0x55608fb88d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.381+0000 7f51577fe640 1 --2- 172.21.15.62:0/3026147690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb88d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.381+0000 7f51577fe640 1 -- 172.21.15.62:0/3026147690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 msgr2=0x55608fb89280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.381+0000 7f51577fe640 1 --2- 172.21.15.62:0/3026147690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb89280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.381+0000 7f51577fe640 1 -- 172.21.15.62:0/3026147690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55608fb6a9f0 con 0x55608fb694a0 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.381+0000 7f5157fff640 1 --2- 172.21.15.62:0/3026147690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb89280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.381+0000 7f51577fe640 1 --2- 172.21.15.62:0/3026147690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb6fd90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f51480980e0 tx=0x7f5148009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=366ea04657ff2303 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.381+0000 7f513d7fa640 1 -- 172.21.15.62:0/3026147690 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51480ab030 con 0x55608fb694a0 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.381+0000 7f513d7fa640 1 -- 172.21.15.62:0/3026147690 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51480cf2a0 con 0x55608fb694a0 2024-09-12T00:05:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.381+0000 7f513d7fa640 1 -- 172.21.15.62:0/3026147690 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514809b330 con 0x55608fb694a0 2024-09-12T00:05:09.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.382+0000 7f51603f9740 1 -- 172.21.15.62:0/3026147690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 msgr2=0x55608fb6fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:09.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.382+0000 7f51603f9740 1 --2- 172.21.15.62:0/3026147690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb6fd90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f51480980e0 tx=0x7f5148009700 comp rx=0 tx=0).stop 2024-09-12T00:05:09.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.382+0000 7f51603f9740 1 -- 172.21.15.62:0/3026147690 shutdown_connections 2024-09-12T00:05:09.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.382+0000 7f51603f9740 1 --2- 172.21.15.62:0/3026147690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb74680 0x55608fb89280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.382+0000 7f51603f9740 1 --2- 172.21.15.62:0/3026147690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb88d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.382+0000 7f51603f9740 1 --2- 172.21.15.62:0/3026147690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb694a0 0x55608fb6fd90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.382+0000 7f51603f9740 1 -- 172.21.15.62:0/3026147690 >> 172.21.15.62:0/3026147690 conn(0x55608fb65a30 msgr2=0x55608fb73540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.382+0000 7f51603f9740 1 -- 172.21.15.62:0/3026147690 shutdown_connections 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.382+0000 7f51603f9740 1 -- 172.21.15.62:0/3026147690 wait complete. 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.382+0000 7f51603f9740 1 -- start start 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f51603f9740 1 Processor -- start 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f51603f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb85050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f51603f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb85590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f51603f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb74680 0x55608fb804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f51603f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51581a9d70 con 0x55608fb694a0 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f51603f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51581ab1c0 con 0x55608fb71f40 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f51603f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51581aa420 con 0x55608fb74680 2024-09-12T00:05:09.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f51603f9740 10 client.0 _ll_register_callbacks cb 0x7f514800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f5156ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb85590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f51577fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb85050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f5156ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb85590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:51338/0 (socket says 172.21.15.62:51338) 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f51577fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb85050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38888/0 (socket says 172.21.15.62:38888) 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f5156ffd640 1 -- 172.21.15.62:0/2769790573 learned_addr learned my addr 172.21.15.62:0/2769790573 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.383+0000 7f5157fff640 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb74680 0x55608fb804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f5156ffd640 1 -- 172.21.15.62:0/2769790573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb74680 msgr2=0x55608fb804d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f5156ffd640 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb74680 0x55608fb804d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f5156ffd640 1 -- 172.21.15.62:0/2769790573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 msgr2=0x55608fb85050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f5156ffd640 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb85050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f5156ffd640 1 -- 172.21.15.62:0/2769790573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55608fb80cd0 con 0x55608fb71f40 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f5156ffd640 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb85590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f514c006010 tx=0x7f514c009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7067063b35f9b998 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f5136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:05:09.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f5136ffd640 1 -- 172.21.15.62:0/2769790573 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c07c060 con 0x55608fb71f40 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f51603f9740 1 -- 172.21.15.62:0/2769790573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51480cf2a0 con 0x55608fb71f40 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f5136ffd640 1 -- 172.21.15.62:0/2769790573 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f514c004940 con 0x55608fb71f40 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f5136ffd640 1 -- 172.21.15.62:0/2769790573 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f514c035040 con 0x55608fb71f40 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f51603f9740 10 client.27257 fetch_fsmap learned FSMap version 37 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.384+0000 7f51603f9740 1 -- 172.21.15.62:0/2769790573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55608fb77710 con 0x55608fb71f40 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f5136ffd640 1 -- 172.21.15.62:0/2769790573 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514c073030 con 0x55608fb71f40 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f5136ffd640 1 -- 172.21.15.62:0/2769790573 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f514c03a040 con 0x55608fb71f40 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f51603f9740 10 client.27257 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f51603f9740 10 client.27257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f51603f9740 20 client.27257 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f51603f9740 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55608fb77aa0 0x55608fb79ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f51603f9740 4 client.27257 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:05:09.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f51603f9740 1 -- 172.21.15.62:0/2769790573 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f51480cf2a0 con 0x55608fb77aa0 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f51577fe640 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55608fb77aa0 0x55608fb79ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f51577fe640 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55608fb77aa0 0x55608fb79ec0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b37a7b698e9e00 server_cookie=e378fc312a09b4a1 in_seq=0 out_seq=0 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f5136ffd640 10 client.27257 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f5136ffd640 1 -- 172.21.15.62:0/2769790573 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f51480cf2a0 con 0x55608fb77aa0 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f5136ffd640 10 client.27257 handle_command_reply: tid=0 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.385+0000 7f51603f9740 1 client.27257 shutdown 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 1 -- 172.21.15.62:0/2769790573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 msgr2=0x55608fb85590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb85590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f514c006010 tx=0x7f514c009520 comp rx=0 tx=0).stop 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 1 -- 172.21.15.62:0/2769790573 shutdown_connections 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55608fb77aa0 0x55608fb79ec0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55608fb74680 0x55608fb804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f5136ffd640 0 client.27257 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55608fb71f40 0x55608fb85590 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 1 --2- 172.21.15.62:0/2769790573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55608fb694a0 0x55608fb85050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 1 -- 172.21.15.62:0/2769790573 >> 172.21.15.62:0/2769790573 conn(0x55608fb65a30 msgr2=0x55608fb64350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:09.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 1 -- 172.21.15.62:0/2769790573 shutdown_connections 2024-09-12T00:05:09.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 1 -- 172.21.15.62:0/2769790573 wait complete. 2024-09-12T00:05:09.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.386+0000 7f51603f9740 20 client.27257 trim_cache size 0 max 16384 2024-09-12T00:05:09.391 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:05:09.391 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:05:09.391 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-12T00:05:09.391 INFO:teuthology.orchestra.run.smithi062.stdout: "fe27fd46-3577-4469-b0a6-2200665e52c4": { 2024-09-12T00:05:09.392 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-12T00:05:09.392 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "fe27fd46-3577-4469-b0a6-2200665e52c4", 2024-09-12T00:05:09.392 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-12T00:05:09.392 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:05:09.392 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:05:09.392 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:05:09.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 -- 172.21.15.62:0/2847743441 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f51400a2ec0 msgr2=0x7f51400a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:09.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f51400a2ec0 0x7f51400a5380 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f514c005c20 tx=0x7f514c005b70 comp rx=0 tx=0).stop 2024-09-12T00:05:09.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 -- 172.21.15.62:0/2847743441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158101df0 msgr2=0x7f51580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:09.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158101df0 0x7f51580744f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f514800e9f0 tx=0x7f514800eeb0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 -- 172.21.15.62:0/2847743441 shutdown_connections 2024-09-12T00:05:09.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f51400a2ec0 0x7f51400a5380 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5158106f20 0x7f5158074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51581027a0 0x7f5158074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 --2- 172.21.15.62:0/2847743441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158101df0 0x7f51580744f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:09.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 -- 172.21.15.62:0/2847743441 >> 172.21.15.62:0/2847743441 conn(0x7f51580fd8f0 msgr2=0x7f51580fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:09.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 -- 172.21.15.62:0/2847743441 shutdown_connections 2024-09-12T00:05:09.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:09.387+0000 7f515e020640 1 -- 172.21.15.62:0/2847743441 wait complete. 2024-09-12T00:05:10.165 INFO:tasks.cephfs.filesystem:scrub status for tag:fe27fd46-3577-4469-b0a6-2200665e52c4 - {'path': '/', 'tag': 'fe27fd46-3577-4469-b0a6-2200665e52c4', 'options': 'recursive,force'} 2024-09-12T00:05:11.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:10 smithi083 ceph-mon[34805]: pgmap v407: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.48k op/s 2024-09-12T00:05:11.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:10 smithi062 ceph-mon[30958]: pgmap v407: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.48k op/s 2024-09-12T00:05:11.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:10 smithi120 ceph-mon[35182]: pgmap v407: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.48k op/s 2024-09-12T00:05:12.444 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 750.0 s, 2512.6 tps, lat 39.711 ms stddev 42.313 2024-09-12T00:05:13.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:12 smithi083 ceph-mon[34805]: pgmap v408: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-12T00:05:13.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:12 smithi062 ceph-mon[30958]: pgmap v408: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-12T00:05:13.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:12 smithi120 ceph-mon[35182]: pgmap v408: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-12T00:05:15.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:14 smithi083 ceph-mon[34805]: pgmap v409: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 MiB/s wr, 1.71k op/s 2024-09-12T00:05:15.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:14 smithi062 ceph-mon[30958]: pgmap v409: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 MiB/s wr, 1.71k op/s 2024-09-12T00:05:15.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:14 smithi120 ceph-mon[35182]: pgmap v409: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 MiB/s wr, 1.71k op/s 2024-09-12T00:05:16.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:15 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:05:16.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:15 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:05:16.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:15 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:05:17.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:16 smithi083 ceph-mon[34805]: pgmap v410: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 546 op/s 2024-09-12T00:05:17.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:16 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:05:17.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:16 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:05:17.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:16 smithi062 ceph-mon[30958]: pgmap v410: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 546 op/s 2024-09-12T00:05:17.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:16 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:05:17.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:16 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:05:17.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:16 smithi120 ceph-mon[35182]: pgmap v410: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 546 op/s 2024-09-12T00:05:17.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:16 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:05:17.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:16 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:05:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 755.0 s, 2135.6 tps, lat 46.934 ms stddev 53.034 2024-09-12T00:05:19.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:18 smithi083 ceph-mon[34805]: pgmap v411: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.6 MiB/s wr, 798 op/s 2024-09-12T00:05:19.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:18 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:05:19.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:18 smithi062 ceph-mon[30958]: pgmap v411: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.6 MiB/s wr, 798 op/s 2024-09-12T00:05:19.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:18 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:05:19.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:18 smithi120 ceph-mon[35182]: pgmap v411: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.6 MiB/s wr, 798 op/s 2024-09-12T00:05:19.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:18 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:05:21.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:20 smithi083 ceph-mon[34805]: pgmap v412: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-09-12T00:05:21.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:20 smithi062 ceph-mon[30958]: pgmap v412: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-09-12T00:05:21.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:20 smithi120 ceph-mon[35182]: pgmap v412: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-09-12T00:05:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 760.0 s, 2205.4 tps, lat 45.033 ms stddev 50.078 2024-09-12T00:05:23.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:22 smithi083 ceph-mon[34805]: pgmap v413: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 607 op/s 2024-09-12T00:05:23.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:22 smithi062 ceph-mon[30958]: pgmap v413: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 607 op/s 2024-09-12T00:05:23.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:22 smithi120 ceph-mon[35182]: pgmap v413: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 607 op/s 2024-09-12T00:05:25.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:24 smithi083 ceph-mon[34805]: pgmap v414: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.4 MiB/s wr, 870 op/s 2024-09-12T00:05:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:24 smithi062 ceph-mon[30958]: pgmap v414: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.4 MiB/s wr, 870 op/s 2024-09-12T00:05:25.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:24 smithi120 ceph-mon[35182]: pgmap v414: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.4 MiB/s wr, 870 op/s 2024-09-12T00:05:27.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:26 smithi083 ceph-mon[34805]: pgmap v415: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-09-12T00:05:27.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:26 smithi062 ceph-mon[30958]: pgmap v415: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-09-12T00:05:27.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:26 smithi120 ceph-mon[35182]: pgmap v415: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-09-12T00:05:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 765.0 s, 2273.0 tps, lat 44.199 ms stddev 50.283 2024-09-12T00:05:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:28 smithi062 ceph-mon[30958]: pgmap v416: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.8 MiB/s wr, 808 op/s 2024-09-12T00:05:29.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:28 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:05:29.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:28 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:05:29.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:28 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:05:29.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:28 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:05:29.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:28 smithi120 ceph-mon[35182]: pgmap v416: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.8 MiB/s wr, 808 op/s 2024-09-12T00:05:29.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:28 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:05:29.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:28 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:05:29.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:28 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:05:29.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:28 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:05:29.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:28 smithi083 ceph-mon[34805]: pgmap v416: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.8 MiB/s wr, 808 op/s 2024-09-12T00:05:29.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:28 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:05:29.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:28 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:05:29.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:28 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:05:29.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:28 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:05:31.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:30 smithi062 ceph-mon[30958]: pgmap v417: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.6 MiB/s wr, 592 op/s 2024-09-12T00:05:31.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:30 smithi120 ceph-mon[35182]: pgmap v417: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.6 MiB/s wr, 592 op/s 2024-09-12T00:05:31.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:30 smithi083 ceph-mon[34805]: pgmap v417: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.6 MiB/s wr, 592 op/s 2024-09-12T00:05:32.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 770.0 s, 1339.2 tps, lat 73.786 ms stddev 77.831 2024-09-12T00:05:32.681 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T00:05:32.681 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T00:05:32.682 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:05:32.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:32 smithi062 ceph-mon[30958]: pgmap v418: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.1 MiB/s wr, 537 op/s 2024-09-12T00:05:33.080 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:33.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:32 smithi120 ceph-mon[35182]: pgmap v418: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.1 MiB/s wr, 537 op/s 2024-09-12T00:05:33.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:32 smithi083 ceph-mon[34805]: pgmap v418: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.1 MiB/s wr, 537 op/s 2024-09-12T00:05:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.588+0000 7f811ebd6640 1 -- 172.21.15.62:0/688140118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8118100e20 msgr2=0x7f8118101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.588+0000 7f811ebd6640 1 --2- 172.21.15.62:0/688140118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8118100e20 0x7f8118101200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f810c009930 tx=0x7f810c02a4d0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.588+0000 7f811ebd6640 1 -- 172.21.15.62:0/688140118 shutdown_connections 2024-09-12T00:05:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.588+0000 7f811ebd6640 1 --2- 172.21.15.62:0/688140118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8118102ef0 0x7f811810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.588+0000 7f811ebd6640 1 --2- 172.21.15.62:0/688140118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8118102570 0x7f81181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.588+0000 7f811ebd6640 1 --2- 172.21.15.62:0/688140118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8118100e20 0x7f8118101200 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.588+0000 7f811ebd6640 1 -- 172.21.15.62:0/688140118 >> 172.21.15.62:0/688140118 conn(0x7f81180fc7d0 msgr2=0x7f81180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.588+0000 7f811ebd6640 1 -- 172.21.15.62:0/688140118 shutdown_connections 2024-09-12T00:05:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.588+0000 7f811ebd6640 1 -- 172.21.15.62:0/688140118 wait complete. 2024-09-12T00:05:34.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.589+0000 7f811ebd6640 1 Processor -- start 2024-09-12T00:05:34.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.589+0000 7f811ebd6640 1 -- start start 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f811ebd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8118100e20 0x7f81180fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f811ebd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8118102570 0x7f81180fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f811ebd6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8118102ef0 0x7f81180fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f811ebd6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8118110fa0 con 0x7f8118102ef0 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f811ebd6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8118110e20 con 0x7f8118102570 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f811ebd6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8118111120 con 0x7f8118100e20 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f811c972640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8118100e20 0x7f81180fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f8117fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8118102570 0x7f81180fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f8117fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8118102570 0x7f81180fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:44374/0 (socket says 172.21.15.62:44374) 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f8117fff640 1 -- 172.21.15.62:0/3766362048 learned_addr learned my addr 172.21.15.62:0/3766362048 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.590+0000 7f811d173640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8118102ef0 0x7f81180fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.591+0000 7f8117fff640 1 -- 172.21.15.62:0/3766362048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8118100e20 msgr2=0x7f81180fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.591+0000 7f8117fff640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8118100e20 0x7f81180fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.591+0000 7f8117fff640 1 -- 172.21.15.62:0/3766362048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8118102ef0 msgr2=0x7f81180fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.591+0000 7f8117fff640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8118102ef0 0x7f81180fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.591+0000 7f8117fff640 1 -- 172.21.15.62:0/3766362048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811819c760 con 0x7f8118102570 2024-09-12T00:05:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.591+0000 7f811c972640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8118100e20 0x7f81180fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.591+0000 7f8117fff640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8118102570 0x7f81180fd1c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f810400e9f0 tx=0x7f810400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.591+0000 7f8115ffb640 1 -- 172.21.15.62:0/3766362048 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810400cd80 con 0x7f8118102570 2024-09-12T00:05:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.592+0000 7f8115ffb640 1 -- 172.21.15.62:0/3766362048 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8104010910 con 0x7f8118102570 2024-09-12T00:05:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.592+0000 7f8115ffb640 1 -- 172.21.15.62:0/3766362048 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810400cd80 con 0x7f8118102570 2024-09-12T00:05:34.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.592+0000 7f811ebd6640 1 -- 172.21.15.62:0/3766362048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811819ca50 con 0x7f8118102570 2024-09-12T00:05:34.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.592+0000 7f811ebd6640 1 -- 172.21.15.62:0/3766362048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f811819cf90 con 0x7f8118102570 2024-09-12T00:05:34.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.593+0000 7f811ebd6640 1 -- 172.21.15.62:0/3766362048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81181021b0 con 0x7f8118102570 2024-09-12T00:05:34.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.595+0000 7f8115ffb640 1 -- 172.21.15.62:0/3766362048 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8104010ac0 con 0x7f8118102570 2024-09-12T00:05:34.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.596+0000 7f8115ffb640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f81080a2ef0 0x7f81080a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:34.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.596+0000 7f8115ffb640 1 -- 172.21.15.62:0/3766362048 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f81040cf5e0 con 0x7f8118102570 2024-09-12T00:05:34.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.596+0000 7f811c972640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f81080a2ef0 0x7f81080a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:34.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.597+0000 7f811c972640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f81080a2ef0 0x7f81080a53b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f810c009900 tx=0x7f810c0058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:34.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.601+0000 7f8115ffb640 1 -- 172.21.15.62:0/3766362048 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81040541b0 con 0x7f8118102570 2024-09-12T00:05:34.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.950+0000 7f811ebd6640 1 -- 172.21.15.62:0/3766362048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81180fe1b0 con 0x7f8118102570 2024-09-12T00:05:34.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.950+0000 7f8115ffb640 1 -- 172.21.15.62:0/3766362048 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11910 (secure 0 0 0) 0x7f810409c200 con 0x7f8118102570 2024-09-12T00:05:34.950 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:05:34.950 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"btime":"2024-09-12T00:05:17:893610+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:05:17.091935+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:05:34.951 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-09-12T00:05:34.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.953+0000 7f811ebd6640 1 -- 172.21.15.62:0/3766362048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f81080a2ef0 msgr2=0x7f81080a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:34.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.953+0000 7f811ebd6640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f81080a2ef0 0x7f81080a53b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f810c009900 tx=0x7f810c0058d0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.953+0000 7f811ebd6640 1 -- 172.21.15.62:0/3766362048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8118102570 msgr2=0x7f81180fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:34.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.953+0000 7f811ebd6640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8118102570 0x7f81180fd1c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f810400e9f0 tx=0x7f810400eeb0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.953+0000 7f811ebd6640 1 -- 172.21.15.62:0/3766362048 shutdown_connections 2024-09-12T00:05:34.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.953+0000 7f811ebd6640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f81080a2ef0 0x7f81080a53b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.953+0000 7f811ebd6640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8118102ef0 0x7f81180fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.953+0000 7f811ebd6640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8118102570 0x7f81180fd1c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.953+0000 7f811ebd6640 1 --2- 172.21.15.62:0/3766362048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8118100e20 0x7f81180fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:34.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.953+0000 7f811ebd6640 1 -- 172.21.15.62:0/3766362048 >> 172.21.15.62:0/3766362048 conn(0x7f81180fc7d0 msgr2=0x7f811806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:34.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.954+0000 7f811ebd6640 1 -- 172.21.15.62:0/3766362048 shutdown_connections 2024-09-12T00:05:34.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:34.954+0000 7f811ebd6640 1 -- 172.21.15.62:0/3766362048 wait complete. 2024-09-12T00:05:35.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:34 smithi062 ceph-mon[30958]: pgmap v419: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.6 MiB/s wr, 806 op/s 2024-09-12T00:05:35.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:34 smithi120 ceph-mon[35182]: pgmap v419: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.6 MiB/s wr, 806 op/s 2024-09-12T00:05:35.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:34 smithi083 ceph-mon[34805]: pgmap v419: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.6 MiB/s wr, 806 op/s 2024-09-12T00:05:36.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:35 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3766362048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:05:36.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:35 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3766362048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:05:36.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:35 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3766362048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:05:36.600 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T00:05:36.601 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T00:05:36.972 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:37.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:36 smithi062 ceph-mon[30958]: pgmap v420: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-09-12T00:05:37.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:36 smithi120 ceph-mon[35182]: pgmap v420: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-09-12T00:05:37.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:36 smithi083 ceph-mon[34805]: pgmap v420: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-09-12T00:05:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 775.0 s, 1637.4 tps, lat 60.915 ms stddev 73.327 2024-09-12T00:05:37.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.547+0000 7f25961e7640 1 -- 172.21.15.62:0/1554745328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2590102780 msgr2=0x7f25901067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:37.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.547+0000 7f25961e7640 1 --2- 172.21.15.62:0/1554745328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2590102780 0x7f25901067d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f2584009900 tx=0x7f258402a410 comp rx=0 tx=0).stop 2024-09-12T00:05:37.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.547+0000 7f25961e7640 1 -- 172.21.15.62:0/1554745328 shutdown_connections 2024-09-12T00:05:37.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.547+0000 7f25961e7640 1 --2- 172.21.15.62:0/1554745328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590106f00 0x7f259010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.547+0000 7f25961e7640 1 --2- 172.21.15.62:0/1554745328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2590102780 0x7f25901067d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.547+0000 7f25961e7640 1 --2- 172.21.15.62:0/1554745328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2590101dd0 0x7f25901021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.547+0000 7f25961e7640 1 -- 172.21.15.62:0/1554745328 >> 172.21.15.62:0/1554745328 conn(0x7f25900fd870 msgr2=0x7f25900ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:37.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.547+0000 7f25961e7640 1 -- 172.21.15.62:0/1554745328 shutdown_connections 2024-09-12T00:05:37.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.547+0000 7f25961e7640 1 -- 172.21.15.62:0/1554745328 wait complete. 2024-09-12T00:05:37.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.548+0000 7f25961e7640 1 Processor -- start 2024-09-12T00:05:37.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f25961e7640 1 -- start start 2024-09-12T00:05:37.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f25961e7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2590101dd0 0x7f2590198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f25961e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590102780 0x7f25901989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f25961e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2590106f00 0x7f259019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f258f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2590101dd0 0x7f2590198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f25961e7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2590111f60 con 0x7f2590101dd0 2024-09-12T00:05:37.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f258f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2590101dd0 0x7f2590198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52052/0 (socket says 172.21.15.62:52052) 2024-09-12T00:05:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f258f7fe640 1 -- 172.21.15.62:0/3590149054 learned_addr learned my addr 172.21.15.62:0/3590149054 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f25961e7640 1 -- 172.21.15.62:0/3590149054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2590111de0 con 0x7f2590106f00 2024-09-12T00:05:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f25961e7640 1 -- 172.21.15.62:0/3590149054 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f25901120e0 con 0x7f2590102780 2024-09-12T00:05:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f258ffff640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2590106f00 0x7f259019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.549+0000 7f258effd640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590102780 0x7f25901989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f258f7fe640 1 -- 172.21.15.62:0/3590149054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590102780 msgr2=0x7f25901989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f258f7fe640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590102780 0x7f25901989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f258f7fe640 1 -- 172.21.15.62:0/3590149054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2590106f00 msgr2=0x7f259019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f258f7fe640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2590106f00 0x7f259019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f258f7fe640 1 -- 172.21.15.62:0/3590149054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259019d410 con 0x7f2590101dd0 2024-09-12T00:05:37.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f258f7fe640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2590101dd0 0x7f2590198460 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f258000e9f0 tx=0x7f258000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:37.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f258cff9640 1 -- 172.21.15.62:0/3590149054 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258000cd80 con 0x7f2590101dd0 2024-09-12T00:05:37.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f25961e7640 1 -- 172.21.15.62:0/3590149054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259019d700 con 0x7f2590101dd0 2024-09-12T00:05:37.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f258cff9640 1 -- 172.21.15.62:0/3590149054 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2580010910 con 0x7f2590101dd0 2024-09-12T00:05:37.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f258cff9640 1 -- 172.21.15.62:0/3590149054 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580018990 con 0x7f2590101dd0 2024-09-12T00:05:37.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.550+0000 7f25961e7640 1 -- 172.21.15.62:0/3590149054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25901a4fe0 con 0x7f2590101dd0 2024-09-12T00:05:37.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.555+0000 7f258cff9640 1 -- 172.21.15.62:0/3590149054 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2580010ac0 con 0x7f2590101dd0 2024-09-12T00:05:37.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.556+0000 7f258cff9640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25780a2df0 0x7f25780a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.556+0000 7f258cff9640 1 -- 172.21.15.62:0/3590149054 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f25800cf870 con 0x7f2590101dd0 2024-09-12T00:05:37.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.556+0000 7f258effd640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25780a2df0 0x7f25780a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.557+0000 7f258effd640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25780a2df0 0x7f25780a52b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f25900fdc90 tx=0x7f2584005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:37.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.557+0000 7f25985c0740 1 Processor -- start 2024-09-12T00:05:37.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.557+0000 7f25985c0740 1 -- start start 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.557+0000 7f25985c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81712a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.557+0000 7f25985c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81712f80 0x555a81721810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.557+0000 7f25985c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81721d50 0x555a81724140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.557+0000 7f25985c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555a816782a0 con 0x555a81721d50 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.557+0000 7f25985c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a8161fb10 con 0x555a81712660 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.557+0000 7f25985c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a81674260 con 0x555a81712f80 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81712a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81712f80 0x555a81721810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81712a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:44414/0 (socket says 172.21.15.62:44414) 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81712f80 0x555a81721810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:38650/0 (socket says 172.21.15.62:38650) 2024-09-12T00:05:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258f7fe640 1 -- 172.21.15.62:0/2314414883 learned_addr learned my addr 172.21.15.62:0/2314414883 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258ffff640 1 --2- 172.21.15.62:0/2314414883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81721d50 0x555a81724140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258f7fe640 1 -- 172.21.15.62:0/2314414883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81712f80 msgr2=0x555a81721810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258f7fe640 1 --2- 172.21.15.62:0/2314414883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81712f80 0x555a81721810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258f7fe640 1 -- 172.21.15.62:0/2314414883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81721d50 msgr2=0x555a81724140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258f7fe640 1 --2- 172.21.15.62:0/2314414883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81721d50 0x555a81724140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258f7fe640 1 -- 172.21.15.62:0/2314414883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a813e9b90 con 0x555a81712660 2024-09-12T00:05:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f258f7fe640 1 --2- 172.21.15.62:0/2314414883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81712a40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f25800a4470 tx=0x7f258009c570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89cfe827b9e7aad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f2575ffb640 1 -- 172.21.15.62:0/2314414883 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800983a0 con 0x555a81712660 2024-09-12T00:05:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f2575ffb640 1 -- 172.21.15.62:0/2314414883 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25800cd1e0 con 0x555a81712660 2024-09-12T00:05:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.558+0000 7f2575ffb640 1 -- 172.21.15.62:0/2314414883 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258009b3e0 con 0x555a81712660 2024-09-12T00:05:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.559+0000 7f25985c0740 1 -- 172.21.15.62:0/2314414883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 msgr2=0x555a81712a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.559+0000 7f25985c0740 1 --2- 172.21.15.62:0/2314414883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81712a40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f25800a4470 tx=0x7f258009c570 comp rx=0 tx=0).stop 2024-09-12T00:05:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.559+0000 7f25985c0740 1 -- 172.21.15.62:0/2314414883 shutdown_connections 2024-09-12T00:05:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.559+0000 7f25985c0740 1 --2- 172.21.15.62:0/2314414883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81721d50 0x555a81724140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.559+0000 7f25985c0740 1 --2- 172.21.15.62:0/2314414883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81712f80 0x555a81721810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.559+0000 7f25985c0740 1 --2- 172.21.15.62:0/2314414883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81712a40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.559+0000 7f25985c0740 1 -- 172.21.15.62:0/2314414883 >> 172.21.15.62:0/2314414883 conn(0x555a8170ecf0 msgr2=0x555a8170f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.559+0000 7f25985c0740 1 -- 172.21.15.62:0/2314414883 shutdown_connections 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.560+0000 7f25985c0740 1 -- 172.21.15.62:0/2314414883 wait complete. 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.560+0000 7f25985c0740 1 -- start start 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.560+0000 7f25985c0740 1 Processor -- start 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f25985c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a8171ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f25985c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a8171f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f25985c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8171f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f25985c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555a81724980 con 0x555a81712f80 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f25985c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a816006e0 con 0x555a81712660 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f25985c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a81724b00 con 0x555a81721d50 2024-09-12T00:05:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f258f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a8171ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f25985c0740 10 client.0 _ll_register_callbacks cb 0x555a81624a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f258f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a8171ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:44430/0 (socket says 172.21.15.62:44430) 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f258f7fe640 1 -- 172.21.15.62:0/2391967105 learned_addr learned my addr 172.21.15.62:0/2391967105 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f258effd640 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a8171f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.561+0000 7f258ffff640 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8171f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.562+0000 7f258ffff640 1 -- 172.21.15.62:0/2391967105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 msgr2=0x555a8171ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.562+0000 7f258ffff640 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a8171ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.562+0000 7f258ffff640 1 -- 172.21.15.62:0/2391967105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 msgr2=0x555a8171f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.562+0000 7f258ffff640 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a8171f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.562+0000 7f258ffff640 1 -- 172.21.15.62:0/2391967105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8172dee0 con 0x555a81721d50 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.562+0000 7f258f7fe640 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a8171ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:05:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.562+0000 7f258effd640 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a8171f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.562+0000 7f258ffff640 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8171f6f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25880127b0 tx=0x7f2588012c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36603fb4f86cc465 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.562+0000 7f256effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f256effd640 1 -- 172.21.15.62:0/2391967105 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258800cde0 con 0x555a81721d50 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f256effd640 1 -- 172.21.15.62:0/2391967105 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f258802b3f0 con 0x555a81721d50 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f25985c0740 1 -- 172.21.15.62:0/2391967105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25800cd1e0 con 0x555a81721d50 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f256effd640 1 -- 172.21.15.62:0/2391967105 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f258802b5f0 con 0x555a81721d50 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f25985c0740 10 client.27166 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f25985c0740 1 -- 172.21.15.62:0/2391967105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555a81716c70 con 0x555a81721d50 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f256effd640 1 -- 172.21.15.62:0/2391967105 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25800cd1e0 con 0x555a81721d50 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f256effd640 1 -- 172.21.15.62:0/2391967105 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f258800f040 con 0x555a81721d50 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f25985c0740 10 client.27166 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f25985c0740 10 client.27166 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:05:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f25985c0740 20 client.27166 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f25985c0740 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a81718180 0x555a817185a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f25985c0740 4 client.27166 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.563+0000 7f25985c0740 1 -- 172.21.15.62:0/2391967105 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a816859d0 con 0x555a81718180 2024-09-12T00:05:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.564+0000 7f258f7fe640 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a81718180 0x555a817185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.564+0000 7f256effd640 10 client.27166 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.564+0000 7f258f7fe640 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a81718180 0x555a817185a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0175fe4fcd6fb13 server_cookie=c1d90f7b1badc1e2 in_seq=0 out_seq=0 2024-09-12T00:05:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.565+0000 7f256effd640 1 -- 172.21.15.62:0/2391967105 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a816859d0 con 0x555a81718180 2024-09-12T00:05:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.565+0000 7f256effd640 10 client.27166 handle_command_reply: tid=0 2024-09-12T00:05:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.565+0000 7f25985c0740 1 client.27166 shutdown 2024-09-12T00:05:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 1 -- 172.21.15.62:0/2391967105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 msgr2=0x555a8171f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8171f6f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25880127b0 tx=0x7f2588012c70 comp rx=0 tx=0).stop 2024-09-12T00:05:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 1 -- 172.21.15.62:0/2391967105 shutdown_connections 2024-09-12T00:05:37.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a81718180 0x555a817185a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8171f6f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a8171f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 1 --2- 172.21.15.62:0/2391967105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a8171ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f256effd640 0 client.27166 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:37.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 1 -- 172.21.15.62:0/2391967105 >> 172.21.15.62:0/2391967105 conn(0x555a8170ecf0 msgr2=0x555a81714a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:37.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 1 -- 172.21.15.62:0/2391967105 shutdown_connections 2024-09-12T00:05:37.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 1 -- 172.21.15.62:0/2391967105 wait complete. 2024-09-12T00:05:37.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.566+0000 7f25985c0740 20 client.27166 trim_cache size 0 max 16384 2024-09-12T00:05:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f25985c0740 1 Processor -- start 2024-09-12T00:05:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f25985c0740 1 -- start start 2024-09-12T00:05:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f25985c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81719660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f25985c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a81720be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f25985c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8173e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f25985c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f25901a5820 con 0x555a81712f80 2024-09-12T00:05:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f25985c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25900744a0 con 0x555a81712660 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f25985c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2590072310 con 0x555a81721d50 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a81720be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81719660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a81720be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52076/0 (socket says 172.21.15.62:52076) 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258effd640 1 -- 172.21.15.62:0/1019993242 learned_addr learned my addr 172.21.15.62:0/1019993242 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258ffff640 1 --2- 172.21.15.62:0/1019993242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8173e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258f7fe640 1 -- 172.21.15.62:0/1019993242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 msgr2=0x555a8173e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258f7fe640 1 --2- 172.21.15.62:0/1019993242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8173e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258f7fe640 1 -- 172.21.15.62:0/1019993242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 msgr2=0x555a81720be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258f7fe640 1 --2- 172.21.15.62:0/1019993242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a81720be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258f7fe640 1 -- 172.21.15.62:0/1019993242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8170a260 con 0x555a81712660 2024-09-12T00:05:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258ffff640 1 --2- 172.21.15.62:0/1019993242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8173e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.590+0000 7f258effd640 1 --2- 172.21.15.62:0/1019993242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a81720be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.591+0000 7f258f7fe640 1 --2- 172.21.15.62:0/1019993242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81719660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f258009c5e0 tx=0x7f2580009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d782bd2a32a1449 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.591+0000 7f25757fa640 1 -- 172.21.15.62:0/1019993242 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580018e60 con 0x555a81712660 2024-09-12T00:05:37.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.591+0000 7f25757fa640 1 -- 172.21.15.62:0/1019993242 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25800983a0 con 0x555a81712660 2024-09-12T00:05:37.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.591+0000 7f25757fa640 1 -- 172.21.15.62:0/1019993242 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580097a80 con 0x555a81712660 2024-09-12T00:05:37.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.592+0000 7f25985c0740 1 -- 172.21.15.62:0/1019993242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 msgr2=0x555a81719660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:37.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.592+0000 7f25985c0740 1 --2- 172.21.15.62:0/1019993242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81719660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f258009c5e0 tx=0x7f2580009310 comp rx=0 tx=0).stop 2024-09-12T00:05:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.594+0000 7f25985c0740 1 -- 172.21.15.62:0/1019993242 shutdown_connections 2024-09-12T00:05:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.594+0000 7f25985c0740 1 --2- 172.21.15.62:0/1019993242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8173e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.595+0000 7f25985c0740 1 --2- 172.21.15.62:0/1019993242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712f80 0x555a81720be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.595+0000 7f25985c0740 1 --2- 172.21.15.62:0/1019993242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712660 0x555a81719660 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.595+0000 7f25985c0740 1 -- 172.21.15.62:0/1019993242 >> 172.21.15.62:0/1019993242 conn(0x555a8170ecf0 msgr2=0x555a817204d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.597+0000 7f25985c0740 1 -- 172.21.15.62:0/1019993242 shutdown_connections 2024-09-12T00:05:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.599+0000 7f25985c0740 1 -- 172.21.15.62:0/1019993242 wait complete. 2024-09-12T00:05:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.599+0000 7f25985c0740 1 -- start start 2024-09-12T00:05:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.600+0000 7f25985c0740 1 Processor -- start 2024-09-12T00:05:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.600+0000 7f25985c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712660 0x555a81739c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.600+0000 7f25985c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712f80 0x555a8173a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.601+0000 7f25985c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8179f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.601+0000 7f25985c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2590072960 con 0x555a81712660 2024-09-12T00:05:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.601+0000 7f25985c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2590073dd0 con 0x555a81712f80 2024-09-12T00:05:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.601+0000 7f25985c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2590073030 con 0x555a81721d50 2024-09-12T00:05:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.601+0000 7f25985c0740 10 client.0 _ll_register_callbacks cb 0x7f25880123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.603+0000 7f258effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712f80 0x555a8173a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.603+0000 7f258effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712f80 0x555a8173a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:44458/0 (socket says 172.21.15.62:44458) 2024-09-12T00:05:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.603+0000 7f258effd640 1 -- 172.21.15.62:0/2080759710 learned_addr learned my addr 172.21.15.62:0/2080759710 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.604+0000 7f258ffff640 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8179f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.605+0000 7f258f7fe640 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712660 0x555a81739c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.609+0000 7f258f7fe640 1 -- 172.21.15.62:0/2080759710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 msgr2=0x555a8179f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.609+0000 7f258f7fe640 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8179f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.609+0000 7f258f7fe640 1 -- 172.21.15.62:0/2080759710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712f80 msgr2=0x555a8173a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.609+0000 7f258f7fe640 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712f80 0x555a8173a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.609+0000 7f258f7fe640 1 -- 172.21.15.62:0/2080759710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8179fd50 con 0x555a81712660 2024-09-12T00:05:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.609+0000 7f258f7fe640 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712660 0x555a81739c40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2580098190 tx=0x7f25800cd4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1e1125777a8941 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:37.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.612+0000 7f256effd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:37.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.612+0000 7f256effd640 1 -- 172.21.15.62:0/2080759710 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800af060 con 0x555a81712660 2024-09-12T00:05:37.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.612+0000 7f256effd640 1 -- 172.21.15.62:0/2080759710 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f2580010430 con 0x555a81712660 2024-09-12T00:05:37.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.613+0000 7f25985c0740 1 -- 172.21.15.62:0/2080759710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25800983a0 con 0x555a81712660 2024-09-12T00:05:37.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.616+0000 7f256effd640 1 -- 172.21.15.62:0/2080759710 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f258008b050 con 0x555a81712660 2024-09-12T00:05:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.619+0000 7f25985c0740 10 client.17748 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.619+0000 7f25985c0740 1 -- 172.21.15.62:0/2080759710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555a817a0fb0 con 0x555a81712660 2024-09-12T00:05:37.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.621+0000 7f256effd640 1 -- 172.21.15.62:0/2080759710 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258009b490 con 0x555a81712660 2024-09-12T00:05:37.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.621+0000 7f256effd640 1 -- 172.21.15.62:0/2080759710 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f25800a1270 con 0x555a81712660 2024-09-12T00:05:37.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.623+0000 7f25985c0740 10 client.17748 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.623+0000 7f25985c0740 10 client.17748 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:05:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.623+0000 7f25985c0740 20 client.17748 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.623+0000 7f25985c0740 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a817a1180 0x555a817a3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.623+0000 7f25985c0740 4 client.17748 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T00:05:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.623+0000 7f25985c0740 1 -- 172.21.15.62:0/2080759710 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7f25800983a0 con 0x555a817a1180 2024-09-12T00:05:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.626+0000 7f258effd640 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a817a1180 0x555a817a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.627+0000 7f258effd640 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a817a1180 0x555a817a3570 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76c5b3047fc2cee0 server_cookie=3914fa3c3c1e08d7 in_seq=0 out_seq=0 2024-09-12T00:05:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.627+0000 7f256effd640 10 client.17748 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:37.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.855+0000 7f256effd640 1 -- 172.21.15.62:0/2080759710 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f25800983a0 con 0x555a817a1180 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.855+0000 7f256effd640 10 client.17748 handle_command_reply: tid=0 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.855+0000 7f25985c0740 1 client.17748 shutdown 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 1 -- 172.21.15.62:0/2080759710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712660 msgr2=0x555a81739c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712660 0x555a81739c40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2580098190 tx=0x7f25800cd4b0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 1 -- 172.21.15.62:0/2080759710 shutdown_connections 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x555a817a1180 0x555a817a3570 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a81721d50 0x555a8179f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a81712f80 0x555a8173a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 1 --2- 172.21.15.62:0/2080759710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a81712660 0x555a81739c40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f256effd640 0 client.17748 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 1 -- 172.21.15.62:0/2080759710 >> 172.21.15.62:0/2080759710 conn(0x555a8170ecf0 msgr2=0x555a817204d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:37.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 1 -- 172.21.15.62:0/2080759710 shutdown_connections 2024-09-12T00:05:37.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 1 -- 172.21.15.62:0/2080759710 wait complete. 2024-09-12T00:05:37.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.856+0000 7f25985c0740 20 client.17748 trim_cache size 0 max 16384 2024-09-12T00:05:37.857 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:05:37.857 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 38, 2024-09-12T00:05:37.857 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:05:37.857 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 50, 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "aebadee8": { 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 50, 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:05:37.858 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:05:37.859 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:05:37.859 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:05:37.859 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:05:37.859 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:05:37.859 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:05:37.859 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:05:37.859 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:05:37.859 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:05:37.859 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:05:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 -- 172.21.15.62:0/3590149054 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25780a2df0 msgr2=0x7f25780a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25780a2df0 0x7f25780a52b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f25900fdc90 tx=0x7f2584005a20 comp rx=0 tx=0).stop 2024-09-12T00:05:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 -- 172.21.15.62:0/3590149054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2590101dd0 msgr2=0x7f2590198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2590101dd0 0x7f2590198460 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f258000e9f0 tx=0x7f258000eeb0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 -- 172.21.15.62:0/3590149054 shutdown_connections 2024-09-12T00:05:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f25780a2df0 0x7f25780a52b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2590106f00 0x7f259019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590102780 0x7f25901989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 --2- 172.21.15.62:0/3590149054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2590101dd0 0x7f2590198460 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 -- 172.21.15.62:0/3590149054 >> 172.21.15.62:0/3590149054 conn(0x7f25900fd870 msgr2=0x7f25900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 -- 172.21.15.62:0/3590149054 shutdown_connections 2024-09-12T00:05:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:37.857+0000 7f25961e7640 1 -- 172.21.15.62:0/3590149054 wait complete. 2024-09-12T00:05:38.516 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'aebadee8', quiesce took 5.8 seconds. Will release after: 24.2 2024-09-12T00:05:39.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:38 smithi062 ceph-mon[30958]: pgmap v421: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.2 MiB/s wr, 758 op/s 2024-09-12T00:05:39.218 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:38 smithi083 ceph-mon[34805]: pgmap v421: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.2 MiB/s wr, 758 op/s 2024-09-12T00:05:39.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:38 smithi120 ceph-mon[35182]: pgmap v421: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.2 MiB/s wr, 758 op/s 2024-09-12T00:05:40.166 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:05:40.438 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:40.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.925+0000 7f8ac1b89640 1 -- 172.21.15.62:0/719354691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc106a90 msgr2=0x7f8abc106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:40.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.925+0000 7f8ac1b89640 1 --2- 172.21.15.62:0/719354691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc106a90 0x7f8abc106e70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009930 tx=0x7f8aac02a420 comp rx=0 tx=0).stop 2024-09-12T00:05:40.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.926+0000 7f8ac1b89640 1 -- 172.21.15.62:0/719354691 shutdown_connections 2024-09-12T00:05:40.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.926+0000 7f8ac1b89640 1 --2- 172.21.15.62:0/719354691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8abc101430 0x7f8abc10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.926+0000 7f8ac1b89640 1 --2- 172.21.15.62:0/719354691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8abc100a90 0x7f8abc100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.926+0000 7f8ac1b89640 1 --2- 172.21.15.62:0/719354691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc106a90 0x7f8abc106e70 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.926+0000 7f8ac1b89640 1 -- 172.21.15.62:0/719354691 >> 172.21.15.62:0/719354691 conn(0x7f8abc0fc7d0 msgr2=0x7f8abc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:40.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.926+0000 7f8ac1b89640 1 -- 172.21.15.62:0/719354691 shutdown_connections 2024-09-12T00:05:40.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.926+0000 7f8ac1b89640 1 -- 172.21.15.62:0/719354691 wait complete. 2024-09-12T00:05:40.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.926+0000 7f8ac1b89640 1 Processor -- start 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8ac1b89640 1 -- start start 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8ac1b89640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc100a90 0x7f8abc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8ac1b89640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8abc101430 0x7f8abc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8ac1b89640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8abc106a90 0x7f8abc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8ac1b89640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc110e90 con 0x7f8abc100a90 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8ac1b89640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc110d10 con 0x7f8abc101430 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8ac1b89640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc111010 con 0x7f8abc106a90 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8abb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc100a90 0x7f8abc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8abb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc100a90 0x7f8abc198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52100/0 (socket says 172.21.15.62:52100) 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8abb7fe640 1 -- 172.21.15.62:0/3766482362 learned_addr learned my addr 172.21.15.62:0/3766482362 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8abaffd640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8abc101430 0x7f8abc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8abb7fe640 1 -- 172.21.15.62:0/3766482362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8abc106a90 msgr2=0x7f8abc19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8abc106a90 0x7f8abc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8abb7fe640 1 -- 172.21.15.62:0/3766482362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8abc101430 msgr2=0x7f8abc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8abc101430 0x7f8abc198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.927+0000 7f8abb7fe640 1 -- 172.21.15.62:0/3766482362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc19d3d0 con 0x7f8abc100a90 2024-09-12T00:05:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.928+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc100a90 0x7f8abc198420 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009230 tx=0x7f8aac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.928+0000 7f8ab8ff9640 1 -- 172.21.15.62:0/3766482362 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac03f030 con 0x7f8abc100a90 2024-09-12T00:05:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.928+0000 7f8ab8ff9640 1 -- 172.21.15.62:0/3766482362 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aac003ad0 con 0x7f8abc100a90 2024-09-12T00:05:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.928+0000 7f8ac1b89640 1 -- 172.21.15.62:0/3766482362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc19d660 con 0x7f8abc100a90 2024-09-12T00:05:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.928+0000 7f8ab8ff9640 1 -- 172.21.15.62:0/3766482362 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac035070 con 0x7f8abc100a90 2024-09-12T00:05:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.928+0000 7f8ac1b89640 1 -- 172.21.15.62:0/3766482362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc1a4fa0 con 0x7f8abc100a90 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.930+0000 7f8ab8ff9640 1 -- 172.21.15.62:0/3766482362 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8aac042030 con 0x7f8abc100a90 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.930+0000 7f8ac3f62740 1 Processor -- start 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.930+0000 7f8ac3f62740 1 -- start start 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.930+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cc2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 0x562877ccd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ccfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562877c282e0 con 0x562877ccd680 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562877bcfce0 con 0x562877cc24a0 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562877c242a0 con 0x562877cc2dc0 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8ab8ff9640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8aa40a2cf0 0x7f8aa40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ccfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ccfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52106/0 (socket says 172.21.15.62:52106) 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abaffd640 1 -- 172.21.15.62:0/198943804 learned_addr learned my addr 172.21.15.62:0/198943804 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/198943804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 0x562877ccd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abbfff640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8aa40a2cf0 0x7f8aa40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abbfff640 1 --2- 172.21.15.62:0/198943804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cc2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8ab8ff9640 1 -- 172.21.15.62:0/3766482362 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f8aac0ef9f0 con 0x7f8abc100a90 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abaffd640 1 -- 172.21.15.62:0/198943804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 msgr2=0x562877ccd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abaffd640 1 --2- 172.21.15.62:0/198943804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 0x562877ccd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abaffd640 1 -- 172.21.15.62:0/198943804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 msgr2=0x562877cc2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abaffd640 1 --2- 172.21.15.62:0/198943804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cc2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abaffd640 1 -- 172.21.15.62:0/198943804 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562877bb4440 con 0x562877ccd680 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abbfff640 1 --2- 172.21.15.62:0/198943804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cc2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:05:40.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abaffd640 1 --2- 172.21.15.62:0/198943804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ccfa70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009b60 tx=0x7f8ab000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fc8cfe0d287baa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8abbfff640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8aa40a2cf0 0x7f8aa40a51b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009fd0 tx=0x7f8ab40092b0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.931+0000 7f8aa1ffb640 1 -- 172.21.15.62:0/198943804 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0016070 con 0x562877ccd680 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8aa1ffb640 1 -- 172.21.15.62:0/198943804 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab0009e50 con 0x562877ccd680 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8aa1ffb640 1 -- 172.21.15.62:0/198943804 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0011470 con 0x562877ccd680 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8ac3f62740 1 -- 172.21.15.62:0/198943804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 msgr2=0x562877ccfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/198943804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ccfa70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009b60 tx=0x7f8ab000e6c0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8ac3f62740 1 -- 172.21.15.62:0/198943804 shutdown_connections 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/198943804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ccfa70 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/198943804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 0x562877ccd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/198943804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cc2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8ac3f62740 1 -- 172.21.15.62:0/198943804 >> 172.21.15.62:0/198943804 conn(0x562877cbea30 msgr2=0x562877cbee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8ac3f62740 1 -- 172.21.15.62:0/198943804 shutdown_connections 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.932+0000 7f8ac3f62740 1 -- 172.21.15.62:0/198943804 wait complete. 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8ac3f62740 1 -- start start 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8ac3f62740 1 Processor -- start 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc24a0 0x562877cccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc2dc0 0x562877cca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877cca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562877cd0250 con 0x562877ccd680 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562877be3050 con 0x562877cc2dc0 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562877cd03d0 con 0x562877cc24a0 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8ac3f62740 10 client.0 _ll_register_callbacks cb 0x562877c23160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877cca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877cca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52114/0 (socket says 172.21.15.62:52114) 2024-09-12T00:05:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8abbfff640 1 -- 172.21.15.62:0/3406969080 learned_addr learned my addr 172.21.15.62:0/3406969080 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.933+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc24a0 0x562877cccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8abaffd640 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc2dc0 0x562877cca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8abbfff640 1 -- 172.21.15.62:0/3406969080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc24a0 msgr2=0x562877cccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8abbfff640 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc24a0 0x562877cccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8abbfff640 1 -- 172.21.15.62:0/3406969080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc2dc0 msgr2=0x562877cca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8abbfff640 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc2dc0 0x562877cca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8abbfff640 1 -- 172.21.15.62:0/3406969080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628779993c0 con 0x562877ccd680 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8abbfff640 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877cca770 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009cd0 tx=0x7f8ab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa320827eca6a656 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8a9affd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8a9affd640 1 -- 172.21.15.62:0/3406969080 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4021070 con 0x562877ccd680 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8ac3f62740 1 -- 172.21.15.62:0/3406969080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ab0009e50 con 0x562877ccd680 2024-09-12T00:05:40.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8a9affd640 1 -- 172.21.15.62:0/3406969080 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f8ab4011a30 con 0x562877ccd680 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8a9affd640 1 -- 172.21.15.62:0/3406969080 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8ab402d050 con 0x562877ccd680 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8ac3f62740 10 client.17760 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.934+0000 7f8ac3f62740 1 -- 172.21.15.62:0/3406969080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562877cdffd0 con 0x562877ccd680 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.935+0000 7f8a9affd640 1 -- 172.21.15.62:0/3406969080 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab0009e50 con 0x562877ccd680 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.935+0000 7f8a9affd640 1 -- 172.21.15.62:0/3406969080 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f8ab400f040 con 0x562877ccd680 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.935+0000 7f8ac3f62740 10 client.17760 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.935+0000 7f8ac3f62740 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.935+0000 7f8ac3f62740 20 client.17760 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.935+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562877ce1430 0x562877ce38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.935+0000 7f8ac3f62740 4 client.17760 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.935+0000 7f8ac3f62740 1 -- 172.21.15.62:0/3406969080 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562877c35850 con 0x562877ce1430 2024-09-12T00:05:40.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.935+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562877ce1430 0x562877ce38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562877ce1430 0x562877ce38a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e494ad1829d3d7 server_cookie=180194e1bfa5099b in_seq=0 out_seq=0 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8a9affd640 10 client.17760 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8a9affd640 1 -- 172.21.15.62:0/3406969080 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562877c35850 con 0x562877ce1430 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8a9affd640 10 client.17760 handle_command_reply: tid=0 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8ac3f62740 1 client.17760 shutdown 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8ac3f62740 1 -- 172.21.15.62:0/3406969080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 msgr2=0x562877cca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877cca770 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009cd0 tx=0x7f8ab4004a40 comp rx=0 tx=0).stop 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8ac3f62740 1 -- 172.21.15.62:0/3406969080 shutdown_connections 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562877ce1430 0x562877ce38a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877cca770 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc2dc0 0x562877cca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/3406969080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc24a0 0x562877cccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8a9affd640 0 client.17760 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.936+0000 7f8ac3f62740 1 -- 172.21.15.62:0/3406969080 >> 172.21.15.62:0/3406969080 conn(0x562877cbea30 msgr2=0x562877cc4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.937+0000 7f8ac3f62740 1 -- 172.21.15.62:0/3406969080 shutdown_connections 2024-09-12T00:05:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.937+0000 7f8ac3f62740 1 -- 172.21.15.62:0/3406969080 wait complete. 2024-09-12T00:05:40.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.937+0000 7f8ac3f62740 20 client.17760 trim_cache size 0 max 16384 2024-09-12T00:05:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8ac3f62740 1 Processor -- start 2024-09-12T00:05:40.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8ac3f62740 1 -- start start 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cd94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 0x562877ce9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ce96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1adeb0 con 0x562877ccd680 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1ab780 con 0x562877cc24a0 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1a9630 con 0x562877cc2dc0 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ce96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ce96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52118/0 (socket says 172.21.15.62:52118) 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8abbfff640 1 -- 172.21.15.62:0/290124978 learned_addr learned my addr 172.21.15.62:0/290124978 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8abaffd640 1 --2- 172.21.15.62:0/290124978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 0x562877ce9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/290124978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cd94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.954+0000 7f8abbfff640 1 -- 172.21.15.62:0/290124978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 msgr2=0x562877ce9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.955+0000 7f8abbfff640 1 --2- 172.21.15.62:0/290124978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 0x562877ce9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.955+0000 7f8abbfff640 1 -- 172.21.15.62:0/290124978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 msgr2=0x562877cd94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.955+0000 7f8abbfff640 1 --2- 172.21.15.62:0/290124978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cd94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.955+0000 7f8abbfff640 1 -- 172.21.15.62:0/290124978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562877ccad70 con 0x562877ccd680 2024-09-12T00:05:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.955+0000 7f8abaffd640 1 --2- 172.21.15.62:0/290124978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 0x562877ce9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:05:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.955+0000 7f8abbfff640 1 --2- 172.21.15.62:0/290124978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ce96b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40257b0 tx=0x7f8ab40257e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837a852b3391d1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.955+0000 7f8aa17fa640 1 -- 172.21.15.62:0/290124978 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4036070 con 0x562877ccd680 2024-09-12T00:05:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.955+0000 7f8aa17fa640 1 -- 172.21.15.62:0/290124978 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab4003e50 con 0x562877ccd680 2024-09-12T00:05:40.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.955+0000 7f8aa17fa640 1 -- 172.21.15.62:0/290124978 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4021070 con 0x562877ccd680 2024-09-12T00:05:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 -- 172.21.15.62:0/290124978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 msgr2=0x562877ce96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/290124978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ce96b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40257b0 tx=0x7f8ab40257e0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 -- 172.21.15.62:0/290124978 shutdown_connections 2024-09-12T00:05:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/290124978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877ccd680 0x562877ce96b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/290124978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877cc2dc0 0x562877ce9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/290124978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cd94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 -- 172.21.15.62:0/290124978 >> 172.21.15.62:0/290124978 conn(0x562877cbea30 msgr2=0x562877cc8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 -- 172.21.15.62:0/290124978 shutdown_connections 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 -- 172.21.15.62:0/290124978 wait complete. 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 -- start start 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.956+0000 7f8ac3f62740 1 Processor -- start 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cd1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877cc2dc0 0x562877cd1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8ac3f62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877ccd680 0x562877cd1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1a9c80 con 0x562877cc2dc0 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1ab0b0 con 0x562877cc24a0 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8ac3f62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc1aa310 con 0x562877ccd680 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8ac3f62740 10 client.0 _ll_register_callbacks cb 0x7f8ab4010f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877cc2dc0 0x562877cd1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877cc2dc0 0x562877cd1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52128/0 (socket says 172.21.15.62:52128) 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abaffd640 1 -- 172.21.15.62:0/2029125086 learned_addr learned my addr 172.21.15.62:0/2029125086 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cd1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abbfff640 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877ccd680 0x562877cd1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abaffd640 1 -- 172.21.15.62:0/2029125086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877ccd680 msgr2=0x562877cd1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abaffd640 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877ccd680 0x562877cd1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abaffd640 1 -- 172.21.15.62:0/2029125086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 msgr2=0x562877cd1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:40.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abaffd640 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cd1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abaffd640 1 -- 172.21.15.62:0/2029125086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562877d4b2a0 con 0x562877cc2dc0 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8abaffd640 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877cc2dc0 0x562877cd1a40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009b60 tx=0x7f8ab0004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ded700186f47b77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.957+0000 7f8a9affd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8a9affd640 1 -- 172.21.15.62:0/2029125086 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0016040 con 0x562877cc2dc0 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8a9affd640 1 -- 172.21.15.62:0/2029125086 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f8ab0011880 con 0x562877cc2dc0 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8ac3f62740 1 -- 172.21.15.62:0/2029125086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ab4003e50 con 0x562877cc2dc0 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8a9affd640 1 -- 172.21.15.62:0/2029125086 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8ab0011ac0 con 0x562877cc2dc0 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8ac3f62740 10 client.17766 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8ac3f62740 1 -- 172.21.15.62:0/2029125086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562877d4c5a0 con 0x562877cc2dc0 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8a9affd640 1 -- 172.21.15.62:0/2029125086 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab4003e50 con 0x562877cc2dc0 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8a9affd640 1 -- 172.21.15.62:0/2029125086 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f8ab000f040 con 0x562877cc2dc0 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8ac3f62740 10 client.17766 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8ac3f62740 10 client.17766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8ac3f62740 20 client.17766 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562877d4cac0 0x562877d4eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8ac3f62740 4 client.17766 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8ac3f62740 1 -- 172.21.15.62:0/2029125086 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562877ccab00 con 0x562877d4cac0 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.958+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562877d4cac0 0x562877d4eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8abb7fe640 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562877d4cac0 0x562877d4eeb0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40de3d2879c0cd server_cookie=596c845361165cf6 in_seq=0 out_seq=0 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8a9affd640 10 client.17766 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8a9affd640 1 -- 172.21.15.62:0/2029125086 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562877ccab00 con 0x562877d4cac0 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8a9affd640 10 client.17766 handle_command_reply: tid=0 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 client.17766 shutdown 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 -- 172.21.15.62:0/2029125086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877cc2dc0 msgr2=0x562877cd1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877cc2dc0 0x562877cd1a40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009b60 tx=0x7f8ab0004990 comp rx=0 tx=0).stop 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 -- 172.21.15.62:0/2029125086 shutdown_connections 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562877d4cac0 0x562877d4eeb0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562877ccd680 0x562877cd1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562877cc2dc0 0x562877cd1a40 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8a9affd640 0 client.17766 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 --2- 172.21.15.62:0/2029125086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562877cc24a0 0x562877cd1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 -- 172.21.15.62:0/2029125086 >> 172.21.15.62:0/2029125086 conn(0x562877cbea30 msgr2=0x562877ccc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 -- 172.21.15.62:0/2029125086 shutdown_connections 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.959+0000 7f8ac3f62740 1 -- 172.21.15.62:0/2029125086 wait complete. 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac3f62740 20 client.17766 trim_cache size 0 max 16384 2024-09-12T00:05:40.965 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:05:40.966 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:05:40.966 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:05:40.966 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac1b89640 1 -- 172.21.15.62:0/3766482362 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8aa40a2cf0 msgr2=0x7f8aa40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac1b89640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8aa40a2cf0 0x7f8aa40a51b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009fd0 tx=0x7f8ab40092b0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac1b89640 1 -- 172.21.15.62:0/3766482362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc100a90 msgr2=0x7f8abc198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac1b89640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc100a90 0x7f8abc198420 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009230 tx=0x7f8aac004270 comp rx=0 tx=0).stop 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac1b89640 1 -- 172.21.15.62:0/3766482362 shutdown_connections 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac1b89640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f8aa40a2cf0 0x7f8aa40a51b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac1b89640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8abc106a90 0x7f8abc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac1b89640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8abc101430 0x7f8abc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac1b89640 1 --2- 172.21.15.62:0/3766482362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc100a90 0x7f8abc198420 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.960+0000 7f8ac1b89640 1 -- 172.21.15.62:0/3766482362 >> 172.21.15.62:0/3766482362 conn(0x7f8abc0fc7d0 msgr2=0x7f8abc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.961+0000 7f8ac1b89640 1 -- 172.21.15.62:0/3766482362 shutdown_connections 2024-09-12T00:05:40.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:40.961+0000 7f8ac1b89640 1 -- 172.21.15.62:0/3766482362 wait complete. 2024-09-12T00:05:41.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:40 smithi062 ceph-mon[30958]: pgmap v422: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 551 op/s 2024-09-12T00:05:41.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:40 smithi120 ceph-mon[35182]: pgmap v422: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 551 op/s 2024-09-12T00:05:41.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:40 smithi083 ceph-mon[34805]: pgmap v422: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 551 op/s 2024-09-12T00:05:41.865 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:05:41.865 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:05:42.129 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:42.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 780.0 s, 92.7 tps, lat 63.759 ms stddev 70.753 2024-09-12T00:05:42.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.593+0000 7f836e59b640 1 -- 172.21.15.62:0/1214214605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8368102790 msgr2=0x7f83681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.593+0000 7f836e59b640 1 --2- 172.21.15.62:0/1214214605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8368102790 0x7f83681067e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8350009900 tx=0x7f835002a410 comp rx=0 tx=0).stop 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.593+0000 7f836e59b640 1 -- 172.21.15.62:0/1214214605 shutdown_connections 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.593+0000 7f836e59b640 1 --2- 172.21.15.62:0/1214214605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368106f10 0x7f836810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.593+0000 7f836e59b640 1 --2- 172.21.15.62:0/1214214605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8368102790 0x7f83681067e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.593+0000 7f836e59b640 1 --2- 172.21.15.62:0/1214214605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8368101de0 0x7f83681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.593+0000 7f836e59b640 1 -- 172.21.15.62:0/1214214605 >> 172.21.15.62:0/1214214605 conn(0x7f83680fd8c0 msgr2=0x7f83680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.593+0000 7f836e59b640 1 -- 172.21.15.62:0/1214214605 shutdown_connections 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.593+0000 7f836e59b640 1 -- 172.21.15.62:0/1214214605 wait complete. 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.593+0000 7f836e59b640 1 Processor -- start 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f836e59b640 1 -- start start 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f836e59b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8368101de0 0x7f83680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f836e59b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8368102790 0x7f8368074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f836e59b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368106f10 0x7f8368074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f836e59b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8368111f70 con 0x7f8368101de0 2024-09-12T00:05:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f836e59b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8368111df0 con 0x7f8368102790 2024-09-12T00:05:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f836e59b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83681120f0 con 0x7f8368106f10 2024-09-12T00:05:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f8367fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8368101de0 0x7f83680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f8367fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8368101de0 0x7f83680744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42590/0 (socket says 172.21.15.62:42590) 2024-09-12T00:05:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f8367fff640 1 -- 172.21.15.62:0/3201597841 learned_addr learned my addr 172.21.15.62:0/3201597841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f836cb38640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368106f10 0x7f8368074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f835ffff640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8368102790 0x7f8368074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:42.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f8367fff640 1 -- 172.21.15.62:0/3201597841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368106f10 msgr2=0x7f8368074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:42.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f8367fff640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368106f10 0x7f8368074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:42.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f8367fff640 1 -- 172.21.15.62:0/3201597841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8368102790 msgr2=0x7f8368074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:42.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f8367fff640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8368102790 0x7f8368074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:42.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f8367fff640 1 -- 172.21.15.62:0/3201597841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83681a68e0 con 0x7f8368101de0 2024-09-12T00:05:42.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f835ffff640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8368102790 0x7f8368074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:05:42.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.594+0000 7f8367fff640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8368101de0 0x7f83680744f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f835800d950 tx=0x7f835800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:42.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.595+0000 7f8365ffb640 1 -- 172.21.15.62:0/3201597841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358017030 con 0x7f8368101de0 2024-09-12T00:05:42.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.595+0000 7f836e59b640 1 -- 172.21.15.62:0/3201597841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83681a6bd0 con 0x7f8368101de0 2024-09-12T00:05:42.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.595+0000 7f8365ffb640 1 -- 172.21.15.62:0/3201597841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8358004020 con 0x7f8368101de0 2024-09-12T00:05:42.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.595+0000 7f8365ffb640 1 -- 172.21.15.62:0/3201597841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580047a0 con 0x7f8368101de0 2024-09-12T00:05:42.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.595+0000 7f836e59b640 1 -- 172.21.15.62:0/3201597841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681a7160 con 0x7f8368101de0 2024-09-12T00:05:42.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.596+0000 7f836e59b640 1 -- 172.21.15.62:0/3201597841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83681021c0 con 0x7f8368101de0 2024-09-12T00:05:42.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.597+0000 7f8365ffb640 1 -- 172.21.15.62:0/3201597841 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f83580041d0 con 0x7f8368101de0 2024-09-12T00:05:42.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.598+0000 7f8365ffb640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83540a2cf0 0x7f83540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:42.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.598+0000 7f835ffff640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83540a2cf0 0x7f83540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:42.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.598+0000 7f8365ffb640 1 -- 172.21.15.62:0/3201597841 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f83580cf4d0 con 0x7f8368101de0 2024-09-12T00:05:42.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.598+0000 7f835ffff640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83540a2cf0 0x7f83540a51b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8368108800 tx=0x7f8350005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:42.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.601+0000 7f8365ffb640 1 -- 172.21.15.62:0/3201597841 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f835809c0f0 con 0x7f8368101de0 2024-09-12T00:05:42.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.813+0000 7f836e59b640 1 -- 172.21.15.62:0/3201597841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f836810a150 con 0x7f8368101de0 2024-09-12T00:05:42.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.814+0000 7f8365ffb640 1 -- 172.21.15.62:0/3201597841 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11910 (secure 0 0 0) 0x7f835809b9f0 con 0x7f8368101de0 2024-09-12T00:05:42.815 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:05:42.815 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"btime":"2024-09-12T00:05:17:893610+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:05:17.091935+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:05:42.815 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-09-12T00:05:42.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 -- 172.21.15.62:0/3201597841 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83540a2cf0 msgr2=0x7f83540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:42.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83540a2cf0 0x7f83540a51b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8368108800 tx=0x7f8350005a20 comp rx=0 tx=0).stop 2024-09-12T00:05:42.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 -- 172.21.15.62:0/3201597841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8368101de0 msgr2=0x7f83680744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:42.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8368101de0 0x7f83680744f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f835800d950 tx=0x7f835800de10 comp rx=0 tx=0).stop 2024-09-12T00:05:42.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 -- 172.21.15.62:0/3201597841 shutdown_connections 2024-09-12T00:05:42.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f83540a2cf0 0x7f83540a51b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:42.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368106f10 0x7f8368074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:42.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8368102790 0x7f8368074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:42.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 --2- 172.21.15.62:0/3201597841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8368101de0 0x7f83680744f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:42.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 -- 172.21.15.62:0/3201597841 >> 172.21.15.62:0/3201597841 conn(0x7f83680fd8c0 msgr2=0x7f83680feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:42.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 -- 172.21.15.62:0/3201597841 shutdown_connections 2024-09-12T00:05:42.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:42.816+0000 7f836e59b640 1 -- 172.21.15.62:0/3201597841 wait complete. 2024-09-12T00:05:43.189 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:42 smithi062 ceph-mon[30958]: pgmap v423: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 516 op/s 2024-09-12T00:05:43.189 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:42 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3201597841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:05:43.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:42 smithi120 ceph-mon[35182]: pgmap v423: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 516 op/s 2024-09-12T00:05:43.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:42 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3201597841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:05:43.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:42 smithi083 ceph-mon[34805]: pgmap v423: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 516 op/s 2024-09-12T00:05:43.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:42 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3201597841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:05:43.495 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:05:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.295+0000 7fdf563cb640 1 -- 172.21.15.62:0/3251697376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf50102790 msgr2=0x7fdf501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.295+0000 7fdf563cb640 1 --2- 172.21.15.62:0/3251697376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf50102790 0x7fdf501067e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009900 tx=0x7fdf4402a410 comp rx=0 tx=0).stop 2024-09-12T00:05:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.295+0000 7fdf563cb640 1 -- 172.21.15.62:0/3251697376 shutdown_connections 2024-09-12T00:05:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.295+0000 7fdf563cb640 1 --2- 172.21.15.62:0/3251697376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf50106f10 0x7fdf5010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.295+0000 7fdf563cb640 1 --2- 172.21.15.62:0/3251697376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf50102790 0x7fdf501067e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.295+0000 7fdf563cb640 1 --2- 172.21.15.62:0/3251697376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50101de0 0x7fdf501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.295+0000 7fdf563cb640 1 -- 172.21.15.62:0/3251697376 >> 172.21.15.62:0/3251697376 conn(0x7fdf500fd8c0 msgr2=0x7fdf500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.295+0000 7fdf563cb640 1 -- 172.21.15.62:0/3251697376 shutdown_connections 2024-09-12T00:05:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.295+0000 7fdf563cb640 1 -- 172.21.15.62:0/3251697376 wait complete. 2024-09-12T00:05:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf563cb640 1 Processor -- start 2024-09-12T00:05:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf563cb640 1 -- start start 2024-09-12T00:05:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf563cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf50101de0 0x7fdf50198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf563cb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf50102790 0x7fdf501989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf563cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50106f10 0x7fdf5019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf563cb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50111f70 con 0x7fdf50102790 2024-09-12T00:05:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf563cb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50111df0 con 0x7fdf50106f10 2024-09-12T00:05:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf563cb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501120f0 con 0x7fdf50101de0 2024-09-12T00:05:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf4f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf50102790 0x7fdf501989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf4f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf50102790 0x7fdf501989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42608/0 (socket says 172.21.15.62:42608) 2024-09-12T00:05:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.296+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/2590741752 learned_addr learned my addr 172.21.15.62:0/2590741752 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf50101de0 0x7fdf50198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf54968640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50106f10 0x7fdf5019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/2590741752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf50101de0 msgr2=0x7fdf50198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf50101de0 0x7fdf50198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/2590741752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50106f10 msgr2=0x7fdf5019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50106f10 0x7fdf5019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/2590741752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5019d440 con 0x7fdf50102790 2024-09-12T00:05:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf54968640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50106f10 0x7fdf5019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:05:44.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf50102790 0x7fdf501989d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fdf44006010 tx=0x7fdf440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:44.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf4d7fa640 1 -- 172.21.15.62:0/2590741752 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4403f030 con 0x7fdf50102790 2024-09-12T00:05:44.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf4d7fa640 1 -- 172.21.15.62:0/2590741752 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf44003ce0 con 0x7fdf50102790 2024-09-12T00:05:44.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf4d7fa640 1 -- 172.21.15.62:0/2590741752 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44035070 con 0x7fdf50102790 2024-09-12T00:05:44.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf563cb640 1 -- 172.21.15.62:0/2590741752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5019d6d0 con 0x7fdf50102790 2024-09-12T00:05:44.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.297+0000 7fdf563cb640 1 -- 172.21.15.62:0/2590741752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf501a5010 con 0x7fdf50102790 2024-09-12T00:05:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf4d7fa640 1 -- 172.21.15.62:0/2590741752 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdf44042030 con 0x7fdf50102790 2024-09-12T00:05:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf587a4740 1 Processor -- start 2024-09-12T00:05:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf587a4740 1 -- start start 2024-09-12T00:05:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c025b40 0x56527c024090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c024a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c02f370 0x56527c031760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf587a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56527bf8a2f0 con 0x56527c0245d0 2024-09-12T00:05:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf587a4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56527bf31cf0 con 0x56527c025b40 2024-09-12T00:05:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf587a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56527bf862b0 con 0x56527c02f370 2024-09-12T00:05:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c024a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c024a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42614/0 (socket says 172.21.15.62:42614) 2024-09-12T00:05:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf4ffff640 1 -- 172.21.15.62:0/3746315859 learned_addr learned my addr 172.21.15.62:0/3746315859 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf54968640 1 --2- 172.21.15.62:0/3746315859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c025b40 0x56527c024090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.300+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/3746315859 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c02f370 0x56527c031760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf4d7fa640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf380a2cf0 0x7fdf380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf4ffff640 1 -- 172.21.15.62:0/3746315859 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c02f370 msgr2=0x56527c031760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/3746315859 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c02f370 0x56527c031760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf4ffff640 1 -- 172.21.15.62:0/3746315859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c025b40 msgr2=0x56527c024090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/3746315859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c025b40 0x56527c024090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf4ffff640 1 -- 172.21.15.62:0/3746315859 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56527bf163a0 con 0x56527c0245d0 2024-09-12T00:05:44.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf54968640 1 --2- 172.21.15.62:0/3746315859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c025b40 0x56527c024090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:44.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf54968640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf380a2cf0 0x7fdf380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf4d7fa640 1 -- 172.21.15.62:0/2590741752 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fdf440efa30 con 0x7fdf50102790 2024-09-12T00:05:44.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf54968640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf380a2cf0 0x7fdf380a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdf48005fd0 tx=0x7fdf48004a40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:44.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/3746315859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c024a10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000bd10 tx=0x7fdf400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408968cf785b81a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:44.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf367fc640 1 -- 172.21.15.62:0/3746315859 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40004030 con 0x56527c0245d0 2024-09-12T00:05:44.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf367fc640 1 -- 172.21.15.62:0/3746315859 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf400041d0 con 0x56527c0245d0 2024-09-12T00:05:44.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.301+0000 7fdf367fc640 1 -- 172.21.15.62:0/3746315859 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf400117e0 con 0x56527c0245d0 2024-09-12T00:05:44.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.302+0000 7fdf587a4740 1 -- 172.21.15.62:0/3746315859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 msgr2=0x56527c024a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:44.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.302+0000 7fdf587a4740 1 --2- 172.21.15.62:0/3746315859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c024a10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000bd10 tx=0x7fdf400107a0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.302+0000 7fdf587a4740 1 -- 172.21.15.62:0/3746315859 shutdown_connections 2024-09-12T00:05:44.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.302+0000 7fdf587a4740 1 --2- 172.21.15.62:0/3746315859 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c02f370 0x56527c031760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.302+0000 7fdf587a4740 1 --2- 172.21.15.62:0/3746315859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c024a10 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.302+0000 7fdf587a4740 1 --2- 172.21.15.62:0/3746315859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c025b40 0x56527c024090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.302+0000 7fdf587a4740 1 -- 172.21.15.62:0/3746315859 >> 172.21.15.62:0/3746315859 conn(0x56527bfed310 msgr2=0x56527c020c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:44.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.302+0000 7fdf587a4740 1 -- 172.21.15.62:0/3746315859 shutdown_connections 2024-09-12T00:05:44.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.302+0000 7fdf587a4740 1 -- 172.21.15.62:0/3746315859 wait complete. 2024-09-12T00:05:44.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.302+0000 7fdf587a4740 1 -- start start 2024-09-12T00:05:44.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.303+0000 7fdf587a4740 1 Processor -- start 2024-09-12T00:05:44.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c0245d0 0x56527c02ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 0x56527c02bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c02f370 0x56527c02c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf587a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56527c032070 con 0x56527c02f370 2024-09-12T00:05:44.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf587a4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56527bf45060 con 0x56527c0245d0 2024-09-12T00:05:44.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf587a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56527c0321f0 con 0x56527c025b40 2024-09-12T00:05:44.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf587a4740 10 client.0 _ll_register_callbacks cb 0x56527bc988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:44.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf54968640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c02f370 0x56527c02c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf4f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 0x56527c02bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf54968640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c02f370 0x56527c02c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42618/0 (socket says 172.21.15.62:42618) 2024-09-12T00:05:44.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf54968640 1 -- 172.21.15.62:0/847870385 learned_addr learned my addr 172.21.15.62:0/847870385 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:44.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c0245d0 0x56527c02ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/847870385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c0245d0 msgr2=0x56527c02ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c0245d0 0x56527c02ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/847870385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c02f370 msgr2=0x56527c02c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c02f370 0x56527c02c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/847870385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56527bcfb3c0 con 0x56527c025b40 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 0x56527c02bff0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf440ed180 tx=0x7fdf44005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fae2ab27831e8366 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.304+0000 7fdf2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/847870385 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44049040 con 0x56527c025b40 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/847870385 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fdf440bc270 con 0x56527c025b40 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf587a4740 1 -- 172.21.15.62:0/847870385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf400041d0 con 0x56527c025b40 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/847870385 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdf440bbb50 con 0x56527c025b40 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf587a4740 10 client.27199 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:44.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf587a4740 1 -- 172.21.15.62:0/847870385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56527c032070 con 0x56527c025b40 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/847870385 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf400041d0 con 0x56527c025b40 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/847870385 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fdf440c47f0 con 0x56527c025b40 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf587a4740 10 client.27199 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf587a4740 10 client.27199 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf587a4740 20 client.27199 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf587a4740 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56527c03eee0 0x56527c041300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf587a4740 4 client.27199 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf587a4740 1 -- 172.21.15.62:0/847870385 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56527bf97860 con 0x56527c03eee0 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.305+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56527c03eee0 0x56527c041300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.306+0000 7fdf2f7fe640 10 client.27199 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.306+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56527c03eee0 0x56527c041300 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81ca25a5f5b4e303 server_cookie=8bda7c783adbbdaf in_seq=0 out_seq=0 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.306+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/847870385 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56527bf97860 con 0x56527c03eee0 2024-09-12T00:05:44.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.306+0000 7fdf2f7fe640 10 client.27199 handle_command_reply: tid=0 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 client.27199 shutdown 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 -- 172.21.15.62:0/847870385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 msgr2=0x56527c02bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 0x56527c02bff0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf440ed180 tx=0x7fdf44005e50 comp rx=0 tx=0).stop 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 -- 172.21.15.62:0/847870385 shutdown_connections 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56527c03eee0 0x56527c041300 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c02f370 0x56527c02c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 0x56527c02bff0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 --2- 172.21.15.62:0/847870385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c0245d0 0x56527c02ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf2f7fe640 0 client.27199 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 -- 172.21.15.62:0/847870385 >> 172.21.15.62:0/847870385 conn(0x56527bfed310 msgr2=0x56527c026960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 -- 172.21.15.62:0/847870385 shutdown_connections 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 1 -- 172.21.15.62:0/847870385 wait complete. 2024-09-12T00:05:44.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.307+0000 7fdf587a4740 20 client.27199 trim_cache size 0 max 16384 2024-09-12T00:05:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.320+0000 7fdf587a4740 1 Processor -- start 2024-09-12T00:05:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.320+0000 7fdf587a4740 1 -- start start 2024-09-12T00:05:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.320+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c020e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.320+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 0x56527c03b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.320+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c02f370 0x56527c046c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.320+0000 7fdf587a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501b0190 con 0x56527c0245d0 2024-09-12T00:05:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.320+0000 7fdf587a4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501ada60 con 0x56527c02f370 2024-09-12T00:05:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.320+0000 7fdf587a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501ab8d0 con 0x56527c025b40 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.320+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c020e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c020e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42630/0 (socket says 172.21.15.62:42630) 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf4ffff640 1 -- 172.21.15.62:0/2506704156 learned_addr learned my addr 172.21.15.62:0/2506704156 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/2506704156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 0x56527c03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf54968640 1 --2- 172.21.15.62:0/2506704156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c02f370 0x56527c046c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf4ffff640 1 -- 172.21.15.62:0/2506704156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 msgr2=0x56527c03b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/2506704156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 0x56527c03b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf4ffff640 1 -- 172.21.15.62:0/2506704156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c02f370 msgr2=0x56527c046c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/2506704156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c02f370 0x56527c046c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf4ffff640 1 -- 172.21.15.62:0/2506704156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56527c027120 con 0x56527c0245d0 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/2506704156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 0x56527c03b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:05:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/2506704156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c020e10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000bc30 tx=0x7fdf4000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4608195a6353bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf35ffb640 1 -- 172.21.15.62:0/2506704156 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40005240 con 0x56527c0245d0 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf35ffb640 1 -- 172.21.15.62:0/2506704156 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf400053e0 con 0x56527c0245d0 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.321+0000 7fdf35ffb640 1 -- 172.21.15.62:0/2506704156 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf400116a0 con 0x56527c0245d0 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.322+0000 7fdf587a4740 1 -- 172.21.15.62:0/2506704156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 msgr2=0x56527c020e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.322+0000 7fdf587a4740 1 --2- 172.21.15.62:0/2506704156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c020e10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000bc30 tx=0x7fdf4000bf90 comp rx=0 tx=0).stop 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.322+0000 7fdf587a4740 1 -- 172.21.15.62:0/2506704156 shutdown_connections 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.322+0000 7fdf587a4740 1 --2- 172.21.15.62:0/2506704156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c02f370 0x56527c046c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.322+0000 7fdf587a4740 1 --2- 172.21.15.62:0/2506704156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c025b40 0x56527c03b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.322+0000 7fdf587a4740 1 --2- 172.21.15.62:0/2506704156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c0245d0 0x56527c020e10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.322+0000 7fdf587a4740 1 -- 172.21.15.62:0/2506704156 >> 172.21.15.62:0/2506704156 conn(0x56527bfed310 msgr2=0x56527c020700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.322+0000 7fdf587a4740 1 -- 172.21.15.62:0/2506704156 shutdown_connections 2024-09-12T00:05:44.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.322+0000 7fdf587a4740 1 -- 172.21.15.62:0/2506704156 wait complete. 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.322+0000 7fdf587a4740 1 -- start start 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf587a4740 1 Processor -- start 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c0245d0 0x56527c0377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c025b40 0x56527c037ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf587a4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c02f370 0x56527c034590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf587a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501abf20 con 0x56527c025b40 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf587a4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501ad390 con 0x56527c0245d0 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf587a4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501ac5f0 con 0x56527c02f370 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf587a4740 10 client.0 _ll_register_callbacks cb 0x7fdf44005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf4f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c025b40 0x56527c037ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf4f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c025b40 0x56527c037ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42638/0 (socket says 172.21.15.62:42638) 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/2523707477 learned_addr learned my addr 172.21.15.62:0/2523707477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:44.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c0245d0 0x56527c0377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf54968640 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c02f370 0x56527c034590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/2523707477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c02f370 msgr2=0x56527c034590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c02f370 0x56527c034590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/2523707477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c0245d0 msgr2=0x56527c0377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c0245d0 0x56527c0377a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf4f7fe640 1 -- 172.21.15.62:0/2523707477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56527c034dc0 con 0x56527c025b40 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf4f7fe640 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c025b40 0x56527c037ce0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdf440ed780 tx=0x7fdf440c4a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd7120504678985 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.323+0000 7fdf2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/2523707477 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44049070 con 0x56527c025b40 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/2523707477 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fdf440bc470 con 0x56527c025b40 2024-09-12T00:05:44.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf587a4740 1 -- 172.21.15.62:0/2523707477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf400053e0 con 0x56527c025b40 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/2523707477 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdf440044e0 con 0x56527c025b40 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf587a4740 10 client.17790 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf587a4740 1 -- 172.21.15.62:0/2523707477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56527c034dc0 con 0x56527c025b40 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/2523707477 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf400053e0 con 0x56527c025b40 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/2523707477 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fdf4403bbb0 con 0x56527c025b40 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf587a4740 10 client.17790 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf587a4740 10 client.17790 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf587a4740 20 client.17790 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf587a4740 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56527c0aabb0 0x56527c0acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf587a4740 4 client.17790 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.324+0000 7fdf587a4740 1 -- 172.21.15.62:0/2523707477 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56527bf91d70 con 0x56527c0aabb0 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56527c0aabb0 0x56527c0acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf4ffff640 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56527c0aabb0 0x56527c0acfa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c3851855a013c77 server_cookie=dcb8b45251ae0bf in_seq=0 out_seq=0 2024-09-12T00:05:44.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf2f7fe640 10 client.17790 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf2f7fe640 1 -- 172.21.15.62:0/2523707477 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56527bf91d70 con 0x56527c0aabb0 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf2f7fe640 10 client.17790 handle_command_reply: tid=0 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 client.17790 shutdown 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 -- 172.21.15.62:0/2523707477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c025b40 msgr2=0x56527c037ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c025b40 0x56527c037ce0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdf440ed780 tx=0x7fdf440c4a00 comp rx=0 tx=0).stop 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 -- 172.21.15.62:0/2523707477 shutdown_connections 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x56527c0aabb0 0x56527c0acfa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56527c02f370 0x56527c034590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527c025b40 0x56527c037ce0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf2f7fe640 0 client.17790 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 --2- 172.21.15.62:0/2523707477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527c0245d0 0x56527c0377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 -- 172.21.15.62:0/2523707477 >> 172.21.15.62:0/2523707477 conn(0x56527bfed310 msgr2=0x56527c01f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:44.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 -- 172.21.15.62:0/2523707477 shutdown_connections 2024-09-12T00:05:44.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.325+0000 7fdf587a4740 1 -- 172.21.15.62:0/2523707477 wait complete. 2024-09-12T00:05:44.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf587a4740 20 client.17790 trim_cache size 0 max 16384 2024-09-12T00:05:44.338 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 -- 172.21.15.62:0/2590741752 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf380a2cf0 msgr2=0x7fdf380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf380a2cf0 0x7fdf380a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdf48005fd0 tx=0x7fdf48004a40 comp rx=0 tx=0).stop 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 -- 172.21.15.62:0/2590741752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf50102790 msgr2=0x7fdf501989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf50102790 0x7fdf501989d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fdf44006010 tx=0x7fdf440042f0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 -- 172.21.15.62:0/2590741752 shutdown_connections 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdf380a2cf0 0x7fdf380a51b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50106f10 0x7fdf5019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf50102790 0x7fdf501989d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 --2- 172.21.15.62:0/2590741752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf50101de0 0x7fdf50198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 -- 172.21.15.62:0/2590741752 >> 172.21.15.62:0/2590741752 conn(0x7fdf500fd8c0 msgr2=0x7fdf500fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 -- 172.21.15.62:0/2590741752 shutdown_connections 2024-09-12T00:05:44.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:44.326+0000 7fdf563cb640 1 -- 172.21.15.62:0/2590741752 wait complete. 2024-09-12T00:05:45.006 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:05:45.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:44 smithi062 ceph-mon[30958]: pgmap v424: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.0 MiB/s wr, 539 op/s 2024-09-12T00:05:45.271 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:45.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:44 smithi120 ceph-mon[35182]: pgmap v424: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.0 MiB/s wr, 539 op/s 2024-09-12T00:05:45.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:44 smithi083 ceph-mon[34805]: pgmap v424: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.0 MiB/s wr, 539 op/s 2024-09-12T00:05:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.722+0000 7fadd7c98640 1 -- 172.21.15.62:0/3521717736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadd01082e0 msgr2=0x7fadd010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.722+0000 7fadd7c98640 1 --2- 172.21.15.62:0/3521717736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadd01082e0 0x7fadd010ec50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009cd0 tx=0x7fadc002a8e0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.722+0000 7fadd7c98640 1 -- 172.21.15.62:0/3521717736 shutdown_connections 2024-09-12T00:05:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.722+0000 7fadd7c98640 1 --2- 172.21.15.62:0/3521717736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadd01082e0 0x7fadd010ec50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.722+0000 7fadd7c98640 1 --2- 172.21.15.62:0/3521717736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadd0103760 0x7fadd0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.722+0000 7fadd7c98640 1 --2- 172.21.15.62:0/3521717736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadd0102db0 0x7fadd0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.722+0000 7fadd7c98640 1 -- 172.21.15.62:0/3521717736 >> 172.21.15.62:0/3521717736 conn(0x7fadd00fc7d0 msgr2=0x7fadd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.722+0000 7fadd7c98640 1 -- 172.21.15.62:0/3521717736 shutdown_connections 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.722+0000 7fadd7c98640 1 -- 172.21.15.62:0/3521717736 wait complete. 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd7c98640 1 Processor -- start 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd7c98640 1 -- start start 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd7c98640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadd0102db0 0x7fadd0100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd7c98640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadd0103760 0x7fadd0100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd7c98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadd01082e0 0x7fadd0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd7c98640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01120b0 con 0x7fadd0103760 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd7c98640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0111f30 con 0x7fadd01082e0 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd7c98640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0112230 con 0x7fadd0102db0 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd5233640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadd0103760 0x7fadd0100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd5233640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadd0103760 0x7fadd0100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42646/0 (socket says 172.21.15.62:42646) 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd5233640 1 -- 172.21.15.62:0/25272578 learned_addr learned my addr 172.21.15.62:0/25272578 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd5a34640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadd0102db0 0x7fadd0100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd6235640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadd01082e0 0x7fadd0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd5233640 1 -- 172.21.15.62:0/25272578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadd0102db0 msgr2=0x7fadd0100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd5233640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadd0102db0 0x7fadd0100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd5233640 1 -- 172.21.15.62:0/25272578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadd01082e0 msgr2=0x7fadd0101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd5233640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadd01082e0 0x7fadd0101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.723+0000 7fadd5233640 1 -- 172.21.15.62:0/25272578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd01a6a70 con 0x7fadd0103760 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.724+0000 7fadd5233640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadd0103760 0x7fadd0100f50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fadc400c970 tx=0x7fadc400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.724+0000 7fadceffd640 1 -- 172.21.15.62:0/25272578 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4004220 con 0x7fadd0103760 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.724+0000 7fadd7c98640 1 -- 172.21.15.62:0/25272578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd01a6d60 con 0x7fadd0103760 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.724+0000 7fadceffd640 1 -- 172.21.15.62:0/25272578 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc40043c0 con 0x7fadd0103760 2024-09-12T00:05:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.724+0000 7fadceffd640 1 -- 172.21.15.62:0/25272578 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4004a00 con 0x7fadd0103760 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.724+0000 7fadd7c98640 1 -- 172.21.15.62:0/25272578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd01a71f0 con 0x7fadd0103760 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.726+0000 7fadceffd640 1 -- 172.21.15.62:0/25272578 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fadc40079b0 con 0x7fadd0103760 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.726+0000 7fadda071740 1 Processor -- start 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadda071740 1 -- start start 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadda071740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a354ab40 0x55b4a3549090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadda071740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a35495d0 0x55b4a3549a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadda071740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a3556760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadda071740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b4a34af2f0 con 0x55b4a3554370 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadda071740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b4a3456cf0 con 0x55b4a354ab40 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadda071740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b4a34ab2b0 con 0x55b4a35495d0 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadd5233640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a3556760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadd5233640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a3556760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42650/0 (socket says 172.21.15.62:42650) 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadd5233640 1 -- 172.21.15.62:0/2684997923 learned_addr learned my addr 172.21.15.62:0/2684997923 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadceffd640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadb80a2cf0 0x7fadb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadd5a34640 1 --2- 172.21.15.62:0/2684997923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a35495d0 0x55b4a3549a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadd6235640 1 --2- 172.21.15.62:0/2684997923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a354ab40 0x55b4a3549090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadd6235640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadb80a2cf0 0x7fadb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadceffd640 1 -- 172.21.15.62:0/25272578 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fadc40cf000 con 0x7fadd0103760 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadd5233640 1 -- 172.21.15.62:0/2684997923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a35495d0 msgr2=0x55b4a3549a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadd5233640 1 --2- 172.21.15.62:0/2684997923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a35495d0 0x55b4a3549a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadd5233640 1 -- 172.21.15.62:0/2684997923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a354ab40 msgr2=0x55b4a3549090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.727+0000 7fadd5233640 1 --2- 172.21.15.62:0/2684997923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a354ab40 0x55b4a3549090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.728+0000 7fadd5233640 1 -- 172.21.15.62:0/2684997923 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a343b3a0 con 0x55b4a3554370 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.728+0000 7fadd5233640 1 --2- 172.21.15.62:0/2684997923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a3556760 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fadc4097fa0 tx=0x7fadc40053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12451c18255abb54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:45.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.728+0000 7fadd6235640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadb80a2cf0 0x7fadb80a51b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009a60 tx=0x7fadc0009800 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.728+0000 7fadb7fff640 1 -- 172.21.15.62:0/2684997923 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc401d070 con 0x55b4a3554370 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.728+0000 7fadb7fff640 1 -- 172.21.15.62:0/2684997923 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc409b5d0 con 0x55b4a3554370 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.728+0000 7fadb7fff640 1 -- 172.21.15.62:0/2684997923 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4097c40 con 0x55b4a3554370 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 -- 172.21.15.62:0/2684997923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 msgr2=0x55b4a3556760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 --2- 172.21.15.62:0/2684997923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a3556760 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fadc4097fa0 tx=0x7fadc40053f0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 -- 172.21.15.62:0/2684997923 shutdown_connections 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 --2- 172.21.15.62:0/2684997923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a3556760 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 --2- 172.21.15.62:0/2684997923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a35495d0 0x55b4a3549a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 --2- 172.21.15.62:0/2684997923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a354ab40 0x55b4a3549090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 -- 172.21.15.62:0/2684997923 >> 172.21.15.62:0/2684997923 conn(0x55b4a3512310 msgr2=0x55b4a3545c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 -- 172.21.15.62:0/2684997923 shutdown_connections 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 -- 172.21.15.62:0/2684997923 wait complete. 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 -- start start 2024-09-12T00:05:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.729+0000 7fadda071740 1 Processor -- start 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadda071740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a3553db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadda071740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a3551060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadda071740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 0x55b4a35517f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadda071740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b4a3556fb0 con 0x55b4a35495d0 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadda071740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b4a346a060 con 0x55b4a3554370 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadda071740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b4a3557130 con 0x55b4a354ab40 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadda071740 10 client.0 _ll_register_callbacks cb 0x55b4a31bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5a34640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a3553db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5a34640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a3553db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42666/0 (socket says 172.21.15.62:42666) 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5a34640 1 -- 172.21.15.62:0/3509302136 learned_addr learned my addr 172.21.15.62:0/3509302136 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd6235640 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 0x55b4a35517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5233640 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a3551060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5a34640 1 -- 172.21.15.62:0/3509302136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 msgr2=0x55b4a3551060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5a34640 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a3551060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5a34640 1 -- 172.21.15.62:0/3509302136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 msgr2=0x55b4a35517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5a34640 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 0x55b4a35517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5a34640 1 -- 172.21.15.62:0/3509302136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a32203c0 con 0x55b4a35495d0 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5233640 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a3551060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadd5a34640 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a3553db0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fadc8010820 tx=0x7fadc8010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe78e8cb6e17447c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadb57fa640 1 -- 172.21.15.62:0/3509302136 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc800fb90 con 0x55b4a35495d0 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadb57fa640 1 -- 172.21.15.62:0/3509302136 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fadc800fd30 con 0x55b4a35495d0 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.730+0000 7fadda071740 1 -- 172.21.15.62:0/3509302136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc409b5d0 con 0x55b4a35495d0 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadb57fa640 1 -- 172.21.15.62:0/3509302136 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fadc8005020 con 0x55b4a35495d0 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadda071740 10 client.17802 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadda071740 1 -- 172.21.15.62:0/3509302136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b4a3562b00 con 0x55b4a35495d0 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadb57fa640 1 -- 172.21.15.62:0/3509302136 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc409b5d0 con 0x55b4a35495d0 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadb57fa640 1 -- 172.21.15.62:0/3509302136 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fadc8022460 con 0x55b4a35495d0 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadda071740 10 client.17802 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadda071740 10 client.17802 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadda071740 20 client.17802 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadda071740 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55b4a3563fc0 0x55b4a3566460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadda071740 4 client.17802 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadda071740 1 -- 172.21.15.62:0/3509302136 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4a34bc860 con 0x55b4a3563fc0 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.731+0000 7fadd5233640 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55b4a3563fc0 0x55b4a3566460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.732+0000 7fadb57fa640 10 client.17802 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:05:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.732+0000 7fadd5233640 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55b4a3563fc0 0x55b4a3566460 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bec6d34b639b8734 server_cookie=aa9d6c94e0145e45 in_seq=0 out_seq=0 2024-09-12T00:05:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadb57fa640 1 -- 172.21.15.62:0/3509302136 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4a34bc860 con 0x55b4a3563fc0 2024-09-12T00:05:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadb57fa640 10 client.17802 handle_command_reply: tid=0 2024-09-12T00:05:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 client.17802 shutdown 2024-09-12T00:05:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 -- 172.21.15.62:0/3509302136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 msgr2=0x55b4a3553db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a3553db0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fadc8010820 tx=0x7fadc8010c20 comp rx=0 tx=0).stop 2024-09-12T00:05:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 -- 172.21.15.62:0/3509302136 shutdown_connections 2024-09-12T00:05:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55b4a3563fc0 0x55b4a3566460 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 0x55b4a35517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a3551060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 --2- 172.21.15.62:0/3509302136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a3553db0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadb57fa640 0 client.17802 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 -- 172.21.15.62:0/3509302136 >> 172.21.15.62:0/3509302136 conn(0x55b4a3512310 msgr2=0x55b4a354b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 -- 172.21.15.62:0/3509302136 shutdown_connections 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 1 -- 172.21.15.62:0/3509302136 wait complete. 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.733+0000 7fadda071740 20 client.17802 trim_cache size 0 max 16384 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadda071740 1 Processor -- start 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadda071740 1 -- start start 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadda071740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a35495d0 0x55b4a354c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadda071740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a354c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadda071740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a356bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadda071740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01b23b0 con 0x55b4a3554370 2024-09-12T00:05:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadda071740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01afc80 con 0x55b4a35495d0 2024-09-12T00:05:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadda071740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01adaf0 con 0x55b4a354ab40 2024-09-12T00:05:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadd6235640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a356bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadd6235640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a356bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42668/0 (socket says 172.21.15.62:42668) 2024-09-12T00:05:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadd6235640 1 -- 172.21.15.62:0/4048398276 learned_addr learned my addr 172.21.15.62:0/4048398276 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadd5233640 1 --2- 172.21.15.62:0/4048398276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a354c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadd5a34640 1 --2- 172.21.15.62:0/4048398276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a35495d0 0x55b4a354c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadd6235640 1 -- 172.21.15.62:0/4048398276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 msgr2=0x55b4a354c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadd6235640 1 --2- 172.21.15.62:0/4048398276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a354c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadd6235640 1 -- 172.21.15.62:0/4048398276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a35495d0 msgr2=0x55b4a354c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadd6235640 1 --2- 172.21.15.62:0/4048398276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a35495d0 0x55b4a354c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.746+0000 7fadd6235640 1 -- 172.21.15.62:0/4048398276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a3551b60 con 0x55b4a3554370 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadd6235640 1 --2- 172.21.15.62:0/4048398276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a356bd80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009cd0 tx=0x7fadc0036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69592cf31601b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadb77fe640 1 -- 172.21.15.62:0/4048398276 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0059070 con 0x55b4a3554370 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadb77fe640 1 -- 172.21.15.62:0/4048398276 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc0036070 con 0x55b4a3554370 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadb77fe640 1 -- 172.21.15.62:0/4048398276 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc004a440 con 0x55b4a3554370 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadda071740 1 -- 172.21.15.62:0/4048398276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 msgr2=0x55b4a356bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadda071740 1 --2- 172.21.15.62:0/4048398276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a356bd80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009cd0 tx=0x7fadc0036000 comp rx=0 tx=0).stop 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadda071740 1 -- 172.21.15.62:0/4048398276 shutdown_connections 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadda071740 1 --2- 172.21.15.62:0/4048398276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a3554370 0x55b4a356bd80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadda071740 1 --2- 172.21.15.62:0/4048398276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a354c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadda071740 1 --2- 172.21.15.62:0/4048398276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a35495d0 0x55b4a354c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadda071740 1 -- 172.21.15.62:0/4048398276 >> 172.21.15.62:0/4048398276 conn(0x55b4a3512310 msgr2=0x55b4a3550150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadda071740 1 -- 172.21.15.62:0/4048398276 shutdown_connections 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.747+0000 7fadda071740 1 -- 172.21.15.62:0/4048398276 wait complete. 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadda071740 1 -- start start 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadda071740 1 Processor -- start 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadda071740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a355c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadda071740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a3557cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadda071740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 0x55b4a35581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadda071740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01ae140 con 0x55b4a35495d0 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadda071740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01af5b0 con 0x55b4a3554370 2024-09-12T00:05:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadda071740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01ae810 con 0x55b4a354ab40 2024-09-12T00:05:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadda071740 10 client.0 _ll_register_callbacks cb 0x7fadc800a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadd5a34640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a355c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadd5a34640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a355c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42680/0 (socket says 172.21.15.62:42680) 2024-09-12T00:05:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadd5a34640 1 -- 172.21.15.62:0/4132687923 learned_addr learned my addr 172.21.15.62:0/4132687923 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.748+0000 7fadd5233640 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a3557cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadd6235640 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 0x55b4a35581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadd5a34640 1 -- 172.21.15.62:0/4132687923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 msgr2=0x55b4a3557cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadd5a34640 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a3557cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadd5a34640 1 -- 172.21.15.62:0/4132687923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 msgr2=0x55b4a35581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadd5a34640 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 0x55b4a35581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadd5a34640 1 -- 172.21.15.62:0/4132687923 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a3558860 con 0x55b4a35495d0 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadd6235640 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 0x55b4a35581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadd5a34640 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a355c980 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fadc80107f0 tx=0x7fadc8022af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55f0d3d514689d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadb57fa640 1 -- 172.21.15.62:0/4132687923 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc800fb90 con 0x55b4a35495d0 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadb57fa640 1 -- 172.21.15.62:0/4132687923 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fadc800fd30 con 0x55b4a35495d0 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadda071740 1 -- 172.21.15.62:0/4132687923 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc0036070 con 0x55b4a35495d0 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadb57fa640 1 -- 172.21.15.62:0/4132687923 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fadc8038050 con 0x55b4a35495d0 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadda071740 10 client.17808 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadda071740 1 -- 172.21.15.62:0/4132687923 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b4a35cebc0 con 0x55b4a35495d0 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.749+0000 7fadb57fa640 1 -- 172.21.15.62:0/4132687923 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc0036070 con 0x55b4a35495d0 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.750+0000 7fadb57fa640 1 -- 172.21.15.62:0/4132687923 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fadc802a3f0 con 0x55b4a35495d0 2024-09-12T00:05:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.750+0000 7fadda071740 10 client.17808 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:45.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.750+0000 7fadda071740 10 client.17808 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:05:45.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.750+0000 7fadda071740 20 client.17808 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:45.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.750+0000 7fadda071740 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55b4a35cf390 0x55b4a35d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:45.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.750+0000 7fadda071740 4 client.17808 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:05:45.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.750+0000 7fadda071740 1 -- 172.21.15.62:0/4132687923 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4a34b6d70 con 0x55b4a35cf390 2024-09-12T00:05:45.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.750+0000 7fadd5233640 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55b4a35cf390 0x55b4a35d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:45.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.750+0000 7fadb57fa640 10 client.17808 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:05:45.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.750+0000 7fadd5233640 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55b4a35cf390 0x55b4a35d17b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=353e8f5e6773c87 server_cookie=b4c16b3c39cbbb97 in_seq=0 out_seq=0 2024-09-12T00:05:45.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadb57fa640 1 -- 172.21.15.62:0/4132687923 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4a34b6d70 con 0x55b4a35cf390 2024-09-12T00:05:45.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadb57fa640 10 client.17808 handle_command_reply: tid=0 2024-09-12T00:05:45.772 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:05:45.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 client.17808 shutdown 2024-09-12T00:05:45.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 -- 172.21.15.62:0/4132687923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 msgr2=0x55b4a355c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a355c980 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fadc80107f0 tx=0x7fadc8022af0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 -- 172.21.15.62:0/4132687923 shutdown_connections 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x55b4a35cf390 0x55b4a35d17b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadb57fa640 0 client.17808 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4a3554370 0x55b4a35581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4a354ab40 0x55b4a3557cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 --2- 172.21.15.62:0/4132687923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4a35495d0 0x55b4a355c980 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 -- 172.21.15.62:0/4132687923 >> 172.21.15.62:0/4132687923 conn(0x55b4a3512310 msgr2=0x55b4a35444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 -- 172.21.15.62:0/4132687923 shutdown_connections 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 1 -- 172.21.15.62:0/4132687923 wait complete. 2024-09-12T00:05:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.751+0000 7fadda071740 20 client.17808 trim_cache size 0 max 16384 2024-09-12T00:05:45.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 -- 172.21.15.62:0/25272578 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadb80a2cf0 msgr2=0x7fadb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:45.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadb80a2cf0 0x7fadb80a51b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009a60 tx=0x7fadc0009800 comp rx=0 tx=0).stop 2024-09-12T00:05:45.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 -- 172.21.15.62:0/25272578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadd0103760 msgr2=0x7fadd0100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:45.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadd0103760 0x7fadd0100f50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fadc400c970 tx=0x7fadc400ce30 comp rx=0 tx=0).stop 2024-09-12T00:05:45.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 -- 172.21.15.62:0/25272578 shutdown_connections 2024-09-12T00:05:45.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fadb80a2cf0 0x7fadb80a51b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadd01082e0 0x7fadd0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadd0103760 0x7fadd0100f50 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 --2- 172.21.15.62:0/25272578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadd0102db0 0x7fadd0100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:45.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 -- 172.21.15.62:0/25272578 >> 172.21.15.62:0/25272578 conn(0x7fadd00fc7d0 msgr2=0x7fadd0105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:45.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 -- 172.21.15.62:0/25272578 shutdown_connections 2024-09-12T00:05:45.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:45.752+0000 7fadd7c98640 1 -- 172.21.15.62:0/25272578 wait complete. 2024-09-12T00:05:46.534 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:05:46.795 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:47.169 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:46 smithi062 ceph-mon[30958]: pgmap v425: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.5 MiB/s wr, 270 op/s 2024-09-12T00:05:47.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 -- 172.21.15.62:0/4137027507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70fc100a80 msgr2=0x7f70fc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:47.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 --2- 172.21.15.62:0/4137027507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70fc100a80 0x7f70fc100ee0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009930 tx=0x7f70f002a420 comp rx=0 tx=0).stop 2024-09-12T00:05:47.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 -- 172.21.15.62:0/4137027507 shutdown_connections 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 --2- 172.21.15.62:0/4137027507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70fc101420 0x7f70fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 --2- 172.21.15.62:0/4137027507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70fc100a80 0x7f70fc100ee0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 --2- 172.21.15.62:0/4137027507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70fc106a80 0x7f70fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 -- 172.21.15.62:0/4137027507 >> 172.21.15.62:0/4137027507 conn(0x7f70fc0fc7a0 msgr2=0x7f70fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 -- 172.21.15.62:0/4137027507 shutdown_connections 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 -- 172.21.15.62:0/4137027507 wait complete. 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 Processor -- start 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.249+0000 7f7102241640 1 -- start start 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f7102241640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70fc100a80 0x7f70fc1109d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f7102241640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70fc101420 0x7f70fc110f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f7102241640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70fc106a80 0x7f70fc111450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f7102241640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc06ad80 con 0x7f70fc101420 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f7102241640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc06ac00 con 0x7f70fc100a80 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f7102241640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc06af00 con 0x7f70fc106a80 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70faffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70fc101420 0x7f70fc110f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70faffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70fc101420 0x7f70fc110f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42704/0 (socket says 172.21.15.62:42704) 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70faffd640 1 -- 172.21.15.62:0/3174555856 learned_addr learned my addr 172.21.15.62:0/3174555856 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70fc100a80 0x7f70fc1109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70fbfff640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70fc106a80 0x7f70fc111450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70faffd640 1 -- 172.21.15.62:0/3174555856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70fc106a80 msgr2=0x7f70fc111450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70faffd640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70fc106a80 0x7f70fc111450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70faffd640 1 -- 172.21.15.62:0/3174555856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70fc100a80 msgr2=0x7f70fc1109d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:47.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70faffd640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70fc100a80 0x7f70fc1109d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70faffd640 1 -- 172.21.15.62:0/3174555856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc1a6a30 con 0x7f70fc101420 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70fc100a80 0x7f70fc1109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.250+0000 7f70faffd640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70fc101420 0x7f70fc110f10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f70f002a940 tx=0x7f70f0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.251+0000 7f70f8ff9640 1 -- 172.21.15.62:0/3174555856 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f003f070 con 0x7f70fc101420 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.251+0000 7f70f8ff9640 1 -- 172.21.15.62:0/3174555856 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f0031c90 con 0x7f70fc101420 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.251+0000 7f7102241640 1 -- 172.21.15.62:0/3174555856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70fc1a6cc0 con 0x7f70fc101420 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.251+0000 7f70f8ff9640 1 -- 172.21.15.62:0/3174555856 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0035030 con 0x7f70fc101420 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.251+0000 7f7102241640 1 -- 172.21.15.62:0/3174555856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70fc1a71a0 con 0x7f70fc101420 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.253+0000 7f70f8ff9640 1 -- 172.21.15.62:0/3174555856 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f70f0042070 con 0x7f70fc101420 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.253+0000 7f710461a740 1 Processor -- start 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.253+0000 7f710461a740 1 -- start start 2024-09-12T00:05:47.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f710461a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1d3b40 0x56418c1d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f710461a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d25d0 0x56418c1d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f710461a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1dd370 0x56418c1df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f710461a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56418c1382f0 con 0x56418c1dd370 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f710461a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56418c0dfcf0 con 0x56418c1d25d0 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f710461a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56418c1342b0 con 0x56418c1d3b40 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70f8ff9640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f70e40a2cf0 0x7f70e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70fbfff640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f70e40a2cf0 0x7f70e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70faffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1dd370 0x56418c1df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70faffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1dd370 0x56418c1df760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42718/0 (socket says 172.21.15.62:42718) 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70faffd640 1 -- 172.21.15.62:0/933926717 learned_addr learned my addr 172.21.15.62:0/933926717 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70fbfff640 1 --2- 172.21.15.62:0/933926717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1d3b40 0x56418c1d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70f8ff9640 1 -- 172.21.15.62:0/3174555856 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f70f00efca0 con 0x7f70fc101420 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/933926717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d25d0 0x56418c1d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70faffd640 1 -- 172.21.15.62:0/933926717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1d3b40 msgr2=0x56418c1d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70faffd640 1 --2- 172.21.15.62:0/933926717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1d3b40 0x56418c1d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70faffd640 1 -- 172.21.15.62:0/933926717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d25d0 msgr2=0x56418c1d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70faffd640 1 --2- 172.21.15.62:0/933926717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d25d0 0x56418c1d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70faffd640 1 -- 172.21.15.62:0/933926717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418c0c43a0 con 0x56418c1dd370 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70fbfff640 1 --2- 172.21.15.62:0/933926717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1d3b40 0x56418c1d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70fbfff640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f70e40a2cf0 0x7f70e40a51b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f70f4005fd0 tx=0x7f70f4009a30 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.254+0000 7f70faffd640 1 --2- 172.21.15.62:0/933926717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1dd370 0x56418c1df760 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009900 tx=0x7f70f002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c8f78fd0c6a2e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f70e1ffb640 1 -- 172.21.15.62:0/933926717 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f00bbb20 con 0x56418c1dd370 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f70e1ffb640 1 -- 172.21.15.62:0/933926717 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f00bc8c0 con 0x56418c1dd370 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f70e1ffb640 1 -- 172.21.15.62:0/933926717 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f00d1ab0 con 0x56418c1dd370 2024-09-12T00:05:47.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f710461a740 1 -- 172.21.15.62:0/933926717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1dd370 msgr2=0x56418c1df760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f710461a740 1 --2- 172.21.15.62:0/933926717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1dd370 0x56418c1df760 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009900 tx=0x7f70f002a420 comp rx=0 tx=0).stop 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f710461a740 1 -- 172.21.15.62:0/933926717 shutdown_connections 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f710461a740 1 --2- 172.21.15.62:0/933926717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1dd370 0x56418c1df760 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f710461a740 1 --2- 172.21.15.62:0/933926717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d25d0 0x56418c1d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f710461a740 1 --2- 172.21.15.62:0/933926717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1d3b40 0x56418c1d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f710461a740 1 -- 172.21.15.62:0/933926717 >> 172.21.15.62:0/933926717 conn(0x56418c19b310 msgr2=0x56418c1cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f710461a740 1 -- 172.21.15.62:0/933926717 shutdown_connections 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.255+0000 7f710461a740 1 -- 172.21.15.62:0/933926717 wait complete. 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.256+0000 7f710461a740 1 -- start start 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.256+0000 7f710461a740 1 Processor -- start 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.256+0000 7f710461a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.256+0000 7f710461a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.256+0000 7f710461a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1da400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.256+0000 7f710461a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56418c1dfe30 con 0x56418c1d25d0 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.256+0000 7f710461a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56418c0f3060 con 0x56418c1d3b40 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.256+0000 7f710461a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56418c1dffb0 con 0x56418c1dd370 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.256+0000 7f710461a740 10 client.0 _ll_register_callbacks cb 0x56418be468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.256+0000 7f70fb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70fb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1dcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42726/0 (socket says 172.21.15.62:42726) 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70fb7fe640 1 -- 172.21.15.62:0/1323059016 learned_addr learned my addr 172.21.15.62:0/1323059016 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70faffd640 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70fbfff640 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1da400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70fb7fe640 1 -- 172.21.15.62:0/1323059016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 msgr2=0x56418c1da400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:47.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1da400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70fb7fe640 1 -- 172.21.15.62:0/1323059016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 msgr2=0x56418c1d9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70fb7fe640 1 -- 172.21.15.62:0/1323059016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418bea93c0 con 0x56418c1d25d0 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70fbfff640 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1da400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1dcc10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00f4f0 tx=0x7f70ec00f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3d95e501834cee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70daffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70daffd640 1 -- 172.21.15.62:0/1323059016 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec013850 con 0x56418c1d25d0 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70daffd640 1 -- 172.21.15.62:0/1323059016 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f70ec0139f0 con 0x56418c1d25d0 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f710461a740 1 -- 172.21.15.62:0/1323059016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f00bc8c0 con 0x56418c1d25d0 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f70daffd640 1 -- 172.21.15.62:0/1323059016 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f70ec013bf0 con 0x56418c1d25d0 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f710461a740 10 client.17820 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.257+0000 7f710461a740 1 -- 172.21.15.62:0/1323059016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56418c1eb7b0 con 0x56418c1d25d0 2024-09-12T00:05:47.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f70daffd640 1 -- 172.21.15.62:0/1323059016 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f00bc8c0 con 0x56418c1d25d0 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f70daffd640 1 -- 172.21.15.62:0/1323059016 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f70ec0113e0 con 0x56418c1d25d0 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f710461a740 10 client.17820 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f710461a740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f710461a740 20 client.17820 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f710461a740 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56418c1ecc70 0x56418c1ef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f710461a740 4 client.17820 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f710461a740 1 -- 172.21.15.62:0/1323059016 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56418c145860 con 0x56418c1ecc70 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f70faffd640 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56418c1ecc70 0x56418c1ef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f70daffd640 10 client.17820 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.258+0000 7f70faffd640 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56418c1ecc70 0x56418c1ef0e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c034bbfc8dd364 server_cookie=2295d39faa9a7161 in_seq=0 out_seq=0 2024-09-12T00:05:47.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f70daffd640 1 -- 172.21.15.62:0/1323059016 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56418c145860 con 0x56418c1ecc70 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f70daffd640 10 client.17820 handle_command_reply: tid=0 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 client.17820 shutdown 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 -- 172.21.15.62:0/1323059016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 msgr2=0x56418c1dcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1dcc10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00f4f0 tx=0x7f70ec00f9b0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 -- 172.21.15.62:0/1323059016 shutdown_connections 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56418c1ecc70 0x56418c1ef0e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1da400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 --2- 172.21.15.62:0/1323059016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1dcc10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f70daffd640 0 client.17820 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 -- 172.21.15.62:0/1323059016 >> 172.21.15.62:0/1323059016 conn(0x56418c19b310 msgr2=0x56418c1d4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 -- 172.21.15.62:0/1323059016 shutdown_connections 2024-09-12T00:05:47.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.259+0000 7f710461a740 1 -- 172.21.15.62:0/1323059016 wait complete. 2024-09-12T00:05:47.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.260+0000 7f710461a740 20 client.17820 trim_cache size 0 max 16384 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f710461a740 1 Processor -- start 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f710461a740 1 -- start start 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f710461a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f710461a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f710461a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f710461a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc1b2330 con 0x56418c1d25d0 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f710461a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc1afc00 con 0x56418c1d3b40 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f710461a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc1ada70 con 0x56418c1dd370 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f70fb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f70fb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1d5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42736/0 (socket says 172.21.15.62:42736) 2024-09-12T00:05:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f70fb7fe640 1 -- 172.21.15.62:0/1098803830 learned_addr learned my addr 172.21.15.62:0/1098803830 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f70fbfff640 1 --2- 172.21.15.62:0/1098803830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.275+0000 7f70faffd640 1 --2- 172.21.15.62:0/1098803830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.276+0000 7f70fb7fe640 1 -- 172.21.15.62:0/1098803830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 msgr2=0x56418c1cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.276+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/1098803830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.276+0000 7f70fb7fe640 1 -- 172.21.15.62:0/1098803830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 msgr2=0x56418c1d56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.276+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/1098803830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.276+0000 7f70fb7fe640 1 -- 172.21.15.62:0/1098803830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418c1cf960 con 0x56418c1d25d0 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.276+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/1098803830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1d5180 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00a120 tx=0x7f70ec006c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d57b4d8fcf980e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.276+0000 7f70e17fa640 1 -- 172.21.15.62:0/1098803830 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec028070 con 0x56418c1d25d0 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.276+0000 7f70e17fa640 1 -- 172.21.15.62:0/1098803830 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70ec011a40 con 0x56418c1d25d0 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.276+0000 7f70e17fa640 1 -- 172.21.15.62:0/1098803830 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec006040 con 0x56418c1d25d0 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.277+0000 7f710461a740 1 -- 172.21.15.62:0/1098803830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 msgr2=0x56418c1d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.277+0000 7f710461a740 1 --2- 172.21.15.62:0/1098803830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1d5180 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00a120 tx=0x7f70ec006c20 comp rx=0 tx=0).stop 2024-09-12T00:05:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.277+0000 7f710461a740 1 -- 172.21.15.62:0/1098803830 shutdown_connections 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.277+0000 7f710461a740 1 --2- 172.21.15.62:0/1098803830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.277+0000 7f710461a740 1 --2- 172.21.15.62:0/1098803830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.277+0000 7f710461a740 1 --2- 172.21.15.62:0/1098803830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1d5180 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.277+0000 7f710461a740 1 -- 172.21.15.62:0/1098803830 >> 172.21.15.62:0/1098803830 conn(0x56418c19b310 msgr2=0x56418c1cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.277+0000 7f710461a740 1 -- 172.21.15.62:0/1098803830 shutdown_connections 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.277+0000 7f710461a740 1 -- 172.21.15.62:0/1098803830 wait complete. 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.277+0000 7f710461a740 1 -- start start 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f710461a740 1 Processor -- start 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f710461a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1e5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f710461a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f710461a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f710461a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc1ae0c0 con 0x56418c1d25d0 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f710461a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc1af530 con 0x56418c1d3b40 2024-09-12T00:05:47.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f710461a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc1ae790 con 0x56418c1dd370 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f710461a740 10 client.0 _ll_register_callbacks cb 0x7f70ec00ae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f70fb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1e5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f70fb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1e5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42744/0 (socket says 172.21.15.62:42744) 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f70fb7fe640 1 -- 172.21.15.62:0/897244464 learned_addr learned my addr 172.21.15.62:0/897244464 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f70fbfff640 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f70faffd640 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f70fb7fe640 1 -- 172.21.15.62:0/897244464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 msgr2=0x56418c1d7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1d7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f70fb7fe640 1 -- 172.21.15.62:0/897244464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 msgr2=0x56418c1d7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.278+0000 7f70fb7fe640 1 -- 172.21.15.62:0/897244464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418c1d7d20 con 0x56418c1d25d0 2024-09-12T00:05:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f70fb7fe640 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1e5680 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00f9f0 tx=0x7f70ec011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e649504c154bd852 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f70daffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f70daffd640 1 -- 172.21.15.62:0/897244464 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec028040 con 0x56418c1d25d0 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f70daffd640 1 -- 172.21.15.62:0/897244464 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f70ec013850 con 0x56418c1d25d0 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f710461a740 1 -- 172.21.15.62:0/897244464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70ec011a40 con 0x56418c1d25d0 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f70daffd640 1 -- 172.21.15.62:0/897244464 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f70ec011e00 con 0x56418c1d25d0 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f710461a740 10 client.17826 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f710461a740 1 -- 172.21.15.62:0/897244464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56418c257980 con 0x56418c1d25d0 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f70daffd640 1 -- 172.21.15.62:0/897244464 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70ec011a40 con 0x56418c1d25d0 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f70daffd640 1 -- 172.21.15.62:0/897244464 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f70ec037610 con 0x56418c1d25d0 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f710461a740 10 client.17826 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f710461a740 10 client.17826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f710461a740 20 client.17826 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f710461a740 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56418c257e70 0x56418c25a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f710461a740 4 client.17826 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.279+0000 7f710461a740 1 -- 172.21.15.62:0/897244464 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56418c13fd70 con 0x56418c257e70 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f70faffd640 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56418c257e70 0x56418c25a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f70daffd640 10 client.17826 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f70faffd640 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56418c257e70 0x56418c25a290 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c582c37b02948b server_cookie=f0909cc30a08e243 in_seq=0 out_seq=0 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f70daffd640 1 -- 172.21.15.62:0/897244464 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56418c13fd70 con 0x56418c257e70 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f70daffd640 10 client.17826 handle_command_reply: tid=0 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f710461a740 1 client.17826 shutdown 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f710461a740 1 -- 172.21.15.62:0/897244464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 msgr2=0x56418c1e5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f710461a740 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1e5680 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00f9f0 tx=0x7f70ec011690 comp rx=0 tx=0).stop 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f710461a740 1 -- 172.21.15.62:0/897244464 shutdown_connections 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f710461a740 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x56418c257e70 0x56418c25a290 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f710461a740 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56418c1dd370 0x56418c1d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f70daffd640 0 client.17826 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f710461a740 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56418c1d3b40 0x56418c1d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f710461a740 1 --2- 172.21.15.62:0/897244464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56418c1d25d0 0x56418c1e5680 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.280+0000 7f710461a740 1 -- 172.21.15.62:0/897244464 >> 172.21.15.62:0/897244464 conn(0x56418c19b310 msgr2=0x56418c1dbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f710461a740 1 -- 172.21.15.62:0/897244464 shutdown_connections 2024-09-12T00:05:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f710461a740 1 -- 172.21.15.62:0/897244464 wait complete. 2024-09-12T00:05:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f710461a740 20 client.17826 trim_cache size 0 max 16384 2024-09-12T00:05:47.287 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:05:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 -- 172.21.15.62:0/3174555856 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f70e40a2cf0 msgr2=0x7f70e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f70e40a2cf0 0x7f70e40a51b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f70f4005fd0 tx=0x7f70f4009a30 comp rx=0 tx=0).stop 2024-09-12T00:05:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 -- 172.21.15.62:0/3174555856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70fc101420 msgr2=0x7f70fc110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70fc101420 0x7f70fc110f10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f70f002a940 tx=0x7f70f0003b90 comp rx=0 tx=0).stop 2024-09-12T00:05:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 -- 172.21.15.62:0/3174555856 shutdown_connections 2024-09-12T00:05:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f70e40a2cf0 0x7f70e40a51b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70fc106a80 0x7f70fc111450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70fc101420 0x7f70fc110f10 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 --2- 172.21.15.62:0/3174555856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70fc100a80 0x7f70fc1109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 -- 172.21.15.62:0/3174555856 >> 172.21.15.62:0/3174555856 conn(0x7f70fc0fc7a0 msgr2=0x7f70fc0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 -- 172.21.15.62:0/3174555856 shutdown_connections 2024-09-12T00:05:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:47.281+0000 7f7102241640 1 -- 172.21.15.62:0/3174555856 wait complete. 2024-09-12T00:05:47.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:46 smithi120 ceph-mon[35182]: pgmap v425: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.5 MiB/s wr, 270 op/s 2024-09-12T00:05:47.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:46 smithi083 ceph-mon[34805]: pgmap v425: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.5 MiB/s wr, 270 op/s 2024-09-12T00:05:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 785.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:05:48.968 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:05:48.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:05:49.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:48 smithi062 ceph-mon[30958]: pgmap v426: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.0 MiB/s wr, 379 op/s 2024-09-12T00:05:49.241 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:49.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:48 smithi120 ceph-mon[35182]: pgmap v426: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.0 MiB/s wr, 379 op/s 2024-09-12T00:05:49.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:48 smithi083 ceph-mon[34805]: pgmap v426: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.0 MiB/s wr, 379 op/s 2024-09-12T00:05:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 -- 172.21.15.62:0/2632403952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba5c101420 msgr2=0x7fba5c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 --2- 172.21.15.62:0/2632403952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba5c101420 0x7fba5c10d950 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fba54009cd0 tx=0x7fba5402a8e0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 -- 172.21.15.62:0/2632403952 shutdown_connections 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 --2- 172.21.15.62:0/2632403952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba5c101420 0x7fba5c10d950 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 --2- 172.21.15.62:0/2632403952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba5c100a80 0x7fba5c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 --2- 172.21.15.62:0/2632403952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c106a80 0x7fba5c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 -- 172.21.15.62:0/2632403952 >> 172.21.15.62:0/2632403952 conn(0x7fba5c0fc7a0 msgr2=0x7fba5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 -- 172.21.15.62:0/2632403952 shutdown_connections 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 -- 172.21.15.62:0/2632403952 wait complete. 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 Processor -- start 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.695+0000 7fba626a4640 1 -- start start 2024-09-12T00:05:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba626a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba5c100a80 0x7fba5c113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba626a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba5c101420 0x7fba5c111650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba626a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c106a80 0x7fba5c111c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba626a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c110ed0 con 0x7fba5c100a80 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba626a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c110d50 con 0x7fba5c106a80 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba626a4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c111050 con 0x7fba5c101420 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba5bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba5c100a80 0x7fba5c113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba5bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba5c100a80 0x7fba5c113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42762/0 (socket says 172.21.15.62:42762) 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba5bfff640 1 -- 172.21.15.62:0/544593708 learned_addr learned my addr 172.21.15.62:0/544593708 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba53fff640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba5c101420 0x7fba5c111650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba60c41640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c106a80 0x7fba5c111c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba5bfff640 1 -- 172.21.15.62:0/544593708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba5c101420 msgr2=0x7fba5c111650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba5bfff640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba5c101420 0x7fba5c111650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba5bfff640 1 -- 172.21.15.62:0/544593708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c106a80 msgr2=0x7fba5c111c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba5bfff640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c106a80 0x7fba5c111c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba5bfff640 1 -- 172.21.15.62:0/544593708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c1a69f0 con 0x7fba5c100a80 2024-09-12T00:05:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba53fff640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba5c101420 0x7fba5c111650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:05:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba5bfff640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba5c100a80 0x7fba5c113260 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00e9f0 tx=0x7fba4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.696+0000 7fba59ffb640 1 -- 172.21.15.62:0/544593708 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00cd80 con 0x7fba5c100a80 2024-09-12T00:05:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.697+0000 7fba626a4640 1 -- 172.21.15.62:0/544593708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5c1a6bc0 con 0x7fba5c100a80 2024-09-12T00:05:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.697+0000 7fba59ffb640 1 -- 172.21.15.62:0/544593708 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c010910 con 0x7fba5c100a80 2024-09-12T00:05:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.697+0000 7fba59ffb640 1 -- 172.21.15.62:0/544593708 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0188f0 con 0x7fba5c100a80 2024-09-12T00:05:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.697+0000 7fba626a4640 1 -- 172.21.15.62:0/544593708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5c1a70c0 con 0x7fba5c100a80 2024-09-12T00:05:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.699+0000 7fba59ffb640 1 -- 172.21.15.62:0/544593708 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fba4c010ac0 con 0x7fba5c100a80 2024-09-12T00:05:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.699+0000 7fba64a7d740 1 Processor -- start 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.699+0000 7fba64a7d740 1 -- start start 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.699+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e793ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.699+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793f480 0x5606e7949800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.699+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e794c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.699+0000 7fba64a7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5606e78a4280 con 0x5606e7949d40 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.699+0000 7fba64a7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5606e784bb10 con 0x5606e793eb60 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.699+0000 7fba64a7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5606e78a0240 con 0x5606e793f480 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba53fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e794c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba60c41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e793ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba53fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e794c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42778/0 (socket says 172.21.15.62:42778) 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba53fff640 1 -- 172.21.15.62:0/2320283442 learned_addr learned my addr 172.21.15.62:0/2320283442 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba5bfff640 1 --2- 172.21.15.62:0/2320283442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793f480 0x5606e7949800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba59ffb640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fba480a2cf0 0x7fba480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba60c41640 1 -- 172.21.15.62:0/2320283442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793f480 msgr2=0x5606e7949800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba60c41640 1 --2- 172.21.15.62:0/2320283442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793f480 0x5606e7949800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba60c41640 1 -- 172.21.15.62:0/2320283442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 msgr2=0x5606e794c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba60c41640 1 --2- 172.21.15.62:0/2320283442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e794c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba60c41640 1 -- 172.21.15.62:0/2320283442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e78581f0 con 0x5606e793eb60 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba59ffb640 1 -- 172.21.15.62:0/544593708 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fba4c0cf920 con 0x7fba5c100a80 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba60c41640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fba480a2cf0 0x7fba480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.700+0000 7fba60c41640 1 --2- 172.21.15.62:0/2320283442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e793ef40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fba5402ae00 tx=0x7fba54005fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e953528b1556bea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba527fc640 1 -- 172.21.15.62:0/2320283442 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5404d070 con 0x5606e793eb60 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba527fc640 1 -- 172.21.15.62:0/2320283442 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba54043070 con 0x5606e793eb60 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba527fc640 1 -- 172.21.15.62:0/2320283442 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54030b60 con 0x5606e793eb60 2024-09-12T00:05:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba60c41640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fba480a2cf0 0x7fba480a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5606e793faa0 tx=0x7fba5404b000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba64a7d740 1 -- 172.21.15.62:0/2320283442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 msgr2=0x5606e793ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba64a7d740 1 --2- 172.21.15.62:0/2320283442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e793ef40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fba5402ae00 tx=0x7fba54005fc0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba64a7d740 1 -- 172.21.15.62:0/2320283442 shutdown_connections 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba64a7d740 1 --2- 172.21.15.62:0/2320283442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e794c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba64a7d740 1 --2- 172.21.15.62:0/2320283442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793f480 0x5606e7949800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba64a7d740 1 --2- 172.21.15.62:0/2320283442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e793ef40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba64a7d740 1 -- 172.21.15.62:0/2320283442 >> 172.21.15.62:0/2320283442 conn(0x5606e793b0f0 msgr2=0x5606e793b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba64a7d740 1 -- 172.21.15.62:0/2320283442 shutdown_connections 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.701+0000 7fba64a7d740 1 -- 172.21.15.62:0/2320283442 wait complete. 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba64a7d740 1 -- start start 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba64a7d740 1 Processor -- start 2024-09-12T00:05:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e7946bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e793f480 0x5606e7947130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e7949d40 0x5606e7947670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba64a7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5606e794c910 con 0x5606e793f480 2024-09-12T00:05:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba64a7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5606e785ef10 con 0x5606e793eb60 2024-09-12T00:05:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba64a7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5606e794ca90 con 0x5606e7949d40 2024-09-12T00:05:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba64a7d740 10 client.0 _ll_register_callbacks cb 0x5606e7827a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba53fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e793f480 0x5606e7947130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba53fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e793f480 0x5606e7947130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42792/0 (socket says 172.21.15.62:42792) 2024-09-12T00:05:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba53fff640 1 -- 172.21.15.62:0/472784568 learned_addr learned my addr 172.21.15.62:0/472784568 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.702+0000 7fba5bfff640 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e7946bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba60c41640 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e7949d40 0x5606e7947670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba53fff640 1 -- 172.21.15.62:0/472784568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e7949d40 msgr2=0x5606e7947670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba53fff640 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e7949d40 0x5606e7947670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba53fff640 1 -- 172.21.15.62:0/472784568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 msgr2=0x5606e7946bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba53fff640 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e7946bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba53fff640 1 -- 172.21.15.62:0/472784568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e76156a0 con 0x5606e793f480 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba5bfff640 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e7946bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba53fff640 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e793f480 0x5606e7947130 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fba44012770 tx=0x7fba44012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287506ba1445cc90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba3b7fe640 1 -- 172.21.15.62:0/472784568 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44011bb0 con 0x5606e793f480 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba3b7fe640 1 -- 172.21.15.62:0/472784568 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fba44011d50 con 0x5606e793f480 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba64a7d740 1 -- 172.21.15.62:0/472784568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba54043070 con 0x5606e793f480 2024-09-12T00:05:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba3b7fe640 1 -- 172.21.15.62:0/472784568 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fba4400f460 con 0x5606e793f480 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba64a7d740 10 client.17838 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.703+0000 7fba64a7d740 1 -- 172.21.15.62:0/472784568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5606e7958540 con 0x5606e793f480 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba3b7fe640 1 -- 172.21.15.62:0/472784568 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba54043070 con 0x5606e793f480 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba3b7fe640 1 -- 172.21.15.62:0/472784568 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fba4401c7c0 con 0x5606e793f480 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba64a7d740 10 client.17838 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba64a7d740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba64a7d740 20 client.17838 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba64a7d740 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5606e7959a50 0x5606e795bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba64a7d740 4 client.17838 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba64a7d740 1 -- 172.21.15.62:0/472784568 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606e78b17f0 con 0x5606e7959a50 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba5bfff640 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5606e7959a50 0x5606e795bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba5bfff640 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5606e7959a50 0x5606e795bed0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb44670e228502f9 server_cookie=cd04c5203443be09 in_seq=0 out_seq=0 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.704+0000 7fba3b7fe640 10 client.17838 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba3b7fe640 1 -- 172.21.15.62:0/472784568 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606e78b17f0 con 0x5606e7959a50 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba3b7fe640 10 client.17838 handle_command_reply: tid=0 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba64a7d740 1 client.17838 shutdown 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba64a7d740 1 -- 172.21.15.62:0/472784568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e793f480 msgr2=0x5606e7947130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba64a7d740 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e793f480 0x5606e7947130 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fba44012770 tx=0x7fba44012c30 comp rx=0 tx=0).stop 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba64a7d740 1 -- 172.21.15.62:0/472784568 shutdown_connections 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba64a7d740 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5606e7959a50 0x5606e795bed0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba64a7d740 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e7949d40 0x5606e7947670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba3b7fe640 0 client.17838 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba64a7d740 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e793f480 0x5606e7947130 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba64a7d740 1 --2- 172.21.15.62:0/472784568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e7946bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.705+0000 7fba64a7d740 1 -- 172.21.15.62:0/472784568 >> 172.21.15.62:0/472784568 conn(0x5606e793b0f0 msgr2=0x5606e793a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.706+0000 7fba64a7d740 1 -- 172.21.15.62:0/472784568 shutdown_connections 2024-09-12T00:05:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.706+0000 7fba64a7d740 1 -- 172.21.15.62:0/472784568 wait complete. 2024-09-12T00:05:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.706+0000 7fba64a7d740 20 client.17838 trim_cache size 0 max 16384 2024-09-12T00:05:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba64a7d740 1 Processor -- start 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba64a7d740 1 -- start start 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e79617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793f480 0x5606e7961d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7939a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba64a7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1b23f0 con 0x5606e7949d40 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba64a7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1afcc0 con 0x5606e793eb60 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba64a7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1adb30 con 0x5606e793f480 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba60c41640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7939a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba60c41640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7939a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42808/0 (socket says 172.21.15.62:42808) 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba60c41640 1 -- 172.21.15.62:0/689919829 learned_addr learned my addr 172.21.15.62:0/689919829 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba53fff640 1 --2- 172.21.15.62:0/689919829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793f480 0x5606e7961d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.723+0000 7fba5bfff640 1 --2- 172.21.15.62:0/689919829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e79617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.724+0000 7fba60c41640 1 -- 172.21.15.62:0/689919829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793f480 msgr2=0x5606e7961d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.724+0000 7fba60c41640 1 --2- 172.21.15.62:0/689919829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793f480 0x5606e7961d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.724+0000 7fba60c41640 1 -- 172.21.15.62:0/689919829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 msgr2=0x5606e79617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.724+0000 7fba60c41640 1 --2- 172.21.15.62:0/689919829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e79617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.724+0000 7fba60c41640 1 -- 172.21.15.62:0/689919829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e79494c0 con 0x5606e7949d40 2024-09-12T00:05:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.724+0000 7fba60c41640 1 --2- 172.21.15.62:0/689919829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7939a50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fba54005e30 tx=0x7fba540317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58288ce23bfac82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.724+0000 7fba51ffb640 1 -- 172.21.15.62:0/689919829 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5404d030 con 0x5606e7949d40 2024-09-12T00:05:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.724+0000 7fba51ffb640 1 -- 172.21.15.62:0/689919829 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba54043070 con 0x5606e7949d40 2024-09-12T00:05:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.724+0000 7fba51ffb640 1 -- 172.21.15.62:0/689919829 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54004780 con 0x5606e7949d40 2024-09-12T00:05:49.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 -- 172.21.15.62:0/689919829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 msgr2=0x5606e7939a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:49.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 --2- 172.21.15.62:0/689919829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7939a50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fba54005e30 tx=0x7fba540317d0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 -- 172.21.15.62:0/689919829 shutdown_connections 2024-09-12T00:05:49.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 --2- 172.21.15.62:0/689919829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7939a50 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 --2- 172.21.15.62:0/689919829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793f480 0x5606e7961d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 --2- 172.21.15.62:0/689919829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793eb60 0x5606e79617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 -- 172.21.15.62:0/689919829 >> 172.21.15.62:0/689919829 conn(0x5606e793b0f0 msgr2=0x5606e7945ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 -- 172.21.15.62:0/689919829 shutdown_connections 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 -- 172.21.15.62:0/689919829 wait complete. 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 -- start start 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.725+0000 7fba64a7d740 1 Processor -- start 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793eb60 0x5606e7951ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793f480 0x5606e7952400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba64a7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7943400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba64a7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1ae180 con 0x5606e7949d40 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba64a7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1af5f0 con 0x5606e793f480 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba64a7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1ae850 con 0x5606e793eb60 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba64a7d740 10 client.0 _ll_register_callbacks cb 0x7fba44009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:49.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba60c41640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7943400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba5bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793eb60 0x5606e7951ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793f480 0x5606e7952400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba5bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793eb60 0x5606e7951ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:37508/0 (socket says 172.21.15.62:37508) 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba5bfff640 1 -- 172.21.15.62:0/1889440581 learned_addr learned my addr 172.21.15.62:0/1889440581 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba60c41640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7943400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42812/0 (socket says 172.21.15.62:42812) 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba5bfff640 1 -- 172.21.15.62:0/1889440581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793f480 msgr2=0x5606e7952400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba5bfff640 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793f480 0x5606e7952400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba5bfff640 1 -- 172.21.15.62:0/1889440581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 msgr2=0x5606e7943400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba5bfff640 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7943400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba5bfff640 1 -- 172.21.15.62:0/1889440581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e7943c00 con 0x5606e793eb60 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba60c41640 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7943400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba5bfff640 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793eb60 0x5606e7951ec0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009380 tx=0x7fba4c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2efcfb4e89ba83b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.726+0000 7fba3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba3b7fe640 1 -- 172.21.15.62:0/1889440581 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c09e020 con 0x5606e793eb60 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba3b7fe640 1 -- 172.21.15.62:0/1889440581 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fba4c09b490 con 0x5606e793eb60 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba64a7d740 1 -- 172.21.15.62:0/1889440581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba54043070 con 0x5606e793eb60 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba3b7fe640 1 -- 172.21.15.62:0/1889440581 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fba4c09be40 con 0x5606e793eb60 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba64a7d740 10 client.27247 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba64a7d740 1 -- 172.21.15.62:0/1889440581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5606e79cca80 con 0x5606e793eb60 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba3b7fe640 1 -- 172.21.15.62:0/1889440581 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba54043070 con 0x5606e793eb60 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba3b7fe640 1 -- 172.21.15.62:0/1889440581 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fba4c09ad00 con 0x5606e793eb60 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba64a7d740 10 client.27247 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba64a7d740 10 client.27247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba64a7d740 20 client.27247 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba64a7d740 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5606e79cd1d0 0x5606e79cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba64a7d740 4 client.27247 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:05:49.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.727+0000 7fba64a7d740 1 -- 172.21.15.62:0/1889440581 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5606e78abd00 con 0x5606e79cd1d0 2024-09-12T00:05:49.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.728+0000 7fba53fff640 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5606e79cd1d0 0x5606e79cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:49.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.728+0000 7fba53fff640 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5606e79cd1d0 0x5606e79cf5f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b010f7ee93c9783b server_cookie=65dcbde2e6fb7bb3 in_seq=0 out_seq=0 2024-09-12T00:05:49.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.728+0000 7fba3b7fe640 10 client.27247 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba3b7fe640 1 -- 172.21.15.62:0/1889440581 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5606e78abd00 con 0x5606e79cd1d0 2024-09-12T00:05:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba3b7fe640 10 client.27247 handle_command_reply: tid=0 2024-09-12T00:05:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 client.27247 shutdown 2024-09-12T00:05:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 -- 172.21.15.62:0/1889440581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793eb60 msgr2=0x5606e7951ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793eb60 0x5606e7951ec0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009380 tx=0x7fba4c005f10 comp rx=0 tx=0).stop 2024-09-12T00:05:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 -- 172.21.15.62:0/1889440581 shutdown_connections 2024-09-12T00:05:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5606e79cd1d0 0x5606e79cf5f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606e7949d40 0x5606e7943400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e793f480 0x5606e7952400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 --2- 172.21.15.62:0/1889440581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606e793eb60 0x5606e7951ec0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba3b7fe640 0 client.27247 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:49.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 -- 172.21.15.62:0/1889440581 >> 172.21.15.62:0/1889440581 conn(0x5606e793b0f0 msgr2=0x5606e7939a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:49.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 -- 172.21.15.62:0/1889440581 shutdown_connections 2024-09-12T00:05:49.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.729+0000 7fba64a7d740 1 -- 172.21.15.62:0/1889440581 wait complete. 2024-09-12T00:05:49.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba64a7d740 20 client.27247 trim_cache size 0 max 16384 2024-09-12T00:05:49.734 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:05:49.734 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-12T00:05:49.734 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "3eb53ae5-7bbf-453c-b12f-a714e6bd8d56", 2024-09-12T00:05:49.734 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-12T00:05:49.735 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:05:49.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 -- 172.21.15.62:0/544593708 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fba480a2cf0 msgr2=0x7fba480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:49.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fba480a2cf0 0x7fba480a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5606e793faa0 tx=0x7fba5404b000 comp rx=0 tx=0).stop 2024-09-12T00:05:49.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 -- 172.21.15.62:0/544593708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba5c100a80 msgr2=0x7fba5c113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:49.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba5c100a80 0x7fba5c113260 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00e9f0 tx=0x7fba4c00eeb0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 -- 172.21.15.62:0/544593708 shutdown_connections 2024-09-12T00:05:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fba480a2cf0 0x7fba480a51b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c106a80 0x7fba5c111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba5c101420 0x7fba5c111650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 --2- 172.21.15.62:0/544593708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba5c100a80 0x7fba5c113260 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 -- 172.21.15.62:0/544593708 >> 172.21.15.62:0/544593708 conn(0x7fba5c0fc7a0 msgr2=0x7fba5c102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 -- 172.21.15.62:0/544593708 shutdown_connections 2024-09-12T00:05:49.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:49.730+0000 7fba626a4640 1 -- 172.21.15.62:0/544593708 wait complete. 2024-09-12T00:05:50.366 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:49 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-12T00:05:50.366 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:49 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:50.366 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:49 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:05:50.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:49 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:05:50.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:49 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:50.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:49 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:05:50.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:49 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:50.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:49 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-12T00:05:50.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:49 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:50.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:49 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:05:50.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:49 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:05:50.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:49 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:50.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:49 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:05:50.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:49 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:50.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:49 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-12T00:05:50.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:49 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:50.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:49 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:05:50.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:49 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:05:50.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:49 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:50.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:49 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:05:50.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:49 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:50.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:05:50.715 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:05:51.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:50 smithi062 ceph-mon[30958]: pgmap v427: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-12T00:05:51.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:50 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:05:51.089 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:50 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:51.089 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:50 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:51.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.169+0000 7fe39fef5640 1 -- 172.21.15.62:0/1747610624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398100e20 msgr2=0x7fe398101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.169+0000 7fe39fef5640 1 --2- 172.21.15.62:0/1747610624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398100e20 0x7fe398101200 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe394009900 tx=0x7fe39402a480 comp rx=0 tx=0).stop 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.169+0000 7fe39fef5640 1 -- 172.21.15.62:0/1747610624 shutdown_connections 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.169+0000 7fe39fef5640 1 --2- 172.21.15.62:0/1747610624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe398102ef0 0x7fe39810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.169+0000 7fe39fef5640 1 --2- 172.21.15.62:0/1747610624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe398102570 0x7fe3981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.169+0000 7fe39fef5640 1 --2- 172.21.15.62:0/1747610624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398100e20 0x7fe398101200 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.169+0000 7fe39fef5640 1 -- 172.21.15.62:0/1747610624 >> 172.21.15.62:0/1747610624 conn(0x7fe3980fc7d0 msgr2=0x7fe3980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.169+0000 7fe39fef5640 1 -- 172.21.15.62:0/1747610624 shutdown_connections 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.169+0000 7fe39fef5640 1 -- 172.21.15.62:0/1747610624 wait complete. 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39fef5640 1 Processor -- start 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39fef5640 1 -- start start 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39fef5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe398100e20 0x7fe3981984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39fef5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102570 0x7fe3981989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39fef5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe398102ef0 0x7fe39819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39fef5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe398110f80 con 0x7fe398102570 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39fef5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe398110e00 con 0x7fe398100e20 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39fef5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe398111100 con 0x7fe398102ef0 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39d490640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102570 0x7fe3981989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39d490640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102570 0x7fe3981989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38716/0 (socket says 172.21.15.62:38716) 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39d490640 1 -- 172.21.15.62:0/135140830 learned_addr learned my addr 172.21.15.62:0/135140830 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39e492640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe398102ef0 0x7fe39819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39dc91640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe398100e20 0x7fe3981984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39d490640 1 -- 172.21.15.62:0/135140830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe398102ef0 msgr2=0x7fe39819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39d490640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe398102ef0 0x7fe39819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39d490640 1 -- 172.21.15.62:0/135140830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe398100e20 msgr2=0x7fe3981984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:51.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.170+0000 7fe39d490640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe398100e20 0x7fe3981984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.171+0000 7fe39d490640 1 -- 172.21.15.62:0/135140830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39819d460 con 0x7fe398102570 2024-09-12T00:05:51.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.171+0000 7fe39d490640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102570 0x7fe3981989f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe38800d770 tx=0x7fe38800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:51.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.171+0000 7fe38effd640 1 -- 172.21.15.62:0/135140830 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388017070 con 0x7fe398102570 2024-09-12T00:05:51.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.171+0000 7fe38effd640 1 -- 172.21.15.62:0/135140830 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe388005500 con 0x7fe398102570 2024-09-12T00:05:51.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.171+0000 7fe39fef5640 1 -- 172.21.15.62:0/135140830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39819d750 con 0x7fe398102570 2024-09-12T00:05:51.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.171+0000 7fe38effd640 1 -- 172.21.15.62:0/135140830 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388004e00 con 0x7fe398102570 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.171+0000 7fe39fef5640 1 -- 172.21.15.62:0/135140830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3981a5030 con 0x7fe398102570 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.173+0000 7fe38effd640 1 -- 172.21.15.62:0/135140830 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe388004fa0 con 0x7fe398102570 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.173+0000 7fe3a22ce740 1 Processor -- start 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe3a22ce740 1 -- start start 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeaa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeaadc0 0x55f68eeb5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeb5680 0x55f68eeb7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f68ee102e0 con 0x55f68eeaa4a0 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f68edb7ce0 con 0x55f68eeb5680 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f68ee0c2a0 con 0x55f68eeaadc0 2024-09-12T00:05:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe39e492640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeaa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe39e492640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeaa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38720/0 (socket says 172.21.15.62:38720) 2024-09-12T00:05:51.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe39e492640 1 -- 172.21.15.62:0/1500101101 learned_addr learned my addr 172.21.15.62:0/1500101101 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:51.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe39d490640 1 --2- 172.21.15.62:0/1500101101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeb5680 0x55f68eeb7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe38effd640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe3840a2cf0 0x7fe3840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.174+0000 7fe39dc91640 1 --2- 172.21.15.62:0/1500101101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeaadc0 0x55f68eeb5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe39e492640 1 -- 172.21.15.62:0/1500101101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeaadc0 msgr2=0x55f68eeb5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:51.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe38effd640 1 -- 172.21.15.62:0/135140830 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fe3880cee80 con 0x7fe398102570 2024-09-12T00:05:51.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe39e492640 1 --2- 172.21.15.62:0/1500101101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeaadc0 0x55f68eeb5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe39e492640 1 -- 172.21.15.62:0/1500101101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeb5680 msgr2=0x55f68eeb7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:51.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe39e492640 1 --2- 172.21.15.62:0/1500101101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeb5680 0x55f68eeb7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe39e492640 1 -- 172.21.15.62:0/1500101101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68ed9c440 con 0x55f68eeaa4a0 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe39d490640 1 --2- 172.21.15.62:0/1500101101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeb5680 0x55f68eeb7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe39e492640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe3840a2cf0 0x7fe3840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe39e492640 1 --2- 172.21.15.62:0/1500101101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeaa880 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe39000f440 tx=0x7fe39000f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2bf47238ce5fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe383fff640 1 -- 172.21.15.62:0/1500101101 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390019870 con 0x55f68eeaa4a0 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe383fff640 1 -- 172.21.15.62:0/1500101101 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39001a410 con 0x55f68eeaa4a0 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.175+0000 7fe383fff640 1 -- 172.21.15.62:0/1500101101 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39000d540 con 0x55f68eeaa4a0 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 -- 172.21.15.62:0/1500101101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 msgr2=0x55f68eeaa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe39e492640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe3840a2cf0 0x7fe3840a51b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe390020000 tx=0x7fe39001d000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/1500101101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeaa880 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe39000f440 tx=0x7fe39000f900 comp rx=0 tx=0).stop 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 -- 172.21.15.62:0/1500101101 shutdown_connections 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/1500101101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeb5680 0x55f68eeb7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/1500101101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeaadc0 0x55f68eeb5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/1500101101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeaa880 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 -- 172.21.15.62:0/1500101101 >> 172.21.15.62:0/1500101101 conn(0x55f68eea6a30 msgr2=0x55f68eea6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 -- 172.21.15.62:0/1500101101 shutdown_connections 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 -- 172.21.15.62:0/1500101101 wait complete. 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 -- start start 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.176+0000 7fe3a22ce740 1 Processor -- start 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeb4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaadc0 0x55f68eeb2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 0x55f68eeb2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f68eebc660 con 0x55f68eeaa4a0 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f68edcb050 con 0x55f68eeaadc0 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f68eebc7e0 con 0x55f68eeb5680 2024-09-12T00:05:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe3a22ce740 10 client.0 _ll_register_callbacks cb 0x55f68ee0b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39dc91640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeb4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39dc91640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeb4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38734/0 (socket says 172.21.15.62:38734) 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39dc91640 1 -- 172.21.15.62:0/3032567421 learned_addr learned my addr 172.21.15.62:0/3032567421 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39d490640 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaadc0 0x55f68eeb2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39e492640 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 0x55f68eeb2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39dc91640 1 -- 172.21.15.62:0/3032567421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 msgr2=0x55f68eeb2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39dc91640 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 0x55f68eeb2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39dc91640 1 -- 172.21.15.62:0/3032567421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaadc0 msgr2=0x55f68eeb2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39dc91640 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaadc0 0x55f68eeb2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39dc91640 1 -- 172.21.15.62:0/3032567421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68eb813c0 con 0x55f68eeaa4a0 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.177+0000 7fe39e492640 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 0x55f68eeb2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:05:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe39dc91640 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeb4f80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe39402a9a0 tx=0x7fe394004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdb7d52c6957e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3817fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3817fa640 1 -- 172.21.15.62:0/3032567421 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394035070 con 0x55f68eeaa4a0 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3a22ce740 1 -- 172.21.15.62:0/3032567421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe39001a410 con 0x55f68eeaa4a0 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3817fa640 1 -- 172.21.15.62:0/3032567421 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fe394003ac0 con 0x55f68eeaa4a0 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3817fa640 1 -- 172.21.15.62:0/3032567421 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe394003cc0 con 0x55f68eeaa4a0 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3a22ce740 10 client.17856 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3a22ce740 1 -- 172.21.15.62:0/3032567421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f68eec7fd0 con 0x55f68eeaa4a0 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3817fa640 1 -- 172.21.15.62:0/3032567421 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39001a410 con 0x55f68eeaa4a0 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3817fa640 1 -- 172.21.15.62:0/3032567421 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fe39404e6b0 con 0x55f68eeaa4a0 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3a22ce740 10 client.17856 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3a22ce740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3a22ce740 20 client.17856 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f68eec9430 0x55f68eecb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3a22ce740 4 client.17856 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.178+0000 7fe3a22ce740 1 -- 172.21.15.62:0/3032567421 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f68ee1d850 con 0x55f68eec9430 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.179+0000 7fe39d490640 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f68eec9430 0x55f68eecb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.179+0000 7fe39d490640 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f68eec9430 0x55f68eecb880 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d68511629a5fda81 server_cookie=135d8cad7807c82b in_seq=0 out_seq=0 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.179+0000 7fe3817fa640 10 client.17856 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3817fa640 1 -- 172.21.15.62:0/3032567421 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f68ee1d850 con 0x55f68eec9430 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3817fa640 10 client.17856 handle_command_reply: tid=0 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 client.17856 shutdown 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 -- 172.21.15.62:0/3032567421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 msgr2=0x55f68eeb4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeb4f80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe39402a9a0 tx=0x7fe394004440 comp rx=0 tx=0).stop 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 -- 172.21.15.62:0/3032567421 shutdown_connections 2024-09-12T00:05:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f68eec9430 0x55f68eecb880 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 0x55f68eeb2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaadc0 0x55f68eeb2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/3032567421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaa4a0 0x55f68eeb4f80 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3817fa640 0 client.17856 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 -- 172.21.15.62:0/3032567421 >> 172.21.15.62:0/3032567421 conn(0x55f68eea6a30 msgr2=0x55f68eeac710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 -- 172.21.15.62:0/3032567421 shutdown_connections 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 1 -- 172.21.15.62:0/3032567421 wait complete. 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.180+0000 7fe3a22ce740 20 client.17856 trim_cache size 0 max 16384 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe3a22ce740 1 Processor -- start 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe3a22ce740 1 -- start start 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaa4a0 0x55f68eec58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaadc0 0x55f68eea5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 0x55f68eea5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3981ade00 con 0x55f68eeaadc0 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3981ab6d0 con 0x55f68eeaa4a0 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3981a9540 con 0x55f68eeb5680 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39d490640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaadc0 0x55f68eea5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39d490640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaadc0 0x55f68eea5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38746/0 (socket says 172.21.15.62:38746) 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39d490640 1 -- 172.21.15.62:0/2577935338 learned_addr learned my addr 172.21.15.62:0/2577935338 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39dc91640 1 --2- 172.21.15.62:0/2577935338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaa4a0 0x55f68eec58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39e492640 1 --2- 172.21.15.62:0/2577935338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 0x55f68eea5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39d490640 1 -- 172.21.15.62:0/2577935338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 msgr2=0x55f68eea5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39d490640 1 --2- 172.21.15.62:0/2577935338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 0x55f68eea5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39d490640 1 -- 172.21.15.62:0/2577935338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaa4a0 msgr2=0x55f68eec58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39d490640 1 --2- 172.21.15.62:0/2577935338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaa4a0 0x55f68eec58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39d490640 1 -- 172.21.15.62:0/2577935338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68eeaa2d0 con 0x55f68eeaadc0 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.198+0000 7fe39e492640 1 --2- 172.21.15.62:0/2577935338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 0x55f68eea5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe39d490640 1 --2- 172.21.15.62:0/2577935338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaadc0 0x55f68eea5930 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe38809b380 tx=0x7fe3880d12d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addaaaaea68f6176 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe3837fe640 1 -- 172.21.15.62:0/2577935338 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388020040 con 0x55f68eeaadc0 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe3837fe640 1 -- 172.21.15.62:0/2577935338 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe388005290 con 0x55f68eeaadc0 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe3837fe640 1 -- 172.21.15.62:0/2577935338 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3880cc7f0 con 0x55f68eeaadc0 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2577935338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaadc0 msgr2=0x55f68eea5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/2577935338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaadc0 0x55f68eea5930 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe38809b380 tx=0x7fe3880d12d0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2577935338 shutdown_connections 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/2577935338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeb5680 0x55f68eea5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/2577935338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeaadc0 0x55f68eea5930 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/2577935338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaa4a0 0x55f68eec58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.199+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2577935338 >> 172.21.15.62:0/2577935338 conn(0x55f68eea6a30 msgr2=0x55f68eea5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2577935338 shutdown_connections 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2577935338 wait complete. 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 1 -- start start 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 1 Processor -- start 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeaa4a0 0x55f68eebf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaadc0 0x55f68eebfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeb5680 0x55f68eec0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3981a9b90 con 0x55f68eeb5680 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3981ab000 con 0x55f68eeaadc0 2024-09-12T00:05:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3981aa260 con 0x55f68eeaa4a0 2024-09-12T00:05:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe3a22ce740 10 client.0 _ll_register_callbacks cb 0x7fe394009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:05:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe39e492640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeb5680 0x55f68eec0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe39e492640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeb5680 0x55f68eec0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38758/0 (socket says 172.21.15.62:38758) 2024-09-12T00:05:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.200+0000 7fe39e492640 1 -- 172.21.15.62:0/2386710460 learned_addr learned my addr 172.21.15.62:0/2386710460 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:05:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe39dc91640 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeaa4a0 0x55f68eebf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe39d490640 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaadc0 0x55f68eebfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe39e492640 1 -- 172.21.15.62:0/2386710460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeaa4a0 msgr2=0x55f68eebf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe39e492640 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeaa4a0 0x55f68eebf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe39e492640 1 -- 172.21.15.62:0/2386710460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaadc0 msgr2=0x55f68eebfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:05:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe39e492640 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaadc0 0x55f68eebfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe39e492640 1 -- 172.21.15.62:0/2386710460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68ef332a0 con 0x55f68eeb5680 2024-09-12T00:05:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe39e492640 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeb5680 0x55f68eec0390 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fe390010010 tx=0x7fe390019a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cad1bf81f90997 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:05:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe3817fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:05:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe3817fa640 1 -- 172.21.15.62:0/2386710460 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390035060 con 0x55f68eeb5680 2024-09-12T00:05:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe3817fa640 1 -- 172.21.15.62:0/2386710460 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fe39001a410 con 0x55f68eeb5680 2024-09-12T00:05:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2386710460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe388005290 con 0x55f68eeb5680 2024-09-12T00:05:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe3817fa640 1 -- 172.21.15.62:0/2386710460 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe390004980 con 0x55f68eeb5680 2024-09-12T00:05:51.225 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stdout: "3eb53ae5-7bbf-453c-b12f-a714e6bd8d56": { 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "3eb53ae5-7bbf-453c-b12f-a714e6bd8d56", 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe3a22ce740 10 client.17862 fetch_fsmap learned FSMap version 38 2024-09-12T00:05:51.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.201+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2386710460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f68ef344e0 con 0x55f68eeb5680 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.202+0000 7fe3817fa640 1 -- 172.21.15.62:0/2386710460 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe388005290 con 0x55f68eeb5680 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.202+0000 7fe3817fa640 1 -- 172.21.15.62:0/2386710460 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fe390025440 con 0x55f68eeb5680 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.202+0000 7fe3a22ce740 10 client.17862 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.202+0000 7fe3a22ce740 10 client.17862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.202+0000 7fe3a22ce740 20 client.17862 populate_metadata read hostname 'smithi062' 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.202+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f68ef349b0 0x55f68ef36dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.202+0000 7fe3a22ce740 4 client.17862 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.202+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2386710460 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f68ee17d60 con 0x55f68ef349b0 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.202+0000 7fe39dc91640 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f68ef349b0 0x55f68ef36dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3817fa640 10 client.17862 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe39dc91640 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f68ef349b0 0x55f68ef36dd0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7761fb9da65196 server_cookie=e3d51adf685d9d4b in_seq=0 out_seq=0 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3817fa640 1 -- 172.21.15.62:0/2386710460 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f68ee17d60 con 0x55f68ef349b0 2024-09-12T00:05:51.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3817fa640 10 client.17862 handle_command_reply: tid=0 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 client.17862 shutdown 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2386710460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeb5680 msgr2=0x55f68eec0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeb5680 0x55f68eec0390 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fe390010010 tx=0x7fe390019a80 comp rx=0 tx=0).stop 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2386710460 shutdown_connections 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55f68ef349b0 0x55f68ef36dd0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f68eeb5680 0x55f68eec0390 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f68eeaadc0 0x55f68eebfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 --2- 172.21.15.62:0/2386710460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f68eeaa4a0 0x55f68eebf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3817fa640 0 client.17862 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2386710460 >> 172.21.15.62:0/2386710460 conn(0x55f68eea6a30 msgr2=0x55f68eea5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2386710460 shutdown_connections 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 1 -- 172.21.15.62:0/2386710460 wait complete. 2024-09-12T00:05:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.203+0000 7fe3a22ce740 20 client.17862 trim_cache size 0 max 16384 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 -- 172.21.15.62:0/135140830 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe3840a2cf0 msgr2=0x7fe3840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe3840a2cf0 0x7fe3840a51b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe390020000 tx=0x7fe39001d000 comp rx=0 tx=0).stop 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 -- 172.21.15.62:0/135140830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102570 msgr2=0x7fe3981989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102570 0x7fe3981989f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe38800d770 tx=0x7fe38800dc30 comp rx=0 tx=0).stop 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 -- 172.21.15.62:0/135140830 shutdown_connections 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe3840a2cf0 0x7fe3840a51b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe398102ef0 0x7fe39819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102570 0x7fe3981989f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 --2- 172.21.15.62:0/135140830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe398100e20 0x7fe3981984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 -- 172.21.15.62:0/135140830 >> 172.21.15.62:0/135140830 conn(0x7fe3980fc7d0 msgr2=0x7fe39806a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 -- 172.21.15.62:0/135140830 shutdown_connections 2024-09-12T00:05:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:05:51.204+0000 7fe39fef5640 1 -- 172.21.15.62:0/135140830 wait complete. 2024-09-12T00:05:51.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:50 smithi120 ceph-mon[35182]: pgmap v427: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-12T00:05:51.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:50 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:05:51.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:50 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:51.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:50 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:51.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:50 smithi083 ceph-mon[34805]: pgmap v427: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-12T00:05:51.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:50 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:05:51.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:50 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:51.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:50 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:05:51.928 INFO:tasks.cephfs.filesystem:scrub status for tag:3eb53ae5-7bbf-453c-b12f-a714e6bd8d56 - {'path': '/', 'tag': '3eb53ae5-7bbf-453c-b12f-a714e6bd8d56', 'options': 'recursive,force'} 2024-09-12T00:05:52.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 790.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:05:53.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:52 smithi120 ceph-mon[35182]: pgmap v428: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-09-12T00:05:53.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:52 smithi083 ceph-mon[34805]: pgmap v428: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-09-12T00:05:53.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:52 smithi062 ceph-mon[30958]: pgmap v428: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-09-12T00:05:55.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:54 smithi120 ceph-mon[35182]: pgmap v429: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-09-12T00:05:55.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:54 smithi083 ceph-mon[34805]: pgmap v429: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-09-12T00:05:55.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:54 smithi062 ceph-mon[30958]: pgmap v429: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-09-12T00:05:56.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:55 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:05:56.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:55 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:05:56.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:55 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:05:56.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:55 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:05:56.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:55 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:05:56.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:55 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:05:57.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:56 smithi120 ceph-mon[35182]: pgmap v430: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-09-12T00:05:57.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:56 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:05:57.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:56 smithi083 ceph-mon[34805]: pgmap v430: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-09-12T00:05:57.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:56 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:05:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 795.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:05:57.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:56 smithi062 ceph-mon[30958]: pgmap v430: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-09-12T00:05:57.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:56 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:05:59.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:05:58 smithi120 ceph-mon[35182]: pgmap v431: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-12T00:05:59.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:05:58 smithi083 ceph-mon[34805]: pgmap v431: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-12T00:05:59.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:05:58 smithi062 ceph-mon[30958]: pgmap v431: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-12T00:06:01.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:00 smithi120 ceph-mon[35182]: pgmap v432: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 352 KiB/s wr, 31 op/s 2024-09-12T00:06:01.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:00 smithi083 ceph-mon[34805]: pgmap v432: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 352 KiB/s wr, 31 op/s 2024-09-12T00:06:01.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:00 smithi062 ceph-mon[30958]: pgmap v432: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 352 KiB/s wr, 31 op/s 2024-09-12T00:06:02.366 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:02 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:06:02.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:02 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:06:02.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:02 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:06:02.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 800.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:06:02.718 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:06:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:03.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:03 smithi062 ceph-mon[30958]: pgmap v433: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-09-12T00:06:03.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:03 smithi120 ceph-mon[35182]: pgmap v433: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-09-12T00:06:03.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:03 smithi083 ceph-mon[34805]: pgmap v433: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-09-12T00:06:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.490+0000 7f1a38912640 1 -- 172.21.15.62:0/3879060310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a34100e20 msgr2=0x7f1a34101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.490+0000 7f1a38912640 1 --2- 172.21.15.62:0/3879060310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a34100e20 0x7f1a34101200 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009900 tx=0x7f1a2402a480 comp rx=0 tx=0).stop 2024-09-12T00:06:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.490+0000 7f1a38912640 1 -- 172.21.15.62:0/3879060310 shutdown_connections 2024-09-12T00:06:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.490+0000 7f1a38912640 1 --2- 172.21.15.62:0/3879060310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a34102ef0 0x7f1a3410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.490+0000 7f1a38912640 1 --2- 172.21.15.62:0/3879060310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a34102570 0x7f1a341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.490+0000 7f1a38912640 1 --2- 172.21.15.62:0/3879060310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a34100e20 0x7f1a34101200 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.490+0000 7f1a38912640 1 -- 172.21.15.62:0/3879060310 >> 172.21.15.62:0/3879060310 conn(0x7f1a340fc7d0 msgr2=0x7f1a340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.490+0000 7f1a38912640 1 -- 172.21.15.62:0/3879060310 shutdown_connections 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.490+0000 7f1a38912640 1 -- 172.21.15.62:0/3879060310 wait complete. 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a38912640 1 Processor -- start 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a38912640 1 -- start start 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a38912640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a34100e20 0x7f1a341984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a38912640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a34102570 0x7f1a34198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a38912640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a34102ef0 0x7f1a3419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a38912640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34110f80 con 0x7f1a34102570 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a38912640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34110e00 con 0x7f1a34102ef0 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a38912640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34111100 con 0x7f1a34100e20 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a2bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a34102570 0x7f1a34198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a2bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a34102570 0x7f1a34198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41304/0 (socket says 172.21.15.62:41304) 2024-09-12T00:06:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a2bfff640 1 -- 172.21.15.62:0/658575025 learned_addr learned my addr 172.21.15.62:0/658575025 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.491+0000 7f1a32d9d640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a34102ef0 0x7f1a3419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a3259c640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a34100e20 0x7f1a341984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a2bfff640 1 -- 172.21.15.62:0/658575025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a34100e20 msgr2=0x7f1a341984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a2bfff640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a34100e20 0x7f1a341984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a2bfff640 1 -- 172.21.15.62:0/658575025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a34102ef0 msgr2=0x7f1a3419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a2bfff640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a34102ef0 0x7f1a3419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a2bfff640 1 -- 172.21.15.62:0/658575025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3419d480 con 0x7f1a34102570 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a2bfff640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a34102570 0x7f1a34198a10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c920 tx=0x7f1a1c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a2b7fe640 1 -- 172.21.15.62:0/658575025 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c007c70 con 0x7f1a34102570 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a2b7fe640 1 -- 172.21.15.62:0/658575025 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1c007e10 con 0x7f1a34102570 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a38912640 1 -- 172.21.15.62:0/658575025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3419d770 con 0x7f1a34102570 2024-09-12T00:06:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a2b7fe640 1 -- 172.21.15.62:0/658575025 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c018890 con 0x7f1a34102570 2024-09-12T00:06:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.492+0000 7f1a38912640 1 -- 172.21.15.62:0/658575025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a341a5050 con 0x7f1a34102570 2024-09-12T00:06:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.493+0000 7f1a38912640 1 -- 172.21.15.62:0/658575025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a34101200 con 0x7f1a34102570 2024-09-12T00:06:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.496+0000 7f1a2b7fe640 1 -- 172.21.15.62:0/658575025 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1a1c018a50 con 0x7f1a34102570 2024-09-12T00:06:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.497+0000 7f1a2b7fe640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1a200a2d70 0x7f1a200a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.497+0000 7f1a2b7fe640 1 -- 172.21.15.62:0/658575025 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f1a1c0cf720 con 0x7f1a34102570 2024-09-12T00:06:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.497+0000 7f1a3259c640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1a200a2d70 0x7f1a200a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:03.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.497+0000 7f1a3259c640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1a200a2d70 0x7f1a200a5230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1a24000c00 tx=0x7f1a240058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:03.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.502+0000 7f1a2b7fe640 1 -- 172.21.15.62:0/658575025 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a1c014030 con 0x7f1a34102570 2024-09-12T00:06:03.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.719+0000 7f1a38912640 1 -- 172.21.15.62:0/658575025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a3410a0d0 con 0x7f1a34102570 2024-09-12T00:06:03.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.720+0000 7f1a2b7fe640 1 -- 172.21.15.62:0/658575025 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11909 (secure 0 0 0) 0x7f1a1c09c450 con 0x7f1a34102570 2024-09-12T00:06:03.720 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:06:03.720 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":39,"btime":"2024-09-12T00:06:00:989943+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:06:00.711094+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:06:03.720 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 39 2024-09-12T00:06:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 -- 172.21.15.62:0/658575025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1a200a2d70 msgr2=0x7f1a200a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1a200a2d70 0x7f1a200a5230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1a24000c00 tx=0x7f1a240058d0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 -- 172.21.15.62:0/658575025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a34102570 msgr2=0x7f1a34198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a34102570 0x7f1a34198a10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c920 tx=0x7f1a1c00cde0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 -- 172.21.15.62:0/658575025 shutdown_connections 2024-09-12T00:06:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1a200a2d70 0x7f1a200a5230 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a34102ef0 0x7f1a3419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a34102570 0x7f1a34198a10 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 --2- 172.21.15.62:0/658575025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a34100e20 0x7f1a341984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 -- 172.21.15.62:0/658575025 >> 172.21.15.62:0/658575025 conn(0x7f1a340fc7d0 msgr2=0x7f1a3406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 -- 172.21.15.62:0/658575025 shutdown_connections 2024-09-12T00:06:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:03.722+0000 7f1a38912640 1 -- 172.21.15.62:0/658575025 wait complete. 2024-09-12T00:06:04.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:04 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/658575025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:04.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:04 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/658575025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:04.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:04 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/658575025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:04.621 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id aebadee8 --release --await' 2024-09-12T00:06:04.622 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id aebadee8 --release --await 2024-09-12T00:06:04.882 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:05.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:05 smithi062 ceph-mon[30958]: pgmap v434: 97 pgs: 97 active+clean; 4.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 934 op/s 2024-09-12T00:06:05.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:05 smithi120 ceph-mon[35182]: pgmap v434: 97 pgs: 97 active+clean; 4.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 934 op/s 2024-09-12T00:06:05.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.370+0000 7fed766b6640 1 -- 172.21.15.62:0/649721509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed70101420 msgr2=0x7fed7010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:05.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.370+0000 7fed766b6640 1 --2- 172.21.15.62:0/649721509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed70101420 0x7fed7010d950 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fed68009cd0 tx=0x7fed6802a8e0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.370+0000 7fed766b6640 1 -- 172.21.15.62:0/649721509 shutdown_connections 2024-09-12T00:06:05.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.370+0000 7fed766b6640 1 --2- 172.21.15.62:0/649721509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed70101420 0x7fed7010d950 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.370+0000 7fed766b6640 1 --2- 172.21.15.62:0/649721509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70100a80 0x7fed70100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.370+0000 7fed766b6640 1 --2- 172.21.15.62:0/649721509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed70106a80 0x7fed70106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.370+0000 7fed766b6640 1 -- 172.21.15.62:0/649721509 >> 172.21.15.62:0/649721509 conn(0x7fed700fc7a0 msgr2=0x7fed700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.370+0000 7fed766b6640 1 -- 172.21.15.62:0/649721509 shutdown_connections 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.370+0000 7fed766b6640 1 -- 172.21.15.62:0/649721509 wait complete. 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed766b6640 1 Processor -- start 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed766b6640 1 -- start start 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed766b6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed70100a80 0x7fed700fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed766b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70101420 0x7fed700fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed766b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed70106a80 0x7fed700fd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed766b6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fed70110ea0 con 0x7fed70100a80 2024-09-12T00:06:05.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed766b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed70110d20 con 0x7fed70101420 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed766b6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed70111020 con 0x7fed70106a80 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed70100a80 0x7fed700fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed70100a80 0x7fed700fccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41326/0 (socket says 172.21.15.62:41326) 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed6ffff640 1 -- 172.21.15.62:0/836193330 learned_addr learned my addr 172.21.15.62:0/836193330 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed67fff640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70101420 0x7fed700fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed6ffff640 1 -- 172.21.15.62:0/836193330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed70106a80 msgr2=0x7fed700fd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.371+0000 7fed6ffff640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed70106a80 0x7fed700fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.372+0000 7fed6ffff640 1 -- 172.21.15.62:0/836193330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70101420 msgr2=0x7fed700fd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.372+0000 7fed6ffff640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70101420 0x7fed700fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.372+0000 7fed6ffff640 1 -- 172.21.15.62:0/836193330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed701a6900 con 0x7fed70100a80 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.372+0000 7fed67fff640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70101420 0x7fed700fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:05.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.372+0000 7fed6ffff640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed70100a80 0x7fed700fccc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fed6000c920 tx=0x7fed6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.372+0000 7fed6dffb640 1 -- 172.21.15.62:0/836193330 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60007c70 con 0x7fed70100a80 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.372+0000 7fed6dffb640 1 -- 172.21.15.62:0/836193330 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed60007e10 con 0x7fed70100a80 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.372+0000 7fed766b6640 1 -- 172.21.15.62:0/836193330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed701a6bf0 con 0x7fed70100a80 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.372+0000 7fed6dffb640 1 -- 172.21.15.62:0/836193330 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60018890 con 0x7fed70100a80 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.372+0000 7fed766b6640 1 -- 172.21.15.62:0/836193330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed701a7180 con 0x7fed70100a80 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.374+0000 7fed6dffb640 1 -- 172.21.15.62:0/836193330 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fed60018a70 con 0x7fed70100a80 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed78a8f740 1 Processor -- start 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed78a8f740 1 -- start start 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d31440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d31980 0x565392d3bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d3c240 0x565392d3e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed78a8f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565392c96620 con 0x565392d3c240 2024-09-12T00:06:05.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed78a8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565392c3de50 con 0x565392d31980 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed78a8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x565392c925e0 con 0x565392d31060 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed6dffb640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fed5c0a2d70 0x7fed5c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed67fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d3c240 0x565392d3e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed67fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d3c240 0x565392d3e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41338/0 (socket says 172.21.15.62:41338) 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed67fff640 1 -- 172.21.15.62:0/632066226 learned_addr learned my addr 172.21.15.62:0/632066226 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed74c53640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fed5c0a2d70 0x7fed5c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed6dffb640 1 -- 172.21.15.62:0/836193330 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fed600cf5b0 con 0x7fed70100a80 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed6ffff640 1 --2- 172.21.15.62:0/632066226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d31980 0x565392d3bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed74c53640 1 --2- 172.21.15.62:0/632066226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d31440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed67fff640 1 -- 172.21.15.62:0/632066226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 msgr2=0x565392d31440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed67fff640 1 --2- 172.21.15.62:0/632066226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d31440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed67fff640 1 -- 172.21.15.62:0/632066226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d31980 msgr2=0x565392d3bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed67fff640 1 --2- 172.21.15.62:0/632066226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d31980 0x565392d3bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.375+0000 7fed67fff640 1 -- 172.21.15.62:0/632066226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565392c2ed40 con 0x565392d3c240 2024-09-12T00:06:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.376+0000 7fed67fff640 1 --2- 172.21.15.62:0/632066226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d3c240 0x565392d3e630 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fed5800a9d0 tx=0x7fed5800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1c842881df4cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.376+0000 7fed74c53640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fed5c0a2d70 0x7fed5c0a5230 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fed68009430 tx=0x7fed680093a0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.376+0000 7fed667fc640 1 -- 172.21.15.62:0/632066226 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58015070 con 0x565392d3c240 2024-09-12T00:06:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.376+0000 7fed667fc640 1 -- 172.21.15.62:0/632066226 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed5800cc60 con 0x565392d3c240 2024-09-12T00:06:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.376+0000 7fed667fc640 1 -- 172.21.15.62:0/632066226 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed580185c0 con 0x565392d3c240 2024-09-12T00:06:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 -- 172.21.15.62:0/632066226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d3c240 msgr2=0x565392d3e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 --2- 172.21.15.62:0/632066226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d3c240 0x565392d3e630 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fed5800a9d0 tx=0x7fed5800ae90 comp rx=0 tx=0).stop 2024-09-12T00:06:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 -- 172.21.15.62:0/632066226 shutdown_connections 2024-09-12T00:06:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 --2- 172.21.15.62:0/632066226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d3c240 0x565392d3e630 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 --2- 172.21.15.62:0/632066226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d31980 0x565392d3bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 --2- 172.21.15.62:0/632066226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d31440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 -- 172.21.15.62:0/632066226 >> 172.21.15.62:0/632066226 conn(0x565392d2d5f0 msgr2=0x565392d2d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:05.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 -- 172.21.15.62:0/632066226 shutdown_connections 2024-09-12T00:06:05.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 -- 172.21.15.62:0/632066226 wait complete. 2024-09-12T00:06:05.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 -- start start 2024-09-12T00:06:05.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.377+0000 7fed78a8f740 1 Processor -- start 2024-09-12T00:06:05.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d39070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d39af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed78a8f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565392d3ee10 con 0x565392d31980 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed78a8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565392c1ef10 con 0x565392d3c240 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed78a8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x565392d3ef90 con 0x565392d31060 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed78a8f740 10 client.0 _ll_register_callbacks cb 0x565392c42db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed67fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed67fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d395b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41350/0 (socket says 172.21.15.62:41350) 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed67fff640 1 -- 172.21.15.62:0/2316088438 learned_addr learned my addr 172.21.15.62:0/2316088438 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed6ffff640 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d39070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed74c53640 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d39af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed67fff640 1 -- 172.21.15.62:0/2316088438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 msgr2=0x565392d39070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed67fff640 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d39070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed67fff640 1 -- 172.21.15.62:0/2316088438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 msgr2=0x565392d39af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed67fff640 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d39af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed67fff640 1 -- 172.21.15.62:0/2316088438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653929a5580 con 0x565392d31980 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed67fff640 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d395b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fed58004700 tx=0x7fed58006cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e227f6025273dcb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed4f7fe640 1 -- 172.21.15.62:0/2316088438 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58020070 con 0x565392d31980 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed4f7fe640 1 -- 172.21.15.62:0/2316088438 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fed5801a650 con 0x565392d31980 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.378+0000 7fed78a8f740 1 -- 172.21.15.62:0/2316088438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed5800cc60 con 0x565392d31980 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed4f7fe640 1 -- 172.21.15.62:0/2316088438 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fed58015040 con 0x565392d31980 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed78a8f740 10 client.17880 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed78a8f740 1 -- 172.21.15.62:0/2316088438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565392a6f960 con 0x565392d31980 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed4f7fe640 1 -- 172.21.15.62:0/2316088438 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed5800cc60 con 0x565392d31980 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed4f7fe640 1 -- 172.21.15.62:0/2316088438 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fed58024830 con 0x565392d31980 2024-09-12T00:06:05.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed78a8f740 10 client.17880 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:05.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed78a8f740 10 client.17880 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:06:05.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed78a8f740 20 client.17880 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:05.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed78a8f740 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565392d4bc50 0x565392d4e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed78a8f740 4 client.17880 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:06:05.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed78a8f740 1 -- 172.21.15.62:0/2316088438 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565392ca3b90 con 0x565392d4bc50 2024-09-12T00:06:05.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.379+0000 7fed6ffff640 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565392d4bc50 0x565392d4e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.380+0000 7fed6ffff640 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565392d4bc50 0x565392d4e040 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566535ec43dafb15 server_cookie=a333942d11be909a in_seq=0 out_seq=0 2024-09-12T00:06:05.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.380+0000 7fed4f7fe640 10 client.17880 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:05.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed4f7fe640 1 -- 172.21.15.62:0/2316088438 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565392ca3b90 con 0x565392d4bc50 2024-09-12T00:06:05.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed4f7fe640 10 client.17880 handle_command_reply: tid=0 2024-09-12T00:06:05.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 client.17880 shutdown 2024-09-12T00:06:05.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 -- 172.21.15.62:0/2316088438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 msgr2=0x565392d395b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:05.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d395b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fed58004700 tx=0x7fed58006cf0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 -- 172.21.15.62:0/2316088438 shutdown_connections 2024-09-12T00:06:05.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565392d4bc50 0x565392d4e040 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d39af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d395b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 --2- 172.21.15.62:0/2316088438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d39070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 -- 172.21.15.62:0/2316088438 >> 172.21.15.62:0/2316088438 conn(0x565392d2d5f0 msgr2=0x565392d332d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:05.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed4f7fe640 0 client.17880 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:05.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 -- 172.21.15.62:0/2316088438 shutdown_connections 2024-09-12T00:06:05.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.381+0000 7fed78a8f740 1 -- 172.21.15.62:0/2316088438 wait complete. 2024-09-12T00:06:05.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.382+0000 7fed78a8f740 20 client.17880 trim_cache size 0 max 16384 2024-09-12T00:06:05.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed78a8f740 1 Processor -- start 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed78a8f740 1 -- start start 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d53670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d53bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed78a8f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fed701b23a0 con 0x565392d31980 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed78a8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed701afc70 con 0x565392d3c240 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed78a8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed701adae0 con 0x565392d31060 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed67fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d53670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed67fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d53670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41362/0 (socket says 172.21.15.62:41362) 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed67fff640 1 -- 172.21.15.62:0/1918882527 learned_addr learned my addr 172.21.15.62:0/1918882527 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed6ffff640 1 --2- 172.21.15.62:0/1918882527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed67fff640 1 -- 172.21.15.62:0/1918882527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 msgr2=0x565392d331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:05.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed74c53640 1 --2- 172.21.15.62:0/1918882527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d53bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed67fff640 1 --2- 172.21.15.62:0/1918882527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d331c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed67fff640 1 -- 172.21.15.62:0/1918882527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 msgr2=0x565392d53bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:05.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed67fff640 1 --2- 172.21.15.62:0/1918882527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d53bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed67fff640 1 -- 172.21.15.62:0/1918882527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565392d2d420 con 0x565392d31980 2024-09-12T00:06:05.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed74c53640 1 --2- 172.21.15.62:0/1918882527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d53bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:05.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.399+0000 7fed67fff640 1 --2- 172.21.15.62:0/1918882527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d53670 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fed5800aed0 tx=0x7fed58006e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8965636299fcb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:05.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.400+0000 7fed65ffb640 1 -- 172.21.15.62:0/1918882527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58020040 con 0x565392d31980 2024-09-12T00:06:05.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.400+0000 7fed65ffb640 1 -- 172.21.15.62:0/1918882527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed580248f0 con 0x565392d31980 2024-09-12T00:06:05.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.400+0000 7fed65ffb640 1 -- 172.21.15.62:0/1918882527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5801a7b0 con 0x565392d31980 2024-09-12T00:06:05.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.400+0000 7fed78a8f740 1 -- 172.21.15.62:0/1918882527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 msgr2=0x565392d53670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:05.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.400+0000 7fed78a8f740 1 --2- 172.21.15.62:0/1918882527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d53670 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fed5800aed0 tx=0x7fed58006e80 comp rx=0 tx=0).stop 2024-09-12T00:06:05.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.400+0000 7fed78a8f740 1 -- 172.21.15.62:0/1918882527 shutdown_connections 2024-09-12T00:06:05.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.400+0000 7fed78a8f740 1 --2- 172.21.15.62:0/1918882527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d53bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.400+0000 7fed78a8f740 1 --2- 172.21.15.62:0/1918882527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31980 0x565392d53670 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.400+0000 7fed78a8f740 1 --2- 172.21.15.62:0/1918882527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31060 0x565392d331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.400+0000 7fed78a8f740 1 -- 172.21.15.62:0/1918882527 >> 172.21.15.62:0/1918882527 conn(0x565392d2d5f0 msgr2=0x565392d32ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:05.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.401+0000 7fed78a8f740 1 -- 172.21.15.62:0/1918882527 shutdown_connections 2024-09-12T00:06:05.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.401+0000 7fed78a8f740 1 -- 172.21.15.62:0/1918882527 wait complete. 2024-09-12T00:06:05.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.401+0000 7fed78a8f740 1 -- start start 2024-09-12T00:06:05.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.401+0000 7fed78a8f740 1 Processor -- start 2024-09-12T00:06:05.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.401+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31060 0x565392d441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31980 0x565392d44720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed78a8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d411b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed78a8f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fed701ae130 con 0x565392d31060 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed78a8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed701af5a0 con 0x565392d3c240 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed78a8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed701ae800 con 0x565392d31980 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed78a8f740 10 client.0 _ll_register_callbacks cb 0x7fed58011860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31060 0x565392d441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31060 0x565392d441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41376/0 (socket says 172.21.15.62:41376) 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed6ffff640 1 -- 172.21.15.62:0/1148093328 learned_addr learned my addr 172.21.15.62:0/1148093328 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed67fff640 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31980 0x565392d44720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed74c53640 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed6ffff640 1 -- 172.21.15.62:0/1148093328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31980 msgr2=0x565392d44720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:05.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed6ffff640 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31980 0x565392d44720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed6ffff640 1 -- 172.21.15.62:0/1148093328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 msgr2=0x565392d411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed6ffff640 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed6ffff640 1 -- 172.21.15.62:0/1148093328 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565392d416f0 con 0x565392d31060 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed6ffff640 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31060 0x565392d441e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fed60007590 tx=0x7fed60000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b80ff87760d9820 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed4f7fe640 1 -- 172.21.15.62:0/1148093328 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6007e020 con 0x565392d31060 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed4f7fe640 1 -- 172.21.15.62:0/1148093328 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7fed60018d60 con 0x565392d31060 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.402+0000 7fed78a8f740 1 -- 172.21.15.62:0/1148093328 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed580248f0 con 0x565392d31060 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed4f7fe640 1 -- 172.21.15.62:0/1148093328 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fed6000fb30 con 0x565392d31060 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed78a8f740 10 client.17886 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed78a8f740 1 -- 172.21.15.62:0/1148093328 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565392db6e80 con 0x565392d31060 2024-09-12T00:06:05.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed4f7fe640 1 -- 172.21.15.62:0/1148093328 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed580248f0 con 0x565392d31060 2024-09-12T00:06:05.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed4f7fe640 1 -- 172.21.15.62:0/1148093328 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fed600a42b0 con 0x565392d31060 2024-09-12T00:06:05.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed78a8f740 10 client.17886 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:05.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed78a8f740 10 client.17886 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:06:05.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed78a8f740 20 client.17886 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:05.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed78a8f740 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565392db7530 0x565392db9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:05.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed78a8f740 4 client.17886 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "aebadee8", "release": true, "await": true}] 2024-09-12T00:06:05.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed78a8f740 1 -- 172.21.15.62:0/1148093328 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "aebadee8", "release": true, "await": true}) -- 0x565392c9e0a0 con 0x565392db7530 2024-09-12T00:06:05.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.403+0000 7fed67fff640 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565392db7530 0x565392db9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:05.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.404+0000 7fed4f7fe640 10 client.17886 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:05.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.404+0000 7fed67fff640 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565392db7530 0x565392db9950 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413dd90581efd59c server_cookie=48a168cc174a0011 in_seq=0 out_seq=0 2024-09-12T00:06:05.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:05 smithi083 ceph-mon[34805]: pgmap v434: 97 pgs: 97 active+clean; 4.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 934 op/s 2024-09-12T00:06:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed4f7fe640 1 -- 172.21.15.62:0/1148093328 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565392c9e0a0 con 0x565392db7530 2024-09-12T00:06:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed4f7fe640 10 client.17886 handle_command_reply: tid=0 2024-09-12T00:06:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 client.17886 shutdown 2024-09-12T00:06:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 -- 172.21.15.62:0/1148093328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31060 msgr2=0x565392d441e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31060 0x565392d441e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fed60007590 tx=0x7fed60000bc0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 -- 172.21.15.62:0/1148093328 shutdown_connections 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x565392db7530 0x565392db9950 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565392d3c240 0x565392d411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565392d31980 0x565392d44720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 --2- 172.21.15.62:0/1148093328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565392d31060 0x565392d441e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 -- 172.21.15.62:0/1148093328 >> 172.21.15.62:0/1148093328 conn(0x565392d2d5f0 msgr2=0x565392d332d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed4f7fe640 0 client.17886 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 -- 172.21.15.62:0/1148093328 shutdown_connections 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.466+0000 7fed78a8f740 1 -- 172.21.15.62:0/1148093328 wait complete. 2024-09-12T00:06:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.467+0000 7fed78a8f740 20 client.17886 trim_cache size 0 max 16384 2024-09-12T00:06:05.469 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:06:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 39, 2024-09-12T00:06:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:06:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 52, 2024-09-12T00:06:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:06:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "aebadee8": { 2024-09-12T00:06:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 52, 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:06:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:06:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:06:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:05.471 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.467+0000 7fed766b6640 1 -- 172.21.15.62:0/836193330 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fed5c0a2d70 msgr2=0x7fed5c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.467+0000 7fed766b6640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fed5c0a2d70 0x7fed5c0a5230 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fed68009430 tx=0x7fed680093a0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.468+0000 7fed766b6640 1 -- 172.21.15.62:0/836193330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed70100a80 msgr2=0x7fed700fccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.468+0000 7fed766b6640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed70100a80 0x7fed700fccc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fed6000c920 tx=0x7fed6000cde0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.468+0000 7fed766b6640 1 -- 172.21.15.62:0/836193330 shutdown_connections 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.468+0000 7fed766b6640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fed5c0a2d70 0x7fed5c0a5230 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.468+0000 7fed766b6640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed70106a80 0x7fed700fd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.468+0000 7fed766b6640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed70101420 0x7fed700fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.468+0000 7fed766b6640 1 --2- 172.21.15.62:0/836193330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed70100a80 0x7fed700fccc0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.468+0000 7fed766b6640 1 -- 172.21.15.62:0/836193330 >> 172.21.15.62:0/836193330 conn(0x7fed700fc7a0 msgr2=0x7fed7006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.468+0000 7fed766b6640 1 -- 172.21.15.62:0/836193330 shutdown_connections 2024-09-12T00:06:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:05.468+0000 7fed766b6640 1 -- 172.21.15.62:0/836193330 wait complete. 2024-09-12T00:06:06.252 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'aebadee8', total seconds elapsed: 33.6 2024-09-12T00:06:06.252 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T00:06:06.252 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:06:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:07.166 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:07 smithi062 ceph-mon[30958]: pgmap v435: 97 pgs: 97 active+clean; 4.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 930 op/s 2024-09-12T00:06:07.166 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:07 smithi062 ceph-mon[30958]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T00:06:07.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.303+0000 7fe165197640 1 -- 172.21.15.62:0/1317713998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe16011b140 msgr2=0x7fe16011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:07.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.303+0000 7fe165197640 1 --2- 172.21.15.62:0/1317713998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe16011b140 0x7fe16011d530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe158009ca0 tx=0x7fe15802a8f0 comp rx=0 tx=0).stop 2024-09-12T00:06:07.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.306+0000 7fe165197640 1 -- 172.21.15.62:0/1317713998 shutdown_connections 2024-09-12T00:06:07.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.306+0000 7fe165197640 1 --2- 172.21.15.62:0/1317713998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe16011b140 0x7fe16011d530 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:07.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.306+0000 7fe165197640 1 --2- 172.21.15.62:0/1317713998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe160109830 0x7fe160109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:07.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.306+0000 7fe165197640 1 --2- 172.21.15.62:0/1317713998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe160109450 0x7fe160110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:07.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.306+0000 7fe165197640 1 -- 172.21.15.62:0/1317713998 >> 172.21.15.62:0/1317713998 conn(0x7fe16006e140 msgr2=0x7fe16006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:07.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.311+0000 7fe165197640 1 -- 172.21.15.62:0/1317713998 shutdown_connections 2024-09-12T00:06:07.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.315+0000 7fe165197640 1 -- 172.21.15.62:0/1317713998 wait complete. 2024-09-12T00:06:07.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.316+0000 7fe165197640 1 Processor -- start 2024-09-12T00:06:07.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.317+0000 7fe165197640 1 -- start start 2024-09-12T00:06:07.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.317+0000 7fe165197640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe160109450 0x7fe16011a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.317+0000 7fe165197640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe160109830 0x7fe160114830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.317+0000 7fe165197640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16011b140 0x7fe160114d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.317+0000 7fe165197640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe160111330 con 0x7fe16011b140 2024-09-12T00:06:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.317+0000 7fe165197640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601111b0 con 0x7fe160109830 2024-09-12T00:06:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.317+0000 7fe165197640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601114b0 con 0x7fe160109450 2024-09-12T00:06:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.317+0000 7fe15f59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16011b140 0x7fe160114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:07.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.318+0000 7fe15f59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16011b140 0x7fe160114d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41402/0 (socket says 172.21.15.62:41402) 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.318+0000 7fe15f59e640 1 -- 172.21.15.62:0/2714612711 learned_addr learned my addr 172.21.15.62:0/2714612711 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.318+0000 7fe15f59e640 1 -- 172.21.15.62:0/2714612711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe160109450 msgr2=0x7fe16011a7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.318+0000 7fe15f59e640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe160109450 0x7fe16011a7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.318+0000 7fe15f59e640 1 -- 172.21.15.62:0/2714612711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe160109830 msgr2=0x7fe160114830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.318+0000 7fe15f59e640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe160109830 0x7fe160114830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.318+0000 7fe15f59e640 1 -- 172.21.15.62:0/2714612711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1601153b0 con 0x7fe16011b140 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.318+0000 7fe15f59e640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16011b140 0x7fe160114d70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe158031e70 tx=0x7fe15802f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.319+0000 7fe147fff640 1 -- 172.21.15.62:0/2714612711 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15802fa50 con 0x7fe16011b140 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.319+0000 7fe147fff640 1 -- 172.21.15.62:0/2714612711 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15802d070 con 0x7fe16011b140 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.319+0000 7fe147fff640 1 -- 172.21.15.62:0/2714612711 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580466e0 con 0x7fe16011b140 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.319+0000 7fe165197640 1 -- 172.21.15.62:0/2714612711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe160115640 con 0x7fe16011b140 2024-09-12T00:06:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.319+0000 7fe165197640 1 -- 172.21.15.62:0/2714612711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1601bc420 con 0x7fe16011b140 2024-09-12T00:06:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.325+0000 7fe165197640 1 -- 172.21.15.62:0/2714612711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe160073b00 con 0x7fe16011b140 2024-09-12T00:06:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.327+0000 7fe147fff640 1 -- 172.21.15.62:0/2714612711 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe1580305f0 con 0x7fe16011b140 2024-09-12T00:06:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.328+0000 7fe147fff640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1480a2f00 0x7fe1480a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.328+0000 7fe147fff640 1 -- 172.21.15.62:0/2714612711 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8557+0+0 (secure 0 0 0) 0x7fe1580f0d30 con 0x7fe16011b140 2024-09-12T00:06:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.332+0000 7fe15ed9d640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1480a2f00 0x7fe1480a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.334+0000 7fe147fff640 1 -- 172.21.15.62:0/2714612711 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe15804b060 con 0x7fe16011b140 2024-09-12T00:06:07.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.340+0000 7fe15ed9d640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1480a2f00 0x7fe1480a53c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe15000b990 tx=0x7fe15000b370 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:07.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:07 smithi120 ceph-mon[35182]: pgmap v435: 97 pgs: 97 active+clean; 4.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 930 op/s 2024-09-12T00:06:07.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:07 smithi120 ceph-mon[35182]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T00:06:07.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:07 smithi083 ceph-mon[34805]: pgmap v435: 97 pgs: 97 active+clean; 4.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 930 op/s 2024-09-12T00:06:07.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:07 smithi083 ceph-mon[34805]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T00:06:07.444 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 805.0 s, 453.5 tps, lat 1314.200 ms stddev 5750.925 2024-09-12T00:06:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.667+0000 7fe165197640 1 -- 172.21.15.62:0/2714612711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe16010d5c0 con 0x7fe16011b140 2024-09-12T00:06:07.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.668+0000 7fe147fff640 1 -- 172.21.15.62:0/2714612711 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11909 (secure 0 0 0) 0x7fe158039380 con 0x7fe16011b140 2024-09-12T00:06:07.667 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:06:07.667 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":39,"btime":"2024-09-12T00:06:00:989943+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:06:00.711094+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:06:07.667 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 39 2024-09-12T00:06:07.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 -- 172.21.15.62:0/2714612711 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1480a2f00 msgr2=0x7fe1480a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1480a2f00 0x7fe1480a53c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe15000b990 tx=0x7fe15000b370 comp rx=0 tx=0).stop 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 -- 172.21.15.62:0/2714612711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16011b140 msgr2=0x7fe160114d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16011b140 0x7fe160114d70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe158031e70 tx=0x7fe15802f480 comp rx=0 tx=0).stop 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 -- 172.21.15.62:0/2714612711 shutdown_connections 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe1480a2f00 0x7fe1480a53c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16011b140 0x7fe160114d70 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe160109830 0x7fe160114830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 --2- 172.21.15.62:0/2714612711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe160109450 0x7fe16011a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 -- 172.21.15.62:0/2714612711 >> 172.21.15.62:0/2714612711 conn(0x7fe16006e140 msgr2=0x7fe160072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 -- 172.21.15.62:0/2714612711 shutdown_connections 2024-09-12T00:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:07.671+0000 7fe165197640 1 -- 172.21.15.62:0/2714612711 wait complete. 2024-09-12T00:06:08.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:08 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2714612711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:08.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:08 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2714612711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:08.417 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:08 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2714612711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:08.457 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T00:06:08.457 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T00:06:08.855 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:09.271 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:09 smithi120 ceph-mon[35182]: pgmap v437: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 22 MiB/s wr, 1.38k op/s 2024-09-12T00:06:09.295 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:09 smithi062 ceph-mon[30958]: pgmap v437: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 22 MiB/s wr, 1.38k op/s 2024-09-12T00:06:09.317 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:09 smithi083 ceph-mon[34805]: pgmap v437: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 22 MiB/s wr, 1.38k op/s 2024-09-12T00:06:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.434+0000 7f15faaf9640 1 -- 172.21.15.62:0/1740917670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f4106a80 msgr2=0x7f15f4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.434+0000 7f15faaf9640 1 --2- 172.21.15.62:0/1740917670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f4106a80 0x7f15f4106e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009860 tx=0x7f15e802a340 comp rx=0 tx=0).stop 2024-09-12T00:06:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.435+0000 7f15faaf9640 1 -- 172.21.15.62:0/1740917670 shutdown_connections 2024-09-12T00:06:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.435+0000 7f15faaf9640 1 --2- 172.21.15.62:0/1740917670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15f4101420 0x7f15f410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.435+0000 7f15faaf9640 1 --2- 172.21.15.62:0/1740917670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15f4100a80 0x7f15f4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.435+0000 7f15faaf9640 1 --2- 172.21.15.62:0/1740917670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f4106a80 0x7f15f4106e60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.435+0000 7f15faaf9640 1 -- 172.21.15.62:0/1740917670 >> 172.21.15.62:0/1740917670 conn(0x7f15f40fc7a0 msgr2=0x7f15f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.435+0000 7f15faaf9640 1 -- 172.21.15.62:0/1740917670 shutdown_connections 2024-09-12T00:06:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.435+0000 7f15faaf9640 1 -- 172.21.15.62:0/1740917670 wait complete. 2024-09-12T00:06:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.436+0000 7f15faaf9640 1 Processor -- start 2024-09-12T00:06:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.436+0000 7f15faaf9640 1 -- start start 2024-09-12T00:06:09.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15faaf9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15f4100a80 0x7f15f419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15faaf9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15f4101420 0x7f15f419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15faaf9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f4106a80 0x7f15f4196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15f8895640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15f4100a80 0x7f15f419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15f8895640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15f4100a80 0x7f15f419e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41418/0 (socket says 172.21.15.62:41418) 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15f4101420 0x7f15f419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15f4101420 0x7f15f419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:41578/0 (socket says 172.21.15.62:41578) 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15f8895640 1 -- 172.21.15.62:0/3053173282 learned_addr learned my addr 172.21.15.62:0/3053173282 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4110e80 con 0x7f15f4100a80 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4110d00 con 0x7f15f4106a80 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4111000 con 0x7f15f4101420 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.437+0000 7f15f9096640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f4106a80 0x7f15f4196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.438+0000 7f15f9096640 1 -- 172.21.15.62:0/3053173282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15f4101420 msgr2=0x7f15f419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.438+0000 7f15f9096640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15f4101420 0x7f15f419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.438+0000 7f15f9096640 1 -- 172.21.15.62:0/3053173282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15f4100a80 msgr2=0x7f15f419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.438+0000 7f15f9096640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15f4100a80 0x7f15f419e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.438+0000 7f15f9096640 1 -- 172.21.15.62:0/3053173282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f4196c50 con 0x7f15f4106a80 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.438+0000 7f15f3fff640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15f4101420 0x7f15f419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.438+0000 7f15f8895640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15f4100a80 0x7f15f419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.438+0000 7f15f9096640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f4106a80 0x7f15f4196390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00efc0 tx=0x7f15ec00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.439+0000 7f15f1ffb640 1 -- 172.21.15.62:0/3053173282 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec003970 con 0x7f15f4106a80 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.439+0000 7f15f1ffb640 1 -- 172.21.15.62:0/3053173282 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ec004020 con 0x7f15f4106a80 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.439+0000 7f15f1ffb640 1 -- 172.21.15.62:0/3053173282 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00b9f0 con 0x7f15f4106a80 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.439+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f4196f40 con 0x7f15f4106a80 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.439+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f419ecf0 con 0x7f15f4106a80 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.442+0000 7f15f1ffb640 1 -- 172.21.15.62:0/3053173282 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f15ec01d020 con 0x7f15f4106a80 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.442+0000 7f15fced2740 1 Processor -- start 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.442+0000 7f15fced2740 1 -- start start 2024-09-12T00:06:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.443+0000 7f15fced2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c46760 0x5622e9c46b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.443+0000 7f15fced2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 0x5622e9c51400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.443+0000 7f15fced2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c51940 0x5622e9c53d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.443+0000 7f15fced2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5622e9bac2a0 con 0x5622e9c51940 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.443+0000 7f15fced2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622e9b53b10 con 0x5622e9c47080 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.443+0000 7f15fced2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5622e9ba8260 con 0x5622e9c46760 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c46760 0x5622e9c46b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c46760 0x5622e9c46b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:41582/0 (socket says 172.21.15.62:41582) 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f3fff640 1 -- 172.21.15.62:0/687660643 learned_addr learned my addr 172.21.15.62:0/687660643 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f3fff640 1 --2- 172.21.15.62:0/687660643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c51940 0x5622e9c53d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f8895640 1 --2- 172.21.15.62:0/687660643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 0x5622e9c51400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f3fff640 1 -- 172.21.15.62:0/687660643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 msgr2=0x5622e9c51400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f3fff640 1 --2- 172.21.15.62:0/687660643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 0x5622e9c51400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f3fff640 1 -- 172.21.15.62:0/687660643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c51940 msgr2=0x5622e9c53d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f3fff640 1 --2- 172.21.15.62:0/687660643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c51940 0x5622e9c53d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f3fff640 1 -- 172.21.15.62:0/687660643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e991db90 con 0x5622e9c46760 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f8895640 1 --2- 172.21.15.62:0/687660643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 0x5622e9c51400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f3fff640 1 --2- 172.21.15.62:0/687660643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c46760 0x5622e9c46b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009710 tx=0x7f15e0009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9a8d7287114732 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15f1ffb640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15e40a2d70 0x7f15e40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.444+0000 7f15deffd640 1 -- 172.21.15.62:0/687660643 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e000f8c0 con 0x5622e9c46760 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.445+0000 7f15f8895640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15e40a2d70 0x7f15e40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.445+0000 7f15deffd640 1 -- 172.21.15.62:0/687660643 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e001e420 con 0x5622e9c46760 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.445+0000 7f15deffd640 1 -- 172.21.15.62:0/687660643 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e000e5f0 con 0x5622e9c46760 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.445+0000 7f15f1ffb640 1 -- 172.21.15.62:0/3053173282 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8557+0+0 (secure 0 0 0) 0x7f15ec0cf1d0 con 0x7f15f4106a80 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15fced2740 1 -- 172.21.15.62:0/687660643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c46760 msgr2=0x5622e9c46b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15fced2740 1 --2- 172.21.15.62:0/687660643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c46760 0x5622e9c46b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009710 tx=0x7f15e0009be0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15f8895640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15e40a2d70 0x7f15e40a5230 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f15e802a930 tx=0x7f15e80056d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15fced2740 1 -- 172.21.15.62:0/687660643 shutdown_connections 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15fced2740 1 --2- 172.21.15.62:0/687660643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c51940 0x5622e9c53d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15fced2740 1 --2- 172.21.15.62:0/687660643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 0x5622e9c51400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15fced2740 1 --2- 172.21.15.62:0/687660643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c46760 0x5622e9c46b40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15fced2740 1 -- 172.21.15.62:0/687660643 >> 172.21.15.62:0/687660643 conn(0x5622e9c42cf0 msgr2=0x5622e9c430d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15fced2740 1 -- 172.21.15.62:0/687660643 shutdown_connections 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15fced2740 1 -- 172.21.15.62:0/687660643 wait complete. 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.446+0000 7f15fced2740 1 -- start start 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15fced2740 1 Processor -- start 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15fced2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c46760 0x5622e9c4e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15fced2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 0x5622e9c4ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15fced2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c51940 0x5622e9c4f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15f8895640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c46760 0x5622e9c4e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15f8895640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c46760 0x5622e9c4e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:60268/0 (socket says 172.21.15.62:60268) 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15fced2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5622e9c54510 con 0x5622e9c51940 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15f8895640 1 -- 172.21.15.62:0/341032221 learned_addr learned my addr 172.21.15.62:0/341032221 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15fced2740 1 -- 172.21.15.62:0/341032221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622e9b346e0 con 0x5622e9c46760 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15fced2740 1 -- 172.21.15.62:0/341032221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5622e9c54690 con 0x5622e9c47080 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15fced2740 10 client.0 _ll_register_callbacks cb 0x5622e9b58a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15f3fff640 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 0x5622e9c4ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.447+0000 7f15f9096640 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c51940 0x5622e9c4f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.448+0000 7f15f8895640 1 -- 172.21.15.62:0/341032221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 msgr2=0x5622e9c4ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.448+0000 7f15f8895640 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 0x5622e9c4ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.448+0000 7f15f8895640 1 -- 172.21.15.62:0/341032221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c51940 msgr2=0x5622e9c4f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.448+0000 7f15f8895640 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c51940 0x5622e9c4f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.448+0000 7f15f8895640 1 -- 172.21.15.62:0/341032221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e9c5da60 con 0x5622e9c46760 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.448+0000 7f15f8895640 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c46760 0x5622e9c4e850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f15e8004090 tx=0x7f15e8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45fc5ddcdefc459f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.448+0000 7f15d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.448+0000 7f15d3fff640 1 -- 172.21.15.62:0/341032221 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8037040 con 0x5622e9c46760 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.448+0000 7f15d3fff640 1 -- 172.21.15.62:0/341032221 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8557+0+0 (secure 0 0 0) 0x7f15e8003a00 con 0x5622e9c46760 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.449+0000 7f15fced2740 1 -- 172.21.15.62:0/341032221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e001e420 con 0x5622e9c46760 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.449+0000 7f15d3fff640 1 -- 172.21.15.62:0/341032221 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f15e8003c00 con 0x5622e9c46760 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.449+0000 7f15fced2740 10 client.27392 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.449+0000 7f15fced2740 1 -- 172.21.15.62:0/341032221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5622e9c60340 con 0x5622e9c46760 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.449+0000 7f15d3fff640 1 -- 172.21.15.62:0/341032221 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e804d070 con 0x5622e9c46760 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.449+0000 7f15d3fff640 1 -- 172.21.15.62:0/341032221 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f15e80525a0 con 0x5622e9c46760 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.449+0000 7f15fced2740 10 client.27392 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.450+0000 7f15fced2740 10 client.27392 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:06:09.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.450+0000 7f15fced2740 20 client.27392 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.450+0000 7f15fced2740 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5622e9c61800 0x5622e9c63c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.450+0000 7f15fced2740 4 client.27392 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.450+0000 7f15fced2740 1 -- 172.21.15.62:0/341032221 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f15e001e420 con 0x5622e9c61800 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.450+0000 7f15f3fff640 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5622e9c61800 0x5622e9c63c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.451+0000 7f15f3fff640 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5622e9c61800 0x5622e9c63c50 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d83ab12c8cf4fa server_cookie=b202702f341f3e5e in_seq=0 out_seq=0 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.451+0000 7f15d3fff640 10 client.27392 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.451+0000 7f15d3fff640 1 -- 172.21.15.62:0/341032221 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f15e001e420 con 0x5622e9c61800 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.451+0000 7f15d3fff640 10 client.27392 handle_command_reply: tid=0 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.451+0000 7f15fced2740 1 client.27392 shutdown 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.451+0000 7f15fced2740 1 -- 172.21.15.62:0/341032221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c46760 msgr2=0x5622e9c4e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.451+0000 7f15fced2740 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c46760 0x5622e9c4e850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f15e8004090 tx=0x7f15e8005f90 comp rx=0 tx=0).stop 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.452+0000 7f15fced2740 1 -- 172.21.15.62:0/341032221 shutdown_connections 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.452+0000 7f15fced2740 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5622e9c61800 0x5622e9c63c50 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.452+0000 7f15d3fff640 0 client.27392 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.452+0000 7f15fced2740 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c51940 0x5622e9c4f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.452+0000 7f15fced2740 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 0x5622e9c4ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.452+0000 7f15fced2740 1 --2- 172.21.15.62:0/341032221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c46760 0x5622e9c4e850 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.452+0000 7f15fced2740 1 -- 172.21.15.62:0/341032221 >> 172.21.15.62:0/341032221 conn(0x5622e9c42cf0 msgr2=0x5622e9c489d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.452+0000 7f15fced2740 1 -- 172.21.15.62:0/341032221 shutdown_connections 2024-09-12T00:06:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.452+0000 7f15fced2740 1 -- 172.21.15.62:0/341032221 wait complete. 2024-09-12T00:06:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.452+0000 7f15fced2740 20 client.27392 trim_cache size 0 max 16384 2024-09-12T00:06:09.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.475+0000 7f15fced2740 1 Processor -- start 2024-09-12T00:06:09.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.475+0000 7f15fced2740 1 -- start start 2024-09-12T00:06:09.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15fced2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c4ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15fced2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 0x5622e9c69520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15fced2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c51940 0x5622e9c69a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f8895640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c4ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f8895640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c4ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41436/0 (socket says 172.21.15.62:41436) 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f8895640 1 -- 172.21.15.62:0/3589603871 learned_addr learned my addr 172.21.15.62:0/3589603871 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15fced2740 1 -- 172.21.15.62:0/3589603871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15f41a9ff0 con 0x5622e9c46760 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15fced2740 1 -- 172.21.15.62:0/3589603871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15f41a7900 con 0x5622e9c51940 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15fced2740 1 -- 172.21.15.62:0/3589603871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15f41971d0 con 0x5622e9c47080 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f3fff640 1 --2- 172.21.15.62:0/3589603871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 0x5622e9c69520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f9096640 1 --2- 172.21.15.62:0/3589603871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c51940 0x5622e9c69a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f3fff640 1 -- 172.21.15.62:0/3589603871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c51940 msgr2=0x5622e9c69a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f3fff640 1 --2- 172.21.15.62:0/3589603871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c51940 0x5622e9c69a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f3fff640 1 -- 172.21.15.62:0/3589603871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 msgr2=0x5622e9c4ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f3fff640 1 --2- 172.21.15.62:0/3589603871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c4ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f3fff640 1 -- 172.21.15.62:0/3589603871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e9c642d0 con 0x5622e9c47080 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.476+0000 7f15f3fff640 1 --2- 172.21.15.62:0/3589603871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 0x5622e9c69520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f15e00096e0 tx=0x7f15e001ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1cb8cf0edf988c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.477+0000 7f15de7fc640 1 -- 172.21.15.62:0/3589603871 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e001e6f0 con 0x5622e9c47080 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.477+0000 7f15de7fc640 1 -- 172.21.15.62:0/3589603871 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e0005a70 con 0x5622e9c47080 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.477+0000 7f15de7fc640 1 -- 172.21.15.62:0/3589603871 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e00052a0 con 0x5622e9c47080 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.478+0000 7f15fced2740 1 -- 172.21.15.62:0/3589603871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 msgr2=0x5622e9c69520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.478+0000 7f15fced2740 1 --2- 172.21.15.62:0/3589603871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 0x5622e9c69520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f15e00096e0 tx=0x7f15e001ee40 comp rx=0 tx=0).stop 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.478+0000 7f15fced2740 1 -- 172.21.15.62:0/3589603871 shutdown_connections 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.478+0000 7f15fced2740 1 --2- 172.21.15.62:0/3589603871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c51940 0x5622e9c69a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.478+0000 7f15fced2740 1 --2- 172.21.15.62:0/3589603871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c47080 0x5622e9c69520 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.478+0000 7f15fced2740 1 --2- 172.21.15.62:0/3589603871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c4ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.478+0000 7f15fced2740 1 -- 172.21.15.62:0/3589603871 >> 172.21.15.62:0/3589603871 conn(0x5622e9c42cf0 msgr2=0x5622e9c4e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.478+0000 7f15fced2740 1 -- 172.21.15.62:0/3589603871 shutdown_connections 2024-09-12T00:06:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.478+0000 7f15fced2740 1 -- 172.21.15.62:0/3589603871 wait complete. 2024-09-12T00:06:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.478+0000 7f15fced2740 1 -- start start 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15fced2740 1 Processor -- start 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15fced2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15fced2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 0x5622e9c55e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15fced2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c51940 0x5622e9c56340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15fced2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15f41a5e00 con 0x5622e9c46760 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15fced2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15f41a7230 con 0x5622e9c47080 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15fced2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15f41a6490 con 0x5622e9c51940 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15fced2740 10 client.0 _ll_register_callbacks cb 0x7f15e80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15f8895640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15f8895640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41444/0 (socket says 172.21.15.62:41444) 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15f8895640 1 -- 172.21.15.62:0/928206058 learned_addr learned my addr 172.21.15.62:0/928206058 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.479+0000 7f15f9096640 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c51940 0x5622e9c56340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15f3fff640 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 0x5622e9c55e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15f8895640 1 -- 172.21.15.62:0/928206058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c51940 msgr2=0x5622e9c56340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15f8895640 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c51940 0x5622e9c56340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15f8895640 1 -- 172.21.15.62:0/928206058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 msgr2=0x5622e9c55e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15f8895640 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 0x5622e9c55e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15f8895640 1 -- 172.21.15.62:0/928206058 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e9ccb740 con 0x5622e9c46760 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15f3fff640 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 0x5622e9c55e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15f8895640 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c558c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f15e8052a40 tx=0x7f15e8003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd402dd90c39232 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15d3fff640 1 -- 172.21.15.62:0/928206058 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8037040 con 0x5622e9c46760 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15d3fff640 1 -- 172.21.15.62:0/928206058 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8557+0+0 (secure 0 0 0) 0x7f15e8032510 con 0x5622e9c46760 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.480+0000 7f15fced2740 1 -- 172.21.15.62:0/928206058 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e0005a70 con 0x5622e9c46760 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.481+0000 7f15d3fff640 1 -- 172.21.15.62:0/928206058 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f15e8032710 con 0x5622e9c46760 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.481+0000 7f15fced2740 10 client.17910 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.481+0000 7f15fced2740 1 -- 172.21.15.62:0/928206058 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5622e9ccca00 con 0x5622e9c46760 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.481+0000 7f15d3fff640 1 -- 172.21.15.62:0/928206058 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e0005a70 con 0x5622e9c46760 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.482+0000 7f15d3fff640 1 -- 172.21.15.62:0/928206058 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f15e805a070 con 0x5622e9c46760 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.482+0000 7f15fced2740 10 client.17910 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.482+0000 7f15fced2740 10 client.17910 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.482+0000 7f15fced2740 20 client.17910 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.482+0000 7f15fced2740 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5622e9ccd180 0x5622e9ccf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.482+0000 7f15fced2740 4 client.17910 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.482+0000 7f15fced2740 1 -- 172.21.15.62:0/928206058 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5622e9bb99d0 con 0x5622e9ccd180 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.483+0000 7f15f3fff640 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5622e9ccd180 0x5622e9ccf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.483+0000 7f15f3fff640 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5622e9ccd180 0x5622e9ccf590 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b6702aec22c661 server_cookie=80adea46312d2f4 in_seq=0 out_seq=0 2024-09-12T00:06:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.483+0000 7f15d3fff640 10 client.17910 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:09.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.682+0000 7f15d3fff640 1 -- 172.21.15.62:0/928206058 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5622e9bb99d0 con 0x5622e9ccd180 2024-09-12T00:06:09.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.682+0000 7f15d3fff640 10 client.17910 handle_command_reply: tid=0 2024-09-12T00:06:09.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.682+0000 7f15fced2740 1 client.17910 shutdown 2024-09-12T00:06:09.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 1 -- 172.21.15.62:0/928206058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 msgr2=0x5622e9c558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:09.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c558c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f15e8052a40 tx=0x7f15e8003b00 comp rx=0 tx=0).stop 2024-09-12T00:06:09.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 1 -- 172.21.15.62:0/928206058 shutdown_connections 2024-09-12T00:06:09.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5622e9ccd180 0x5622e9ccf590 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622e9c51940 0x5622e9c56340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622e9c47080 0x5622e9c55e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 1 --2- 172.21.15.62:0/928206058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e9c46760 0x5622e9c558c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15d3fff640 0 client.17910 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 1 -- 172.21.15.62:0/928206058 >> 172.21.15.62:0/928206058 conn(0x5622e9c42cf0 msgr2=0x5622e9c4f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 1 -- 172.21.15.62:0/928206058 shutdown_connections 2024-09-12T00:06:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 1 -- 172.21.15.62:0/928206058 wait complete. 2024-09-12T00:06:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.683+0000 7f15fced2740 20 client.17910 trim_cache size 0 max 16384 2024-09-12T00:06:09.683 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:06:09.683 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 39, 2024-09-12T00:06:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:06:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 54, 2024-09-12T00:06:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:06:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: "3dc716a2": { 2024-09-12T00:06:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 54, 2024-09-12T00:06:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:06:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:06:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:06:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:09.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:09.686 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:06:09.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15e40a2d70 msgr2=0x7f15e40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:09.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15e40a2d70 0x7f15e40a5230 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f15e802a930 tx=0x7f15e80056d0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f4106a80 msgr2=0x7f15f4196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:09.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f4106a80 0x7f15f4196390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00efc0 tx=0x7f15ec00c800 comp rx=0 tx=0).stop 2024-09-12T00:06:09.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 shutdown_connections 2024-09-12T00:06:09.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f15e40a2d70 0x7f15e40a5230 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f4106a80 0x7f15f4196390 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15f4101420 0x7f15f419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 --2- 172.21.15.62:0/3053173282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15f4100a80 0x7f15f419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:09.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 >> 172.21.15.62:0/3053173282 conn(0x7f15f40fc7a0 msgr2=0x7f15f406a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:09.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 shutdown_connections 2024-09-12T00:06:09.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:09.684+0000 7f15faaf9640 1 -- 172.21.15.62:0/3053173282 wait complete. 2024-09-12T00:06:10.511 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3dc716a2', quiesce took 4.3 seconds. Will release after: 25.7 2024-09-12T00:06:11.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:11 smithi120 ceph-mon[35182]: pgmap v438: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-12T00:06:11.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:11 smithi083 ceph-mon[34805]: pgmap v438: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-12T00:06:11.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:11 smithi062 ceph-mon[30958]: pgmap v438: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-12T00:06:12.447 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 810.0 s, 755.3 tps, lat 54.667 ms stddev 58.701 2024-09-12T00:06:13.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:12 smithi120 ceph-mon[35182]: pgmap v439: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-12T00:06:13.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:12 smithi120 ceph-mon[35182]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T00:06:13.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:12 smithi083 ceph-mon[34805]: pgmap v439: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-12T00:06:13.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:12 smithi083 ceph-mon[34805]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T00:06:13.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:12 smithi062 ceph-mon[30958]: pgmap v439: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-12T00:06:13.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:12 smithi062 ceph-mon[30958]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T00:06:15.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:14 smithi120 ceph-mon[35182]: pgmap v441: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 6.2 MiB/s wr, 536 op/s 2024-09-12T00:06:15.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:14 smithi083 ceph-mon[34805]: pgmap v441: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 6.2 MiB/s wr, 536 op/s 2024-09-12T00:06:15.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:14 smithi062 ceph-mon[30958]: pgmap v441: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 6.2 MiB/s wr, 536 op/s 2024-09-12T00:06:17.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:16 smithi120 ceph-mon[35182]: pgmap v442: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.1 MiB/s wr, 443 op/s 2024-09-12T00:06:17.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:16 smithi083 ceph-mon[34805]: pgmap v442: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.1 MiB/s wr, 443 op/s 2024-09-12T00:06:17.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:16 smithi062 ceph-mon[30958]: pgmap v442: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.1 MiB/s wr, 443 op/s 2024-09-12T00:06:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 815.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:06:19.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:18 smithi120 ceph-mon[35182]: pgmap v443: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-12T00:06:19.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:18 smithi083 ceph-mon[34805]: pgmap v443: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-12T00:06:19.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:18 smithi062 ceph-mon[30958]: pgmap v443: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-12T00:06:21.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:20 smithi120 ceph-mon[35182]: pgmap v444: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-12T00:06:21.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:20 smithi083 ceph-mon[34805]: pgmap v444: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-12T00:06:21.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:20 smithi062 ceph-mon[30958]: pgmap v444: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-12T00:06:21.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:06:22.255 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:22.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 820.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:06:22.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.733+0000 7fc90091c640 1 -- 172.21.15.62:0/1559384203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc101de0 msgr2=0x7fc8fc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:22.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.733+0000 7fc90091c640 1 --2- 172.21.15.62:0/1559384203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc101de0 0x7fc8fc1021c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009900 tx=0x7fc8ec02a410 comp rx=0 tx=0).stop 2024-09-12T00:06:22.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.733+0000 7fc90091c640 1 -- 172.21.15.62:0/1559384203 shutdown_connections 2024-09-12T00:06:22.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.733+0000 7fc90091c640 1 --2- 172.21.15.62:0/1559384203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8fc106f10 0x7fc8fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.733+0000 7fc90091c640 1 --2- 172.21.15.62:0/1559384203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8fc102790 0x7fc8fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.733+0000 7fc90091c640 1 --2- 172.21.15.62:0/1559384203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc101de0 0x7fc8fc1021c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.733+0000 7fc90091c640 1 -- 172.21.15.62:0/1559384203 >> 172.21.15.62:0/1559384203 conn(0x7fc8fc0fd8c0 msgr2=0x7fc8fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:22.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.733+0000 7fc90091c640 1 -- 172.21.15.62:0/1559384203 shutdown_connections 2024-09-12T00:06:22.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.733+0000 7fc90091c640 1 -- 172.21.15.62:0/1559384203 wait complete. 2024-09-12T00:06:22.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc90091c640 1 Processor -- start 2024-09-12T00:06:22.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc90091c640 1 -- start start 2024-09-12T00:06:22.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc90091c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8fc101de0 0x7fc8fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc90091c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc102790 0x7fc8fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc90091c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8fc106f10 0x7fc8fc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc90091c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc111f90 con 0x7fc8fc102790 2024-09-12T00:06:22.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc90091c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc111e10 con 0x7fc8fc101de0 2024-09-12T00:06:22.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc90091c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc112110 con 0x7fc8fc106f10 2024-09-12T00:06:22.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc8fa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8fc101de0 0x7fc8fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc8fa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8fc101de0 0x7fc8fc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57066/0 (socket says 172.21.15.62:57066) 2024-09-12T00:06:22.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc8fa59c640 1 -- 172.21.15.62:0/1192251975 learned_addr learned my addr 172.21.15.62:0/1192251975 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:22.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.734+0000 7fc8f3fff640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc102790 0x7fc8fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8fc106f10 0x7fc8fc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8fa59c640 1 -- 172.21.15.62:0/1192251975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8fc106f10 msgr2=0x7fc8fc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:22.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8fa59c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8fc106f10 0x7fc8fc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8fa59c640 1 -- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc102790 msgr2=0x7fc8fc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:22.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8fa59c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc102790 0x7fc8fc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8fa59c640 1 -- 172.21.15.62:0/1192251975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc1aad80 con 0x7fc8fc101de0 2024-09-12T00:06:22.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8f3fff640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc102790 0x7fc8fc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:06:22.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8fa59c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8fc101de0 0x7fc8fc0744f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec005ab0 tx=0x7fc8ec0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:22.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8f37fe640 1 -- 172.21.15.62:0/1192251975 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec03f030 con 0x7fc8fc101de0 2024-09-12T00:06:22.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8f37fe640 1 -- 172.21.15.62:0/1192251975 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8ec003ce0 con 0x7fc8fc101de0 2024-09-12T00:06:22.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc90091c640 1 -- 172.21.15.62:0/1192251975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc1aaf50 con 0x7fc8fc101de0 2024-09-12T00:06:22.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc8f37fe640 1 -- 172.21.15.62:0/1192251975 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec035070 con 0x7fc8fc101de0 2024-09-12T00:06:22.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.735+0000 7fc90091c640 1 -- 172.21.15.62:0/1192251975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc1ab3b0 con 0x7fc8fc101de0 2024-09-12T00:06:22.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc902cf5740 1 Processor -- start 2024-09-12T00:06:22.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc902cf5740 1 -- start start 2024-09-12T00:06:22.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f5f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 0x55a19f604140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc8f37fe640 1 -- 172.21.15.62:0/1192251975 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc8ec042030 con 0x7fc8fc101de0 2024-09-12T00:06:22.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f604680 0x55a19f606a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc902cf5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a19f55f2e0 con 0x55a19f5f94a0 2024-09-12T00:06:22.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc902cf5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a19f506ce0 con 0x55a19f5f9dc0 2024-09-12T00:06:22.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc902cf5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a19f55b2a0 con 0x55a19f604680 2024-09-12T00:06:22.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc8fad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f5f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc8fad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f5f9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55136/0 (socket says 172.21.15.62:55136) 2024-09-12T00:06:22.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.738+0000 7fc8fad9d640 1 -- 172.21.15.62:0/964933088 learned_addr learned my addr 172.21.15.62:0/964933088 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:22.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8fa59c640 1 --2- 172.21.15.62:0/964933088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 0x55a19f604140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8f3fff640 1 --2- 172.21.15.62:0/964933088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f604680 0x55a19f606a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8fad9d640 1 -- 172.21.15.62:0/964933088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f604680 msgr2=0x55a19f606a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:22.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/964933088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f604680 0x55a19f606a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8fad9d640 1 -- 172.21.15.62:0/964933088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 msgr2=0x55a19f604140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:22.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/964933088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 0x55a19f604140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8fad9d640 1 -- 172.21.15.62:0/964933088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19f4eb440 con 0x55a19f5f94a0 2024-09-12T00:06:22.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8fa59c640 1 --2- 172.21.15.62:0/964933088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 0x55a19f604140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:22.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/964933088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f5f9880 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400a430 tx=0x7fc8f4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9338123b3f8c1a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:22.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8f37fe640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc8e80a2cf0 0x7fc8e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8f0ff9640 1 -- 172.21.15.62:0/964933088 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4013790 con 0x55a19f5f94a0 2024-09-12T00:06:22.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8f0ff9640 1 -- 172.21.15.62:0/964933088 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4019070 con 0x55a19f5f94a0 2024-09-12T00:06:22.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8f0ff9640 1 -- 172.21.15.62:0/964933088 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4014da0 con 0x55a19f5f94a0 2024-09-12T00:06:22.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8fa59c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc8e80a2cf0 0x7fc8e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.739+0000 7fc8f37fe640 1 -- 172.21.15.62:0/1192251975 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fc8ec0efd20 con 0x7fc8fc101de0 2024-09-12T00:06:22.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc8fa59c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc8e80a2cf0 0x7fc8e80a51b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55a19f5f6890 tx=0x7fc8ec0b8480 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:22.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 -- 172.21.15.62:0/964933088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 msgr2=0x55a19f5f9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:22.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 --2- 172.21.15.62:0/964933088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f5f9880 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400a430 tx=0x7fc8f4009650 comp rx=0 tx=0).stop 2024-09-12T00:06:22.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 -- 172.21.15.62:0/964933088 shutdown_connections 2024-09-12T00:06:22.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 --2- 172.21.15.62:0/964933088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f604680 0x55a19f606a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 --2- 172.21.15.62:0/964933088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 0x55a19f604140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 --2- 172.21.15.62:0/964933088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f5f9880 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 -- 172.21.15.62:0/964933088 >> 172.21.15.62:0/964933088 conn(0x55a19f5f5a30 msgr2=0x55a19f5f5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:22.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 -- 172.21.15.62:0/964933088 shutdown_connections 2024-09-12T00:06:22.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 -- 172.21.15.62:0/964933088 wait complete. 2024-09-12T00:06:22.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 -- start start 2024-09-12T00:06:22.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.740+0000 7fc902cf5740 1 Processor -- start 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f94a0 0x55a19f603f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f9dc0 0x55a19f6011f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f604680 0x55a19f601730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc902cf5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a19f607250 con 0x55a19f5f9dc0 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc902cf5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a19f51a050 con 0x55a19f5f94a0 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc902cf5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a19f6073d0 con 0x55a19f604680 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc902cf5740 10 client.0 _ll_register_callbacks cb 0x55a19f55a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8f3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f9dc0 0x55a19f6011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8f3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f9dc0 0x55a19f6011f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55146/0 (socket says 172.21.15.62:55146) 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8f3fff640 1 -- 172.21.15.62:0/213208680 learned_addr learned my addr 172.21.15.62:0/213208680 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:22.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f94a0 0x55a19f603f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f604680 0x55a19f601730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8f3fff640 1 -- 172.21.15.62:0/213208680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f604680 msgr2=0x55a19f601730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8f3fff640 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f604680 0x55a19f601730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8f3fff640 1 -- 172.21.15.62:0/213208680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f94a0 msgr2=0x55a19f603f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8f3fff640 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f94a0 0x55a19f603f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8f3fff640 1 -- 172.21.15.62:0/213208680 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19f2d03c0 con 0x55a19f5f9dc0 2024-09-12T00:06:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8f3fff640 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f9dc0 0x55a19f6011f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4011a00 tx=0x7fc8e4011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f5180c222a034 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/213208680 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e401f060 con 0x55a19f5f9dc0 2024-09-12T00:06:22.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/213208680 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fc8e4012450 con 0x55a19f5f9dc0 2024-09-12T00:06:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.741+0000 7fc902cf5740 1 -- 172.21.15.62:0/213208680 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f4019070 con 0x55a19f5f9dc0 2024-09-12T00:06:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/213208680 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc8e400fe70 con 0x55a19f5f9dc0 2024-09-12T00:06:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc902cf5740 10 client.17922 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc902cf5740 1 -- 172.21.15.62:0/213208680 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a19f612b70 con 0x55a19f5f9dc0 2024-09-12T00:06:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/213208680 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4019070 con 0x55a19f5f9dc0 2024-09-12T00:06:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/213208680 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc8e4012650 con 0x55a19f5f9dc0 2024-09-12T00:06:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc902cf5740 10 client.17922 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc902cf5740 10 client.17922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:06:22.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc902cf5740 20 client.17922 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc902cf5740 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a19f614080 0x55a19f6164c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc902cf5740 4 client.17922 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc902cf5740 1 -- 172.21.15.62:0/213208680 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a19f56c850 con 0x55a19f614080 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.742+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a19f614080 0x55a19f6164c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.743+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a19f614080 0x55a19f6164c0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50e0d7306f48326 server_cookie=908ae8bddd42e0bb in_seq=0 out_seq=0 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.743+0000 7fc8e1ffb640 10 client.17922 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.743+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/213208680 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a19f56c850 con 0x55a19f614080 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.743+0000 7fc8e1ffb640 10 client.17922 handle_command_reply: tid=0 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.743+0000 7fc902cf5740 1 client.17922 shutdown 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.743+0000 7fc902cf5740 1 -- 172.21.15.62:0/213208680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f9dc0 msgr2=0x55a19f6011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.743+0000 7fc902cf5740 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f9dc0 0x55a19f6011f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4011a00 tx=0x7fc8e4011ec0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.744+0000 7fc902cf5740 1 -- 172.21.15.62:0/213208680 shutdown_connections 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.744+0000 7fc902cf5740 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a19f614080 0x55a19f6164c0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.744+0000 7fc902cf5740 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f604680 0x55a19f601730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.744+0000 7fc902cf5740 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f9dc0 0x55a19f6011f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.744+0000 7fc8e1ffb640 0 client.17922 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.744+0000 7fc902cf5740 1 --2- 172.21.15.62:0/213208680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f94a0 0x55a19f603f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.744+0000 7fc902cf5740 1 -- 172.21.15.62:0/213208680 >> 172.21.15.62:0/213208680 conn(0x55a19f5f5a30 msgr2=0x55a19f5f4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.744+0000 7fc902cf5740 1 -- 172.21.15.62:0/213208680 shutdown_connections 2024-09-12T00:06:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.744+0000 7fc902cf5740 1 -- 172.21.15.62:0/213208680 wait complete. 2024-09-12T00:06:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.744+0000 7fc902cf5740 20 client.17922 trim_cache size 0 max 16384 2024-09-12T00:06:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc902cf5740 1 Processor -- start 2024-09-12T00:06:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc902cf5740 1 -- start start 2024-09-12T00:06:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f5f94a0 0x55a19f6104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 0x55a19f61bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f604680 0x55a19f61c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc902cf5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1b6710 con 0x55a19f604680 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc902cf5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1b3fe0 con 0x55a19f5f9dc0 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc902cf5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1b1e50 con 0x55a19f5f94a0 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc8fad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f604680 0x55a19f61c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc8fad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f604680 0x55a19f61c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55148/0 (socket says 172.21.15.62:55148) 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc8fad9d640 1 -- 172.21.15.62:0/1268244614 learned_addr learned my addr 172.21.15.62:0/1268244614 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/1268244614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f5f94a0 0x55a19f6104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.761+0000 7fc8f3fff640 1 --2- 172.21.15.62:0/1268244614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 0x55a19f61bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.762+0000 7fc8fad9d640 1 -- 172.21.15.62:0/1268244614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f5f94a0 msgr2=0x55a19f6104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.762+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/1268244614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f5f94a0 0x55a19f6104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.762+0000 7fc8fad9d640 1 -- 172.21.15.62:0/1268244614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 msgr2=0x55a19f61bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.762+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/1268244614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 0x55a19f61bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.762+0000 7fc8fad9d640 1 -- 172.21.15.62:0/1268244614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19f5f6ae0 con 0x55a19f604680 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.762+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/1268244614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f604680 0x55a19f61c2d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4000f80 tx=0x7fc8f4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b418cbb3bf6ac3f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.762+0000 7fc8e3fff640 1 -- 172.21.15.62:0/1268244614 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40139d0 con 0x55a19f604680 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.762+0000 7fc8e3fff640 1 -- 172.21.15.62:0/1268244614 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4019030 con 0x55a19f604680 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.762+0000 7fc8e3fff640 1 -- 172.21.15.62:0/1268244614 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f402ccb0 con 0x55a19f604680 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 -- 172.21.15.62:0/1268244614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f604680 msgr2=0x55a19f61c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 --2- 172.21.15.62:0/1268244614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f604680 0x55a19f61c2d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4000f80 tx=0x7fc8f4004b50 comp rx=0 tx=0).stop 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 -- 172.21.15.62:0/1268244614 shutdown_connections 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 --2- 172.21.15.62:0/1268244614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f604680 0x55a19f61c2d0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 --2- 172.21.15.62:0/1268244614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f5f9dc0 0x55a19f61bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 --2- 172.21.15.62:0/1268244614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f5f94a0 0x55a19f6104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 -- 172.21.15.62:0/1268244614 >> 172.21.15.62:0/1268244614 conn(0x55a19f5f5a30 msgr2=0x55a19f6002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 -- 172.21.15.62:0/1268244614 shutdown_connections 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 -- 172.21.15.62:0/1268244614 wait complete. 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 -- start start 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 Processor -- start 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f60cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f5f9dc0 0x55a19f607fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f604680 0x55a19f6084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1b24a0 con 0x55a19f5f94a0 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1b3910 con 0x55a19f604680 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1b2b70 con 0x55a19f5f9dc0 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc8fad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f60cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc902cf5740 10 client.0 _ll_register_callbacks cb 0x7fc8e4011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.763+0000 7fc8fad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f60cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55164/0 (socket says 172.21.15.62:55164) 2024-09-12T00:06:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8fad9d640 1 -- 172.21.15.62:0/1319045274 learned_addr learned my addr 172.21.15.62:0/1319045274 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8f3fff640 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f5f9dc0 0x55a19f607fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f604680 0x55a19f6084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8fad9d640 1 -- 172.21.15.62:0/1319045274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f5f9dc0 msgr2=0x55a19f607fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f5f9dc0 0x55a19f607fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8fad9d640 1 -- 172.21.15.62:0/1319045274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f604680 msgr2=0x55a19f6084e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f604680 0x55a19f6084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8fad9d640 1 -- 172.21.15.62:0/1319045274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19f608d10 con 0x55a19f5f94a0 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8fad9d640 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f60cc70 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400ad80 tx=0x7fc8f4009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de98e94697963858 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/1319045274 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f402c930 con 0x55a19f5f94a0 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/1319045274 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fc8f4002030 con 0x55a19f5f94a0 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc902cf5740 1 -- 172.21.15.62:0/1319045274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f4019030 con 0x55a19f5f94a0 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/1319045274 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc8f402cad0 con 0x55a19f5f94a0 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc902cf5740 10 client.17928 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.764+0000 7fc902cf5740 1 -- 172.21.15.62:0/1319045274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a19f67ec70 con 0x55a19f5f94a0 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/1319045274 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4019030 con 0x55a19f5f94a0 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/1319045274 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc8f403a4c0 con 0x55a19f5f94a0 2024-09-12T00:06:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc902cf5740 10 client.17928 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc902cf5740 10 client.17928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc902cf5740 20 client.17928 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc902cf5740 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a19f67f0f0 0x55a19f6814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc902cf5740 4 client.17928 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc902cf5740 1 -- 172.21.15.62:0/1319045274 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a19f566d60 con 0x55a19f67f0f0 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc8f3fff640 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a19f67f0f0 0x55a19f6814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc8e1ffb640 10 client.17928 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.765+0000 7fc8f3fff640 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a19f67f0f0 0x55a19f6814e0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc63e96c355bbd1 server_cookie=39ab665e60eeb145 in_seq=0 out_seq=0 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc8e1ffb640 1 -- 172.21.15.62:0/1319045274 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a19f566d60 con 0x55a19f67f0f0 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc8e1ffb640 10 client.17928 handle_command_reply: tid=0 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 client.17928 shutdown 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 -- 172.21.15.62:0/1319045274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 msgr2=0x55a19f60cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f60cc70 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400ad80 tx=0x7fc8f4009320 comp rx=0 tx=0).stop 2024-09-12T00:06:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 -- 172.21.15.62:0/1319045274 shutdown_connections 2024-09-12T00:06:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a19f67f0f0 0x55a19f6814e0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a19f604680 0x55a19f6084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a19f5f9dc0 0x55a19f607fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 --2- 172.21.15.62:0/1319045274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a19f5f94a0 0x55a19f60cc70 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 -- 172.21.15.62:0/1319045274 >> 172.21.15.62:0/1319045274 conn(0x55a19f5f5a30 msgr2=0x55a19f5fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc8e1ffb640 0 client.17928 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 -- 172.21.15.62:0/1319045274 shutdown_connections 2024-09-12T00:06:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 1 -- 172.21.15.62:0/1319045274 wait complete. 2024-09-12T00:06:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc902cf5740 20 client.17928 trim_cache size 0 max 16384 2024-09-12T00:06:22.776 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:06:22.776 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:06:22.776 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:06:22.776 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc90091c640 1 -- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc8e80a2cf0 msgr2=0x7fc8e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc90091c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc8e80a2cf0 0x7fc8e80a51b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55a19f5f6890 tx=0x7fc8ec0b8480 comp rx=0 tx=0).stop 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc90091c640 1 -- 172.21.15.62:0/1192251975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8fc101de0 msgr2=0x7fc8fc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.766+0000 7fc90091c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8fc101de0 0x7fc8fc0744f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec005ab0 tx=0x7fc8ec0042f0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.767+0000 7fc90091c640 1 -- 172.21.15.62:0/1192251975 shutdown_connections 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.767+0000 7fc90091c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fc8e80a2cf0 0x7fc8e80a51b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.767+0000 7fc90091c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8fc106f10 0x7fc8fc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.767+0000 7fc90091c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc102790 0x7fc8fc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.767+0000 7fc90091c640 1 --2- 172.21.15.62:0/1192251975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8fc101de0 0x7fc8fc0744f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.767+0000 7fc90091c640 1 -- 172.21.15.62:0/1192251975 >> 172.21.15.62:0/1192251975 conn(0x7fc8fc0fd8c0 msgr2=0x7fc8fc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.767+0000 7fc90091c640 1 -- 172.21.15.62:0/1192251975 shutdown_connections 2024-09-12T00:06:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:22.767+0000 7fc90091c640 1 -- 172.21.15.62:0/1192251975 wait complete. 2024-09-12T00:06:22.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:22 smithi062 ceph-mon[30958]: pgmap v445: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-12T00:06:23.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:22 smithi120 ceph-mon[35182]: pgmap v445: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-12T00:06:23.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:22 smithi083 ceph-mon[34805]: pgmap v445: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-12T00:06:23.525 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:06:23.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:06:23.786 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.257+0000 7fef2ccd2640 1 -- 172.21.15.62:0/3961260291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef28102570 msgr2=0x7fef281029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.257+0000 7fef2ccd2640 1 --2- 172.21.15.62:0/3961260291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef28102570 0x7fef281029b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fef10009900 tx=0x7fef1002a480 comp rx=0 tx=0).stop 2024-09-12T00:06:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.257+0000 7fef2ccd2640 1 -- 172.21.15.62:0/3961260291 shutdown_connections 2024-09-12T00:06:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.257+0000 7fef2ccd2640 1 --2- 172.21.15.62:0/3961260291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef28102ef0 0x7fef2810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.257+0000 7fef2ccd2640 1 --2- 172.21.15.62:0/3961260291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef28102570 0x7fef281029b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.257+0000 7fef2ccd2640 1 --2- 172.21.15.62:0/3961260291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef28100e20 0x7fef28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.257+0000 7fef2ccd2640 1 -- 172.21.15.62:0/3961260291 >> 172.21.15.62:0/3961260291 conn(0x7fef280fc7d0 msgr2=0x7fef280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.257+0000 7fef2ccd2640 1 -- 172.21.15.62:0/3961260291 shutdown_connections 2024-09-12T00:06:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.257+0000 7fef2ccd2640 1 -- 172.21.15.62:0/3961260291 wait complete. 2024-09-12T00:06:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef2ccd2640 1 Processor -- start 2024-09-12T00:06:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef2ccd2640 1 -- start start 2024-09-12T00:06:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef2ccd2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef28100e20 0x7fef281985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef2ccd2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef28102570 0x7fef28198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef2ccd2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef28102ef0 0x7fef2819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef2ccd2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fef28110f80 con 0x7fef28102570 2024-09-12T00:06:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef2ccd2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef28110e00 con 0x7fef28102ef0 2024-09-12T00:06:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef2ccd2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef28111100 con 0x7fef28100e20 2024-09-12T00:06:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef25d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef28102570 0x7fef28198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef25d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef28102570 0x7fef28198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55184/0 (socket says 172.21.15.62:55184) 2024-09-12T00:06:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef25d9b640 1 -- 172.21.15.62:0/2079269768 learned_addr learned my addr 172.21.15.62:0/2079269768 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef26d9d640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef28102ef0 0x7fef2819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.258+0000 7fef2659c640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef28100e20 0x7fef281985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef25d9b640 1 -- 172.21.15.62:0/2079269768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef28100e20 msgr2=0x7fef281985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef25d9b640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef28100e20 0x7fef281985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef25d9b640 1 -- 172.21.15.62:0/2079269768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef28102ef0 msgr2=0x7fef2819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef25d9b640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef28102ef0 0x7fef2819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef25d9b640 1 -- 172.21.15.62:0/2079269768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2819d570 con 0x7fef28102570 2024-09-12T00:06:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef26d9d640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef28102ef0 0x7fef2819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:06:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef25d9b640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef28102570 0x7fef28198b00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fef10006010 tx=0x7fef10003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef1f7fe640 1 -- 172.21.15.62:0/2079269768 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1003f070 con 0x7fef28102570 2024-09-12T00:06:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef2ccd2640 1 -- 172.21.15.62:0/2079269768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2819d800 con 0x7fef28102570 2024-09-12T00:06:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef1f7fe640 1 -- 172.21.15.62:0/2079269768 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef10031c30 con 0x7fef28102570 2024-09-12T00:06:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef1f7fe640 1 -- 172.21.15.62:0/2079269768 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10035030 con 0x7fef28102570 2024-09-12T00:06:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.259+0000 7fef2ccd2640 1 -- 172.21.15.62:0/2079269768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef281a5140 con 0x7fef28102570 2024-09-12T00:06:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.260+0000 7fef2ccd2640 1 -- 172.21.15.62:0/2079269768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef281021b0 con 0x7fef28102570 2024-09-12T00:06:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.262+0000 7fef1f7fe640 1 -- 172.21.15.62:0/2079269768 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fef10042070 con 0x7fef28102570 2024-09-12T00:06:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.263+0000 7fef1f7fe640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fef140a2cf0 0x7fef140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.263+0000 7fef2659c640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fef140a2cf0 0x7fef140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.263+0000 7fef1f7fe640 1 -- 172.21.15.62:0/2079269768 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fef100f00e0 con 0x7fef28102570 2024-09-12T00:06:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.264+0000 7fef2659c640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fef140a2cf0 0x7fef140a51b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef20009b60 tx=0x7fef20005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.266+0000 7fef1f7fe640 1 -- 172.21.15.62:0/2079269768 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef100bcd00 con 0x7fef28102570 2024-09-12T00:06:24.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.479+0000 7fef2ccd2640 1 -- 172.21.15.62:0/2079269768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef28101200 con 0x7fef28102570 2024-09-12T00:06:24.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.480+0000 7fef1f7fe640 1 -- 172.21.15.62:0/2079269768 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11909 (secure 0 0 0) 0x7fef28101200 con 0x7fef28102570 2024-09-12T00:06:24.479 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:06:24.479 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":39,"btime":"2024-09-12T00:06:00:989943+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:06:00.711094+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:06:24.479 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 39 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 -- 172.21.15.62:0/2079269768 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fef140a2cf0 msgr2=0x7fef140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fef140a2cf0 0x7fef140a51b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef20009b60 tx=0x7fef20005f90 comp rx=0 tx=0).stop 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 -- 172.21.15.62:0/2079269768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef28102570 msgr2=0x7fef28198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef28102570 0x7fef28198b00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fef10006010 tx=0x7fef10003c10 comp rx=0 tx=0).stop 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 -- 172.21.15.62:0/2079269768 shutdown_connections 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fef140a2cf0 0x7fef140a51b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef28102ef0 0x7fef2819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef28102570 0x7fef28198b00 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 --2- 172.21.15.62:0/2079269768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef28100e20 0x7fef281985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 -- 172.21.15.62:0/2079269768 >> 172.21.15.62:0/2079269768 conn(0x7fef280fc7d0 msgr2=0x7fef2806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:24.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 -- 172.21.15.62:0/2079269768 shutdown_connections 2024-09-12T00:06:24.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:24.482+0000 7fef2ccd2640 1 -- 172.21.15.62:0/2079269768 wait complete. 2024-09-12T00:06:25.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:24 smithi120 ceph-mon[35182]: pgmap v446: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.0 MiB/s wr, 348 op/s 2024-09-12T00:06:25.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:24 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2079269768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:25.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:24 smithi083 ceph-mon[34805]: pgmap v446: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.0 MiB/s wr, 348 op/s 2024-09-12T00:06:25.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:24 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2079269768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:24 smithi062 ceph-mon[30958]: pgmap v446: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.0 MiB/s wr, 348 op/s 2024-09-12T00:06:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:24 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2079269768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:25.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:06:25.646 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:26.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.160+0000 7f891744e640 1 -- 172.21.15.62:0/964462353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8910100d00 msgr2=0x7f8910106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.160+0000 7f891744e640 1 --2- 172.21.15.62:0/964462353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8910100d00 0x7f8910106840 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8900009930 tx=0x7f890002a420 comp rx=0 tx=0).stop 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.160+0000 7f891744e640 1 -- 172.21.15.62:0/964462353 shutdown_connections 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.160+0000 7f891744e640 1 --2- 172.21.15.62:0/964462353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8910106f50 0x7f891010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.160+0000 7f891744e640 1 --2- 172.21.15.62:0/964462353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8910100d00 0x7f8910106840 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.160+0000 7f891744e640 1 --2- 172.21.15.62:0/964462353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8910100350 0x7f8910100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.160+0000 7f891744e640 1 -- 172.21.15.62:0/964462353 >> 172.21.15.62:0/964462353 conn(0x7f8910077210 msgr2=0x7f89100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.161+0000 7f891744e640 1 -- 172.21.15.62:0/964462353 shutdown_connections 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.161+0000 7f891744e640 1 -- 172.21.15.62:0/964462353 wait complete. 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.161+0000 7f891744e640 1 Processor -- start 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.161+0000 7f891744e640 1 -- start start 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.161+0000 7f891744e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8910100350 0x7f89101984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.161+0000 7f891744e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8910100d00 0x7f89101989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.161+0000 7f891744e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8910106f50 0x7f891019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.161+0000 7f891744e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8910112030 con 0x7f8910106f50 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.161+0000 7f891744e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8910111eb0 con 0x7f8910100d00 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.161+0000 7f891744e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89101121b0 con 0x7f8910100350 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89159eb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8910106f50 0x7f891019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89159eb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8910106f50 0x7f891019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55212/0 (socket says 172.21.15.62:55212) 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89159eb640 1 -- 172.21.15.62:0/869881988 learned_addr learned my addr 172.21.15.62:0/869881988 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89151ea640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8910100350 0x7f89101984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89149e9640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8910100d00 0x7f89101989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89159eb640 1 -- 172.21.15.62:0/869881988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8910100350 msgr2=0x7f89101984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89159eb640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8910100350 0x7f89101984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89159eb640 1 -- 172.21.15.62:0/869881988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8910100d00 msgr2=0x7f89101989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89159eb640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8910100d00 0x7f89101989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89159eb640 1 -- 172.21.15.62:0/869881988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891019d460 con 0x7f8910106f50 2024-09-12T00:06:26.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89159eb640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8910106f50 0x7f891019cd80 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f890800b970 tx=0x7f890800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:26.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89067fc640 1 -- 172.21.15.62:0/869881988 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908019070 con 0x7f8910106f50 2024-09-12T00:06:26.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f89067fc640 1 -- 172.21.15.62:0/869881988 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8908014440 con 0x7f8910106f50 2024-09-12T00:06:26.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.162+0000 7f891744e640 1 -- 172.21.15.62:0/869881988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891019d750 con 0x7f8910106f50 2024-09-12T00:06:26.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.163+0000 7f89067fc640 1 -- 172.21.15.62:0/869881988 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89080075d0 con 0x7f8910106f50 2024-09-12T00:06:26.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.163+0000 7f891744e640 1 -- 172.21.15.62:0/869881988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89101a5030 con 0x7f8910106f50 2024-09-12T00:06:26.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.165+0000 7f89067fc640 1 -- 172.21.15.62:0/869881988 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f890801c070 con 0x7f8910106f50 2024-09-12T00:06:26.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.165+0000 7f8919827740 1 Processor -- start 2024-09-12T00:06:26.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.165+0000 7f8919827740 1 -- start start 2024-09-12T00:06:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f8919827740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a151d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f8919827740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a151d5d0 0x55c9a151da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f8919827740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a152a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f8919827740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c9a14832f0 con 0x55c9a1528370 2024-09-12T00:06:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f8919827740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c9a142acf0 con 0x55c9a151d5d0 2024-09-12T00:06:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f8919827740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9a147f2b0 con 0x55c9a151eb40 2024-09-12T00:06:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89067fc640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88fc0a2d70 0x7f88fc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89149e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a152a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89149e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a152a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55226/0 (socket says 172.21.15.62:55226) 2024-09-12T00:06:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89149e9640 1 -- 172.21.15.62:0/4010446211 learned_addr learned my addr 172.21.15.62:0/4010446211 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89151ea640 1 --2- 172.21.15.62:0/4010446211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a151d5d0 0x55c9a151da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89149e9640 1 --2- 172.21.15.62:0/4010446211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a151d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89159eb640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88fc0a2d70 0x7f88fc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89067fc640 1 -- 172.21.15.62:0/869881988 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f89080ce890 con 0x7f8910106f50 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89149e9640 1 -- 172.21.15.62:0/4010446211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 msgr2=0x55c9a151d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89149e9640 1 --2- 172.21.15.62:0/4010446211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a151d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89149e9640 1 -- 172.21.15.62:0/4010446211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a151d5d0 msgr2=0x55c9a151da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89149e9640 1 --2- 172.21.15.62:0/4010446211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a151d5d0 0x55c9a151da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.166+0000 7f89149e9640 1 -- 172.21.15.62:0/4010446211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9a140f3a0 con 0x55c9a1528370 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.167+0000 7f89149e9640 1 --2- 172.21.15.62:0/4010446211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a152a760 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8900031550 tx=0x7f8900005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85f2a9e7e5437a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.167+0000 7f89159eb640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88fc0a2d70 0x7f88fc0a5230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f890809aea0 tx=0x7f890800f300 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.167+0000 7f88fb7fe640 1 -- 172.21.15.62:0/4010446211 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900035030 con 0x55c9a1528370 2024-09-12T00:06:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.167+0000 7f88fb7fe640 1 -- 172.21.15.62:0/4010446211 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8900005500 con 0x55c9a1528370 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.167+0000 7f88fb7fe640 1 -- 172.21.15.62:0/4010446211 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900004e60 con 0x55c9a1528370 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 -- 172.21.15.62:0/4010446211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 msgr2=0x55c9a152a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 --2- 172.21.15.62:0/4010446211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a152a760 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8900031550 tx=0x7f8900005fd0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 -- 172.21.15.62:0/4010446211 shutdown_connections 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 --2- 172.21.15.62:0/4010446211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a152a760 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 --2- 172.21.15.62:0/4010446211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a151d5d0 0x55c9a151da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 --2- 172.21.15.62:0/4010446211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a151d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 -- 172.21.15.62:0/4010446211 >> 172.21.15.62:0/4010446211 conn(0x55c9a14e6310 msgr2=0x55c9a1519c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 -- 172.21.15.62:0/4010446211 shutdown_connections 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 -- 172.21.15.62:0/4010446211 wait complete. 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 -- start start 2024-09-12T00:06:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.168+0000 7f8919827740 1 Processor -- start 2024-09-12T00:06:26.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f8919827740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a151d5d0 0x55c9a1527d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f8919827740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a1525020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f8919827740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a1525610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f8919827740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c9a152b010 con 0x55c9a1528370 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f8919827740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c9a143e060 con 0x55c9a151d5d0 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f8919827740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9a152b190 con 0x55c9a151eb40 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f8919827740 10 client.0 _ll_register_callbacks cb 0x55c9a11918c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89149e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a1525610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89149e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a1525610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55232/0 (socket says 172.21.15.62:55232) 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89149e9640 1 -- 172.21.15.62:0/2307310937 learned_addr learned my addr 172.21.15.62:0/2307310937 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89149e9640 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a151d5d0 0x55c9a1527d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89151ea640 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a1525020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89149e9640 1 -- 172.21.15.62:0/2307310937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 msgr2=0x55c9a1525020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89149e9640 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a1525020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89149e9640 1 -- 172.21.15.62:0/2307310937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a151d5d0 msgr2=0x55c9a1527d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89149e9640 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a151d5d0 0x55c9a1527d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89149e9640 1 -- 172.21.15.62:0/2307310937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9a11f43c0 con 0x55c9a1528370 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f89149e9640 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a1525610 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8900004da0 tx=0x7f8900004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4481d02e3b46684 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f88f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f88f8ff9640 1 -- 172.21.15.62:0/2307310937 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900035030 con 0x55c9a1528370 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f8919827740 1 -- 172.21.15.62:0/2307310937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8900005500 con 0x55c9a1528370 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.169+0000 7f88f8ff9640 1 -- 172.21.15.62:0/2307310937 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f8900004f80 con 0x55c9a1528370 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f88f8ff9640 1 -- 172.21.15.62:0/2307310937 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8900005180 con 0x55c9a1528370 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f8919827740 10 client.17946 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f8919827740 1 -- 172.21.15.62:0/2307310937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c9a1536ab0 con 0x55c9a1528370 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f88f8ff9640 1 -- 172.21.15.62:0/2307310937 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8900005500 con 0x55c9a1528370 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f88f8ff9640 1 -- 172.21.15.62:0/2307310937 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f89000316d0 con 0x55c9a1528370 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f8919827740 10 client.17946 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f8919827740 10 client.17946 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f8919827740 20 client.17946 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f8919827740 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55c9a1537e00 0x55c9a153a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f8919827740 4 client.17946 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f8919827740 1 -- 172.21.15.62:0/2307310937 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9a1490860 con 0x55c9a1537e00 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.170+0000 7f89151ea640 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55c9a1537e00 0x55c9a153a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.171+0000 7f89151ea640 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55c9a1537e00 0x55c9a153a1f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c04457c80cb00d85 server_cookie=9c36b3faa5ebe356 in_seq=0 out_seq=0 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.171+0000 7f88f8ff9640 10 client.17946 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f88f8ff9640 1 -- 172.21.15.62:0/2307310937 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9a1490860 con 0x55c9a1537e00 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f88f8ff9640 10 client.17946 handle_command_reply: tid=0 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 client.17946 shutdown 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 -- 172.21.15.62:0/2307310937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 msgr2=0x55c9a1525610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a1525610 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8900004da0 tx=0x7f8900004ca0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 -- 172.21.15.62:0/2307310937 shutdown_connections 2024-09-12T00:06:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55c9a1537e00 0x55c9a153a1f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a1528370 0x55c9a1525610 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a1525020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 --2- 172.21.15.62:0/2307310937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a151d5d0 0x55c9a1527d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 -- 172.21.15.62:0/2307310937 >> 172.21.15.62:0/2307310937 conn(0x55c9a14e6310 msgr2=0x55c9a151f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f88f8ff9640 0 client.17946 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:06:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 -- 172.21.15.62:0/2307310937 shutdown_connections 2024-09-12T00:06:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 1 -- 172.21.15.62:0/2307310937 wait complete. 2024-09-12T00:06:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.172+0000 7f8919827740 20 client.17946 trim_cache size 0 max 16384 2024-09-12T00:06:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.185+0000 7f8919827740 1 Processor -- start 2024-09-12T00:06:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.185+0000 7f8919827740 1 -- start start 2024-09-12T00:06:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.185+0000 7f8919827740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a1519700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.185+0000 7f8919827740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a1519c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.185+0000 7f8919827740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a1528370 0x55c9a153ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.185+0000 7f8919827740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b00f0 con 0x55c9a151d5d0 2024-09-12T00:06:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.185+0000 7f8919827740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89101ad9c0 con 0x55c9a1528370 2024-09-12T00:06:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.185+0000 7f8919827740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89101ab830 con 0x55c9a151eb40 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89149e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a1519700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89149e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a1519700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55242/0 (socket says 172.21.15.62:55242) 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89149e9640 1 -- 172.21.15.62:0/2048948483 learned_addr learned my addr 172.21.15.62:0/2048948483 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89151ea640 1 --2- 172.21.15.62:0/2048948483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a1519c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89149e9640 1 -- 172.21.15.62:0/2048948483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 msgr2=0x55c9a1519c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89149e9640 1 --2- 172.21.15.62:0/2048948483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a1519c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89149e9640 1 -- 172.21.15.62:0/2048948483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a1528370 msgr2=0x55c9a153ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89149e9640 1 --2- 172.21.15.62:0/2048948483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a1528370 0x55c9a153ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89149e9640 1 -- 172.21.15.62:0/2048948483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9a151a180 con 0x55c9a151d5d0 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89151ea640 1 --2- 172.21.15.62:0/2048948483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a1519c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f89149e9640 1 --2- 172.21.15.62:0/2048948483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a1519700 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8900004d70 tx=0x7f8900004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813b7e26ccc27abb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f88faffd640 1 -- 172.21.15.62:0/2048948483 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900035040 con 0x55c9a151d5d0 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f88faffd640 1 -- 172.21.15.62:0/2048948483 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890005b070 con 0x55c9a151d5d0 2024-09-12T00:06:26.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.186+0000 7f88faffd640 1 -- 172.21.15.62:0/2048948483 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900005740 con 0x55c9a151d5d0 2024-09-12T00:06:26.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 -- 172.21.15.62:0/2048948483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 msgr2=0x55c9a1519700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:26.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 --2- 172.21.15.62:0/2048948483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a1519700 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8900004d70 tx=0x7f8900004310 comp rx=0 tx=0).stop 2024-09-12T00:06:26.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 -- 172.21.15.62:0/2048948483 shutdown_connections 2024-09-12T00:06:26.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 --2- 172.21.15.62:0/2048948483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a1528370 0x55c9a153ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 --2- 172.21.15.62:0/2048948483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a1519c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 --2- 172.21.15.62:0/2048948483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a1519700 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 -- 172.21.15.62:0/2048948483 >> 172.21.15.62:0/2048948483 conn(0x55c9a14e6310 msgr2=0x55c9a1524fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:26.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 -- 172.21.15.62:0/2048948483 shutdown_connections 2024-09-12T00:06:26.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 -- 172.21.15.62:0/2048948483 wait complete. 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 -- start start 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.187+0000 7f8919827740 1 Processor -- start 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f8919827740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a153a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f8919827740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a153ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f8919827740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a1528370 0x55c9a159db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f8919827740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89101abe80 con 0x55c9a151d5d0 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f8919827740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89101ad2f0 con 0x55c9a1528370 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f8919827740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89101ac550 con 0x55c9a151eb40 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f8919827740 10 client.0 _ll_register_callbacks cb 0x7f890c00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89149e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a153a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89149e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a153a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55250/0 (socket says 172.21.15.62:55250) 2024-09-12T00:06:26.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89149e9640 1 -- 172.21.15.62:0/696127725 learned_addr learned my addr 172.21.15.62:0/696127725 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89151ea640 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a153ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89149e9640 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a1528370 0x55c9a159db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89149e9640 1 -- 172.21.15.62:0/696127725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 msgr2=0x55c9a153ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89149e9640 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a153ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89149e9640 1 -- 172.21.15.62:0/696127725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a1528370 msgr2=0x55c9a159db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89149e9640 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a1528370 0x55c9a159db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89149e9640 1 -- 172.21.15.62:0/696127725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9a159e1a0 con 0x55c9a151d5d0 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f89149e9640 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a153a6d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f890002a940 tx=0x7f8900005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc3def669b6f8fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.188+0000 7f88f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f88f8ff9640 1 -- 172.21.15.62:0/696127725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900035040 con 0x55c9a151d5d0 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f88f8ff9640 1 -- 172.21.15.62:0/696127725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f89000053c0 con 0x55c9a151d5d0 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f8919827740 1 -- 172.21.15.62:0/696127725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f890005b070 con 0x55c9a151d5d0 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f88f8ff9640 1 -- 172.21.15.62:0/696127725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f89000055c0 con 0x55c9a151d5d0 2024-09-12T00:06:26.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f8919827740 10 client.17952 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f8919827740 1 -- 172.21.15.62:0/696127725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c9a15a6680 con 0x55c9a151d5d0 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f88f8ff9640 1 -- 172.21.15.62:0/696127725 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890005b070 con 0x55c9a151d5d0 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f88f8ff9640 1 -- 172.21.15.62:0/696127725 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f8900056470 con 0x55c9a151d5d0 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f8919827740 10 client.17952 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f8919827740 10 client.17952 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f8919827740 20 client.17952 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f8919827740 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55c9a15a6d00 0x55c9a15a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f8919827740 4 client.17952 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.189+0000 7f8919827740 1 -- 172.21.15.62:0/696127725 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9a148ad70 con 0x55c9a15a6d00 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f89151ea640 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55c9a15a6d00 0x55c9a15a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f89151ea640 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55c9a15a6d00 0x55c9a15a9120 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d10692dac930be4 server_cookie=fc4440da58ee638f in_seq=0 out_seq=0 2024-09-12T00:06:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f88f8ff9640 10 client.17952 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f88f8ff9640 1 -- 172.21.15.62:0/696127725 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9a148ad70 con 0x55c9a15a6d00 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f88f8ff9640 10 client.17952 handle_command_reply: tid=0 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f8919827740 1 client.17952 shutdown 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f8919827740 1 -- 172.21.15.62:0/696127725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 msgr2=0x55c9a153a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f8919827740 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a153a6d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f890002a940 tx=0x7f8900005320 comp rx=0 tx=0).stop 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f8919827740 1 -- 172.21.15.62:0/696127725 shutdown_connections 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f8919827740 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55c9a15a6d00 0x55c9a15a9120 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f8919827740 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a1528370 0x55c9a159db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f8919827740 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9a151eb40 0x55c9a153ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f8919827740 1 --2- 172.21.15.62:0/696127725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9a151d5d0 0x55c9a153a6d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f8919827740 1 -- 172.21.15.62:0/696127725 >> 172.21.15.62:0/696127725 conn(0x55c9a14e6310 msgr2=0x55c9a15234e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.190+0000 7f88f8ff9640 0 client.17952 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:06:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f8919827740 1 -- 172.21.15.62:0/696127725 shutdown_connections 2024-09-12T00:06:26.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f8919827740 1 -- 172.21.15.62:0/696127725 wait complete. 2024-09-12T00:06:26.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f8919827740 20 client.17952 trim_cache size 0 max 16384 2024-09-12T00:06:26.195 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:06:26.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 -- 172.21.15.62:0/869881988 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88fc0a2d70 msgr2=0x7f88fc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88fc0a2d70 0x7f88fc0a5230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f890809aea0 tx=0x7f890800f300 comp rx=0 tx=0).stop 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 -- 172.21.15.62:0/869881988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8910106f50 msgr2=0x7f891019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8910106f50 0x7f891019cd80 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f890800b970 tx=0x7f890800bd70 comp rx=0 tx=0).stop 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 -- 172.21.15.62:0/869881988 shutdown_connections 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f88fc0a2d70 0x7f88fc0a5230 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8910106f50 0x7f891019cd80 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8910100d00 0x7f89101989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 --2- 172.21.15.62:0/869881988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8910100350 0x7f89101984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 -- 172.21.15.62:0/869881988 >> 172.21.15.62:0/869881988 conn(0x7f8910077210 msgr2=0x7f8910105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 -- 172.21.15.62:0/869881988 shutdown_connections 2024-09-12T00:06:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:26.191+0000 7f891744e640 1 -- 172.21.15.62:0/869881988 wait complete. 2024-09-12T00:06:26.920 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:06:27.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:26 smithi120 ceph-mon[35182]: pgmap v447: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 236 op/s 2024-09-12T00:06:27.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:26 smithi083 ceph-mon[34805]: pgmap v447: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 236 op/s 2024-09-12T00:06:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:27.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:26 smithi062 ceph-mon[30958]: pgmap v447: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 236 op/s 2024-09-12T00:06:27.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 825.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:06:27.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 -- 172.21.15.62:0/2434242359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1b0102790 msgr2=0x7fa1b01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:27.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 --2- 172.21.15.62:0/2434242359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1b0102790 0x7fa1b01067e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa198009900 tx=0x7fa19802a410 comp rx=0 tx=0).stop 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 -- 172.21.15.62:0/2434242359 shutdown_connections 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 --2- 172.21.15.62:0/2434242359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0106f10 0x7fa1b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 --2- 172.21.15.62:0/2434242359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1b0102790 0x7fa1b01067e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 --2- 172.21.15.62:0/2434242359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b0101de0 0x7fa1b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 -- 172.21.15.62:0/2434242359 >> 172.21.15.62:0/2434242359 conn(0x7fa1b00fd8c0 msgr2=0x7fa1b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 -- 172.21.15.62:0/2434242359 shutdown_connections 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 -- 172.21.15.62:0/2434242359 wait complete. 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 Processor -- start 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.672+0000 7fa1b649e640 1 -- start start 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b649e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101de0 0x7fa1b0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b649e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b0102790 0x7fa1b0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b649e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1b0106f10 0x7fa1b019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b649e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0111f90 con 0x7fa1b0106f10 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b649e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0111e10 con 0x7fa1b0102790 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b649e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0112110 con 0x7fa1b0101de0 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b4a3b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1b0106f10 0x7fa1b019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b4a3b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1b0106f10 0x7fa1b019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55264/0 (socket says 172.21.15.62:55264) 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b4a3b640 1 -- 172.21.15.62:0/943520152 learned_addr learned my addr 172.21.15.62:0/943520152 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1affff640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101de0 0x7fa1b0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b0102790 0x7fa1b0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b4a3b640 1 -- 172.21.15.62:0/943520152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101de0 msgr2=0x7fa1b0198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:27.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b4a3b640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101de0 0x7fa1b0198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b4a3b640 1 -- 172.21.15.62:0/943520152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b0102790 msgr2=0x7fa1b0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:27.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b4a3b640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b0102790 0x7fa1b0198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b4a3b640 1 -- 172.21.15.62:0/943520152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b019d520 con 0x7fa1b0106f10 2024-09-12T00:06:27.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1affff640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101de0 0x7fa1b0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:06:27.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1b4a3b640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1b0106f10 0x7fa1b019ce40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800eaa0 tx=0x7fa1a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:27.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.673+0000 7fa1adffb640 1 -- 172.21.15.62:0/943520152 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800cdb0 con 0x7fa1b0106f10 2024-09-12T00:06:27.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.674+0000 7fa1b649e640 1 -- 172.21.15.62:0/943520152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b019d810 con 0x7fa1b0106f10 2024-09-12T00:06:27.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.674+0000 7fa1adffb640 1 -- 172.21.15.62:0/943520152 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a8005020 con 0x7fa1b0106f10 2024-09-12T00:06:27.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.674+0000 7fa1adffb640 1 -- 172.21.15.62:0/943520152 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80116b0 con 0x7fa1b0106f10 2024-09-12T00:06:27.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.674+0000 7fa1b649e640 1 -- 172.21.15.62:0/943520152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b01a50f0 con 0x7fa1b0106f10 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.676+0000 7fa1adffb640 1 -- 172.21.15.62:0/943520152 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa1a8005560 con 0x7fa1b0106f10 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.676+0000 7fa1b8877740 1 Processor -- start 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.676+0000 7fa1b8877740 1 -- start start 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.676+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 0x56404ae82090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.676+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae825d0 0x56404ae82a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 0x56404ae8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1b8877740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56404ade82f0 con 0x56404ae825d0 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1b8877740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56404ad8fcf0 con 0x56404ae8d370 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1b8877740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56404ade42b0 con 0x56404ae83b40 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1affff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae825d0 0x56404ae82a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1affff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae825d0 0x56404ae82a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55278/0 (socket says 172.21.15.62:55278) 2024-09-12T00:06:27.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1affff640 1 -- 172.21.15.62:0/1633742428 learned_addr learned my addr 172.21.15.62:0/1633742428 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:27.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/1633742428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 0x56404ae82090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1adffb640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa19c0a2cf0 0x7fa19c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/1633742428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 0x56404ae8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1affff640 1 -- 172.21.15.62:0/1633742428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 msgr2=0x56404ae82090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:27.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1affff640 1 --2- 172.21.15.62:0/1633742428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 0x56404ae82090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1affff640 1 -- 172.21.15.62:0/1633742428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 msgr2=0x56404ae8f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:27.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1affff640 1 --2- 172.21.15.62:0/1633742428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 0x56404ae8f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1b4a3b640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa19c0a2cf0 0x7fa19c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1affff640 1 -- 172.21.15.62:0/1633742428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56404ad743a0 con 0x56404ae825d0 2024-09-12T00:06:27.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/1633742428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 0x56404ae8f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:27.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1adffb640 1 -- 172.21.15.62:0/943520152 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fa1a80cf780 con 0x7fa1b0106f10 2024-09-12T00:06:27.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1affff640 1 --2- 172.21.15.62:0/1633742428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae825d0 0x56404ae82a10 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0007850 tx=0x7fa1a0007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc83747102bfe8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:27.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.677+0000 7fa1a67fc640 1 -- 172.21.15.62:0/1633742428 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000d950 con 0x56404ae825d0 2024-09-12T00:06:27.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1a67fc640 1 -- 172.21.15.62:0/1633742428 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0018070 con 0x56404ae825d0 2024-09-12T00:06:27.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1b4a3b640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa19c0a2cf0 0x7fa19c0a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009270 tx=0x7fa1a806bb00 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:27.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1a67fc640 1 -- 172.21.15.62:0/1633742428 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00123e0 con 0x56404ae825d0 2024-09-12T00:06:27.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1b8877740 1 -- 172.21.15.62:0/1633742428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae825d0 msgr2=0x56404ae82a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:27.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1b8877740 1 --2- 172.21.15.62:0/1633742428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae825d0 0x56404ae82a10 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0007850 tx=0x7fa1a0007d10 comp rx=0 tx=0).stop 2024-09-12T00:06:27.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1b8877740 1 -- 172.21.15.62:0/1633742428 shutdown_connections 2024-09-12T00:06:27.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1b8877740 1 --2- 172.21.15.62:0/1633742428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 0x56404ae8f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1b8877740 1 --2- 172.21.15.62:0/1633742428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae825d0 0x56404ae82a10 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1b8877740 1 --2- 172.21.15.62:0/1633742428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 0x56404ae82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1b8877740 1 -- 172.21.15.62:0/1633742428 >> 172.21.15.62:0/1633742428 conn(0x56404ae4b310 msgr2=0x56404ae7ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:27.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1b8877740 1 -- 172.21.15.62:0/1633742428 shutdown_connections 2024-09-12T00:06:27.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.678+0000 7fa1b8877740 1 -- 172.21.15.62:0/1633742428 wait complete. 2024-09-12T00:06:27.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1b8877740 1 -- start start 2024-09-12T00:06:27.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1b8877740 1 Processor -- start 2024-09-12T00:06:27.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae825d0 0x56404ae8cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 0x56404ae8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae8a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1b8877740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56404ae90070 con 0x56404ae8d370 2024-09-12T00:06:27.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1b8877740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56404ada3060 con 0x56404ae825d0 2024-09-12T00:06:27.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1b8877740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56404ae901f0 con 0x56404ae83b40 2024-09-12T00:06:27.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1b8877740 10 client.0 _ll_register_callbacks cb 0x56404aaf68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:27.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae8a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae8a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55288/0 (socket says 172.21.15.62:55288) 2024-09-12T00:06:27.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1a7fff640 1 -- 172.21.15.62:0/389607715 learned_addr learned my addr 172.21.15.62:0/389607715 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:27.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae825d0 0x56404ae8cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.679+0000 7fa1affff640 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 0x56404ae8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa1a7fff640 1 -- 172.21.15.62:0/389607715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 msgr2=0x56404ae8a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:27.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 0x56404ae8a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa1a7fff640 1 -- 172.21.15.62:0/389607715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae825d0 msgr2=0x56404ae8cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:27.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae825d0 0x56404ae8cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa1a7fff640 1 -- 172.21.15.62:0/389607715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56404ab593c0 con 0x56404ae8d370 2024-09-12T00:06:27.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa1affff640 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 0x56404ae8a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:06:27.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae8a640 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa198031550 tx=0x7fa198009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d98805757eebdd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:27.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa18f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:27.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa18f7fe640 1 -- 172.21.15.62:0/389607715 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198035040 con 0x56404ae8d370 2024-09-12T00:06:27.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa18f7fe640 1 -- 172.21.15.62:0/389607715 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fa198031810 con 0x56404ae8d370 2024-09-12T00:06:27.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa1b8877740 1 -- 172.21.15.62:0/389607715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a0018070 con 0x56404ae8d370 2024-09-12T00:06:27.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa18f7fe640 1 -- 172.21.15.62:0/389607715 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa198031a10 con 0x56404ae8d370 2024-09-12T00:06:27.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa1b8877740 10 client.17964 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:27.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa1b8877740 1 -- 172.21.15.62:0/389607715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56404ae9bab0 con 0x56404ae8d370 2024-09-12T00:06:27.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.680+0000 7fa18f7fe640 1 -- 172.21.15.62:0/389607715 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0018070 con 0x56404ae8d370 2024-09-12T00:06:27.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.681+0000 7fa18f7fe640 1 -- 172.21.15.62:0/389607715 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa19803a030 con 0x56404ae8d370 2024-09-12T00:06:27.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.681+0000 7fa1b8877740 10 client.17964 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:27.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.681+0000 7fa1b8877740 10 client.17964 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:06:27.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.681+0000 7fa1b8877740 20 client.17964 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:27.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.681+0000 7fa1b8877740 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56404ae9cf30 0x56404ae9f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.681+0000 7fa1b8877740 4 client.17964 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:06:27.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.681+0000 7fa1b8877740 1 -- 172.21.15.62:0/389607715 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56404adf5860 con 0x56404ae9cf30 2024-09-12T00:06:27.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.681+0000 7fa1affff640 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56404ae9cf30 0x56404ae9f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.681+0000 7fa18f7fe640 10 client.17964 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:06:27.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.681+0000 7fa1affff640 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56404ae9cf30 0x56404ae9f360 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1b2d556c0dfa18e server_cookie=fb330bb3bbc18cf in_seq=0 out_seq=0 2024-09-12T00:06:27.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa18f7fe640 1 -- 172.21.15.62:0/389607715 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56404adf5860 con 0x56404ae9cf30 2024-09-12T00:06:27.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa18f7fe640 10 client.17964 handle_command_reply: tid=0 2024-09-12T00:06:27.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa1b8877740 1 client.17964 shutdown 2024-09-12T00:06:27.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa1b8877740 1 -- 172.21.15.62:0/389607715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 msgr2=0x56404ae8a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:27.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa1b8877740 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae8a640 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa198031550 tx=0x7fa198009260 comp rx=0 tx=0).stop 2024-09-12T00:06:27.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa1b8877740 1 -- 172.21.15.62:0/389607715 shutdown_connections 2024-09-12T00:06:27.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa1b8877740 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56404ae9cf30 0x56404ae9f360 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa1b8877740 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae8a640 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa1b8877740 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae83b40 0x56404ae8a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa18f7fe640 0 client.17964 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:06:27.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa1b8877740 1 --2- 172.21.15.62:0/389607715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae825d0 0x56404ae8cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.682+0000 7fa1b8877740 1 -- 172.21.15.62:0/389607715 >> 172.21.15.62:0/389607715 conn(0x56404ae4b310 msgr2=0x56404ae84960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:27.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.683+0000 7fa1b8877740 1 -- 172.21.15.62:0/389607715 shutdown_connections 2024-09-12T00:06:27.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.683+0000 7fa1b8877740 1 -- 172.21.15.62:0/389607715 wait complete. 2024-09-12T00:06:27.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.683+0000 7fa1b8877740 20 client.17964 trim_cache size 0 max 16384 2024-09-12T00:06:27.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.695+0000 7fa1b8877740 1 Processor -- start 2024-09-12T00:06:27.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.695+0000 7fa1b8877740 1 -- start start 2024-09-12T00:06:27.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae825d0 0x56404ae8bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae83b40 0x56404ae85120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae85660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1b8877740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b0230 con 0x56404ae8d370 2024-09-12T00:06:27.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1b8877740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01adb00 con 0x56404ae83b40 2024-09-12T00:06:27.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1b8877740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01ab970 con 0x56404ae825d0 2024-09-12T00:06:27.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae85660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae85660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55298/0 (socket says 172.21.15.62:55298) 2024-09-12T00:06:27.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1a7fff640 1 -- 172.21.15.62:0/2720089466 learned_addr learned my addr 172.21.15.62:0/2720089466 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:27.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1affff640 1 --2- 172.21.15.62:0/2720089466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae83b40 0x56404ae85120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/2720089466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae825d0 0x56404ae8bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1a7fff640 1 -- 172.21.15.62:0/2720089466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae825d0 msgr2=0x56404ae8bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/2720089466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae825d0 0x56404ae8bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1a7fff640 1 -- 172.21.15.62:0/2720089466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae83b40 msgr2=0x56404ae85120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/2720089466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae83b40 0x56404ae85120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1a7fff640 1 -- 172.21.15.62:0/2720089466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56404ae7e4f0 con 0x56404ae8d370 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1affff640 1 --2- 172.21.15.62:0/2720089466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae83b40 0x56404ae85120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.696+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/2720089466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae85660 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa198005c20 tx=0x7fa19802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4244cc9c5c9843aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1a5ffb640 1 -- 172.21.15.62:0/2720089466 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198058070 con 0x56404ae8d370 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1a5ffb640 1 -- 172.21.15.62:0/2720089466 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa198003ad0 con 0x56404ae8d370 2024-09-12T00:06:27.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1a5ffb640 1 -- 172.21.15.62:0/2720089466 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198035040 con 0x56404ae8d370 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1b8877740 1 -- 172.21.15.62:0/2720089466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 msgr2=0x56404ae85660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1b8877740 1 --2- 172.21.15.62:0/2720089466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae85660 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa198005c20 tx=0x7fa19802a410 comp rx=0 tx=0).stop 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1b8877740 1 -- 172.21.15.62:0/2720089466 shutdown_connections 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1b8877740 1 --2- 172.21.15.62:0/2720089466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae8d370 0x56404ae85660 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1b8877740 1 --2- 172.21.15.62:0/2720089466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae83b40 0x56404ae85120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1b8877740 1 --2- 172.21.15.62:0/2720089466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae825d0 0x56404ae8bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1b8877740 1 -- 172.21.15.62:0/2720089466 >> 172.21.15.62:0/2720089466 conn(0x56404ae4b310 msgr2=0x56404ae8b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1b8877740 1 -- 172.21.15.62:0/2720089466 shutdown_connections 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1b8877740 1 -- 172.21.15.62:0/2720089466 wait complete. 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.697+0000 7fa1b8877740 1 -- start start 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1b8877740 1 Processor -- start 2024-09-12T00:06:27.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae825d0 0x56404ae95880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae83b40 0x56404ae86a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1b8877740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 0x56404ae86f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1b8877740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01abfc0 con 0x56404ae83b40 2024-09-12T00:06:27.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1b8877740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01ad430 con 0x56404ae8d370 2024-09-12T00:06:27.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1b8877740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01ac690 con 0x56404ae825d0 2024-09-12T00:06:27.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1b8877740 10 client.0 _ll_register_callbacks cb 0x7fa19803d8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:27.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1affff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae83b40 0x56404ae86a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1affff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae83b40 0x56404ae86a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55300/0 (socket says 172.21.15.62:55300) 2024-09-12T00:06:27.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1affff640 1 -- 172.21.15.62:0/2243234222 learned_addr learned my addr 172.21.15.62:0/2243234222 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:27.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 0x56404ae86f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae825d0 0x56404ae95880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1affff640 1 -- 172.21.15.62:0/2243234222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae825d0 msgr2=0x56404ae95880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1affff640 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae825d0 0x56404ae95880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1affff640 1 -- 172.21.15.62:0/2243234222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 msgr2=0x56404ae86f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1affff640 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 0x56404ae86f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.698+0000 7fa1affff640 1 -- 172.21.15.62:0/2243234222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56404ae874a0 con 0x56404ae83b40 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa1affff640 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae83b40 0x56404ae86a20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x56404ae7fac0 tx=0x7fa1a0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ec871c49862119 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa18f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa18f7fe640 1 -- 172.21.15.62:0/2243234222 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a002b030 con 0x56404ae83b40 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa18f7fe640 1 -- 172.21.15.62:0/2243234222 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fa1a0007420 con 0x56404ae83b40 2024-09-12T00:06:27.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa1b8877740 1 -- 172.21.15.62:0/2243234222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa198003ad0 con 0x56404ae83b40 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa18f7fe640 1 -- 172.21.15.62:0/2243234222 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa1a00136a0 con 0x56404ae83b40 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa1b8877740 10 client.17970 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa1b8877740 1 -- 172.21.15.62:0/2243234222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56404af07870 con 0x56404ae83b40 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa18f7fe640 1 -- 172.21.15.62:0/2243234222 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa198003ad0 con 0x56404ae83b40 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa18f7fe640 1 -- 172.21.15.62:0/2243234222 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa1a0027750 con 0x56404ae83b40 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa1b8877740 10 client.17970 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa1b8877740 10 client.17970 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa1b8877740 20 client.17970 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa1b8877740 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56404af07cc0 0x56404af0a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa1b8877740 4 client.17970 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:06:27.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.699+0000 7fa1b8877740 1 -- 172.21.15.62:0/2243234222 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56404adefd70 con 0x56404af07cc0 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.700+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56404af07cc0 0x56404af0a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.700+0000 7fa18f7fe640 10 client.17970 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.700+0000 7fa1a7fff640 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56404af07cc0 0x56404af0a0e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be3500ab1aacc45f server_cookie=6587b4dc4ac88bfb in_seq=0 out_seq=0 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.700+0000 7fa18f7fe640 1 -- 172.21.15.62:0/2243234222 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56404adefd70 con 0x56404af07cc0 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.700+0000 7fa18f7fe640 10 client.17970 handle_command_reply: tid=0 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.700+0000 7fa1b8877740 1 client.17970 shutdown 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.700+0000 7fa1b8877740 1 -- 172.21.15.62:0/2243234222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae83b40 msgr2=0x56404ae86a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.700+0000 7fa1b8877740 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae83b40 0x56404ae86a20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x56404ae7fac0 tx=0x7fa1a0007780 comp rx=0 tx=0).stop 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b8877740 1 -- 172.21.15.62:0/2243234222 shutdown_connections 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b8877740 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x56404af07cc0 0x56404af0a0e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b8877740 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56404ae8d370 0x56404ae86f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b8877740 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56404ae83b40 0x56404ae86a20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b8877740 1 --2- 172.21.15.62:0/2243234222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56404ae825d0 0x56404ae95880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b8877740 1 -- 172.21.15.62:0/2243234222 >> 172.21.15.62:0/2243234222 conn(0x56404ae4b310 msgr2=0x56404ae8c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:27.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa18f7fe640 0 client.17970 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:06:27.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b8877740 1 -- 172.21.15.62:0/2243234222 shutdown_connections 2024-09-12T00:06:27.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b8877740 1 -- 172.21.15.62:0/2243234222 wait complete. 2024-09-12T00:06:27.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b8877740 20 client.17970 trim_cache size 0 max 16384 2024-09-12T00:06:27.707 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:06:27.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 -- 172.21.15.62:0/943520152 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa19c0a2cf0 msgr2=0x7fa19c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:27.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa19c0a2cf0 0x7fa19c0a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009270 tx=0x7fa1a806bb00 comp rx=0 tx=0).stop 2024-09-12T00:06:27.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 -- 172.21.15.62:0/943520152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1b0106f10 msgr2=0x7fa1b019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1b0106f10 0x7fa1b019ce40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800eaa0 tx=0x7fa1a800ef60 comp rx=0 tx=0).stop 2024-09-12T00:06:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 -- 172.21.15.62:0/943520152 shutdown_connections 2024-09-12T00:06:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fa19c0a2cf0 0x7fa19c0a51b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1b0106f10 0x7fa1b019ce40 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b0102790 0x7fa1b0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 --2- 172.21.15.62:0/943520152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101de0 0x7fa1b0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 -- 172.21.15.62:0/943520152 >> 172.21.15.62:0/943520152 conn(0x7fa1b00fd8c0 msgr2=0x7fa1b00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 -- 172.21.15.62:0/943520152 shutdown_connections 2024-09-12T00:06:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:27.701+0000 7fa1b649e640 1 -- 172.21.15.62:0/943520152 wait complete. 2024-09-12T00:06:28.419 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:06:28.674 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:29.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:28 smithi062 ceph-mon[30958]: pgmap v448: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-12T00:06:29.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:28 smithi120 ceph-mon[35182]: pgmap v448: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-12T00:06:29.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.140+0000 7f029052b640 1 -- 172.21.15.62:0/1210306596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0288106f40 msgr2=0x7f028810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.140+0000 7f029052b640 1 --2- 172.21.15.62:0/1210306596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0288106f40 0x7f028810eac0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f0280009ca0 tx=0x7f028002a8c0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.140+0000 7f029052b640 1 -- 172.21.15.62:0/1210306596 shutdown_connections 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.140+0000 7f029052b640 1 --2- 172.21.15.62:0/1210306596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0288106f40 0x7f028810eac0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.140+0000 7f029052b640 1 --2- 172.21.15.62:0/1210306596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02881027c0 0x7f0288106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.140+0000 7f029052b640 1 --2- 172.21.15.62:0/1210306596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0288101e10 0x7f02881021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.140+0000 7f029052b640 1 -- 172.21.15.62:0/1210306596 >> 172.21.15.62:0/1210306596 conn(0x7f02880fd8d0 msgr2=0x7f02880ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.140+0000 7f029052b640 1 -- 172.21.15.62:0/1210306596 shutdown_connections 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.140+0000 7f029052b640 1 -- 172.21.15.62:0/1210306596 wait complete. 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f029052b640 1 Processor -- start 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f029052b640 1 -- start start 2024-09-12T00:06:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f029052b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101e10 0x7f02881983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f029052b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02881027c0 0x7f02881988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f029052b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0288106f40 0x7f028819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f029052b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0288111fa0 con 0x7f0288106f40 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f029052b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0288111e20 con 0x7f02881027c0 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f029052b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0288112120 con 0x7f0288101e10 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f028dac6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02881027c0 0x7f02881988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f028e2c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101e10 0x7f02881983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f028dac6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02881027c0 0x7f02881988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57268/0 (socket says 172.21.15.62:57268) 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f028dac6640 1 -- 172.21.15.62:0/4062442029 learned_addr learned my addr 172.21.15.62:0/4062442029 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f028eac8640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0288106f40 0x7f028819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.141+0000 7f028e2c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101e10 0x7f02881983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:60390/0 (socket says 172.21.15.62:60390) 2024-09-12T00:06:29.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f028dac6640 1 -- 172.21.15.62:0/4062442029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101e10 msgr2=0x7f02881983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f028dac6640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101e10 0x7f02881983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f028dac6640 1 -- 172.21.15.62:0/4062442029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0288106f40 msgr2=0x7f028819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f028dac6640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0288106f40 0x7f028819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f028dac6640 1 -- 172.21.15.62:0/4062442029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028819d360 con 0x7f02881027c0 2024-09-12T00:06:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f028dac6640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02881027c0 0x7f02881988f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9f0 tx=0x7f027c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f027b7fe640 1 -- 172.21.15.62:0/4062442029 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c00cd80 con 0x7f02881027c0 2024-09-12T00:06:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f029052b640 1 -- 172.21.15.62:0/4062442029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028819d650 con 0x7f02881027c0 2024-09-12T00:06:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f027b7fe640 1 -- 172.21.15.62:0/4062442029 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f027c010910 con 0x7f02881027c0 2024-09-12T00:06:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f027b7fe640 1 -- 172.21.15.62:0/4062442029 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0188f0 con 0x7f02881027c0 2024-09-12T00:06:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.142+0000 7f029052b640 1 -- 172.21.15.62:0/4062442029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02881a4f30 con 0x7f02881027c0 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.145+0000 7f0292904740 1 Processor -- start 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.145+0000 7f0292904740 1 -- start start 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.145+0000 7f0292904740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db297d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.145+0000 7f0292904740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db297da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.145+0000 7f0292904740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db298c780 0x559db298eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.145+0000 7f0292904740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559db28e32f0 con 0x559db297eb40 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.145+0000 7f0292904740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559db288acf0 con 0x559db298c780 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.145+0000 7f0292904740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559db28df2b0 con 0x559db297d5d0 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028eac8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db297d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f027b7fe640 1 -- 172.21.15.62:0/4062442029 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f027c010ac0 con 0x7f02881027c0 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028eac8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db297d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55312/0 (socket says 172.21.15.62:55312) 2024-09-12T00:06:29.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028eac8640 1 -- 172.21.15.62:0/3203702379 learned_addr learned my addr 172.21.15.62:0/3203702379 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028e2c7640 1 --2- 172.21.15.62:0/3203702379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db297da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028dac6640 1 --2- 172.21.15.62:0/3203702379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db298c780 0x559db298eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028eac8640 1 -- 172.21.15.62:0/3203702379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 msgr2=0x559db297da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028eac8640 1 --2- 172.21.15.62:0/3203702379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db297da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028eac8640 1 -- 172.21.15.62:0/3203702379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db298c780 msgr2=0x559db298eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028eac8640 1 --2- 172.21.15.62:0/3203702379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db298c780 0x559db298eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028eac8640 1 -- 172.21.15.62:0/3203702379 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db286f3a0 con 0x559db297eb40 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028e2c7640 1 --2- 172.21.15.62:0/3203702379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db297da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.146+0000 7f028eac8640 1 --2- 172.21.15.62:0/3203702379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db297d090 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f0280009ca0 tx=0x7f0280005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804dc209e0c74265 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.147+0000 7f0278ff9640 1 -- 172.21.15.62:0/3203702379 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280031db0 con 0x559db297eb40 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.147+0000 7f0278ff9640 1 -- 172.21.15.62:0/3203702379 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0280036040 con 0x559db297eb40 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.147+0000 7f0278ff9640 1 -- 172.21.15.62:0/3203702379 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02800042f0 con 0x559db297eb40 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.147+0000 7f027b7fe640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02740a2cf0 0x7f02740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.147+0000 7f028e2c7640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02740a2cf0 0x7f02740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.147+0000 7f0292904740 1 -- 172.21.15.62:0/3203702379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 msgr2=0x559db297d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.147+0000 7f0292904740 1 --2- 172.21.15.62:0/3203702379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db297d090 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f0280009ca0 tx=0x7f0280005f10 comp rx=0 tx=0).stop 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.147+0000 7f027b7fe640 1 -- 172.21.15.62:0/4062442029 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f027c0cf7f0 con 0x7f02881027c0 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 -- 172.21.15.62:0/3203702379 shutdown_connections 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 --2- 172.21.15.62:0/3203702379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db298c780 0x559db298eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 --2- 172.21.15.62:0/3203702379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db297da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 --2- 172.21.15.62:0/3203702379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db297d090 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 -- 172.21.15.62:0/3203702379 >> 172.21.15.62:0/3203702379 conn(0x559db2946310 msgr2=0x559db2979c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 -- 172.21.15.62:0/3203702379 shutdown_connections 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 -- 172.21.15.62:0/3203702379 wait complete. 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f028e2c7640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02740a2cf0 0x7f02740a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x559db297aac0 tx=0x7f0284005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 -- start start 2024-09-12T00:06:29.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 Processor -- start 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db298bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.148+0000 7f0292904740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db2984fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f0292904740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db298c780 0x559db2985510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f0292904740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559db298f2a0 con 0x559db297eb40 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f0292904740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559db289e060 con 0x559db298c780 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f0292904740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559db298f420 con 0x559db297d5d0 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f0292904740 10 client.0 _ll_register_callbacks cb 0x559db25f18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028dac6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db2984fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028dac6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db2984fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55318/0 (socket says 172.21.15.62:55318) 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028dac6640 1 -- 172.21.15.62:0/25881908 learned_addr learned my addr 172.21.15.62:0/25881908 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028e2c7640 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db298bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028eac8640 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db298c780 0x559db2985510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028dac6640 1 -- 172.21.15.62:0/25881908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 msgr2=0x559db298bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028dac6640 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db298bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028dac6640 1 -- 172.21.15.62:0/25881908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db298c780 msgr2=0x559db2985510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028dac6640 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db298c780 0x559db2985510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028dac6640 1 -- 172.21.15.62:0/25881908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db26543c0 con 0x559db297eb40 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028e2c7640 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db298bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f028dac6640 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db2984fd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f027c0ccf40 tx=0x7f027c09b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60987047edc234f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.149+0000 7f0271ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0271ffb640 1 -- 172.21.15.62:0/25881908 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0cd370 con 0x559db297eb40 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0271ffb640 1 -- 172.21.15.62:0/25881908 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f027c0cfde0 con 0x559db297eb40 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0292904740 1 -- 172.21.15.62:0/25881908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0280036040 con 0x559db297eb40 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0271ffb640 1 -- 172.21.15.62:0/25881908 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f027c0a0060 con 0x559db297eb40 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0292904740 10 client.17982 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:29.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0292904740 1 -- 172.21.15.62:0/25881908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559db299f1c0 con 0x559db297eb40 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0271ffb640 1 -- 172.21.15.62:0/25881908 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0280036040 con 0x559db297eb40 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0271ffb640 1 -- 172.21.15.62:0/25881908 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f027c097cb0 con 0x559db297eb40 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0292904740 10 client.17982 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0292904740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0292904740 20 client.17982 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0292904740 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559db29a0680 0x559db29a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0292904740 4 client.17982 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.150+0000 7f0292904740 1 -- 172.21.15.62:0/25881908 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559db28f0860 con 0x559db29a0680 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.151+0000 7f028e2c7640 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559db29a0680 0x559db29a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.151+0000 7f0271ffb640 10 client.17982 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.151+0000 7f028e2c7640 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559db29a0680 0x559db29a2a70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b27e05161524238 server_cookie=f4e9bea068210940 in_seq=0 out_seq=0 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0271ffb640 1 -- 172.21.15.62:0/25881908 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559db28f0860 con 0x559db29a0680 2024-09-12T00:06:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0271ffb640 10 client.17982 handle_command_reply: tid=0 2024-09-12T00:06:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 client.17982 shutdown 2024-09-12T00:06:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 -- 172.21.15.62:0/25881908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 msgr2=0x559db2984fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db2984fd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f027c0ccf40 tx=0x7f027c09b990 comp rx=0 tx=0).stop 2024-09-12T00:06:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 -- 172.21.15.62:0/25881908 shutdown_connections 2024-09-12T00:06:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559db29a0680 0x559db29a2a70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db298c780 0x559db2985510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0271ffb640 0 client.17982 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db297eb40 0x559db2984fd0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 --2- 172.21.15.62:0/25881908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db298bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 -- 172.21.15.62:0/25881908 >> 172.21.15.62:0/25881908 conn(0x559db2946310 msgr2=0x559db297f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:29.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 -- 172.21.15.62:0/25881908 shutdown_connections 2024-09-12T00:06:29.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 1 -- 172.21.15.62:0/25881908 wait complete. 2024-09-12T00:06:29.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.152+0000 7f0292904740 20 client.17982 trim_cache size 0 max 16384 2024-09-12T00:06:29.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.166+0000 7f0292904740 1 Processor -- start 2024-09-12T00:06:29.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.166+0000 7f0292904740 1 -- start start 2024-09-12T00:06:29.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.166+0000 7f0292904740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db2974c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.166+0000 7f0292904740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297eb40 0x559db2980120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.166+0000 7f0292904740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2980660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:28 smithi083 ceph-mon[34805]: pgmap v448: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.166+0000 7f0292904740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f02881ade50 con 0x559db298c780 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.166+0000 7f0292904740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02881ab720 con 0x559db297eb40 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.166+0000 7f0292904740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02881a9590 con 0x559db297d5d0 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028eac8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2980660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028eac8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2980660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55324/0 (socket says 172.21.15.62:55324) 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028eac8640 1 -- 172.21.15.62:0/3646731833 learned_addr learned my addr 172.21.15.62:0/3646731833 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028dac6640 1 --2- 172.21.15.62:0/3646731833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297eb40 0x559db2980120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028e2c7640 1 --2- 172.21.15.62:0/3646731833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db2974c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028eac8640 1 -- 172.21.15.62:0/3646731833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 msgr2=0x559db2974c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028eac8640 1 --2- 172.21.15.62:0/3646731833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db2974c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028eac8640 1 -- 172.21.15.62:0/3646731833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297eb40 msgr2=0x559db2980120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028eac8640 1 --2- 172.21.15.62:0/3646731833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297eb40 0x559db2980120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028eac8640 1 -- 172.21.15.62:0/3646731833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db2983f80 con 0x559db298c780 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028dac6640 1 --2- 172.21.15.62:0/3646731833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297eb40 0x559db2980120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f028eac8640 1 --2- 172.21.15.62:0/3646731833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2980660 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0280009ca0 tx=0x7f0280000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b3d167b62315ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.167+0000 7f0273fff640 1 -- 172.21.15.62:0/3646731833 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280052070 con 0x559db298c780 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0273fff640 1 -- 172.21.15.62:0/3646731833 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0280036040 con 0x559db298c780 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0273fff640 1 -- 172.21.15.62:0/3646731833 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280004370 con 0x559db298c780 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0292904740 1 -- 172.21.15.62:0/3646731833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 msgr2=0x559db2980660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0292904740 1 --2- 172.21.15.62:0/3646731833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2980660 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0280009ca0 tx=0x7f0280000f40 comp rx=0 tx=0).stop 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0292904740 1 -- 172.21.15.62:0/3646731833 shutdown_connections 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0292904740 1 --2- 172.21.15.62:0/3646731833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2980660 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0292904740 1 --2- 172.21.15.62:0/3646731833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297eb40 0x559db2980120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0292904740 1 --2- 172.21.15.62:0/3646731833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297d5d0 0x559db2974c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0292904740 1 -- 172.21.15.62:0/3646731833 >> 172.21.15.62:0/3646731833 conn(0x559db2946310 msgr2=0x559db2989060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0292904740 1 -- 172.21.15.62:0/3646731833 shutdown_connections 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.168+0000 7f0292904740 1 -- 172.21.15.62:0/3646731833 wait complete. 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f0292904740 1 -- start start 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f0292904740 1 Processor -- start 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f0292904740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297d5d0 0x559db299d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f0292904740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297eb40 0x559db2998cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f0292904740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2999440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f0292904740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f02881a9be0 con 0x559db298c780 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f0292904740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02881ab050 con 0x559db297d5d0 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f0292904740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02881aa2b0 con 0x559db297eb40 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f0292904740 10 client.0 _ll_register_callbacks cb 0x7f027c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f028eac8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2999440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f028e2c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297d5d0 0x559db299d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f028e2c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297d5d0 0x559db299d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:57306/0 (socket says 172.21.15.62:57306) 2024-09-12T00:06:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f028e2c7640 1 -- 172.21.15.62:0/2984973131 learned_addr learned my addr 172.21.15.62:0/2984973131 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f028eac8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2999440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55332/0 (socket says 172.21.15.62:55332) 2024-09-12T00:06:29.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.169+0000 7f028dac6640 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297eb40 0x559db2998cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f028e2c7640 1 -- 172.21.15.62:0/2984973131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297eb40 msgr2=0x559db2998cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f028e2c7640 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297eb40 0x559db2998cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f028e2c7640 1 -- 172.21.15.62:0/2984973131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 msgr2=0x559db2999440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:29.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f028e2c7640 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2999440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f028e2c7640 1 -- 172.21.15.62:0/2984973131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db298c0b0 con 0x559db297d5d0 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f028eac8640 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2999440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f028e2c7640 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297d5d0 0x559db299d980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f02840037c0 tx=0x7f02840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=112f405925967801 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f0271ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f0271ffb640 1 -- 172.21.15.62:0/2984973131 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284005020 con 0x559db297d5d0 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f0292904740 1 -- 172.21.15.62:0/2984973131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0280036040 con 0x559db297d5d0 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f0271ffb640 1 -- 172.21.15.62:0/2984973131 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f02840051c0 con 0x559db297d5d0 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f0271ffb640 1 -- 172.21.15.62:0/2984973131 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f02840053c0 con 0x559db297d5d0 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f0292904740 10 client.27449 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.170+0000 7f0292904740 1 -- 172.21.15.62:0/2984973131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559db2a0b320 con 0x559db297d5d0 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0271ffb640 1 -- 172.21.15.62:0/2984973131 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0280036040 con 0x559db297d5d0 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0271ffb640 1 -- 172.21.15.62:0/2984973131 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f028402b780 con 0x559db297d5d0 2024-09-12T00:06:29.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0292904740 10 client.27449 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0292904740 10 client.27449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0292904740 20 client.27449 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0292904740 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559db29a0680 0x559db2a0da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0292904740 4 client.27449 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0292904740 1 -- 172.21.15.62:0/2984973131 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559db28ead70 con 0x559db29a0680 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f028dac6640 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559db29a0680 0x559db2a0da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f028dac6640 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559db29a0680 0x559db2a0da40 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15e90548130fb6f0 server_cookie=a7f2e296d62b3c77 in_seq=0 out_seq=0 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0271ffb640 10 client.27449 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0271ffb640 1 -- 172.21.15.62:0/2984973131 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559db28ead70 con 0x559db29a0680 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0271ffb640 10 client.27449 handle_command_reply: tid=0 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.171+0000 7f0292904740 1 client.27449 shutdown 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 1 -- 172.21.15.62:0/2984973131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297d5d0 msgr2=0x559db299d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297d5d0 0x559db299d980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f02840037c0 tx=0x7f02840046c0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 1 -- 172.21.15.62:0/2984973131 shutdown_connections 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559db29a0680 0x559db2a0da40 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559db298c780 0x559db2999440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db297eb40 0x559db2998cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 1 --2- 172.21.15.62:0/2984973131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559db297d5d0 0x559db299d980 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 1 -- 172.21.15.62:0/2984973131 >> 172.21.15.62:0/2984973131 conn(0x559db2946310 msgr2=0x559db2984890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0271ffb640 0 client.27449 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 1 -- 172.21.15.62:0/2984973131 shutdown_connections 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 1 -- 172.21.15.62:0/2984973131 wait complete. 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f0292904740 20 client.27449 trim_cache size 0 max 16384 2024-09-12T00:06:29.177 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:06:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f029052b640 1 -- 172.21.15.62:0/4062442029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02740a2cf0 msgr2=0x7f02740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f029052b640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02740a2cf0 0x7f02740a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x559db297aac0 tx=0x7f0284005f90 comp rx=0 tx=0).stop 2024-09-12T00:06:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f029052b640 1 -- 172.21.15.62:0/4062442029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02881027c0 msgr2=0x7f02881988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:29.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.172+0000 7f029052b640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02881027c0 0x7f02881988f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9f0 tx=0x7f027c00eeb0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.173+0000 7f029052b640 1 -- 172.21.15.62:0/4062442029 shutdown_connections 2024-09-12T00:06:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.173+0000 7f029052b640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02740a2cf0 0x7f02740a51b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.173+0000 7f029052b640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0288106f40 0x7f028819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.173+0000 7f029052b640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02881027c0 0x7f02881988f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.173+0000 7f029052b640 1 --2- 172.21.15.62:0/4062442029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0288101e10 0x7f02881983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.173+0000 7f029052b640 1 -- 172.21.15.62:0/4062442029 >> 172.21.15.62:0/4062442029 conn(0x7f02880fd8d0 msgr2=0x7f02880fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.173+0000 7f029052b640 1 -- 172.21.15.62:0/4062442029 shutdown_connections 2024-09-12T00:06:29.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:29.173+0000 7f029052b640 1 -- 172.21.15.62:0/4062442029 wait complete. 2024-09-12T00:06:30.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:29 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:06:30.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:29 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:06:30.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:29 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:06:30.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:29 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:06:30.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:29 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:06:30.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:29 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:06:30.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:29 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:06:30.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:29 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:06:30.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:29 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:06:30.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:29 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:06:30.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:29 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:06:30.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:29 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:06:30.997 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:06:30.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:06:31.269 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:31.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:31 smithi120 ceph-mon[35182]: pgmap v449: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 MiB/s wr, 263 op/s 2024-09-12T00:06:31.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:31 smithi062 ceph-mon[30958]: pgmap v449: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 MiB/s wr, 263 op/s 2024-09-12T00:06:31.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:31 smithi083 ceph-mon[34805]: pgmap v449: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 MiB/s wr, 263 op/s 2024-09-12T00:06:31.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.717+0000 7fbbea7de640 1 -- 172.21.15.62:0/1764470960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe4100e20 msgr2=0x7fbbe4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:31.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.717+0000 7fbbea7de640 1 --2- 172.21.15.62:0/1764470960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe4100e20 0x7fbbe4101200 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009900 tx=0x7fbbd402a480 comp rx=0 tx=0).stop 2024-09-12T00:06:31.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.717+0000 7fbbea7de640 1 -- 172.21.15.62:0/1764470960 shutdown_connections 2024-09-12T00:06:31.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.717+0000 7fbbea7de640 1 --2- 172.21.15.62:0/1764470960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe4102ef0 0x7fbbe410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.717+0000 7fbbea7de640 1 --2- 172.21.15.62:0/1764470960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe4102570 0x7fbbe41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.717+0000 7fbbea7de640 1 --2- 172.21.15.62:0/1764470960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe4100e20 0x7fbbe4101200 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.717+0000 7fbbea7de640 1 -- 172.21.15.62:0/1764470960 >> 172.21.15.62:0/1764470960 conn(0x7fbbe40fc7d0 msgr2=0x7fbbe40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.717+0000 7fbbea7de640 1 -- 172.21.15.62:0/1764470960 shutdown_connections 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.717+0000 7fbbea7de640 1 -- 172.21.15.62:0/1764470960 wait complete. 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.718+0000 7fbbea7de640 1 Processor -- start 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.718+0000 7fbbea7de640 1 -- start start 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.718+0000 7fbbea7de640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe4100e20 0x7fbbe4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.718+0000 7fbbea7de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe4102570 0x7fbbe4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.718+0000 7fbbea7de640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe4102ef0 0x7fbbe419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.718+0000 7fbbea7de640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4110f80 con 0x7fbbe4100e20 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.718+0000 7fbbea7de640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4110e00 con 0x7fbbe4102ef0 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.718+0000 7fbbea7de640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4111100 con 0x7fbbe4102570 2024-09-12T00:06:31.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe4100e20 0x7fbbe4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe4100e20 0x7fbbe4198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43908/0 (socket says 172.21.15.62:43908) 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe3fff640 1 -- 172.21.15.62:0/4225740201 learned_addr learned my addr 172.21.15.62:0/4225740201 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe4102570 0x7fbbe4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe8d7b640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe4102ef0 0x7fbbe419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe3fff640 1 -- 172.21.15.62:0/4225740201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe4102570 msgr2=0x7fbbe4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe4102570 0x7fbbe4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe3fff640 1 -- 172.21.15.62:0/4225740201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe4102ef0 msgr2=0x7fbbe419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe4102ef0 0x7fbbe419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe3fff640 1 -- 172.21.15.62:0/4225740201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe419d4e0 con 0x7fbbe4100e20 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe8d7b640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe4102ef0 0x7fbbe419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe4100e20 0x7fbbe4198530 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4006010 tx=0x7fbbd4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe17fa640 1 -- 172.21.15.62:0/4225740201 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd403f070 con 0x7fbbe4100e20 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe17fa640 1 -- 172.21.15.62:0/4225740201 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd4031c30 con 0x7fbbe4100e20 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbea7de640 1 -- 172.21.15.62:0/4225740201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe419d770 con 0x7fbbe4100e20 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbe17fa640 1 -- 172.21.15.62:0/4225740201 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4035030 con 0x7fbbe4100e20 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.719+0000 7fbbea7de640 1 -- 172.21.15.62:0/4225740201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe41a50b0 con 0x7fbbe4100e20 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbe17fa640 1 -- 172.21.15.62:0/4225740201 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbbd4042070 con 0x7fbbe4100e20 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbecbb7740 1 Processor -- start 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbecbb7740 1 -- start start 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666fc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562f66650280 con 0x562f666ef890 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562f665f7b10 con 0x562f666fa150 2024-09-12T00:06:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f6664c240 con 0x562f666eef70 2024-09-12T00:06:31.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43920/0 (socket says 172.21.15.62:43920) 2024-09-12T00:06:31.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbe3fff640 1 -- 172.21.15.62:0/211137760 learned_addr learned my addr 172.21.15.62:0/211137760 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:31.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbe8d7b640 1 --2- 172.21.15.62:0/211137760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/211137760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.722+0000 7fbbe17fa640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbcc0a2cf0 0x7fbbcc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbe8d7b640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbcc0a2cf0 0x7fbbcc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbe17fa640 1 -- 172.21.15.62:0/4225740201 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fbbd40efcb0 con 0x7fbbe4100e20 2024-09-12T00:06:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbe3fff640 1 -- 172.21.15.62:0/211137760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 msgr2=0x562f666ef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/211137760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666ef350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbe3fff640 1 -- 172.21.15.62:0/211137760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 msgr2=0x562f666fc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/211137760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666fc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbe3fff640 1 -- 172.21.15.62:0/211137760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f666041f0 con 0x562f666ef890 2024-09-12T00:06:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/211137760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/211137760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f9c10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40bbed0 tx=0x7fbbd402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3bd8cec94104a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbe8d7b640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbcc0a2cf0 0x7fbbcc0a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc005fd0 tx=0x7fbbdc004ab0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbca7fc640 1 -- 172.21.15.62:0/211137760 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4049070 con 0x562f666ef890 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbca7fc640 1 -- 172.21.15.62:0/211137760 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd4031de0 con 0x562f666ef890 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.723+0000 7fbbca7fc640 1 -- 172.21.15.62:0/211137760 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40c0580 con 0x562f666ef890 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.724+0000 7fbbecbb7740 1 -- 172.21.15.62:0/211137760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 msgr2=0x562f666f9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.724+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/211137760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f9c10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40bbed0 tx=0x7fbbd402a480 comp rx=0 tx=0).stop 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.724+0000 7fbbecbb7740 1 -- 172.21.15.62:0/211137760 shutdown_connections 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.724+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/211137760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.724+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/211137760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f9c10 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.724+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/211137760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666ef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.724+0000 7fbbecbb7740 1 -- 172.21.15.62:0/211137760 >> 172.21.15.62:0/211137760 conn(0x562f666eb500 msgr2=0x562f666eb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.724+0000 7fbbecbb7740 1 -- 172.21.15.62:0/211137760 shutdown_connections 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.724+0000 7fbbecbb7740 1 -- 172.21.15.62:0/211137760 wait complete. 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbecbb7740 1 -- start start 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbecbb7740 1 Processor -- start 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562f666fcae0 con 0x562f666ef890 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562f6660af10 con 0x562f666fa150 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f666fcc60 con 0x562f666eef70 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbecbb7740 10 client.0 _ll_register_callbacks cb 0x562f665d3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbe37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbe37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43936/0 (socket says 172.21.15.62:43936) 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbe37fe640 1 -- 172.21.15.62:0/846776400 learned_addr learned my addr 172.21.15.62:0/846776400 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.725+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbe8d7b640 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbe37fe640 1 -- 172.21.15.62:0/846776400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 msgr2=0x562f666f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbe37fe640 1 -- 172.21.15.62:0/846776400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 msgr2=0x562f666f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbe37fe640 1 -- 172.21.15.62:0/846776400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f663c16a0 con 0x562f666ef890 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f7530 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8010850 tx=0x7fbbd8010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf907bef8f65dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbc37fe640 1 -- 172.21.15.62:0/846776400 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd801e060 con 0x562f666ef890 2024-09-12T00:06:31.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbc37fe640 1 -- 172.21.15.62:0/846776400 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fbbd800fb90 con 0x562f666ef890 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbecbb7740 1 -- 172.21.15.62:0/846776400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd4031de0 con 0x562f666ef890 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbc37fe640 1 -- 172.21.15.62:0/846776400 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbbd800fd90 con 0x562f666ef890 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbecbb7740 10 client.18000 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbecbb7740 1 -- 172.21.15.62:0/846776400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562f66708680 con 0x562f666ef890 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.726+0000 7fbbc37fe640 1 -- 172.21.15.62:0/846776400 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd4031de0 con 0x562f666ef890 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.727+0000 7fbbc37fe640 1 -- 172.21.15.62:0/846776400 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fbbd8018500 con 0x562f666ef890 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.727+0000 7fbbecbb7740 10 client.18000 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.727+0000 7fbbecbb7740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.727+0000 7fbbecbb7740 20 client.18000 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.727+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562f66709b90 0x562f6670bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.727+0000 7fbbecbb7740 4 client.18000 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.727+0000 7fbbecbb7740 1 -- 172.21.15.62:0/846776400 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f6665d7f0 con 0x562f66709b90 2024-09-12T00:06:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.727+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562f66709b90 0x562f6670bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.727+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562f66709b90 0x562f6670bfd0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88eecb924734e634 server_cookie=787a8cf3098edb11 in_seq=0 out_seq=0 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.727+0000 7fbbc37fe640 10 client.18000 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbc37fe640 1 -- 172.21.15.62:0/846776400 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f6665d7f0 con 0x562f66709b90 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbc37fe640 10 client.18000 handle_command_reply: tid=0 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbecbb7740 1 client.18000 shutdown 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbecbb7740 1 -- 172.21.15.62:0/846776400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 msgr2=0x562f666f7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f7530 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8010850 tx=0x7fbbd8010d10 comp rx=0 tx=0).stop 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbecbb7740 1 -- 172.21.15.62:0/846776400 shutdown_connections 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562f66709b90 0x562f6670bfd0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666f7530 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbc37fe640 0 client.18000 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/846776400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.728+0000 7fbbecbb7740 1 -- 172.21.15.62:0/846776400 >> 172.21.15.62:0/846776400 conn(0x562f666eb500 msgr2=0x562f666e6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:31.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.729+0000 7fbbecbb7740 1 -- 172.21.15.62:0/846776400 shutdown_connections 2024-09-12T00:06:31.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.729+0000 7fbbecbb7740 1 -- 172.21.15.62:0/846776400 wait complete. 2024-09-12T00:06:31.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.729+0000 7fbbecbb7740 20 client.18000 trim_cache size 0 max 16384 2024-09-12T00:06:31.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.746+0000 7fbbecbb7740 1 Processor -- start 2024-09-12T00:06:31.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.746+0000 7fbbecbb7740 1 -- start start 2024-09-12T00:06:31.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.746+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666eef70 0x562f666f10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.746+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f667118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.746+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666fa150 0x562f66711de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.746+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41adf50 con 0x562f666ef890 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.746+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41ab820 con 0x562f666eef70 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.746+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41a9690 con 0x562f666fa150 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f667118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f667118a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43946/0 (socket says 172.21.15.62:43946) 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe37fe640 1 -- 172.21.15.62:0/81474869 learned_addr learned my addr 172.21.15.62:0/81474869 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe8d7b640 1 --2- 172.21.15.62:0/81474869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666fa150 0x562f66711de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/81474869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666eef70 0x562f666f10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe37fe640 1 -- 172.21.15.62:0/81474869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666fa150 msgr2=0x562f66711de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/81474869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666fa150 0x562f66711de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe37fe640 1 -- 172.21.15.62:0/81474869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666eef70 msgr2=0x562f666f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/81474869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666eef70 0x562f666f10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe37fe640 1 -- 172.21.15.62:0/81474869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6670c650 con 0x562f666ef890 2024-09-12T00:06:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/81474869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666eef70 0x562f666f10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/81474869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f667118a0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80189a0 tx=0x7fbbd800fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46dc7a8b4e71dc76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbc9ffb640 1 -- 172.21.15.62:0/81474869 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8018b90 con 0x562f666ef890 2024-09-12T00:06:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbc9ffb640 1 -- 172.21.15.62:0/81474869 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd8018d30 con 0x562f666ef890 2024-09-12T00:06:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.747+0000 7fbbc9ffb640 1 -- 172.21.15.62:0/81474869 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8004660 con 0x562f666ef890 2024-09-12T00:06:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 -- 172.21.15.62:0/81474869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 msgr2=0x562f667118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/81474869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f667118a0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80189a0 tx=0x7fbbd800fd20 comp rx=0 tx=0).stop 2024-09-12T00:06:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 -- 172.21.15.62:0/81474869 shutdown_connections 2024-09-12T00:06:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/81474869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666fa150 0x562f66711de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/81474869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f667118a0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/81474869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666eef70 0x562f666f10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 -- 172.21.15.62:0/81474869 >> 172.21.15.62:0/81474869 conn(0x562f666eb500 msgr2=0x562f666f09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 -- 172.21.15.62:0/81474869 shutdown_connections 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 -- 172.21.15.62:0/81474869 wait complete. 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 -- start start 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.748+0000 7fbbecbb7740 1 Processor -- start 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666fe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbecbb7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41a9ce0 con 0x562f666ef890 2024-09-12T00:06:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41ab150 con 0x562f666fa150 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbecbb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41aa3b0 con 0x562f666eef70 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbecbb7740 10 client.0 _ll_register_callbacks cb 0x7fbbd800a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666fe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666fe0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43960/0 (socket says 172.21.15.62:43960) 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe37fe640 1 -- 172.21.15.62:0/231737528 learned_addr learned my addr 172.21.15.62:0/231737528 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe8d7b640 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe37fe640 1 -- 172.21.15.62:0/231737528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 msgr2=0x562f666fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666fdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe37fe640 1 -- 172.21.15.62:0/231737528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 msgr2=0x562f666fe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666fe600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe37fe640 1 -- 172.21.15.62:0/231737528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f667739d0 con 0x562f666ef890 2024-09-12T00:06:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe8d7b640 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbe37fe640 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666fe0c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8018920 tx=0x7fbbd80056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4c7b36db298513 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.749+0000 7fbbc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbc37fe640 1 -- 172.21.15.62:0/231737528 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8038060 con 0x562f666ef890 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbc37fe640 1 -- 172.21.15.62:0/231737528 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fbbd8018580 con 0x562f666ef890 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbecbb7740 1 -- 172.21.15.62:0/231737528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd8018d30 con 0x562f666ef890 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbc37fe640 1 -- 172.21.15.62:0/231737528 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbbd801e040 con 0x562f666ef890 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbecbb7740 10 client.18006 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbecbb7740 1 -- 172.21.15.62:0/231737528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562f66774c20 con 0x562f666ef890 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbc37fe640 1 -- 172.21.15.62:0/231737528 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd8018d30 con 0x562f666ef890 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbc37fe640 1 -- 172.21.15.62:0/231737528 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fbbd802b430 con 0x562f666ef890 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbecbb7740 10 client.18006 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbecbb7740 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:06:31.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbecbb7740 20 client.18006 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562f66775320 0x562f66777740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbecbb7740 4 client.18006 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.750+0000 7fbbecbb7740 1 -- 172.21.15.62:0/231737528 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562f66657d00 con 0x562f66775320 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.751+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562f66775320 0x562f66777740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.751+0000 7fbbe3fff640 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562f66775320 0x562f66777740 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82891ab6644f2c0c server_cookie=e51db9b170a13ef6 in_seq=0 out_seq=0 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.751+0000 7fbbc37fe640 10 client.18006 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbc37fe640 1 -- 172.21.15.62:0/231737528 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562f66657d00 con 0x562f66775320 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbc37fe640 10 client.18006 handle_command_reply: tid=0 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 client.18006 shutdown 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 -- 172.21.15.62:0/231737528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 msgr2=0x562f666fe0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666fe0c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8018920 tx=0x7fbbd80056b0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 -- 172.21.15.62:0/231737528 shutdown_connections 2024-09-12T00:06:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562f66775320 0x562f66777740 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f666fa150 0x562f666fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f666ef890 0x562f666fe0c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 --2- 172.21.15.62:0/231737528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f666eef70 0x562f666fdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 -- 172.21.15.62:0/231737528 >> 172.21.15.62:0/231737528 conn(0x562f666eb500 msgr2=0x562f666f11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbc37fe640 0 client.18006 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 -- 172.21.15.62:0/231737528 shutdown_connections 2024-09-12T00:06:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 1 -- 172.21.15.62:0/231737528 wait complete. 2024-09-12T00:06:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.752+0000 7fbbecbb7740 20 client.18006 trim_cache size 0 max 16384 2024-09-12T00:06:31.757 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:06:31.758 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-12T00:06:31.758 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "4ab6a655-98b0-4779-b479-21a601a65210", 2024-09-12T00:06:31.758 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-12T00:06:31.758 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:06:31.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 -- 172.21.15.62:0/4225740201 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbcc0a2cf0 msgr2=0x7fbbcc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:31.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbcc0a2cf0 0x7fbbcc0a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc005fd0 tx=0x7fbbdc004ab0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 -- 172.21.15.62:0/4225740201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe4100e20 msgr2=0x7fbbe4198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe4100e20 0x7fbbe4198530 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4006010 tx=0x7fbbd4003c10 comp rx=0 tx=0).stop 2024-09-12T00:06:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 -- 172.21.15.62:0/4225740201 shutdown_connections 2024-09-12T00:06:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbbcc0a2cf0 0x7fbbcc0a51b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe4102ef0 0x7fbbe419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbe4102570 0x7fbbe4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 --2- 172.21.15.62:0/4225740201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe4100e20 0x7fbbe4198530 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 -- 172.21.15.62:0/4225740201 >> 172.21.15.62:0/4225740201 conn(0x7fbbe40fc7d0 msgr2=0x7fbbe406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 -- 172.21.15.62:0/4225740201 shutdown_connections 2024-09-12T00:06:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:31.753+0000 7fbbea7de640 1 -- 172.21.15.62:0/4225740201 wait complete. 2024-09-12T00:06:32.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 830.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:06:32.450 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:06:32.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:32 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-12T00:06:32.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:32 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:32 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:06:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:32 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:06:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:32 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:32 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:06:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:32 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:06:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:32 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.621 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:32 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:32 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-12T00:06:32.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:32 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:32 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:06:32.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:32 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:06:32.668 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:32 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:32 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:06:32.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:32 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:06:32.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:32 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.669 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:32 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.707 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:32 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-12T00:06:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:32 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:32 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:06:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:32 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:06:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:32 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:32 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:06:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:32 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:06:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:32 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:32.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:32 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.209+0000 7feee3b15640 1 -- 172.21.15.62:0/439111395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feedc106f10 msgr2=0x7feedc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.209+0000 7feee3b15640 1 --2- 172.21.15.62:0/439111395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feedc106f10 0x7feedc10ea90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7feed4009900 tx=0x7feed402a410 comp rx=0 tx=0).stop 2024-09-12T00:06:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.209+0000 7feee3b15640 1 -- 172.21.15.62:0/439111395 shutdown_connections 2024-09-12T00:06:33.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.209+0000 7feee3b15640 1 --2- 172.21.15.62:0/439111395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feedc106f10 0x7feedc10ea90 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.209+0000 7feee3b15640 1 --2- 172.21.15.62:0/439111395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feedc102790 0x7feedc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.209+0000 7feee3b15640 1 --2- 172.21.15.62:0/439111395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101de0 0x7feedc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.209+0000 7feee3b15640 1 -- 172.21.15.62:0/439111395 >> 172.21.15.62:0/439111395 conn(0x7feedc0fd8c0 msgr2=0x7feedc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.209+0000 7feee3b15640 1 -- 172.21.15.62:0/439111395 shutdown_connections 2024-09-12T00:06:33.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.209+0000 7feee3b15640 1 -- 172.21.15.62:0/439111395 wait complete. 2024-09-12T00:06:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.209+0000 7feee3b15640 1 Processor -- start 2024-09-12T00:06:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee3b15640 1 -- start start 2024-09-12T00:06:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee3b15640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101de0 0x7feedc198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee3b15640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feedc102790 0x7feedc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee3b15640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feedc106f10 0x7feedc19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee3b15640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feedc111f90 con 0x7feedc106f10 2024-09-12T00:06:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee3b15640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feedc111e10 con 0x7feedc101de0 2024-09-12T00:06:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee3b15640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feedc112110 con 0x7feedc102790 2024-09-12T00:06:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee20b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feedc106f10 0x7feedc19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee20b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feedc106f10 0x7feedc19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43970/0 (socket says 172.21.15.62:43970) 2024-09-12T00:06:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee20b2640 1 -- 172.21.15.62:0/3226983838 learned_addr learned my addr 172.21.15.62:0/3226983838 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee10b0640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feedc102790 0x7feedc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee18b1640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101de0 0x7feedc198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee20b2640 1 -- 172.21.15.62:0/3226983838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feedc102790 msgr2=0x7feedc198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee20b2640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feedc102790 0x7feedc198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee20b2640 1 -- 172.21.15.62:0/3226983838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101de0 msgr2=0x7feedc198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee20b2640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101de0 0x7feedc198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.210+0000 7feee20b2640 1 -- 172.21.15.62:0/3226983838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc19d500 con 0x7feedc106f10 2024-09-12T00:06:33.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.211+0000 7feee20b2640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feedc106f10 0x7feedc19ce20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7feed4005ab0 tx=0x7feed40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.211+0000 7feeceffd640 1 -- 172.21.15.62:0/3226983838 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed403f030 con 0x7feedc106f10 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.211+0000 7feeceffd640 1 -- 172.21.15.62:0/3226983838 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed4003ce0 con 0x7feedc106f10 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.211+0000 7feee3b15640 1 -- 172.21.15.62:0/3226983838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc19d790 con 0x7feedc106f10 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.211+0000 7feeceffd640 1 -- 172.21.15.62:0/3226983838 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4035070 con 0x7feedc106f10 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.211+0000 7feee3b15640 1 -- 172.21.15.62:0/3226983838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc1a50d0 con 0x7feedc106f10 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.214+0000 7feeceffd640 1 -- 172.21.15.62:0/3226983838 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feed4042030 con 0x7feedc106f10 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.214+0000 7feee5eee740 1 Processor -- start 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.214+0000 7feee5eee740 1 -- start start 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.214+0000 7feee5eee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 0x560b6a1c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.214+0000 7feee5eee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.214+0000 7feee5eee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.214+0000 7feee5eee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560b6a12e2e0 con 0x560b6a1d3680 2024-09-12T00:06:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.214+0000 7feee5eee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560b6a0d5ce0 con 0x560b6a1c8dc0 2024-09-12T00:06:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.214+0000 7feee5eee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560b6a12a2a0 con 0x560b6a1c84a0 2024-09-12T00:06:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.214+0000 7feee10b0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee10b0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43984/0 (socket says 172.21.15.62:43984) 2024-09-12T00:06:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee10b0640 1 -- 172.21.15.62:0/2906384722 learned_addr learned my addr 172.21.15.62:0/2906384722 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee10b0640 1 --2- 172.21.15.62:0/2906384722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 0x560b6a1c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee18b1640 1 --2- 172.21.15.62:0/2906384722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee10b0640 1 -- 172.21.15.62:0/2906384722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 msgr2=0x560b6a1c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee10b0640 1 --2- 172.21.15.62:0/2906384722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 0x560b6a1c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee10b0640 1 -- 172.21.15.62:0/2906384722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c8dc0 msgr2=0x560b6a1d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee10b0640 1 --2- 172.21.15.62:0/2906384722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee10b0640 1 -- 172.21.15.62:0/2906384722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b6a0ba440 con 0x560b6a1d3680 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feeceffd640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feec80a2d70 0x7feec80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee10b0640 1 --2- 172.21.15.62:0/2906384722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d5a70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7feed8016530 tx=0x7feed800a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad342ec4c506d127 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feee18b1640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feec80a2d70 0x7feec80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feeceffd640 1 -- 172.21.15.62:0/3226983838 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7feed40ef8d0 con 0x7feedc106f10 2024-09-12T00:06:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feec7fff640 1 -- 172.21.15.62:0/2906384722 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8018430 con 0x560b6a1d3680 2024-09-12T00:06:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feec7fff640 1 -- 172.21.15.62:0/2906384722 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed8018ab0 con 0x560b6a1d3680 2024-09-12T00:06:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.215+0000 7feec7fff640 1 -- 172.21.15.62:0/2906384722 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8017a40 con 0x560b6a1d3680 2024-09-12T00:06:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.216+0000 7feee5eee740 1 -- 172.21.15.62:0/2906384722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 msgr2=0x560b6a1d5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.216+0000 7feee18b1640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feec80a2d70 0x7feec80a5230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x560b6a1c5890 tx=0x7feed0009580 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.216+0000 7feee5eee740 1 --2- 172.21.15.62:0/2906384722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d5a70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7feed8016530 tx=0x7feed800a2b0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.216+0000 7feee5eee740 1 -- 172.21.15.62:0/2906384722 shutdown_connections 2024-09-12T00:06:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.216+0000 7feee5eee740 1 --2- 172.21.15.62:0/2906384722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d5a70 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.216+0000 7feee5eee740 1 --2- 172.21.15.62:0/2906384722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.216+0000 7feee5eee740 1 --2- 172.21.15.62:0/2906384722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 0x560b6a1c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.216+0000 7feee5eee740 1 -- 172.21.15.62:0/2906384722 >> 172.21.15.62:0/2906384722 conn(0x560b6a1c4a30 msgr2=0x560b6a1c4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.216+0000 7feee5eee740 1 -- 172.21.15.62:0/2906384722 shutdown_connections 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.216+0000 7feee5eee740 1 -- 172.21.15.62:0/2906384722 wait complete. 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee5eee740 1 -- start start 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee5eee740 1 Processor -- start 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee5eee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c84a0 0x560b6a1d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee5eee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee5eee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee5eee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560b6a1d61f0 con 0x560b6a1d3680 2024-09-12T00:06:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee5eee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560b6a0e9050 con 0x560b6a1c84a0 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee5eee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560b6a1d6370 con 0x560b6a1c8dc0 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee5eee740 10 client.0 _ll_register_callbacks cb 0x560b6a129160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee20b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee18b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c84a0 0x560b6a1d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee20b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43986/0 (socket says 172.21.15.62:43986) 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee20b2640 1 -- 172.21.15.62:0/2099967424 learned_addr learned my addr 172.21.15.62:0/2099967424 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.217+0000 7feee10b0640 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feee20b2640 1 -- 172.21.15.62:0/2099967424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c8dc0 msgr2=0x560b6a1d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feee20b2640 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feee20b2640 1 -- 172.21.15.62:0/2099967424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c84a0 msgr2=0x560b6a1d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feee20b2640 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c84a0 0x560b6a1d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feee20b2640 1 -- 172.21.15.62:0/2099967424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b69e9f3c0 con 0x560b6a1d3680 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feee18b1640 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c84a0 0x560b6a1d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feee20b2640 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d0740 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7feed40b87c0 tx=0x7feed40b8150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f18ad8abdf1c3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feec57fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feec57fa640 1 -- 172.21.15.62:0/2099967424 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40bf060 con 0x560b6a1d3680 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feec57fa640 1 -- 172.21.15.62:0/2099967424 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7feed40ed2a0 con 0x560b6a1d3680 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feee5eee740 1 -- 172.21.15.62:0/2099967424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed8018ab0 con 0x560b6a1d3680 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feec57fa640 1 -- 172.21.15.62:0/2099967424 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7feed4049040 con 0x560b6a1d3680 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feee5eee740 10 client.18018 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feee5eee740 1 -- 172.21.15.62:0/2099967424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560b6a1e1b40 con 0x560b6a1d3680 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.218+0000 7feec57fa640 1 -- 172.21.15.62:0/2099967424 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed8018ab0 con 0x560b6a1d3680 2024-09-12T00:06:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.219+0000 7feec57fa640 1 -- 172.21.15.62:0/2099967424 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7feed40b8460 con 0x560b6a1d3680 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.219+0000 7feee5eee740 10 client.18018 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.219+0000 7feee5eee740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.219+0000 7feee5eee740 20 client.18018 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.219+0000 7feee5eee740 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560b6a1e3040 0x560b6a1e5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.219+0000 7feee5eee740 4 client.18018 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.219+0000 7feee5eee740 1 -- 172.21.15.62:0/2099967424 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b6a13b850 con 0x560b6a1e3040 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.219+0000 7feee18b1640 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560b6a1e3040 0x560b6a1e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.219+0000 7feec57fa640 10 client.18018 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.219+0000 7feee18b1640 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560b6a1e3040 0x560b6a1e5490 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9922a46d01915b server_cookie=faf1e86ad3dfc379 in_seq=0 out_seq=0 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feec57fa640 1 -- 172.21.15.62:0/2099967424 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b6a13b850 con 0x560b6a1e3040 2024-09-12T00:06:33.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feec57fa640 10 client.18018 handle_command_reply: tid=0 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 client.18018 shutdown 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 -- 172.21.15.62:0/2099967424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 msgr2=0x560b6a1d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d0740 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7feed40b87c0 tx=0x7feed40b8150 comp rx=0 tx=0).stop 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 -- 172.21.15.62:0/2099967424 shutdown_connections 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560b6a1e3040 0x560b6a1e5490 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1d3680 0x560b6a1d0740 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 --2- 172.21.15.62:0/2099967424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c84a0 0x560b6a1d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feec57fa640 0 client.18018 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 -- 172.21.15.62:0/2099967424 >> 172.21.15.62:0/2099967424 conn(0x560b6a1c4a30 msgr2=0x560b6a1ca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 -- 172.21.15.62:0/2099967424 shutdown_connections 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.220+0000 7feee5eee740 1 -- 172.21.15.62:0/2099967424 wait complete. 2024-09-12T00:06:33.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.221+0000 7feee5eee740 20 client.18018 trim_cache size 0 max 16384 2024-09-12T00:06:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee5eee740 1 Processor -- start 2024-09-12T00:06:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee5eee740 1 -- start start 2024-09-12T00:06:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee5eee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 0x560b6a1df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee5eee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 0x560b6a1d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee5eee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c8dc0 0x560b6a1cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee5eee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1b00b0 con 0x560b6a1c8dc0 2024-09-12T00:06:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee5eee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1ad980 con 0x560b6a1c4a30 2024-09-12T00:06:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee5eee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1ab7f0 con 0x560b6a1c84a0 2024-09-12T00:06:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee20b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c8dc0 0x560b6a1cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee20b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c8dc0 0x560b6a1cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44002/0 (socket says 172.21.15.62:44002) 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee20b2640 1 -- 172.21.15.62:0/3171751798 learned_addr learned my addr 172.21.15.62:0/3171751798 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee18b1640 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 0x560b6a1df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee10b0640 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 0x560b6a1d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.238+0000 7feee20b2640 1 -- 172.21.15.62:0/3171751798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 msgr2=0x560b6a1d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.239+0000 7feee20b2640 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 0x560b6a1d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.239+0000 7feee20b2640 1 -- 172.21.15.62:0/3171751798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 msgr2=0x560b6a1df450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.239+0000 7feee20b2640 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 0x560b6a1df450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.239+0000 7feee20b2640 1 -- 172.21.15.62:0/3171751798 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b6a1e5ae0 con 0x560b6a1c8dc0 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.239+0000 7feee18b1640 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 0x560b6a1df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.239+0000 7feee10b0640 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 0x560b6a1d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.239+0000 7feee20b2640 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c8dc0 0x560b6a1cb7a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7feed40ed0d0 tx=0x7feed40c00d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8122864f7eb7d17c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.239+0000 7feec77fe640 1 -- 172.21.15.62:0/3171751798 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40c01a0 con 0x560b6a1c8dc0 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.239+0000 7feec77fe640 1 -- 172.21.15.62:0/3171751798 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed40bbdf0 con 0x560b6a1c8dc0 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.239+0000 7feec77fe640 1 -- 172.21.15.62:0/3171751798 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40bc050 con 0x560b6a1c8dc0 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 -- 172.21.15.62:0/3171751798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c8dc0 msgr2=0x560b6a1cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c8dc0 0x560b6a1cb7a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7feed40ed0d0 tx=0x7feed40c00d0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 -- 172.21.15.62:0/3171751798 shutdown_connections 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c8dc0 0x560b6a1cb7a0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c84a0 0x560b6a1d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 --2- 172.21.15.62:0/3171751798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 0x560b6a1df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 -- 172.21.15.62:0/3171751798 >> 172.21.15.62:0/3171751798 conn(0x560b6a1d08a0 msgr2=0x560b6a1cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 -- 172.21.15.62:0/3171751798 shutdown_connections 2024-09-12T00:06:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 -- 172.21.15.62:0/3171751798 wait complete. 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 -- start start 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 Processor -- start 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.240+0000 7feee5eee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 0x560b6a1d7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee5eee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c84a0 0x560b6a1d7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee5eee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee5eee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1abe40 con 0x560b6a1c84a0 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee5eee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1ad2b0 con 0x560b6a1c4a30 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee5eee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1ac510 con 0x560b6a1c8dc0 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee5eee740 10 client.0 _ll_register_callbacks cb 0x7feed4009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee10b0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c84a0 0x560b6a1d7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee10b0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c84a0 0x560b6a1d7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44010/0 (socket says 172.21.15.62:44010) 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee10b0640 1 -- 172.21.15.62:0/345274130 learned_addr learned my addr 172.21.15.62:0/345274130 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee20b2640 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee18b1640 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 0x560b6a1d7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee10b0640 1 -- 172.21.15.62:0/345274130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c8dc0 msgr2=0x560b6a1d7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee10b0640 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee10b0640 1 -- 172.21.15.62:0/345274130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 msgr2=0x560b6a1d7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee10b0640 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 0x560b6a1d7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee10b0640 1 -- 172.21.15.62:0/345274130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b6a1c4860 con 0x560b6a1c84a0 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee10b0640 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c84a0 0x560b6a1d7aa0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7feed8017690 tx=0x7feed80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6238ae6ae5a49747 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feec57fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feec57fa640 1 -- 172.21.15.62:0/345274130 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8028070 con 0x560b6a1c84a0 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feec57fa640 1 -- 172.21.15.62:0/345274130 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7feed8018430 con 0x560b6a1c84a0 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.241+0000 7feee5eee740 1 -- 172.21.15.62:0/345274130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed40bbdf0 con 0x560b6a1c84a0 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feec57fa640 1 -- 172.21.15.62:0/345274130 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7feed8034050 con 0x560b6a1c84a0 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feee5eee740 10 client.18024 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feee5eee740 1 -- 172.21.15.62:0/345274130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560b6a24e020 con 0x560b6a1c84a0 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feec57fa640 1 -- 172.21.15.62:0/345274130 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed40bbdf0 con 0x560b6a1c84a0 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feec57fa640 1 -- 172.21.15.62:0/345274130 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7feed8012050 con 0x560b6a1c84a0 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feee5eee740 10 client.18024 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feee5eee740 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feee5eee740 20 client.18024 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feee5eee740 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560b6a1d3680 0x560b6a250400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feee5eee740 4 client.18024 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:06:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feee5eee740 1 -- 172.21.15.62:0/345274130 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b6a135d60 con 0x560b6a1d3680 2024-09-12T00:06:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.242+0000 7feee18b1640 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560b6a1d3680 0x560b6a250400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee18b1640 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560b6a1d3680 0x560b6a250400 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f58563023a4d1dc server_cookie=1542a46a3f665dbf in_seq=0 out_seq=0 2024-09-12T00:06:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feec57fa640 10 client.18024 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feec57fa640 1 -- 172.21.15.62:0/345274130 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560b6a135d60 con 0x560b6a1d3680 2024-09-12T00:06:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feec57fa640 10 client.18024 handle_command_reply: tid=0 2024-09-12T00:06:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 client.18024 shutdown 2024-09-12T00:06:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 -- 172.21.15.62:0/345274130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c84a0 msgr2=0x560b6a1d7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c84a0 0x560b6a1d7aa0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7feed8017690 tx=0x7feed80046c0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 -- 172.21.15.62:0/345274130 shutdown_connections 2024-09-12T00:06:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x560b6a1d3680 0x560b6a250400 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b6a1c8dc0 0x560b6a1d7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feec57fa640 0 client.18024 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b6a1c84a0 0x560b6a1d7aa0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 --2- 172.21.15.62:0/345274130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b6a1c4a30 0x560b6a1d7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 -- 172.21.15.62:0/345274130 >> 172.21.15.62:0/345274130 conn(0x560b6a1d08a0 msgr2=0x560b6a1cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 -- 172.21.15.62:0/345274130 shutdown_connections 2024-09-12T00:06:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 1 -- 172.21.15.62:0/345274130 wait complete. 2024-09-12T00:06:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.243+0000 7feee5eee740 20 client.18024 trim_cache size 0 max 16384 2024-09-12T00:06:33.251 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:06:33.251 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:06:33.251 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-12T00:06:33.251 INFO:teuthology.orchestra.run.smithi062.stdout: "4ab6a655-98b0-4779-b479-21a601a65210": { 2024-09-12T00:06:33.251 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-12T00:06:33.251 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "4ab6a655-98b0-4779-b479-21a601a65210", 2024-09-12T00:06:33.252 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-12T00:06:33.252 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:33.252 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:33.253 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:06:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 -- 172.21.15.62:0/3226983838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feec80a2d70 msgr2=0x7feec80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feec80a2d70 0x7feec80a5230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x560b6a1c5890 tx=0x7feed0009580 comp rx=0 tx=0).stop 2024-09-12T00:06:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 -- 172.21.15.62:0/3226983838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feedc106f10 msgr2=0x7feedc19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feedc106f10 0x7feedc19ce20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7feed4005ab0 tx=0x7feed40042f0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 -- 172.21.15.62:0/3226983838 shutdown_connections 2024-09-12T00:06:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7feec80a2d70 0x7feec80a5230 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feedc106f10 0x7feedc19ce20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feedc102790 0x7feedc198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 --2- 172.21.15.62:0/3226983838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feedc101de0 0x7feedc198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 -- 172.21.15.62:0/3226983838 >> 172.21.15.62:0/3226983838 conn(0x7feedc0fd8c0 msgr2=0x7feedc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 -- 172.21.15.62:0/3226983838 shutdown_connections 2024-09-12T00:06:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:33.244+0000 7feee3b15640 1 -- 172.21.15.62:0/3226983838 wait complete. 2024-09-12T00:06:33.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:33 smithi062 ceph-mon[30958]: pgmap v450: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 MiB/s wr, 263 op/s 2024-09-12T00:06:33.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:33 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:33 smithi120 ceph-mon[35182]: pgmap v450: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 MiB/s wr, 263 op/s 2024-09-12T00:06:33.620 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:33 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:33.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:33 smithi083 ceph-mon[34805]: pgmap v450: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 MiB/s wr, 263 op/s 2024-09-12T00:06:33.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:33 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:06:33.976 INFO:tasks.cephfs.filesystem:scrub status for tag:4ab6a655-98b0-4779-b479-21a601a65210 - {'path': '/', 'tag': '4ab6a655-98b0-4779-b479-21a601a65210', 'options': 'recursive,force'} 2024-09-12T00:06:35.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:35 smithi083 ceph-mon[34805]: pgmap v451: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-12T00:06:35.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:35 smithi062 ceph-mon[30958]: pgmap v451: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-12T00:06:35.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:35 smithi120 ceph-mon[35182]: pgmap v451: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-12T00:06:36.212 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:06:36.489 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:36.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 -- 172.21.15.62:0/3803936328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab8101de0 msgr2=0x7f1ab81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 --2- 172.21.15.62:0/3803936328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab8101de0 0x7f1ab81021c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009900 tx=0x7f1ab402a410 comp rx=0 tx=0).stop 2024-09-12T00:06:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 -- 172.21.15.62:0/3803936328 shutdown_connections 2024-09-12T00:06:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 --2- 172.21.15.62:0/3803936328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab8106f10 0x7f1ab810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 --2- 172.21.15.62:0/3803936328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8102790 0x7f1ab81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 --2- 172.21.15.62:0/3803936328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab8101de0 0x7f1ab81021c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 -- 172.21.15.62:0/3803936328 >> 172.21.15.62:0/3803936328 conn(0x7f1ab80fd8c0 msgr2=0x7f1ab80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 -- 172.21.15.62:0/3803936328 shutdown_connections 2024-09-12T00:06:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 -- 172.21.15.62:0/3803936328 wait complete. 2024-09-12T00:06:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 Processor -- start 2024-09-12T00:06:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.986+0000 7f1ac0479640 1 -- start start 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1ac0479640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab8101de0 0x7f1ab81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1ac0479640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8102790 0x7f1ab8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1ac0479640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab8106f10 0x7f1ab819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1ac0479640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8111f70 con 0x7f1ab8106f10 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1ac0479640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8111df0 con 0x7f1ab8102790 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1ac0479640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81120f0 con 0x7f1ab8101de0 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abea16640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab8106f10 0x7f1ab819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abea16640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab8106f10 0x7f1ab819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44024/0 (socket says 172.21.15.62:44024) 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abe215640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab8101de0 0x7f1ab81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abea16640 1 -- 172.21.15.62:0/1651077013 learned_addr learned my addr 172.21.15.62:0/1651077013 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abda14640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8102790 0x7f1ab8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:36.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abea16640 1 -- 172.21.15.62:0/1651077013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab8101de0 msgr2=0x7f1ab81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:36.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abea16640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab8101de0 0x7f1ab81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:36.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abea16640 1 -- 172.21.15.62:0/1651077013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8102790 msgr2=0x7f1ab8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:36.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abea16640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8102790 0x7f1ab8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:36.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abea16640 1 -- 172.21.15.62:0/1651077013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab819d380 con 0x7f1ab8106f10 2024-09-12T00:06:36.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.987+0000 7f1abea16640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab8106f10 0x7f1ab819cca0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000cce0 tx=0x7f1ab000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:36.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.988+0000 7f1aaf7fe640 1 -- 172.21.15.62:0/1651077013 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000f450 con 0x7f1ab8106f10 2024-09-12T00:06:36.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.988+0000 7f1aaf7fe640 1 -- 172.21.15.62:0/1651077013 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab000fad0 con 0x7f1ab8106f10 2024-09-12T00:06:36.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.988+0000 7f1ac0479640 1 -- 172.21.15.62:0/1651077013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab819d670 con 0x7f1ab8106f10 2024-09-12T00:06:36.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.988+0000 7f1aaf7fe640 1 -- 172.21.15.62:0/1651077013 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0018c10 con 0x7f1ab8106f10 2024-09-12T00:06:36.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.988+0000 7f1ac0479640 1 -- 172.21.15.62:0/1651077013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab81a4f50 con 0x7f1ab8106f10 2024-09-12T00:06:36.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.989+0000 7f1ac0479640 1 -- 172.21.15.62:0/1651077013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab81021c0 con 0x7f1ab8106f10 2024-09-12T00:06:36.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.990+0000 7f1aaf7fe640 1 -- 172.21.15.62:0/1651077013 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1ab000f5f0 con 0x7f1ab8106f10 2024-09-12T00:06:36.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.991+0000 7f1aaf7fe640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1aa40a2cf0 0x7f1aa40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:36.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.991+0000 7f1abe215640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1aa40a2cf0 0x7f1aa40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:36.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.991+0000 7f1aaf7fe640 1 -- 172.21.15.62:0/1651077013 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f1ab00cfab0 con 0x7f1ab8106f10 2024-09-12T00:06:36.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.991+0000 7f1abe215640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1aa40a2cf0 0x7f1aa40a51b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009230 tx=0x7f1ab40058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:36.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:36.994+0000 7f1aaf7fe640 1 -- 172.21.15.62:0/1651077013 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ab0014030 con 0x7f1ab8106f10 2024-09-12T00:06:37.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.206+0000 7f1ac0479640 1 -- 172.21.15.62:0/1651077013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ab8199730 con 0x7f1ab8106f10 2024-09-12T00:06:37.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.207+0000 7f1aaf7fe640 1 -- 172.21.15.62:0/1651077013 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11909 (secure 0 0 0) 0x7f1ab009c760 con 0x7f1ab8106f10 2024-09-12T00:06:37.207 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:06:37.207 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":39,"btime":"2024-09-12T00:06:00:989943+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:06:00.711094+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:06:37.208 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 39 2024-09-12T00:06:37.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 -- 172.21.15.62:0/1651077013 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1aa40a2cf0 msgr2=0x7f1aa40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:37.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1aa40a2cf0 0x7f1aa40a51b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009230 tx=0x7f1ab40058d0 comp rx=0 tx=0).stop 2024-09-12T00:06:37.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 -- 172.21.15.62:0/1651077013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab8106f10 msgr2=0x7f1ab819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:37.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab8106f10 0x7f1ab819cca0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000cce0 tx=0x7f1ab000a620 comp rx=0 tx=0).stop 2024-09-12T00:06:37.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 -- 172.21.15.62:0/1651077013 shutdown_connections 2024-09-12T00:06:37.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f1aa40a2cf0 0x7f1aa40a51b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:37.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab8106f10 0x7f1ab819cca0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:37.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8102790 0x7f1ab8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:37.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 --2- 172.21.15.62:0/1651077013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab8101de0 0x7f1ab81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:37.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 -- 172.21.15.62:0/1651077013 >> 172.21.15.62:0/1651077013 conn(0x7f1ab80fd8c0 msgr2=0x7f1ab80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:37.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 -- 172.21.15.62:0/1651077013 shutdown_connections 2024-09-12T00:06:37.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:37.209+0000 7f1ac0479640 1 -- 172.21.15.62:0/1651077013 wait complete. 2024-09-12T00:06:37.449 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 835.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-12T00:06:37.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:37 smithi062 ceph-mon[30958]: pgmap v452: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 233 KiB/s wr, 32 op/s 2024-09-12T00:06:37.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:37 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1651077013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:37.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:37 smithi083 ceph-mon[34805]: pgmap v452: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 233 KiB/s wr, 32 op/s 2024-09-12T00:06:37.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:37 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1651077013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:37.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:37 smithi120 ceph-mon[35182]: pgmap v452: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 233 KiB/s wr, 32 op/s 2024-09-12T00:06:37.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:37 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1651077013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:06:37.910 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 3dc716a2 --release --await' 2024-09-12T00:06:37.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 3dc716a2 --release --await 2024-09-12T00:06:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:06:38.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.642+0000 7f5750158640 1 -- 172.21.15.62:0/1686764174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5748102570 msgr2=0x7f57481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.642+0000 7f5750158640 1 --2- 172.21.15.62:0/1686764174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5748102570 0x7f57481029b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5738009900 tx=0x7f573802a480 comp rx=0 tx=0).stop 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.642+0000 7f5750158640 1 -- 172.21.15.62:0/1686764174 shutdown_connections 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.642+0000 7f5750158640 1 --2- 172.21.15.62:0/1686764174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5748102ef0 0x7f574810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.642+0000 7f5750158640 1 --2- 172.21.15.62:0/1686764174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5748102570 0x7f57481029b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.642+0000 7f5750158640 1 --2- 172.21.15.62:0/1686764174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5748100e20 0x7f5748101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.642+0000 7f5750158640 1 -- 172.21.15.62:0/1686764174 >> 172.21.15.62:0/1686764174 conn(0x7f57480fc7d0 msgr2=0x7f57480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.642+0000 7f5750158640 1 -- 172.21.15.62:0/1686764174 shutdown_connections 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.642+0000 7f5750158640 1 -- 172.21.15.62:0/1686764174 wait complete. 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.643+0000 7f5750158640 1 Processor -- start 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.643+0000 7f5750158640 1 -- start start 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.643+0000 7f5750158640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5748100e20 0x7f5748199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.643+0000 7f5750158640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5748102570 0x7f5748199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.643+0000 7f5750158640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5748102ef0 0x7f57481a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.643+0000 7f5750158640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5748110f80 con 0x7f5748102570 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.643+0000 7f5750158640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5748110e00 con 0x7f5748100e20 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.643+0000 7f5750158640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5748111100 con 0x7f5748102ef0 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574d6f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5748102570 0x7f5748199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574d6f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5748102570 0x7f5748199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44046/0 (socket says 172.21.15.62:44046) 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574d6f3640 1 -- 172.21.15.62:0/3449408222 learned_addr learned my addr 172.21.15.62:0/3449408222 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574e6f5640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5748102ef0 0x7f57481a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574def4640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5748100e20 0x7f5748199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574d6f3640 1 -- 172.21.15.62:0/3449408222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5748102ef0 msgr2=0x7f57481a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574d6f3640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5748102ef0 0x7f57481a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574d6f3640 1 -- 172.21.15.62:0/3449408222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5748100e20 msgr2=0x7f5748199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574d6f3640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5748100e20 0x7f5748199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574d6f3640 1 -- 172.21.15.62:0/3449408222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57481a48d0 con 0x7f5748102570 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574def4640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5748100e20 0x7f5748199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f574d6f3640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5748102570 0x7f5748199850 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5738006010 tx=0x7f5738003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f573effd640 1 -- 172.21.15.62:0/3449408222 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573803f070 con 0x7f5748102570 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f573effd640 1 -- 172.21.15.62:0/3449408222 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5738031c30 con 0x7f5748102570 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f5750158640 1 -- 172.21.15.62:0/3449408222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57481a4aa0 con 0x7f5748102570 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f573effd640 1 -- 172.21.15.62:0/3449408222 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738035030 con 0x7f5748102570 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.644+0000 7f5750158640 1 -- 172.21.15.62:0/3449408222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57481a4ef0 con 0x7f5748102570 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.647+0000 7f573effd640 1 -- 172.21.15.62:0/3449408222 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5738042070 con 0x7f5748102570 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.647+0000 7f5752531740 1 Processor -- start 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.647+0000 7f5752531740 1 -- start start 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.647+0000 7f5752531740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947bd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.647+0000 7f5752531740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.647+0000 7f5752531740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 0x5615947cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.647+0000 7f5752531740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561594722620 con 0x5615947bd860 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.647+0000 7f5752531740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5615946c9e50 con 0x5615947bcf40 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.647+0000 7f5752531740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56159471e5e0 con 0x5615947cc650 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574def4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574def4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44062/0 (socket says 172.21.15.62:44062) 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574def4640 1 -- 172.21.15.62:0/2273539262 learned_addr learned my addr 172.21.15.62:0/2273539262 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574e6f5640 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947bd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574d6f3640 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 0x5615947cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574def4640 1 -- 172.21.15.62:0/2273539262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 msgr2=0x5615947cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574def4640 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 0x5615947cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574def4640 1 -- 172.21.15.62:0/2273539262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 msgr2=0x5615947bd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574def4640 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947bd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574def4640 1 -- 172.21.15.62:0/2273539262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615946bad40 con 0x5615947bd860 2024-09-12T00:06:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f573effd640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f57340a2cf0 0x7f57340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574e6f5640 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574d6f3640 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 0x5615947cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574e6f5640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f57340a2cf0 0x7f57340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f574def4640 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947cc110 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f574400bb70 tx=0x7f574400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45612569c43b1ac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f573effd640 1 -- 172.21.15.62:0/3449408222 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f57380efd60 con 0x7f5748102570 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f5733fff640 1 -- 172.21.15.62:0/2273539262 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744017620 con 0x5615947bd860 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f5733fff640 1 -- 172.21.15.62:0/2273539262 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5744017ca0 con 0x5615947bd860 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.648+0000 7f5733fff640 1 -- 172.21.15.62:0/2273539262 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574400fd10 con 0x5615947bd860 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.649+0000 7f5752531740 1 -- 172.21.15.62:0/2273539262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 msgr2=0x5615947cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.649+0000 7f574e6f5640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f57340a2cf0 0x7f57340a51b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5740005fd0 tx=0x7f57400094a0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.649+0000 7f5752531740 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947cc110 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f574400bb70 tx=0x7f574400d710 comp rx=0 tx=0).stop 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.649+0000 7f5752531740 1 -- 172.21.15.62:0/2273539262 shutdown_connections 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.649+0000 7f5752531740 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 0x5615947cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.649+0000 7f5752531740 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947cc110 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.649+0000 7f5752531740 1 --2- 172.21.15.62:0/2273539262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.649+0000 7f5752531740 1 -- 172.21.15.62:0/2273539262 >> 172.21.15.62:0/2273539262 conn(0x5615947b95f0 msgr2=0x5615947b99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.649+0000 7f5752531740 1 -- 172.21.15.62:0/2273539262 shutdown_connections 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.649+0000 7f5752531740 1 -- 172.21.15.62:0/2273539262 wait complete. 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f5752531740 1 -- start start 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f5752531740 1 Processor -- start 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f5752531740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947bcf40 0x5615947c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f5752531740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f5752531740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947cc650 0x5615947c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f5752531740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5615947cf220 con 0x5615947bd860 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f5752531740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5615946aaf10 con 0x5615947cc650 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f5752531740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615947cf3a0 con 0x5615947bcf40 2024-09-12T00:06:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f5752531740 10 client.0 _ll_register_callbacks cb 0x5615946cedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f574d6f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f574d6f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44064/0 (socket says 172.21.15.62:44064) 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.650+0000 7f574d6f3640 1 -- 172.21.15.62:0/2737847354 learned_addr learned my addr 172.21.15.62:0/2737847354 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f574def4640 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947bcf40 0x5615947c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f574e6f5640 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947cc650 0x5615947c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f574d6f3640 1 -- 172.21.15.62:0/2737847354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947bcf40 msgr2=0x5615947c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f574d6f3640 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947bcf40 0x5615947c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f574d6f3640 1 -- 172.21.15.62:0/2737847354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947cc650 msgr2=0x5615947c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f574d6f3640 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947cc650 0x5615947c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f574d6f3640 1 -- 172.21.15.62:0/2737847354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561594431580 con 0x5615947bd860 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f574d6f3640 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947c99d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f57380c50b0 tx=0x7f57380bc110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb26f83d0a62cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f57317fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f57317fa640 1 -- 172.21.15.62:0/2737847354 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738049070 con 0x5615947bd860 2024-09-12T00:06:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f57317fa640 1 -- 172.21.15.62:0/2737847354 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f573803b460 con 0x5615947bd860 2024-09-12T00:06:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f5752531740 1 -- 172.21.15.62:0/2737847354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5744017ca0 con 0x5615947bd860 2024-09-12T00:06:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f57317fa640 1 -- 172.21.15.62:0/2737847354 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5738031de0 con 0x5615947bd860 2024-09-12T00:06:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f5752531740 10 client.18042 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f5752531740 1 -- 172.21.15.62:0/2737847354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5615944fb960 con 0x5615947bd860 2024-09-12T00:06:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.651+0000 7f57317fa640 1 -- 172.21.15.62:0/2737847354 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5744017ca0 con 0x5615947bd860 2024-09-12T00:06:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.652+0000 7f57317fa640 1 -- 172.21.15.62:0/2737847354 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f57380bc4e0 con 0x5615947bd860 2024-09-12T00:06:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.652+0000 7f5752531740 10 client.18042 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.652+0000 7f5752531740 10 client.18042 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.652+0000 7f5752531740 20 client.18042 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.652+0000 7f5752531740 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615947dbfe0 0x5615947de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.652+0000 7f5752531740 4 client.18042 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.652+0000 7f5752531740 1 -- 172.21.15.62:0/2737847354 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56159472fb90 con 0x5615947dbfe0 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.652+0000 7f574def4640 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615947dbfe0 0x5615947de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.652+0000 7f57317fa640 10 client.18042 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.652+0000 7f574def4640 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615947dbfe0 0x5615947de3d0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c1dcbf5c023617 server_cookie=482bfffb60b84a2 in_seq=0 out_seq=0 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.653+0000 7f57317fa640 1 -- 172.21.15.62:0/2737847354 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56159472fb90 con 0x5615947dbfe0 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.653+0000 7f57317fa640 10 client.18042 handle_command_reply: tid=0 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.653+0000 7f5752531740 1 client.18042 shutdown 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.653+0000 7f5752531740 1 -- 172.21.15.62:0/2737847354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 msgr2=0x5615947c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.653+0000 7f5752531740 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947c99d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f57380c50b0 tx=0x7f57380bc110 comp rx=0 tx=0).stop 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.654+0000 7f5752531740 1 -- 172.21.15.62:0/2737847354 shutdown_connections 2024-09-12T00:06:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.654+0000 7f5752531740 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5615947dbfe0 0x5615947de3d0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.654+0000 7f5752531740 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947cc650 0x5615947c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.654+0000 7f5752531740 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947c99d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.654+0000 7f57317fa640 0 client.18042 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.654+0000 7f5752531740 1 --2- 172.21.15.62:0/2737847354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947bcf40 0x5615947c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.654+0000 7f5752531740 1 -- 172.21.15.62:0/2737847354 >> 172.21.15.62:0/2737847354 conn(0x5615947b95f0 msgr2=0x5615947bf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.654+0000 7f5752531740 1 -- 172.21.15.62:0/2737847354 shutdown_connections 2024-09-12T00:06:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.654+0000 7f5752531740 1 -- 172.21.15.62:0/2737847354 wait complete. 2024-09-12T00:06:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.654+0000 7f5752531740 20 client.18042 trim_cache size 0 max 16384 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f5752531740 1 Processor -- start 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f5752531740 1 -- start start 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f5752531740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f5752531740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947bd860 0x5615947e4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f5752531740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947cc650 0x5615947c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f5752531740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f57481ade20 con 0x5615947cc650 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f5752531740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57481ab6f0 con 0x5615947bcf40 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f5752531740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57481a9540 con 0x5615947bd860 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f574e6f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947cc650 0x5615947c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f574e6f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947cc650 0x5615947c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44080/0 (socket says 172.21.15.62:44080) 2024-09-12T00:06:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f574e6f5640 1 -- 172.21.15.62:0/4121289428 learned_addr learned my addr 172.21.15.62:0/4121289428 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f574def4640 1 --2- 172.21.15.62:0/4121289428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.671+0000 7f574d6f3640 1 --2- 172.21.15.62:0/4121289428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947bd860 0x5615947e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.672+0000 7f574e6f5640 1 -- 172.21.15.62:0/4121289428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947bd860 msgr2=0x5615947e4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.672+0000 7f574e6f5640 1 --2- 172.21.15.62:0/4121289428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947bd860 0x5615947e4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.672+0000 7f574e6f5640 1 -- 172.21.15.62:0/4121289428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 msgr2=0x5615947e3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.672+0000 7f574e6f5640 1 --2- 172.21.15.62:0/4121289428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947e3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.672+0000 7f574e6f5640 1 -- 172.21.15.62:0/4121289428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615947de740 con 0x5615947cc650 2024-09-12T00:06:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.672+0000 7f574def4640 1 --2- 172.21.15.62:0/4121289428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:06:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.672+0000 7f574e6f5640 1 --2- 172.21.15.62:0/4121289428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947cc650 0x5615947c66f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5740012660 tx=0x7f5740014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922c8db00d64665b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.672+0000 7f57337fe640 1 -- 172.21.15.62:0/4121289428 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574001e770 con 0x5615947cc650 2024-09-12T00:06:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.672+0000 7f57337fe640 1 -- 172.21.15.62:0/4121289428 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f574001edf0 con 0x5615947cc650 2024-09-12T00:06:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.672+0000 7f57337fe640 1 -- 172.21.15.62:0/4121289428 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740028380 con 0x5615947cc650 2024-09-12T00:06:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 -- 172.21.15.62:0/4121289428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947cc650 msgr2=0x5615947c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 --2- 172.21.15.62:0/4121289428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947cc650 0x5615947c66f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5740012660 tx=0x7f5740014000 comp rx=0 tx=0).stop 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 -- 172.21.15.62:0/4121289428 shutdown_connections 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 --2- 172.21.15.62:0/4121289428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947cc650 0x5615947c66f0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 --2- 172.21.15.62:0/4121289428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947bd860 0x5615947e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 --2- 172.21.15.62:0/4121289428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 -- 172.21.15.62:0/4121289428 >> 172.21.15.62:0/4121289428 conn(0x5615947b95f0 msgr2=0x5615947c7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 -- 172.21.15.62:0/4121289428 shutdown_connections 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 -- 172.21.15.62:0/4121289428 wait complete. 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 -- start start 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.673+0000 7f5752531740 1 Processor -- start 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f5752531740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f5752531740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f5752531740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 0x5615947d42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f5752531740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f57481a9bb0 con 0x5615947bd860 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f5752531740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57481ab020 con 0x5615947bcf40 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f5752531740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57481aa280 con 0x5615947cc650 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f5752531740 10 client.0 _ll_register_callbacks cb 0x7f57380b8520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574d6f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574d6f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947d1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44086/0 (socket says 172.21.15.62:44086) 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574d6f3640 1 -- 172.21.15.62:0/1304087987 learned_addr learned my addr 172.21.15.62:0/1304087987 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574e6f5640 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 0x5615947d42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574def4640 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574d6f3640 1 -- 172.21.15.62:0/1304087987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 msgr2=0x5615947d42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574d6f3640 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 0x5615947d42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574d6f3640 1 -- 172.21.15.62:0/1304087987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 msgr2=0x5615947d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574d6f3640 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574d6f3640 1 -- 172.21.15.62:0/1304087987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615947d4990 con 0x5615947bd860 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f574d6f3640 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947d1230 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f57380bc850 tx=0x7f57380d5150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61454402900762b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f57317fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.674+0000 7f57317fa640 1 -- 172.21.15.62:0/1304087987 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738049040 con 0x5615947bd860 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f57317fa640 1 -- 172.21.15.62:0/1304087987 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f57380dc070 con 0x5615947bd860 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f5752531740 1 -- 172.21.15.62:0/1304087987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f574001edf0 con 0x5615947bd860 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f57317fa640 1 -- 172.21.15.62:0/1304087987 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f57380bc980 con 0x5615947bd860 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f5752531740 10 client.18048 fetch_fsmap learned FSMap version 39 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f5752531740 1 -- 172.21.15.62:0/1304087987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5615948476a0 con 0x5615947bd860 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f57317fa640 1 -- 172.21.15.62:0/1304087987 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f574001edf0 con 0x5615947bd860 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f57317fa640 1 -- 172.21.15.62:0/1304087987 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f57380bb6d0 con 0x5615947bd860 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f5752531740 10 client.18048 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:06:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f5752531740 10 client.18048 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:06:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f5752531740 20 client.18048 populate_metadata read hostname 'smithi062' 2024-09-12T00:06:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f5752531740 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561594847c90 0x56159484a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:06:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f5752531740 4 client.18048 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3dc716a2", "release": true, "await": true}] 2024-09-12T00:06:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f5752531740 1 -- 172.21.15.62:0/1304087987 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3dc716a2", "release": true, "await": true}) -- 0x56159472a0a0 con 0x561594847c90 2024-09-12T00:06:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.675+0000 7f574def4640 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561594847c90 0x56159484a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:06:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.676+0000 7f57317fa640 10 client.18048 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.676+0000 7f574def4640 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561594847c90 0x56159484a0b0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f78be4e62499020 server_cookie=af3d490084151c96 in_seq=0 out_seq=0 2024-09-12T00:06:38.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.734+0000 7f57317fa640 1 -- 172.21.15.62:0/1304087987 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56159472a0a0 con 0x561594847c90 2024-09-12T00:06:38.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.734+0000 7f57317fa640 10 client.18048 handle_command_reply: tid=0 2024-09-12T00:06:38.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.734+0000 7f5752531740 1 client.18048 shutdown 2024-09-12T00:06:38.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f5752531740 1 -- 172.21.15.62:0/1304087987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 msgr2=0x5615947d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:38.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f5752531740 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947d1230 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f57380bc850 tx=0x7f57380d5150 comp rx=0 tx=0).stop 2024-09-12T00:06:38.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f5752531740 1 -- 172.21.15.62:0/1304087987 shutdown_connections 2024-09-12T00:06:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f5752531740 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x561594847c90 0x56159484a0b0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f5752531740 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615947cc650 0x5615947d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f57317fa640 0 client.18048 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:06:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f5752531740 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615947bd860 0x5615947d1230 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f5752531740 1 --2- 172.21.15.62:0/1304087987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615947bcf40 0x5615947d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f5752531740 1 -- 172.21.15.62:0/1304087987 >> 172.21.15.62:0/1304087987 conn(0x5615947b95f0 msgr2=0x5615947b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f5752531740 1 -- 172.21.15.62:0/1304087987 shutdown_connections 2024-09-12T00:06:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.735+0000 7f5752531740 1 -- 172.21.15.62:0/1304087987 wait complete. 2024-09-12T00:06:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.736+0000 7f5752531740 20 client.18048 trim_cache size 0 max 16384 2024-09-12T00:06:38.736 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 39, 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 56, 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "3dc716a2": { 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 56, 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:06:38.737 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:06:38.738 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:06:38.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 -- 172.21.15.62:0/3449408222 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f57340a2cf0 msgr2=0x7f57340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:38.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f57340a2cf0 0x7f57340a51b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5740005fd0 tx=0x7f57400094a0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 -- 172.21.15.62:0/3449408222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5748102570 msgr2=0x7f5748199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:06:38.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5748102570 0x7f5748199850 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5738006010 tx=0x7f5738003c10 comp rx=0 tx=0).stop 2024-09-12T00:06:38.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 -- 172.21.15.62:0/3449408222 shutdown_connections 2024-09-12T00:06:38.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f57340a2cf0 0x7f57340a51b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5748102ef0 0x7f57481a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5748102570 0x7f5748199850 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 --2- 172.21.15.62:0/3449408222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5748100e20 0x7f5748199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:06:38.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 -- 172.21.15.62:0/3449408222 >> 172.21.15.62:0/3449408222 conn(0x7f57480fc7d0 msgr2=0x7f57480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:06:38.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 -- 172.21.15.62:0/3449408222 shutdown_connections 2024-09-12T00:06:38.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:06:38.737+0000 7f5750158640 1 -- 172.21.15.62:0/3449408222 wait complete. 2024-09-12T00:06:39.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:39 smithi062 ceph-mon[30958]: pgmap v453: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 237 KiB/s wr, 36 op/s 2024-09-12T00:06:39.483 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3dc716a2', total seconds elapsed: 33.2 2024-09-12T00:06:39.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:39 smithi083 ceph-mon[34805]: pgmap v453: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 237 KiB/s wr, 36 op/s 2024-09-12T00:06:39.672 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:39 smithi120 ceph-mon[35182]: pgmap v453: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 237 KiB/s wr, 36 op/s 2024-09-12T00:06:41.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:41 smithi083 ceph-mon[34805]: pgmap v454: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-12T00:06:41.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:41 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:06:41.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:41 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:06:41.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:41 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:06:41.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:41 smithi062 ceph-mon[30958]: pgmap v454: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-12T00:06:41.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:41 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:06:41.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:41 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:06:41.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:41 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:06:41.877 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:41 smithi120 ceph-mon[35182]: pgmap v454: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-12T00:06:41.877 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:41 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:06:41.877 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:41 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:06:41.877 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:41 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:06:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 840.0 s, 1338.5 tps, lat 492.691 ms stddev 3550.960 2024-09-12T00:06:43.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:43 smithi083 ceph-mon[34805]: pgmap v455: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-12T00:06:43.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:43 smithi062 ceph-mon[30958]: pgmap v455: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-12T00:06:43.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:43 smithi120 ceph-mon[35182]: pgmap v455: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-12T00:06:45.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:45 smithi083 ceph-mon[34805]: pgmap v456: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.1 MiB/s wr, 300 op/s 2024-09-12T00:06:45.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:45 smithi062 ceph-mon[30958]: pgmap v456: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.1 MiB/s wr, 300 op/s 2024-09-12T00:06:45.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:45 smithi120 ceph-mon[35182]: pgmap v456: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.1 MiB/s wr, 300 op/s 2024-09-12T00:06:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 845.0 s, 2007.8 tps, lat 49.372 ms stddev 56.979 2024-09-12T00:06:47.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:47 smithi083 ceph-mon[34805]: pgmap v457: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.1 MiB/s wr, 296 op/s 2024-09-12T00:06:47.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:47 smithi062 ceph-mon[30958]: pgmap v457: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.1 MiB/s wr, 296 op/s 2024-09-12T00:06:47.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:47 smithi120 ceph-mon[35182]: pgmap v457: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.1 MiB/s wr, 296 op/s 2024-09-12T00:06:49.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:49 smithi083 ceph-mon[34805]: pgmap v458: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 15 MiB/s wr, 553 op/s 2024-09-12T00:06:49.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:49 smithi062 ceph-mon[30958]: pgmap v458: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 15 MiB/s wr, 553 op/s 2024-09-12T00:06:49.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:49 smithi120 ceph-mon[35182]: pgmap v458: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 15 MiB/s wr, 553 op/s 2024-09-12T00:06:51.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:51 smithi083 ceph-mon[34805]: pgmap v459: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 15 MiB/s wr, 593 op/s 2024-09-12T00:06:51.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:51 smithi062 ceph-mon[30958]: pgmap v459: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 15 MiB/s wr, 593 op/s 2024-09-12T00:06:51.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:51 smithi120 ceph-mon[35182]: pgmap v459: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 15 MiB/s wr, 593 op/s 2024-09-12T00:06:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 850.0 s, 2030.4 tps, lat 49.314 ms stddev 55.348 2024-09-12T00:06:53.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:53 smithi083 ceph-mon[34805]: pgmap v460: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 15 MiB/s wr, 593 op/s 2024-09-12T00:06:53.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:53 smithi062 ceph-mon[30958]: pgmap v460: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 15 MiB/s wr, 593 op/s 2024-09-12T00:06:53.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:53 smithi120 ceph-mon[35182]: pgmap v460: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 15 MiB/s wr, 593 op/s 2024-09-12T00:06:55.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:55 smithi083 ceph-mon[34805]: pgmap v461: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 22 MiB/s wr, 859 op/s 2024-09-12T00:06:55.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:55 smithi062 ceph-mon[30958]: pgmap v461: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 22 MiB/s wr, 859 op/s 2024-09-12T00:06:55.873 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:55 smithi120 ceph-mon[35182]: pgmap v461: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 22 MiB/s wr, 859 op/s 2024-09-12T00:06:57.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 855.0 s, 2228.2 tps, lat 45.152 ms stddev 53.483 2024-09-12T00:06:57.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:57 smithi083 ceph-mon[34805]: pgmap v462: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 15 MiB/s wr, 565 op/s 2024-09-12T00:06:57.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:57 smithi062 ceph-mon[30958]: pgmap v462: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 15 MiB/s wr, 565 op/s 2024-09-12T00:06:57.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:57 smithi120 ceph-mon[35182]: pgmap v462: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 15 MiB/s wr, 565 op/s 2024-09-12T00:06:59.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:06:59 smithi083 ceph-mon[34805]: pgmap v463: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 20 MiB/s wr, 795 op/s 2024-09-12T00:06:59.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:06:59 smithi062 ceph-mon[30958]: pgmap v463: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 20 MiB/s wr, 795 op/s 2024-09-12T00:06:59.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:06:59 smithi120 ceph-mon[35182]: pgmap v463: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 20 MiB/s wr, 795 op/s 2024-09-12T00:07:01.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:01 smithi083 ceph-mon[34805]: pgmap v464: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 584 op/s 2024-09-12T00:07:01.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:01 smithi062 ceph-mon[30958]: pgmap v464: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 584 op/s 2024-09-12T00:07:01.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:01 smithi120 ceph-mon[35182]: pgmap v464: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 584 op/s 2024-09-12T00:07:02.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 860.0 s, 1311.2 tps, lat 74.880 ms stddev 97.272 2024-09-12T00:07:03.666 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:03 smithi083 ceph-mon[34805]: pgmap v465: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 541 op/s 2024-09-12T00:07:03.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:03 smithi062 ceph-mon[30958]: pgmap v465: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 541 op/s 2024-09-12T00:07:03.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:03 smithi120 ceph-mon[35182]: pgmap v465: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 541 op/s 2024-09-12T00:07:03.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:07:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:05.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:05 smithi062 ceph-mon[30958]: pgmap v466: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 17 MiB/s wr, 767 op/s 2024-09-12T00:07:05.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:05 smithi120 ceph-mon[35182]: pgmap v466: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 17 MiB/s wr, 767 op/s 2024-09-12T00:07:05.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:05 smithi083 ceph-mon[34805]: pgmap v466: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 17 MiB/s wr, 767 op/s 2024-09-12T00:07:06.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.249+0000 7f19feaf7640 1 -- 172.21.15.62:0/3918201643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f81036e0 msgr2=0x7f19f8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:06.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.249+0000 7f19feaf7640 1 --2- 172.21.15.62:0/3918201643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f81036e0 0x7f19f8107b90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009cd0 tx=0x7f19e802a8e0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.249+0000 7f19feaf7640 1 -- 172.21.15.62:0/3918201643 shutdown_connections 2024-09-12T00:07:06.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.249+0000 7f19feaf7640 1 --2- 172.21.15.62:0/3918201643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f81082a0 0x7f19f810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.249+0000 7f19feaf7640 1 --2- 172.21.15.62:0/3918201643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f81036e0 0x7f19f8107b90 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.249+0000 7f19feaf7640 1 --2- 172.21.15.62:0/3918201643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f8102d30 0x7f19f8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.249+0000 7f19feaf7640 1 -- 172.21.15.62:0/3918201643 >> 172.21.15.62:0/3918201643 conn(0x7f19f80fc7d0 msgr2=0x7f19f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:06.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19feaf7640 1 -- 172.21.15.62:0/3918201643 shutdown_connections 2024-09-12T00:07:06.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19feaf7640 1 -- 172.21.15.62:0/3918201643 wait complete. 2024-09-12T00:07:06.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19feaf7640 1 Processor -- start 2024-09-12T00:07:06.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19feaf7640 1 -- start start 2024-09-12T00:07:06.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19feaf7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f8102d30 0x7f19f8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19feaf7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f81036e0 0x7f19f8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19feaf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f81082a0 0x7f19f8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19feaf7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81120a0 con 0x7f19f8102d30 2024-09-12T00:07:06.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19feaf7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19f8111f20 con 0x7f19f81082a0 2024-09-12T00:07:06.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19feaf7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f19f8112220 con 0x7f19f81036e0 2024-09-12T00:07:06.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.250+0000 7f19fc893640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f8102d30 0x7f19f8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19fc893640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f8102d30 0x7f19f8100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45704/0 (socket says 172.21.15.62:45704) 2024-09-12T00:07:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f81036e0 0x7f19f8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19fc893640 1 -- 172.21.15.62:0/1181209913 learned_addr learned my addr 172.21.15.62:0/1181209913 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19fd094640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f81082a0 0x7f19f8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19fc893640 1 -- 172.21.15.62:0/1181209913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f81036e0 msgr2=0x7f19f8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19fc893640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f81036e0 0x7f19f8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19fc893640 1 -- 172.21.15.62:0/1181209913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f81082a0 msgr2=0x7f19f8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:06.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19fc893640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f81082a0 0x7f19f8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19fc893640 1 -- 172.21.15.62:0/1181209913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f81a6aa0 con 0x7f19f8102d30 2024-09-12T00:07:06.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19fc893640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f8102d30 0x7f19f8100990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f19f000d770 tx=0x7f19f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:06.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19f5ffb640 1 -- 172.21.15.62:0/1181209913 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0017070 con 0x7f19f8102d30 2024-09-12T00:07:06.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19f5ffb640 1 -- 172.21.15.62:0/1181209913 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f0005500 con 0x7f19f8102d30 2024-09-12T00:07:06.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19feaf7640 1 -- 172.21.15.62:0/1181209913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f81a6d90 con 0x7f19f8102d30 2024-09-12T00:07:06.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.251+0000 7f19f5ffb640 1 -- 172.21.15.62:0/1181209913 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0004e00 con 0x7f19f8102d30 2024-09-12T00:07:06.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.252+0000 7f19feaf7640 1 -- 172.21.15.62:0/1181209913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f81a7240 con 0x7f19f8102d30 2024-09-12T00:07:06.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.254+0000 7f19f5ffb640 1 -- 172.21.15.62:0/1181209913 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f19f0004fa0 con 0x7f19f8102d30 2024-09-12T00:07:06.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.254+0000 7f1a00ed0740 1 Processor -- start 2024-09-12T00:07:06.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.254+0000 7f1a00ed0740 1 -- start start 2024-09-12T00:07:06.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.254+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b01880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.254+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b0c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.254+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.254+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562df4a672e0 con 0x562df4b0c680 2024-09-12T00:07:06.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.254+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562df4a0ece0 con 0x562df4b01dc0 2024-09-12T00:07:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.254+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562df4a632a0 con 0x562df4b014a0 2024-09-12T00:07:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19f5ffb640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19e40a2d70 0x7f19e40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b01880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b01880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40712/0 (socket says 172.21.15.62:40712) 2024-09-12T00:07:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 -- 172.21.15.62:0/2162152174 learned_addr learned my addr 172.21.15.62:0/2162152174 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fc893640 1 --2- 172.21.15.62:0/2162152174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b0c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19f5ffb640 1 -- 172.21.15.62:0/1181209913 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f19f00ced60 con 0x7f19f8102d30 2024-09-12T00:07:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19e40a2d70 0x7f19e40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 -- 172.21.15.62:0/2162152174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 msgr2=0x562df4b0c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 --2- 172.21.15.62:0/2162152174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b0c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 -- 172.21.15.62:0/2162152174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 msgr2=0x562df4b0ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 --2- 172.21.15.62:0/2162152174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b0ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 -- 172.21.15.62:0/2162152174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df49f3440 con 0x562df4b014a0 2024-09-12T00:07:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 --2- 172.21.15.62:0/2162152174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b01880 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f19e000bd20 tx=0x7f19e00039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=576840949d3008ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19eeffd640 1 -- 172.21.15.62:0/2162152174 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e000b700 con 0x562df4b014a0 2024-09-12T00:07:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19eeffd640 1 -- 172.21.15.62:0/2162152174 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e00054b0 con 0x562df4b014a0 2024-09-12T00:07:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19eeffd640 1 -- 172.21.15.62:0/2162152174 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e001ecb0 con 0x562df4b014a0 2024-09-12T00:07:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.255+0000 7f19fd094640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19e40a2d70 0x7f19e40a5230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f19e0021000 tx=0x7f19e0020000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.256+0000 7f1a00ed0740 1 -- 172.21.15.62:0/2162152174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 msgr2=0x562df4b01880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.256+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/2162152174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b01880 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f19e000bd20 tx=0x7f19e00039c0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.256+0000 7f1a00ed0740 1 -- 172.21.15.62:0/2162152174 shutdown_connections 2024-09-12T00:07:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.256+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/2162152174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.256+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/2162152174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.256+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/2162152174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b01880 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.256+0000 7f1a00ed0740 1 -- 172.21.15.62:0/2162152174 >> 172.21.15.62:0/2162152174 conn(0x562df4afda30 msgr2=0x562df4afde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.256+0000 7f1a00ed0740 1 -- 172.21.15.62:0/2162152174 shutdown_connections 2024-09-12T00:07:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.256+0000 7f1a00ed0740 1 -- 172.21.15.62:0/2162152174 wait complete. 2024-09-12T00:07:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f1a00ed0740 1 -- start start 2024-09-12T00:07:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f1a00ed0740 1 Processor -- start 2024-09-12T00:07:06.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b0bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b09200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b09740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562df4b0f250 con 0x562df4b0c680 2024-09-12T00:07:06.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562df4a22050 con 0x562df4b01dc0 2024-09-12T00:07:06.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562df4b0f3d0 con 0x562df4b014a0 2024-09-12T00:07:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f19fd094640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b09740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f1a00ed0740 10 client.0 _ll_register_callbacks cb 0x562df4a62160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f19fd094640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b09740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45734/0 (socket says 172.21.15.62:45734) 2024-09-12T00:07:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f19fd094640 1 -- 172.21.15.62:0/1151759082 learned_addr learned my addr 172.21.15.62:0/1151759082 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f19fc893640 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b0bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.257+0000 7f19f7fff640 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b09200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f19f7fff640 1 -- 172.21.15.62:0/1151759082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 msgr2=0x562df4b0bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f19f7fff640 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b0bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f19f7fff640 1 -- 172.21.15.62:0/1151759082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 msgr2=0x562df4b09740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f19f7fff640 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b09740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f19f7fff640 1 -- 172.21.15.62:0/1151759082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df47d83c0 con 0x562df4b01dc0 2024-09-12T00:07:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f19fd094640 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b09740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f19f7fff640 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b09200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f19e802ae00 tx=0x7f19e8031c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=946e0817b5f502bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f19d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:07:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f19d7fff640 1 -- 172.21.15.62:0/1151759082 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8048020 con 0x562df4b01dc0 2024-09-12T00:07:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f1a00ed0740 1 -- 172.21.15.62:0/1151759082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e00054b0 con 0x562df4b01dc0 2024-09-12T00:07:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.258+0000 7f19d7fff640 1 -- 172.21.15.62:0/1151759082 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f19e8003b80 con 0x562df4b01dc0 2024-09-12T00:07:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f19d7fff640 1 -- 172.21.15.62:0/1151759082 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f19e8031d20 con 0x562df4b01dc0 2024-09-12T00:07:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f1a00ed0740 10 client.27500 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f1a00ed0740 1 -- 172.21.15.62:0/1151759082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562df4b1abf0 con 0x562df4b01dc0 2024-09-12T00:07:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f19d7fff640 1 -- 172.21.15.62:0/1151759082 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e00054b0 con 0x562df4b01dc0 2024-09-12T00:07:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f19d7fff640 1 -- 172.21.15.62:0/1151759082 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f19e8043070 con 0x562df4b01dc0 2024-09-12T00:07:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f1a00ed0740 10 client.27500 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f1a00ed0740 10 client.27500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:07:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f1a00ed0740 20 client.27500 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562df4b1c050 0x562df4b1e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f1a00ed0740 4 client.27500 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:07:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f1a00ed0740 1 -- 172.21.15.62:0/1151759082 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562df4a74850 con 0x562df4b1c050 2024-09-12T00:07:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.259+0000 7f19fc893640 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562df4b1c050 0x562df4b1e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f19fc893640 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562df4b1c050 0x562df4b1e460 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b7d571f960c6b9 server_cookie=1ea2d7d5af8546f0 in_seq=0 out_seq=0 2024-09-12T00:07:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f19d7fff640 10 client.27500 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f19d7fff640 1 -- 172.21.15.62:0/1151759082 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562df4a74850 con 0x562df4b1c050 2024-09-12T00:07:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f19d7fff640 10 client.27500 handle_command_reply: tid=0 2024-09-12T00:07:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f1a00ed0740 1 client.27500 shutdown 2024-09-12T00:07:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f1a00ed0740 1 -- 172.21.15.62:0/1151759082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 msgr2=0x562df4b09200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b09200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f19e802ae00 tx=0x7f19e8031c30 comp rx=0 tx=0).stop 2024-09-12T00:07:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f1a00ed0740 1 -- 172.21.15.62:0/1151759082 shutdown_connections 2024-09-12T00:07:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562df4b1c050 0x562df4b1e460 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b09740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b09200 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f19d7fff640 0 client.27500 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:06.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/1151759082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b0bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.260+0000 7f1a00ed0740 1 -- 172.21.15.62:0/1151759082 >> 172.21.15.62:0/1151759082 conn(0x562df4afda30 msgr2=0x562df4b03710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:06.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.261+0000 7f1a00ed0740 1 -- 172.21.15.62:0/1151759082 shutdown_connections 2024-09-12T00:07:06.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.261+0000 7f1a00ed0740 1 -- 172.21.15.62:0/1151759082 wait complete. 2024-09-12T00:07:06.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.261+0000 7f1a00ed0740 20 client.27500 trim_cache size 0 max 16384 2024-09-12T00:07:06.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.278+0000 7f1a00ed0740 1 Processor -- start 2024-09-12T00:07:06.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.278+0000 7f1a00ed0740 1 -- start start 2024-09-12T00:07:06.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.278+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b014a0 0x562df4b184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.278+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b01dc0 0x562df4b04900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.278+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b0c680 0x562df4b081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81b22c0 con 0x562df4b01dc0 2024-09-12T00:07:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81afb90 con 0x562df4b014a0 2024-09-12T00:07:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81ada00 con 0x562df4b0c680 2024-09-12T00:07:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b01dc0 0x562df4b04900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b01dc0 0x562df4b04900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45744/0 (socket says 172.21.15.62:45744) 2024-09-12T00:07:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19f7fff640 1 -- 172.21.15.62:0/907948590 learned_addr learned my addr 172.21.15.62:0/907948590 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19fd094640 1 --2- 172.21.15.62:0/907948590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b0c680 0x562df4b081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19fc893640 1 --2- 172.21.15.62:0/907948590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b014a0 0x562df4b184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19f7fff640 1 -- 172.21.15.62:0/907948590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b0c680 msgr2=0x562df4b081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19f7fff640 1 --2- 172.21.15.62:0/907948590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b0c680 0x562df4b081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19f7fff640 1 -- 172.21.15.62:0/907948590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b014a0 msgr2=0x562df4b184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19f7fff640 1 --2- 172.21.15.62:0/907948590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b014a0 0x562df4b184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19f7fff640 1 -- 172.21.15.62:0/907948590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df4b01bf0 con 0x562df4b01dc0 2024-09-12T00:07:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19f7fff640 1 --2- 172.21.15.62:0/907948590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b01dc0 0x562df4b04900 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f19e8030940 tx=0x7f19e802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d9efd962be14fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19ee7fc640 1 -- 172.21.15.62:0/907948590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8048040 con 0x562df4b01dc0 2024-09-12T00:07:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19ee7fc640 1 -- 172.21.15.62:0/907948590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8036040 con 0x562df4b01dc0 2024-09-12T00:07:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.279+0000 7f19ee7fc640 1 -- 172.21.15.62:0/907948590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8055920 con 0x562df4b01dc0 2024-09-12T00:07:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.280+0000 7f1a00ed0740 1 -- 172.21.15.62:0/907948590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b01dc0 msgr2=0x562df4b04900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.280+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/907948590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b01dc0 0x562df4b04900 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f19e8030940 tx=0x7f19e802a8e0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.280+0000 7f1a00ed0740 1 -- 172.21.15.62:0/907948590 shutdown_connections 2024-09-12T00:07:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.280+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/907948590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b0c680 0x562df4b081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.280+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/907948590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b01dc0 0x562df4b04900 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.280+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/907948590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b014a0 0x562df4b184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.280+0000 7f1a00ed0740 1 -- 172.21.15.62:0/907948590 >> 172.21.15.62:0/907948590 conn(0x562df4afda30 msgr2=0x562df4b07ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:06.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.280+0000 7f1a00ed0740 1 -- 172.21.15.62:0/907948590 shutdown_connections 2024-09-12T00:07:06.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.280+0000 7f1a00ed0740 1 -- 172.21.15.62:0/907948590 wait complete. 2024-09-12T00:07:06.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.280+0000 7f1a00ed0740 1 -- start start 2024-09-12T00:07:06.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f1a00ed0740 1 Processor -- start 2024-09-12T00:07:06.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b14c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b0ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f1a00ed0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81ae050 con 0x562df4b0c680 2024-09-12T00:07:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81af4c0 con 0x562df4b01dc0 2024-09-12T00:07:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f1a00ed0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81ae720 con 0x562df4b014a0 2024-09-12T00:07:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f1a00ed0740 10 client.0 _ll_register_callbacks cb 0x7f19e80098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b0ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19fd094640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19fc893640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b14c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b0ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:47800/0 (socket says 172.21.15.62:47800) 2024-09-12T00:07:06.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19f7fff640 1 -- 172.21.15.62:0/3423601902 learned_addr learned my addr 172.21.15.62:0/3423601902 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:06.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19fd094640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b104b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45756/0 (socket says 172.21.15.62:45756) 2024-09-12T00:07:06.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19f7fff640 1 -- 172.21.15.62:0/3423601902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 msgr2=0x562df4b14c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:06.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19f7fff640 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b14c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19f7fff640 1 -- 172.21.15.62:0/3423601902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 msgr2=0x562df4b104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:06.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19f7fff640 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19f7fff640 1 -- 172.21.15.62:0/3423601902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df4b10ba0 con 0x562df4b01dc0 2024-09-12T00:07:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.281+0000 7f19fc893640 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b14c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.282+0000 7f19fd094640 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.282+0000 7f19f7fff640 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b0ff70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f19e8005e30 tx=0x7f19e8055b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3997601f1bf2a5fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.282+0000 7f19d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:07:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.282+0000 7f19d7fff640 1 -- 172.21.15.62:0/3423601902 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8059070 con 0x562df4b01dc0 2024-09-12T00:07:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.282+0000 7f19d7fff640 1 -- 172.21.15.62:0/3423601902 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f19e803b610 con 0x562df4b01dc0 2024-09-12T00:07:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.282+0000 7f1a00ed0740 1 -- 172.21.15.62:0/3423601902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e8036040 con 0x562df4b01dc0 2024-09-12T00:07:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.282+0000 7f19d7fff640 1 -- 172.21.15.62:0/3423601902 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f19e8065050 con 0x562df4b01dc0 2024-09-12T00:07:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.282+0000 7f1a00ed0740 10 client.27506 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.282+0000 7f1a00ed0740 1 -- 172.21.15.62:0/3423601902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562df4b86bd0 con 0x562df4b01dc0 2024-09-12T00:07:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.282+0000 7f19d7fff640 1 -- 172.21.15.62:0/3423601902 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8036040 con 0x562df4b01dc0 2024-09-12T00:07:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f19d7fff640 1 -- 172.21.15.62:0/3423601902 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f19e8053020 con 0x562df4b01dc0 2024-09-12T00:07:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f1a00ed0740 10 client.27506 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f1a00ed0740 10 client.27506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:07:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f1a00ed0740 20 client.27506 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562df4b872e0 0x562df4b89700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f1a00ed0740 4 client.27506 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:07:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f1a00ed0740 1 -- 172.21.15.62:0/3423601902 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562df4a6ed60 con 0x562df4b872e0 2024-09-12T00:07:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f19fc893640 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562df4b872e0 0x562df4b89700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f19fc893640 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562df4b872e0 0x562df4b89700 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88800c4bfb50c987 server_cookie=7288aadf8af380d5 in_seq=0 out_seq=0 2024-09-12T00:07:06.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f19d7fff640 10 client.27506 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:06.403 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:07:06.403 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:07:06.404 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:07:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:07:06.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f19d7fff640 1 -- 172.21.15.62:0/3423601902 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562df4a6ed60 con 0x562df4b872e0 2024-09-12T00:07:06.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f19d7fff640 10 client.27506 handle_command_reply: tid=0 2024-09-12T00:07:06.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f1a00ed0740 1 client.27506 shutdown 2024-09-12T00:07:06.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f1a00ed0740 1 -- 172.21.15.62:0/3423601902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 msgr2=0x562df4b0ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:06.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.283+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b0ff70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f19e8005e30 tx=0x7f19e8055b20 comp rx=0 tx=0).stop 2024-09-12T00:07:06.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f1a00ed0740 1 -- 172.21.15.62:0/3423601902 shutdown_connections 2024-09-12T00:07:06.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x562df4b872e0 0x562df4b89700 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562df4b0c680 0x562df4b104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562df4b01dc0 0x562df4b0ff70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f1a00ed0740 1 --2- 172.21.15.62:0/3423601902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562df4b014a0 0x562df4b14c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f1a00ed0740 1 -- 172.21.15.62:0/3423601902 >> 172.21.15.62:0/3423601902 conn(0x562df4afda30 msgr2=0x562df4afc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:06.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f19d7fff640 0 client.27506 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:06.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f1a00ed0740 1 -- 172.21.15.62:0/3423601902 shutdown_connections 2024-09-12T00:07:06.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f1a00ed0740 1 -- 172.21.15.62:0/3423601902 wait complete. 2024-09-12T00:07:06.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f1a00ed0740 20 client.27506 trim_cache size 0 max 16384 2024-09-12T00:07:06.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f19feaf7640 1 -- 172.21.15.62:0/1181209913 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19e40a2d70 msgr2=0x7f19e40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:06.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f19feaf7640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19e40a2d70 0x7f19e40a5230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f19e0021000 tx=0x7f19e0020000 comp rx=0 tx=0).stop 2024-09-12T00:07:06.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f19feaf7640 1 -- 172.21.15.62:0/1181209913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f8102d30 msgr2=0x7f19f8100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:06.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f19feaf7640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f8102d30 0x7f19f8100990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f19f000d770 tx=0x7f19f000dc30 comp rx=0 tx=0).stop 2024-09-12T00:07:06.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f19feaf7640 1 -- 172.21.15.62:0/1181209913 shutdown_connections 2024-09-12T00:07:06.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f19feaf7640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f19e40a2d70 0x7f19e40a5230 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f19feaf7640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f81082a0 0x7f19f8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f19feaf7640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19f81036e0 0x7f19f8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.284+0000 7f19feaf7640 1 --2- 172.21.15.62:0/1181209913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f8102d30 0x7f19f8100990 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:06.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.285+0000 7f19feaf7640 1 -- 172.21.15.62:0/1181209913 >> 172.21.15.62:0/1181209913 conn(0x7f19f80fc7d0 msgr2=0x7f19f8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:06.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.285+0000 7f19feaf7640 1 -- 172.21.15.62:0/1181209913 shutdown_connections 2024-09-12T00:07:06.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:05.285+0000 7f19feaf7640 1 -- 172.21.15.62:0/1181209913 wait complete. 2024-09-12T00:07:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 865.0 s, 589.4 tps, lat 172.659 ms stddev 299.675 2024-09-12T00:07:07.642 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:07:07.642 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:07:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:07 smithi062 ceph-mon[30958]: pgmap v467: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 501 op/s 2024-09-12T00:07:07.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:07 smithi062 ceph-mon[30958]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T00:07:07.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:07 smithi120 ceph-mon[35182]: pgmap v467: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 501 op/s 2024-09-12T00:07:07.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:07 smithi120 ceph-mon[35182]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T00:07:07.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:07 smithi083 ceph-mon[34805]: pgmap v467: 97 pgs: 97 active+clean; 4.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 501 op/s 2024-09-12T00:07:07.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:07 smithi083 ceph-mon[34805]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T00:07:08.016 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.605+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3189871367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a541011e0 msgr2=0x7f3a54105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.605+0000 7f3a58ee0640 1 --2- 172.21.15.62:0/3189871367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a541011e0 0x7f3a54105980 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009ca0 tx=0x7f3a4c02a8d0 comp rx=0 tx=0).stop 2024-09-12T00:07:08.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.610+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3189871367 shutdown_connections 2024-09-12T00:07:08.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.610+0000 7f3a58ee0640 1 --2- 172.21.15.62:0/3189871367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a54105ec0 0x7f3a54106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:08.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.610+0000 7f3a58ee0640 1 --2- 172.21.15.62:0/3189871367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a541011e0 0x7f3a54105980 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:08.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.610+0000 7f3a58ee0640 1 --2- 172.21.15.62:0/3189871367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a541008c0 0x7f3a54100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:08.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.610+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3189871367 >> 172.21.15.62:0/3189871367 conn(0x7f3a540fc7d0 msgr2=0x7f3a540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:08.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.614+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3189871367 shutdown_connections 2024-09-12T00:07:08.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.614+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3189871367 wait complete. 2024-09-12T00:07:08.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.616+0000 7f3a58ee0640 1 Processor -- start 2024-09-12T00:07:08.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.625+0000 7f3a58ee0640 1 -- start start 2024-09-12T00:07:08.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.626+0000 7f3a58ee0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a541008c0 0x7f3a54073b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:08.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.626+0000 7f3a58ee0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a541011e0 0x7f3a5406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:08.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.626+0000 7f3a58ee0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a54105ec0 0x7f3a5406e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:08.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.626+0000 7f3a58ee0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5406c640 con 0x7f3a541008c0 2024-09-12T00:07:08.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.626+0000 7f3a58ee0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5406c4c0 con 0x7f3a54105ec0 2024-09-12T00:07:08.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.626+0000 7f3a58ee0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5406c7c0 con 0x7f3a541011e0 2024-09-12T00:07:08.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.627+0000 7f3a51d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a541011e0 0x7f3a5406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:08.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.627+0000 7f3a51d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a541011e0 0x7f3a5406dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:40760/0 (socket says 172.21.15.62:40760) 2024-09-12T00:07:08.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.627+0000 7f3a51d9b640 1 -- 172.21.15.62:0/3626383161 learned_addr learned my addr 172.21.15.62:0/3626383161 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:08.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.628+0000 7f3a51d9b640 1 -- 172.21.15.62:0/3626383161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a54105ec0 msgr2=0x7f3a5406e140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:07:08.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.628+0000 7f3a51d9b640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a54105ec0 0x7f3a5406e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:08.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.628+0000 7f3a51d9b640 1 -- 172.21.15.62:0/3626383161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a541008c0 msgr2=0x7f3a54073b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:08.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.629+0000 7f3a51d9b640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a541008c0 0x7f3a54073b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:08.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.629+0000 7f3a51d9b640 1 -- 172.21.15.62:0/3626383161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5406e8c0 con 0x7f3a541011e0 2024-09-12T00:07:08.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.631+0000 7f3a51d9b640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a541011e0 0x7f3a5406dc00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c005e30 tx=0x7f3a4c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:08.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.632+0000 7f3a4b7fe640 1 -- 172.21.15.62:0/3626383161 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c003b70 con 0x7f3a541011e0 2024-09-12T00:07:08.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.632+0000 7f3a4b7fe640 1 -- 172.21.15.62:0/3626383161 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a4c003d10 con 0x7f3a541011e0 2024-09-12T00:07:08.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.632+0000 7f3a4b7fe640 1 -- 172.21.15.62:0/3626383161 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c02fb50 con 0x7f3a541011e0 2024-09-12T00:07:08.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.632+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3626383161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a541b3980 con 0x7f3a541011e0 2024-09-12T00:07:08.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.632+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3626383161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a541b3de0 con 0x7f3a541011e0 2024-09-12T00:07:08.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.636+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3626383161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a340052c0 con 0x7f3a541011e0 2024-09-12T00:07:08.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.638+0000 7f3a4b7fe640 1 -- 172.21.15.62:0/3626383161 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3a4c02fcf0 con 0x7f3a541011e0 2024-09-12T00:07:08.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.640+0000 7f3a4b7fe640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3a400a2f00 0x7f3a400a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:08.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.640+0000 7f3a4b7fe640 1 -- 172.21.15.62:0/3626383161 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8557+0+0 (secure 0 0 0) 0x7f3a4c0f08e0 con 0x7f3a541011e0 2024-09-12T00:07:08.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.644+0000 7f3a5259c640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3a400a2f00 0x7f3a400a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:08.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.646+0000 7f3a5259c640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3a400a2f00 0x7f3a400a53c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3a44005fd0 tx=0x7f3a44005850 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:08.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.646+0000 7f3a4b7fe640 1 -- 172.21.15.62:0/3626383161 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a4c0bd590 con 0x7f3a541011e0 2024-09-12T00:07:08.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.971+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3626383161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a34005880 con 0x7f3a541011e0 2024-09-12T00:07:08.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.972+0000 7f3a4b7fe640 1 -- 172.21.15.62:0/3626383161 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11909 (secure 0 0 0) 0x7f3a4c039e10 con 0x7f3a541011e0 2024-09-12T00:07:08.971 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:08.972 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":39,"btime":"2024-09-12T00:06:00:989943+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:06:00.711094+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:07:08.972 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 39 2024-09-12T00:07:08.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3626383161 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3a400a2f00 msgr2=0x7f3a400a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3a400a2f00 0x7f3a400a53c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3a44005fd0 tx=0x7f3a44005850 comp rx=0 tx=0).stop 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3626383161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a541011e0 msgr2=0x7f3a5406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a541011e0 0x7f3a5406dc00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c005e30 tx=0x7f3a4c031a20 comp rx=0 tx=0).stop 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3626383161 shutdown_connections 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f3a400a2f00 0x7f3a400a53c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a54105ec0 0x7f3a5406e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a541011e0 0x7f3a5406dc00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 --2- 172.21.15.62:0/3626383161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a541008c0 0x7f3a54073b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3626383161 >> 172.21.15.62:0/3626383161 conn(0x7f3a540fc7d0 msgr2=0x7f3a5410f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3626383161 shutdown_connections 2024-09-12T00:07:08.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:08.975+0000 7f3a58ee0640 1 -- 172.21.15.62:0/3626383161 wait complete. 2024-09-12T00:07:09.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:09 smithi062 ceph-mon[30958]: pgmap v469: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2024-09-12T00:07:09.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:09 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3626383161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:07:09.718 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:09 smithi083 ceph-mon[34805]: pgmap v469: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2024-09-12T00:07:09.718 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:09 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3626383161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:07:09.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:09 smithi120 ceph-mon[35182]: pgmap v469: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2024-09-12T00:07:09.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:09 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3626383161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:07:10.702 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:07:11.114 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:11.573 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:11 smithi062 ceph-mon[30958]: pgmap v470: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2024-09-12T00:07:11.775 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:07:11.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.698+0000 7ffa77ecc640 1 -- 172.21.15.62:0/3054516613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa70100e20 msgr2=0x7ffa70101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:11.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.698+0000 7ffa77ecc640 1 --2- 172.21.15.62:0/3054516613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa70100e20 0x7ffa70101200 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009900 tx=0x7ffa6c02a480 comp rx=0 tx=0).stop 2024-09-12T00:07:11.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.698+0000 7ffa77ecc640 1 -- 172.21.15.62:0/3054516613 shutdown_connections 2024-09-12T00:07:11.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.698+0000 7ffa77ecc640 1 --2- 172.21.15.62:0/3054516613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa70102ef0 0x7ffa7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.698+0000 7ffa77ecc640 1 --2- 172.21.15.62:0/3054516613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa70102570 0x7ffa701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.698+0000 7ffa77ecc640 1 --2- 172.21.15.62:0/3054516613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa70100e20 0x7ffa70101200 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.698+0000 7ffa77ecc640 1 -- 172.21.15.62:0/3054516613 >> 172.21.15.62:0/3054516613 conn(0x7ffa700fc7d0 msgr2=0x7ffa700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:11.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.698+0000 7ffa77ecc640 1 -- 172.21.15.62:0/3054516613 shutdown_connections 2024-09-12T00:07:11.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.698+0000 7ffa77ecc640 1 -- 172.21.15.62:0/3054516613 wait complete. 2024-09-12T00:07:11.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.699+0000 7ffa77ecc640 1 Processor -- start 2024-09-12T00:07:11.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.699+0000 7ffa77ecc640 1 -- start start 2024-09-12T00:07:11.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.699+0000 7ffa77ecc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa70100e20 0x7ffa70198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.699+0000 7ffa77ecc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa70102570 0x7ffa70198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.699+0000 7ffa77ecc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa70102ef0 0x7ffa7019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.699+0000 7ffa77ecc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70110f80 con 0x7ffa70102570 2024-09-12T00:07:11.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.699+0000 7ffa77ecc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70110e00 con 0x7ffa70102ef0 2024-09-12T00:07:11.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.699+0000 7ffa77ecc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70111100 con 0x7ffa70100e20 2024-09-12T00:07:11.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.700+0000 7ffa76469640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa70102ef0 0x7ffa7019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.700+0000 7ffa76469640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa70102ef0 0x7ffa7019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37572/0 (socket says 172.21.15.62:37572) 2024-09-12T00:07:11.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.700+0000 7ffa76469640 1 -- 172.21.15.62:0/486854659 learned_addr learned my addr 172.21.15.62:0/486854659 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:11.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.701+0000 7ffa76469640 1 -- 172.21.15.62:0/486854659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa70100e20 msgr2=0x7ffa70198500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T00:07:11.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.701+0000 7ffa76469640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa70100e20 0x7ffa70198500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.701+0000 7ffa76469640 1 -- 172.21.15.62:0/486854659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa70102570 msgr2=0x7ffa70198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:11.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.701+0000 7ffa76469640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa70102570 0x7ffa70198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.702+0000 7ffa76469640 1 -- 172.21.15.62:0/486854659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7019d4b0 con 0x7ffa70102ef0 2024-09-12T00:07:11.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.702+0000 7ffa76469640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa70102ef0 0x7ffa7019cdd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009fd0 tx=0x7ffa6800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:11.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.703+0000 7ffa66ffd640 1 -- 172.21.15.62:0/486854659 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68017070 con 0x7ffa70102ef0 2024-09-12T00:07:11.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.703+0000 7ffa66ffd640 1 -- 172.21.15.62:0/486854659 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa68003e50 con 0x7ffa70102ef0 2024-09-12T00:07:11.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.703+0000 7ffa77ecc640 1 -- 172.21.15.62:0/486854659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7019d7a0 con 0x7ffa70102ef0 2024-09-12T00:07:11.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.703+0000 7ffa66ffd640 1 -- 172.21.15.62:0/486854659 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa680106c0 con 0x7ffa70102ef0 2024-09-12T00:07:11.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.703+0000 7ffa77ecc640 1 -- 172.21.15.62:0/486854659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa701a5080 con 0x7ffa70102ef0 2024-09-12T00:07:11.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.705+0000 7ffa66ffd640 1 -- 172.21.15.62:0/486854659 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffa68003970 con 0x7ffa70102ef0 2024-09-12T00:07:11.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.705+0000 7ffa7a2a5740 1 Processor -- start 2024-09-12T00:07:11.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.706+0000 7ffa7a2a5740 1 -- start start 2024-09-12T00:07:11.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.706+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8591b40 0x55e5d8590090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.706+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d85905d0 0x55e5d8590a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.706+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d859b370 0x55e5d859d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.706+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e5d84f62f0 con 0x55e5d859b370 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.706+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5d849dcf0 con 0x55e5d85905d0 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.706+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5d84f22b0 con 0x55e5d8591b40 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75467640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8591b40 0x55e5d8590090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75467640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8591b40 0x55e5d8590090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:44080/0 (socket says 172.21.15.62:44080) 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75467640 1 -- 172.21.15.62:0/4133628614 learned_addr learned my addr 172.21.15.62:0/4133628614 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75c68640 1 --2- 172.21.15.62:0/4133628614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d85905d0 0x55e5d8590a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75467640 1 --2- 172.21.15.62:0/4133628614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d859b370 0x55e5d859d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa66ffd640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa5c0a2d70 0x7ffa5c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa76469640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa5c0a2d70 0x7ffa5c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75c68640 1 -- 172.21.15.62:0/4133628614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8591b40 msgr2=0x55e5d8590090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa66ffd640 1 -- 172.21.15.62:0/486854659 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8557+0+0 (secure 0 0 0) 0x7ffa680cf270 con 0x7ffa70102ef0 2024-09-12T00:07:11.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75c68640 1 --2- 172.21.15.62:0/4133628614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8591b40 0x55e5d8590090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75c68640 1 -- 172.21.15.62:0/4133628614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d859b370 msgr2=0x55e5d859d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75c68640 1 --2- 172.21.15.62:0/4133628614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d859b370 0x55e5d859d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75c68640 1 -- 172.21.15.62:0/4133628614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d84823a0 con 0x55e5d85905d0 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.707+0000 7ffa75467640 1 --2- 172.21.15.62:0/4133628614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8591b40 0x55e5d8590090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.708+0000 7ffa75c68640 1 --2- 172.21.15.62:0/4133628614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d85905d0 0x55e5d8590a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c02a9a0 tx=0x7ffa6c031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b34743f3ca62459e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.708+0000 7ffa76469640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa5c0a2d70 0x7ffa5c0a5230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffa68098120 tx=0x7ffa68086040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.708+0000 7ffa5bfff640 1 -- 172.21.15.62:0/4133628614 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c03e030 con 0x55e5d85905d0 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.708+0000 7ffa5bfff640 1 -- 172.21.15.62:0/4133628614 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c0035f0 con 0x55e5d85905d0 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.708+0000 7ffa5bfff640 1 -- 172.21.15.62:0/4133628614 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c035040 con 0x55e5d85905d0 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/4133628614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d85905d0 msgr2=0x55e5d8590a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/4133628614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d85905d0 0x55e5d8590a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c02a9a0 tx=0x7ffa6c031490 comp rx=0 tx=0).stop 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/4133628614 shutdown_connections 2024-09-12T00:07:11.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/4133628614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d859b370 0x55e5d859d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/4133628614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d85905d0 0x55e5d8590a10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/4133628614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8591b40 0x55e5d8590090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/4133628614 >> 172.21.15.62:0/4133628614 conn(0x55e5d8559310 msgr2=0x55e5d858cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/4133628614 shutdown_connections 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/4133628614 wait complete. 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 -- start start 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.709+0000 7ffa7a2a5740 1 Processor -- start 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d8597f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d859b370 0x55e5d85984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e5d85a2420 con 0x55e5d85905d0 2024-09-12T00:07:11.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5d84b1060 con 0x55e5d8591b40 2024-09-12T00:07:11.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5d85a25a0 con 0x55e5d859b370 2024-09-12T00:07:11.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa7a2a5740 10 client.0 _ll_register_callbacks cb 0x55e5d82048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:11.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa75467640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa75c68640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d8597f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa75467640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56534/0 (socket says 172.21.15.62:56534) 2024-09-12T00:07:11.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa75467640 1 -- 172.21.15.62:0/3112267887 learned_addr learned my addr 172.21.15.62:0/3112267887 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:11.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.710+0000 7ffa75467640 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d859b370 0x55e5d85984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.711+0000 7ffa75467640 1 -- 172.21.15.62:0/3112267887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d859b370 msgr2=0x55e5d85984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:11.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.711+0000 7ffa75467640 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d859b370 0x55e5d85984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.711+0000 7ffa75467640 1 -- 172.21.15.62:0/3112267887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 msgr2=0x55e5d8597f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.711+0000 7ffa75467640 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d8597f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.711+0000 7ffa75467640 1 -- 172.21.15.62:0/3112267887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d82673c0 con 0x55e5d85905d0 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.711+0000 7ffa75467640 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859acb0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ffa60011d60 tx=0x7ffa60005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1b5148847f509e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.711+0000 7ffa597fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.711+0000 7ffa597fa640 1 -- 172.21.15.62:0/3112267887 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000edf0 con 0x55e5d85905d0 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.711+0000 7ffa597fa640 1 -- 172.21.15.62:0/3112267887 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8557+0+0 (secure 0 0 0) 0x7ffa6001f440 con 0x55e5d85905d0 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.713+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3112267887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6c0035f0 con 0x55e5d85905d0 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.713+0000 7ffa597fa640 1 -- 172.21.15.62:0/3112267887 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa6001f640 con 0x55e5d85905d0 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.713+0000 7ffa7a2a5740 10 client.18078 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:11.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.713+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3112267887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e5d85ade90 con 0x55e5d85905d0 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.714+0000 7ffa597fa640 1 -- 172.21.15.62:0/3112267887 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c0035f0 con 0x55e5d85905d0 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.714+0000 7ffa597fa640 1 -- 172.21.15.62:0/3112267887 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ffa6001d020 con 0x55e5d85905d0 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.714+0000 7ffa7a2a5740 10 client.18078 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.714+0000 7ffa7a2a5740 10 client.18078 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.714+0000 7ffa7a2a5740 20 client.18078 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.714+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e5d85af2a0 0x55e5d85b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.714+0000 7ffa7a2a5740 4 client.18078 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.714+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3112267887 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5d8503860 con 0x55e5d85af2a0 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.715+0000 7ffa75c68640 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e5d85af2a0 0x55e5d85b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.715+0000 7ffa597fa640 10 client.18078 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.715+0000 7ffa75c68640 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e5d85af2a0 0x55e5d85b1720 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76d1f22a479d81e9 server_cookie=502eacc30fe71236 in_seq=0 out_seq=0 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa597fa640 1 -- 172.21.15.62:0/3112267887 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5d8503860 con 0x55e5d85af2a0 2024-09-12T00:07:11.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa597fa640 10 client.18078 handle_command_reply: tid=0 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 client.18078 shutdown 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3112267887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 msgr2=0x55e5d859acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859acb0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ffa60011d60 tx=0x7ffa60005f90 comp rx=0 tx=0).stop 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3112267887 shutdown_connections 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e5d85af2a0 0x55e5d85b1720 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d859b370 0x55e5d85984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d8597f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa597fa640 0 client.18078 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/3112267887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859acb0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3112267887 >> 172.21.15.62:0/3112267887 conn(0x55e5d8559310 msgr2=0x55e5d8592960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3112267887 shutdown_connections 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3112267887 wait complete. 2024-09-12T00:07:11.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.716+0000 7ffa7a2a5740 20 client.18078 trim_cache size 0 max 16384 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.732+0000 7ffa7a2a5740 1 Processor -- start 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa7a2a5740 1 -- start start 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8559310 0x55e5d858bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d859cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701aded0 con 0x55e5d85905d0 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701ab7a0 con 0x55e5d8591b40 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701a95f0 con 0x55e5d8559310 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa75c68640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa75c68640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56540/0 (socket says 172.21.15.62:56540) 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa75c68640 1 -- 172.21.15.62:0/3533249559 learned_addr learned my addr 172.21.15.62:0/3533249559 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa75467640 1 --2- 172.21.15.62:0/3533249559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8559310 0x55e5d858bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa75467640 1 --2- 172.21.15.62:0/3533249559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d859cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa75c68640 1 -- 172.21.15.62:0/3533249559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8559310 msgr2=0x55e5d858bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa75c68640 1 --2- 172.21.15.62:0/3533249559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8559310 0x55e5d858bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa75c68640 1 -- 172.21.15.62:0/3533249559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 msgr2=0x55e5d859cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa75c68640 1 --2- 172.21.15.62:0/3533249559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d859cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.733+0000 7ffa75c68640 1 -- 172.21.15.62:0/3533249559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d8591300 con 0x55e5d85905d0 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.734+0000 7ffa75c68640 1 --2- 172.21.15.62:0/3533249559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859c760 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c02a9a0 tx=0x7ffa6c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfb893241eb950a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.734+0000 7ffa5b7fe640 1 -- 172.21.15.62:0/3533249559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c035040 con 0x55e5d85905d0 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.734+0000 7ffa5b7fe640 1 -- 172.21.15.62:0/3533249559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c051030 con 0x55e5d85905d0 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.734+0000 7ffa5b7fe640 1 -- 172.21.15.62:0/3533249559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c031960 con 0x55e5d85905d0 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.734+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3533249559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 msgr2=0x55e5d859c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.734+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/3533249559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859c760 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c02a9a0 tx=0x7ffa6c02a480 comp rx=0 tx=0).stop 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.735+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3533249559 shutdown_connections 2024-09-12T00:07:11.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.735+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/3533249559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d859cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.735+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/3533249559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d85905d0 0x55e5d859c760 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.735+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/3533249559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d8559310 0x55e5d858bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.735+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3533249559 >> 172.21.15.62:0/3533249559 conn(0x55e5d8598600 msgr2=0x55e5d858b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.735+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3533249559 shutdown_connections 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.735+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/3533249559 wait complete. 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.736+0000 7ffa7a2a5740 1 -- start start 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.736+0000 7ffa7a2a5740 1 Processor -- start 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d8559310 0x55e5d85a7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d85905d0 0x55e5d85a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa7a2a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d85a2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701a9c60 con 0x55e5d8559310 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701ab0d0 con 0x55e5d8591b40 2024-09-12T00:07:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa7a2a5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701aa330 con 0x55e5d85905d0 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa75467640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d8559310 0x55e5d85a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa75467640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d8559310 0x55e5d85a7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56554/0 (socket says 172.21.15.62:56554) 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa75467640 1 -- 172.21.15.62:0/944016152 learned_addr learned my addr 172.21.15.62:0/944016152 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa7a2a5740 10 client.0 _ll_register_callbacks cb 0x7ffa60009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa75c68640 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d85905d0 0x55e5d85a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.737+0000 7ffa75467640 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d85a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.738+0000 7ffa75467640 1 -- 172.21.15.62:0/944016152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d85905d0 msgr2=0x55e5d85a7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.738+0000 7ffa75467640 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d85905d0 0x55e5d85a7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.738+0000 7ffa75467640 1 -- 172.21.15.62:0/944016152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 msgr2=0x55e5d85a2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.738+0000 7ffa75467640 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d85a2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.738+0000 7ffa75467640 1 -- 172.21.15.62:0/944016152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d85a2de0 con 0x55e5d8559310 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.738+0000 7ffa75467640 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d8559310 0x55e5d85a7870 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ffa60011b30 tx=0x7ffa60005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73e34694cd0f2be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.738+0000 7ffa597fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.738+0000 7ffa597fa640 1 -- 172.21.15.62:0/944016152 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6002f070 con 0x55e5d8559310 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.738+0000 7ffa597fa640 1 -- 172.21.15.62:0/944016152 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8557+0+0 (secure 0 0 0) 0x7ffa60029d50 con 0x55e5d8559310 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.738+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/944016152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6c051030 con 0x55e5d8559310 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa597fa640 1 -- 172.21.15.62:0/944016152 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa6003e050 con 0x55e5d8559310 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa7a2a5740 10 client.18084 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/944016152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e5d861a860 con 0x55e5d8559310 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa597fa640 1 -- 172.21.15.62:0/944016152 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c051030 con 0x55e5d8559310 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa597fa640 1 -- 172.21.15.62:0/944016152 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ffa60035040 con 0x55e5d8559310 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa7a2a5740 10 client.18084 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa7a2a5740 10 client.18084 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa7a2a5740 20 client.18084 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e5d859b370 0x55e5d861cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:11.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa7a2a5740 4 client.18084 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/944016152 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5d84fdd70 con 0x55e5d859b370 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.739+0000 7ffa75c68640 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e5d859b370 0x55e5d861cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.740+0000 7ffa75c68640 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e5d859b370 0x55e5d861cf40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97a4abb39978e37a server_cookie=710ddf8f476aca31 in_seq=0 out_seq=0 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.740+0000 7ffa597fa640 10 client.18084 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.740+0000 7ffa597fa640 1 -- 172.21.15.62:0/944016152 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5d84fdd70 con 0x55e5d859b370 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.740+0000 7ffa597fa640 10 client.18084 handle_command_reply: tid=0 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.740+0000 7ffa7a2a5740 1 client.18084 shutdown 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.740+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/944016152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d8559310 msgr2=0x55e5d85a7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.740+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d8559310 0x55e5d85a7870 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ffa60011b30 tx=0x7ffa60005f90 comp rx=0 tx=0).stop 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.744+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/944016152 shutdown_connections 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.744+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x55e5d859b370 0x55e5d861cf40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.744+0000 7ffa597fa640 0 client.18084 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:07:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.744+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5d8591b40 0x55e5d85a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.744+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5d85905d0 0x55e5d85a7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.744+0000 7ffa7a2a5740 1 --2- 172.21.15.62:0/944016152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5d8559310 0x55e5d85a7870 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.744+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/944016152 >> 172.21.15.62:0/944016152 conn(0x55e5d8598600 msgr2=0x55e5d858b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.745+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/944016152 shutdown_connections 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.745+0000 7ffa7a2a5740 1 -- 172.21.15.62:0/944016152 wait complete. 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.745+0000 7ffa7a2a5740 20 client.18084 trim_cache size 0 max 16384 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.746+0000 7ffa77ecc640 1 -- 172.21.15.62:0/486854659 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa5c0a2d70 msgr2=0x7ffa5c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.746+0000 7ffa77ecc640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa5c0a2d70 0x7ffa5c0a5230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffa68098120 tx=0x7ffa68086040 comp rx=0 tx=0).stop 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.746+0000 7ffa77ecc640 1 -- 172.21.15.62:0/486854659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa70102ef0 msgr2=0x7ffa7019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.746+0000 7ffa77ecc640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa70102ef0 0x7ffa7019cdd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009fd0 tx=0x7ffa6800b260 comp rx=0 tx=0).stop 2024-09-12T00:07:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.747+0000 7ffa77ecc640 1 -- 172.21.15.62:0/486854659 shutdown_connections 2024-09-12T00:07:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.747+0000 7ffa77ecc640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ffa5c0a2d70 0x7ffa5c0a5230 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.747+0000 7ffa77ecc640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa70102ef0 0x7ffa7019cdd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.747+0000 7ffa77ecc640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa70102570 0x7ffa70198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.747+0000 7ffa77ecc640 1 --2- 172.21.15.62:0/486854659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa70100e20 0x7ffa70198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.747+0000 7ffa77ecc640 1 -- 172.21.15.62:0/486854659 >> 172.21.15.62:0/486854659 conn(0x7ffa700fc7d0 msgr2=0x7ffa7006a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.747+0000 7ffa77ecc640 1 -- 172.21.15.62:0/486854659 shutdown_connections 2024-09-12T00:07:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:11.747+0000 7ffa77ecc640 1 -- 172.21.15.62:0/486854659 wait complete. 2024-09-12T00:07:11.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:11 smithi120 ceph-mon[35182]: pgmap v470: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2024-09-12T00:07:11.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:11 smithi083 ceph-mon[34805]: pgmap v470: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2024-09-12T00:07:12.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 870.0 s, 1937.8 tps, lat 50.885 ms stddev 70.002 2024-09-12T00:07:12.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:07:13.371 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:13.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:13 smithi062 ceph-mon[30958]: pgmap v471: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 24 MiB/s wr, 1.83k op/s 2024-09-12T00:07:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.858+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/17904183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbcc102570 msgr2=0x7fdbcc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.858+0000 7fdbd1aaf640 1 --2- 172.21.15.62:0/17904183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbcc102570 0x7fdbcc1029b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009930 tx=0x7fdbc002a4d0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.860+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/17904183 shutdown_connections 2024-09-12T00:07:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.860+0000 7fdbd1aaf640 1 --2- 172.21.15.62:0/17904183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbcc102ef0 0x7fdbcc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.860+0000 7fdbd1aaf640 1 --2- 172.21.15.62:0/17904183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbcc102570 0x7fdbcc1029b0 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009930 tx=0x7fdbc002a4d0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.860+0000 7fdbd1aaf640 1 --2- 172.21.15.62:0/17904183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc100e20 0x7fdbcc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.860+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/17904183 >> 172.21.15.62:0/17904183 conn(0x7fdbcc0fc7d0 msgr2=0x7fdbcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.860+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/17904183 shutdown_connections 2024-09-12T00:07:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.861+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/17904183 wait complete. 2024-09-12T00:07:13.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.861+0000 7fdbd1aaf640 1 Processor -- start 2024-09-12T00:07:13.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbd1aaf640 1 -- start start 2024-09-12T00:07:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbd1aaf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc100e20 0x7fdbcc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbd1aaf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbcc102ef0 0x7fdbcc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbd1aaf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbcc0fd700 0x7fdbcc1a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbd1aaf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc110f80 con 0x7fdbcc0fd700 2024-09-12T00:07:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbd1aaf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc110e00 con 0x7fdbcc100e20 2024-09-12T00:07:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbd1aaf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc111100 con 0x7fdbcc102ef0 2024-09-12T00:07:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbcbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbcc0fd700 0x7fdbcc1a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbcb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc100e20 0x7fdbcc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbcbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbcc0fd700 0x7fdbcc1a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56568/0 (socket says 172.21.15.62:56568) 2024-09-12T00:07:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbcbfff640 1 -- 172.21.15.62:0/3120601245 learned_addr learned my addr 172.21.15.62:0/3120601245 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.862+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbcc102ef0 0x7fdbcc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbcb7fe640 1 -- 172.21.15.62:0/3120601245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbcc102ef0 msgr2=0x7fdbcc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbcb7fe640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbcc102ef0 0x7fdbcc0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbcb7fe640 1 -- 172.21.15.62:0/3120601245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbcc0fd700 msgr2=0x7fdbcc1a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbcb7fe640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbcc0fd700 0x7fdbcc1a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbcb7fe640 1 -- 172.21.15.62:0/3120601245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc1a7040 con 0x7fdbcc100e20 2024-09-12T00:07:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbcc102ef0 0x7fdbcc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:07:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbcb7fe640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc100e20 0x7fdbcc0fcc80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00ea20 tx=0x7fdbbc00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbc8ff9640 1 -- 172.21.15.62:0/3120601245 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc00ccc0 con 0x7fdbcc100e20 2024-09-12T00:07:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbc8ff9640 1 -- 172.21.15.62:0/3120601245 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbbc00ce60 con 0x7fdbcc100e20 2024-09-12T00:07:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/3120601245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbcc1a7330 con 0x7fdbcc100e20 2024-09-12T00:07:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbc8ff9640 1 -- 172.21.15.62:0/3120601245 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc004a00 con 0x7fdbcc100e20 2024-09-12T00:07:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.863+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/3120601245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbcc1a77f0 con 0x7fdbcc100e20 2024-09-12T00:07:13.875 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:13 smithi120 ceph-mon[35182]: pgmap v471: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 24 MiB/s wr, 1.83k op/s 2024-09-12T00:07:13.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.866+0000 7fdbc8ff9640 1 -- 172.21.15.62:0/3120601245 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdbbc004bc0 con 0x7fdbcc100e20 2024-09-12T00:07:13.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.867+0000 7fdbd3e88740 1 Processor -- start 2024-09-12T00:07:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.867+0000 7fdbd3e88740 1 -- start start 2024-09-12T00:07:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.867+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 0x5634b083c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.867+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 0x5634b083ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.867+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b0849760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.867+0000 7fdbd3e88740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5634b07a22f0 con 0x5634b083db40 2024-09-12T00:07:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.867+0000 7fdbd3e88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634b0749cf0 con 0x5634b0847370 2024-09-12T00:07:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.867+0000 7fdbd3e88740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634b079e2b0 con 0x5634b083c5d0 2024-09-12T00:07:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.868+0000 7fdbcaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b0849760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.868+0000 7fdbcaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b0849760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37634/0 (socket says 172.21.15.62:37634) 2024-09-12T00:07:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.868+0000 7fdbcaffd640 1 -- 172.21.15.62:0/1565192572 learned_addr learned my addr 172.21.15.62:0/1565192572 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.868+0000 7fdbcb7fe640 1 --2- 172.21.15.62:0/1565192572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 0x5634b083ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.868+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/1565192572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 0x5634b083c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.869+0000 7fdbcbfff640 1 -- 172.21.15.62:0/1565192572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 msgr2=0x5634b083ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.869+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/1565192572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 0x5634b083ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.869+0000 7fdbcbfff640 1 -- 172.21.15.62:0/1565192572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 msgr2=0x5634b0849760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.869+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/1565192572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b0849760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.869+0000 7fdbcbfff640 1 -- 172.21.15.62:0/1565192572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634b072e3a0 con 0x5634b083db40 2024-09-12T00:07:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.869+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/1565192572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b0849760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.869+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/1565192572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 0x5634b083c090 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009fc0 tx=0x7fdbc4009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40daf040483b7b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.870+0000 7fdbc8ff9640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdbb40a2cf0 0x7fdbb40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.870+0000 7fdbb1ffb640 1 -- 172.21.15.62:0/1565192572 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4004b20 con 0x5634b083db40 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.870+0000 7fdbb1ffb640 1 -- 172.21.15.62:0/1565192572 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc4004cc0 con 0x5634b083db40 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.870+0000 7fdbb1ffb640 1 -- 172.21.15.62:0/1565192572 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4005520 con 0x5634b083db40 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.870+0000 7fdbc8ff9640 1 -- 172.21.15.62:0/3120601245 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8557+0+0 (secure 0 0 0) 0x7fdbbc0cf4f0 con 0x7fdbcc100e20 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.870+0000 7fdbcb7fe640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdbb40a2cf0 0x7fdbb40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.871+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1565192572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 msgr2=0x5634b083c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.871+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/1565192572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 0x5634b083c090 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009fc0 tx=0x7fdbc4009960 comp rx=0 tx=0).stop 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.871+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1565192572 shutdown_connections 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.871+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/1565192572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b0849760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.871+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/1565192572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 0x5634b083ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.871+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/1565192572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 0x5634b083c090 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.871+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1565192572 >> 172.21.15.62:0/1565192572 conn(0x5634b0805310 msgr2=0x5634b0838c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.871+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1565192572 shutdown_connections 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.871+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1565192572 wait complete. 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.872+0000 7fdbcb7fe640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdbb40a2cf0 0x7fdbb40a51b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5634b0839ac0 tx=0x7fdbbc0054e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.872+0000 7fdbd3e88740 1 -- start start 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.872+0000 7fdbd3e88740 1 Processor -- start 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.873+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b083c5d0 0x5634b0846cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.873+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 0x5634b0843f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.873+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b0847370 0x5634b08444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.873+0000 7fdbd3e88740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5634b084a070 con 0x5634b0847370 2024-09-12T00:07:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.873+0000 7fdbd3e88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634b075d060 con 0x5634b083c5d0 2024-09-12T00:07:13.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbd3e88740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634b084a1f0 con 0x5634b083db40 2024-09-12T00:07:13.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbd3e88740 10 client.0 _ll_register_callbacks cb 0x5634b04b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbcbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b083c5d0 0x5634b0846cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbcbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b083c5d0 0x5634b0846cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37642/0 (socket says 172.21.15.62:37642) 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbcbfff640 1 -- 172.21.15.62:0/230381019 learned_addr learned my addr 172.21.15.62:0/230381019 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 0x5634b0843f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b0847370 0x5634b08444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbcbfff640 1 -- 172.21.15.62:0/230381019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 msgr2=0x5634b0843f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 0x5634b0843f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbcbfff640 1 -- 172.21.15.62:0/230381019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b083c5d0 msgr2=0x5634b0846cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b083c5d0 0x5634b0846cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.874+0000 7fdbcbfff640 1 -- 172.21.15.62:0/230381019 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634b05133c0 con 0x5634b0847370 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.875+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b0847370 0x5634b08444e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4000f80 tx=0x7fdbc40050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521d50463c8ea2b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.875+0000 7fdbaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.875+0000 7fdbaaffd640 1 -- 172.21.15.62:0/230381019 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4003970 con 0x5634b0847370 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.875+0000 7fdbaaffd640 1 -- 172.21.15.62:0/230381019 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8557+0+0 (secure 0 0 0) 0x7fdbc401f040 con 0x5634b0847370 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.875+0000 7fdbd3e88740 1 -- 172.21.15.62:0/230381019 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc4004cc0 con 0x5634b0847370 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.875+0000 7fdbaaffd640 1 -- 172.21.15.62:0/230381019 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdbc4003d20 con 0x5634b0847370 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.875+0000 7fdbd3e88740 10 client.18096 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.875+0000 7fdbd3e88740 1 -- 172.21.15.62:0/230381019 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5634b0855ab0 con 0x5634b0847370 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.875+0000 7fdbaaffd640 1 -- 172.21.15.62:0/230381019 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc4004cc0 con 0x5634b0847370 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.876+0000 7fdbaaffd640 1 -- 172.21.15.62:0/230381019 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fdbc4003fe0 con 0x5634b0847370 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.876+0000 7fdbd3e88740 10 client.18096 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.876+0000 7fdbd3e88740 10 client.18096 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.876+0000 7fdbd3e88740 20 client.18096 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.876+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5634b0856f10 0x5634b0859300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.876+0000 7fdbd3e88740 4 client.18096 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:07:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.876+0000 7fdbd3e88740 1 -- 172.21.15.62:0/230381019 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634b07af860 con 0x5634b0856f10 2024-09-12T00:07:13.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.876+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5634b0856f10 0x5634b0859300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.877+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5634b0856f10 0x5634b0859300 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bf6a372eaf0cf3b server_cookie=34f34bcfce2013bd in_seq=0 out_seq=0 2024-09-12T00:07:13.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.877+0000 7fdbaaffd640 10 client.18096 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:07:13.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.880+0000 7fdbaaffd640 1 -- 172.21.15.62:0/230381019 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634b07af860 con 0x5634b0856f10 2024-09-12T00:07:13.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.880+0000 7fdbaaffd640 10 client.18096 handle_command_reply: tid=0 2024-09-12T00:07:13.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.884+0000 7fdbd3e88740 1 client.18096 shutdown 2024-09-12T00:07:13.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.894+0000 7fdbd3e88740 1 -- 172.21.15.62:0/230381019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b0847370 msgr2=0x5634b08444e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:13.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.894+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b0847370 0x5634b08444e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4000f80 tx=0x7fdbc40050c0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.907+0000 7fdbd3e88740 1 -- 172.21.15.62:0/230381019 shutdown_connections 2024-09-12T00:07:13.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.907+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5634b0856f10 0x5634b0859300 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.907+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b0847370 0x5634b08444e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.907+0000 7fdbaaffd640 0 client.18096 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:07:13.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.907+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 0x5634b0843f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.907+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/230381019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b083c5d0 0x5634b0846cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.907+0000 7fdbd3e88740 1 -- 172.21.15.62:0/230381019 >> 172.21.15.62:0/230381019 conn(0x5634b0805310 msgr2=0x5634b083e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.913+0000 7fdbd3e88740 1 -- 172.21.15.62:0/230381019 shutdown_connections 2024-09-12T00:07:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.916+0000 7fdbd3e88740 1 -- 172.21.15.62:0/230381019 wait complete. 2024-09-12T00:07:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.917+0000 7fdbd3e88740 20 client.18096 trim_cache size 0 max 16384 2024-09-12T00:07:13.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:13 smithi083 ceph-mon[34805]: pgmap v471: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 24 MiB/s wr, 1.83k op/s 2024-09-12T00:07:13.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.941+0000 7fdbd3e88740 1 Processor -- start 2024-09-12T00:07:13.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.942+0000 7fdbd3e88740 1 -- start start 2024-09-12T00:07:13.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.942+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083c5d0 0x5634b0838e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.943+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 0x5634b0853340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.943+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b0845ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.943+0000 7fdbcaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 0x5634b0853340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.943+0000 7fdbcaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 0x5634b0853340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:44164/0 (socket says 172.21.15.62:44164) 2024-09-12T00:07:13.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.943+0000 7fdbcbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083c5d0 0x5634b0838e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.943+0000 7fdbcbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083c5d0 0x5634b0838e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56596/0 (socket says 172.21.15.62:56596) 2024-09-12T00:07:13.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.943+0000 7fdbcaffd640 1 -- 172.21.15.62:0/1765873888 learned_addr learned my addr 172.21.15.62:0/1765873888 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:13.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.944+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1765873888 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1b2a90 con 0x5634b083c5d0 2024-09-12T00:07:13.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.944+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1765873888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1b0360 con 0x5634b0847370 2024-09-12T00:07:13.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.944+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1765873888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1ae1d0 con 0x5634b083db40 2024-09-12T00:07:13.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.945+0000 7fdbcaffd640 1 -- 172.21.15.62:0/1765873888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 msgr2=0x5634b0845ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:13.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.945+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/1765873888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b0845ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.945+0000 7fdbcaffd640 1 -- 172.21.15.62:0/1765873888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083c5d0 msgr2=0x5634b0838e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:13.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.945+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/1765873888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083c5d0 0x5634b0838e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.945+0000 7fdbcaffd640 1 -- 172.21.15.62:0/1765873888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634b083f120 con 0x5634b083db40 2024-09-12T00:07:13.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.945+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/1765873888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 0x5634b0853340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0006010 tx=0x7fdbc0005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6467ea1232fb549 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:13.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.945+0000 7fdbb17fa640 1 -- 172.21.15.62:0/1765873888 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc004a070 con 0x5634b083db40 2024-09-12T00:07:13.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.945+0000 7fdbb17fa640 1 -- 172.21.15.62:0/1765873888 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc00317b0 con 0x5634b083db40 2024-09-12T00:07:13.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.945+0000 7fdbb17fa640 1 -- 172.21.15.62:0/1765873888 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0000e60 con 0x5634b083db40 2024-09-12T00:07:13.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.946+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1765873888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 msgr2=0x5634b0853340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:13.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.946+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/1765873888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 0x5634b0853340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0006010 tx=0x7fdbc0005b50 comp rx=0 tx=0).stop 2024-09-12T00:07:13.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.947+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1765873888 shutdown_connections 2024-09-12T00:07:13.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.947+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/1765873888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b0845ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.947+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/1765873888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083db40 0x5634b0853340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.947+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/1765873888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083c5d0 0x5634b0838e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.947+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1765873888 >> 172.21.15.62:0/1765873888 conn(0x5634b0805310 msgr2=0x5634b08457e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:13.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.947+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1765873888 shutdown_connections 2024-09-12T00:07:13.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.947+0000 7fdbd3e88740 1 -- 172.21.15.62:0/1765873888 wait complete. 2024-09-12T00:07:13.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.948+0000 7fdbd3e88740 1 -- start start 2024-09-12T00:07:13.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.948+0000 7fdbd3e88740 1 Processor -- start 2024-09-12T00:07:13.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.949+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 0x5634b084a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.949+0000 7fdbd3e88740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 0x5634b084ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.949+0000 7fdbcbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 0x5634b084a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.949+0000 7fdbcbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 0x5634b084a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:44170/0 (socket says 172.21.15.62:44170) 2024-09-12T00:07:13.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.949+0000 7fdbcbfff640 1 -- 172.21.15.62:0/3884365782 learned_addr learned my addr 172.21.15.62:0/3884365782 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:13.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.949+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b08bf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.950+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1ae820 con 0x5634b083db40 2024-09-12T00:07:13.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.950+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1afc90 con 0x5634b0847370 2024-09-12T00:07:13.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.950+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1aeef0 con 0x5634b083c5d0 2024-09-12T00:07:13.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.950+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b08bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.951+0000 7fdbd3e88740 10 client.0 _ll_register_callbacks cb 0x7fdbc400f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:13.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.951+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 0x5634b084ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.952+0000 7fdbcbfff640 1 -- 172.21.15.62:0/3884365782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 msgr2=0x5634b08bf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:13.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.952+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b08bf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.952+0000 7fdbcbfff640 1 -- 172.21.15.62:0/3884365782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 msgr2=0x5634b084ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.952+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 0x5634b084ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.952+0000 7fdbcbfff640 1 -- 172.21.15.62:0/3884365782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634b08bfec0 con 0x5634b083c5d0 2024-09-12T00:07:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.952+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 0x5634b084ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:07:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.953+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 0x5634b084a850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4005c40 tx=0x7fdbc40021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fef244e62b4db5c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.953+0000 7fdbaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:07:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.953+0000 7fdbaaffd640 1 -- 172.21.15.62:0/3884365782 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc400fd00 con 0x5634b083c5d0 2024-09-12T00:07:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.953+0000 7fdbaaffd640 1 -- 172.21.15.62:0/3884365782 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8557+0+0 (secure 0 0 0) 0x7fdbc4016040 con 0x5634b083c5d0 2024-09-12T00:07:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.953+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc00317b0 con 0x5634b083c5d0 2024-09-12T00:07:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.954+0000 7fdbaaffd640 1 -- 172.21.15.62:0/3884365782 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdbc4003ec0 con 0x5634b083c5d0 2024-09-12T00:07:13.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.954+0000 7fdbd3e88740 10 client.27448 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:13.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.954+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5634b08c1070 con 0x5634b083c5d0 2024-09-12T00:07:13.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.955+0000 7fdbaaffd640 1 -- 172.21.15.62:0/3884365782 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc00317b0 con 0x5634b083c5d0 2024-09-12T00:07:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.956+0000 7fdbaaffd640 1 -- 172.21.15.62:0/3884365782 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fdbc4005130 con 0x5634b083c5d0 2024-09-12T00:07:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.956+0000 7fdbd3e88740 10 client.27448 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.956+0000 7fdbd3e88740 10 client.27448 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:07:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.956+0000 7fdbd3e88740 20 client.27448 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.956+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5634b0856f10 0x5634b08c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.956+0000 7fdbd3e88740 4 client.27448 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:07:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.956+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634b07a9d70 con 0x5634b0856f10 2024-09-12T00:07:13.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.956+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5634b0856f10 0x5634b08c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:13.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.957+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5634b0856f10 0x5634b08c35e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=346eae29cb1e8f39 server_cookie=d39524567ab82e06 in_seq=0 out_seq=0 2024-09-12T00:07:13.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.957+0000 7fdbaaffd640 10 client.27448 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:07:13.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.961+0000 7fdbaaffd640 1 -- 172.21.15.62:0/3884365782 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634b07a9d70 con 0x5634b0856f10 2024-09-12T00:07:13.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.961+0000 7fdbaaffd640 10 client.27448 handle_command_reply: tid=0 2024-09-12T00:07:13.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.961+0000 7fdbd3e88740 1 client.27448 shutdown 2024-09-12T00:07:13.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.961+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 msgr2=0x5634b084a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:13.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.962+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 0x5634b084a850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4005c40 tx=0x7fdbc40021d0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.962+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 shutdown_connections 2024-09-12T00:07:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.962+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5634b0856f10 0x5634b08c35e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.962+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634b0847370 0x5634b08bf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.962+0000 7fdbaaffd640 0 client.27448 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:07:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.962+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634b083db40 0x5634b084ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.962+0000 7fdbd3e88740 1 --2- 172.21.15.62:0/3884365782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634b083c5d0 0x5634b084a850 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.963+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 >> 172.21.15.62:0/3884365782 conn(0x5634b0805310 msgr2=0x5634b083e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:13.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.963+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 shutdown_connections 2024-09-12T00:07:13.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.965+0000 7fdbd3e88740 1 -- 172.21.15.62:0/3884365782 wait complete. 2024-09-12T00:07:13.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.965+0000 7fdbd3e88740 20 client.27448 trim_cache size 0 max 16384 2024-09-12T00:07:13.964 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:07:13.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.967+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/3120601245 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdbb40a2cf0 msgr2=0x7fdbb40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:13.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.967+0000 7fdbd1aaf640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdbb40a2cf0 0x7fdbb40a51b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5634b0839ac0 tx=0x7fdbbc0054e0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.967+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/3120601245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc100e20 msgr2=0x7fdbcc0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:13.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.967+0000 7fdbd1aaf640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc100e20 0x7fdbcc0fcc80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00ea20 tx=0x7fdbbc00eee0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.967+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/3120601245 shutdown_connections 2024-09-12T00:07:13.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.967+0000 7fdbd1aaf640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdbb40a2cf0 0x7fdbb40a51b0 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5634b0839ac0 tx=0x7fdbbc0054e0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.967+0000 7fdbd1aaf640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbcc0fd700 0x7fdbcc1a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.967+0000 7fdbd1aaf640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbcc102ef0 0x7fdbcc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.967+0000 7fdbd1aaf640 1 --2- 172.21.15.62:0/3120601245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc100e20 0x7fdbcc0fcc80 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00ea20 tx=0x7fdbbc00eee0 comp rx=0 tx=0).stop 2024-09-12T00:07:13.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.967+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/3120601245 >> 172.21.15.62:0/3120601245 conn(0x7fdbcc0fc7d0 msgr2=0x7fdbcc06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:13.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.968+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/3120601245 shutdown_connections 2024-09-12T00:07:13.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:13.970+0000 7fdbd1aaf640 1 -- 172.21.15.62:0/3120601245 wait complete. 2024-09-12T00:07:14.751 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:07:15.107 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:15.505 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:15 smithi062 ceph-mon[30958]: pgmap v472: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 1.95k op/s 2024-09-12T00:07:15.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.690+0000 7f7b60aad640 1 -- 172.21.15.62:0/1814348718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b5c109110 msgr2=0x7f7b5c110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:15.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.690+0000 7f7b60aad640 1 --2- 172.21.15.62:0/1814348718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b5c109110 0x7f7b5c110c90 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009900 tx=0x7f7b4c02a410 comp rx=0 tx=0).stop 2024-09-12T00:07:15.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.691+0000 7f7b60aad640 1 -- 172.21.15.62:0/1814348718 shutdown_connections 2024-09-12T00:07:15.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.691+0000 7f7b60aad640 1 --2- 172.21.15.62:0/1814348718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b5c109110 0x7f7b5c110c90 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.691+0000 7f7b60aad640 1 --2- 172.21.15.62:0/1814348718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b5c104990 0x7f7b5c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.691+0000 7f7b60aad640 1 --2- 172.21.15.62:0/1814348718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5c103fe0 0x7f7b5c1043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.691+0000 7f7b60aad640 1 -- 172.21.15.62:0/1814348718 >> 172.21.15.62:0/1814348718 conn(0x7f7b5c0ffac0 msgr2=0x7f7b5c101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:15.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.692+0000 7f7b60aad640 1 -- 172.21.15.62:0/1814348718 shutdown_connections 2024-09-12T00:07:15.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.692+0000 7f7b60aad640 1 -- 172.21.15.62:0/1814348718 wait complete. 2024-09-12T00:07:15.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.693+0000 7f7b60aad640 1 Processor -- start 2024-09-12T00:07:15.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.693+0000 7f7b60aad640 1 -- start start 2024-09-12T00:07:15.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.693+0000 7f7b60aad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5c103fe0 0x7f7b5c110b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.693+0000 7f7b60aad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b5c104990 0x7f7b5c10dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.693+0000 7f7b60aad640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b5c109110 0x7f7b5c10e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.693+0000 7f7b60aad640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c114140 con 0x7f7b5c109110 2024-09-12T00:07:15.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.693+0000 7f7b60aad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c113fc0 con 0x7f7b5c103fe0 2024-09-12T00:07:15.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.693+0000 7f7b60aad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c1142c0 con 0x7f7b5c104990 2024-09-12T00:07:15.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.694+0000 7f7b5ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b5c109110 0x7f7b5c10e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.694+0000 7f7b59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b5c104990 0x7f7b5c10dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.694+0000 7f7b59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b5c104990 0x7f7b5c10dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:44178/0 (socket says 172.21.15.62:44178) 2024-09-12T00:07:15.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.694+0000 7f7b59d9b640 1 -- 172.21.15.62:0/177129310 learned_addr learned my addr 172.21.15.62:0/177129310 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:15.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.695+0000 7f7b5ad9d640 1 -- 172.21.15.62:0/177129310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b5c104990 msgr2=0x7f7b5c10dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:15.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.695+0000 7f7b5ad9d640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b5c104990 0x7f7b5c10dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.695+0000 7f7b5ad9d640 1 -- 172.21.15.62:0/177129310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5c103fe0 msgr2=0x7f7b5c110b20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T00:07:15.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.695+0000 7f7b5ad9d640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5c103fe0 0x7f7b5c110b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.695+0000 7f7b5ad9d640 1 -- 172.21.15.62:0/177129310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c1a0660 con 0x7f7b5c109110 2024-09-12T00:07:15.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.695+0000 7f7b5ad9d640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b5c109110 0x7f7b5c10e110 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009230 tx=0x7f7b4c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.696+0000 7f7b4b7fe640 1 -- 172.21.15.62:0/177129310 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c03f030 con 0x7f7b5c109110 2024-09-12T00:07:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.696+0000 7f7b60aad640 1 -- 172.21.15.62:0/177129310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5c1a0890 con 0x7f7b5c109110 2024-09-12T00:07:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.696+0000 7f7b4b7fe640 1 -- 172.21.15.62:0/177129310 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4c004350 con 0x7f7b5c109110 2024-09-12T00:07:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.696+0000 7f7b4b7fe640 1 -- 172.21.15.62:0/177129310 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c035070 con 0x7f7b5c109110 2024-09-12T00:07:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.696+0000 7f7b60aad640 1 -- 172.21.15.62:0/177129310 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5c1a0dc0 con 0x7f7b5c109110 2024-09-12T00:07:15.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.699+0000 7f7b62e86740 1 Processor -- start 2024-09-12T00:07:15.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.700+0000 7f7b4b7fe640 1 -- 172.21.15.62:0/177129310 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7b4c042030 con 0x7f7b5c109110 2024-09-12T00:07:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b62e86740 1 -- start start 2024-09-12T00:07:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de761090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de761a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de76c370 0x55a3de76e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b62e86740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a3de6c72f0 con 0x55a3de76c370 2024-09-12T00:07:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b62e86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a3de66ecf0 con 0x55a3de7615d0 2024-09-12T00:07:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b62e86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a3de6c32b0 con 0x55a3de762b40 2024-09-12T00:07:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de761090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de761090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:44194/0 (socket says 172.21.15.62:44194) 2024-09-12T00:07:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b59d9b640 1 -- 172.21.15.62:0/1810672560 learned_addr learned my addr 172.21.15.62:0/1810672560 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:15.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b59d9b640 1 --2- 172.21.15.62:0/1810672560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de76c370 0x55a3de76e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.701+0000 7f7b5a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de761a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.702+0000 7f7b59d9b640 1 -- 172.21.15.62:0/1810672560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 msgr2=0x55a3de761a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:15.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.702+0000 7f7b59d9b640 1 --2- 172.21.15.62:0/1810672560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de761a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.702+0000 7f7b59d9b640 1 -- 172.21.15.62:0/1810672560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de76c370 msgr2=0x55a3de76e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:15.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.702+0000 7f7b59d9b640 1 --2- 172.21.15.62:0/1810672560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de76c370 0x55a3de76e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.702+0000 7f7b59d9b640 1 -- 172.21.15.62:0/1810672560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3de6533a0 con 0x55a3de762b40 2024-09-12T00:07:15.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.702+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/1810672560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de761a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:15.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.702+0000 7f7b59d9b640 1 --2- 172.21.15.62:0/1810672560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de761090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b54012ae0 tx=0x7f7b54009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e8ae730c8763e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:15.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.703+0000 7f7b48ff9640 1 -- 172.21.15.62:0/1810672560 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54010890 con 0x55a3de762b40 2024-09-12T00:07:15.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.703+0000 7f7b48ff9640 1 -- 172.21.15.62:0/1810672560 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b5401e3c0 con 0x55a3de762b40 2024-09-12T00:07:15.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.703+0000 7f7b48ff9640 1 -- 172.21.15.62:0/1810672560 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5400e610 con 0x55a3de762b40 2024-09-12T00:07:15.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.703+0000 7f7b4b7fe640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7b440a2cf0 0x7f7b440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.703+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7b440a2cf0 0x7f7b440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.703+0000 7f7b4b7fe640 1 -- 172.21.15.62:0/177129310 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8557+0+0 (secure 0 0 0) 0x7f7b4c0ef530 con 0x7f7b5c109110 2024-09-12T00:07:15.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.704+0000 7f7b62e86740 1 -- 172.21.15.62:0/1810672560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 msgr2=0x55a3de761090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:15.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.704+0000 7f7b62e86740 1 --2- 172.21.15.62:0/1810672560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de761090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b54012ae0 tx=0x7f7b54009a70 comp rx=0 tx=0).stop 2024-09-12T00:07:15.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.704+0000 7f7b62e86740 1 -- 172.21.15.62:0/1810672560 shutdown_connections 2024-09-12T00:07:15.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.704+0000 7f7b62e86740 1 --2- 172.21.15.62:0/1810672560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de76c370 0x55a3de76e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.704+0000 7f7b62e86740 1 --2- 172.21.15.62:0/1810672560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de761a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.704+0000 7f7b62e86740 1 --2- 172.21.15.62:0/1810672560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de761090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.704+0000 7f7b62e86740 1 -- 172.21.15.62:0/1810672560 >> 172.21.15.62:0/1810672560 conn(0x55a3de72a310 msgr2=0x55a3de75dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.705+0000 7f7b62e86740 1 -- 172.21.15.62:0/1810672560 shutdown_connections 2024-09-12T00:07:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.705+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7b440a2cf0 0x7f7b440a51b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55a3de75eac0 tx=0x7f7b50005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.705+0000 7f7b62e86740 1 -- 172.21.15.62:0/1810672560 wait complete. 2024-09-12T00:07:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b62e86740 1 -- start start 2024-09-12T00:07:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b62e86740 1 Processor -- start 2024-09-12T00:07:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de76bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de768f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de76c370 0x55a3de7694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b62e86740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a3de76f0c0 con 0x55a3de762b40 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b62e86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a3de682060 con 0x55a3de7615d0 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b62e86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a3de76f240 con 0x55a3de76c370 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b62e86740 10 client.0 _ll_register_callbacks cb 0x55a3de3d58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b59d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de768f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b5a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de76bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b59d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de768f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56652/0 (socket says 172.21.15.62:56652) 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b5ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de76c370 0x55a3de7694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b5a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de76bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37704/0 (socket says 172.21.15.62:37704) 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.706+0000 7f7b59d9b640 1 -- 172.21.15.62:0/1716415773 learned_addr learned my addr 172.21.15.62:0/1716415773 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.707+0000 7f7b59d9b640 1 -- 172.21.15.62:0/1716415773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de76c370 msgr2=0x55a3de7694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.707+0000 7f7b59d9b640 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de76c370 0x55a3de7694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.707+0000 7f7b59d9b640 1 -- 172.21.15.62:0/1716415773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 msgr2=0x55a3de76bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:15.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.707+0000 7f7b59d9b640 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de76bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.707+0000 7f7b59d9b640 1 -- 172.21.15.62:0/1716415773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3de4383c0 con 0x55a3de762b40 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.707+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de76bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.707+0000 7f7b59d9b640 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de768f80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7b54012ab0 tx=0x7f7b5401e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5caa527bf19fd3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.707+0000 7f7b41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.708+0000 7f7b41ffb640 1 -- 172.21.15.62:0/1716415773 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54009480 con 0x55a3de762b40 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.708+0000 7f7b62e86740 1 -- 172.21.15.62:0/1716415773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b5401e3c0 con 0x55a3de762b40 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.708+0000 7f7b41ffb640 1 -- 172.21.15.62:0/1716415773 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8557+0+0 (secure 0 0 0) 0x7f7b54004c50 con 0x55a3de762b40 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.708+0000 7f7b41ffb640 1 -- 172.21.15.62:0/1716415773 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7b540127a0 con 0x55a3de762b40 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b62e86740 10 client.18111 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b62e86740 1 -- 172.21.15.62:0/1716415773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a3de77ab00 con 0x55a3de762b40 2024-09-12T00:07:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b41ffb640 1 -- 172.21.15.62:0/1716415773 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b54004e50 con 0x55a3de762b40 2024-09-12T00:07:15.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b41ffb640 1 -- 172.21.15.62:0/1716415773 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7b54005560 con 0x55a3de762b40 2024-09-12T00:07:15.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b62e86740 10 client.18111 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:15.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b62e86740 10 client.18111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b62e86740 20 client.18111 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b62e86740 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a3de77beb0 0x55a3de77e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b62e86740 4 client.18111 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b62e86740 1 -- 172.21.15.62:0/1716415773 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b5401e3c0 con 0x55a3de77beb0 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.709+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a3de77beb0 0x55a3de77e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.710+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a3de77beb0 0x55a3de77e2a0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fbc8b1f07b26c59 server_cookie=66c18bc73e7aee2b in_seq=0 out_seq=0 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.710+0000 7f7b41ffb640 10 client.18111 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.711+0000 7f7b41ffb640 1 -- 172.21.15.62:0/1716415773 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7b5401e3c0 con 0x55a3de77beb0 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.711+0000 7f7b41ffb640 10 client.18111 handle_command_reply: tid=0 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.711+0000 7f7b62e86740 1 client.18111 shutdown 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b62e86740 1 -- 172.21.15.62:0/1716415773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 msgr2=0x55a3de768f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b62e86740 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de768f80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7b54012ab0 tx=0x7f7b5401e800 comp rx=0 tx=0).stop 2024-09-12T00:07:15.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b62e86740 1 -- 172.21.15.62:0/1716415773 shutdown_connections 2024-09-12T00:07:15.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b62e86740 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a3de77beb0 0x55a3de77e2a0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b62e86740 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de76c370 0x55a3de7694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b62e86740 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de768f80 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b62e86740 1 --2- 172.21.15.62:0/1716415773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de76bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b41ffb640 0 client.18111 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:15.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b62e86740 1 -- 172.21.15.62:0/1716415773 >> 172.21.15.62:0/1716415773 conn(0x55a3de72a310 msgr2=0x55a3de763960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:15.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b62e86740 1 -- 172.21.15.62:0/1716415773 shutdown_connections 2024-09-12T00:07:15.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.712+0000 7f7b62e86740 1 -- 172.21.15.62:0/1716415773 wait complete. 2024-09-12T00:07:15.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.713+0000 7f7b62e86740 20 client.18111 trim_cache size 0 max 16384 2024-09-12T00:07:15.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.730+0000 7f7b62e86740 1 Processor -- start 2024-09-12T00:07:15.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.730+0000 7f7b62e86740 1 -- start start 2024-09-12T00:07:15.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de7615d0 0x55a3de778390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de764120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de76c370 0x55a3de764660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b62e86740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c1bf1e0 con 0x55a3de762b40 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b62e86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c1bcab0 con 0x55a3de76c370 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b62e86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c1ba920 con 0x55a3de7615d0 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b59d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de764120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b5a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de7615d0 0x55a3de778390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b5a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de7615d0 0x55a3de778390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:44208/0 (socket says 172.21.15.62:44208) 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b5a59c640 1 -- 172.21.15.62:0/4104885042 learned_addr learned my addr 172.21.15.62:0/4104885042 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b59d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de764120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56666/0 (socket says 172.21.15.62:56666) 2024-09-12T00:07:15.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b5ad9d640 1 --2- 172.21.15.62:0/4104885042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de76c370 0x55a3de764660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b5a59c640 1 -- 172.21.15.62:0/4104885042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de76c370 msgr2=0x55a3de764660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/4104885042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de76c370 0x55a3de764660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b5a59c640 1 -- 172.21.15.62:0/4104885042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 msgr2=0x55a3de764120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/4104885042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de764120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b5a59c640 1 -- 172.21.15.62:0/4104885042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3de76c040 con 0x55a3de7615d0 2024-09-12T00:07:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b5ad9d640 1 --2- 172.21.15.62:0/4104885042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de76c370 0x55a3de764660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.731+0000 7f7b59d9b640 1 --2- 172.21.15.62:0/4104885042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de764120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.732+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/4104885042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de7615d0 0x55a3de778390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009fd0 tx=0x7f7b50014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a024f551344c015 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.732+0000 7f7b43fff640 1 -- 172.21.15.62:0/4104885042 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5002d070 con 0x55a3de7615d0 2024-09-12T00:07:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.732+0000 7f7b43fff640 1 -- 172.21.15.62:0/4104885042 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b50004030 con 0x55a3de7615d0 2024-09-12T00:07:15.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.732+0000 7f7b43fff640 1 -- 172.21.15.62:0/4104885042 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5001e7b0 con 0x55a3de7615d0 2024-09-12T00:07:15.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 -- 172.21.15.62:0/4104885042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de7615d0 msgr2=0x55a3de778390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 --2- 172.21.15.62:0/4104885042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de7615d0 0x55a3de778390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009fd0 tx=0x7f7b50014000 comp rx=0 tx=0).stop 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 -- 172.21.15.62:0/4104885042 shutdown_connections 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 --2- 172.21.15.62:0/4104885042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de76c370 0x55a3de764660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 --2- 172.21.15.62:0/4104885042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de762b40 0x55a3de764120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 --2- 172.21.15.62:0/4104885042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de7615d0 0x55a3de778390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 -- 172.21.15.62:0/4104885042 >> 172.21.15.62:0/4104885042 conn(0x55a3de72a310 msgr2=0x55a3de75cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 -- 172.21.15.62:0/4104885042 shutdown_connections 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 -- 172.21.15.62:0/4104885042 wait complete. 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 -- start start 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.733+0000 7f7b62e86740 1 Processor -- start 2024-09-12T00:07:15.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de774a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de765a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b62e86740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de76c370 0x55a3de765fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b5a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de774a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b5a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de774a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37730/0 (socket says 172.21.15.62:37730) 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de765a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de765a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:44210/0 (socket says 172.21.15.62:44210) 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b5a59c640 1 -- 172.21.15.62:0/3006268102 learned_addr learned my addr 172.21.15.62:0/3006268102 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c1baf70 con 0x55a3de76c370 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c1bc3e0 con 0x55a3de7615d0 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.734+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c1bb640 con 0x55a3de762b40 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b62e86740 10 client.0 _ll_register_callbacks cb 0x7f7b540125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b5ad9d640 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de76c370 0x55a3de765fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b5a59c640 1 -- 172.21.15.62:0/3006268102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 msgr2=0x55a3de765a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de765a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b5a59c640 1 -- 172.21.15.62:0/3006268102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de76c370 msgr2=0x55a3de765fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de76c370 0x55a3de765fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b5a59c640 1 -- 172.21.15.62:0/3006268102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3de7665c0 con 0x55a3de7615d0 2024-09-12T00:07:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b5a59c640 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de774a80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7b50004650 tx=0x7f7b500050d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6825c2866d76c174 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:07:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b41ffb640 1 -- 172.21.15.62:0/3006268102 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5002d070 con 0x55a3de7615d0 2024-09-12T00:07:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b50004030 con 0x55a3de7615d0 2024-09-12T00:07:15.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.735+0000 7f7b41ffb640 1 -- 172.21.15.62:0/3006268102 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8557+0+0 (secure 0 0 0) 0x7f7b50005590 con 0x55a3de7615d0 2024-09-12T00:07:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.736+0000 7f7b41ffb640 1 -- 172.21.15.62:0/3006268102 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7b500046c0 con 0x55a3de7615d0 2024-09-12T00:07:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.736+0000 7f7b62e86740 10 client.27557 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.736+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a3de7e6770 con 0x55a3de7615d0 2024-09-12T00:07:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.736+0000 7f7b41ffb640 1 -- 172.21.15.62:0/3006268102 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b50022020 con 0x55a3de7615d0 2024-09-12T00:07:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.736+0000 7f7b41ffb640 1 -- 172.21.15.62:0/3006268102 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7b500279a0 con 0x55a3de7615d0 2024-09-12T00:07:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.736+0000 7f7b62e86740 10 client.27557 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.736+0000 7f7b62e86740 10 client.27557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:07:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.736+0000 7f7b62e86740 20 client.27557 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.736+0000 7f7b62e86740 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a3de77beb0 0x55a3de7e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.736+0000 7f7b62e86740 4 client.27557 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:07:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.737+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7b50004030 con 0x55a3de77beb0 2024-09-12T00:07:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.737+0000 7f7b59d9b640 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a3de77beb0 0x55a3de7e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.737+0000 7f7b41ffb640 10 client.27557 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.737+0000 7f7b59d9b640 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a3de77beb0 0x55a3de7e8ea0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e3dce70de8b9f0 server_cookie=6ea9fc19f925e746 in_seq=0 out_seq=0 2024-09-12T00:07:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b41ffb640 1 -- 172.21.15.62:0/3006268102 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7b50004030 con 0x55a3de77beb0 2024-09-12T00:07:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b41ffb640 10 client.27557 handle_command_reply: tid=0 2024-09-12T00:07:15.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 client.27557 shutdown 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 msgr2=0x55a3de774a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de774a80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7b50004650 tx=0x7f7b500050d0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 shutdown_connections 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55a3de77beb0 0x55a3de7e8ea0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3de76c370 0x55a3de765fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3de762b40 0x55a3de765a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b41ffb640 0 client.27557 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 --2- 172.21.15.62:0/3006268102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3de7615d0 0x55a3de774a80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 >> 172.21.15.62:0/3006268102 conn(0x55a3de72a310 msgr2=0x55a3de767720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 shutdown_connections 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 1 -- 172.21.15.62:0/3006268102 wait complete. 2024-09-12T00:07:15.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.738+0000 7f7b62e86740 20 client.27557 trim_cache size 0 max 16384 2024-09-12T00:07:15.743 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:07:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 -- 172.21.15.62:0/177129310 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7b440a2cf0 msgr2=0x7f7b440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7b440a2cf0 0x7f7b440a51b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55a3de75eac0 tx=0x7f7b50005f90 comp rx=0 tx=0).stop 2024-09-12T00:07:15.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 -- 172.21.15.62:0/177129310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b5c109110 msgr2=0x7f7b5c10e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b5c109110 0x7f7b5c10e110 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009230 tx=0x7f7b4c005e70 comp rx=0 tx=0).stop 2024-09-12T00:07:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 -- 172.21.15.62:0/177129310 shutdown_connections 2024-09-12T00:07:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f7b440a2cf0 0x7f7b440a51b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b5c109110 0x7f7b5c10e110 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b5c104990 0x7f7b5c10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 --2- 172.21.15.62:0/177129310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b5c103fe0 0x7f7b5c110b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 -- 172.21.15.62:0/177129310 >> 172.21.15.62:0/177129310 conn(0x7f7b5c0ffac0 msgr2=0x7f7b5c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 -- 172.21.15.62:0/177129310 shutdown_connections 2024-09-12T00:07:15.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:15.739+0000 7f7b60aad640 1 -- 172.21.15.62:0/177129310 wait complete. 2024-09-12T00:07:15.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:15 smithi120 ceph-mon[35182]: pgmap v472: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 1.95k op/s 2024-09-12T00:07:15.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:15 smithi083 ceph-mon[34805]: pgmap v472: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 1.95k op/s 2024-09-12T00:07:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 875.0 s, 1928.4 tps, lat 52.614 ms stddev 58.791 2024-09-12T00:07:17.662 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:07:17.662 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:07:17.713 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:17 smithi062 ceph-mon[30958]: pgmap v473: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 1.95k op/s 2024-09-12T00:07:17.713 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:17 smithi062 ceph-mon[30958]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T00:07:17.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:17 smithi120 ceph-mon[35182]: pgmap v473: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 1.95k op/s 2024-09-12T00:07:17.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:17 smithi120 ceph-mon[35182]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T00:07:17.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:17 smithi083 ceph-mon[34805]: pgmap v473: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 1.95k op/s 2024-09-12T00:07:17.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:17 smithi083 ceph-mon[34805]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T00:07:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:18.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.651+0000 7f0628e4f640 1 -- 172.21.15.62:0/1951479824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f062411b140 msgr2=0x7f062411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:18.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.651+0000 7f0628e4f640 1 --2- 172.21.15.62:0/1951479824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f062411b140 0x7f062411d530 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0614009900 tx=0x7f061402a440 comp rx=0 tx=0).stop 2024-09-12T00:07:18.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.652+0000 7f0628e4f640 1 -- 172.21.15.62:0/1951479824 shutdown_connections 2024-09-12T00:07:18.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.652+0000 7f0628e4f640 1 --2- 172.21.15.62:0/1951479824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f062411b140 0x7f062411d530 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.652+0000 7f0628e4f640 1 --2- 172.21.15.62:0/1951479824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0624109830 0x7f0624109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.652+0000 7f0628e4f640 1 --2- 172.21.15.62:0/1951479824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0624109450 0x7f0624110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.652+0000 7f0628e4f640 1 -- 172.21.15.62:0/1951479824 >> 172.21.15.62:0/1951479824 conn(0x7f062406e140 msgr2=0x7f062406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:18.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.654+0000 7f0628e4f640 1 -- 172.21.15.62:0/1951479824 shutdown_connections 2024-09-12T00:07:18.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.656+0000 7f0628e4f640 1 -- 172.21.15.62:0/1951479824 wait complete. 2024-09-12T00:07:18.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.658+0000 7f0628e4f640 1 Processor -- start 2024-09-12T00:07:18.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.659+0000 7f0628e4f640 1 -- start start 2024-09-12T00:07:18.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.660+0000 7f0628e4f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0624109450 0x7f06241a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.660+0000 7f0628e4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0624109830 0x7f06241a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.661+0000 7f0628e4f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f062411b140 0x7f06241abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.661+0000 7f0621d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0624109830 0x7f06241a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.661+0000 7f0621d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0624109830 0x7f06241a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:37748/0 (socket says 172.21.15.62:37748) 2024-09-12T00:07:18.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.661+0000 7f062259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0624109450 0x7f06241a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.661+0000 7f062259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0624109450 0x7f06241a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56682/0 (socket says 172.21.15.62:56682) 2024-09-12T00:07:18.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.662+0000 7f0621d9b640 1 -- 172.21.15.62:0/1940918862 learned_addr learned my addr 172.21.15.62:0/1940918862 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:18.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.662+0000 7f0622d9d640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f062411b140 0x7f06241abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.662+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0624120e10 con 0x7f0624109450 2024-09-12T00:07:18.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.662+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0624120c90 con 0x7f0624109830 2024-09-12T00:07:18.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.663+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0624120f90 con 0x7f062411b140 2024-09-12T00:07:18.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.663+0000 7f0621d9b640 1 -- 172.21.15.62:0/1940918862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f062411b140 msgr2=0x7f06241abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:18.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.663+0000 7f0621d9b640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f062411b140 0x7f06241abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.663+0000 7f0621d9b640 1 -- 172.21.15.62:0/1940918862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0624109450 msgr2=0x7f06241a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:18.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.663+0000 7f0621d9b640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0624109450 0x7f06241a72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.663+0000 7f0621d9b640 1 -- 172.21.15.62:0/1940918862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06241ac2a0 con 0x7f0624109830 2024-09-12T00:07:18.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.664+0000 7f0621d9b640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0624109830 0x7f06241a7830 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f061c00ba50 tx=0x7f061c00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:18.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.664+0000 7f06137fe640 1 -- 172.21.15.62:0/1940918862 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c019070 con 0x7f0624109830 2024-09-12T00:07:18.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.664+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06241ac590 con 0x7f0624109830 2024-09-12T00:07:18.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.664+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06241b3e70 con 0x7f0624109830 2024-09-12T00:07:18.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.664+0000 7f06137fe640 1 -- 172.21.15.62:0/1940918862 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c00cbe0 con 0x7f0624109830 2024-09-12T00:07:18.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.664+0000 7f06137fe640 1 -- 172.21.15.62:0/1940918862 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0076e0 con 0x7f0624109830 2024-09-12T00:07:18.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.669+0000 7f06137fe640 1 -- 172.21.15.62:0/1940918862 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f061c01c070 con 0x7f0624109830 2024-09-12T00:07:18.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.671+0000 7f062b228740 1 Processor -- start 2024-09-12T00:07:18.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f062b228740 1 -- start start 2024-09-12T00:07:18.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f06137fe640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f060c0a2d70 0x7f060c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f062b228740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a700f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f062b228740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 0x559e2a70b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f062b228740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a70bd40 0x559e2a70e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f062b228740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559e2a666280 con 0x559e2a701480 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f062b228740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e2a60db10 con 0x559e2a700b60 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f062b228740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559e2a662240 con 0x559e2a70bd40 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f0622d9d640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f060c0a2d70 0x7f060c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f06137fe640 1 -- 172.21.15.62:0/1940918862 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f061c0ce890 con 0x7f0624109830 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f0621d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 0x559e2a70b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f0622d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a70bd40 0x559e2a70e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f0621d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 0x559e2a70b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56698/0 (socket says 172.21.15.62:56698) 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f0621d9b640 1 -- 172.21.15.62:0/1694646194 learned_addr learned my addr 172.21.15.62:0/1694646194 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:18.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.672+0000 7f062259c640 1 --2- 172.21.15.62:0/1694646194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a700f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0621d9b640 1 -- 172.21.15.62:0/1694646194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a70bd40 msgr2=0x559e2a70e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0621d9b640 1 --2- 172.21.15.62:0/1694646194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a70bd40 0x559e2a70e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0621d9b640 1 -- 172.21.15.62:0/1694646194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 msgr2=0x559e2a700f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0621d9b640 1 --2- 172.21.15.62:0/1694646194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a700f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0621d9b640 1 -- 172.21.15.62:0/1694646194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e2a61a1f0 con 0x559e2a701480 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0622d9d640 1 --2- 172.21.15.62:0/1694646194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a70bd40 0x559e2a70e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0622d9d640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f060c0a2d70 0x7f060c0a5230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f061402a960 tx=0x7f0614003680 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0621d9b640 1 --2- 172.21.15.62:0/1694646194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 0x559e2a70b800 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f061c0a3600 tx=0x7f061c09aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1510a51c9b4da6ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0610ff9640 1 -- 172.21.15.62:0/1694646194 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0167b0 con 0x559e2a701480 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0610ff9640 1 -- 172.21.15.62:0/1694646194 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c09af70 con 0x559e2a701480 2024-09-12T00:07:18.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.673+0000 7f0610ff9640 1 -- 172.21.15.62:0/1694646194 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0975e0 con 0x559e2a701480 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.674+0000 7f062b228740 1 -- 172.21.15.62:0/1694646194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 msgr2=0x559e2a70b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.674+0000 7f062b228740 1 --2- 172.21.15.62:0/1694646194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 0x559e2a70b800 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f061c0a3600 tx=0x7f061c09aab0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.674+0000 7f062b228740 1 -- 172.21.15.62:0/1694646194 shutdown_connections 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.674+0000 7f062b228740 1 --2- 172.21.15.62:0/1694646194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a70bd40 0x559e2a70e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.674+0000 7f062b228740 1 --2- 172.21.15.62:0/1694646194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 0x559e2a70b800 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.674+0000 7f062b228740 1 --2- 172.21.15.62:0/1694646194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a700f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.674+0000 7f062b228740 1 -- 172.21.15.62:0/1694646194 >> 172.21.15.62:0/1694646194 conn(0x559e2a6fd0f0 msgr2=0x559e2a6fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.674+0000 7f062b228740 1 -- 172.21.15.62:0/1694646194 shutdown_connections 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.674+0000 7f062b228740 1 -- 172.21.15.62:0/1694646194 wait complete. 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.675+0000 7f062b228740 1 -- start start 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.675+0000 7f062b228740 1 Processor -- start 2024-09-12T00:07:18.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.675+0000 7f062b228740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a708c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.675+0000 7f062b228740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a709150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.675+0000 7f062b228740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 0x559e2a709690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.675+0000 7f062b228740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559e2a70e910 con 0x559e2a70bd40 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.675+0000 7f062b228740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e2a620f10 con 0x559e2a700b60 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.675+0000 7f062b228740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559e2a70ea90 con 0x559e2a701480 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.675+0000 7f062b228740 10 client.0 _ll_register_callbacks cb 0x559e2a5e9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.676+0000 7f0621d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a709150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.676+0000 7f0621d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a709150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:44250/0 (socket says 172.21.15.62:44250) 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.676+0000 7f0621d9b640 1 -- 172.21.15.62:0/1224334698 learned_addr learned my addr 172.21.15.62:0/1224334698 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.676+0000 7f0622d9d640 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 0x559e2a709690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.676+0000 7f0621d9b640 1 -- 172.21.15.62:0/1224334698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 msgr2=0x559e2a708c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.676+0000 7f0621d9b640 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a708c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.676+0000 7f0621d9b640 1 -- 172.21.15.62:0/1224334698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 msgr2=0x559e2a709690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:18.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.676+0000 7f0621d9b640 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 0x559e2a709690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.676+0000 7f0621d9b640 1 -- 172.21.15.62:0/1224334698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e2a3d76a0 con 0x559e2a701480 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.677+0000 7f0621d9b640 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a709150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f061c097940 tx=0x7f061c003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e6758810c65a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.677+0000 7f0609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.677+0000 7f0609ffb640 1 -- 172.21.15.62:0/1224334698 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0b7070 con 0x559e2a701480 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.677+0000 7f0609ffb640 1 -- 172.21.15.62:0/1224334698 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f061c09b3e0 con 0x559e2a701480 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.677+0000 7f062b228740 1 -- 172.21.15.62:0/1224334698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061c09af70 con 0x559e2a701480 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f0609ffb640 1 -- 172.21.15.62:0/1224334698 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f061c080050 con 0x559e2a701480 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f062b228740 10 client.27478 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f062b228740 1 -- 172.21.15.62:0/1224334698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559e2a70e910 con 0x559e2a701480 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f0609ffb640 1 -- 172.21.15.62:0/1224334698 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c09af70 con 0x559e2a701480 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f0609ffb640 1 -- 172.21.15.62:0/1224334698 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f061c00dd90 con 0x559e2a701480 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f062b228740 10 client.27478 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:18.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f062b228740 10 client.27478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f062b228740 20 client.27478 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f062b228740 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559e2a72ca80 0x559e2a72ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f062b228740 4 client.27478 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f062b228740 1 -- 172.21.15.62:0/1224334698 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e2a6737f0 con 0x559e2a72ca80 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.678+0000 7f062259c640 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559e2a72ca80 0x559e2a72ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.679+0000 7f062259c640 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559e2a72ca80 0x559e2a72ee90 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3eb505c2c27b875 server_cookie=3f77a3b3115af0ca in_seq=0 out_seq=0 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.679+0000 7f0609ffb640 10 client.27478 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.680+0000 7f0609ffb640 1 -- 172.21.15.62:0/1224334698 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e2a6737f0 con 0x559e2a72ca80 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.680+0000 7f0609ffb640 10 client.27478 handle_command_reply: tid=0 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.680+0000 7f062b228740 1 client.27478 shutdown 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 1 -- 172.21.15.62:0/1224334698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 msgr2=0x559e2a709150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a709150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f061c097940 tx=0x7f061c003ba0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 1 -- 172.21.15.62:0/1224334698 shutdown_connections 2024-09-12T00:07:18.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559e2a72ca80 0x559e2a72ee90 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 0x559e2a709690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a709150 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 1 --2- 172.21.15.62:0/1224334698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a708c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 1 -- 172.21.15.62:0/1224334698 >> 172.21.15.62:0/1224334698 conn(0x559e2a6fd0f0 msgr2=0x559e2a702e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:18.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f0609ffb640 0 client.27478 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:18.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 1 -- 172.21.15.62:0/1224334698 shutdown_connections 2024-09-12T00:07:18.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 1 -- 172.21.15.62:0/1224334698 wait complete. 2024-09-12T00:07:18.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.681+0000 7f062b228740 20 client.27478 trim_cache size 0 max 16384 2024-09-12T00:07:18.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.706+0000 7f062b228740 1 Processor -- start 2024-09-12T00:07:18.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.706+0000 7f062b228740 1 -- start start 2024-09-12T00:07:18.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.706+0000 7f062b228740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a703590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.707+0000 7f062b228740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a7347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.707+0000 7f062b228740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 0x559e2a734ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.707+0000 7f062b228740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06241b6790 con 0x559e2a70bd40 2024-09-12T00:07:18.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.707+0000 7f062b228740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06241b4090 con 0x559e2a700b60 2024-09-12T00:07:18.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.707+0000 7f062b228740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0624116fc0 con 0x559e2a701480 2024-09-12T00:07:18.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.707+0000 7f062259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a703590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.707+0000 7f0621d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a7347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.707+0000 7f0621d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a7347a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:44266/0 (socket says 172.21.15.62:44266) 2024-09-12T00:07:18.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.707+0000 7f0621d9b640 1 -- 172.21.15.62:0/1848731940 learned_addr learned my addr 172.21.15.62:0/1848731940 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:18.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.708+0000 7f0622d9d640 1 --2- 172.21.15.62:0/1848731940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 0x559e2a734ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.708+0000 7f062259c640 1 -- 172.21.15.62:0/1848731940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 msgr2=0x559e2a7347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:18.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.708+0000 7f062259c640 1 --2- 172.21.15.62:0/1848731940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a7347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.708+0000 7f062259c640 1 -- 172.21.15.62:0/1848731940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 msgr2=0x559e2a734ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:18.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.708+0000 7f062259c640 1 --2- 172.21.15.62:0/1848731940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 0x559e2a734ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.708+0000 7f062259c640 1 -- 172.21.15.62:0/1848731940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e2a72f510 con 0x559e2a700b60 2024-09-12T00:07:18.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.708+0000 7f0622d9d640 1 --2- 172.21.15.62:0/1848731940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 0x559e2a734ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:07:18.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.708+0000 7f062259c640 1 --2- 172.21.15.62:0/1848731940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a703590 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x559e2a72f490 tx=0x7f0618010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def638967224d432 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:18.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.709+0000 7f060bfff640 1 -- 172.21.15.62:0/1848731940 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618022070 con 0x559e2a700b60 2024-09-12T00:07:18.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.709+0000 7f060bfff640 1 -- 172.21.15.62:0/1848731940 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06180035f0 con 0x559e2a700b60 2024-09-12T00:07:18.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.709+0000 7f060bfff640 1 -- 172.21.15.62:0/1848731940 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061801c680 con 0x559e2a700b60 2024-09-12T00:07:18.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.710+0000 7f062b228740 1 -- 172.21.15.62:0/1848731940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 msgr2=0x559e2a703590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:18.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.710+0000 7f062b228740 1 --2- 172.21.15.62:0/1848731940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a703590 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x559e2a72f490 tx=0x7f0618010800 comp rx=0 tx=0).stop 2024-09-12T00:07:18.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.710+0000 7f062b228740 1 -- 172.21.15.62:0/1848731940 shutdown_connections 2024-09-12T00:07:18.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.710+0000 7f062b228740 1 --2- 172.21.15.62:0/1848731940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a70bd40 0x559e2a734ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.710+0000 7f062b228740 1 --2- 172.21.15.62:0/1848731940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a701480 0x559e2a7347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.710+0000 7f062b228740 1 --2- 172.21.15.62:0/1848731940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a700b60 0x559e2a703590 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.710+0000 7f062b228740 1 -- 172.21.15.62:0/1848731940 >> 172.21.15.62:0/1848731940 conn(0x559e2a6fd0f0 msgr2=0x559e2a70ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:18.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.716+0000 7f062b228740 1 -- 172.21.15.62:0/1848731940 shutdown_connections 2024-09-12T00:07:18.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.720+0000 7f062b228740 1 -- 172.21.15.62:0/1848731940 wait complete. 2024-09-12T00:07:18.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.720+0000 7f062b228740 1 -- start start 2024-09-12T00:07:18.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.721+0000 7f062b228740 1 Processor -- start 2024-09-12T00:07:18.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.721+0000 7f062b228740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a700b60 0x559e2a710150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.721+0000 7f062b228740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 0x559e2a710690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.722+0000 7f062b228740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a70bd40 0x559e2a796770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.722+0000 7f062b228740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0624117610 con 0x559e2a701480 2024-09-12T00:07:18.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.722+0000 7f062b228740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0624118a80 con 0x559e2a70bd40 2024-09-12T00:07:18.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.722+0000 7f062b228740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0624117ce0 con 0x559e2a700b60 2024-09-12T00:07:18.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.722+0000 7f062b228740 10 client.0 _ll_register_callbacks cb 0x7f06180097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.722+0000 7f0622d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a70bd40 0x559e2a796770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.722+0000 7f062259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a700b60 0x559e2a710150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.722+0000 7f062259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a700b60 0x559e2a710150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:44280/0 (socket says 172.21.15.62:44280) 2024-09-12T00:07:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.722+0000 7f062259c640 1 -- 172.21.15.62:0/3526848699 learned_addr learned my addr 172.21.15.62:0/3526848699 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.723+0000 7f0621d9b640 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 0x559e2a710690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.724+0000 7f062259c640 1 -- 172.21.15.62:0/3526848699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a70bd40 msgr2=0x559e2a796770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.724+0000 7f062259c640 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a70bd40 0x559e2a796770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.724+0000 7f062259c640 1 -- 172.21.15.62:0/3526848699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 msgr2=0x559e2a710690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.724+0000 7f062259c640 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 0x559e2a710690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.724+0000 7f062259c640 1 -- 172.21.15.62:0/3526848699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e2a6fcf20 con 0x559e2a700b60 2024-09-12T00:07:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.724+0000 7f062259c640 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a700b60 0x559e2a710150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0618009aa0 tx=0x7f0618009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6307717def3bb499 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.724+0000 7f0609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:07:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.725+0000 7f0609ffb640 1 -- 172.21.15.62:0/3526848699 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618022040 con 0x559e2a700b60 2024-09-12T00:07:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.725+0000 7f0609ffb640 1 -- 172.21.15.62:0/3526848699 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f06180042d0 con 0x559e2a700b60 2024-09-12T00:07:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.725+0000 7f062b228740 1 -- 172.21.15.62:0/3526848699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06180035f0 con 0x559e2a700b60 2024-09-12T00:07:18.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.726+0000 7f0609ffb640 1 -- 172.21.15.62:0/3526848699 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06180044d0 con 0x559e2a700b60 2024-09-12T00:07:18.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.727+0000 7f062b228740 10 client.27484 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:18.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.727+0000 7f062b228740 1 -- 172.21.15.62:0/3526848699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559e2a6f8af0 con 0x559e2a700b60 2024-09-12T00:07:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.728+0000 7f0609ffb640 1 -- 172.21.15.62:0/3526848699 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061802c020 con 0x559e2a700b60 2024-09-12T00:07:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.729+0000 7f0609ffb640 1 -- 172.21.15.62:0/3526848699 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f0618039080 con 0x559e2a700b60 2024-09-12T00:07:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.729+0000 7f062b228740 10 client.27484 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.729+0000 7f062b228740 10 client.27484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:07:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.729+0000 7f062b228740 20 client.27484 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.729+0000 7f062b228740 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559e2a798150 0x559e2a79a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.729+0000 7f062b228740 4 client.27484 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:07:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.729+0000 7f062b228740 1 -- 172.21.15.62:0/3526848699 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f06180035f0 con 0x559e2a798150 2024-09-12T00:07:18.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.731+0000 7f0621d9b640 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559e2a798150 0x559e2a79a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:18.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.742+0000 7f0621d9b640 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559e2a798150 0x559e2a79a540 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857882353f7916c5 server_cookie=14635787ba823be1 in_seq=0 out_seq=0 2024-09-12T00:07:18.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.742+0000 7f0609ffb640 10 client.27484 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:18.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.747+0000 7f0609ffb640 1 -- 172.21.15.62:0/3526848699 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f06180035f0 con 0x559e2a798150 2024-09-12T00:07:18.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.747+0000 7f0609ffb640 10 client.27484 handle_command_reply: tid=0 2024-09-12T00:07:18.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.747+0000 7f062b228740 1 client.27484 shutdown 2024-09-12T00:07:18.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.747+0000 7f062b228740 1 -- 172.21.15.62:0/3526848699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a700b60 msgr2=0x559e2a710150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:18.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.747+0000 7f062b228740 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a700b60 0x559e2a710150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0618009aa0 tx=0x7f0618009e10 comp rx=0 tx=0).stop 2024-09-12T00:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.751+0000 7f062b228740 1 -- 172.21.15.62:0/3526848699 shutdown_connections 2024-09-12T00:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.751+0000 7f062b228740 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559e2a798150 0x559e2a79a540 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.751+0000 7f062b228740 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e2a70bd40 0x559e2a796770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.751+0000 7f062b228740 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e2a701480 0x559e2a710690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.751+0000 7f062b228740 1 --2- 172.21.15.62:0/3526848699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e2a700b60 0x559e2a710150 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.751+0000 7f062b228740 1 -- 172.21.15.62:0/3526848699 >> 172.21.15.62:0/3526848699 conn(0x559e2a6fd0f0 msgr2=0x559e2a70a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.751+0000 7f0609ffb640 0 client.27484 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:18.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.755+0000 7f062b228740 1 -- 172.21.15.62:0/3526848699 shutdown_connections 2024-09-12T00:07:18.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.755+0000 7f062b228740 1 -- 172.21.15.62:0/3526848699 wait complete. 2024-09-12T00:07:18.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.759+0000 7f062b228740 20 client.27484 trim_cache size 0 max 16384 2024-09-12T00:07:18.757 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:07:18.758 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-09-12T00:07:18.758 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "d948402a-766e-410a-aa32-a3db77be0207", 2024-09-12T00:07:18.758 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-09-12T00:07:18.758 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:07:18.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.761+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f060c0a2d70 msgr2=0x7f060c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:18.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.761+0000 7f0628e4f640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f060c0a2d70 0x7f060c0a5230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f061402a960 tx=0x7f0614003680 comp rx=0 tx=0).stop 2024-09-12T00:07:18.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.761+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0624109830 msgr2=0x7f06241a7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:18.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.761+0000 7f0628e4f640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0624109830 0x7f06241a7830 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f061c00ba50 tx=0x7f061c00bf10 comp rx=0 tx=0).stop 2024-09-12T00:07:18.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.761+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 shutdown_connections 2024-09-12T00:07:18.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.761+0000 7f0628e4f640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f060c0a2d70 0x7f060c0a5230 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.761+0000 7f0628e4f640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f062411b140 0x7f06241abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.761+0000 7f0628e4f640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0624109830 0x7f06241a7830 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.761+0000 7f0628e4f640 1 --2- 172.21.15.62:0/1940918862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0624109450 0x7f06241a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:18.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.761+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 >> 172.21.15.62:0/1940918862 conn(0x7f062406e140 msgr2=0x7f062411bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:18.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.762+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 shutdown_connections 2024-09-12T00:07:18.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:18.762+0000 7f0628e4f640 1 -- 172.21.15.62:0/1940918862 wait complete. 2024-09-12T00:07:19.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:19 smithi062 ceph-mon[30958]: pgmap v475: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 913 op/s 2024-09-12T00:07:19.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:19 smithi062 ceph-mon[30958]: scrub queued for path: / 2024-09-12T00:07:19.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:19 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:19.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:19 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:07:19.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:19 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:07:19.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:19 smithi062 ceph-mon[30958]: scrub summary: active paths [/] 2024-09-12T00:07:19.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:07:19.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:19 smithi120 ceph-mon[35182]: pgmap v475: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 913 op/s 2024-09-12T00:07:19.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:19 smithi120 ceph-mon[35182]: scrub queued for path: / 2024-09-12T00:07:19.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:19 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:19.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:19 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:07:19.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:19 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:07:19.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:19 smithi120 ceph-mon[35182]: scrub summary: active paths [/] 2024-09-12T00:07:19.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:19 smithi083 ceph-mon[34805]: pgmap v475: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 913 op/s 2024-09-12T00:07:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:19 smithi083 ceph-mon[34805]: scrub queued for path: / 2024-09-12T00:07:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:19 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:19 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:07:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:19 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:07:19.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:19 smithi083 ceph-mon[34805]: scrub summary: active paths [/] 2024-09-12T00:07:20.219 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:20.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:20 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:20.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:20 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:20.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:20 smithi062 ceph-mon[30958]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:20.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:20 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:20.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:20 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:20.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:20 smithi120 ceph-mon[35182]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:20.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:20 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:20 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:20.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:20 smithi083 ceph-mon[34805]: scrub summary: idle+waiting paths [/] 2024-09-12T00:07:21.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.303+0000 7f5354974640 1 -- 172.21.15.62:0/2922731304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53501009b0 msgr2=0x7f5350100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:21.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.303+0000 7f5354974640 1 --2- 172.21.15.62:0/2922731304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53501009b0 0x7f5350100d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5340009860 tx=0x7f534002a340 comp rx=0 tx=0).stop 2024-09-12T00:07:21.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.303+0000 7f5354974640 1 -- 172.21.15.62:0/2922731304 shutdown_connections 2024-09-12T00:07:21.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.303+0000 7f5354974640 1 --2- 172.21.15.62:0/2922731304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535010e4d0 0x7f53501108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.303+0000 7f5354974640 1 --2- 172.21.15.62:0/2922731304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5350101360 0x7f535010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.303+0000 7f5354974640 1 --2- 172.21.15.62:0/2922731304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53501009b0 0x7f5350100d90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.303+0000 7f5354974640 1 -- 172.21.15.62:0/2922731304 >> 172.21.15.62:0/2922731304 conn(0x7f53500fc7d0 msgr2=0x7f53500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:21.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.303+0000 7f5354974640 1 -- 172.21.15.62:0/2922731304 shutdown_connections 2024-09-12T00:07:21.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.303+0000 7f5354974640 1 -- 172.21.15.62:0/2922731304 wait complete. 2024-09-12T00:07:21.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.304+0000 7f5354974640 1 Processor -- start 2024-09-12T00:07:21.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.304+0000 7f5354974640 1 -- start start 2024-09-12T00:07:21.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f5354974640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53501009b0 0x7f53501a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f5354974640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5350101360 0x7f53501a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f5354974640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535010e4d0 0x7f535010cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f534e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53501009b0 0x7f53501a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f534e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53501009b0 0x7f53501a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41214/0 (socket says 172.21.15.62:41214) 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f534e59c640 1 -- 172.21.15.62:0/2822544653 learned_addr learned my addr 172.21.15.62:0/2822544653 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5350103fe0 con 0x7f53501009b0 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f534dd9b640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5350101360 0x7f53501a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5350103e60 con 0x7f535010e4d0 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5350104160 con 0x7f5350101360 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.305+0000 7f534ed9d640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535010e4d0 0x7f535010cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f534dd9b640 1 -- 172.21.15.62:0/2822544653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535010e4d0 msgr2=0x7f535010cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f534dd9b640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535010e4d0 0x7f535010cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f534dd9b640 1 -- 172.21.15.62:0/2822544653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53501009b0 msgr2=0x7f53501a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:21.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f534dd9b640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53501009b0 0x7f53501a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f534dd9b640 1 -- 172.21.15.62:0/2822544653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535010d3e0 con 0x7f5350101360 2024-09-12T00:07:21.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f534ed9d640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535010e4d0 0x7f535010cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:21.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f534dd9b640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5350101360 0x7f53501a4e90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f534400e9f0 tx=0x7f534400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:21.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f533f7fe640 1 -- 172.21.15.62:0/2822544653 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534400ccc0 con 0x7f5350101360 2024-09-12T00:07:21.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f533f7fe640 1 -- 172.21.15.62:0/2822544653 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5344005020 con 0x7f5350101360 2024-09-12T00:07:21.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f533f7fe640 1 -- 172.21.15.62:0/2822544653 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53440049e0 con 0x7f5350101360 2024-09-12T00:07:21.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535010d6d0 con 0x7f5350101360 2024-09-12T00:07:21.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.306+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53501ab7b0 con 0x7f5350101360 2024-09-12T00:07:21.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.310+0000 7f533f7fe640 1 -- 172.21.15.62:0/2822544653 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5344005560 con 0x7f5350101360 2024-09-12T00:07:21.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.310+0000 7f5356d4d740 1 Processor -- start 2024-09-12T00:07:21.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.310+0000 7f5356d4d740 1 -- start start 2024-09-12T00:07:21.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.310+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ff5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.310+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c7000140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.311+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 0x5602c7002a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.311+0000 7f5356d4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5602c6f5b2e0 con 0x5602c6ff5dc0 2024-09-12T00:07:21.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.311+0000 7f5356d4d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602c6f02ce0 con 0x5602c7000680 2024-09-12T00:07:21.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.311+0000 7f5356d4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5602c6f572a0 con 0x5602c6ff54a0 2024-09-12T00:07:21.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.311+0000 7f534e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c7000140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.311+0000 7f534e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c7000140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41226/0 (socket says 172.21.15.62:41226) 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.311+0000 7f534e59c640 1 -- 172.21.15.62:0/1499124318 learned_addr learned my addr 172.21.15.62:0/1499124318 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.311+0000 7f534ed9d640 1 --2- 172.21.15.62:0/1499124318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ff5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f534dd9b640 1 --2- 172.21.15.62:0/1499124318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 0x5602c7002a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f533f7fe640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f53380a2df0 0x7f53380a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f534e59c640 1 -- 172.21.15.62:0/1499124318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 msgr2=0x5602c6ff5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f534e59c640 1 --2- 172.21.15.62:0/1499124318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ff5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f534e59c640 1 -- 172.21.15.62:0/1499124318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 msgr2=0x5602c7002a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f534e59c640 1 --2- 172.21.15.62:0/1499124318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 0x5602c7002a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f534e59c640 1 -- 172.21.15.62:0/1499124318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c6ee7440 con 0x5602c6ff5dc0 2024-09-12T00:07:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f533f7fe640 1 -- 172.21.15.62:0/2822544653 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f53440cf3c0 con 0x7f5350101360 2024-09-12T00:07:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f534ed9d640 1 --2- 172.21.15.62:0/1499124318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ff5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f534ed9d640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f53380a2df0 0x7f53380a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.312+0000 7f534e59c640 1 --2- 172.21.15.62:0/1499124318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c7000140 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5340009510 tx=0x7f5340009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8c8aae41997afb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.313+0000 7f533cff9640 1 -- 172.21.15.62:0/1499124318 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340041070 con 0x5602c6ff5dc0 2024-09-12T00:07:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.313+0000 7f533cff9640 1 -- 172.21.15.62:0/1499124318 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5340030820 con 0x5602c6ff5dc0 2024-09-12T00:07:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.313+0000 7f533cff9640 1 -- 172.21.15.62:0/1499124318 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340037040 con 0x5602c6ff5dc0 2024-09-12T00:07:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 -- 172.21.15.62:0/1499124318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 msgr2=0x5602c7000140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f534ed9d640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f53380a2df0 0x7f53380a52b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5348009590 tx=0x7f5348010040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 --2- 172.21.15.62:0/1499124318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c7000140 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5340009510 tx=0x7f5340009540 comp rx=0 tx=0).stop 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 -- 172.21.15.62:0/1499124318 shutdown_connections 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 --2- 172.21.15.62:0/1499124318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 0x5602c7002a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 --2- 172.21.15.62:0/1499124318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c7000140 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 --2- 172.21.15.62:0/1499124318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ff5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 -- 172.21.15.62:0/1499124318 >> 172.21.15.62:0/1499124318 conn(0x5602c6ff1a30 msgr2=0x5602c6ff1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 -- 172.21.15.62:0/1499124318 shutdown_connections 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 -- 172.21.15.62:0/1499124318 wait complete. 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 -- start start 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.314+0000 7f5356d4d740 1 Processor -- start 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ffff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c6ffd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 0x5602c6ffd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f5356d4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5602c7003250 con 0x5602c6ff5dc0 2024-09-12T00:07:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f5356d4d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602c6f16050 con 0x5602c7000680 2024-09-12T00:07:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f5356d4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5602c70033d0 con 0x5602c6ff54a0 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f534dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c6ffd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f5356d4d740 10 client.0 _ll_register_callbacks cb 0x5602c6f56160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f534dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c6ffd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41236/0 (socket says 172.21.15.62:41236) 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f534dd9b640 1 -- 172.21.15.62:0/3724434452 learned_addr learned my addr 172.21.15.62:0/3724434452 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f534e59c640 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ffff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.315+0000 7f534ed9d640 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 0x5602c6ffd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.316+0000 7f534ed9d640 1 -- 172.21.15.62:0/3724434452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 msgr2=0x5602c6ffff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.316+0000 7f534ed9d640 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ffff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.316+0000 7f534ed9d640 1 -- 172.21.15.62:0/3724434452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 msgr2=0x5602c6ffd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.316+0000 7f534ed9d640 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c6ffd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.316+0000 7f534ed9d640 1 -- 172.21.15.62:0/3724434452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c6ccc3c0 con 0x5602c7000680 2024-09-12T00:07:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.316+0000 7f534e59c640 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ffff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:07:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.317+0000 7f534ed9d640 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 0x5602c6ffd740 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5348009fd0 tx=0x7f5348004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50f12c694f722c9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.317+0000 7f5335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-12T00:07:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.317+0000 7f5335ffb640 1 -- 172.21.15.62:0/3724434452 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534802b060 con 0x5602c7000680 2024-09-12T00:07:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.317+0000 7f5356d4d740 1 -- 172.21.15.62:0/3724434452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5340030820 con 0x5602c7000680 2024-09-12T00:07:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.318+0000 7f5335ffb640 1 -- 172.21.15.62:0/3724434452 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f53480045a0 con 0x5602c7000680 2024-09-12T00:07:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.318+0000 7f5335ffb640 1 -- 172.21.15.62:0/3724434452 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f53480047a0 con 0x5602c7000680 2024-09-12T00:07:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.318+0000 7f5356d4d740 10 client.27581 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.318+0000 7f5356d4d740 1 -- 172.21.15.62:0/3724434452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5602c700ec50 con 0x5602c7000680 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.318+0000 7f5335ffb640 1 -- 172.21.15.62:0/3724434452 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5340030820 con 0x5602c7000680 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.319+0000 7f5335ffb640 1 -- 172.21.15.62:0/3724434452 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f534800b040 con 0x5602c7000680 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.319+0000 7f5356d4d740 10 client.27581 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.319+0000 7f5356d4d740 10 client.27581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.319+0000 7f5356d4d740 20 client.27581 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.319+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c7010440 0x5602c7012900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.319+0000 7f5356d4d740 4 client.27581 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.319+0000 7f5356d4d740 1 -- 172.21.15.62:0/3724434452 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602c6f68850 con 0x5602c7010440 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.319+0000 7f534e59c640 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c7010440 0x5602c7012900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.320+0000 7f534e59c640 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c7010440 0x5602c7012900 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a4f9d82d8b6088 server_cookie=d9002be350858957 in_seq=0 out_seq=0 2024-09-12T00:07:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.320+0000 7f5335ffb640 10 client.27581 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5335ffb640 1 -- 172.21.15.62:0/3724434452 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602c6f68850 con 0x5602c7010440 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5335ffb640 10 client.27581 handle_command_reply: tid=0 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 client.27581 shutdown 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 -- 172.21.15.62:0/3724434452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 msgr2=0x5602c6ffd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 0x5602c6ffd740 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5348009fd0 tx=0x7f5348004a40 comp rx=0 tx=0).stop 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 -- 172.21.15.62:0/3724434452 shutdown_connections 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c7010440 0x5602c7012900 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c7000680 0x5602c6ffd740 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c6ffd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5335ffb640 0 client.27581 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3724434452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ffff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 -- 172.21.15.62:0/3724434452 >> 172.21.15.62:0/3724434452 conn(0x5602c6ff1a30 msgr2=0x5602c6ff7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 -- 172.21.15.62:0/3724434452 shutdown_connections 2024-09-12T00:07:21.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 1 -- 172.21.15.62:0/3724434452 wait complete. 2024-09-12T00:07:21.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.321+0000 7f5356d4d740 20 client.27581 trim_cache size 0 max 16384 2024-09-12T00:07:21.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.347+0000 7f5356d4d740 1 Processor -- start 2024-09-12T00:07:21.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.348+0000 7f5356d4d740 1 -- start start 2024-09-12T00:07:21.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.348+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff1a30 0x5602c700c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.348+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ff87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.348+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff5dc0 0x5602c7018210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.348+0000 7f5356d4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f53501b6780 con 0x5602c6ff1a30 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.348+0000 7f5356d4d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53501b4050 con 0x5602c6ff5dc0 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.348+0000 7f5356d4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53501b1ec0 con 0x5602c6ff54a0 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.348+0000 7f534ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff5dc0 0x5602c7018210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.348+0000 7f534ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff5dc0 0x5602c7018210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:59182/0 (socket says 172.21.15.62:59182) 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.348+0000 7f534ed9d640 1 -- 172.21.15.62:0/1736250758 learned_addr learned my addr 172.21.15.62:0/1736250758 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.349+0000 7f534ed9d640 1 -- 172.21.15.62:0/1736250758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 msgr2=0x5602c6ff87a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.349+0000 7f534ed9d640 1 --2- 172.21.15.62:0/1736250758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ff87a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.349+0000 7f534ed9d640 1 -- 172.21.15.62:0/1736250758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff1a30 msgr2=0x5602c700c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.349+0000 7f534ed9d640 1 --2- 172.21.15.62:0/1736250758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff1a30 0x5602c700c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.349+0000 7f534ed9d640 1 -- 172.21.15.62:0/1736250758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c6ff5bf0 con 0x5602c6ff5dc0 2024-09-12T00:07:21.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.349+0000 7f534ed9d640 1 --2- 172.21.15.62:0/1736250758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff5dc0 0x5602c7018210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f534800c010 tx=0x7f534800b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99dc6296406986 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:21.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.349+0000 7f5337fff640 1 -- 172.21.15.62:0/1736250758 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348009de0 con 0x5602c6ff5dc0 2024-09-12T00:07:21.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.350+0000 7f5337fff640 1 -- 172.21.15.62:0/1736250758 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f534800b040 con 0x5602c6ff5dc0 2024-09-12T00:07:21.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.350+0000 7f5337fff640 1 -- 172.21.15.62:0/1736250758 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534801a680 con 0x5602c6ff5dc0 2024-09-12T00:07:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.351+0000 7f5356d4d740 1 -- 172.21.15.62:0/1736250758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff5dc0 msgr2=0x5602c7018210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.351+0000 7f5356d4d740 1 --2- 172.21.15.62:0/1736250758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff5dc0 0x5602c7018210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f534800c010 tx=0x7f534800b780 comp rx=0 tx=0).stop 2024-09-12T00:07:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.351+0000 7f5356d4d740 1 -- 172.21.15.62:0/1736250758 shutdown_connections 2024-09-12T00:07:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.351+0000 7f5356d4d740 1 --2- 172.21.15.62:0/1736250758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff5dc0 0x5602c7018210 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.351+0000 7f5356d4d740 1 --2- 172.21.15.62:0/1736250758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c6ff87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.351+0000 7f5356d4d740 1 --2- 172.21.15.62:0/1736250758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff1a30 0x5602c700c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.351+0000 7f5356d4d740 1 -- 172.21.15.62:0/1736250758 >> 172.21.15.62:0/1736250758 conn(0x5602c6ffd8a0 msgr2=0x5602c6ff8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.352+0000 7f5356d4d740 1 -- 172.21.15.62:0/1736250758 shutdown_connections 2024-09-12T00:07:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.352+0000 7f5356d4d740 1 -- 172.21.15.62:0/1736250758 wait complete. 2024-09-12T00:07:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.352+0000 7f5356d4d740 1 -- start start 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.352+0000 7f5356d4d740 1 Processor -- start 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.353+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff1a30 0x5602c7008b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.353+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c7003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.353+0000 7f5356d4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c70043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.353+0000 7f5356d4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f53501b2510 con 0x5602c6ff5dc0 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.353+0000 7f5356d4d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53501b3980 con 0x5602c6ff1a30 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.353+0000 7f5356d4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53501b2be0 con 0x5602c6ff54a0 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.353+0000 7f5356d4d740 10 client.0 _ll_register_callbacks cb 0x7f534800ff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.353+0000 7f534ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c70043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.353+0000 7f534ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c70043d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41252/0 (socket says 172.21.15.62:41252) 2024-09-12T00:07:21.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.353+0000 7f534ed9d640 1 -- 172.21.15.62:0/3456402353 learned_addr learned my addr 172.21.15.62:0/3456402353 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:21.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.359+0000 7f534ed9d640 1 -- 172.21.15.62:0/3456402353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 msgr2=0x5602c7003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:21.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.359+0000 7f534ed9d640 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c7003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.359+0000 7f534ed9d640 1 -- 172.21.15.62:0/3456402353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff1a30 msgr2=0x5602c7008b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:07:21.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.359+0000 7f534ed9d640 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff1a30 0x5602c7008b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.359+0000 7f534ed9d640 1 -- 172.21.15.62:0/3456402353 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c70049c0 con 0x5602c6ff5dc0 2024-09-12T00:07:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.360+0000 7f534ed9d640 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c70043d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5348026640 tx=0x7f53480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d74777a1cd6cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.363+0000 7f5335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:07:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.363+0000 7f5335ffb640 1 -- 172.21.15.62:0/3456402353 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534802b070 con 0x5602c6ff5dc0 2024-09-12T00:07:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.363+0000 7f5356d4d740 1 -- 172.21.15.62:0/3456402353 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f534800b040 con 0x5602c6ff5dc0 2024-09-12T00:07:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.363+0000 7f5335ffb640 1 -- 172.21.15.62:0/3456402353 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f5348026990 con 0x5602c6ff5dc0 2024-09-12T00:07:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.366+0000 7f5335ffb640 1 -- 172.21.15.62:0/3456402353 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5348069050 con 0x5602c6ff5dc0 2024-09-12T00:07:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.369+0000 7f5356d4d740 10 client.18144 fetch_fsmap learned FSMap version 39 2024-09-12T00:07:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.369+0000 7f5356d4d740 1 -- 172.21.15.62:0/3456402353 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5602c707aff0 con 0x5602c6ff5dc0 2024-09-12T00:07:21.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.373+0000 7f5335ffb640 1 -- 172.21.15.62:0/3456402353 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f534805a920 con 0x5602c6ff5dc0 2024-09-12T00:07:21.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.373+0000 7f5335ffb640 1 -- 172.21.15.62:0/3456402353 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f534801a5e0 con 0x5602c6ff5dc0 2024-09-12T00:07:21.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.373+0000 7f5356d4d740 10 client.18144 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T00:07:21.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.373+0000 7f5356d4d740 10 client.18144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:07:21.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.373+0000 7f5356d4d740 20 client.18144 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:21.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.373+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c7000680 0x5602c707d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:21.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.373+0000 7f5356d4d740 4 client.18144 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:07:21.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.373+0000 7f5356d4d740 1 -- 172.21.15.62:0/3456402353 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f534800b040 con 0x5602c7000680 2024-09-12T00:07:21.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.385+0000 7f534e59c640 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c7000680 0x5602c707d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:21.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.397+0000 7f534e59c640 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c7000680 0x5602c707d710 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c9eb91b3e9c5c86 server_cookie=c5d07244d8bd70b2 in_seq=0 out_seq=0 2024-09-12T00:07:21.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.397+0000 7f5335ffb640 10 client.18144 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:21.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.401+0000 7f5335ffb640 1 -- 172.21.15.62:0/3456402353 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f534800b040 con 0x5602c7000680 2024-09-12T00:07:21.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.401+0000 7f5335ffb640 10 client.18144 handle_command_reply: tid=0 2024-09-12T00:07:21.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.401+0000 7f5356d4d740 1 client.18144 shutdown 2024-09-12T00:07:21.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.405+0000 7f5356d4d740 1 -- 172.21.15.62:0/3456402353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 msgr2=0x5602c70043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:21.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.405+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c70043d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5348026640 tx=0x7f53480095c0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.409+0000 7f5356d4d740 1 -- 172.21.15.62:0/3456402353 shutdown_connections 2024-09-12T00:07:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.409+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x5602c7000680 0x5602c707d710 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.409+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602c6ff5dc0 0x5602c70043d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.409+0000 7f5335ffb640 0 client.18144 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.409+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602c6ff54a0 0x5602c7003e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.409+0000 7f5356d4d740 1 --2- 172.21.15.62:0/3456402353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c6ff1a30 0x5602c7008b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.409+0000 7f5356d4d740 1 -- 172.21.15.62:0/3456402353 >> 172.21.15.62:0/3456402353 conn(0x5602c6ffd8a0 msgr2=0x5602c6fff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:21.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.409+0000 7f5356d4d740 1 -- 172.21.15.62:0/3456402353 shutdown_connections 2024-09-12T00:07:21.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.409+0000 7f5356d4d740 1 -- 172.21.15.62:0/3456402353 wait complete. 2024-09-12T00:07:21.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.409+0000 7f5356d4d740 20 client.18144 trim_cache size 0 max 16384 2024-09-12T00:07:21.409 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:07:21.409 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:07:21.409 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-09-12T00:07:21.409 INFO:teuthology.orchestra.run.smithi062.stdout: "d948402a-766e-410a-aa32-a3db77be0207": { 2024-09-12T00:07:21.409 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-09-12T00:07:21.410 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "d948402a-766e-410a-aa32-a3db77be0207", 2024-09-12T00:07:21.410 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-09-12T00:07:21.410 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:07:21.410 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:07:21.410 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:07:21.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.410+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f53380a2df0 msgr2=0x7f53380a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:21.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.410+0000 7f5354974640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f53380a2df0 0x7f53380a52b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5348009590 tx=0x7f5348010040 comp rx=0 tx=0).stop 2024-09-12T00:07:21.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.410+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5350101360 msgr2=0x7f53501a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:21.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.410+0000 7f5354974640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5350101360 0x7f53501a4e90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f534400e9f0 tx=0x7f534400eeb0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.413+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 shutdown_connections 2024-09-12T00:07:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.413+0000 7f5354974640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f53380a2df0 0x7f53380a52b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.413+0000 7f5354974640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535010e4d0 0x7f535010cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.413+0000 7f5354974640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5350101360 0x7f53501a4e90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.413+0000 7f5354974640 1 --2- 172.21.15.62:0/2822544653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53501009b0 0x7f53501a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.413+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 >> 172.21.15.62:0/2822544653 conn(0x7f53500fc7d0 msgr2=0x7f53500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:21.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.415+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 shutdown_connections 2024-09-12T00:07:21.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:21.416+0000 7f5354974640 1 -- 172.21.15.62:0/2822544653 wait complete. 2024-09-12T00:07:21.715 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:21 smithi062 ceph-mon[30958]: pgmap v476: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 771 op/s 2024-09-12T00:07:21.873 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:21 smithi120 ceph-mon[35182]: pgmap v476: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 771 op/s 2024-09-12T00:07:21.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:21 smithi083 ceph-mon[34805]: pgmap v476: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 771 op/s 2024-09-12T00:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 880.0 s, 1601.2 tps, lat 61.825 ms stddev 97.656 2024-09-12T00:07:22.663 INFO:tasks.cephfs.filesystem:scrub status for tag:d948402a-766e-410a-aa32-a3db77be0207 - {'path': '/', 'tag': 'd948402a-766e-410a-aa32-a3db77be0207', 'options': 'recursive,force'} 2024-09-12T00:07:23.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:23 smithi120 ceph-mon[35182]: pgmap v477: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 873 op/s 2024-09-12T00:07:23.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:23 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:07:23.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:23 smithi083 ceph-mon[34805]: pgmap v477: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 873 op/s 2024-09-12T00:07:23.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:23 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:07:23.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:23 smithi062 ceph-mon[30958]: pgmap v477: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 873 op/s 2024-09-12T00:07:23.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:23 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:07:25.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:25 smithi120 ceph-mon[35182]: pgmap v478: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2024-09-12T00:07:25.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:25 smithi083 ceph-mon[34805]: pgmap v478: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2024-09-12T00:07:25.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:25 smithi062 ceph-mon[30958]: pgmap v478: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2024-09-12T00:07:26.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:26 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:07:26.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:26 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:07:26.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:26 smithi120 ceph-mon[35182]: scrub summary: idle 2024-09-12T00:07:26.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:26 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:07:26.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:26 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:07:26.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:26 smithi083 ceph-mon[34805]: scrub summary: idle 2024-09-12T00:07:26.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:26 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:07:26.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:26 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:07:26.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:26 smithi062 ceph-mon[30958]: scrub summary: idle 2024-09-12T00:07:27.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 885.0 s, 1628.4 tps, lat 61.707 ms stddev 83.761 2024-09-12T00:07:27.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:27 smithi120 ceph-mon[35182]: pgmap v479: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2024-09-12T00:07:27.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:27 smithi083 ceph-mon[34805]: pgmap v479: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2024-09-12T00:07:27.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:27 smithi062 ceph-mon[30958]: pgmap v479: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2024-09-12T00:07:29.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:29 smithi120 ceph-mon[35182]: pgmap v480: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 23 MiB/s wr, 1.91k op/s 2024-09-12T00:07:29.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:29 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:07:29.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:29 smithi083 ceph-mon[34805]: pgmap v480: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 23 MiB/s wr, 1.91k op/s 2024-09-12T00:07:29.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:29 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:07:29.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:29 smithi062 ceph-mon[30958]: pgmap v480: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 23 MiB/s wr, 1.91k op/s 2024-09-12T00:07:29.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:29 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:07:30.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:30 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:07:30.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:30 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:07:30.871 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:30 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:07:30.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:30 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:07:30.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:30 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:07:30.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:30 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:07:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:30 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:07:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:30 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:07:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:30 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:07:31.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:31 smithi120 ceph-mon[35182]: pgmap v481: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 22 MiB/s wr, 1.94k op/s 2024-09-12T00:07:31.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:31 smithi083 ceph-mon[34805]: pgmap v481: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 22 MiB/s wr, 1.94k op/s 2024-09-12T00:07:31.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:31 smithi062 ceph-mon[30958]: pgmap v481: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 22 MiB/s wr, 1.94k op/s 2024-09-12T00:07:32.450 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 890.0 s, 218.0 tps, lat 46.088 ms stddev 65.526 2024-09-12T00:07:33.767 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:33 smithi062 ceph-mon[30958]: pgmap v482: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-09-12T00:07:33.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:33 smithi120 ceph-mon[35182]: pgmap v482: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-09-12T00:07:33.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:33 smithi083 ceph-mon[34805]: pgmap v482: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-09-12T00:07:35.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:35 smithi120 ceph-mon[35182]: pgmap v483: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 21 MiB/s wr, 1.83k op/s 2024-09-12T00:07:35.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:35 smithi083 ceph-mon[34805]: pgmap v483: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 21 MiB/s wr, 1.83k op/s 2024-09-12T00:07:35.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:35 smithi062 ceph-mon[30958]: pgmap v483: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 21 MiB/s wr, 1.83k op/s 2024-09-12T00:07:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 895.0 s, 314.7 tps, lat 605.942 ms stddev 2197.217 2024-09-12T00:07:37.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:37 smithi120 ceph-mon[35182]: pgmap v484: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.1 MiB/s wr, 834 op/s 2024-09-12T00:07:37.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:37 smithi083 ceph-mon[34805]: pgmap v484: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.1 MiB/s wr, 834 op/s 2024-09-12T00:07:37.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:37 smithi062 ceph-mon[30958]: pgmap v484: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.1 MiB/s wr, 834 op/s 2024-09-12T00:07:39.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:39 smithi120 ceph-mon[35182]: pgmap v485: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 9.4 MiB/s wr, 854 op/s 2024-09-12T00:07:39.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:39 smithi083 ceph-mon[34805]: pgmap v485: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 9.4 MiB/s wr, 854 op/s 2024-09-12T00:07:39.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:39 smithi062 ceph-mon[30958]: pgmap v485: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 9.4 MiB/s wr, 854 op/s 2024-09-12T00:07:41.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:41 smithi120 ceph-mon[35182]: pgmap v486: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.8 MiB/s wr, 503 op/s 2024-09-12T00:07:41.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:41 smithi083 ceph-mon[34805]: pgmap v486: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.8 MiB/s wr, 503 op/s 2024-09-12T00:07:41.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:41 smithi062 ceph-mon[30958]: pgmap v486: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.8 MiB/s wr, 503 op/s 2024-09-12T00:07:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:progress: 900.0 s, 1971.2 tps, lat 50.404 ms stddev 57.320 2024-09-12T00:07:42.569 INFO:teuthology.orchestra.run.smithi062.stdout:transaction type: 2024-09-12T00:07:42.569 INFO:teuthology.orchestra.run.smithi062.stdout:scaling factor: 32 2024-09-12T00:07:42.570 INFO:teuthology.orchestra.run.smithi062.stdout:query mode: simple 2024-09-12T00:07:42.570 INFO:teuthology.orchestra.run.smithi062.stdout:number of clients: 100 2024-09-12T00:07:42.570 INFO:teuthology.orchestra.run.smithi062.stdout:number of threads: 4 2024-09-12T00:07:42.570 INFO:teuthology.orchestra.run.smithi062.stdout:duration: 900 s 2024-09-12T00:07:42.570 INFO:teuthology.orchestra.run.smithi062.stdout:number of transactions actually processed: 1550500 2024-09-12T00:07:42.570 INFO:teuthology.orchestra.run.smithi062.stdout:latency average = 58.040 ms 2024-09-12T00:07:42.570 INFO:teuthology.orchestra.run.smithi062.stdout:latency stddev = 654.619 ms 2024-09-12T00:07:42.570 INFO:teuthology.orchestra.run.smithi062.stdout:tps = 1722.539817 (including connections establishing) 2024-09-12T00:07:42.571 INFO:teuthology.orchestra.run.smithi062.stdout:tps = 1722.545205 (excluding connections establishing) 2024-09-12T00:07:42.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-12T00:07:43.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:43 smithi120 ceph-mon[35182]: pgmap v487: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 2.2 MiB/s wr, 177 op/s 2024-09-12T00:07:43.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:43 smithi083 ceph-mon[34805]: pgmap v487: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 2.2 MiB/s wr, 177 op/s 2024-09-12T00:07:43.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:43 smithi062 ceph-mon[30958]: pgmap v487: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 2.2 MiB/s wr, 177 op/s 2024-09-12T00:07:45.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:45 smithi120 ceph-mon[35182]: pgmap v488: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.0 MiB/s wr, 358 op/s 2024-09-12T00:07:45.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:45 smithi083 ceph-mon[34805]: pgmap v488: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.0 MiB/s wr, 358 op/s 2024-09-12T00:07:45.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:45 smithi062 ceph-mon[30958]: pgmap v488: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.0 MiB/s wr, 358 op/s 2024-09-12T00:07:47.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:47 smithi120 ceph-mon[35182]: pgmap v489: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.6 MiB/s wr, 325 op/s 2024-09-12T00:07:47.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:47 smithi083 ceph-mon[34805]: pgmap v489: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.6 MiB/s wr, 325 op/s 2024-09-12T00:07:47.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:47 smithi062 ceph-mon[30958]: pgmap v489: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.6 MiB/s wr, 325 op/s 2024-09-12T00:07:49.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:49 smithi120 ceph-mon[35182]: pgmap v490: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 9.0 MiB/s wr, 750 op/s 2024-09-12T00:07:49.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:49 smithi083 ceph-mon[34805]: pgmap v490: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 9.0 MiB/s wr, 750 op/s 2024-09-12T00:07:49.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:49 smithi062 ceph-mon[30958]: pgmap v490: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 9.0 MiB/s wr, 750 op/s 2024-09-12T00:07:49.976 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-12T00:07:50.044 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/: 2024-09-12T00:07:50.044 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.044 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 . 2024-09-12T00:07:50.044 INFO:teuthology.orchestra.run.smithi062.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 .. 2024-09-12T00:07:50.044 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 data 2024-09-12T00:07:50.044 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.044 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data: 2024-09-12T00:07:50.044 INFO:teuthology.orchestra.run.smithi062.stdout:total 36 2024-09-12T00:07:50.044 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 . 2024-09-12T00:07:50.045 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 .. 2024-09-12T00:07:50.045 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 11 23:52 base 2024-09-12T00:07:50.045 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 12 00:00 current_logfiles 2024-09-12T00:07:50.045 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 11 23:52 global 2024-09-12T00:07:50.045 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 00:00 log 2024-09-12T00:07:50.045 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_commit_ts 2024-09-12T00:07:50.045 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_dynshmem 2024-09-12T00:07:50.045 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 11 23:52 pg_hba.conf 2024-09-12T00:07:50.045 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 11 23:52 pg_ident.conf 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 00:07 pg_logical 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 pg_multixact 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_notify 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_replslot 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_serial 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_snapshots 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 00:07 pg_stat 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 00:07 pg_stat_tmp 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 00:07 pg_subtrans 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_tblspc 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pg_twophase 2024-09-12T00:07:50.046 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 PG_VERSION 2024-09-12T00:07:50.047 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 12 00:07 pg_wal 2024-09-12T00:07:50.047 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 00:01 pg_xact 2024-09-12T00:07:50.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 11 23:52 postgresql.auto.conf 2024-09-12T00:07:50.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 11 23:52 postgresql.conf 2024-09-12T00:07:50.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 11 23:52 postmaster.opts 2024-09-12T00:07:50.047 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.047 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/base: 2024-09-12T00:07:50.047 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.047 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 11 23:52 . 2024-09-12T00:07:50.047 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 11 23:52 1 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 11 23:52 13493 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 11 23:57 13494 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 pgsql_tmp 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:total 7777 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 11 23:52 . 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 11 23:52 .. 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 112 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 113 2024-09-12T00:07:50.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 1247 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1247_fsm 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1247_vm 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 11 23:52 1249 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1249_fsm 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1249_vm 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 11 23:52 1255 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1255_fsm 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1255_vm 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 11 23:52 1259 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1259_fsm 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1259_vm 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 11 23:52 13344 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13344_fsm 2024-09-12T00:07:50.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13344_vm 2024-09-12T00:07:50.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13346 2024-09-12T00:07:50.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13348 2024-09-12T00:07:50.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349 2024-09-12T00:07:50.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13349_fsm 2024-09-12T00:07:50.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349_vm 2024-09-12T00:07:50.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13351 2024-09-12T00:07:50.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13353 2024-09-12T00:07:50.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354 2024-09-12T00:07:50.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13354_fsm 2024-09-12T00:07:50.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354_vm 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13356 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13358 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13359_fsm 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359_vm 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13361 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13363 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1417 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1418 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 174 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 175 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2187 2024-09-12T00:07:50.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2224 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2328 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2336 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2337 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2579 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2600 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2600_fsm 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2600_vm 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2601_fsm 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601_vm 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2602 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2602_fsm 2024-09-12T00:07:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2602_vm 2024-09-12T00:07:50.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2603 2024-09-12T00:07:50.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2603_fsm 2024-09-12T00:07:50.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2603_vm 2024-09-12T00:07:50.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2604 2024-09-12T00:07:50.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2605 2024-09-12T00:07:50.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2605_fsm 2024-09-12T00:07:50.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2605_vm 2024-09-12T00:07:50.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606 2024-09-12T00:07:50.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2606_fsm 2024-09-12T00:07:50.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606_vm 2024-09-12T00:07:50.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2607 2024-09-12T00:07:50.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2607_fsm 2024-09-12T00:07:50.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2607_vm 2024-09-12T00:07:50.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 11 23:52 2608 2024-09-12T00:07:50.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2608_fsm 2024-09-12T00:07:50.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2608_vm 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 11 23:52 2609 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2609_fsm 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2609_vm 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2610 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2610_fsm 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2610_vm 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2611 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2612_fsm 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612_vm 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2613 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615 2024-09-12T00:07:50.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2615_fsm 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615_vm 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2616 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2616_fsm 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2616_vm 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 11 23:52 2617 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2617_fsm 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2617_vm 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 11 23:52 2618 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2618_fsm 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2618_vm 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 11 23:52 2619 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2619_fsm 2024-09-12T00:07:50.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2619_vm 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2620 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2650 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2651 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2652 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2653 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2654 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2655 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2656 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2657 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 11 23:52 2658 2024-09-12T00:07:50.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 2659 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2660 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2661 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2662 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2663 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2664 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2665 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2666 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2667 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2668 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2669 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2670 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 11 23:52 2673 2024-09-12T00:07:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 11 23:52 2674 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 11 23:52 2675 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2678 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2679 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2680 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2681 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2682 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2683 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2684 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2685 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2686 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2687 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2688 2024-09-12T00:07:50.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2689 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 11 23:52 2690 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 11 23:52 2691 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2692 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2693 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2696 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2699 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2701 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2702 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2703 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2704 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2753 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2753_fsm 2024-09-12T00:07:50.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2753_vm 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2754 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2755 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2756 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2757 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2830 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2831 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2832 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2833 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2834 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2835 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2836 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2837 2024-09-12T00:07:50.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 11 23:52 2838 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2838_fsm 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2838_vm 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2839 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840_fsm 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2840_vm 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2841 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2995 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2996 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3079_fsm 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079_vm 2024-09-12T00:07:50.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3080 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3081 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3085 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3118 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3119 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 11 23:52 3164 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3256 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3257 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3258 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3350 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3351 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3379 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3380 2024-09-12T00:07:50.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3381 2024-09-12T00:07:50.064 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3394 2024-09-12T00:07:50.064 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3394_fsm 2024-09-12T00:07:50.064 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3394_vm 2024-09-12T00:07:50.064 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3395 2024-09-12T00:07:50.064 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3429 2024-09-12T00:07:50.064 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3430 2024-09-12T00:07:50.064 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3431 2024-09-12T00:07:50.064 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3433 2024-09-12T00:07:50.065 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3439 2024-09-12T00:07:50.065 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3440 2024-09-12T00:07:50.065 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3455 2024-09-12T00:07:50.065 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 11 23:52 3456 2024-09-12T00:07:50.065 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3456_fsm 2024-09-12T00:07:50.065 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3456_vm 2024-09-12T00:07:50.065 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3466 2024-09-12T00:07:50.065 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3467 2024-09-12T00:07:50.065 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3468 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3501 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3502 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3503 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3534 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3541_fsm 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541_vm 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3542 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3574 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3575 2024-09-12T00:07:50.066 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3576 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3596 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3597 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3598 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3599 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3600_fsm 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600_vm 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3601_fsm 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601_vm 2024-09-12T00:07:50.067 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602 2024-09-12T00:07:50.068 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3602_fsm 2024-09-12T00:07:50.068 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602_vm 2024-09-12T00:07:50.068 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603 2024-09-12T00:07:50.068 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603_fsm 2024-09-12T00:07:50.068 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3603_vm 2024-09-12T00:07:50.068 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3604 2024-09-12T00:07:50.068 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3605 2024-09-12T00:07:50.068 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3606 2024-09-12T00:07:50.068 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3607 2024-09-12T00:07:50.068 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3608 2024-09-12T00:07:50.069 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 3609 2024-09-12T00:07:50.069 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3712 2024-09-12T00:07:50.069 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764 2024-09-12T00:07:50.069 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3764_fsm 2024-09-12T00:07:50.069 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764_vm 2024-09-12T00:07:50.069 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3766 2024-09-12T00:07:50.070 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3767 2024-09-12T00:07:50.070 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3997 2024-09-12T00:07:50.070 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4143 2024-09-12T00:07:50.070 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4144 2024-09-12T00:07:50.070 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4145 2024-09-12T00:07:50.070 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4146 2024-09-12T00:07:50.071 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4147 2024-09-12T00:07:50.071 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4148 2024-09-12T00:07:50.071 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4149 2024-09-12T00:07:50.071 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4150 2024-09-12T00:07:50.071 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4151 2024-09-12T00:07:50.071 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4152 2024-09-12T00:07:50.072 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4153 2024-09-12T00:07:50.072 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4154 2024-09-12T00:07:50.072 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4155 2024-09-12T00:07:50.072 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4156 2024-09-12T00:07:50.072 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4157 2024-09-12T00:07:50.073 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4158 2024-09-12T00:07:50.073 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4159 2024-09-12T00:07:50.073 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4160 2024-09-12T00:07:50.073 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4161 2024-09-12T00:07:50.073 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4162 2024-09-12T00:07:50.074 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4163 2024-09-12T00:07:50.075 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4164 2024-09-12T00:07:50.075 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4165 2024-09-12T00:07:50.075 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4166 2024-09-12T00:07:50.075 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4167 2024-09-12T00:07:50.075 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4168 2024-09-12T00:07:50.075 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4169 2024-09-12T00:07:50.075 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4170 2024-09-12T00:07:50.075 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4171 2024-09-12T00:07:50.075 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4172 2024-09-12T00:07:50.076 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4173 2024-09-12T00:07:50.076 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4174 2024-09-12T00:07:50.076 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 5002 2024-09-12T00:07:50.076 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 548 2024-09-12T00:07:50.076 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 549 2024-09-12T00:07:50.076 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6102 2024-09-12T00:07:50.076 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6104 2024-09-12T00:07:50.076 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6106 2024-09-12T00:07:50.076 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6110 2024-09-12T00:07:50.077 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6111 2024-09-12T00:07:50.077 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6112 2024-09-12T00:07:50.077 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6113 2024-09-12T00:07:50.077 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6117 2024-09-12T00:07:50.077 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 826 2024-09-12T00:07:50.077 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 827 2024-09-12T00:07:50.077 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 828 2024-09-12T00:07:50.077 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 11 23:52 pg_filenode.map 2024-09-12T00:07:50.077 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 PG_VERSION 2024-09-12T00:07:50.077 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:total 7777 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 11 23:52 . 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 11 23:52 .. 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 112 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 113 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 1247 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1247_fsm 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1247_vm 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 11 23:52 1249 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1249_fsm 2024-09-12T00:07:50.078 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1249_vm 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 11 23:52 1255 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1255_fsm 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1255_vm 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 11 23:52 1259 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1259_fsm 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1259_vm 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 11 23:52 13344 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13344_fsm 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13344_vm 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13346 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13348 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349 2024-09-12T00:07:50.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13349_fsm 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349_vm 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13351 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13353 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13354_fsm 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354_vm 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13356 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13358 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13359_fsm 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359_vm 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13361 2024-09-12T00:07:50.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13363 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1417 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1418 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 174 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 175 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2187 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2224 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2328 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2336 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2337 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2579 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2600 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2600_fsm 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2600_vm 2024-09-12T00:07:50.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2601_fsm 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601_vm 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2602 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2602_fsm 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2602_vm 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2603 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2603_fsm 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2603_vm 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2604 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2605 2024-09-12T00:07:50.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2605_fsm 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2605_vm 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2606_fsm 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606_vm 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2607 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2607_fsm 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2607_vm 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 11 23:52 2608 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2608_fsm 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2608_vm 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 11 23:52 2609 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2609_fsm 2024-09-12T00:07:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2609_vm 2024-09-12T00:07:50.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2610 2024-09-12T00:07:50.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2610_fsm 2024-09-12T00:07:50.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2610_vm 2024-09-12T00:07:50.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2611 2024-09-12T00:07:50.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612 2024-09-12T00:07:50.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2612_fsm 2024-09-12T00:07:50.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612_vm 2024-09-12T00:07:50.085 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2613 2024-09-12T00:07:50.085 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615 2024-09-12T00:07:50.085 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2615_fsm 2024-09-12T00:07:50.085 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615_vm 2024-09-12T00:07:50.085 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2616 2024-09-12T00:07:50.085 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2616_fsm 2024-09-12T00:07:50.085 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2616_vm 2024-09-12T00:07:50.085 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 11 23:52 2617 2024-09-12T00:07:50.085 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2617_fsm 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2617_vm 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 11 23:52 2618 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2618_fsm 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2618_vm 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 11 23:52 2619 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2619_fsm 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2619_vm 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2620 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2650 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2651 2024-09-12T00:07:50.086 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2652 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2653 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2654 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2655 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2656 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2657 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 11 23:52 2658 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 2659 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2660 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2661 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2662 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2663 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2664 2024-09-12T00:07:50.087 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2665 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2666 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2667 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2668 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2669 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2670 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 11 23:52 2673 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 11 23:52 2674 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 11 23:52 2675 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2678 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2679 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2680 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2681 2024-09-12T00:07:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2682 2024-09-12T00:07:50.089 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2683 2024-09-12T00:07:50.089 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2684 2024-09-12T00:07:50.089 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2685 2024-09-12T00:07:50.089 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2686 2024-09-12T00:07:50.089 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2687 2024-09-12T00:07:50.089 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2688 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2689 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 11 23:52 2690 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 11 23:52 2691 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2692 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2693 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2696 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2699 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2701 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2702 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2703 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2704 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2753 2024-09-12T00:07:50.090 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2753_fsm 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2753_vm 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2754 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2755 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2756 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2757 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2830 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2831 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2832 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2833 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2834 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2835 2024-09-12T00:07:50.091 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2836 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2837 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 11 23:52 2838 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2838_fsm 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2838_vm 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2839 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840_fsm 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2840_vm 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2841 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2995 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2996 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079 2024-09-12T00:07:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3079_fsm 2024-09-12T00:07:50.093 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079_vm 2024-09-12T00:07:50.093 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3080 2024-09-12T00:07:50.093 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3081 2024-09-12T00:07:50.093 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3085 2024-09-12T00:07:50.093 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3118 2024-09-12T00:07:50.093 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3119 2024-09-12T00:07:50.093 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 11 23:52 3164 2024-09-12T00:07:50.093 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3256 2024-09-12T00:07:50.096 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3257 2024-09-12T00:07:50.097 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3258 2024-09-12T00:07:50.097 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3350 2024-09-12T00:07:50.097 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3351 2024-09-12T00:07:50.097 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3379 2024-09-12T00:07:50.097 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3380 2024-09-12T00:07:50.097 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3381 2024-09-12T00:07:50.097 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3394 2024-09-12T00:07:50.097 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3394_fsm 2024-09-12T00:07:50.097 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3394_vm 2024-09-12T00:07:50.097 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3395 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3429 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3430 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3431 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3433 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3439 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3440 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3455 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 11 23:52 3456 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3456_fsm 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3456_vm 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3466 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3467 2024-09-12T00:07:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3468 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3501 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3502 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3503 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3534 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3541_fsm 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541_vm 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3542 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3574 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3575 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3576 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3596 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3597 2024-09-12T00:07:50.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3598 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3599 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3600_fsm 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600_vm 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3601_fsm 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601_vm 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3602_fsm 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602_vm 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603_fsm 2024-09-12T00:07:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3603_vm 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3604 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3605 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3606 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3607 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3608 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 3609 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3712 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3764_fsm 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764_vm 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3766 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3767 2024-09-12T00:07:50.101 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3997 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4143 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4144 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4145 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4146 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4147 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4148 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4149 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4150 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4151 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4152 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4153 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4154 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4155 2024-09-12T00:07:50.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4156 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4157 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4158 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4159 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4160 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4161 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4162 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4163 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4164 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4165 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4166 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4167 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4168 2024-09-12T00:07:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4169 2024-09-12T00:07:50.104 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4170 2024-09-12T00:07:50.104 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4171 2024-09-12T00:07:50.104 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4172 2024-09-12T00:07:50.105 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4173 2024-09-12T00:07:50.105 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4174 2024-09-12T00:07:50.105 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 5002 2024-09-12T00:07:50.105 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 548 2024-09-12T00:07:50.105 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 549 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6102 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6104 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6106 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6110 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6111 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6112 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6113 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6117 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 826 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 827 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 828 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 11 23:52 pg_filenode.map 2024-09-12T00:07:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 PG_VERSION 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:total 596110 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 11 23:57 . 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 11 23:52 .. 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 112 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 113 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 1247 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1247_fsm 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1247_vm 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 11 23:52 1249 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1249_fsm 2024-09-12T00:07:50.107 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1249_vm 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 11 23:52 1255 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1255_fsm 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1255_vm 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 12 00:06 1259 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1259_fsm 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1259_vm 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 11 23:52 13344 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13344_fsm 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13344_vm 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13346 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13348 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349 2024-09-12T00:07:50.108 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13349_fsm 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13349_vm 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13351 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13353 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13354_fsm 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13354_vm 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13356 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13358 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 13359_fsm 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13359_vm 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 13361 2024-09-12T00:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 13363 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1417 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 1418 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 443146240 Sep 12 00:07 16396 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 12 00:07 16396_fsm 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:56 16396_vm 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1081344 Sep 12 00:07 16397 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 00:07 16397_fsm 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 00:07 16397_vm 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1703936 Sep 12 00:07 16399 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 12 00:06 16399_fsm 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 00:07 16399_vm 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 12 00:07 16400 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 12 00:07 16402 2024-09-12T00:07:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 12 00:07 16404 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 84099072 Sep 12 00:07 16406 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 12 00:07 16406_fsm 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 00:03 16406_vm 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 174 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 175 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2187 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2224 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2328 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2336 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2337 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2579 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2600 2024-09-12T00:07:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2600_fsm 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2600_vm 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2601_fsm 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2601_vm 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2602 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2602_fsm 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2602_vm 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2603 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2603_fsm 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2603_vm 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2604 2024-09-12T00:07:50.112 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2605 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2605_fsm 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2605_vm 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2606_fsm 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2606_vm 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2607 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2607_fsm 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2607_vm 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 11 23:52 2608 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2608_fsm 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2608_vm 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 11 23:52 2609 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2609_fsm 2024-09-12T00:07:50.113 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2609_vm 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2610 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2610_fsm 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2610_vm 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2611 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2612_fsm 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2612_vm 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2613 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2615_fsm 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2615_vm 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2616 2024-09-12T00:07:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2616_fsm 2024-09-12T00:07:50.115 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2616_vm 2024-09-12T00:07:50.115 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 11 23:52 2617 2024-09-12T00:07:50.115 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2617_fsm 2024-09-12T00:07:50.115 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2617_vm 2024-09-12T00:07:50.115 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 11 23:52 2618 2024-09-12T00:07:50.115 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2618_fsm 2024-09-12T00:07:50.115 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2618_vm 2024-09-12T00:07:50.115 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 12 00:06 2619 2024-09-12T00:07:50.115 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:54 2619_fsm 2024-09-12T00:07:50.115 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2619_vm 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2620 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2650 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2651 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2652 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2653 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2654 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2655 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2656 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2657 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 11 23:52 2658 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 11 23:52 2659 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2660 2024-09-12T00:07:50.116 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2661 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2662 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2663 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2664 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2665 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2666 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2667 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2668 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2669 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2670 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 11 23:52 2673 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 11 23:52 2674 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 11 23:52 2675 2024-09-12T00:07:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2678 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2679 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2680 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2681 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2682 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2683 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2684 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2685 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2686 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2687 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2688 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 11 23:52 2689 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 11 23:52 2690 2024-09-12T00:07:50.118 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 11 23:52 2691 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2692 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2693 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:55 2696 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2699 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2701 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2702 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2703 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 2704 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2753 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2753_fsm 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2753_vm 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2754 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2755 2024-09-12T00:07:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2756 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 2757 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2830 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2831 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2832 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2833 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2834 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2835 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2836 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2837 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 11 23:52 2838 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2838_fsm 2024-09-12T00:07:50.120 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2838_vm 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2839 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2840_fsm 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2840_vm 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2841 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2995 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2996 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3079_fsm 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3079_vm 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3080 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3081 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3085 2024-09-12T00:07:50.121 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3118 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3119 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 11 23:52 3164 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3256 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3257 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3258 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3350 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3351 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3379 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3380 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3381 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3394 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3394_fsm 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3394_vm 2024-09-12T00:07:50.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3395 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3429 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3430 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3431 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3433 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3439 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3440 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 11 23:52 3455 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 11 23:52 3456 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3456_fsm 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3456_vm 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3466 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3467 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3468 2024-09-12T00:07:50.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3501 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3502 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3503 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3534 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3541_fsm 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3541_vm 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3542 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3574 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3575 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3576 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3596 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3597 2024-09-12T00:07:50.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3598 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3599 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3600_fsm 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3600_vm 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3601_fsm 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3601_vm 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3602_fsm 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3602_vm 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3603_fsm 2024-09-12T00:07:50.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3603_vm 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3604 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3605 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3606 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3607 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3608 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 11 23:52 3609 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3712 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 3764_fsm 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3764_vm 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3766 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 3767 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3997 2024-09-12T00:07:50.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4143 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4144 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4145 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4146 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4147 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4148 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4149 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4150 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4151 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4152 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4153 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4154 2024-09-12T00:07:50.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4155 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4156 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4157 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4158 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4159 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4160 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4161 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4162 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4163 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4164 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4165 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4166 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4167 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4168 2024-09-12T00:07:50.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4169 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4170 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4171 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4172 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4173 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4174 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 5002 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 548 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 549 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6102 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6104 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6106 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6110 2024-09-12T00:07:50.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6111 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6112 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6113 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6117 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 826 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 827 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 828 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 11 23:52 pg_filenode.map 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 11 23:52 pg_internal.init 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 11 23:52 PG_VERSION 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.130 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 11 23:52 .. 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/global: 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:total 560 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 11 23:52 . 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1213 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1213_fsm 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1213_vm 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1214 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1214_fsm 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1214_vm 2024-09-12T00:07:50.131 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 1232 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 1233 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1260 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1260_fsm 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1260_vm 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1261 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1261_fsm 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1261_vm 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1262 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 1262_fsm 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 1262_vm 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2396 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 11 23:52 2396_fsm 2024-09-12T00:07:50.132 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2396_vm 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2397 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2671 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2672 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2676 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2677 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2694 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2695 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2697 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 11 23:52 2698 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2846 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2847 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2964 2024-09-12T00:07:50.133 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2965 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 2966 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 2967 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 3592 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 3593 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4060 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4061 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4175 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4176 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4177 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4178 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4181 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4182 2024-09-12T00:07:50.134 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4183 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4184 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 4185 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 4186 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6000 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6001 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6002 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 6100 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6114 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 6115 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 12 00:07 pg_control 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 11 23:52 pg_filenode.map 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 11 23:52 pg_internal.init 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.135 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/log: 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:total 2 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 00:00 . 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 507 Sep 12 00:07 postgresql-Thu.log 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 847 Sep 11 23:56 postgresql-Wed.log 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-12T00:07:50.136 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:total 1 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 00:07 . 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 mappings 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 12 00:07 replorigin_checkpoint 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 snapshots 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-12T00:07:50.137 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.138 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.138 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 00:07 .. 2024-09-12T00:07:50.138 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.138 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-12T00:07:50.138 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.138 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.138 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 00:07 .. 2024-09-12T00:07:50.138 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.138 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-12T00:07:50.138 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 . 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 members 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 offsets 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout:total 8 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 . 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 .. 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:52 0000 2024-09-12T00:07:50.139 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout:total 8 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 11 23:52 . 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 11 23:52 .. 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 11 23:57 0000 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.140 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.142 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-12T00:07:50.142 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.142 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.142 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.142 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.142 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-12T00:07:50.142 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.143 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.143 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.143 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.143 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-12T00:07:50.143 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.143 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout:total 15 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 12 00:07 . 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 12 00:07 db_0.stat 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 12 00:07 db_13494.stat 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 12 00:07 global.stat 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-12T00:07:50.144 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 12 00:07 . 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:total 176 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 12 00:07 . 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 12 00:07 0017 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.145 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:total 1048576 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 12 00:07 . 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:07 000000010000000000000093 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 11 23:57 000000010000000000000094 2024-09-12T00:07:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 000000010000000000000095 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 000000010000000000000096 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 000000010000000000000097 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:03 000000010000000000000098 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:05 000000010000000000000099 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:04 00000001000000000000009A 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:03 00000001000000000000009B 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 00000001000000000000009C 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 00000001000000000000009D 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 00000001000000000000009E 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:04 00000001000000000000009F 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:02 0000000100000000000000A0 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000A1 2024-09-12T00:07:50.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000A2 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:03 0000000100000000000000A3 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000A4 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000A5 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000A6 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000A7 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000A8 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:03 0000000100000000000000A9 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:03 0000000100000000000000AA 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000AB 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:02 0000000100000000000000AC 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000AD 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000AE 2024-09-12T00:07:50.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:02 0000000100000000000000AF 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000B0 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:02 0000000100000000000000B1 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000B2 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000B3 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:02 0000000100000000000000B4 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000B5 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:05 0000000100000000000000B6 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:03 0000000100000000000000B7 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000B8 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:03 0000000100000000000000B9 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:01 0000000100000000000000BA 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:07 0000000100000000000000BB 2024-09-12T00:07:50.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000BC 2024-09-12T00:07:50.150 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:07 0000000100000000000000BD 2024-09-12T00:07:50.150 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000BE 2024-09-12T00:07:50.150 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000BF 2024-09-12T00:07:50.150 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000C0 2024-09-12T00:07:50.150 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:07 0000000100000000000000C1 2024-09-12T00:07:50.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000C2 2024-09-12T00:07:50.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:07 0000000100000000000000C3 2024-09-12T00:07:50.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000C4 2024-09-12T00:07:50.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000C5 2024-09-12T00:07:50.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:07 0000000100000000000000C6 2024-09-12T00:07:50.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000C7 2024-09-12T00:07:50.152 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000C8 2024-09-12T00:07:50.152 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:07 0000000100000000000000C9 2024-09-12T00:07:50.152 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000CA 2024-09-12T00:07:50.152 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000CB 2024-09-12T00:07:50.152 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000CC 2024-09-12T00:07:50.152 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000CD 2024-09-12T00:07:50.152 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000CE 2024-09-12T00:07:50.153 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:07 0000000100000000000000CF 2024-09-12T00:07:50.153 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:06 0000000100000000000000D0 2024-09-12T00:07:50.153 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:07 0000000100000000000000D1 2024-09-12T00:07:50.153 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 12 00:07 0000000100000000000000D2 2024-09-12T00:07:50.153 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 archive_status 2024-09-12T00:07:50.153 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.153 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-12T00:07:50.153 INFO:teuthology.orchestra.run.smithi062.stdout:total 0 2024-09-12T00:07:50.153 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 11 23:52 . 2024-09-12T00:07:50.153 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 12 00:07 .. 2024-09-12T00:07:50.154 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:50.154 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-12T00:07:50.154 INFO:teuthology.orchestra.run.smithi062.stdout:total 384 2024-09-12T00:07:50.154 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 12 00:01 . 2024-09-12T00:07:50.154 INFO:teuthology.orchestra.run.smithi062.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 12 00:07 .. 2024-09-12T00:07:50.154 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 12 00:01 0000 2024-09-12T00:07:50.154 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 12 00:07 0001 2024-09-12T00:07:50.155 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-12T00:07:50.175 INFO:tasks.quiescer:joining Quiescers 2024-09-12T00:07:50.175 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-12T00:07:50.176 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:07:50.433 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:51.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.363+0000 7f17464c0640 1 -- 172.21.15.62:0/3023524578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740102570 msgr2=0x7f17401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:51.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.363+0000 7f17464c0640 1 --2- 172.21.15.62:0/3023524578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740102570 0x7f17401029b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f1734009900 tx=0x7f173402a480 comp rx=0 tx=0).stop 2024-09-12T00:07:51.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.363+0000 7f17464c0640 1 -- 172.21.15.62:0/3023524578 shutdown_connections 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.363+0000 7f17464c0640 1 --2- 172.21.15.62:0/3023524578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1740102ef0 0x7f174010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.363+0000 7f17464c0640 1 --2- 172.21.15.62:0/3023524578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740102570 0x7f17401029b0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.363+0000 7f17464c0640 1 --2- 172.21.15.62:0/3023524578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1740100e20 0x7f1740101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.363+0000 7f17464c0640 1 -- 172.21.15.62:0/3023524578 >> 172.21.15.62:0/3023524578 conn(0x7f17400fc7d0 msgr2=0x7f17400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.364+0000 7f17464c0640 1 -- 172.21.15.62:0/3023524578 shutdown_connections 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.364+0000 7f17464c0640 1 -- 172.21.15.62:0/3023524578 wait complete. 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.364+0000 7f17464c0640 1 Processor -- start 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.364+0000 7f17464c0640 1 -- start start 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f17464c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100e20 0x7f17401984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f17464c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1740102570 0x7f1740198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f17464c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1740102ef0 0x7f174019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f17464c0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1740110f80 con 0x7f1740100e20 2024-09-12T00:07:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f17464c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1740110e00 con 0x7f1740102ef0 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f17464c0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1740111100 con 0x7f1740102570 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100e20 0x7f17401984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100e20 0x7f17401984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60872/0 (socket says 172.21.15.62:60872) 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173ffff640 1 -- 172.21.15.62:0/1792304283 learned_addr learned my addr 172.21.15.62:0/1792304283 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173f7fe640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1740102570 0x7f1740198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f1744a5d640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1740102ef0 0x7f174019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173ffff640 1 -- 172.21.15.62:0/1792304283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1740102570 msgr2=0x7f1740198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173ffff640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1740102570 0x7f1740198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173ffff640 1 -- 172.21.15.62:0/1792304283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1740102ef0 msgr2=0x7f174019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173ffff640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1740102ef0 0x7f174019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173ffff640 1 -- 172.21.15.62:0/1792304283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174019d4a0 con 0x7f1740100e20 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f1744a5d640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1740102ef0 0x7f174019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:07:51.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173f7fe640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1740102570 0x7f1740198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.365+0000 7f173ffff640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100e20 0x7f17401984f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f173000e9f0 tx=0x7f173000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.366+0000 7f173d7fa640 1 -- 172.21.15.62:0/1792304283 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173000cd80 con 0x7f1740100e20 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.366+0000 7f173d7fa640 1 -- 172.21.15.62:0/1792304283 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1730010910 con 0x7f1740100e20 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.366+0000 7f17464c0640 1 -- 172.21.15.62:0/1792304283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174019d790 con 0x7f1740100e20 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.366+0000 7f173d7fa640 1 -- 172.21.15.62:0/1792304283 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17300188f0 con 0x7f1740100e20 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.366+0000 7f17464c0640 1 -- 172.21.15.62:0/1792304283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17401a5070 con 0x7f1740100e20 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.367+0000 7f17464c0640 1 -- 172.21.15.62:0/1792304283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1740101200 con 0x7f1740100e20 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.369+0000 7f173d7fa640 1 -- 172.21.15.62:0/1792304283 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1730010ac0 con 0x7f1740100e20 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.369+0000 7f173d7fa640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f17280a2cf0 0x7f17280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.369+0000 7f173f7fe640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f17280a2cf0 0x7f17280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.369+0000 7f173d7fa640 1 -- 172.21.15.62:0/1792304283 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f17300cfd40 con 0x7f1740100e20 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.370+0000 7f173f7fe640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f17280a2cf0 0x7f17280a51b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1734000c00 tx=0x7f1734004100 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.373+0000 7f173d7fa640 1 -- 172.21.15.62:0/1792304283 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f173009c960 con 0x7f1740100e20 2024-09-12T00:07:51.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.588+0000 7f17464c0640 1 -- 172.21.15.62:0/1792304283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17401a5270 con 0x7f1740100e20 2024-09-12T00:07:51.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.589+0000 7f173d7fa640 1 -- 172.21.15.62:0/1792304283 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11912 (secure 0 0 0) 0x7f173009c260 con 0x7f1740100e20 2024-09-12T00:07:51.589 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:51.589 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":40,"btime":"2024-09-12T00:07:22:469216+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:07:22.092246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:07:51.590 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 40 2024-09-12T00:07:51.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 -- 172.21.15.62:0/1792304283 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f17280a2cf0 msgr2=0x7f17280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:51.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f17280a2cf0 0x7f17280a51b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1734000c00 tx=0x7f1734004100 comp rx=0 tx=0).stop 2024-09-12T00:07:51.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 -- 172.21.15.62:0/1792304283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100e20 msgr2=0x7f17401984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:51.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100e20 0x7f17401984f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f173000e9f0 tx=0x7f173000eeb0 comp rx=0 tx=0).stop 2024-09-12T00:07:51.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 -- 172.21.15.62:0/1792304283 shutdown_connections 2024-09-12T00:07:51.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f17280a2cf0 0x7f17280a51b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:51.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1740102ef0 0x7f174019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:51.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1740102570 0x7f1740198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:51.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 --2- 172.21.15.62:0/1792304283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100e20 0x7f17401984f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:51.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 -- 172.21.15.62:0/1792304283 >> 172.21.15.62:0/1792304283 conn(0x7f17400fc7d0 msgr2=0x7f1740069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:51.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 -- 172.21.15.62:0/1792304283 shutdown_connections 2024-09-12T00:07:51.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:51.591+0000 7f17464c0640 1 -- 172.21.15.62:0/1792304283 wait complete. 2024-09-12T00:07:51.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:51 smithi062 ceph-mon[30958]: pgmap v491: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-12T00:07:51.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:51 smithi120 ceph-mon[35182]: pgmap v491: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-12T00:07:51.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:51 smithi083 ceph-mon[34805]: pgmap v491: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-12T00:07:52.384 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --cancel --all' 2024-09-12T00:07:52.385 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --cancel --all 2024-09-12T00:07:52.649 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:52.664 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:07:52.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:52 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1792304283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:07:52.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:52 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1792304283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:07:52.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:52 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1792304283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:07:53.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:53 smithi062 ceph-mon[30958]: pgmap v492: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2024-09-12T00:07:53.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:53 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:07:53.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 -- 172.21.15.62:0/3268261887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c1082a0 msgr2=0x7fbd0c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:53.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 --2- 172.21.15.62:0/3268261887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c1082a0 0x7fbd0c10ec40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009cd0 tx=0x7fbd0402a8e0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 -- 172.21.15.62:0/3268261887 shutdown_connections 2024-09-12T00:07:53.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 --2- 172.21.15.62:0/3268261887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c1082a0 0x7fbd0c10ec40 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 --2- 172.21.15.62:0/3268261887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd0c1036e0 0x7fbd0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 --2- 172.21.15.62:0/3268261887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c102d30 0x7fbd0c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 -- 172.21.15.62:0/3268261887 >> 172.21.15.62:0/3268261887 conn(0x7fbd0c0fc7d0 msgr2=0x7fbd0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 -- 172.21.15.62:0/3268261887 shutdown_connections 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 -- 172.21.15.62:0/3268261887 wait complete. 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 Processor -- start 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 -- start start 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c102d30 0x7fbd0c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.827+0000 7fbd12da9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd0c1036e0 0x7fbd0c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd12da9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c1082a0 0x7fbd0c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd12da9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1120a0 con 0x7fbd0c102d30 2024-09-12T00:07:53.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd12da9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c111f20 con 0x7fbd0c1082a0 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd12da9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c112220 con 0x7fbd0c1036e0 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd10b45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c102d30 0x7fbd0c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd10b45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c102d30 0x7fbd0c198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60890/0 (socket says 172.21.15.62:60890) 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd10b45640 1 -- 172.21.15.62:0/1359725732 learned_addr learned my addr 172.21.15.62:0/1359725732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd0bfff640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd0c1036e0 0x7fbd0c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd11346640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c1082a0 0x7fbd0c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd10b45640 1 -- 172.21.15.62:0/1359725732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd0c1036e0 msgr2=0x7fbd0c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd10b45640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd0c1036e0 0x7fbd0c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd10b45640 1 -- 172.21.15.62:0/1359725732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c1082a0 msgr2=0x7fbd0c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd10b45640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c1082a0 0x7fbd0c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd10b45640 1 -- 172.21.15.62:0/1359725732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c19d4c0 con 0x7fbd0c102d30 2024-09-12T00:07:53.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd0bfff640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd0c1036e0 0x7fbd0c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd10b45640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c102d30 0x7fbd0c198510 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000d950 tx=0x7fbd0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd09ffb640 1 -- 172.21.15.62:0/1359725732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00017030 con 0x7fbd0c102d30 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd12da9640 1 -- 172.21.15.62:0/1359725732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0c19d7b0 con 0x7fbd0c102d30 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd09ffb640 1 -- 172.21.15.62:0/1359725732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd00004020 con 0x7fbd0c102d30 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.828+0000 7fbd09ffb640 1 -- 172.21.15.62:0/1359725732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd000047a0 con 0x7fbd0c102d30 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.829+0000 7fbd12da9640 1 -- 172.21.15.62:0/1359725732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0c1a5090 con 0x7fbd0c102d30 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.831+0000 7fbd09ffb640 1 -- 172.21.15.62:0/1359725732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbd000041d0 con 0x7fbd0c102d30 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.831+0000 7fbd15182740 1 Processor -- start 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.831+0000 7fbd09ffb640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbcfc0a2cf0 0x7fbcfc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.831+0000 7fbd15182740 1 -- start start 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.831+0000 7fbd11346640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbcfc0a2cf0 0x7fbcfc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.831+0000 7fbd15182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb108a00 0x559edb108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.831+0000 7fbd09ffb640 1 -- 172.21.15.62:0/1359725732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fbd000ceef0 con 0x7fbd0c102d30 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.831+0000 7fbd15182740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb1136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.831+0000 7fbd15182740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb113be0 0x559edb115fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.831+0000 7fbd15182740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559edb06d200 con 0x559edb109320 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd15182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559edb013dc0 con 0x559edb108a00 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd15182740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559edb0691c0 con 0x559edb113be0 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd0bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb1136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd0bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb1136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60894/0 (socket says 172.21.15.62:60894) 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd0bfff640 1 -- 172.21.15.62:0/3967480352 learned_addr learned my addr 172.21.15.62:0/3967480352 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd10b45640 1 --2- 172.21.15.62:0/3967480352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb108a00 0x559edb108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd0bfff640 1 -- 172.21.15.62:0/3967480352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb113be0 msgr2=0x559edb115fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:53.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd0bfff640 1 --2- 172.21.15.62:0/3967480352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb113be0 0x559edb115fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd0bfff640 1 -- 172.21.15.62:0/3967480352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb108a00 msgr2=0x559edb108de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd0bfff640 1 --2- 172.21.15.62:0/3967480352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb108a00 0x559edb108de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd0bfff640 1 -- 172.21.15.62:0/3967480352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559edadde620 con 0x559edb109320 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd11346640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbcfc0a2cf0 0x7fbcfc0a51b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbd04006010 tx=0x7fbd04009920 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbd0bfff640 1 --2- 172.21.15.62:0/3967480352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb1136a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8008b30 tx=0x7fbcf800e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72af3d4053017407 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbcf6ffd640 1 -- 172.21.15.62:0/3967480352 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8014070 con 0x559edb109320 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbcf6ffd640 1 -- 172.21.15.62:0/3967480352 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcf8005500 con 0x559edb109320 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.832+0000 7fbcf6ffd640 1 -- 172.21.15.62:0/3967480352 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8010630 con 0x559edb109320 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 -- 172.21.15.62:0/3967480352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 msgr2=0x559edb1136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 --2- 172.21.15.62:0/3967480352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb1136a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8008b30 tx=0x7fbcf800e350 comp rx=0 tx=0).stop 2024-09-12T00:07:53.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 -- 172.21.15.62:0/3967480352 shutdown_connections 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 --2- 172.21.15.62:0/3967480352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb113be0 0x559edb115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 --2- 172.21.15.62:0/3967480352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb1136a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 --2- 172.21.15.62:0/3967480352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb108a00 0x559edb108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 -- 172.21.15.62:0/3967480352 >> 172.21.15.62:0/3967480352 conn(0x559edb104f90 msgr2=0x559edb105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 -- 172.21.15.62:0/3967480352 shutdown_connections 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 -- 172.21.15.62:0/3967480352 wait complete. 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 -- start start 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.833+0000 7fbd15182740 1 Processor -- start 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd15182740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb113470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd15182740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb110720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd15182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb113be0 0x559edb110c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd15182740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559edb1167b0 con 0x559edb109320 2024-09-12T00:07:53.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd15182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559edb007c50 con 0x559edb113be0 2024-09-12T00:07:53.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd15182740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559edb116930 con 0x559edb108a00 2024-09-12T00:07:53.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd15182740 10 client.0 _ll_register_callbacks cb 0x559edb019ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:53.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd0bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd0bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60908/0 (socket says 172.21.15.62:60908) 2024-09-12T00:07:53.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd0bfff640 1 -- 172.21.15.62:0/2976813039 learned_addr learned my addr 172.21.15.62:0/2976813039 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:53.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd11346640 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb113be0 0x559edb110c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd10b45640 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb113470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd0bfff640 1 -- 172.21.15.62:0/2976813039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 msgr2=0x559edb113470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:53.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd0bfff640 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb113470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd0bfff640 1 -- 172.21.15.62:0/2976813039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb113be0 msgr2=0x559edb110c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:53.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd0bfff640 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb113be0 0x559edb110c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.834+0000 7fbd0bfff640 1 -- 172.21.15.62:0/2976813039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559edb111440 con 0x559edb109320 2024-09-12T00:07:53.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbd0bfff640 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb110720 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbcf800ef60 tx=0x7fbcf8005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc163f9c410888d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:53.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbcebfff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:07:53.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbcebfff640 1 -- 172.21.15.62:0/2976813039 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8027060 con 0x559edb109320 2024-09-12T00:07:53.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbcebfff640 1 -- 172.21.15.62:0/2976813039 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fbcf8010640 con 0x559edb109320 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbd15182740 1 -- 172.21.15.62:0/2976813039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcf8005500 con 0x559edb109320 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbcebfff640 1 -- 172.21.15.62:0/2976813039 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbcf8014070 con 0x559edb109320 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbd15182740 10 client.18162 fetch_fsmap learned FSMap version 41 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbd15182740 1 -- 172.21.15.62:0/2976813039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559edb122280 con 0x559edb109320 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbcebfff640 1 -- 172.21.15.62:0/2976813039 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcf8005500 con 0x559edb109320 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbcebfff640 1 -- 172.21.15.62:0/2976813039 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7fbcf8018710 con 0x559edb109320 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbd15182740 10 client.18162 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbd15182740 10 client.18162 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbd15182740 20 client.18162 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbd15182740 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559edb123740 0x559edb125b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbd15182740 4 client.18162 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:07:53.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.835+0000 7fbd15182740 1 -- 172.21.15.62:0/2976813039 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559edb07a770 con 0x559edb123740 2024-09-12T00:07:53.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.836+0000 7fbd10b45640 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559edb123740 0x559edb125b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.836+0000 7fbcebfff640 10 client.18162 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:53.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.836+0000 7fbd10b45640 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559edb123740 0x559edb125b30 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965452341a72c4c6 server_cookie=d27cf321c88c163a in_seq=0 out_seq=0 2024-09-12T00:07:53.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbcebfff640 1 -- 172.21.15.62:0/2976813039 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559edb07a770 con 0x559edb123740 2024-09-12T00:07:53.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbcebfff640 10 client.18162 handle_command_reply: tid=0 2024-09-12T00:07:53.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 client.18162 shutdown 2024-09-12T00:07:53.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 -- 172.21.15.62:0/2976813039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 msgr2=0x559edb110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:53.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb110720 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbcf800ef60 tx=0x7fbcf8005fc0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 -- 172.21.15.62:0/2976813039 shutdown_connections 2024-09-12T00:07:53.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559edb123740 0x559edb125b30 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb113be0 0x559edb110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb110720 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbcebfff640 0 client.18162 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:53.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 --2- 172.21.15.62:0/2976813039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb113470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 -- 172.21.15.62:0/2976813039 >> 172.21.15.62:0/2976813039 conn(0x559edb104f90 msgr2=0x559edb1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:53.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 -- 172.21.15.62:0/2976813039 shutdown_connections 2024-09-12T00:07:53.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 1 -- 172.21.15.62:0/2976813039 wait complete. 2024-09-12T00:07:53.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.837+0000 7fbd15182740 20 client.18162 trim_cache size 0 max 16384 2024-09-12T00:07:53.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd15182740 1 Processor -- start 2024-09-12T00:07:53.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd15182740 1 -- start start 2024-09-12T00:07:53.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd15182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb104f90 0x559edb12b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd15182740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb12b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd15182740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb10d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd15182740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b0200 con 0x559edb109320 2024-09-12T00:07:53.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd15182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1ada40 con 0x559edb104f90 2024-09-12T00:07:53.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd15182740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1ab8b0 con 0x559edb108a00 2024-09-12T00:07:53.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd11346640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb10d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd11346640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb10d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60918/0 (socket says 172.21.15.62:60918) 2024-09-12T00:07:53.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.855+0000 7fbd11346640 1 -- 172.21.15.62:0/244579446 learned_addr learned my addr 172.21.15.62:0/244579446 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.856+0000 7fbd0bfff640 1 --2- 172.21.15.62:0/244579446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb12b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.856+0000 7fbd10b45640 1 --2- 172.21.15.62:0/244579446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb104f90 0x559edb12b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.856+0000 7fbd11346640 1 -- 172.21.15.62:0/244579446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 msgr2=0x559edb12b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.856+0000 7fbd11346640 1 --2- 172.21.15.62:0/244579446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb12b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.856+0000 7fbd11346640 1 -- 172.21.15.62:0/244579446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb104f90 msgr2=0x559edb12b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.856+0000 7fbd11346640 1 --2- 172.21.15.62:0/244579446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb104f90 0x559edb12b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.856+0000 7fbd11346640 1 -- 172.21.15.62:0/244579446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559edb104dc0 con 0x559edb109320 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.856+0000 7fbd11346640 1 --2- 172.21.15.62:0/244579446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb10d9a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbd04031800 tx=0x7fbd0403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc845aad58ac60ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.856+0000 7fbcf67fc640 1 -- 172.21.15.62:0/244579446 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0404d070 con 0x559edb109320 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.856+0000 7fbcf67fc640 1 -- 172.21.15.62:0/244579446 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd04036040 con 0x559edb109320 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.857+0000 7fbcf67fc640 1 -- 172.21.15.62:0/244579446 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd040308f0 con 0x559edb109320 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.857+0000 7fbd15182740 1 -- 172.21.15.62:0/244579446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 msgr2=0x559edb10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:53.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.857+0000 7fbd15182740 1 --2- 172.21.15.62:0/244579446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb10d9a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbd04031800 tx=0x7fbd0403e000 comp rx=0 tx=0).stop 2024-09-12T00:07:53.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.857+0000 7fbd15182740 1 -- 172.21.15.62:0/244579446 shutdown_connections 2024-09-12T00:07:53.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.857+0000 7fbd15182740 1 --2- 172.21.15.62:0/244579446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb10d9a0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.857+0000 7fbd15182740 1 --2- 172.21.15.62:0/244579446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb12b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.857+0000 7fbd15182740 1 --2- 172.21.15.62:0/244579446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb104f90 0x559edb12b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.857+0000 7fbd15182740 1 -- 172.21.15.62:0/244579446 >> 172.21.15.62:0/244579446 conn(0x559edb110dc0 msgr2=0x559edb10eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:53.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.857+0000 7fbd15182740 1 -- 172.21.15.62:0/244579446 shutdown_connections 2024-09-12T00:07:53.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.857+0000 7fbd15182740 1 -- 172.21.15.62:0/244579446 wait complete. 2024-09-12T00:07:53.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd15182740 1 -- start start 2024-09-12T00:07:53.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd15182740 1 Processor -- start 2024-09-12T00:07:53.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd15182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb104f90 0x559edb11c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd15182740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb1193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd15182740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb119910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd15182740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1abf00 con 0x559edb109320 2024-09-12T00:07:53.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd15182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1ad370 con 0x559edb104f90 2024-09-12T00:07:53.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd15182740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1ac5d0 con 0x559edb108a00 2024-09-12T00:07:53.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd15182740 10 client.0 _ll_register_callbacks cb 0x7fbcf80056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd11346640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb119910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd11346640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb119910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60926/0 (socket says 172.21.15.62:60926) 2024-09-12T00:07:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.858+0000 7fbd11346640 1 -- 172.21.15.62:0/3405698799 learned_addr learned my addr 172.21.15.62:0/3405698799 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbd0bfff640 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb1193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbd11346640 1 -- 172.21.15.62:0/3405698799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 msgr2=0x559edb1193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbd11346640 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb1193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbd11346640 1 -- 172.21.15.62:0/3405698799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb104f90 msgr2=0x559edb11c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbd11346640 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb104f90 0x559edb11c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbd11346640 1 -- 172.21.15.62:0/3405698799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559edb119ed0 con 0x559edb109320 2024-09-12T00:07:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbd11346640 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb119910 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbd04030b00 tx=0x7fbd04030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf99c40a31c72e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbcebfff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbcebfff640 1 -- 172.21.15.62:0/3405698799 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0405a020 con 0x559edb109320 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbcebfff640 1 -- 172.21.15.62:0/3405698799 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fbd04052460 con 0x559edb109320 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.859+0000 7fbd15182740 1 -- 172.21.15.62:0/3405698799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd04036040 con 0x559edb109320 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbcebfff640 1 -- 172.21.15.62:0/3405698799 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbd0404d070 con 0x559edb109320 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbd15182740 10 client.18168 fetch_fsmap learned FSMap version 41 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbd15182740 1 -- 172.21.15.62:0/3405698799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559edb192280 con 0x559edb109320 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbcebfff640 1 -- 172.21.15.62:0/3405698799 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd04052b40 con 0x559edb109320 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbcebfff640 1 -- 172.21.15.62:0/3405698799 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7fbd04030070 con 0x559edb109320 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbd15182740 10 client.18168 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbd15182740 10 client.18168 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-12T00:07:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbd15182740 20 client.18168 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbd15182740 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559edb113be0 0x559edb1949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbd15182740 4 client.18168 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbd15182740 1 -- 172.21.15.62:0/3405698799 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x7fbd04036040 con 0x559edb113be0 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.860+0000 7fbd10b45640 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559edb113be0 0x559edb1949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbcebfff640 10 client.18168 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd10b45640 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559edb113be0 0x559edb1949f0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfa703ebdd71ac0 server_cookie=2f35114c4d31d956 in_seq=0 out_seq=0 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbcebfff640 1 -- 172.21.15.62:0/3405698799 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x7fbd04036040 con 0x559edb113be0 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbcebfff640 10 client.18168 handle_command_reply: tid=0 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 client.18168 shutdown 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 -- 172.21.15.62:0/3405698799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 msgr2=0x559edb119910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb119910 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbd04030b00 tx=0x7fbd04030b30 comp rx=0 tx=0).stop 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 -- 172.21.15.62:0/3405698799 shutdown_connections 2024-09-12T00:07:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x559edb113be0 0x559edb1949f0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edb109320 0x559edb119910 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559edb108a00 0x559edb1193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 --2- 172.21.15.62:0/3405698799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559edb104f90 0x559edb11c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 -- 172.21.15.62:0/3405698799 >> 172.21.15.62:0/3405698799 conn(0x559edb110dc0 msgr2=0x559edb111f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbcebfff640 0 client.18168 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 -- 172.21.15.62:0/3405698799 shutdown_connections 2024-09-12T00:07:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.861+0000 7fbd15182740 1 -- 172.21.15.62:0/3405698799 wait complete. 2024-09-12T00:07:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd15182740 20 client.18168 trim_cache size 0 max 16384 2024-09-12T00:07:53.869 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:07:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 41, 2024-09-12T00:07:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24488, 2024-09-12T00:07:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 56, 2024-09-12T00:07:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": {} 2024-09-12T00:07:53.869 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:07:53.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 -- 172.21.15.62:0/1359725732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbcfc0a2cf0 msgr2=0x7fbcfc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:53.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbcfc0a2cf0 0x7fbcfc0a51b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbd04006010 tx=0x7fbd04009920 comp rx=0 tx=0).stop 2024-09-12T00:07:53.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 -- 172.21.15.62:0/1359725732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c102d30 msgr2=0x7fbd0c198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:53.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c102d30 0x7fbd0c198510 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000d950 tx=0x7fbd0000de10 comp rx=0 tx=0).stop 2024-09-12T00:07:53.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 -- 172.21.15.62:0/1359725732 shutdown_connections 2024-09-12T00:07:53.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fbcfc0a2cf0 0x7fbcfc0a51b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c1082a0 0x7fbd0c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd0c1036e0 0x7fbd0c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 --2- 172.21.15.62:0/1359725732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c102d30 0x7fbd0c198510 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:53.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 -- 172.21.15.62:0/1359725732 >> 172.21.15.62:0/1359725732 conn(0x7fbd0c0fc7d0 msgr2=0x7fbd0c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:53.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 -- 172.21.15.62:0/1359725732 shutdown_connections 2024-09-12T00:07:53.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:53.862+0000 7fbd12da9640 1 -- 172.21.15.62:0/1359725732 wait complete. 2024-09-12T00:07:53.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:53 smithi120 ceph-mon[35182]: pgmap v492: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2024-09-12T00:07:53.872 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:53 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:07:53.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:53 smithi083 ceph-mon[34805]: pgmap v492: 97 pgs: 97 active+clean; 5.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2024-09-12T00:07:53.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:53 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:07:54.109 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:54.655 INFO:tasks.quiescer:done joining Quiescers 2024-09-12T00:07:54.656 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-12T00:07:54.665 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-12T00:07:54.666 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T00:07:54.666 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-09-12T00:07:54.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-12T00:07:54.727 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-12T00:07:54.837 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T00:07:54.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3190966924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5d81035e0 msgr2=0x7fe5d810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:54.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 --2- 172.21.15.62:0/3190966924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5d81035e0 0x7fe5d810bac0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009900 tx=0x7fe5cc02a480 comp rx=0 tx=0).stop 2024-09-12T00:07:54.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3190966924 shutdown_connections 2024-09-12T00:07:54.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 --2- 172.21.15.62:0/3190966924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5d810c110 0x7fe5d810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:54.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 --2- 172.21.15.62:0/3190966924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5d81035e0 0x7fe5d810bac0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:54.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 --2- 172.21.15.62:0/3190966924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d8102cc0 0x7fe5d81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:54.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3190966924 >> 172.21.15.62:0/3190966924 conn(0x7fe5d80fc6b0 msgr2=0x7fe5d80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:54.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3190966924 shutdown_connections 2024-09-12T00:07:54.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3190966924 wait complete. 2024-09-12T00:07:54.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 Processor -- start 2024-09-12T00:07:54.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.946+0000 7fe5de8c9640 1 -- start start 2024-09-12T00:07:54.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5de8c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5d8102cc0 0x7fe5d81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:54.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5de8c9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5d81035e0 0x7fe5d8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:54.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5de8c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d810c110 0x7fe5d819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:54.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5de8c9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d8111ef0 con 0x7fe5d81035e0 2024-09-12T00:07:54.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5de8c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d8111d70 con 0x7fe5d8102cc0 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5de8c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d8112070 con 0x7fe5d810c110 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5d77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5d81035e0 0x7fe5d8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5d77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5d81035e0 0x7fe5d8198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60940/0 (socket says 172.21.15.62:60940) 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5d77fe640 1 -- 172.21.15.62:0/3506902981 learned_addr learned my addr 172.21.15.62:0/3506902981 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5d7fff640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5d8102cc0 0x7fe5d81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5d77fe640 1 -- 172.21.15.62:0/3506902981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d810c110 msgr2=0x7fe5d819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5dce66640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d810c110 0x7fe5d819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5d77fe640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d810c110 0x7fe5d819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5d77fe640 1 -- 172.21.15.62:0/3506902981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5d8102cc0 msgr2=0x7fe5d81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5d77fe640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5d8102cc0 0x7fe5d81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5d77fe640 1 -- 172.21.15.62:0/3506902981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d819d3b0 con 0x7fe5d81035e0 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.947+0000 7fe5d77fe640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5d81035e0 0x7fe5d8198900 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc006010 tx=0x7fe5cc031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.948+0000 7fe5d57fa640 1 -- 172.21.15.62:0/3506902981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc03f030 con 0x7fe5d81035e0 2024-09-12T00:07:54.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.948+0000 7fe5d57fa640 1 -- 172.21.15.62:0/3506902981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5cc0043d0 con 0x7fe5d81035e0 2024-09-12T00:07:54.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.948+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3506902981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d819d640 con 0x7fe5d81035e0 2024-09-12T00:07:54.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.948+0000 7fe5d57fa640 1 -- 172.21.15.62:0/3506902981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc035070 con 0x7fe5d81035e0 2024-09-12T00:07:54.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.948+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3506902981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d81a4f30 con 0x7fe5d81035e0 2024-09-12T00:07:54.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.949+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3506902981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d806abe0 con 0x7fe5d81035e0 2024-09-12T00:07:54.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.950+0000 7fe5d57fa640 1 -- 172.21.15.62:0/3506902981 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe5cc042030 con 0x7fe5d81035e0 2024-09-12T00:07:54.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.950+0000 7fe5d57fa640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5c00a2da0 0x7fe5c00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:54.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.951+0000 7fe5d7fff640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5c00a2da0 0x7fe5c00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:54.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.951+0000 7fe5d57fa640 1 -- 172.21.15.62:0/3506902981 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fe5cc0ef760 con 0x7fe5d81035e0 2024-09-12T00:07:54.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.951+0000 7fe5d7fff640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5c00a2da0 0x7fe5c00a5260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009bd0 tx=0x7fe5c8005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:54.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:54.955+0000 7fe5d57fa640 1 -- 172.21.15.62:0/3506902981 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5cc0bc440 con 0x7fe5d81035e0 2024-09-12T00:07:55.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.157+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3506902981 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fe5d8062630 con 0x7fe5c00a2da0 2024-09-12T00:07:55.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.164+0000 7fe5d57fa640 1 -- 172.21.15.62:0/3506902981 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fe5d8062630 con 0x7fe5c00a2da0 2024-09-12T00:07:55.164 INFO:teuthology.orchestra.run.smithi062.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T23:52:14", "first": "2024-09-11T23:53:00", "last": "2024-09-12T00:07:00", "last_pruned": "2024-09-12T00:07:00", "created_count": 15, "pruned_count": 8, "active": true} 2024-09-12T00:07:55.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3506902981 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5c00a2da0 msgr2=0x7fe5c00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5c00a2da0 0x7fe5c00a5260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009bd0 tx=0x7fe5c8005f90 comp rx=0 tx=0).stop 2024-09-12T00:07:55.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3506902981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5d81035e0 msgr2=0x7fe5d8198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5d81035e0 0x7fe5d8198900 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc006010 tx=0x7fe5cc031b20 comp rx=0 tx=0).stop 2024-09-12T00:07:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3506902981 shutdown_connections 2024-09-12T00:07:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe5c00a2da0 0x7fe5c00a5260 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d810c110 0x7fe5d819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5d81035e0 0x7fe5d8198900 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 --2- 172.21.15.62:0/3506902981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5d8102cc0 0x7fe5d81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3506902981 >> 172.21.15.62:0/3506902981 conn(0x7fe5d80fc6b0 msgr2=0x7fe5d80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.166+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3506902981 shutdown_connections 2024-09-12T00:07:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.167+0000 7fe5de8c9640 1 -- 172.21.15.62:0/3506902981 wait complete. 2024-09-12T00:07:55.177 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T00:07:55.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f43453f6640 1 -- 172.21.15.62:0/1314816519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43401011a0 msgr2=0x7f434010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f43453f6640 1 --2- 172.21.15.62:0/1314816519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43401011a0 0x7f434010def0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4334009900 tx=0x7f433402a3f0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f43453f6640 1 -- 172.21.15.62:0/1314816519 shutdown_connections 2024-09-12T00:07:55.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f43453f6640 1 --2- 172.21.15.62:0/1314816519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434010e430 0x7f4340110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f43453f6640 1 --2- 172.21.15.62:0/1314816519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43401011a0 0x7f434010def0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f43453f6640 1 --2- 172.21.15.62:0/1314816519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4340100880 0x7f4340100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f43453f6640 1 -- 172.21.15.62:0/1314816519 >> 172.21.15.62:0/1314816519 conn(0x7f43400fc6b0 msgr2=0x7f43400fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:55.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f43453f6640 1 -- 172.21.15.62:0/1314816519 shutdown_connections 2024-09-12T00:07:55.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f43453f6640 1 -- 172.21.15.62:0/1314816519 wait complete. 2024-09-12T00:07:55.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f43453f6640 1 Processor -- start 2024-09-12T00:07:55.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f43453f6640 1 -- start start 2024-09-12T00:07:55.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f43453f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4340100880 0x7f434019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f43453f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43401011a0 0x7f434019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f43453f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434010e430 0x7f434019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f43453f6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4340114140 con 0x7f434010e430 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f43453f6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4340113fc0 con 0x7f43401011a0 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f43453f6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43401142c0 con 0x7f4340100880 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4340100880 0x7f434019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4340100880 0x7f434019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:48778/0 (socket says 172.21.15.62:48778) 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434010e430 0x7f434019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433effd640 1 -- 172.21.15.62:0/1430007489 learned_addr learned my addr 172.21.15.62:0/1430007489 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433e7fc640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43401011a0 0x7f434019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433effd640 1 -- 172.21.15.62:0/1430007489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43401011a0 msgr2=0x7f434019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433effd640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43401011a0 0x7f434019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433effd640 1 -- 172.21.15.62:0/1430007489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434010e430 msgr2=0x7f434019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433effd640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434010e430 0x7f434019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433effd640 1 -- 172.21.15.62:0/1430007489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434019f5e0 con 0x7f4340100880 2024-09-12T00:07:55.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433f7fe640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434010e430 0x7f434019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:55.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f433effd640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4340100880 0x7f434019a5f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9f0 tx=0x7f433000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:55.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f4327fff640 1 -- 172.21.15.62:0/1430007489 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433000cd80 con 0x7f4340100880 2024-09-12T00:07:55.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f4327fff640 1 -- 172.21.15.62:0/1430007489 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4330010910 con 0x7f4340100880 2024-09-12T00:07:55.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f4327fff640 1 -- 172.21.15.62:0/1430007489 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330018990 con 0x7f4340100880 2024-09-12T00:07:55.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f43453f6640 1 -- 172.21.15.62:0/1430007489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434019f8d0 con 0x7f4340100880 2024-09-12T00:07:55.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f43453f6640 1 -- 172.21.15.62:0/1430007489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43401a7160 con 0x7f4340100880 2024-09-12T00:07:55.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f43453f6640 1 -- 172.21.15.62:0/1430007489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4340100c60 con 0x7f4340100880 2024-09-12T00:07:55.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f4327fff640 1 -- 172.21.15.62:0/1430007489 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4330010ac0 con 0x7f4340100880 2024-09-12T00:07:55.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f4327fff640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43280a2da0 0x7f43280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f4327fff640 1 -- 172.21.15.62:0/1430007489 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f43300cf850 con 0x7f4340100880 2024-09-12T00:07:55.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f433e7fc640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43280a2da0 0x7f43280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.295+0000 7f433e7fc640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43280a2da0 0x7f43280a5260 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4334000c00 tx=0x7f4334005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:55.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.299+0000 7f4327fff640 1 -- 172.21.15.62:0/1430007489 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f433009c410 con 0x7f4340100880 2024-09-12T00:07:55.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.500+0000 7f43453f6640 1 -- 172.21.15.62:0/1430007489 --> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f434010f7e0 con 0x7f43280a2da0 2024-09-12T00:07:55.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.508+0000 7f4327fff640 1 -- 172.21.15.62:0/1430007489 <== mgr.15141 v2:172.21.15.62:6800/3798569441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f434010f7e0 con 0x7f43280a2da0 2024-09-12T00:07:55.508 INFO:teuthology.orchestra.run.smithi062.stdout:/ 1m 6m3h 2024-09-12T00:07:55.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 -- 172.21.15.62:0/1430007489 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43280a2da0 msgr2=0x7f43280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43280a2da0 0x7f43280a5260 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4334000c00 tx=0x7f4334005a20 comp rx=0 tx=0).stop 2024-09-12T00:07:55.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 -- 172.21.15.62:0/1430007489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4340100880 msgr2=0x7f434019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4340100880 0x7f434019a5f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9f0 tx=0x7f433000eeb0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 -- 172.21.15.62:0/1430007489 shutdown_connections 2024-09-12T00:07:55.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f43280a2da0 0x7f43280a5260 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434010e430 0x7f434019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43401011a0 0x7f434019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 --2- 172.21.15.62:0/1430007489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4340100880 0x7f434019a5f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 -- 172.21.15.62:0/1430007489 >> 172.21.15.62:0/1430007489 conn(0x7f43400fc6b0 msgr2=0x7f43400feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:55.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 -- 172.21.15.62:0/1430007489 shutdown_connections 2024-09-12T00:07:55.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.510+0000 7f43453f6640 1 -- 172.21.15.62:0/1430007489 wait complete. 2024-09-12T00:07:55.521 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-12T00:07:55.531 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-12T00:07:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.283+0000 7f290e601640 1 -- 172.21.15.62:0/1926138208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2908102ef0 msgr2=0x7f290810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.283+0000 7f290e601640 1 --2- 172.21.15.62:0/1926138208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2908102ef0 0x7f290810da40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2900009ca0 tx=0x7f290002a950 comp rx=0 tx=0).stop 2024-09-12T00:07:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.283+0000 7f290e601640 1 -- 172.21.15.62:0/1926138208 shutdown_connections 2024-09-12T00:07:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.283+0000 7f290e601640 1 --2- 172.21.15.62:0/1926138208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2908102ef0 0x7f290810da40 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.283+0000 7f290e601640 1 --2- 172.21.15.62:0/1926138208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2908102570 0x7f29081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.283+0000 7f290e601640 1 --2- 172.21.15.62:0/1926138208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2908100e20 0x7f2908101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.283+0000 7f290e601640 1 -- 172.21.15.62:0/1926138208 >> 172.21.15.62:0/1926138208 conn(0x7f29080fc7d0 msgr2=0x7f29080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.283+0000 7f290e601640 1 -- 172.21.15.62:0/1926138208 shutdown_connections 2024-09-12T00:07:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.283+0000 7f290e601640 1 -- 172.21.15.62:0/1926138208 wait complete. 2024-09-12T00:07:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f290e601640 1 Processor -- start 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f290e601640 1 -- start start 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f290e601640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2908100e20 0x7f2908198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f290e601640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2908102570 0x7f2908198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f290e601640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2908102ef0 0x7f290819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f290e601640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2908110f80 con 0x7f2908100e20 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f290e601640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2908110e00 con 0x7f2908102ef0 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f290e601640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2908111100 con 0x7f2908102570 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f29077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2908102570 0x7f2908198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f29077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2908102570 0x7f2908198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:48750/0 (socket says 172.21.15.62:48750) 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f29077fe640 1 -- 172.21.15.62:0/3681585835 learned_addr learned my addr 172.21.15.62:0/3681585835 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f290cb9e640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2908102ef0 0x7f290819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.284+0000 7f2907fff640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2908100e20 0x7f2908198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f2907fff640 1 -- 172.21.15.62:0/3681585835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2908102570 msgr2=0x7f2908198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f2907fff640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2908102570 0x7f2908198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f2907fff640 1 -- 172.21.15.62:0/3681585835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2908102ef0 msgr2=0x7f290819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f2907fff640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2908102ef0 0x7f290819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f2907fff640 1 -- 172.21.15.62:0/3681585835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290819d400 con 0x7f2908100e20 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f290cb9e640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2908102ef0 0x7f290819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f2907fff640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2908100e20 0x7f2908198450 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9f0 tx=0x7f28f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f29057fa640 1 -- 172.21.15.62:0/3681585835 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f800ccc0 con 0x7f2908100e20 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f29057fa640 1 -- 172.21.15.62:0/3681585835 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f8010910 con 0x7f2908100e20 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f29057fa640 1 -- 172.21.15.62:0/3681585835 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8018960 con 0x7f2908100e20 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f290e601640 1 -- 172.21.15.62:0/3681585835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290819d6f0 con 0x7f2908100e20 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.285+0000 7f290e601640 1 -- 172.21.15.62:0/3681585835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29081a4fd0 con 0x7f2908100e20 2024-09-12T00:07:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f29109da740 1 Processor -- start 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f29109da740 1 -- start start 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f29109da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 0x557c59705720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f29109da740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59714550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f29109da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59714a90 0x557c59716e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f29109da740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557c5966b2e0 con 0x557c59705c60 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f29109da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557c59612ce0 con 0x557c59705340 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f29109da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557c596672a0 con 0x557c59714a90 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f29057fa640 1 -- 172.21.15.62:0/3681585835 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f28f8010ac0 con 0x7f2908100e20 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f2907fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59714550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f290cb9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 0x557c59705720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f290cb9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 0x557c59705720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.62:42592/0 (socket says 172.21.15.62:42592) 2024-09-12T00:07:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f2907fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59714550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60978/0 (socket says 172.21.15.62:60978) 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.288+0000 7f290cb9e640 1 -- 172.21.15.62:0/2221951141 learned_addr learned my addr 172.21.15.62:0/2221951141 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f29077fe640 1 --2- 172.21.15.62:0/2221951141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59714a90 0x557c59716e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f290cb9e640 1 -- 172.21.15.62:0/2221951141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59714a90 msgr2=0x557c59716e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f290cb9e640 1 --2- 172.21.15.62:0/2221951141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59714a90 0x557c59716e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f290cb9e640 1 -- 172.21.15.62:0/2221951141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 msgr2=0x557c59714550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f290cb9e640 1 --2- 172.21.15.62:0/2221951141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59714550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f290cb9e640 1 -- 172.21.15.62:0/2221951141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c595f7440 con 0x557c59705340 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f290cb9e640 1 --2- 172.21.15.62:0/2221951141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 0x557c59705720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2900009ca0 tx=0x7f2900005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aff23acb501de2fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.289+0000 7f29057fa640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f28f00a2cf0 0x7f28f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f28ee7fc640 1 -- 172.21.15.62:0/2221951141 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900048070 con 0x557c59705340 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f29057fa640 1 -- 172.21.15.62:0/3681585835 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f28f80cf7c0 con 0x7f2908100e20 2024-09-12T00:07:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f28ee7fc640 1 -- 172.21.15.62:0/2221951141 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2900003970 con 0x557c59705340 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f2907fff640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f28f00a2cf0 0x7f28f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.290+0000 7f28ee7fc640 1 -- 172.21.15.62:0/2221951141 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900034040 con 0x557c59705340 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 -- 172.21.15.62:0/2221951141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 msgr2=0x557c59705720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 --2- 172.21.15.62:0/2221951141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 0x557c59705720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2900009ca0 tx=0x7f2900005ef0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f2907fff640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f28f00a2cf0 0x7f28f00a51b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28f80ccfc0 tx=0x7f28f80d5550 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 -- 172.21.15.62:0/2221951141 shutdown_connections 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 --2- 172.21.15.62:0/2221951141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59714a90 0x557c59716e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 --2- 172.21.15.62:0/2221951141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59714550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 --2- 172.21.15.62:0/2221951141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 0x557c59705720 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 -- 172.21.15.62:0/2221951141 >> 172.21.15.62:0/2221951141 conn(0x557c59701a30 msgr2=0x557c59701e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 -- 172.21.15.62:0/2221951141 shutdown_connections 2024-09-12T00:07:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 -- 172.21.15.62:0/2221951141 wait complete. 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 -- start start 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.291+0000 7f29109da740 1 Processor -- start 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f29109da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705340 0x557c597144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f29109da740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59711750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f29109da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59714a90 0x557c59711d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f29109da740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557c59717700 con 0x557c59705c60 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f29109da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557c59626050 con 0x557c59714a90 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f29109da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557c59717880 con 0x557c59705340 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f29109da740 10 client.0 _ll_register_callbacks cb 0x557c59666160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f29077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59711750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f29077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59711750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60998/0 (socket says 172.21.15.62:60998) 2024-09-12T00:07:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f29077fe640 1 -- 172.21.15.62:0/2761054219 learned_addr learned my addr 172.21.15.62:0/2761054219 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f290cb9e640 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705340 0x557c597144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.292+0000 7f290cb9e640 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59714a90 0x557c59711d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f29077fe640 1 -- 172.21.15.62:0/2761054219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705340 msgr2=0x557c597144a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f29077fe640 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705340 0x557c597144a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f29077fe640 1 -- 172.21.15.62:0/2761054219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59714a90 msgr2=0x557c59711d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f29077fe640 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59714a90 0x557c59711d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f29077fe640 1 -- 172.21.15.62:0/2761054219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c593dc3c0 con 0x557c59705c60 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f29077fe640 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59711750 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00ba90 tx=0x7f28fc00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441135bd7b7acdb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f28e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f28e77fe640 1 -- 172.21.15.62:0/2761054219 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc011440 con 0x557c59705c60 2024-09-12T00:07:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f29109da740 1 -- 172.21.15.62:0/2761054219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2900003970 con 0x557c59705c60 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f28e77fe640 1 -- 172.21.15.62:0/2761054219 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f28fc0115e0 con 0x557c59705c60 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f28e77fe640 1 -- 172.21.15.62:0/2761054219 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f28fc0117e0 con 0x557c59705c60 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f29109da740 10 client.18192 fetch_fsmap learned FSMap version 41 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.293+0000 7f29109da740 1 -- 172.21.15.62:0/2761054219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557c59723280 con 0x557c59705c60 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f28e77fe640 1 -- 172.21.15.62:0/2761054219 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2900003970 con 0x557c59705c60 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f28e77fe640 1 -- 172.21.15.62:0/2761054219 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f28fc01bc00 con 0x557c59705c60 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f29109da740 10 client.18192 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f29109da740 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f29109da740 20 client.18192 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f29109da740 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557c59724790 0x557c59726bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f29109da740 4 client.18192 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f29109da740 1 -- 172.21.15.62:0/2761054219 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c59678850 con 0x557c59724790 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f290cb9e640 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557c59724790 0x557c59726bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.294+0000 7f28e77fe640 10 client.18192 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.295+0000 7f290cb9e640 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557c59724790 0x557c59726bb0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b30c8ab52ca219e server_cookie=3dadc74841367706 in_seq=0 out_seq=0 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.295+0000 7f28e77fe640 1 -- 172.21.15.62:0/2761054219 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c59678850 con 0x557c59724790 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.295+0000 7f28e77fe640 10 client.18192 handle_command_reply: tid=0 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.295+0000 7f29109da740 1 client.18192 shutdown 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 1 -- 172.21.15.62:0/2761054219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 msgr2=0x557c59711750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59711750 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00ba90 tx=0x7f28fc00bf50 comp rx=0 tx=0).stop 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 1 -- 172.21.15.62:0/2761054219 shutdown_connections 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557c59724790 0x557c59726bb0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59714a90 0x557c59711d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c59711750 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 1 --2- 172.21.15.62:0/2761054219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705340 0x557c597144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 1 -- 172.21.15.62:0/2761054219 >> 172.21.15.62:0/2761054219 conn(0x557c59701a30 msgr2=0x557c59707790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:55.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f28e77fe640 0 client.18192 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 1 -- 172.21.15.62:0/2761054219 shutdown_connections 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 1 -- 172.21.15.62:0/2761054219 wait complete. 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.296+0000 7f29109da740 20 client.18192 trim_cache size 0 max 16384 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.313+0000 7f29109da740 1 Processor -- start 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.313+0000 7f29109da740 1 -- start start 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.313+0000 7f29109da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705340 0x557c597209a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.313+0000 7f29109da740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c5972c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29109da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59714a90 0x557c5972c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29109da740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f29081ade70 con 0x557c59705c60 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29109da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29081ab6b0 con 0x557c59714a90 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29109da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29081a9520 con 0x557c59705340 2024-09-12T00:07:55.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c5972c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c5972c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32778/0 (socket says 172.21.15.62:32778) 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29077fe640 1 -- 172.21.15.62:0/979711203 learned_addr learned my addr 172.21.15.62:0/979711203 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f290cb9e640 1 --2- 172.21.15.62:0/979711203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705340 0x557c597209a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f290cb9e640 1 --2- 172.21.15.62:0/979711203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59714a90 0x557c5972c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29077fe640 1 -- 172.21.15.62:0/979711203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705340 msgr2=0x557c597209a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29077fe640 1 --2- 172.21.15.62:0/979711203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705340 0x557c597209a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29077fe640 1 -- 172.21.15.62:0/979711203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59714a90 msgr2=0x557c5972c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29077fe640 1 --2- 172.21.15.62:0/979711203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59714a90 0x557c5972c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29077fe640 1 -- 172.21.15.62:0/979711203 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5970fca0 con 0x557c59705c60 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f29077fe640 1 --2- 172.21.15.62:0/979711203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c5972c460 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00ba90 tx=0x7f28fc004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89b797d2c1fc574 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f28edffb640 1 -- 172.21.15.62:0/979711203 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc032070 con 0x557c59705c60 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f28edffb640 1 -- 172.21.15.62:0/979711203 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28fc029040 con 0x557c59705c60 2024-09-12T00:07:55.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.314+0000 7f28edffb640 1 -- 172.21.15.62:0/979711203 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc028070 con 0x557c59705c60 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.315+0000 7f29109da740 1 -- 172.21.15.62:0/979711203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 msgr2=0x557c5972c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.315+0000 7f29109da740 1 --2- 172.21.15.62:0/979711203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c5972c460 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00ba90 tx=0x7f28fc004590 comp rx=0 tx=0).stop 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.315+0000 7f29109da740 1 -- 172.21.15.62:0/979711203 shutdown_connections 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.315+0000 7f29109da740 1 --2- 172.21.15.62:0/979711203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59714a90 0x557c5972c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.315+0000 7f29109da740 1 --2- 172.21.15.62:0/979711203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59705c60 0x557c5972c460 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.315+0000 7f29109da740 1 --2- 172.21.15.62:0/979711203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705340 0x557c597209a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.315+0000 7f29109da740 1 -- 172.21.15.62:0/979711203 >> 172.21.15.62:0/979711203 conn(0x557c59701a30 msgr2=0x557c59710840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.315+0000 7f29109da740 1 -- 172.21.15.62:0/979711203 shutdown_connections 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.315+0000 7f29109da740 1 -- 172.21.15.62:0/979711203 wait complete. 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.315+0000 7f29109da740 1 -- start start 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f29109da740 1 Processor -- start 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f29109da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 0x557c59787720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f29109da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705c60 0x557c59787c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f29109da740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59714a90 0x557c59718190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f29109da740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f29081a9b70 con 0x557c59714a90 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f29109da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29081aafe0 con 0x557c59705340 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f29109da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29081aa240 con 0x557c59705c60 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f29109da740 10 client.0 _ll_register_callbacks cb 0x7f28fc0041f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f290cb9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59714a90 0x557c59718190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f290cb9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59714a90 0x557c59718190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32788/0 (socket says 172.21.15.62:32788) 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f290cb9e640 1 -- 172.21.15.62:0/1432944274 learned_addr learned my addr 172.21.15.62:0/1432944274 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f290cb9e640 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 0x557c59787720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f29077fe640 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705c60 0x557c59787c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.316+0000 7f290cb9e640 1 -- 172.21.15.62:0/1432944274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705c60 msgr2=0x557c59787c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f290cb9e640 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705c60 0x557c59787c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f290cb9e640 1 -- 172.21.15.62:0/1432944274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 msgr2=0x557c59787720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f290cb9e640 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 0x557c59787720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f290cb9e640 1 -- 172.21.15.62:0/1432944274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c59718880 con 0x557c59714a90 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f29077fe640 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705c60 0x557c59787c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f290cb9e640 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59714a90 0x557c59718190 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2900046040 tx=0x7f28fc0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6858887b7770597a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f28e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f28e77fe640 1 -- 172.21.15.62:0/1432944274 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900048040 con 0x557c59714a90 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f28e77fe640 1 -- 172.21.15.62:0/1432944274 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f2900040040 con 0x557c59714a90 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f29109da740 1 -- 172.21.15.62:0/1432944274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28fc029040 con 0x557c59714a90 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f28e77fe640 1 -- 172.21.15.62:0/1432944274 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2900003dd0 con 0x557c59714a90 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f29109da740 10 client.18198 fetch_fsmap learned FSMap version 41 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f29109da740 1 -- 172.21.15.62:0/1432944274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557c5978ef90 con 0x557c59714a90 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f28e77fe640 1 -- 172.21.15.62:0/1432944274 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28fc029040 con 0x557c59714a90 2024-09-12T00:07:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f28e77fe640 1 -- 172.21.15.62:0/1432944274 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f290004e040 con 0x557c59714a90 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f29109da740 10 client.18198 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.317+0000 7f29109da740 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.318+0000 7f29109da740 20 client.18198 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.318+0000 7f29109da740 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557c5978f210 0x557c59791600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.318+0000 7f29109da740 4 client.18198 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.318+0000 7f29109da740 1 -- 172.21.15.62:0/1432944274 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c59672d60 con 0x557c5978f210 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.318+0000 7f29077fe640 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557c5978f210 0x557c59791600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.318+0000 7f29077fe640 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557c5978f210 0x557c59791600 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c231385bd21474a1 server_cookie=fe58a67a62d4a5f1 in_seq=0 out_seq=0 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.318+0000 7f28e77fe640 10 client.18198 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.318+0000 7f28e77fe640 1 -- 172.21.15.62:0/1432944274 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557c59672d60 con 0x557c5978f210 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.318+0000 7f28e77fe640 10 client.18198 handle_command_reply: tid=0 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 client.18198 shutdown 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 -- 172.21.15.62:0/1432944274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59714a90 msgr2=0x557c59718190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59714a90 0x557c59718190 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2900046040 tx=0x7f28fc0044a0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 -- 172.21.15.62:0/1432944274 shutdown_connections 2024-09-12T00:07:55.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x557c5978f210 0x557c59791600 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c59714a90 0x557c59718190 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f28e77fe640 0 client.18198 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:07:55.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c59705c60 0x557c59787c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 --2- 172.21.15.62:0/1432944274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c59705340 0x557c59787720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 -- 172.21.15.62:0/1432944274 >> 172.21.15.62:0/1432944274 conn(0x557c59701a30 msgr2=0x557c5970c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:55.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 -- 172.21.15.62:0/1432944274 shutdown_connections 2024-09-12T00:07:55.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 1 -- 172.21.15.62:0/1432944274 wait complete. 2024-09-12T00:07:55.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.319+0000 7f29109da740 20 client.18198 trim_cache size 0 max 16384 2024-09-12T00:07:55.711 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:07:55.711 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-09-12T00:07:55.711 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-09-12T00:07:55.711 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:07:55.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 -- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f28f00a2cf0 msgr2=0x7f28f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f28f00a2cf0 0x7f28f00a51b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28f80ccfc0 tx=0x7f28f80d5550 comp rx=0 tx=0).stop 2024-09-12T00:07:55.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 -- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2908100e20 msgr2=0x7f2908198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:55.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2908100e20 0x7f2908198450 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9f0 tx=0x7f28f800eeb0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 -- 172.21.15.62:0/3681585835 shutdown_connections 2024-09-12T00:07:55.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f28f00a2cf0 0x7f28f00a51b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2908102ef0 0x7f290819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2908102570 0x7f2908198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 --2- 172.21.15.62:0/3681585835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2908100e20 0x7f2908198450 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 -- 172.21.15.62:0/3681585835 >> 172.21.15.62:0/3681585835 conn(0x7f29080fc7d0 msgr2=0x7f290806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 -- 172.21.15.62:0/3681585835 shutdown_connections 2024-09-12T00:07:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:55.320+0000 7f290e601640 1 -- 172.21.15.62:0/3681585835 wait complete. 2024-09-12T00:07:55.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:55 smithi120 ceph-mon[35182]: pgmap v493: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 23 MiB/s wr, 1.89k op/s 2024-09-12T00:07:55.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:55 smithi083 ceph-mon[34805]: pgmap v493: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 23 MiB/s wr, 1.89k op/s 2024-09-12T00:07:55.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:55 smithi062 ceph-mon[30958]: pgmap v493: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 23 MiB/s wr, 1.89k op/s 2024-09-12T00:07:56.597 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:07:56.598 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:07:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:56.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:56 smithi120 ceph-mon[35182]: from='client.18174 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:07:56.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:56 smithi120 ceph-mon[35182]: from='client.27529 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:07:56.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:56 smithi083 ceph-mon[34805]: from='client.18174 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:07:56.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:56 smithi083 ceph-mon[34805]: from='client.27529 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:07:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:56 smithi062 ceph-mon[30958]: from='client.18174 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:07:56.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:56 smithi062 ceph-mon[30958]: from='client.27529 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:07:57.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.322+0000 7f093c0e4640 1 -- 172.21.15.62:0/854969493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0934102790 msgr2=0x7f09341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:57.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.322+0000 7f093c0e4640 1 --2- 172.21.15.62:0/854969493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0934102790 0x7f09341067e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0924009900 tx=0x7f092402a410 comp rx=0 tx=0).stop 2024-09-12T00:07:57.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.322+0000 7f093c0e4640 1 -- 172.21.15.62:0/854969493 shutdown_connections 2024-09-12T00:07:57.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.322+0000 7f093c0e4640 1 --2- 172.21.15.62:0/854969493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0934106f10 0x7f093410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.322+0000 7f093c0e4640 1 --2- 172.21.15.62:0/854969493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0934102790 0x7f09341067e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.322+0000 7f093c0e4640 1 --2- 172.21.15.62:0/854969493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0934101de0 0x7f09341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.322+0000 7f093c0e4640 1 -- 172.21.15.62:0/854969493 >> 172.21.15.62:0/854969493 conn(0x7f09340fd8c0 msgr2=0x7f09340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.322+0000 7f093c0e4640 1 -- 172.21.15.62:0/854969493 shutdown_connections 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.322+0000 7f093c0e4640 1 -- 172.21.15.62:0/854969493 wait complete. 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.323+0000 7f093c0e4640 1 Processor -- start 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.323+0000 7f093c0e4640 1 -- start start 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.323+0000 7f093c0e4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0934101de0 0x7f0934198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.323+0000 7f093c0e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0934102790 0x7f0934198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.323+0000 7f093c0e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0934106f10 0x7f093419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.323+0000 7f093c0e4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0934111f70 con 0x7f0934101de0 2024-09-12T00:07:57.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.323+0000 7f093c0e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0934111df0 con 0x7f0934106f10 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.323+0000 7f093c0e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09341120f0 con 0x7f0934102790 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f0939e80640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0934101de0 0x7f0934198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f0939e80640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0934101de0 0x7f0934198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32818/0 (socket says 172.21.15.62:32818) 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f0939e80640 1 -- 172.21.15.62:0/3155907369 learned_addr learned my addr 172.21.15.62:0/3155907369 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f093967f640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0934102790 0x7f0934198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f093a681640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0934106f10 0x7f093419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f0939e80640 1 -- 172.21.15.62:0/3155907369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0934102790 msgr2=0x7f0934198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f0939e80640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0934102790 0x7f0934198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f0939e80640 1 -- 172.21.15.62:0/3155907369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0934106f10 msgr2=0x7f093419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f0939e80640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0934106f10 0x7f093419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f0939e80640 1 -- 172.21.15.62:0/3155907369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093419d400 con 0x7f0934101de0 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f0939e80640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0934101de0 0x7f0934198450 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f093000c890 tx=0x7f093000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f092affd640 1 -- 172.21.15.62:0/3155907369 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930004220 con 0x7f0934101de0 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f092affd640 1 -- 172.21.15.62:0/3155907369 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09300043c0 con 0x7f0934101de0 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f092affd640 1 -- 172.21.15.62:0/3155907369 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09300049e0 con 0x7f0934101de0 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f093c0e4640 1 -- 172.21.15.62:0/3155907369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093419d6f0 con 0x7f0934101de0 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.324+0000 7f093c0e4640 1 -- 172.21.15.62:0/3155907369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09341a4fd0 con 0x7f0934101de0 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.325+0000 7f093c0e4640 1 -- 172.21.15.62:0/3155907369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09341021c0 con 0x7f0934101de0 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.327+0000 7f092affd640 1 -- 172.21.15.62:0/3155907369 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f09300079b0 con 0x7f0934101de0 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.327+0000 7f092affd640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f09200a2cf0 0x7f09200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.327+0000 7f093967f640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f09200a2cf0 0x7f09200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.327+0000 7f092affd640 1 -- 172.21.15.62:0/3155907369 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f09300cf400 con 0x7f0934101de0 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.328+0000 7f093967f640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f09200a2cf0 0x7f09200a51b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0924009230 tx=0x7f09240058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:57.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.331+0000 7f092affd640 1 -- 172.21.15.62:0/3155907369 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0930014030 con 0x7f0934101de0 2024-09-12T00:07:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.541+0000 7f093c0e4640 1 -- 172.21.15.62:0/3155907369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09341997b0 con 0x7f0934101de0 2024-09-12T00:07:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.542+0000 7f092affd640 1 -- 172.21.15.62:0/3155907369 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11913 (secure 0 0 0) 0x7f093009b9d0 con 0x7f0934101de0 2024-09-12T00:07:57.542 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:07:57.542 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":41,"btime":"2024-09-12T00:07:52:543212+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:07:51.825352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24484,24469]},"id":1}]} 2024-09-12T00:07:57.542 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 41 2024-09-12T00:07:57.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 -- 172.21.15.62:0/3155907369 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f09200a2cf0 msgr2=0x7f09200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:57.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f09200a2cf0 0x7f09200a51b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0924009230 tx=0x7f09240058d0 comp rx=0 tx=0).stop 2024-09-12T00:07:57.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 -- 172.21.15.62:0/3155907369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0934101de0 msgr2=0x7f0934198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:57.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0934101de0 0x7f0934198450 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f093000c890 tx=0x7f093000cd50 comp rx=0 tx=0).stop 2024-09-12T00:07:57.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 -- 172.21.15.62:0/3155907369 shutdown_connections 2024-09-12T00:07:57.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f09200a2cf0 0x7f09200a51b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:57.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0934106f10 0x7f093419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:57.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0934102790 0x7f0934198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:57.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 --2- 172.21.15.62:0/3155907369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0934101de0 0x7f0934198450 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:57.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 -- 172.21.15.62:0/3155907369 >> 172.21.15.62:0/3155907369 conn(0x7f09340fd8c0 msgr2=0x7f09340fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:57.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.544+0000 7f093c0e4640 1 -- 172.21.15.62:0/3155907369 shutdown_connections 2024-09-12T00:07:57.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:57.545+0000 7f093c0e4640 1 -- 172.21.15.62:0/3155907369 wait complete. 2024-09-12T00:07:57.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:57 smithi120 ceph-mon[35182]: pgmap v494: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2024-09-12T00:07:57.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:57 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3155907369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:07:57.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:57 smithi083 ceph-mon[34805]: pgmap v494: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2024-09-12T00:07:57.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:57 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3155907369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:07:57.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:57 smithi062 ceph-mon[30958]: pgmap v494: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2024-09-12T00:07:57.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:57 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3155907369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:07:58.227 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:07:58.486 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:07:58.870 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:07:58 smithi120 ceph-mon[35182]: pgmap v495: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2024-09-12T00:07:58.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:07:58 smithi062 ceph-mon[30958]: pgmap v495: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2024-09-12T00:07:58.917 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:07:58 smithi083 ceph-mon[34805]: pgmap v495: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2024-09-12T00:07:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.961+0000 7fdebda82640 1 -- 172.21.15.62:0/492364945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb810c170 msgr2=0x7fdeb810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.961+0000 7fdebda82640 1 --2- 172.21.15.62:0/492364945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb810c170 0x7fdeb810e610 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009ca0 tx=0x7fdeb002a950 comp rx=0 tx=0).stop 2024-09-12T00:07:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.961+0000 7fdebda82640 1 -- 172.21.15.62:0/492364945 shutdown_connections 2024-09-12T00:07:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.961+0000 7fdebda82640 1 --2- 172.21.15.62:0/492364945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb810c170 0x7fdeb810e610 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.961+0000 7fdebda82640 1 --2- 172.21.15.62:0/492364945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdeb8103660 0x7fdeb810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.961+0000 7fdebda82640 1 --2- 172.21.15.62:0/492364945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb8102d40 0x7fdeb8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.961+0000 7fdebda82640 1 -- 172.21.15.62:0/492364945 >> 172.21.15.62:0/492364945 conn(0x7fdeb80fc7a0 msgr2=0x7fdeb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:58.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.961+0000 7fdebda82640 1 -- 172.21.15.62:0/492364945 shutdown_connections 2024-09-12T00:07:58.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.961+0000 7fdebda82640 1 -- 172.21.15.62:0/492364945 wait complete. 2024-09-12T00:07:58.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdebda82640 1 Processor -- start 2024-09-12T00:07:58.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdebda82640 1 -- start start 2024-09-12T00:07:58.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdebda82640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb8102d40 0x7fdeb8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdebda82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdeb8103660 0x7fdeb81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdebda82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb810c170 0x7fdeb819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdebda82640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb8111f50 con 0x7fdeb8102d40 2024-09-12T00:07:58.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdebda82640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb8111dd0 con 0x7fdeb810c170 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdebda82640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb81120d0 con 0x7fdeb8103660 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdeb77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb8102d40 0x7fdeb8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdeb77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb8102d40 0x7fdeb8198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32842/0 (socket says 172.21.15.62:32842) 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdeb77fe640 1 -- 172.21.15.62:0/699629350 learned_addr learned my addr 172.21.15.62:0/699629350 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdeb8103660 0x7fdeb81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.962+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb810c170 0x7fdeb819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdeb77fe640 1 -- 172.21.15.62:0/699629350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdeb8103660 msgr2=0x7fdeb81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdeb77fe640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdeb8103660 0x7fdeb81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdeb77fe640 1 -- 172.21.15.62:0/699629350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb810c170 msgr2=0x7fdeb819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdeb77fe640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb810c170 0x7fdeb819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdeb77fe640 1 -- 172.21.15.62:0/699629350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb819d410 con 0x7fdeb8102d40 2024-09-12T00:07:58.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdeb77fe640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb8102d40 0x7fdeb8198460 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fdea800e9f0 tx=0x7fdea800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:58.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdeb57fa640 1 -- 172.21.15.62:0/699629350 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea800cd80 con 0x7fdeb8102d40 2024-09-12T00:07:58.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdebda82640 1 -- 172.21.15.62:0/699629350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb819d700 con 0x7fdeb8102d40 2024-09-12T00:07:58.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdeb57fa640 1 -- 172.21.15.62:0/699629350 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea8010910 con 0x7fdeb8102d40 2024-09-12T00:07:58.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdeb57fa640 1 -- 172.21.15.62:0/699629350 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea80188f0 con 0x7fdeb8102d40 2024-09-12T00:07:58.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.963+0000 7fdebda82640 1 -- 172.21.15.62:0/699629350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb81a4fe0 con 0x7fdeb8102d40 2024-09-12T00:07:58.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.965+0000 7fdeb57fa640 1 -- 172.21.15.62:0/699629350 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdea8010ac0 con 0x7fdeb8102d40 2024-09-12T00:07:58.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.965+0000 7fdebfe5b740 1 Processor -- start 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdebfe5b740 1 -- start start 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669a2b40 0x5648669a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x5648669a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669b0780 0x5648669b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5648669072f0 con 0x5648669a2b40 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5648668aecf0 con 0x5648669b0780 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5648669032b0 con 0x5648669a15d0 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669a2b40 0x5648669a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669a2b40 0x5648669a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32846/0 (socket says 172.21.15.62:32846) 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeb7fff640 1 -- 172.21.15.62:0/881972281 learned_addr learned my addr 172.21.15.62:0/881972281 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/881972281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669b0780 0x5648669b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeb77fe640 1 --2- 172.21.15.62:0/881972281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x5648669a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeb7fff640 1 -- 172.21.15.62:0/881972281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 msgr2=0x5648669a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/881972281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x5648669a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeb7fff640 1 -- 172.21.15.62:0/881972281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669b0780 msgr2=0x5648669b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/881972281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669b0780 0x5648669b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeb7fff640 1 -- 172.21.15.62:0/881972281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648668933a0 con 0x5648669a2b40 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.966+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/881972281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669b0780 0x5648669b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.967+0000 7fdeb57fa640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdea40a2cf0 0x7fdea40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.967+0000 7fdeb77fe640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdea40a2cf0 0x7fdea40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.967+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/881972281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669a2b40 0x5648669a1090 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0035000 tx=0x7fdeb00093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28ef27b236ac315 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.967+0000 7fdeb57fa640 1 -- 172.21.15.62:0/699629350 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fdea80cf800 con 0x7fdeb8102d40 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.967+0000 7fdeadffb640 1 -- 172.21.15.62:0/881972281 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0034040 con 0x5648669a2b40 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.967+0000 7fdeadffb640 1 -- 172.21.15.62:0/881972281 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb002f940 con 0x5648669a2b40 2024-09-12T00:07:58.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.967+0000 7fdeadffb640 1 -- 172.21.15.62:0/881972281 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb00042a0 con 0x5648669a2b40 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 -- 172.21.15.62:0/881972281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669a2b40 msgr2=0x5648669a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/881972281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669a2b40 0x5648669a1090 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0035000 tx=0x7fdeb00093e0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdeb77fe640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdea40a2cf0 0x7fdea40a51b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdea80cd000 tx=0x7fdea80d5280 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 -- 172.21.15.62:0/881972281 shutdown_connections 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/881972281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669b0780 0x5648669b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/881972281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x5648669a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/881972281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669a2b40 0x5648669a1090 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 -- 172.21.15.62:0/881972281 >> 172.21.15.62:0/881972281 conn(0x56486696a310 msgr2=0x56486699dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 -- 172.21.15.62:0/881972281 shutdown_connections 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 -- 172.21.15.62:0/881972281 wait complete. 2024-09-12T00:07:58.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 -- start start 2024-09-12T00:07:58.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.968+0000 7fdebfe5b740 1 Processor -- start 2024-09-12T00:07:58.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a15d0 0x5648669b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a2b40 0x5648669ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5648669b3420 con 0x5648669b0780 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5648668c2060 con 0x5648669a15d0 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5648669b35a0 con 0x5648669a2b40 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdebfe5b740 10 client.0 _ll_register_callbacks cb 0x5648666158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32858/0 (socket says 172.21.15.62:32858) 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeb7fff640 1 -- 172.21.15.62:0/3857928741 learned_addr learned my addr 172.21.15.62:0/3857928741 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a2b40 0x5648669ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a15d0 0x5648669b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeb7fff640 1 -- 172.21.15.62:0/3857928741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a2b40 msgr2=0x5648669ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:58.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a2b40 0x5648669ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeb7fff640 1 -- 172.21.15.62:0/3857928741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a15d0 msgr2=0x5648669b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a15d0 0x5648669b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeb7fff640 1 -- 172.21.15.62:0/3857928741 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648666783c0 con 0x5648669b0780 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669ada10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0003ba0 tx=0x7fdeb0003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecf091699802960 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.969+0000 7fde96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fde96ffd640 1 -- 172.21.15.62:0/3857928741 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0049040 con 0x5648669b0780 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fde96ffd640 1 -- 172.21.15.62:0/3857928741 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fdeb0004790 con 0x5648669b0780 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3857928741 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb002f940 con 0x5648669b0780 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fde96ffd640 1 -- 172.21.15.62:0/3857928741 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdeb002f3a0 con 0x5648669b0780 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fdebfe5b740 10 client.18216 fetch_fsmap learned FSMap version 41 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3857928741 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5648669bee90 con 0x5648669b0780 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fde96ffd640 1 -- 172.21.15.62:0/3857928741 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb002f940 con 0x5648669b0780 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fde96ffd640 1 -- 172.21.15.62:0/3857928741 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7fdeb004f040 con 0x5648669b0780 2024-09-12T00:07:58.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fdebfe5b740 10 client.18216 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fdebfe5b740 10 client.18216 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fdebfe5b740 20 client.18216 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5648669c01e0 0x5648669c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fdebfe5b740 4 client.18216 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.970+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3857928741 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564866914860 con 0x5648669c01e0 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.971+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5648669c01e0 0x5648669c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.971+0000 7fde96ffd640 10 client.18216 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.971+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5648669c01e0 0x5648669c25d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acd45d58a12a038d server_cookie=8dc33ceb629e7b88 in_seq=0 out_seq=0 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fde96ffd640 1 -- 172.21.15.62:0/3857928741 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564866914860 con 0x5648669c01e0 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fde96ffd640 10 client.18216 handle_command_reply: tid=0 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 client.18216 shutdown 2024-09-12T00:07:58.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3857928741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 msgr2=0x5648669ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669ada10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0003ba0 tx=0x7fdeb0003c00 comp rx=0 tx=0).stop 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3857928741 shutdown_connections 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x5648669c01e0 0x5648669c25d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669ada10 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a2b40 0x5648669ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3857928741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a15d0 0x5648669b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3857928741 >> 172.21.15.62:0/3857928741 conn(0x56486696a310 msgr2=0x5648669a3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fde96ffd640 0 client.18216 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3857928741 shutdown_connections 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3857928741 wait complete. 2024-09-12T00:07:58.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.972+0000 7fdebfe5b740 20 client.18216 trim_cache size 0 max 16384 2024-09-12T00:07:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.985+0000 7fdebfe5b740 1 Processor -- start 2024-09-12T00:07:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.985+0000 7fdebfe5b740 1 -- start start 2024-09-12T00:07:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.985+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x56486699cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.985+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 0x5648669c5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.985+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669c5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.985+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb81adf10 con 0x5648669b0780 2024-09-12T00:07:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.985+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb81ab750 con 0x5648669a2b40 2024-09-12T00:07:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.985+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb81a95e0 con 0x5648669a15d0 2024-09-12T00:07:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669c5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32868/0 (socket says 172.21.15.62:32868) 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeb7fff640 1 -- 172.21.15.62:0/1702376167 learned_addr learned my addr 172.21.15.62:0/1702376167 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/1702376167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x56486699cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/1702376167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 0x5648669c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeb7fff640 1 -- 172.21.15.62:0/1702376167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 msgr2=0x56486699cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/1702376167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x56486699cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeb7fff640 1 -- 172.21.15.62:0/1702376167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 msgr2=0x5648669c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/1702376167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 0x5648669c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeb7fff640 1 -- 172.21.15.62:0/1702376167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648669afd00 con 0x5648669b0780 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/1702376167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 0x5648669c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/1702376167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669c5750 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fdeb002ff70 tx=0x7fdeb002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7079fe2df056beea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdead7fa640 1 -- 172.21.15.62:0/1702376167 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb002f940 con 0x5648669b0780 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdead7fa640 1 -- 172.21.15.62:0/1702376167 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb0009540 con 0x5648669b0780 2024-09-12T00:07:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.986+0000 7fdead7fa640 1 -- 172.21.15.62:0/1702376167 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0004790 con 0x5648669b0780 2024-09-12T00:07:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 -- 172.21.15.62:0/1702376167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 msgr2=0x5648669c5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/1702376167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669c5750 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fdeb002ff70 tx=0x7fdeb002a950 comp rx=0 tx=0).stop 2024-09-12T00:07:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 -- 172.21.15.62:0/1702376167 shutdown_connections 2024-09-12T00:07:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/1702376167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669c5750 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/1702376167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 0x5648669c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/1702376167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x56486699cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 -- 172.21.15.62:0/1702376167 >> 172.21.15.62:0/1702376167 conn(0x56486696a310 msgr2=0x56486699c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 -- 172.21.15.62:0/1702376167 shutdown_connections 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 -- 172.21.15.62:0/1702376167 wait complete. 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 -- start start 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.987+0000 7fdebfe5b740 1 Processor -- start 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x5648669c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 0x5648669c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdebfe5b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb81a9c30 con 0x5648669b0780 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb81ab080 con 0x5648669a2b40 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdebfe5b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb81aa2e0 con 0x5648669a15d0 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdebfe5b740 10 client.0 _ll_register_callbacks cb 0x7fdea0009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x5648669c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x5648669c5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.62:48876/0 (socket says 172.21.15.62:48876) 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdeb7fff640 1 -- 172.21.15.62:0/3583007807 learned_addr learned my addr 172.21.15.62:0/3583007807 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 0x5648669c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdeb7fff640 1 -- 172.21.15.62:0/3583007807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 msgr2=0x5648669c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 0x5648669c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdeb7fff640 1 -- 172.21.15.62:0/3583007807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 msgr2=0x5648669c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669c31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.988+0000 7fdeb7fff640 1 -- 172.21.15.62:0/3583007807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648669c3a20 con 0x5648669a15d0 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.989+0000 7fdeb7fff640 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x5648669c5960 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdeb003a4b0 tx=0x7fdeb0003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b50996153c95299b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.989+0000 7fde96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.989+0000 7fde96ffd640 1 -- 172.21.15.62:0/3583007807 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0034030 con 0x5648669a15d0 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.989+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3583007807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb0009540 con 0x5648669a15d0 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.989+0000 7fde96ffd640 1 -- 172.21.15.62:0/3583007807 <== mon.1 v2:172.21.15.120:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fdeb0040040 con 0x5648669a15d0 2024-09-12T00:07:58.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.989+0000 7fde96ffd640 1 -- 172.21.15.62:0/3583007807 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdeb0049040 con 0x5648669a15d0 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.989+0000 7fdebfe5b740 10 client.27559 fetch_fsmap learned FSMap version 41 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.989+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3583007807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564866a2aa80 con 0x5648669a15d0 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.989+0000 7fde96ffd640 1 -- 172.21.15.62:0/3583007807 <== mon.1 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb0009540 con 0x5648669a15d0 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.989+0000 7fde96ffd640 1 -- 172.21.15.62:0/3583007807 <== mon.1 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7fdeb004f040 con 0x5648669a15d0 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fdebfe5b740 10 client.27559 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fdebfe5b740 10 client.27559 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fdebfe5b740 20 client.27559 populate_metadata read hostname 'smithi062' 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x564866a2b140 0x564866a2d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fdebfe5b740 4 client.27559 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3583007807 --> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56486690ed70 con 0x564866a2b140 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x564866a2b140 0x564866a2d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fdeaeffd640 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x564866a2b140 0x564866a2d530 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be0f5269479c9f24 server_cookie=b16614d5842e095f in_seq=0 out_seq=0 2024-09-12T00:07:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fde96ffd640 10 client.27559 ms_handle_connect on v2:172.21.15.120:6836/1796168792 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fde96ffd640 1 -- 172.21.15.62:0/3583007807 <== mds.2 v2:172.21.15.120:6836/1796168792 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56486690ed70 con 0x564866a2b140 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fde96ffd640 10 client.27559 handle_command_reply: tid=0 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.990+0000 7fdebfe5b740 1 client.27559 shutdown 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3583007807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 msgr2=0x5648669c5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x5648669c5960 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdeb003a4b0 tx=0x7fdeb0003970 comp rx=0 tx=0).stop 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3583007807 shutdown_connections 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.120:6836/1796168792,v1:172.21.15.120:6837/1796168792] conn(0x564866a2b140 0x564866a2d530 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648669b0780 0x5648669c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fde96ffd640 0 client.27559 ms_handle_reset on v2:172.21.15.120:6836/1796168792 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648669a2b40 0x5648669c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 1 --2- 172.21.15.62:0/3583007807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648669a15d0 0x5648669c5960 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3583007807 >> 172.21.15.62:0/3583007807 conn(0x56486696a310 msgr2=0x5648669af480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3583007807 shutdown_connections 2024-09-12T00:07:58.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 1 -- 172.21.15.62:0/3583007807 wait complete. 2024-09-12T00:07:58.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebfe5b740 20 client.27559 trim_cache size 0 max 16384 2024-09-12T00:07:58.996 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:07:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebda82640 1 -- 172.21.15.62:0/699629350 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdea40a2cf0 msgr2=0x7fdea40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebda82640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdea40a2cf0 0x7fdea40a51b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdea80cd000 tx=0x7fdea80d5280 comp rx=0 tx=0).stop 2024-09-12T00:07:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebda82640 1 -- 172.21.15.62:0/699629350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb8102d40 msgr2=0x7fdeb8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:07:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebda82640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb8102d40 0x7fdeb8198460 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fdea800e9f0 tx=0x7fdea800eeb0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebda82640 1 -- 172.21.15.62:0/699629350 shutdown_connections 2024-09-12T00:07:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebda82640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdea40a2cf0 0x7fdea40a51b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebda82640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb810c170 0x7fdeb819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebda82640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdeb8103660 0x7fdeb81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebda82640 1 --2- 172.21.15.62:0/699629350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb8102d40 0x7fdeb8198460 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:07:58.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.991+0000 7fdebda82640 1 -- 172.21.15.62:0/699629350 >> 172.21.15.62:0/699629350 conn(0x7fdeb80fc7a0 msgr2=0x7fdeb80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:07:58.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.992+0000 7fdebda82640 1 -- 172.21.15.62:0/699629350 shutdown_connections 2024-09-12T00:07:58.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:07:58.992+0000 7fdebda82640 1 -- 172.21.15.62:0/699629350 wait complete. 2024-09-12T00:07:59.704 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:07:59.965 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:00.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 -- 172.21.15.62:0/538911806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff248100e20 msgr2=0x7ff248101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:00.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 --2- 172.21.15.62:0/538911806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff248100e20 0x7ff248101200 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff238009900 tx=0x7ff23802a480 comp rx=0 tx=0).stop 2024-09-12T00:08:00.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 -- 172.21.15.62:0/538911806 shutdown_connections 2024-09-12T00:08:00.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 --2- 172.21.15.62:0/538911806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff248102ef0 0x7ff24810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 --2- 172.21.15.62:0/538911806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff248102570 0x7ff2481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 --2- 172.21.15.62:0/538911806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff248100e20 0x7ff248101200 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 -- 172.21.15.62:0/538911806 >> 172.21.15.62:0/538911806 conn(0x7ff2480fc7d0 msgr2=0x7ff2480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 -- 172.21.15.62:0/538911806 shutdown_connections 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 -- 172.21.15.62:0/538911806 wait complete. 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 Processor -- start 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 -- start start 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.449+0000 7ff24d3ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff248100e20 0x7ff248198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff24d3ac640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff248102570 0x7ff2481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff24d3ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff248102ef0 0x7ff24819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff24d3ac640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff248110f80 con 0x7ff248102570 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff24d3ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff248110e00 con 0x7ff248102ef0 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff24d3ac640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff248111100 con 0x7ff248100e20 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff23ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff248102570 0x7ff2481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff23ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff248102570 0x7ff2481989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32898/0 (socket says 172.21.15.62:32898) 2024-09-12T00:08:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff23ffff640 1 -- 172.21.15.62:0/111093462 learned_addr learned my addr 172.21.15.62:0/111093462 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff246ffd640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff248100e20 0x7ff248198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff2477fe640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff248102ef0 0x7ff24819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff23ffff640 1 -- 172.21.15.62:0/111093462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff248100e20 msgr2=0x7ff248198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff23ffff640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff248100e20 0x7ff248198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff23ffff640 1 -- 172.21.15.62:0/111093462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff248102ef0 msgr2=0x7ff24819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff23ffff640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff248102ef0 0x7ff24819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff23ffff640 1 -- 172.21.15.62:0/111093462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24819d420 con 0x7ff248102570 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff2477fe640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff248102ef0 0x7ff24819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff23ffff640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff248102570 0x7ff2481989b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff23000e9f0 tx=0x7ff23000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff244ff9640 1 -- 172.21.15.62:0/111093462 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23000cd80 con 0x7ff248102570 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff244ff9640 1 -- 172.21.15.62:0/111093462 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff230010910 con 0x7ff248102570 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff24d3ac640 1 -- 172.21.15.62:0/111093462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24819d710 con 0x7ff248102570 2024-09-12T00:08:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.450+0000 7ff244ff9640 1 -- 172.21.15.62:0/111093462 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230018990 con 0x7ff248102570 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.451+0000 7ff24d3ac640 1 -- 172.21.15.62:0/111093462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2481a4ff0 con 0x7ff248102570 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.453+0000 7ff244ff9640 1 -- 172.21.15.62:0/111093462 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff230010ac0 con 0x7ff248102570 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.453+0000 7ff24f785740 1 Processor -- start 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.453+0000 7ff24f785740 1 -- start start 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.453+0000 7ff24f785740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799a0b40 0x56547999f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.453+0000 7ff24f785740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56547999f5d0 0x56547999fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff24f785740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799ae780 0x5654799b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff24f785740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5654799052f0 con 0x5654799a0b40 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff24f785740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654798accf0 con 0x56547999f5d0 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff24f785740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654799012b0 con 0x5654799ae780 2024-09-12T00:08:00.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff2477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799a0b40 0x56547999f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff244ff9640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2340a2cf0 0x7ff2340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff2477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799a0b40 0x56547999f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32904/0 (socket says 172.21.15.62:32904) 2024-09-12T00:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff2477fe640 1 -- 172.21.15.62:0/774847938 learned_addr learned my addr 172.21.15.62:0/774847938 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff246ffd640 1 --2- 172.21.15.62:0/774847938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56547999f5d0 0x56547999fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff23ffff640 1 --2- 172.21.15.62:0/774847938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799ae780 0x5654799b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff2477fe640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2340a2cf0 0x7ff2340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff244ff9640 1 -- 172.21.15.62:0/111093462 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7ff2300cf810 con 0x7ff248102570 2024-09-12T00:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff2477fe640 1 -- 172.21.15.62:0/774847938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799ae780 msgr2=0x5654799b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff2477fe640 1 --2- 172.21.15.62:0/774847938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799ae780 0x5654799b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff2477fe640 1 -- 172.21.15.62:0/774847938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56547999f5d0 msgr2=0x56547999fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff2477fe640 1 --2- 172.21.15.62:0/774847938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56547999f5d0 0x56547999fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff2477fe640 1 -- 172.21.15.62:0/774847938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654798913a0 con 0x5654799a0b40 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff2477fe640 1 --2- 172.21.15.62:0/774847938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799a0b40 0x56547999f090 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff240014a90 tx=0x7ff240014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e8a4487f536151 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.454+0000 7ff23d7fa640 1 -- 172.21.15.62:0/774847938 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240014530 con 0x5654799a0b40 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff23d7fa640 1 -- 172.21.15.62:0/774847938 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff240023070 con 0x5654799a0b40 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff23d7fa640 1 -- 172.21.15.62:0/774847938 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24000cda0 con 0x5654799a0b40 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff24f785740 1 -- 172.21.15.62:0/774847938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799a0b40 msgr2=0x56547999f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff2477fe640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2340a2cf0 0x7ff2340a51b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff240022000 tx=0x7ff240020000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff24f785740 1 --2- 172.21.15.62:0/774847938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799a0b40 0x56547999f090 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff240014a90 tx=0x7ff240014f50 comp rx=0 tx=0).stop 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff24f785740 1 -- 172.21.15.62:0/774847938 shutdown_connections 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff24f785740 1 --2- 172.21.15.62:0/774847938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799ae780 0x5654799b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff24f785740 1 --2- 172.21.15.62:0/774847938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56547999f5d0 0x56547999fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff24f785740 1 --2- 172.21.15.62:0/774847938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799a0b40 0x56547999f090 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff24f785740 1 -- 172.21.15.62:0/774847938 >> 172.21.15.62:0/774847938 conn(0x565479968310 msgr2=0x56547999bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff24f785740 1 -- 172.21.15.62:0/774847938 shutdown_connections 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.455+0000 7ff24f785740 1 -- 172.21.15.62:0/774847938 wait complete. 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff24f785740 1 -- start start 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff24f785740 1 Processor -- start 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff24f785740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56547999f5d0 0x5654799ae150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff24f785740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799a0b40 0x5654799ab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff24f785740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff24f785740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5654799b1480 con 0x5654799ae780 2024-09-12T00:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff24f785740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654798c0060 con 0x5654799a0b40 2024-09-12T00:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff24f785740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654799b1600 con 0x56547999f5d0 2024-09-12T00:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff24f785740 10 client.0 _ll_register_callbacks cb 0x5654796138c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff2477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff2477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32920/0 (socket says 172.21.15.62:32920) 2024-09-12T00:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff2477fe640 1 -- 172.21.15.62:0/1762550053 learned_addr learned my addr 172.21.15.62:0/1762550053 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff23ffff640 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799a0b40 0x5654799ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.456+0000 7ff246ffd640 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56547999f5d0 0x5654799ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff2477fe640 1 -- 172.21.15.62:0/1762550053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56547999f5d0 msgr2=0x5654799ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff2477fe640 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56547999f5d0 0x5654799ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff2477fe640 1 -- 172.21.15.62:0/1762550053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799a0b40 msgr2=0x5654799ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff2477fe640 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799a0b40 0x5654799ab400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff2477fe640 1 -- 172.21.15.62:0/1762550053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654796763c0 con 0x5654799ae780 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff2477fe640 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799ab9f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff240012910 tx=0x7ff240005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f14ba46977cec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff22e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1762550053 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240036060 con 0x5654799ae780 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff24f785740 1 -- 172.21.15.62:0/1762550053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff240023070 con 0x5654799ae780 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1762550053 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7ff240012d70 con 0x5654799ae780 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1762550053 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff24000cb20 con 0x5654799ae780 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff24f785740 10 client.18234 fetch_fsmap learned FSMap version 41 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff24f785740 1 -- 172.21.15.62:0/1762550053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5654799bcef0 con 0x5654799ae780 2024-09-12T00:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1762550053 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff240023070 con 0x5654799ae780 2024-09-12T00:08:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.457+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1762550053 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7ff24000f040 con 0x5654799ae780 2024-09-12T00:08:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.458+0000 7ff24f785740 10 client.18234 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T00:08:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.458+0000 7ff24f785740 10 client.18234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:08:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.458+0000 7ff24f785740 20 client.18234 populate_metadata read hostname 'smithi062' 2024-09-12T00:08:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.458+0000 7ff24f785740 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5654799be300 0x5654799c06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.458+0000 7ff24f785740 4 client.18234 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:08:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.458+0000 7ff24f785740 1 -- 172.21.15.62:0/1762550053 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565479912860 con 0x5654799be300 2024-09-12T00:08:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.458+0000 7ff246ffd640 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5654799be300 0x5654799c06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.458+0000 7ff246ffd640 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5654799be300 0x5654799c06f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d6761b106e08209 server_cookie=fb2143be9d4c25f2 in_seq=0 out_seq=0 2024-09-12T00:08:00.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.458+0000 7ff22e7fc640 10 client.18234 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1762550053 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565479912860 con 0x5654799be300 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff22e7fc640 10 client.18234 handle_command_reply: tid=0 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 client.18234 shutdown 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 -- 172.21.15.62:0/1762550053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 msgr2=0x5654799ab9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799ab9f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff240012910 tx=0x7ff240005f10 comp rx=0 tx=0).stop 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 -- 172.21.15.62:0/1762550053 shutdown_connections 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x5654799be300 0x5654799c06f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799ab9f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff22e7fc640 0 client.18234 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799a0b40 0x5654799ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 --2- 172.21.15.62:0/1762550053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56547999f5d0 0x5654799ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 -- 172.21.15.62:0/1762550053 >> 172.21.15.62:0/1762550053 conn(0x565479968310 msgr2=0x5654799a1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:00.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 -- 172.21.15.62:0/1762550053 shutdown_connections 2024-09-12T00:08:00.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 1 -- 172.21.15.62:0/1762550053 wait complete. 2024-09-12T00:08:00.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.459+0000 7ff24f785740 20 client.18234 trim_cache size 0 max 16384 2024-09-12T00:08:00.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.472+0000 7ff24f785740 1 Processor -- start 2024-09-12T00:08:00.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.472+0000 7ff24f785740 1 -- start start 2024-09-12T00:08:00.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff24f785740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56547999f5d0 0x5654799ba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff24f785740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 0x5654799aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff24f785740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff24f785740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481adf60 con 0x5654799ae780 2024-09-12T00:08:00.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff24f785740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481ab7a0 con 0x56547999f5d0 2024-09-12T00:08:00.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff24f785740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481a9630 con 0x5654799a0b40 2024-09-12T00:08:00.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff2477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff2477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799c5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32926/0 (socket says 172.21.15.62:32926) 2024-09-12T00:08:00.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff2477fe640 1 -- 172.21.15.62:0/2879515767 learned_addr learned my addr 172.21.15.62:0/2879515767 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff246ffd640 1 --2- 172.21.15.62:0/2879515767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56547999f5d0 0x5654799ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff23ffff640 1 --2- 172.21.15.62:0/2879515767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 0x5654799aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff2477fe640 1 -- 172.21.15.62:0/2879515767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 msgr2=0x5654799aa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff2477fe640 1 --2- 172.21.15.62:0/2879515767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 0x5654799aa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff2477fe640 1 -- 172.21.15.62:0/2879515767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56547999f5d0 msgr2=0x5654799ba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff2477fe640 1 --2- 172.21.15.62:0/2879515767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56547999f5d0 0x5654799ba720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff2477fe640 1 -- 172.21.15.62:0/2879515767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56547999bed0 con 0x5654799ae780 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff23ffff640 1 --2- 172.21.15.62:0/2879515767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 0x5654799aa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff2477fe640 1 --2- 172.21.15.62:0/2879515767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799c5d80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff240009f30 tx=0x7ff240009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca990413dd54ca16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff23cff9640 1 -- 172.21.15.62:0/2879515767 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240036070 con 0x5654799ae780 2024-09-12T00:08:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff23cff9640 1 -- 172.21.15.62:0/2879515767 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff240023070 con 0x5654799ae780 2024-09-12T00:08:00.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.473+0000 7ff23cff9640 1 -- 172.21.15.62:0/2879515767 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240027b50 con 0x5654799ae780 2024-09-12T00:08:00.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.474+0000 7ff24f785740 1 -- 172.21.15.62:0/2879515767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 msgr2=0x5654799c5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:00.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.474+0000 7ff24f785740 1 --2- 172.21.15.62:0/2879515767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799c5d80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff240009f30 tx=0x7ff240009800 comp rx=0 tx=0).stop 2024-09-12T00:08:00.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.474+0000 7ff24f785740 1 -- 172.21.15.62:0/2879515767 shutdown_connections 2024-09-12T00:08:00.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.474+0000 7ff24f785740 1 --2- 172.21.15.62:0/2879515767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5654799ae780 0x5654799c5d80 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.474+0000 7ff24f785740 1 --2- 172.21.15.62:0/2879515767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 0x5654799aa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.474+0000 7ff24f785740 1 --2- 172.21.15.62:0/2879515767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56547999f5d0 0x5654799ba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.474+0000 7ff24f785740 1 -- 172.21.15.62:0/2879515767 >> 172.21.15.62:0/2879515767 conn(0x565479968310 msgr2=0x5654799a9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:00.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.474+0000 7ff24f785740 1 -- 172.21.15.62:0/2879515767 shutdown_connections 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.474+0000 7ff24f785740 1 -- 172.21.15.62:0/2879515767 wait complete. 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff24f785740 1 -- start start 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff24f785740 1 Processor -- start 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff24f785740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56547999f5d0 0x5654799b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff24f785740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 0x5654799b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff24f785740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799ae780 0x5654799b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff24f785740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481a9c80 con 0x56547999f5d0 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff24f785740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481ab0d0 con 0x5654799ae780 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff24f785740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481aa330 con 0x5654799a0b40 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff24f785740 10 client.0 _ll_register_callbacks cb 0x7ff238009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:08:00.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff246ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56547999f5d0 0x5654799b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff246ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56547999f5d0 0x5654799b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32936/0 (socket says 172.21.15.62:32936) 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff246ffd640 1 -- 172.21.15.62:0/1337661685 learned_addr learned my addr 172.21.15.62:0/1337661685 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff2477fe640 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799ae780 0x5654799b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff23ffff640 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 0x5654799b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff246ffd640 1 -- 172.21.15.62:0/1337661685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 msgr2=0x5654799b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff246ffd640 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 0x5654799b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff246ffd640 1 -- 172.21.15.62:0/1337661685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799ae780 msgr2=0x5654799b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff246ffd640 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799ae780 0x5654799b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff246ffd640 1 -- 172.21.15.62:0/1337661685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654799b2cc0 con 0x56547999f5d0 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.475+0000 7ff2477fe640 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799ae780 0x5654799b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff246ffd640 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56547999f5d0 0x5654799b6e10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff2380314e0 tx=0x7ff238005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cc91eb49d0731e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:00.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff22e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1337661685 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238035040 con 0x56547999f5d0 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1337661685 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7ff238031b10 con 0x56547999f5d0 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff24f785740 1 -- 172.21.15.62:0/1337661685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff240023070 con 0x56547999f5d0 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1337661685 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff238055050 con 0x56547999f5d0 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff24f785740 10 client.18240 fetch_fsmap learned FSMap version 41 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff24f785740 1 -- 172.21.15.62:0/1337661685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x565479a28b50 con 0x56547999f5d0 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1337661685 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff240023070 con 0x56547999f5d0 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1337661685 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7ff2380035f0 con 0x56547999f5d0 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff24f785740 10 client.18240 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff24f785740 10 client.18240 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff24f785740 20 client.18240 populate_metadata read hostname 'smithi062' 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff24f785740 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x565479a29300 0x565479a2b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:00.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff24f785740 4 client.18240 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.476+0000 7ff24f785740 1 -- 172.21.15.62:0/1337661685 --> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56547990cd70 con 0x565479a29300 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff23ffff640 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x565479a29300 0x565479a2b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff23ffff640 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x565479a29300 0x565479a2b720 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=409ee3e47c8cd4c9 server_cookie=17f34711e9b11b6c in_seq=0 out_seq=0 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff22e7fc640 10 client.18240 ms_handle_connect on v2:172.21.15.120:6838/2283411896 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff22e7fc640 1 -- 172.21.15.62:0/1337661685 <== mds.1 v2:172.21.15.120:6838/2283411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56547990cd70 con 0x565479a29300 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff22e7fc640 10 client.18240 handle_command_reply: tid=0 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff24f785740 1 client.18240 shutdown 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff24f785740 1 -- 172.21.15.62:0/1337661685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56547999f5d0 msgr2=0x5654799b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff24f785740 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56547999f5d0 0x5654799b6e10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff2380314e0 tx=0x7ff238005850 comp rx=0 tx=0).stop 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff24f785740 1 -- 172.21.15.62:0/1337661685 shutdown_connections 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff24f785740 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.120:6838/2283411896,v1:172.21.15.120:6839/2283411896] conn(0x565479a29300 0x565479a2b720 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff24f785740 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654799ae780 0x5654799b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff24f785740 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654799a0b40 0x5654799b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff24f785740 1 --2- 172.21.15.62:0/1337661685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56547999f5d0 0x5654799b6e10 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff22e7fc640 0 client.18240 ms_handle_reset on v2:172.21.15.120:6838/2283411896 2024-09-12T00:08:00.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.477+0000 7ff24f785740 1 -- 172.21.15.62:0/1337661685 >> 172.21.15.62:0/1337661685 conn(0x565479968310 msgr2=0x5654799aa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:00.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24f785740 1 -- 172.21.15.62:0/1337661685 shutdown_connections 2024-09-12T00:08:00.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24f785740 1 -- 172.21.15.62:0/1337661685 wait complete. 2024-09-12T00:08:00.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24f785740 20 client.18240 trim_cache size 0 max 16384 2024-09-12T00:08:00.493 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:08:00.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 -- 172.21.15.62:0/111093462 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2340a2cf0 msgr2=0x7ff2340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:00.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2340a2cf0 0x7ff2340a51b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff240022000 tx=0x7ff240020000 comp rx=0 tx=0).stop 2024-09-12T00:08:00.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 -- 172.21.15.62:0/111093462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff248102570 msgr2=0x7ff2481989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:00.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff248102570 0x7ff2481989b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff23000e9f0 tx=0x7ff23000eeb0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 -- 172.21.15.62:0/111093462 shutdown_connections 2024-09-12T00:08:00.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7ff2340a2cf0 0x7ff2340a51b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff248102ef0 0x7ff24819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff248102570 0x7ff2481989b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 --2- 172.21.15.62:0/111093462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff248100e20 0x7ff248198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:00.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 -- 172.21.15.62:0/111093462 >> 172.21.15.62:0/111093462 conn(0x7ff2480fc7d0 msgr2=0x7ff24806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:00.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 -- 172.21.15.62:0/111093462 shutdown_connections 2024-09-12T00:08:00.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:00.478+0000 7ff24d3ac640 1 -- 172.21.15.62:0/111093462 wait complete. 2024-09-12T00:08:01.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:00 smithi120 ceph-mon[35182]: pgmap v496: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-09-12T00:08:01.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:00 smithi062 ceph-mon[30958]: pgmap v496: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-09-12T00:08:01.161 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:08:01.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:00 smithi083 ceph-mon[34805]: pgmap v496: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-09-12T00:08:01.417 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:01.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.937+0000 7f6c315d3640 1 -- 172.21.15.62:0/3080888747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c106f10 msgr2=0x7f6c2c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:01.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.937+0000 7f6c315d3640 1 --2- 172.21.15.62:0/3080888747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c106f10 0x7f6c2c10ea90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009ca0 tx=0x7f6c2402a8c0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.937+0000 7f6c315d3640 1 -- 172.21.15.62:0/3080888747 shutdown_connections 2024-09-12T00:08:01.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.937+0000 7f6c315d3640 1 --2- 172.21.15.62:0/3080888747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c106f10 0x7f6c2c10ea90 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.937+0000 7f6c315d3640 1 --2- 172.21.15.62:0/3080888747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c2c102790 0x7f6c2c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.937+0000 7f6c315d3640 1 --2- 172.21.15.62:0/3080888747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c2c101de0 0x7f6c2c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.937+0000 7f6c315d3640 1 -- 172.21.15.62:0/3080888747 >> 172.21.15.62:0/3080888747 conn(0x7f6c2c0fd8c0 msgr2=0x7f6c2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.937+0000 7f6c315d3640 1 -- 172.21.15.62:0/3080888747 shutdown_connections 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.937+0000 7f6c315d3640 1 -- 172.21.15.62:0/3080888747 wait complete. 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c315d3640 1 Processor -- start 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c315d3640 1 -- start start 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c315d3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c2c101de0 0x7f6c2c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c315d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c102790 0x7f6c2c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c315d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c2c106f10 0x7f6c2c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c315d3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c111f90 con 0x7f6c2c102790 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c315d3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c111e10 con 0x7f6c2c106f10 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c315d3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c112110 con 0x7f6c2c101de0 2024-09-12T00:08:01.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c227fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c102790 0x7f6c2c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c227fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c102790 0x7f6c2c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46496/0 (socket says 172.21.15.62:46496) 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c227fc640 1 -- 172.21.15.62:0/599458545 learned_addr learned my addr 172.21.15.62:0/599458545 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.938+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c2c106f10 0x7f6c2c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c2affd640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c2c101de0 0x7f6c2c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c227fc640 1 -- 172.21.15.62:0/599458545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c2c101de0 msgr2=0x7f6c2c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c227fc640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c2c101de0 0x7f6c2c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c227fc640 1 -- 172.21.15.62:0/599458545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c2c106f10 msgr2=0x7f6c2c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c227fc640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c2c106f10 0x7f6c2c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c227fc640 1 -- 172.21.15.62:0/599458545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c19f620 con 0x7f6c2c102790 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c227fc640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c102790 0x7f6c2c19abb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400b4a0 tx=0x7f6c1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:01.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c28ff9640 1 -- 172.21.15.62:0/599458545 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14014070 con 0x7f6c2c102790 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c315d3640 1 -- 172.21.15.62:0/599458545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c19f910 con 0x7f6c2c102790 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c28ff9640 1 -- 172.21.15.62:0/599458545 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c14010af0 con 0x7f6c2c102790 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c28ff9640 1 -- 172.21.15.62:0/599458545 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14017d10 con 0x7f6c2c102790 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.939+0000 7f6c315d3640 1 -- 172.21.15.62:0/599458545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c1a71f0 con 0x7f6c2c102790 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.941+0000 7f6c28ff9640 1 -- 172.21.15.62:0/599458545 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6c14017070 con 0x7f6c2c102790 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c339ac740 1 Processor -- start 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c339ac740 1 -- start start 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122247b40 0x55e122246090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c28ff9640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6c180a2cf0 0x7f6c180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1222465d0 0x55e122246a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e122253760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c339ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e1221ac2f0 con 0x55e122247b40 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c339ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e122153cf0 con 0x55e122251370 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c339ac740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e1221a82b0 con 0x55e1222465d0 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c2affd640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6c180a2cf0 0x7f6c180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c2b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122247b40 0x55e122246090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c2b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122247b40 0x55e122246090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46504/0 (socket says 172.21.15.62:46504) 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c2b7fe640 1 -- 172.21.15.62:0/594362916 learned_addr learned my addr 172.21.15.62:0/594362916 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.942+0000 7f6c28ff9640 1 -- 172.21.15.62:0/599458545 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f6c140ce660 con 0x7f6c2c102790 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c227fc640 1 --2- 172.21.15.62:0/594362916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1222465d0 0x55e122246a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/594362916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e122253760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c2b7fe640 1 -- 172.21.15.62:0/594362916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1222465d0 msgr2=0x55e122246a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:01.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/594362916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1222465d0 0x55e122246a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c2b7fe640 1 -- 172.21.15.62:0/594362916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 msgr2=0x55e122253760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/594362916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e122253760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c2b7fe640 1 -- 172.21.15.62:0/594362916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1221383a0 con 0x55e122247b40 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c2affd640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6c180a2cf0 0x7f6c180a51b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0058c0 tx=0x7f6c1c00a2d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/594362916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122247b40 0x55e122246090 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x55e122247070 tx=0x7f6c240093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d550d50706639452 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c217fa640 1 -- 172.21.15.62:0/594362916 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24049070 con 0x55e122247b40 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c217fa640 1 -- 172.21.15.62:0/594362916 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c24036040 con 0x55e122247b40 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.943+0000 7f6c217fa640 1 -- 172.21.15.62:0/594362916 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24004520 con 0x55e122247b40 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.944+0000 7f6c339ac740 1 -- 172.21.15.62:0/594362916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122247b40 msgr2=0x55e122246090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.944+0000 7f6c339ac740 1 --2- 172.21.15.62:0/594362916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122247b40 0x55e122246090 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x55e122247070 tx=0x7f6c240093f0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.944+0000 7f6c339ac740 1 -- 172.21.15.62:0/594362916 shutdown_connections 2024-09-12T00:08:01.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.944+0000 7f6c339ac740 1 --2- 172.21.15.62:0/594362916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e122253760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.944+0000 7f6c339ac740 1 --2- 172.21.15.62:0/594362916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1222465d0 0x55e122246a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.944+0000 7f6c339ac740 1 --2- 172.21.15.62:0/594362916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122247b40 0x55e122246090 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.944+0000 7f6c339ac740 1 -- 172.21.15.62:0/594362916 >> 172.21.15.62:0/594362916 conn(0x55e12220f310 msgr2=0x55e122242c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:01.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.944+0000 7f6c339ac740 1 -- 172.21.15.62:0/594362916 shutdown_connections 2024-09-12T00:08:01.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.944+0000 7f6c339ac740 1 -- 172.21.15.62:0/594362916 wait complete. 2024-09-12T00:08:01.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.944+0000 7f6c339ac740 1 -- start start 2024-09-12T00:08:01.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c339ac740 1 Processor -- start 2024-09-12T00:08:01.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122250dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e12224e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e12224e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c339ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e122254010 con 0x55e1222465d0 2024-09-12T00:08:01.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c339ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e122167060 con 0x55e122251370 2024-09-12T00:08:01.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c339ac740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e122254190 con 0x55e122247b40 2024-09-12T00:08:01.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c339ac740 10 client.0 _ll_register_callbacks cb 0x55e121eba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:08:01.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122250dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122250dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46512/0 (socket says 172.21.15.62:46512) 2024-09-12T00:08:01.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2affd640 1 -- 172.21.15.62:0/2386557719 learned_addr learned my addr 172.21.15.62:0/2386557719 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:01.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c227fc640 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e12224e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e12224e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2affd640 1 -- 172.21.15.62:0/2386557719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 msgr2=0x55e12224e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2affd640 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e12224e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2affd640 1 -- 172.21.15.62:0/2386557719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 msgr2=0x55e12224e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2affd640 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e12224e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2affd640 1 -- 172.21.15.62:0/2386557719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e121f1d3c0 con 0x55e1222465d0 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e12224e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c2affd640 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122250dc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c010a60 tx=0x7f6c1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62b50dad2e93ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.945+0000 7f6c127fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c127fc640 1 -- 172.21.15.62:0/2386557719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c016070 con 0x55e1222465d0 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c339ac740 1 -- 172.21.15.62:0/2386557719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c24036040 con 0x55e1222465d0 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c127fc640 1 -- 172.21.15.62:0/2386557719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f6c1c01bc40 con 0x55e1222465d0 2024-09-12T00:08:01.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c127fc640 1 -- 172.21.15.62:0/2386557719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6c1c01be40 con 0x55e1222465d0 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c339ac740 10 client.18252 fetch_fsmap learned FSMap version 41 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c339ac740 1 -- 172.21.15.62:0/2386557719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e12225fbd0 con 0x55e1222465d0 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c127fc640 1 -- 172.21.15.62:0/2386557719 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c24036040 con 0x55e1222465d0 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c127fc640 1 -- 172.21.15.62:0/2386557719 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f6c1c01c6b0 con 0x55e1222465d0 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c339ac740 10 client.18252 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c339ac740 10 client.18252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c339ac740 20 client.18252 populate_metadata read hostname 'smithi062' 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c339ac740 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e122260fe0 0x55e1222633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c339ac740 4 client.18252 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.946+0000 7f6c339ac740 1 -- 172.21.15.62:0/2386557719 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1221b9860 con 0x55e122260fe0 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.947+0000 7f6c227fc640 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e122260fe0 0x55e1222633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.947+0000 7f6c127fc640 10 client.18252 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.947+0000 7f6c227fc640 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e122260fe0 0x55e1222633d0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ade25f471b5ea2 server_cookie=6eb29992643815a4 in_seq=0 out_seq=0 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c127fc640 1 -- 172.21.15.62:0/2386557719 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1221b9860 con 0x55e122260fe0 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c127fc640 10 client.18252 handle_command_reply: tid=0 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 client.18252 shutdown 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 -- 172.21.15.62:0/2386557719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 msgr2=0x55e122250dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122250dc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c010a60 tx=0x7f6c1c004730 comp rx=0 tx=0).stop 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 -- 172.21.15.62:0/2386557719 shutdown_connections 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e122260fe0 0x55e1222633d0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e12224e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e12224e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c127fc640 0 client.18252 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:08:01.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 --2- 172.21.15.62:0/2386557719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122250dc0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 -- 172.21.15.62:0/2386557719 >> 172.21.15.62:0/2386557719 conn(0x55e12220f310 msgr2=0x55e122248960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:01.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 -- 172.21.15.62:0/2386557719 shutdown_connections 2024-09-12T00:08:01.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 1 -- 172.21.15.62:0/2386557719 wait complete. 2024-09-12T00:08:01.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.948+0000 7f6c339ac740 20 client.18252 trim_cache size 0 max 16384 2024-09-12T00:08:01.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.961+0000 7f6c339ac740 1 Processor -- start 2024-09-12T00:08:01.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.961+0000 7f6c339ac740 1 -- start start 2024-09-12T00:08:01.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1222465d0 0x55e122241c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e122249120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122251370 0x55e122249660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c339ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1b2440 con 0x55e122251370 2024-09-12T00:08:01.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c339ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1afc80 con 0x55e1222465d0 2024-09-12T00:08:01.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c339ac740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1adaf0 con 0x55e122247b40 2024-09-12T00:08:01.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122251370 0x55e122249660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122251370 0x55e122249660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46520/0 (socket says 172.21.15.62:46520) 2024-09-12T00:08:01.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2b7fe640 1 -- 172.21.15.62:0/2485059477 learned_addr learned my addr 172.21.15.62:0/2485059477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:01.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2affd640 1 --2- 172.21.15.62:0/2485059477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1222465d0 0x55e122241c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c227fc640 1 --2- 172.21.15.62:0/2485059477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e122249120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2b7fe640 1 -- 172.21.15.62:0/2485059477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 msgr2=0x55e122249120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:01.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/2485059477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e122249120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2b7fe640 1 -- 172.21.15.62:0/2485059477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1222465d0 msgr2=0x55e122241c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/2485059477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1222465d0 0x55e122241c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2b7fe640 1 -- 172.21.15.62:0/2485059477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e122263740 con 0x55e122251370 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2affd640 1 --2- 172.21.15.62:0/2485059477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1222465d0 0x55e122241c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/2485059477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122251370 0x55e122249660 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6c2402a900 tx=0x7f6c24005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398fc45e350bdd29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.962+0000 7f6c20ff9640 1 -- 172.21.15.62:0/2485059477 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24055070 con 0x55e122251370 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c20ff9640 1 -- 172.21.15.62:0/2485059477 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2404a040 con 0x55e122251370 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c20ff9640 1 -- 172.21.15.62:0/2485059477 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24049050 con 0x55e122251370 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c339ac740 1 -- 172.21.15.62:0/2485059477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122251370 msgr2=0x55e122249660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c339ac740 1 --2- 172.21.15.62:0/2485059477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122251370 0x55e122249660 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6c2402a900 tx=0x7f6c24005ef0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c339ac740 1 -- 172.21.15.62:0/2485059477 shutdown_connections 2024-09-12T00:08:01.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c339ac740 1 --2- 172.21.15.62:0/2485059477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e122251370 0x55e122249660 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c339ac740 1 --2- 172.21.15.62:0/2485059477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e122249120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c339ac740 1 --2- 172.21.15.62:0/2485059477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1222465d0 0x55e122241c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c339ac740 1 -- 172.21.15.62:0/2485059477 >> 172.21.15.62:0/2485059477 conn(0x55e12220f310 msgr2=0x55e122241500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:01.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c339ac740 1 -- 172.21.15.62:0/2485059477 shutdown_connections 2024-09-12T00:08:01.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.963+0000 7f6c339ac740 1 -- 172.21.15.62:0/2485059477 wait complete. 2024-09-12T00:08:01.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c339ac740 1 -- start start 2024-09-12T00:08:01.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c339ac740 1 Processor -- start 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122259d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e12224af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c339ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e12224b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c339ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1ae140 con 0x55e1222465d0 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c339ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1af5b0 con 0x55e122251370 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c339ac740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1ae810 con 0x55e122247b40 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c339ac740 10 client.0 _ll_register_callbacks cb 0x7f6c1c00a850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c2affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122259d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c2affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122259d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46524/0 (socket says 172.21.15.62:46524) 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c2affd640 1 -- 172.21.15.62:0/1128695505 learned_addr learned my addr 172.21.15.62:0/1128695505 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:01.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c2b7fe640 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e12224b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.964+0000 7f6c227fc640 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e12224af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c2affd640 1 -- 172.21.15.62:0/1128695505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 msgr2=0x55e12224af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c2affd640 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e12224af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c2affd640 1 -- 172.21.15.62:0/1128695505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 msgr2=0x55e12224b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c2affd640 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e12224b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c2affd640 1 -- 172.21.15.62:0/1128695505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1222cac20 con 0x55e1222465d0 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c227fc640 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e12224af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c2affd640 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122259d30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0104b0 tx=0x7f6c1c01c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de004dcc53fe5c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c127fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c127fc640 1 -- 172.21.15.62:0/1128695505 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c034050 con 0x55e1222465d0 2024-09-12T00:08:01.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c127fc640 1 -- 172.21.15.62:0/1128695505 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f6c1c027690 con 0x55e1222465d0 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c339ac740 1 -- 172.21.15.62:0/1128695505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2404a040 con 0x55e1222465d0 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c127fc640 1 -- 172.21.15.62:0/1128695505 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6c1c01c8c0 con 0x55e1222465d0 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c339ac740 10 client.18258 fetch_fsmap learned FSMap version 41 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c339ac740 1 -- 172.21.15.62:0/1128695505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e1222cbde0 con 0x55e1222465d0 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.965+0000 7f6c127fc640 1 -- 172.21.15.62:0/1128695505 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2404a040 con 0x55e1222465d0 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.966+0000 7f6c127fc640 1 -- 172.21.15.62:0/1128695505 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f6c1c03b860 con 0x55e1222465d0 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.966+0000 7f6c339ac740 10 client.18258 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.966+0000 7f6c339ac740 10 client.18258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.966+0000 7f6c339ac740 20 client.18258 populate_metadata read hostname 'smithi062' 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.966+0000 7f6c339ac740 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e1222cc580 0x55e1222ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:01.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.966+0000 7f6c339ac740 4 client.18258 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.966+0000 7f6c339ac740 1 -- 172.21.15.62:0/1128695505 --> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1221b3d70 con 0x55e1222cc580 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.966+0000 7f6c227fc640 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e1222cc580 0x55e1222ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.966+0000 7f6c127fc640 10 client.18258 ms_handle_connect on v2:172.21.15.83:6838/2530997346 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.966+0000 7f6c227fc640 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e1222cc580 0x55e1222ce9a0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738134170102925f server_cookie=d1bc1749e04d0104 in_seq=0 out_seq=0 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c127fc640 1 -- 172.21.15.62:0/1128695505 <== mds.0 v2:172.21.15.83:6838/2530997346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1221b3d70 con 0x55e1222cc580 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c127fc640 10 client.18258 handle_command_reply: tid=0 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 client.18258 shutdown 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 -- 172.21.15.62:0/1128695505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 msgr2=0x55e122259d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122259d30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0104b0 tx=0x7f6c1c01c6d0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 -- 172.21.15.62:0/1128695505 shutdown_connections 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.83:6838/2530997346,v1:172.21.15.83:6839/2530997346] conn(0x55e1222cc580 0x55e1222ce9a0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c127fc640 0 client.18258 ms_handle_reset on v2:172.21.15.83:6838/2530997346 2024-09-12T00:08:01.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e122251370 0x55e12224b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e122247b40 0x55e12224af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 --2- 172.21.15.62:0/1128695505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1222465d0 0x55e122259d30 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 -- 172.21.15.62:0/1128695505 >> 172.21.15.62:0/1128695505 conn(0x55e12220f310 msgr2=0x55e122241d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:01.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 -- 172.21.15.62:0/1128695505 shutdown_connections 2024-09-12T00:08:01.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 1 -- 172.21.15.62:0/1128695505 wait complete. 2024-09-12T00:08:01.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.967+0000 7f6c339ac740 20 client.18258 trim_cache size 0 max 16384 2024-09-12T00:08:01.978 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-09-12T00:08:01.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 -- 172.21.15.62:0/599458545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6c180a2cf0 msgr2=0x7f6c180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:01.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6c180a2cf0 0x7f6c180a51b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0058c0 tx=0x7f6c1c00a2d0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 -- 172.21.15.62:0/599458545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c102790 msgr2=0x7f6c2c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:01.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c102790 0x7f6c2c19abb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400b4a0 tx=0x7f6c1400b960 comp rx=0 tx=0).stop 2024-09-12T00:08:01.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 -- 172.21.15.62:0/599458545 shutdown_connections 2024-09-12T00:08:01.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f6c180a2cf0 0x7f6c180a51b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c2c106f10 0x7f6c2c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c102790 0x7f6c2c19abb0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 --2- 172.21.15.62:0/599458545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c2c101de0 0x7f6c2c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:01.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 -- 172.21.15.62:0/599458545 >> 172.21.15.62:0/599458545 conn(0x7f6c2c0fd8c0 msgr2=0x7f6c2c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:01.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 -- 172.21.15.62:0/599458545 shutdown_connections 2024-09-12T00:08:01.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:01.968+0000 7f6c315d3640 1 -- 172.21.15.62:0/599458545 wait complete. 2024-09-12T00:08:02.662 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-12T00:08:02.898 INFO:tasks.fwd_scrub:done joining 2024-09-12T00:08:02.898 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T00:08:02.909 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T00:08:02.910 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:02 smithi062 ceph-mon[30958]: pgmap v497: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 12 MiB/s wr, 930 op/s 2024-09-12T00:08:03.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.018+0000 7efeae282640 1 -- 172.21.15.62:0/122949580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea81069d0 msgr2=0x7efea8106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.018+0000 7efeae282640 1 --2- 172.21.15.62:0/122949580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea81069d0 0x7efea8106db0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7efe98009930 tx=0x7efe9802a420 comp rx=0 tx=0).stop 2024-09-12T00:08:03.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.018+0000 7efeae282640 1 -- 172.21.15.62:0/122949580 shutdown_connections 2024-09-12T00:08:03.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.018+0000 7efeae282640 1 --2- 172.21.15.62:0/122949580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea8101360 0x7efea810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.018+0000 7efeae282640 1 --2- 172.21.15.62:0/122949580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efea81009c0 0x7efea8100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.018+0000 7efeae282640 1 --2- 172.21.15.62:0/122949580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea81069d0 0x7efea8106db0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.018+0000 7efeae282640 1 -- 172.21.15.62:0/122949580 >> 172.21.15.62:0/122949580 conn(0x7efea80fc6a0 msgr2=0x7efea80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:03.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.018+0000 7efeae282640 1 -- 172.21.15.62:0/122949580 shutdown_connections 2024-09-12T00:08:03.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.018+0000 7efeae282640 1 -- 172.21.15.62:0/122949580 wait complete. 2024-09-12T00:08:03.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.019+0000 7efeae282640 1 Processor -- start 2024-09-12T00:08:03.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.019+0000 7efeae282640 1 -- start start 2024-09-12T00:08:03.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.019+0000 7efeae282640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efea81009c0 0x7efea8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.019+0000 7efeae282640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8101360 0x7efea81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.019+0000 7efeae282640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea81069d0 0x7efea819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.019+0000 7efeae282640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efea8111f00 con 0x7efea8101360 2024-09-12T00:08:03.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.019+0000 7efeae282640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efea8111d80 con 0x7efea81069d0 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.019+0000 7efeae282640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efea8112080 con 0x7efea81009c0 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8101360 0x7efea81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8101360 0x7efea81989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46544/0 (socket says 172.21.15.62:46544) 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea77fe640 1 -- 172.21.15.62:0/1022228159 learned_addr learned my addr 172.21.15.62:0/1022228159 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea7fff640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efea81009c0 0x7efea8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efeac81f640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea81069d0 0x7efea819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea77fe640 1 -- 172.21.15.62:0/1022228159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efea81009c0 msgr2=0x7efea8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea77fe640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efea81009c0 0x7efea8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea77fe640 1 -- 172.21.15.62:0/1022228159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea81069d0 msgr2=0x7efea819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea77fe640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea81069d0 0x7efea819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea77fe640 1 -- 172.21.15.62:0/1022228159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea819d470 con 0x7efea8101360 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea77fe640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8101360 0x7efea81989c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00e9f0 tx=0x7efe9c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea57fa640 1 -- 172.21.15.62:0/1022228159 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00cd80 con 0x7efea8101360 2024-09-12T00:08:03.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efeae282640 1 -- 172.21.15.62:0/1022228159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea819d760 con 0x7efea8101360 2024-09-12T00:08:03.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea57fa640 1 -- 172.21.15.62:0/1022228159 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe9c005500 con 0x7efea8101360 2024-09-12T00:08:03.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efea57fa640 1 -- 172.21.15.62:0/1022228159 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c010690 con 0x7efea8101360 2024-09-12T00:08:03.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.020+0000 7efeae282640 1 -- 172.21.15.62:0/1022228159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea81a4ff0 con 0x7efea8101360 2024-09-12T00:08:03.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.021+0000 7efeae282640 1 -- 172.21.15.62:0/1022228159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea8100e20 con 0x7efea8101360 2024-09-12T00:08:03.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.022+0000 7efea57fa640 1 -- 172.21.15.62:0/1022228159 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efe9c0046c0 con 0x7efea8101360 2024-09-12T00:08:03.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.023+0000 7efea57fa640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efe900a2d00 0x7efe900a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.024+0000 7efea7fff640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efe900a2d00 0x7efe900a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.024+0000 7efea57fa640 1 -- 172.21.15.62:0/1022228159 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7efe9c0cf870 con 0x7efea8101360 2024-09-12T00:08:03.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.024+0000 7efea7fff640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efe900a2d00 0x7efe900a51c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efe98009900 tx=0x7efe980058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:03.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.028+0000 7efea57fa640 1 -- 172.21.15.62:0/1022228159 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe9c09c490 con 0x7efea8101360 2024-09-12T00:08:03.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:02 smithi120 ceph-mon[35182]: pgmap v497: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 12 MiB/s wr, 930 op/s 2024-09-12T00:08:03.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:02 smithi083 ceph-mon[34805]: pgmap v497: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 12 MiB/s wr, 930 op/s 2024-09-12T00:08:03.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.207+0000 7efeae282640 1 -- 172.21.15.62:0/1022228159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efea81089a0 con 0x7efea8101360 2024-09-12T00:08:03.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.208+0000 7efea57fa640 1 -- 172.21.15.62:0/1022228159 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23087 (secure 0 0 0) 0x7efe9c09bd90 con 0x7efea8101360 2024-09-12T00:08:03.209 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:08:03.209 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":100,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-12T00:07:16.820310+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":24,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/518295197":"2024-09-12T23:52:05.426127+0000","172.21.15.62:6801/3106394779":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/908998551":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/133064408":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6800/3106394779":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2894445674":"2024-09-12T23:52:05.426127+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":24,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 -- 172.21.15.62:0/1022228159 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efe900a2d00 msgr2=0x7efe900a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efe900a2d00 0x7efe900a51c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efe98009900 tx=0x7efe980058d0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 -- 172.21.15.62:0/1022228159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8101360 msgr2=0x7efea81989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8101360 0x7efea81989c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00e9f0 tx=0x7efe9c00edf0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 -- 172.21.15.62:0/1022228159 shutdown_connections 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7efe900a2d00 0x7efe900a51c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea81069d0 0x7efea819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8101360 0x7efea81989c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 --2- 172.21.15.62:0/1022228159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efea81009c0 0x7efea8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 -- 172.21.15.62:0/1022228159 >> 172.21.15.62:0/1022228159 conn(0x7efea80fc6a0 msgr2=0x7efea80fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 -- 172.21.15.62:0/1022228159 shutdown_connections 2024-09-12T00:08:03.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.211+0000 7efeae282640 1 -- 172.21.15.62:0/1022228159 wait complete. 2024-09-12T00:08:03.222 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:45:29.220992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T23:50:07.698272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T23:50:09.813785+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T00:08:03.223 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T00:08:03.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.332+0000 7f02ccb0a640 1 -- 172.21.15.62:0/607896814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c810c0c0 msgr2=0x7f02c810e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.332+0000 7f02ccb0a640 1 --2- 172.21.15.62:0/607896814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c810c0c0 0x7f02c810e530 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009ca0 tx=0x7f02c002a950 comp rx=0 tx=0).stop 2024-09-12T00:08:03.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.332+0000 7f02ccb0a640 1 -- 172.21.15.62:0/607896814 shutdown_connections 2024-09-12T00:08:03.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.332+0000 7f02ccb0a640 1 --2- 172.21.15.62:0/607896814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c810c0c0 0x7f02c810e530 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.332+0000 7f02ccb0a640 1 --2- 172.21.15.62:0/607896814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02c8103610 0x7f02c810baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.332+0000 7f02ccb0a640 1 --2- 172.21.15.62:0/607896814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02c8102cf0 0x7f02c81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.332+0000 7f02ccb0a640 1 -- 172.21.15.62:0/607896814 >> 172.21.15.62:0/607896814 conn(0x7f02c80779d0 msgr2=0x7f02c80fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.332+0000 7f02ccb0a640 1 -- 172.21.15.62:0/607896814 shutdown_connections 2024-09-12T00:08:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.332+0000 7f02ccb0a640 1 -- 172.21.15.62:0/607896814 wait complete. 2024-09-12T00:08:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02ccb0a640 1 Processor -- start 2024-09-12T00:08:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02ccb0a640 1 -- start start 2024-09-12T00:08:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02ccb0a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c8102cf0 0x7f02c8100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02ccb0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02c8103610 0x7f02c8100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02ccb0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02c810c0c0 0x7f02c81013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02ccb0a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8111e90 con 0x7f02c8102cf0 2024-09-12T00:08:03.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02ccb0a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8111d10 con 0x7f02c8103610 2024-09-12T00:08:03.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02ccb0a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8112010 con 0x7f02c810c0c0 2024-09-12T00:08:03.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c8102cf0 0x7f02c8100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c8102cf0 0x7f02c8100940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46554/0 (socket says 172.21.15.62:46554) 2024-09-12T00:08:03.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c659c640 1 -- 172.21.15.62:0/1057836580 learned_addr learned my addr 172.21.15.62:0/1057836580 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:03.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c6d9d640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02c810c0c0 0x7f02c81013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c5d9b640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02c8103610 0x7f02c8100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c659c640 1 -- 172.21.15.62:0/1057836580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02c810c0c0 msgr2=0x7f02c81013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c659c640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02c810c0c0 0x7f02c81013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c659c640 1 -- 172.21.15.62:0/1057836580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02c8103610 msgr2=0x7f02c8100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c659c640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02c8103610 0x7f02c8100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c659c640 1 -- 172.21.15.62:0/1057836580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c81a68c0 con 0x7f02c8102cf0 2024-09-12T00:08:03.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.333+0000 7f02c5d9b640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02c8103610 0x7f02c8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:08:03.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.334+0000 7f02c659c640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c8102cf0 0x7f02c8100940 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f02b800d950 tx=0x7f02b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.334+0000 7f02b77fe640 1 -- 172.21.15.62:0/1057836580 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8017070 con 0x7f02c8102cf0 2024-09-12T00:08:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.334+0000 7f02ccb0a640 1 -- 172.21.15.62:0/1057836580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c81a6bb0 con 0x7f02c8102cf0 2024-09-12T00:08:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.334+0000 7f02b77fe640 1 -- 172.21.15.62:0/1057836580 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b8009c30 con 0x7f02c8102cf0 2024-09-12T00:08:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.334+0000 7f02b77fe640 1 -- 172.21.15.62:0/1057836580 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b800f6b0 con 0x7f02c8102cf0 2024-09-12T00:08:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.334+0000 7f02ccb0a640 1 -- 172.21.15.62:0/1057836580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c81a7150 con 0x7f02c8102cf0 2024-09-12T00:08:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.335+0000 7f02ccb0a640 1 -- 172.21.15.62:0/1057836580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c81030d0 con 0x7f02c8102cf0 2024-09-12T00:08:03.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.337+0000 7f02b77fe640 1 -- 172.21.15.62:0/1057836580 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f02b80109d0 con 0x7f02c8102cf0 2024-09-12T00:08:03.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.338+0000 7f02b77fe640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02b00a2d00 0x7f02b00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.338+0000 7f02c5d9b640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02b00a2d00 0x7f02b00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.338+0000 7f02b77fe640 1 -- 172.21.15.62:0/1057836580 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f02b80cf070 con 0x7f02c8102cf0 2024-09-12T00:08:03.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.338+0000 7f02c5d9b640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02b00a2d00 0x7f02b00a51c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f02c81023d0 tx=0x7f02bc005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:03.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.341+0000 7f02b77fe640 1 -- 172.21.15.62:0/1057836580 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02b8009de0 con 0x7f02c8102cf0 2024-09-12T00:08:03.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.518+0000 7f02ccb0a640 1 -- 172.21.15.62:0/1057836580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f02c8107170 con 0x7f02c8102cf0 2024-09-12T00:08:03.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.519+0000 7f02b77fe640 1 -- 172.21.15.62:0/1057836580 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f02b809bc90 con 0x7f02c8102cf0 2024-09-12T00:08:03.519 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 -- 172.21.15.62:0/1057836580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02b00a2d00 msgr2=0x7f02b00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02b00a2d00 0x7f02b00a51c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f02c81023d0 tx=0x7f02bc005f90 comp rx=0 tx=0).stop 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 -- 172.21.15.62:0/1057836580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c8102cf0 msgr2=0x7f02c8100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c8102cf0 0x7f02c8100940 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f02b800d950 tx=0x7f02b800de10 comp rx=0 tx=0).stop 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 -- 172.21.15.62:0/1057836580 shutdown_connections 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f02b00a2d00 0x7f02b00a51c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02c810c0c0 0x7f02c81013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02c8103610 0x7f02c8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 --2- 172.21.15.62:0/1057836580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c8102cf0 0x7f02c8100940 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 -- 172.21.15.62:0/1057836580 >> 172.21.15.62:0/1057836580 conn(0x7f02c80779d0 msgr2=0x7f02c80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:03.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 -- 172.21.15.62:0/1057836580 shutdown_connections 2024-09-12T00:08:03.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.521+0000 7f02ccb0a640 1 -- 172.21.15.62:0/1057836580 wait complete. 2024-09-12T00:08:03.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T00:08:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.641+0000 7f47ea6f1640 1 -- 172.21.15.62:0/4150265712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47e4103610 msgr2=0x7f47e410baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.641+0000 7f47ea6f1640 1 --2- 172.21.15.62:0/4150265712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47e4103610 0x7f47e410baf0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009900 tx=0x7f47d802a480 comp rx=0 tx=0).stop 2024-09-12T00:08:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.641+0000 7f47ea6f1640 1 -- 172.21.15.62:0/4150265712 shutdown_connections 2024-09-12T00:08:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.641+0000 7f47ea6f1640 1 --2- 172.21.15.62:0/4150265712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e410c0c0 0x7f47e410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.641+0000 7f47ea6f1640 1 --2- 172.21.15.62:0/4150265712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47e4103610 0x7f47e410baf0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.641+0000 7f47ea6f1640 1 --2- 172.21.15.62:0/4150265712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47e4102cf0 0x7f47e41030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.641+0000 7f47ea6f1640 1 -- 172.21.15.62:0/4150265712 >> 172.21.15.62:0/4150265712 conn(0x7f47e40779d0 msgr2=0x7f47e40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:03.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.641+0000 7f47ea6f1640 1 -- 172.21.15.62:0/4150265712 shutdown_connections 2024-09-12T00:08:03.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.641+0000 7f47ea6f1640 1 -- 172.21.15.62:0/4150265712 wait complete. 2024-09-12T00:08:03.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47ea6f1640 1 Processor -- start 2024-09-12T00:08:03.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47ea6f1640 1 -- start start 2024-09-12T00:08:03.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47ea6f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47e4102cf0 0x7f47e4100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47ea6f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47e4103610 0x7f47e4100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47ea6f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e410c0c0 0x7f47e41013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47ea6f1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4111ee0 con 0x7f47e4103610 2024-09-12T00:08:03.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47ea6f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4111d60 con 0x7f47e410c0c0 2024-09-12T00:08:03.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47ea6f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4112060 con 0x7f47e4102cf0 2024-09-12T00:08:03.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47e37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47e4103610 0x7f47e4100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47e37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47e4103610 0x7f47e4100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46570/0 (socket says 172.21.15.62:46570) 2024-09-12T00:08:03.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47e37fe640 1 -- 172.21.15.62:0/3051732801 learned_addr learned my addr 172.21.15.62:0/3051732801 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:03.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47e8c8e640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e410c0c0 0x7f47e41013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.642+0000 7f47e3fff640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47e4102cf0 0x7f47e4100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47e37fe640 1 -- 172.21.15.62:0/3051732801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47e4102cf0 msgr2=0x7f47e4100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47e37fe640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47e4102cf0 0x7f47e4100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47e37fe640 1 -- 172.21.15.62:0/3051732801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e410c0c0 msgr2=0x7f47e41013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47e37fe640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e410c0c0 0x7f47e41013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47e37fe640 1 -- 172.21.15.62:0/3051732801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e41a6890 con 0x7f47e4103610 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47e37fe640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47e4103610 0x7f47e4100e80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f47d8006010 tx=0x7f47d8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47e17fa640 1 -- 172.21.15.62:0/3051732801 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d803f070 con 0x7f47e4103610 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47e17fa640 1 -- 172.21.15.62:0/3051732801 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d8031c90 con 0x7f47e4103610 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47ea6f1640 1 -- 172.21.15.62:0/3051732801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e41a6b20 con 0x7f47e4103610 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47e17fa640 1 -- 172.21.15.62:0/3051732801 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8035030 con 0x7f47e4103610 2024-09-12T00:08:03.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.643+0000 7f47ea6f1640 1 -- 172.21.15.62:0/3051732801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e41a7010 con 0x7f47e4103610 2024-09-12T00:08:03.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.644+0000 7f47ea6f1640 1 -- 172.21.15.62:0/3051732801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e41030d0 con 0x7f47e4103610 2024-09-12T00:08:03.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.645+0000 7f47e17fa640 1 -- 172.21.15.62:0/3051732801 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f47d8042070 con 0x7f47e4103610 2024-09-12T00:08:03.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.646+0000 7f47e17fa640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f47cc0a2d00 0x7f47cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.646+0000 7f47e3fff640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f47cc0a2d00 0x7f47cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.646+0000 7f47e17fa640 1 -- 172.21.15.62:0/3051732801 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f47d80efd30 con 0x7f47e4103610 2024-09-12T00:08:03.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.647+0000 7f47e3fff640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f47cc0a2d00 0x7f47cc0a51c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f47d4005e10 tx=0x7f47d4005da0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:03.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.650+0000 7f47e17fa640 1 -- 172.21.15.62:0/3051732801 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47d80bc950 con 0x7f47e4103610 2024-09-12T00:08:03.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.831+0000 7f47ea6f1640 1 -- 172.21.15.62:0/3051732801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47e4107110 con 0x7f47e4103610 2024-09-12T00:08:03.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.832+0000 7f47e17fa640 1 -- 172.21.15.62:0/3051732801 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f47d80bc250 con 0x7f47e4103610 2024-09-12T00:08:03.831 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-09-12T00:08:03.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 -- 172.21.15.62:0/3051732801 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f47cc0a2d00 msgr2=0x7f47cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f47cc0a2d00 0x7f47cc0a51c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f47d4005e10 tx=0x7f47d4005da0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 -- 172.21.15.62:0/3051732801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47e4103610 msgr2=0x7f47e4100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47e4103610 0x7f47e4100e80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f47d8006010 tx=0x7f47d8003ba0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 -- 172.21.15.62:0/3051732801 shutdown_connections 2024-09-12T00:08:03.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f47cc0a2d00 0x7f47cc0a51c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e410c0c0 0x7f47e41013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47e4103610 0x7f47e4100e80 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 --2- 172.21.15.62:0/3051732801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47e4102cf0 0x7f47e4100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 -- 172.21.15.62:0/3051732801 >> 172.21.15.62:0/3051732801 conn(0x7f47e40779d0 msgr2=0x7f47e40fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:03.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 -- 172.21.15.62:0/3051732801 shutdown_connections 2024-09-12T00:08:03.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.834+0000 7f47ea6f1640 1 -- 172.21.15.62:0/3051732801 wait complete. 2024-09-12T00:08:03.845 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T00:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.954+0000 7fdcc561e640 1 -- 172.21.15.62:0/318365868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc0103610 msgr2=0x7fdcc010baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.954+0000 7fdcc561e640 1 --2- 172.21.15.62:0/318365868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc0103610 0x7fdcc010baf0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009900 tx=0x7fdca802a480 comp rx=0 tx=0).stop 2024-09-12T00:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.954+0000 7fdcc561e640 1 -- 172.21.15.62:0/318365868 shutdown_connections 2024-09-12T00:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.954+0000 7fdcc561e640 1 --2- 172.21.15.62:0/318365868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc010c0c0 0x7fdcc010e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.954+0000 7fdcc561e640 1 --2- 172.21.15.62:0/318365868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc0103610 0x7fdcc010baf0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.954+0000 7fdcc561e640 1 --2- 172.21.15.62:0/318365868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc0102cf0 0x7fdcc01030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.954+0000 7fdcc561e640 1 -- 172.21.15.62:0/318365868 >> 172.21.15.62:0/318365868 conn(0x7fdcc00779d0 msgr2=0x7fdcc00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.954+0000 7fdcc561e640 1 -- 172.21.15.62:0/318365868 shutdown_connections 2024-09-12T00:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.954+0000 7fdcc561e640 1 -- 172.21.15.62:0/318365868 wait complete. 2024-09-12T00:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.954+0000 7fdcc561e640 1 Processor -- start 2024-09-12T00:08:03.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcc561e640 1 -- start start 2024-09-12T00:08:03.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcc561e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc0102cf0 0x7fdcc0100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcc561e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc0103610 0x7fdcc0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcc561e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc010c0c0 0x7fdcc01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcc561e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111e90 con 0x7fdcc010c0c0 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcc561e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111d10 con 0x7fdcc0102cf0 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcc561e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0112010 con 0x7fdcc0103610 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcbf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc010c0c0 0x7fdcc01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcbf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc010c0c0 0x7fdcc01013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46600/0 (socket says 172.21.15.62:46600) 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcbf7fe640 1 -- 172.21.15.62:0/3099188770 learned_addr learned my addr 172.21.15.62:0/3099188770 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcbe7fc640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc0103610 0x7fdcc0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcbeffd640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc0102cf0 0x7fdcc0100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcbf7fe640 1 -- 172.21.15.62:0/3099188770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc0103610 msgr2=0x7fdcc0100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcbf7fe640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc0103610 0x7fdcc0100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcbf7fe640 1 -- 172.21.15.62:0/3099188770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc0102cf0 msgr2=0x7fdcc0100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcbf7fe640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc0102cf0 0x7fdcc0100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.955+0000 7fdcbf7fe640 1 -- 172.21.15.62:0/3099188770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc01a6880 con 0x7fdcc010c0c0 2024-09-12T00:08:03.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.956+0000 7fdcbf7fe640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc010c0c0 0x7fdcc01013c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c970 tx=0x7fdcb800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:03.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.956+0000 7fdcb7fff640 1 -- 172.21.15.62:0/3099188770 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8007bb0 con 0x7fdcc010c0c0 2024-09-12T00:08:03.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.956+0000 7fdcc561e640 1 -- 172.21.15.62:0/3099188770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc01a6b70 con 0x7fdcc010c0c0 2024-09-12T00:08:03.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.956+0000 7fdcb7fff640 1 -- 172.21.15.62:0/3099188770 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb8007d50 con 0x7fdcc010c0c0 2024-09-12T00:08:03.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.956+0000 7fdcb7fff640 1 -- 172.21.15.62:0/3099188770 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8018800 con 0x7fdcc010c0c0 2024-09-12T00:08:03.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.956+0000 7fdcc561e640 1 -- 172.21.15.62:0/3099188770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01a7040 con 0x7fdcc010c0c0 2024-09-12T00:08:03.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.957+0000 7fdcc561e640 1 -- 172.21.15.62:0/3099188770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc01030d0 con 0x7fdcc010c0c0 2024-09-12T00:08:03.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.958+0000 7fdcb7fff640 1 -- 172.21.15.62:0/3099188770 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdcb800fa50 con 0x7fdcc010c0c0 2024-09-12T00:08:03.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.958+0000 7fdcb7fff640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdcac0a2d00 0x7fdcac0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:03.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.959+0000 7fdcb7fff640 1 -- 172.21.15.62:0/3099188770 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fdcb80cf380 con 0x7fdcc010c0c0 2024-09-12T00:08:03.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.959+0000 7fdcbeffd640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdcac0a2d00 0x7fdcac0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:03.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.959+0000 7fdcbeffd640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdcac0a2d00 0x7fdcac0a51c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0005fd0 tx=0x7fdcb0009680 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:03.963+0000 7fdcb7fff640 1 -- 172.21.15.62:0/3099188770 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdcb8014030 con 0x7fdcc010c0c0 2024-09-12T00:08:04.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:03 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1022228159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:08:04.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:03 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1057836580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T00:08:04.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.140+0000 7fdcc561e640 1 -- 172.21.15.62:0/3099188770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdcc0107110 con 0x7fdcc010c0c0 2024-09-12T00:08:04.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.141+0000 7fdcb7fff640 1 -- 172.21.15.62:0/3099188770 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fdcb809bfa0 con 0x7fdcc010c0c0 2024-09-12T00:08:04.140 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-09-12T00:08:04.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 -- 172.21.15.62:0/3099188770 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdcac0a2d00 msgr2=0x7fdcac0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdcac0a2d00 0x7fdcac0a51c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0005fd0 tx=0x7fdcb0009680 comp rx=0 tx=0).stop 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 -- 172.21.15.62:0/3099188770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc010c0c0 msgr2=0x7fdcc01013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc010c0c0 0x7fdcc01013c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c970 tx=0x7fdcb800ce30 comp rx=0 tx=0).stop 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 -- 172.21.15.62:0/3099188770 shutdown_connections 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fdcac0a2d00 0x7fdcac0a51c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc010c0c0 0x7fdcc01013c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc0103610 0x7fdcc0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 --2- 172.21.15.62:0/3099188770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc0102cf0 0x7fdcc0100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 -- 172.21.15.62:0/3099188770 >> 172.21.15.62:0/3099188770 conn(0x7fdcc00779d0 msgr2=0x7fdcc00fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 -- 172.21.15.62:0/3099188770 shutdown_connections 2024-09-12T00:08:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.143+0000 7fdcc561e640 1 -- 172.21.15.62:0/3099188770 wait complete. 2024-09-12T00:08:04.153 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:03 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1022228159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:08:04.153 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:03 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1057836580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T00:08:04.154 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T00:08:04.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:03 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1022228159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:08:04.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:03 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1057836580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T00:08:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.259+0000 7f2c990ba640 1 -- 172.21.15.62:0/1642094570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9410c0e0 msgr2=0x7f2c9410e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.259+0000 7f2c990ba640 1 --2- 172.21.15.62:0/1642094570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9410c0e0 0x7f2c9410e530 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009ca0 tx=0x7f2c8c02a950 comp rx=0 tx=0).stop 2024-09-12T00:08:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.259+0000 7f2c990ba640 1 -- 172.21.15.62:0/1642094570 shutdown_connections 2024-09-12T00:08:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.259+0000 7f2c990ba640 1 --2- 172.21.15.62:0/1642094570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9410c0e0 0x7f2c9410e530 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.259+0000 7f2c990ba640 1 --2- 172.21.15.62:0/1642094570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c94103630 0x7f2c9410bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.259+0000 7f2c990ba640 1 --2- 172.21.15.62:0/1642094570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c94102d10 0x7f2c941030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.259+0000 7f2c990ba640 1 -- 172.21.15.62:0/1642094570 >> 172.21.15.62:0/1642094570 conn(0x7f2c94077a10 msgr2=0x7f2c940fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.259+0000 7f2c990ba640 1 -- 172.21.15.62:0/1642094570 shutdown_connections 2024-09-12T00:08:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.259+0000 7f2c990ba640 1 -- 172.21.15.62:0/1642094570 wait complete. 2024-09-12T00:08:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c990ba640 1 Processor -- start 2024-09-12T00:08:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c990ba640 1 -- start start 2024-09-12T00:08:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c990ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c94102d10 0x7f2c941982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c990ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c94103630 0x7f2c94198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c990ba640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9410c0e0 0x7f2c9419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c990ba640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94111e40 con 0x7f2c9410c0e0 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c990ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94111cc0 con 0x7f2c94103630 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c990ba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94111fc0 con 0x7f2c94102d10 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c9359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9410c0e0 0x7f2c9419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c9359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9410c0e0 0x7f2c9419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46620/0 (socket says 172.21.15.62:46620) 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c9359e640 1 -- 172.21.15.62:0/131189852 learned_addr learned my addr 172.21.15.62:0/131189852 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c92d9d640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c94102d10 0x7f2c941982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.260+0000 7f2c9259c640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c94103630 0x7f2c94198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c9359e640 1 -- 172.21.15.62:0/131189852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c94102d10 msgr2=0x7f2c941982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c9359e640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c94102d10 0x7f2c941982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c9359e640 1 -- 172.21.15.62:0/131189852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c94103630 msgr2=0x7f2c94198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c9359e640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c94103630 0x7f2c94198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c9359e640 1 -- 172.21.15.62:0/131189852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9419d2e0 con 0x7f2c9410c0e0 2024-09-12T00:08:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c9359e640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9410c0e0 0x7f2c9419cbb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c02ae70 tx=0x7f2c8c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c7bfff640 1 -- 172.21.15.62:0/131189852 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c034040 con 0x7f2c9410c0e0 2024-09-12T00:08:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c990ba640 1 -- 172.21.15.62:0/131189852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9419d570 con 0x7f2c9410c0e0 2024-09-12T00:08:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c7bfff640 1 -- 172.21.15.62:0/131189852 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c8c030920 con 0x7f2c9410c0e0 2024-09-12T00:08:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c7bfff640 1 -- 172.21.15.62:0/131189852 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c004820 con 0x7f2c9410c0e0 2024-09-12T00:08:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.261+0000 7f2c990ba640 1 -- 172.21.15.62:0/131189852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c941a4e60 con 0x7f2c9410c0e0 2024-09-12T00:08:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.263+0000 7f2c990ba640 1 -- 172.21.15.62:0/131189852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c941030f0 con 0x7f2c9410c0e0 2024-09-12T00:08:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.264+0000 7f2c7bfff640 1 -- 172.21.15.62:0/131189852 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2c8c037020 con 0x7f2c9410c0e0 2024-09-12T00:08:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.264+0000 7f2c7bfff640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2c7c0a2d00 0x7f2c7c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.264+0000 7f2c92d9d640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2c7c0a2d00 0x7f2c7c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.264+0000 7f2c7bfff640 1 -- 172.21.15.62:0/131189852 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f2c8c0f01c0 con 0x7f2c9410c0e0 2024-09-12T00:08:04.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.265+0000 7f2c92d9d640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2c7c0a2d00 0x7f2c7c0a51c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400abe0 tx=0x7f2c8400a2d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:04.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.268+0000 7f2c7bfff640 1 -- 172.21.15.62:0/131189852 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c8c0bcef0 con 0x7f2c9410c0e0 2024-09-12T00:08:04.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.537+0000 7f2c990ba640 1 -- 172.21.15.62:0/131189852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f2c941a5030 con 0x7f2c9410c0e0 2024-09-12T00:08:04.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.551+0000 7f2c7bfff640 1 -- 172.21.15.62:0/131189852 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740902 (secure 0 0 0) 0x7f2c8c0bc7f0 con 0x7f2c9410c0e0 2024-09-12T00:08:04.562 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:08:04.562 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:08:04.562 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 25, 2024-09-12T00:08:04.563 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.563 INFO:teuthology.orchestra.run.smithi062.stdout: "active_gid": 15141, 2024-09-12T00:08:04.563 INFO:teuthology.orchestra.run.smithi062.stdout: "active_name": "x", 2024-09-12T00:08:04.563 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addrs": { 2024-09-12T00:08:04.563 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-09-12T00:08:04.563 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.563 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-09-12T00:08:04.563 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6800", 2024-09-12T00:08:04.563 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 3798569441 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6801", 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 3798569441 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addr": "172.21.15.62:6801/3798569441", 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: "active_change": "2024-09-11T23:52:05.426329+0000", 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T00:08:04.564 INFO:teuthology.orchestra.run.smithi062.stdout: "available": true, 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: "standbys": [ 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 24826, 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "z", 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540701547738038271, 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.565 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:04.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:04.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.566 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.566 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.566 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:04.566 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.566 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.566 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.566 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.567 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.568 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.569 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.570 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.571 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.572 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-09-12T00:08:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.574 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.575 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.576 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.577 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.577 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-09-12T00:08:04.577 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.577 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.577 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.577 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.577 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-09-12T00:08:04.577 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-09-12T00:08:04.577 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.577 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.578 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.578 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-09-12T00:08:04.578 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.578 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.578 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.578 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-09-12T00:08:04.578 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.578 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.578 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.579 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.580 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.581 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T00:08:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T00:08:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T00:08:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T00:08:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-09-12T00:08:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-09-12T00:08:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-09-12T00:08:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-09-12T00:08:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-09-12T00:08:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-09-12T00:08:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.595 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.596 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-09-12T00:08:04.597 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.598 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-09-12T00:08:04.599 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-09-12T00:08:04.600 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.600 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-09-12T00:08:04.600 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.600 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.600 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.600 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.600 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-09-12T00:08:04.600 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-09-12T00:08:04.600 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.600 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.601 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.602 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T00:08:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T00:08:04.604 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:04.605 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-09-12T00:08:04.606 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-09-12T00:08:04.607 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.608 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T00:08:04.609 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.610 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.611 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.612 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-09-12T00:08:04.613 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-09-12T00:08:04.614 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-09-12T00:08:04.615 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.616 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.618 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.619 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:04.620 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.621 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T00:08:04.622 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T00:08:04.623 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.624 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.625 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.626 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.627 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.628 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-09-12T00:08:04.629 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-09-12T00:08:04.630 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-09-12T00:08:04.631 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.633 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-09-12T00:08:04.633 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-09-12T00:08:04.633 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.633 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.633 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.633 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T00:08:04.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.636 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.637 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.638 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.639 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-09-12T00:08:04.640 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-09-12T00:08:04.641 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.642 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.644 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.645 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.646 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.647 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.648 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.649 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-09-12T00:08:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.651 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.652 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.653 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.654 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.655 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.656 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-09-12T00:08:04.657 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-09-12T00:08:04.658 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-09-12T00:08:04.659 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-09-12T00:08:04.660 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.661 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.662 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.663 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.664 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.665 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.666 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.667 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.668 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.669 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-09-12T00:08:04.670 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-09-12T00:08:04.671 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.672 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.673 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.674 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.676 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.678 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.679 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.680 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.681 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.682 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.683 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.684 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.685 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.686 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.687 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T00:08:04.688 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T00:08:04.689 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T00:08:04.690 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.691 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.692 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.693 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.694 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.695 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.696 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.697 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.698 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.699 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.700 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.701 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.702 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-09-12T00:08:04.703 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.704 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.705 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.706 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T00:08:04.707 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T00:08:04.708 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.709 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T00:08:04.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T00:08:04.710 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.710 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.710 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.710 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.710 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.710 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.710 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.710 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.710 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.711 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.711 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.711 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T00:08:04.711 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T00:08:04.711 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.711 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T00:08:04.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T00:08:04.713 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.714 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.715 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.720 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.722 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.723 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.724 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-09-12T00:08:04.725 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.726 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.727 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.728 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.729 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.730 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.731 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.732 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-09-12T00:08:04.733 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.734 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.735 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.736 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.737 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.738 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.739 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.740 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.741 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.742 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.745 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.746 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.747 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.748 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-09-12T00:08:04.749 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.750 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.751 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.752 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.753 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.754 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.755 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.757 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-09-12T00:08:04.758 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-09-12T00:08:04.759 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.760 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.761 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.762 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T00:08:04.763 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.764 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.765 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.766 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.767 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.768 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.769 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.770 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.771 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.772 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.773 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.774 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.775 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-09-12T00:08:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.777 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.780 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.781 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.782 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.783 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.785 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.786 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-09-12T00:08:04.788 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.789 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.790 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-09-12T00:08:04.791 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.792 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.793 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.794 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.795 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.796 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.797 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.798 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.799 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.800 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.801 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.802 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.803 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.804 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.805 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.806 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.807 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.808 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.809 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.810 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.811 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.812 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.813 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.814 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.815 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.816 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.817 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.818 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.819 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.820 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.821 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T00:08:04.822 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.823 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.824 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.825 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.826 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.827 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.828 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.829 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.830 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.831 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.832 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.834 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.835 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.836 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.837 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.838 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.839 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.840 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.841 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.842 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.843 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.844 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.845 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.846 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-09-12T00:08:04.847 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-09-12T00:08:04.848 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.849 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.850 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.851 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.852 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.853 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.855 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.856 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.857 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.858 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.859 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.860 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.861 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.862 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.863 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.864 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.865 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.866 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-09-12T00:08:04.867 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-09-12T00:08:04.868 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-09-12T00:08:04.869 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.870 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.871 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.872 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.873 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.874 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.875 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.876 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.877 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.878 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.879 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.880 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.881 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.882 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.883 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.885 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.887 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.889 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.890 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.891 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.892 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.893 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.894 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.895 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.896 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.897 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.898 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.899 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.900 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.901 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.902 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-09-12T00:08:04.903 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.904 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.905 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.906 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.906 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.906 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.906 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.906 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.906 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.906 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.906 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.906 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.907 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.908 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.909 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.910 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-09-12T00:08:04.911 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.912 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.915 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.916 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.917 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-09-12T00:08:04.919 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.920 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.921 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.922 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.923 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.924 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.925 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-09-12T00:08:04.926 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.927 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.928 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.929 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.930 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.931 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.932 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.933 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.934 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.935 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:04.936 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.937 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-09-12T00:08:04.938 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-09-12T00:08:04.939 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-09-12T00:08:04.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-09-12T00:08:04.941 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.942 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.943 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.944 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.945 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.946 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.947 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.948 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.949 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.950 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.951 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.952 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.953 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.954 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.957 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.958 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.959 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.960 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.961 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.962 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.963 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.964 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.965 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.966 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.967 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.968 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.969 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.971 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.972 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.973 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T00:08:04.974 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.975 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.976 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.978 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.979 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.980 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.981 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:04.982 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.983 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.984 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.985 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.986 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.987 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.988 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.989 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.990 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.991 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.992 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.993 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.994 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:04.995 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.996 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:04.997 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:04.998 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:04.999 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.000 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.003 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.004 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.005 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.006 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T00:08:05.007 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.008 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.009 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.011 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.012 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.013 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.014 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.015 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.016 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.017 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.018 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.019 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.020 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.021 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.022 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.023 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 24830, 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "y", 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540701547738038271, 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.024 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.025 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.026 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.027 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.028 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.029 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.030 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.031 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.032 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.033 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.034 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.035 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.036 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.037 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.038 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.040 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T00:08:05.041 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.042 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.043 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.044 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-09-12T00:08:05.045 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.046 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.048 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.049 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.050 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.051 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.052 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-09-12T00:08:05.053 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-09-12T00:08:05.054 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-09-12T00:08:05.055 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.056 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.058 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.059 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-09-12T00:08:05.060 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T00:08:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.062 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.063 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.064 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.065 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.066 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.067 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.068 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.069 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T00:08:05.070 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T00:08:05.071 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.072 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.073 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.074 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.075 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.076 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.077 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.078 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.079 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.080 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-09-12T00:08:05.081 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-09-12T00:08:05.082 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-09-12T00:08:05.083 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.084 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.085 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.086 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.087 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.088 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.089 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T00:08:05.090 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.091 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.092 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.093 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.094 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.095 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.096 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.097 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.098 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.100 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.101 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.102 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.104 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.105 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T00:08:05.106 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.107 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.109 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.110 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.111 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.112 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.115 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.116 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.117 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.118 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.119 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.120 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.121 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.122 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.123 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.124 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.125 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.126 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.127 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.128 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.129 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.130 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.131 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.132 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.133 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-09-12T00:08:05.134 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.135 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.136 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.137 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.138 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.139 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.140 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.141 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.142 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.143 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.145 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.147 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.148 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.149 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-09-12T00:08:05.150 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.151 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.152 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.153 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.154 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.155 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.156 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T00:08:05.157 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T00:08:05.158 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T00:08:05.159 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T00:08:05.160 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T00:08:05.161 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.162 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.163 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.164 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.165 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.166 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.167 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.168 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.169 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.170 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.171 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.172 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-09-12T00:08:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.175 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.176 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.177 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.178 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.179 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.180 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.181 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.182 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-09-12T00:08:05.183 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-09-12T00:08:05.184 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.185 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.186 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.187 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.188 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.189 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.190 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.191 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.192 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.193 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.194 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.195 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.196 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.197 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.198 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.199 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.200 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-09-12T00:08:05.201 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.202 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.203 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.204 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.205 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.206 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.208 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.209 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.210 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.211 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.212 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.213 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.217 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.218 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.219 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.220 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.221 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.222 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-09-12T00:08:05.223 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.224 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.225 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.226 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.227 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.228 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.229 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.230 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.231 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.232 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.233 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.234 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.235 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:05.236 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.237 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.238 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.239 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.240 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.241 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.242 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.243 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.244 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.245 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.246 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.247 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.248 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.249 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.250 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.251 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.252 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.253 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.254 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.255 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.256 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.256 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.256 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.256 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.256 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.257 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.258 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.259 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.260 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.261 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.262 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.263 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.264 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.265 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.266 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.267 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.268 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.269 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-09-12T00:08:05.269 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.269 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.269 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.269 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.269 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-09-12T00:08:05.269 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-09-12T00:08:05.269 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.269 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.269 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.270 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-09-12T00:08:05.271 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.272 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.273 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.274 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.275 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.276 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.277 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.278 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.279 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.280 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.281 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.282 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.283 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.284 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.285 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.286 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.287 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.288 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.289 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.290 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.291 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.296 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.297 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.298 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.299 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.300 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.301 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.303 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.304 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.305 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.306 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.307 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.308 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.310 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.311 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.312 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.313 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.314 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.315 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.316 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.318 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.319 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.320 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.321 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.322 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.323 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.324 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.325 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.326 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-09-12T00:08:05.327 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-09-12T00:08:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.329 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.330 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.332 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.333 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.334 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.335 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.337 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.340 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.341 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.342 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-09-12T00:08:05.343 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.344 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.345 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.346 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.348 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.349 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.350 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.351 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-09-12T00:08:05.352 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.353 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.354 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.355 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.356 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.357 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.359 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.361 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.362 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.363 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.364 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.365 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.366 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.367 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.368 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.369 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.370 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.371 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.372 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.373 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-09-12T00:08:05.374 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.375 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.376 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.377 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.378 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.379 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.380 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-09-12T00:08:05.381 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.382 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.383 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.384 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.385 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.386 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.387 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-09-12T00:08:05.388 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-09-12T00:08:05.389 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.390 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.391 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.392 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.393 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.394 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.395 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.396 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.397 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.398 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.399 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.399 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.399 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.399 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.399 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.399 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.399 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.399 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.399 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.399 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.400 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.401 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.402 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.403 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.404 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.405 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.405 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.405 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.405 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.405 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.405 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.406 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.406 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.406 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.406 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.406 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.406 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.406 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.407 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-09-12T00:08:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.409 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.410 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.411 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.412 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.412 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.412 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.412 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.412 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.412 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.412 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.413 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.413 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.413 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.413 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.413 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.413 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.413 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.414 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.414 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.414 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.414 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.414 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.414 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.415 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.415 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.415 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.415 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.415 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.415 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.416 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.416 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.416 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.416 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.416 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.416 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.416 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.417 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.417 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.417 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.417 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.417 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.417 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-09-12T00:08:05.418 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.418 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.418 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.418 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-09-12T00:08:05.418 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-09-12T00:08:05.418 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.419 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.419 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.419 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T00:08:05.419 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.419 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.419 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.420 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.420 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.420 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.420 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.420 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.421 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.421 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.421 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.421 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.421 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.421 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-09-12T00:08:05.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.422 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.422 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.422 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.422 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.422 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.422 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.423 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.424 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.424 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.424 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.424 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.424 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.425 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.425 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.425 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.425 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.425 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.425 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.428 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.429 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.430 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.431 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.432 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.433 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.434 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.435 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.436 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-09-12T00:08:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-09-12T00:08:05.438 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.440 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.441 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.442 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.443 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.444 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.445 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.446 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.448 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.449 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.450 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.451 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.452 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.453 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.454 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.458 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.459 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.460 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.461 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.462 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.463 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.464 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.465 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.466 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-09-12T00:08:05.467 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.468 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.469 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.470 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.471 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.472 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.473 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.474 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.475 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.476 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.477 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.478 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.479 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-09-12T00:08:05.480 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-09-12T00:08:05.481 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.482 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.483 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: "modules": [ 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard", 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: "iostat", 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: "nfs", 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: "restful", 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: "snap_schedule" 2024-09-12T00:08:05.484 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.485 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.486 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.487 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.488 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.490 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-09-12T00:08:05.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.492 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.493 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-09-12T00:08:05.497 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T00:08:05.498 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.499 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-09-12T00:08:05.500 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-09-12T00:08:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-09-12T00:08:05.502 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.503 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.504 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T00:08:05.505 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-09-12T00:08:05.506 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-09-12T00:08:05.508 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.509 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.510 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.511 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.512 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.513 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.514 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-09-12T00:08:05.515 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.516 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.517 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.518 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.519 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.520 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.521 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.522 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.523 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.524 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.525 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.526 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.527 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-09-12T00:08:05.528 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.529 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.530 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.531 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T00:08:05.532 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.533 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.535 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.536 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-09-12T00:08:05.537 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-09-12T00:08:05.538 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.539 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.540 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-09-12T00:08:05.541 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.542 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.543 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.544 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.545 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T00:08:05.546 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T00:08:05.547 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.548 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.549 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.550 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-09-12T00:08:05.551 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-09-12T00:08:05.552 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.553 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.554 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.555 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T00:08:05.556 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.557 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.558 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.559 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.560 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.561 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.562 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.563 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.565 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.566 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T00:08:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.569 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.570 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.571 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.572 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-09-12T00:08:05.573 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-09-12T00:08:05.574 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.575 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T00:08:05.576 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T00:08:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-09-12T00:08:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-09-12T00:08:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T00:08:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.593 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.594 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.595 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-09-12T00:08:05.597 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.598 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.599 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.600 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.601 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.602 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.603 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.604 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.605 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.606 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.607 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T00:08:05.608 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T00:08:05.609 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T00:08:05.610 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.612 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.613 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.614 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.615 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.616 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.617 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.618 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.619 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T00:08:05.621 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T00:08:05.622 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.623 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.624 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.625 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.626 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.627 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.628 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.629 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.630 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.631 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.632 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.633 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.634 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T00:08:05.635 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.636 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.639 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.640 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.641 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.642 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T00:08:05.643 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.644 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.645 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.646 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.647 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-09-12T00:08:05.648 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.649 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.650 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.651 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.652 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.653 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.654 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-09-12T00:08:05.656 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.657 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.658 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.659 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.660 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.661 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.662 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.663 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.664 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.665 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-09-12T00:08:05.667 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.668 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.669 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.670 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.672 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-09-12T00:08:05.673 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.674 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.676 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.677 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T00:08:05.678 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.679 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.680 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.681 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.682 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.683 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.685 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.686 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.687 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.688 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-09-12T00:08:05.688 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-09-12T00:08:05.688 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.688 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.688 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.688 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-09-12T00:08:05.688 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.689 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.689 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.689 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.689 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.689 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.689 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.689 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-09-12T00:08:05.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-09-12T00:08:05.689 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.690 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-09-12T00:08:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-09-12T00:08:05.692 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.693 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-09-12T00:08:05.693 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-09-12T00:08:05.693 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.693 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.693 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.693 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-09-12T00:08:05.693 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.693 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.693 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.693 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T00:08:05.694 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.694 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.694 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.694 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.694 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-09-12T00:08:05.694 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-09-12T00:08:05.694 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.694 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.694 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-09-12T00:08:05.695 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.695 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.695 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T00:08:05.695 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.695 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.695 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.695 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.695 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-09-12T00:08:05.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-09-12T00:08:05.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.696 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.696 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.696 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.696 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.696 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.696 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.696 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-09-12T00:08:05.696 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.697 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.698 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.699 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.701 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.702 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.703 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-09-12T00:08:05.704 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-09-12T00:08:05.705 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-09-12T00:08:05.706 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.707 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.709 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.710 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.711 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.712 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.713 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.715 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.716 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.717 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.718 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.719 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.720 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.721 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-09-12T00:08:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.723 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.724 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.725 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.725 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.725 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.725 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.725 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.725 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.725 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.725 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.725 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.726 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.727 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.728 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.729 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.730 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.730 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.730 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.730 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.730 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.730 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.730 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.730 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.730 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.731 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.732 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.733 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.734 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.735 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.736 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-09-12T00:08:05.737 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-09-12T00:08:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-09-12T00:08:05.739 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.740 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.741 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-09-12T00:08:05.741 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-09-12T00:08:05.741 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.741 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.741 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.741 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.741 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.742 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-09-12T00:08:05.743 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.743 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.743 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.743 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T00:08:05.743 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T00:08:05.743 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.743 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.743 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.743 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.743 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.744 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.744 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-09-12T00:08:05.744 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.744 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.744 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.744 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.744 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.744 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.744 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.744 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.745 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.746 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.747 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.747 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.747 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.747 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.747 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.747 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.747 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.747 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.748 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.748 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.748 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.748 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.748 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.748 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.748 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.748 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.748 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.750 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.750 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.750 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.750 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.750 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.750 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.750 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.750 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.750 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.750 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.751 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.752 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.752 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.752 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.752 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.752 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.752 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.752 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.752 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.752 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.753 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.753 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.753 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.753 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.753 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.753 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.753 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.753 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.753 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.753 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.754 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.755 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.755 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.755 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.755 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.755 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.755 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.755 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.756 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.757 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.758 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.759 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.760 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.761 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.762 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.763 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-09-12T00:08:05.764 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.766 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.766 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.766 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.766 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.766 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.766 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.766 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.766 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.766 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.767 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.767 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.767 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.767 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.767 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.767 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.767 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.768 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.768 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.768 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.768 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.768 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.769 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.769 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.769 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.769 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.769 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.769 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.770 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.770 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.770 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.770 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.770 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.771 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.771 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.771 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.771 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.771 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.771 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.771 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.771 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.771 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.772 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.773 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.774 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.774 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.774 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.774 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-09-12T00:08:05.774 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-09-12T00:08:05.774 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.774 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.774 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.774 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.774 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.775 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.776 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-09-12T00:08:05.776 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.776 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.776 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.776 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.776 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.776 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.777 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.778 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.779 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.780 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.780 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.780 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.780 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.780 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.780 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.781 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.781 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.781 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.781 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.781 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.781 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.781 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.781 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.781 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.781 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.782 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.783 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.784 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.785 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.786 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.787 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.790 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.791 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.792 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.793 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.794 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.795 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-09-12T00:08:05.796 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-09-12T00:08:05.797 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.798 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.799 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.800 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.801 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.803 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.804 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-09-12T00:08:05.805 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.806 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-09-12T00:08:05.807 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.808 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.809 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.810 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.811 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.814 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.816 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-09-12T00:08:05.817 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T00:08:05.818 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-09-12T00:08:05.819 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-09-12T00:08:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-09-12T00:08:05.821 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-09-12T00:08:05.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-09-12T00:08:05.823 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.824 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.826 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.827 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.828 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.829 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.830 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.830 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.830 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.830 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.830 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.830 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.830 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.830 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.831 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.832 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.833 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.834 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.835 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.836 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.837 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.839 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.840 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.841 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.841 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.841 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.841 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.841 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.841 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.841 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.841 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.841 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.842 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.843 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.844 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.845 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.846 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.847 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.848 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.849 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.850 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.851 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.852 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.854 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.855 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.856 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.857 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.858 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.859 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.860 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.861 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.862 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.863 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.864 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-09-12T00:08:05.865 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-09-12T00:08:05.866 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-09-12T00:08:05.867 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.868 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.869 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.870 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.872 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.873 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.874 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.875 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.876 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.876 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.876 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.876 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.876 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.876 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.876 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.877 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-09-12T00:08:05.877 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-09-12T00:08:05.877 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.877 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.877 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.877 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.877 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-09-12T00:08:05.879 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.880 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.880 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.880 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-09-12T00:08:05.880 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-09-12T00:08:05.880 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.880 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.880 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-09-12T00:08:05.881 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.882 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.883 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.884 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.885 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.887 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.888 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.889 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.890 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.891 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.892 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.893 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.894 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.895 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.896 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.897 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.898 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.899 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.900 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.901 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.902 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.904 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.905 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.906 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.907 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.912 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.913 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.914 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.915 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.917 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.918 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.919 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.920 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.921 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.922 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.923 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.925 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.926 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.927 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.928 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.929 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.930 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.931 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.931 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.931 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.931 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.931 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.931 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.931 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.931 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.931 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.933 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.934 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-09-12T00:08:05.935 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.936 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.937 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.938 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.939 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.940 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.941 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-09-12T00:08:05.942 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.943 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.944 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.945 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.946 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-09-12T00:08:05.947 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.949 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.951 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.952 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.953 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-09-12T00:08:05.953 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-09-12T00:08:05.953 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-09-12T00:08:05.953 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-09-12T00:08:05.953 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-09-12T00:08:05.953 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-09-12T00:08:05.953 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-09-12T00:08:05.953 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.953 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.954 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.955 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-09-12T00:08:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-09-12T00:08:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-09-12T00:08:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-09-12T00:08:05.957 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.958 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "services": { 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard": "https://172.21.15.62:8443/" 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "always_on_modules": { 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "octopus": [ 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-09-12T00:08:05.959 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "pacific": [ 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-09-12T00:08:05.960 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "quincy": [ 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.961 INFO:teuthology.orchestra.run.smithi062.stdout: "reef": [ 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "squid": [ 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-09-12T00:08:05.962 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "last_failure_osd_epoch": 84, 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "active_clients": [ 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.963 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 3505743687 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "libcephsqlite", 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.964 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2973403212 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 312725413 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.965 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2155994841 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.966 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 4021946254 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-09-12T00:08:05.967 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-09-12T00:08:05.968 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-09-12T00:08:05.968 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2060761440 2024-09-12T00:08:05.968 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.968 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:05.968 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:05.968 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-09-12T00:08:05.968 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:08:05.969 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:04 smithi083 ceph-mon[34805]: pgmap v498: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-12T00:08:05.969 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:04 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3051732801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T00:08:05.970 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:04 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3099188770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T00:08:05.970 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:04 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/131189852' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T00:08:05.970 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:04 smithi120 ceph-mon[35182]: pgmap v498: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-12T00:08:05.970 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:04 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3051732801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T00:08:05.970 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:04 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3099188770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T00:08:05.970 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:04 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/131189852' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 -- 172.21.15.62:0/131189852 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2c7c0a2d00 msgr2=0x7f2c7c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2c7c0a2d00 0x7f2c7c0a51c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400abe0 tx=0x7f2c8400a2d0 comp rx=0 tx=0).stop 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 -- 172.21.15.62:0/131189852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9410c0e0 msgr2=0x7f2c9419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9410c0e0 0x7f2c9419cbb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c02ae70 tx=0x7f2c8c02f5d0 comp rx=0 tx=0).stop 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 -- 172.21.15.62:0/131189852 shutdown_connections 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f2c7c0a2d00 0x7f2c7c0a51c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9410c0e0 0x7f2c9419cbb0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c94103630 0x7f2c94198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 --2- 172.21.15.62:0/131189852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c94102d10 0x7f2c941982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 -- 172.21.15.62:0/131189852 >> 172.21.15.62:0/131189852 conn(0x7f2c94077a10 msgr2=0x7f2c940fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:05.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 -- 172.21.15.62:0/131189852 shutdown_connections 2024-09-12T00:08:05.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:04.556+0000 7f2c990ba640 1 -- 172.21.15.62:0/131189852 wait complete. 2024-09-12T00:08:05.978 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T00:08:05.978 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T00:08:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:04 smithi062 ceph-mon[30958]: pgmap v498: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-12T00:08:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:04 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3051732801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T00:08:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:04 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3099188770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T00:08:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:04 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/131189852' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T00:08:06.336 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:06.889 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:08:06.889 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:06.889 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 302, 2024-09-12T00:08:06.889 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 292, 2024-09-12T00:08:06.889 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1466272, 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 179760, 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.051313521, 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.020508281, 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.040682207, 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 291, 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.028532297, 2024-09-12T00:08:06.890 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000098049 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1466272, 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 179760 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 7, 2024-09-12T00:08:06.891 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 226, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3472, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 275840, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.020040340, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.012338348, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.004913171, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 224, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.024026105, 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000107259 2024-09-12T00:08:06.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 275840 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-09-12T00:08:06.893 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-09-12T00:08:06.894 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-09-12T00:08:06.894 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-09-12T00:08:06.896 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.000305374, 2024-09-12T00:08:06.896 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T00:08:06.896 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000225564, 2024-09-12T00:08:06.896 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T00:08:06.897 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:06.898 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:06.899 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:06.900 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:06.900 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:06.900 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:06.900 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:06.900 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:06.900 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:06.900 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:06.900 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:06.900 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:06.900 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148714, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-09-12T00:08:06.902 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 305, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1423668, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 305, 2024-09-12T00:08:06.903 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-09-12T00:08:06.904 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-09-12T00:08:06.904 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 305, 2024-09-12T00:08:06.904 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1423668, 2024-09-12T00:08:06.904 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-09-12T00:08:06.904 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:06.904 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:06.904 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:06.904 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:06.904 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:08:07.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:06 smithi120 ceph-mon[35182]: pgmap v499: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 99 KiB/s wr, 27 op/s 2024-09-12T00:08:07.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:06 smithi120 ceph-mon[35182]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T00:08:07.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:06 smithi083 ceph-mon[34805]: pgmap v499: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 99 KiB/s wr, 27 op/s 2024-09-12T00:08:07.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:06 smithi083 ceph-mon[34805]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T00:08:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:06 smithi062 ceph-mon[30958]: pgmap v499: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 99 KiB/s wr, 27 op/s 2024-09-12T00:08:07.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:06 smithi062 ceph-mon[30958]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T00:08:07.691 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-12T00:08:07.691 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-12T00:08:07.691 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-12T00:08:07.691 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-12T00:08:07.691 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-12T00:08:07.691 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T00:08:07.692 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T00:08:07.948 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:08.471 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:08:08.471 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:08.471 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-09-12T00:08:08.471 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-09-12T00:08:08.471 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-09-12T00:08:08.471 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.001229531, 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000134825, 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.472 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 3, 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 228, 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 480, 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 276096, 2024-09-12T00:08:08.473 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.017773728, 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.012428727, 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.002285592, 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 225, 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.018237346, 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000081054 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:08.474 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 276096 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 311, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 299, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1470624, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 183408, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.056143821, 2024-09-12T00:08:08.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.020510087, 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.047857054, 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 299, 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.021669677, 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000072473 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:08.476 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1470624, 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 183408 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:08.477 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:08.478 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:08.479 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148754, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 31, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-09-12T00:08:08.480 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:08.481 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 310, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1424318, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 310, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-09-12T00:08:08.482 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 310, 2024-09-12T00:08:08.483 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1424318, 2024-09-12T00:08:08.483 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-09-12T00:08:08.483 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:08.483 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:08.483 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:08.483 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:08.483 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:08.483 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:08:09.099 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:08 smithi062 ceph-mon[30958]: pgmap v501: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-09-12T00:08:09.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:08 smithi120 ceph-mon[35182]: pgmap v501: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-09-12T00:08:09.140 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T00:08:09.140 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T00:08:09.140 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-12T00:08:09.140 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-12T00:08:09.141 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-12T00:08:09.141 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T00:08:09.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T00:08:09.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:08 smithi083 ceph-mon[34805]: pgmap v501: 97 pgs: 97 active+clean; 4.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-09-12T00:08:09.399 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:09.914 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:08:09.914 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:09.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 303, 2024-09-12T00:08:09.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 293, 2024-09-12T00:08:09.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1466544, 2024-09-12T00:08:09.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 180400, 2024-09-12T00:08:09.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-09-12T00:08:09.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-09-12T00:08:09.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.049592124, 2024-09-12T00:08:09.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.021120264, 2024-09-12T00:08:09.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.038531458, 2024-09-12T00:08:09.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:09.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 292, 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.024545385, 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000084059 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1466544, 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 180400 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.916 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 7, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 227, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3472, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 276016, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.019061275, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.011764617, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.004282007, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 225, 2024-09-12T00:08:09.917 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.029692797, 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000131967 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 276016 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:09.918 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.000380900, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000293956, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:09.919 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-09-12T00:08:09.920 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:09.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:09.922 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-09-12T00:08:09.923 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148714, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:09.924 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-09-12T00:08:09.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 306, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1423798, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 306, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 306, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1423798, 2024-09-12T00:08:09.926 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-09-12T00:08:09.927 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:09.927 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:09.927 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:09.927 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:09.927 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:09.927 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:08:10.645 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-12T00:08:10.802 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-12T00:08:10.802 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-12T00:08:10.802 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-12T00:08:10.802 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-12T00:08:10.802 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T00:08:10.802 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T00:08:10.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:10 smithi062 ceph-mon[30958]: pgmap v502: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-09-12T00:08:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:11.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:10 smithi120 ceph-mon[35182]: pgmap v502: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-09-12T00:08:11.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:10 smithi083 ceph-mon[34805]: pgmap v502: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-09-12T00:08:11.568 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-09-12T00:08:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 309, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 299, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1468176, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 184240, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.054468466, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.021335975, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.043771462, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 298, 2024-09-12T00:08:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.020982280, 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000070410 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1468176, 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 184240 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 7, 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 231, 2024-09-12T00:08:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3472, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 276720, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.019852345, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.011739678, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.005605228, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 229, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.029737839, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000129859 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 276720 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-09-12T00:08:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.000429512, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000335972, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148731, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 31, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-09-12T00:08:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 312, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1424578, 2024-09-12T00:08:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 312, 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 312, 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1424578, 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-09-12T00:08:11.580 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-09-12T00:08:12.444 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T00:08:12.444 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T00:08:12.444 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-12T00:08:12.445 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-12T00:08:12.445 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-12T00:08:12.445 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T00:08:12.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T00:08:12.704 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:08:13.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:12 smithi120 ceph-mon[35182]: pgmap v503: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 10 op/s 2024-09-12T00:08:13.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:12 smithi120 ceph-mon[35182]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T00:08:13.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:12 smithi083 ceph-mon[34805]: pgmap v503: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 10 op/s 2024-09-12T00:08:13.149 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:12 smithi083 ceph-mon[34805]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T00:08:13.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:12 smithi062 ceph-mon[30958]: pgmap v503: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 10 op/s 2024-09-12T00:08:13.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:12 smithi062 ceph-mon[30958]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T00:08:13.258 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-12T00:08:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 8865, 2024-09-12T00:08:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 8910, 2024-09-12T00:08:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 2269520, 2024-09-12T00:08:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 18540994, 2024-09-12T00:08:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 224, 2024-09-12T00:08:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-09-12T00:08:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.678066349, 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.182062327, 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.175556532, 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.266177473, 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8905, 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.398777103, 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000044781 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 499, 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000061253, 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000122 2024-09-12T00:08:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 99696, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 474384 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 20071, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 22282, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 6231802, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 24719219, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 24, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 6, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 1.151781048, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.389117968, 2024-09-12T00:08:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.630715248, 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.248877385, 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 22283, 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.941094799, 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000042233 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 23846, 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.003626939, 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000152 2024-09-12T00:08:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 491760, 2024-09-12T00:08:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 289360 2024-09-12T00:08:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 9660, 2024-09-12T00:08:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 9908, 2024-09-12T00:08:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 4645148, 2024-09-12T00:08:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 8502981, 2024-09-12T00:08:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 15, 2024-09-12T00:08:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 6, 2024-09-12T00:08:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.563455214, 2024-09-12T00:08:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.208750810, 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.293970477, 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.096095124, 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 9907, 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.492682600, 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000049730 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8493, 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.001331463, 2024-09-12T00:08:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000156 2024-09-12T00:08:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1600640, 2024-09-12T00:08:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 219744 2024-09-12T00:08:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-09-12T00:08:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-09-12T00:08:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-09-12T00:08:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-09-12T00:08:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 17212, 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 5.160613032, 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000299826 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 247, 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.008886250, 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000035976 2024-09-12T00:08:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 3675, 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 3382, 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3382, 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 37.450900190, 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.011073595 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 293, 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 1, 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-09-12T00:08:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 55, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 5, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 1272, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 109, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 1036, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 127, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 1116, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 618, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 775, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 775, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 31, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 5072, 2024-09-12T00:08:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 4143, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 281, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 17, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 60, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 5707, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 11, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 576, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 5, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 285, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-09-12T00:08:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 17307, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 1007, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 1365737271, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 7, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 17307, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 707, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 21, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 686, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 16600, 2024-09-12T00:08:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 26, 2024-09-12T00:08:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 1847, 2024-09-12T00:08:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 2928, 2024-09-12T00:08:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T00:08:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T00:08:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T00:08:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 3571, 2024-09-12T00:08:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 1378, 2024-09-12T00:08:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 77, 2024-09-12T00:08:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 363 2024-09-12T00:08:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-09-12T00:08:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-09-12T00:08:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 17, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 42, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 10, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 119, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 72, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 36, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 3, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-09-12T00:08:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-09-12T00:08:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-09-12T00:08:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-09-12T00:08:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-09-12T00:08:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 14, 2024-09-12T00:08:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 12449, 2024-09-12T00:08:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 21, 2024-09-12T00:08:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 11, 2024-09-12T00:08:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-09-12T00:08:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 5, 2024-09-12T00:08:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-09-12T00:08:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-09-12T00:08:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-09-12T00:08:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-09-12T00:08:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 5136, 2024-09-12T00:08:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-09-12T00:08:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 5136, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 11, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 11, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 35528017, 2024-09-12T00:08:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2945, 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 18.984804108, 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.006446453 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1140, 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 1779, 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 639, 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 107, 2024-09-12T00:08:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 118, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 11, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 1277, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 2238, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 961, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 775, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 2145, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 1370, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 74328, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 231712 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 4194, 2024-09-12T00:08:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 1070, 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 3675, 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 289, 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 2085, 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1079, 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 21.825604281, 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.020227622 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 548, 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3.405619045, 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.006214633 2024-09-12T00:08:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 40, 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.182985319, 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004574632 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-09-12T00:08:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 744, 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 4.187260085, 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005628037 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-09-12T00:08:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-09-12T00:08:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 16, 2024-09-12T00:08:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.002435545, 2024-09-12T00:08:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000152221 2024-09-12T00:08:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-09-12T00:08:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 37, 2024-09-12T00:08:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.292228404, 2024-09-12T00:08:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.007898064 2024-09-12T00:08:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-09-12T00:08:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-09-12T00:08:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 16, 2024-09-12T00:08:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.092682005, 2024-09-12T00:08:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005792625 2024-09-12T00:08:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-09-12T00:08:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 622, 2024-09-12T00:08:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.313602723, 2024-09-12T00:08:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000504184 2024-09-12T00:08:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-09-12T00:08:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 91, 2024-09-12T00:08:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.111494638, 2024-09-12T00:08:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001225215 2024-09-12T00:08:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-09-12T00:08:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 88, 2024-09-12T00:08:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 6.782266247, 2024-09-12T00:08:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.077071207 2024-09-12T00:08:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-09-12T00:08:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-09-12T00:08:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.010706236, 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005353118 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 9, 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.049498976, 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005499886 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-09-12T00:08:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000909367, 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000909367 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 7, 2024-09-12T00:08:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.017535735, 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002505105 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 82, 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.176071584, 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002147214 2024-09-12T00:08:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-09-12T00:08:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-09-12T00:08:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-09-12T00:08:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 4, 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 2, 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 2, 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 212, 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 106, 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 938, 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-09-12T00:08:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 585689, 2024-09-12T00:08:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 1100, 2024-09-12T00:08:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-09-12T00:08:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-09-12T00:08:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 8248, 2024-09-12T00:08:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 182, 2024-09-12T00:08:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-09-12T00:08:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 4281635, 2024-09-12T00:08:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 57132, 2024-09-12T00:08:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-09-12T00:08:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-09-12T00:08:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-09-12T00:08:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-09-12T00:08:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-09-12T00:08:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 21336, 2024-09-12T00:08:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 32027519, 2024-09-12T00:08:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 36, 2024-09-12T00:08:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 21336, 2024-09-12T00:08:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-09-12T00:08:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 21336, 2024-09-12T00:08:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 50.117767801, 2024-09-12T00:08:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002348976 2024-09-12T00:08:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-09-12T00:08:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-09-12T00:08:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 21336, 2024-09-12T00:08:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 21639 2024-09-12T00:08:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 21336, 2024-09-12T00:08:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 17377, 2024-09-12T00:08:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 3959, 2024-09-12T00:08:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-09-12T00:08:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-09-12T00:08:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 0, 2024-09-12T00:08:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 11, 2024-09-12T00:08:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 48, 2024-09-12T00:08:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 2972, 2024-09-12T00:08:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 95, 2024-09-12T00:08:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-09-12T00:08:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-09-12T00:08:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-09-12T00:08:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 3, 2024-09-12T00:08:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 607, 2024-09-12T00:08:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-09-12T00:08:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-09-12T00:08:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-09-12T00:08:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 17328, 2024-09-12T00:08:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 22, 2024-09-12T00:08:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-09-12T00:08:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-09-12T00:08:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-09-12T00:08:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-09-12T00:08:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-09-12T00:08:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-09-12T00:08:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T00:08:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-09-12T00:08:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-09-12T00:08:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-09-12T00:08:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-09-12T00:08:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-09-12T00:08:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-09-12T00:08:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-09-12T00:08:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-09-12T00:08:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-09-12T00:08:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-09-12T00:08:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-09-12T00:08:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-09-12T00:08:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-09-12T00:08:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-09-12T00:08:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-09-12T00:08:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-09-12T00:08:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-09-12T00:08:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-09-12T00:08:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-09-12T00:08:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 102, 2024-09-12T00:08:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-09-12T00:08:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 21, 2024-09-12T00:08:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 12, 2024-09-12T00:08:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 12, 2024-09-12T00:08:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-09-12T00:08:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-09-12T00:08:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 447, 2024-09-12T00:08:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-09-12T00:08:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 104 2024-09-12T00:08:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-09-12T00:08:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-09-12T00:08:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-09-12T00:08:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-09-12T00:08:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-09-12T00:08:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-09-12T00:08:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 12464 2024-09-12T00:08:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-09-12T00:08:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-09-12T00:08:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 16, 2024-09-12T00:08:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-09-12T00:08:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 10, 2024-09-12T00:08:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 86, 2024-09-12T00:08:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 72, 2024-09-12T00:08:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-09-12T00:08:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-12T00:08:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 38592, 2024-09-12T00:08:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 9586152, 2024-09-12T00:08:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 38592, 2024-09-12T00:08:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-12T00:08:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-12T00:08:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 38592, 2024-09-12T00:08:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 9586152, 2024-09-12T00:08:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-09-12T00:08:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-12T00:08:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-12T00:08:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-12T00:08:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 21336, 2024-09-12T00:08:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 32035655, 2024-09-12T00:08:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 21336, 2024-09-12T00:08:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 32035655, 2024-09-12T00:08:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 21336, 2024-09-12T00:08:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 21336, 2024-09-12T00:08:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 21336, 2024-09-12T00:08:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 21336, 2024-09-12T00:08:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 72, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 8152, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 72, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 27, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 8152, 2024-09-12T00:08:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x5578bdb60960": { 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 5136, 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 31333713, 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 5136, 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2945, 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 31333713, 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:13.325 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-12T00:08:14.074 INFO:tasks.check_counter:Daemon mds.b mds.exported=11 2024-09-12T00:08:14.074 INFO:tasks.check_counter:Daemon mds.b mds.imported=5 2024-09-12T00:08:14.074 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-09-12T00:08:14.074 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-09-12T00:08:14.074 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=9 2024-09-12T00:08:14.075 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T00:08:14.075 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T00:08:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:08:14.854 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-12T00:08:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 357, 2024-09-12T00:08:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 327, 2024-09-12T00:08:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1576786, 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 188900, 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.052664756, 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.019374862, 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.046852439, 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000020859, 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 327, 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.028610286, 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000087493 2024-09-12T00:08:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1575952, 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 188576 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 986, 2024-09-12T00:08:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1211, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 242406, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 874389, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.089534558, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.030317305, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.018924478, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.029710165, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1208, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.054170412, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000044843 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 561472 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2283, 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2285, 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 4259650, 2024-09-12T00:08:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 726052, 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.193215382, 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.065846404, 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.052105581, 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.058941734, 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2283, 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.095102744, 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000041656 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-09-12T00:08:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5485, 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.131768457, 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000024023 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-09-12T00:08:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 460, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 14, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 446, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 439, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 10, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-09-12T00:08:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-09-12T00:08:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T00:08:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-09-12T00:08:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-09-12T00:08:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 1321, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 3, 2024-09-12T00:08:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 7779453, 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 7775971, 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 1321 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-09-12T00:08:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 198, 2024-09-12T00:08:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 426, 2024-09-12T00:08:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 228, 2024-09-12T00:08:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 20, 2024-09-12T00:08:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 21, 2024-09-12T00:08:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 1, 2024-09-12T00:08:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 460, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 512, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 52, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 50080, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215328 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-09-12T00:08:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-09-12T00:08:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-09-12T00:08:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-09-12T00:08:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-09-12T00:08:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 4, 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 2, 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 3, 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-09-12T00:08:14.885 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 2831, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:14.886 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:14.887 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-09-12T00:08:14.888 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 200949, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 344, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:14.889 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 8248, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 182, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 930295, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 21173, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-09-12T00:08:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 3268, 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 3268, 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3268, 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 10.610742197, 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003246861 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.891 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3268, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3269 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 3268, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 3268, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 2046, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 1220, 2024-09-12T00:08:14.892 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-09-12T00:08:14.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-09-12T00:08:14.894 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 102, 2024-09-12T00:08:14.895 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 21, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 5, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 6, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 1, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-09-12T00:08:14.896 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-09-12T00:08:14.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 3622, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 5731886, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 3622, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3622, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 5731886, 2024-09-12T00:08:14.898 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-12T00:08:14.899 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3268, 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 3585154, 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3268, 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3585154, 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-09-12T00:08:14.900 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3268, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 3268, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3268, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3268, 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:14.901 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-12T00:08:14.902 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x558947c5eb40": { 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:14.903 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:14.904 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:14.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:14.905 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-12T00:08:14.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:14 smithi062 ceph-mon[30958]: pgmap v505: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-12T00:08:15.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:14 smithi120 ceph-mon[35182]: pgmap v505: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-12T00:08:15.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:14 smithi083 ceph-mon[34805]: pgmap v505: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-09-12T00:08:15.813 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T00:08:15.814 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T00:08:15.814 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-12T00:08:15.814 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-12T00:08:15.814 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T00:08:15.814 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T00:08:15.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T00:08:16.069 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:08:16.588 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-12T00:08:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1424, 2024-09-12T00:08:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1395, 2024-09-12T00:08:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1937312, 2024-09-12T00:08:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 528062, 2024-09-12T00:08:16.588 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.150623309, 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.058447072, 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.066845679, 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.029006767, 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1394, 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.074604119, 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000053518 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.589 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1576784, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 186848 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 5, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 230, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1036, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 652074, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 8, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.013406301, 2024-09-12T00:08:16.590 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.008479661, 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.002573357, 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000065503, 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 227, 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.027507619, 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000121178 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.591 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 651376 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2304, 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2304, 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 6013113, 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 731855, 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.137362873, 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.030631971, 2024-09-12T00:08:16.592 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.043790881, 2024-09-12T00:08:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.054737078, 2024-09-12T00:08:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2303, 2024-09-12T00:08:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.081044452, 2024-09-12T00:08:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000035190 2024-09-12T00:08:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.593 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5707, 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.123541481, 2024-09-12T00:08:16.594 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000021647 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-09-12T00:08:16.595 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-09-12T00:08:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-09-12T00:08:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-09-12T00:08:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-09-12T00:08:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-09-12T00:08:16.596 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 709, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 4, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 705, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 703, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-09-12T00:08:16.597 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 21, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-09-12T00:08:16.598 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-09-12T00:08:16.599 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-09-12T00:08:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-09-12T00:08:16.601 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-09-12T00:08:16.602 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-09-12T00:08:16.602 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-09-12T00:08:16.602 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-09-12T00:08:16.602 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-09-12T00:08:16.602 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-09-12T00:08:16.602 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-09-12T00:08:16.602 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-09-12T00:08:16.602 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 2744, 2024-09-12T00:08:16.603 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 5, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 9620636, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 9617962, 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-09-12T00:08:16.604 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 2744 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 466, 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 734, 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 268, 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 31, 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 45, 2024-09-12T00:08:16.605 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 14, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 709, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 1100, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 391, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 53192, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215328 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-09-12T00:08:16.606 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.607 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.608 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.609 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.610 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-09-12T00:08:16.611 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.612 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-09-12T00:08:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.614 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.615 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.616 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.617 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 3, 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 1, 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 3, 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 2834, 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.619 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:16.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:16.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 201589, 2024-09-12T00:08:16.622 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 575, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 8128, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 179, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:16.623 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 1706966, 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 31440, 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 3375, 2024-09-12T00:08:16.624 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 3375, 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3375, 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3.344918354, 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000991086 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3375, 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3376 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 3375, 2024-09-12T00:08:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 3375, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 2041, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 1332, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-09-12T00:08:16.626 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T00:08:16.627 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-09-12T00:08:16.628 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 102, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 21, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 4, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 6, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 2, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-09-12T00:08:16.629 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-09-12T00:08:16.630 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 3729, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 7594883, 2024-09-12T00:08:16.631 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 3729, 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3729, 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 7594883, 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.632 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-09-12T00:08:16.633 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:16.633 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-12T00:08:16.633 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:16.633 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-12T00:08:16.633 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-12T00:08:16.633 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:16.633 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:16.633 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3375, 2024-09-12T00:08:16.633 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 5426337, 2024-09-12T00:08:16.633 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3375, 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 5426337, 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-12T00:08:16.634 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3375, 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 3375, 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3375, 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3375, 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.635 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.636 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x555934ad8b40": { 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-12T00:08:16.637 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-12T00:08:16.638 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-09-12T00:08:16.638 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-09-12T00:08:16.638 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:16.638 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:16.638 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:16.638 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:16.638 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:16.638 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:16.638 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-12T00:08:17.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:16 smithi120 ceph-mon[35182]: pgmap v506: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-09-12T00:08:17.121 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:16 smithi120 ceph-mon[35182]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:08:17.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:16 smithi083 ceph-mon[34805]: pgmap v506: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-09-12T00:08:17.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:16 smithi083 ceph-mon[34805]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:08:17.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:16 smithi062 ceph-mon[30958]: pgmap v506: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-09-12T00:08:17.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:16 smithi062 ceph-mon[30958]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T00:08:17.563 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T00:08:17.563 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T00:08:17.563 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-12T00:08:17.563 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-12T00:08:17.563 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T00:08:17.563 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T00:08:17.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T00:08:17.810 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:08:18.316 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001308098, 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000514895, 2024-09-12T00:08:18.317 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T00:08:18.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 311, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 301, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1468720, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 185520, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.047175374, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.018288747, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.041573215, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:18.319 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 300, 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.015821182, 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000052737 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1468720, 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 185520 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.320 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 7, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 233, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3472, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 277072, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.020586170, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.009750865, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.003882195, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 231, 2024-09-12T00:08:18.321 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.017200593, 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000074461 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 277072 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-09-12T00:08:18.322 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:18.323 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:18.325 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 149775, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 31, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:18.326 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-12T00:08:18.327 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 314, 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1424838, 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 314, 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 314, 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1424838, 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:18.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-12T00:08:18.329 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-12T00:08:18.916 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:18 smithi083 ceph-mon[34805]: pgmap v507: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 91 KiB/s wr, 12 op/s 2024-09-12T00:08:18.998 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-12T00:08:18.999 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-12T00:08:18.999 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-12T00:08:18.999 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-12T00:08:18.999 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-12T00:08:18.999 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T00:08:18.999 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T00:08:19.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:18 smithi120 ceph-mon[35182]: pgmap v507: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 91 KiB/s wr, 12 op/s 2024-09-12T00:08:19.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:18 smithi062 ceph-mon[30958]: pgmap v507: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 91 KiB/s wr, 12 op/s 2024-09-12T00:08:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:08:20.063 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-12T00:08:20.063 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:20.063 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2538, 2024-09-12T00:08:20.063 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 3684, 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 748197, 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 5834286, 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 20, 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 4, 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.173378440, 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.066964073, 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.099554300, 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.014196686, 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3683, 2024-09-12T00:08:20.064 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.159344859, 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000043264 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 4565, 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000819656, 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000179 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 6784, 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 10816 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 901, 2024-09-12T00:08:20.065 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 871, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 1722565, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 890246, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 49, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.101134292, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.032641895, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.068581817, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.010847382, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 870, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.056886064, 2024-09-12T00:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000065386 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 47, 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000004844, 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000103 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 1594336, 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 210848 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 3484, 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 3452, 2024-09-12T00:08:20.067 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 1806253, 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 1880070, 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 10, 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 6, 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.220569699, 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.089111732, 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.141526455, 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.013257907, 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3449, 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.145536173, 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000042196 2024-09-12T00:08:20.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.069 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:20.069 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5897, 2024-09-12T00:08:20.069 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.001017337, 2024-09-12T00:08:20.069 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000172 2024-09-12T00:08:20.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.069 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T00:08:20.069 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 563888 2024-09-12T00:08:20.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.070 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-12T00:08:20.070 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-09-12T00:08:20.070 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-12T00:08:20.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.071 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-09-12T00:08:20.071 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-12T00:08:20.071 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-12T00:08:20.071 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1051, 2024-09-12T00:08:20.071 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.437985597, 2024-09-12T00:08:20.071 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000416732 2024-09-12T00:08:20.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:20.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.072 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-09-12T00:08:20.072 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-12T00:08:20.072 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-12T00:08:20.072 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 608, 2024-09-12T00:08:20.072 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.021008423, 2024-09-12T00:08:20.073 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000034553 2024-09-12T00:08:20.073 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:20.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.073 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-09-12T00:08:20.073 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 1225, 2024-09-12T00:08:20.073 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 1215, 2024-09-12T00:08:20.073 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-09-12T00:08:20.074 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1215, 2024-09-12T00:08:20.074 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 21.472206110, 2024-09-12T00:08:20.074 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.017672597 2024-09-12T00:08:20.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.074 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-09-12T00:08:20.074 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 10, 2024-09-12T00:08:20.074 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 4, 2024-09-12T00:08:20.075 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-09-12T00:08:20.075 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 11, 2024-09-12T00:08:20.075 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-09-12T00:08:20.075 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-09-12T00:08:20.075 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 546, 2024-09-12T00:08:20.075 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 13, 2024-09-12T00:08:20.076 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 473, 2024-09-12T00:08:20.076 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 60, 2024-09-12T00:08:20.076 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 528, 2024-09-12T00:08:20.076 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 64, 2024-09-12T00:08:20.076 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 343, 2024-09-12T00:08:20.076 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 343, 2024-09-12T00:08:20.077 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 19, 2024-09-12T00:08:20.077 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 2529, 2024-09-12T00:08:20.077 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 2140, 2024-09-12T00:08:20.077 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 8, 2024-09-12T00:08:20.077 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 21, 2024-09-12T00:08:20.077 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 77, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 1203, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 1, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 51, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 15, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 489, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 348, 2024-09-12T00:08:20.078 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 1026, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 1577180583, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 7, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 348, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 348, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 206, 2024-09-12T00:08:20.079 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 672, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 609, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 260, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 131, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 819 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-09-12T00:08:20.080 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 23, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 3, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 2, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 197, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 190, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 5, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-09-12T00:08:20.081 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 14, 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 301, 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 1, 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.082 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 1342, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 1342, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 3, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 3, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-09-12T00:08:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 8035662, 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 0, 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 316, 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2.609107660, 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.008256669 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 0 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.084 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 476, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 717, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 241, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 42, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 43, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 1, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 559, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 813, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 254, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 343, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 580, 2024-09-12T00:08:20.085 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 237, 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 58780, 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 223520 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 1910, 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 16, 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 1225, 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 45, 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 471, 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-09-12T00:08:20.086 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 187, 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 17.018327158, 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.091007097 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 453, 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 3.232519187, 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.007135803 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 15, 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.079858535, 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.005323902 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 222, 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.599860947, 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002702076 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.088 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.089 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.089 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-09-12T00:08:20.089 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-09-12T00:08:20.089 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000157922, 2024-09-12T00:08:20.089 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000157922 2024-09-12T00:08:20.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.089 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-09-12T00:08:20.090 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.090 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.090 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.090 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-09-12T00:08:20.090 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.091 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.091 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.091 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-09-12T00:08:20.091 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.091 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.091 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.092 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-09-12T00:08:20.092 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.092 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.092 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.092 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-09-12T00:08:20.092 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.092 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.092 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-09-12T00:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-09-12T00:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 124, 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.051603682, 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000416158 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 11, 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.006518957, 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000592632 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-09-12T00:08:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 192, 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.459674501, 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002394138 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.095 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.096 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.097 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.098 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 10, 2024-09-12T00:08:20.099 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.023685221, 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002368522 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 2, 2024-09-12T00:08:20.100 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 4, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 2, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 2, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 247, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 123, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 944, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:20.101 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:20.102 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:20.103 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 576490, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 725, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-12T00:08:20.104 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8128, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 179, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 1849472, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 33504, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 1259, 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 3915912, 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 33, 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 1259, 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1259, 2024-09-12T00:08:20.106 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 7.933824808, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.006301687 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1259, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1391 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 1259, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 425, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 834, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 0, 2024-09-12T00:08:20.107 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 10, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 73, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 389, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 117, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 202, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-09-12T00:08:20.108 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 348, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 20, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-09-12T00:08:20.109 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-09-12T00:08:20.110 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 102, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 21, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 12, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 12, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 162, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 8, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 62 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.111 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-09-12T00:08:20.112 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 1, 2024-09-12T00:08:20.112 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-09-12T00:08:20.112 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-09-12T00:08:20.112 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-09-12T00:08:20.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.112 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-09-12T00:08:20.112 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 316 2024-09-12T00:08:20.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.112 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-09-12T00:08:20.112 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 10, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 10, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 190, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 190, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 6919, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 3629269, 2024-09-12T00:08:20.113 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 6919, 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 6919, 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 3629269, 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-09-12T00:08:20.114 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 1259, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 3935374, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 1259, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 3935374, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.115 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 1259, 2024-09-12T00:08:20.116 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 1259, 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 1259, 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 1259, 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:20.117 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 190, 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 19526, 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 190, 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 73, 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 19526, 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:20.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x55ea31d22960": { 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 1342, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 3841358, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 1342, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 316, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 3841358, 2024-09-12T00:08:20.119 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:20.120 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:20.120 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:20.120 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:20.120 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:20.120 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:20.120 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-12T00:08:20.974 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-09-12T00:08:20.974 INFO:tasks.check_counter:Daemon mds.c mds.imported=15 2024-09-12T00:08:20.974 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-09-12T00:08:20.974 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-12T00:08:20.974 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T00:08:20.974 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T00:08:20.975 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T00:08:21.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:20 smithi120 ceph-mon[35182]: pgmap v508: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 91 KiB/s wr, 12 op/s 2024-09-12T00:08:21.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:20 smithi083 ceph-mon[34805]: pgmap v508: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 91 KiB/s wr, 12 op/s 2024-09-12T00:08:21.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:20 smithi062 ceph-mon[30958]: pgmap v508: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 91 KiB/s wr, 12 op/s 2024-09-12T00:08:21.222 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:08:21.764 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-12T00:08:21.764 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:21.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1603, 2024-09-12T00:08:21.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1603, 2024-09-12T00:08:21.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 8886937, 2024-09-12T00:08:21.764 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 508940, 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 7, 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 3, 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.159153189, 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.045321471, 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.048116418, 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.051611117, 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1602, 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.081839516, 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000051085 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.765 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1322, 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1549, 2024-09-12T00:08:21.766 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 15361559, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 984882, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 7, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.117103665, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.038892801, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.035713429, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.033033616, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1546, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.075868183, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000049073 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.767 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 565056 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1047, 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1018, 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 10252003, 2024-09-12T00:08:21.768 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 408985, 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 3, 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.111269938, 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.037169718, 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.063560755, 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.018783217, 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1017, 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.049171633, 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000048349 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:21.769 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 1577760, 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 188928 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-09-12T00:08:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 6072, 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.146771727, 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000024171 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 0, 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 0, 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 0, 2024-09-12T00:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 0, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 1074, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 25, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 1049, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 1038, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 0, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 0, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 0, 2024-09-12T00:08:21.773 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 35, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 0, 2024-09-12T00:08:21.774 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 0, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 1007, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 1365737271, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 7, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-09-12T00:08:21.775 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 0, 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 0 2024-09-12T00:08:21.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-09-12T00:08:21.777 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-09-12T00:08:21.778 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 0, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 5143, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 0, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-09-12T00:08:21.779 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 11, 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 35988242, 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 35917311, 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 5143 2024-09-12T00:08:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 852, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 1745, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 893, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 101, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 112, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 11, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 1074, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 1917, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 843, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 0, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 0, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 0, 2024-09-12T00:08:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 54716, 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 215328 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 0, 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 0, 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 0, 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 0, 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 0, 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.785 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.787 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.788 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.789 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-09-12T00:08:21.789 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.789 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.789 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.789 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-09-12T00:08:21.789 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.789 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.789 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.790 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.792 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.794 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 2, 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 4, 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 2, 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 3, 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 0, 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 0, 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 2837, 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.795 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:21.796 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:21.797 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 244141, 2024-09-12T00:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 1082, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8128, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 179, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:21.799 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 3101866, 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 39033, 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 3611, 2024-09-12T00:08:21.800 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 26, 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 4, 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 3611, 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3611, 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 3.603920479, 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000998039 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3611, 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 3612 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 3611, 2024-09-12T00:08:21.801 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 3611, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 2145, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 1463, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 0, 2024-09-12T00:08:21.802 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 2, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T00:08:21.803 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-09-12T00:08:21.804 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 102, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 21, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 7, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 7, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-09-12T00:08:21.805 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 0, 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 2, 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 0 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 0, 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 0 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.806 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 0, 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 0, 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 0, 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:21.807 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 3968, 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 34122252, 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 3968, 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3968, 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 34122252, 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:21.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 3611, 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 31914677, 2024-09-12T00:08:21.809 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3611, 2024-09-12T00:08:21.810 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 31914677, 2024-09-12T00:08:21.810 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:21.810 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.810 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.810 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.810 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:21.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.810 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 3611, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 3611, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3611, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 3611, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.811 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-12T00:08:21.812 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x560fb2a36b40": { 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-12T00:08:21.813 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:21.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:21.815 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-12T00:08:22.581 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T00:08:22.582 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T00:08:22.582 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-09-12T00:08:22.582 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-12T00:08:22.582 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T00:08:22.582 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T00:08:22.582 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T00:08:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:08:23.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:22 smithi120 ceph-mon[35182]: pgmap v509: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 90 KiB/s wr, 11 op/s 2024-09-12T00:08:23.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:22 smithi083 ceph-mon[34805]: pgmap v509: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 90 KiB/s wr, 11 op/s 2024-09-12T00:08:23.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:22 smithi062 ceph-mon[30958]: pgmap v509: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 90 KiB/s wr, 11 op/s 2024-09-12T00:08:23.469 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-12T00:08:23.469 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:23.469 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2684, 2024-09-12T00:08:23.469 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2893, 2024-09-12T00:08:23.469 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 815768, 2024-09-12T00:08:23.469 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 7070729, 2024-09-12T00:08:23.469 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 15, 2024-09-12T00:08:23.469 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-09-12T00:08:23.469 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.198959882, 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.077249048, 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.049944210, 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.055009736, 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2891, 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.093794344, 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000032443 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 475, 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000101409, 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000213 2024-09-12T00:08:23.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 440512 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 10045, 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 11350, 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 5122620, 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 5384370, 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 8, 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 6, 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.547387253, 2024-09-12T00:08:23.471 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.221208676, 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.381415058, 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.022623463, 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 11351, 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.555537915, 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000048941 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 16981, 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.002879173, 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000169 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.472 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 1599424, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 219568 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1990, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2021, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 549873, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 1515415, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 55, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 10, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.136191323, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.040451309, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.045704228, 2024-09-12T00:08:23.473 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.039317129, 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2017, 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.065524517, 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000032486 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 61, 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000005657, 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000092 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 5632, 2024-09-12T00:08:23.474 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 129152 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5343, 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.392080593, 2024-09-12T00:08:23.475 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000260542 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2, 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000012059, 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000006029 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 2090, 2024-09-12T00:08:23.476 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 2035, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2035, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 56.988452630, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.028004153 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 55, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 12, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 37, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 3, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-09-12T00:08:23.477 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 658, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 61, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 501, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 96, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 570, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 438, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 211, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 211, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 28, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 5042, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 4000, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 48, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 28, 2024-09-12T00:08:23.478 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 280, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 1983, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 9, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 349, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 7, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 468, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 3449, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 1026, 2024-09-12T00:08:23.479 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 1577180583, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 7, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 3449, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 20, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 20, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 3429, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 2, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 992, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 1514, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 1, 2024-09-12T00:08:23.480 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 1768, 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 550, 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 51, 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 689 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 28, 2024-09-12T00:08:23.481 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 21, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 8, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 160, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 0, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 148, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 7, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 14, 2024-09-12T00:08:23.482 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 2352, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 11, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 3, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 2747, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-09-12T00:08:23.483 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 2747, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 5, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 5, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 9622358, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 0, 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-09-12T00:08:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1416, 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 5.695542344, 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.004022275 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 0 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 459, 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 756, 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 297, 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 53, 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 57, 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 4, 2024-09-12T00:08:23.485 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 663, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 1155, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 492, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 211, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 630, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 419, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 62052, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 223520 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 3565, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 228, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 2090, 2024-09-12T00:08:23.486 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 347, 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 767, 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 39, 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.051285694, 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001315017 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 621, 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 34.642492132, 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.055785011 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-09-12T00:08:23.487 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 155, 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.385811920, 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002489109 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 696, 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 20.471160796, 2024-09-12T00:08:23.488 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.029412587 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000117687, 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000117687 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-09-12T00:08:23.489 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-09-12T00:08:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 174, 2024-09-12T00:08:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.107922471, 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000620244 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 37, 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.035446028, 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000958000 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 151, 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.952378354, 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.006307141 2024-09-12T00:08:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.003500721, 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.003500721 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-09-12T00:08:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 160, 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.338336827, 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002114605 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 2, 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 3, 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 1, 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 2, 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 297, 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 148, 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 938, 2024-09-12T00:08:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 741967, 2024-09-12T00:08:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 574, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8128, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 179, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 1877733, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 43662, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 5186, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 5880332, 2024-09-12T00:08:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 28, 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 5186, 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5186, 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 10.594766077, 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002042955 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5186, 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 5454 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 5186, 2024-09-12T00:08:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 3461, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 1725, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 0, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 10, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 0, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 1416, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 61, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 2, 2024-09-12T00:08:23.506 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 8, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 3449, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 20, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T00:08:23.507 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 1, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-09-12T00:08:23.508 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 102, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 21, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 12, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 12, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-09-12T00:08:23.509 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 398, 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 24, 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 65 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 1, 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 2369 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-09-12T00:08:23.510 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 0, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 0, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 0, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 14715, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 5138923, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:23.511 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 14715, 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 14715, 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 5138923, 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-12T00:08:23.512 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 5186, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 5880140, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 5186, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 5880140, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 5186, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 5186, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 5186, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 5186, 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:23.514 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-12T00:08:23.515 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x55d0e361a960": { 2024-09-12T00:08:23.516 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 1722, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 2747, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 5428054, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 2747, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 1416, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 5426332, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:23.517 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:23.518 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:23.518 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:23.518 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:23.518 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-12T00:08:24.376 INFO:tasks.check_counter:Daemon mds.i mds.exported=9 2024-09-12T00:08:24.376 INFO:tasks.check_counter:Daemon mds.i mds.imported=7 2024-09-12T00:08:24.376 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-09-12T00:08:24.376 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-12T00:08:24.376 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T00:08:24.377 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T00:08:24.377 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T00:08:24.622 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:08:25.008 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:24 smithi120 ceph-mon[35182]: pgmap v510: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 82 KiB/s wr, 12 op/s 2024-09-12T00:08:25.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:24 smithi083 ceph-mon[34805]: pgmap v510: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 82 KiB/s wr, 12 op/s 2024-09-12T00:08:25.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:24 smithi062 ceph-mon[30958]: pgmap v510: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 82 KiB/s wr, 12 op/s 2024-09-12T00:08:25.291 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-12T00:08:25.292 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T00:08:25.292 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 313, 2024-09-12T00:08:25.292 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 303, 2024-09-12T00:08:25.292 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 1469264, 2024-09-12T00:08:25.292 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 186800, 2024-09-12T00:08:25.292 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-09-12T00:08:25.292 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 0, 2024-09-12T00:08:25.292 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.048501320, 2024-09-12T00:08:25.292 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.018770841, 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.042671096, 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 302, 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.020496986, 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000067870 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:25.293 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:25.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.294 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 1469264, 2024-09-12T00:08:25.294 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 186800 2024-09-12T00:08:25.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.294 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T00:08:25.294 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 7, 2024-09-12T00:08:25.294 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 235, 2024-09-12T00:08:25.294 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 3472, 2024-09-12T00:08:25.294 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 277424, 2024-09-12T00:08:25.294 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 6, 2024-09-12T00:08:25.295 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 3, 2024-09-12T00:08:25.295 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.015602699, 2024-09-12T00:08:25.295 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.009345228, 2024-09-12T00:08:25.295 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.003720442, 2024-09-12T00:08:25.295 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:25.295 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:25.295 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 233, 2024-09-12T00:08:25.295 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.024963827, 2024-09-12T00:08:25.295 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000107140 2024-09-12T00:08:25.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.296 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:25.296 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:25.296 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:25.296 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:25.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.296 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-09-12T00:08:25.296 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 277424 2024-09-12T00:08:25.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.296 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T00:08:25.296 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 0, 2024-09-12T00:08:25.297 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2, 2024-09-12T00:08:25.297 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 0, 2024-09-12T00:08:25.297 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 0, 2024-09-12T00:08:25.297 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 2, 2024-09-12T00:08:25.297 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 1, 2024-09-12T00:08:25.297 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.000265126, 2024-09-12T00:08:25.297 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T00:08:25.297 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.000184766, 2024-09-12T00:08:25.297 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.298 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 0, 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-12T00:08:25.299 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T00:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-12T00:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T00:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T00:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T00:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T00:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T00:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-12T00:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T00:08:25.300 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T00:08:25.301 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-12T00:08:25.301 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-12T00:08:25.301 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-12T00:08:25.301 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-12T00:08:25.301 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T00:08:25.301 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T00:08:25.301 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T00:08:25.301 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T00:08:25.301 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T00:08:25.302 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 148692, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 30, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 0, 2024-09-12T00:08:25.303 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 928, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 2, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-12T00:08:25.304 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 316, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 1425098, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 316, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 316, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 1425098, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-12T00:08:25.305 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-12T00:08:25.306 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:25.306 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-12T00:08:25.306 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-12T00:08:26.008 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-12T00:08:26.009 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-12T00:08:26.009 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-12T00:08:26.009 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-12T00:08:26.009 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-12T00:08:26.009 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T00:08:26.400 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T00:08:26.401 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-12T00:08:26.401 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T00:08:26.431 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-12T00:08:26.431 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T00:08:26.495 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T00:08:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:27.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:26 smithi120 ceph-mon[35182]: pgmap v511: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 7 op/s 2024-09-12T00:08:27.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:26 smithi083 ceph-mon[34805]: pgmap v511: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 7 op/s 2024-09-12T00:08:27.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:26 smithi062 ceph-mon[30958]: pgmap v511: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 7 op/s 2024-09-12T00:08:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.290+0000 7fe706266640 1 -- 172.21.15.62:0/3597562435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe700102d30 msgr2=0x7fe700103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.290+0000 7fe706266640 1 --2- 172.21.15.62:0/3597562435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe700102d30 0x7fe700103110 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009930 tx=0x7fe6f002a420 comp rx=0 tx=0).stop 2024-09-12T00:08:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.291+0000 7fe706266640 1 -- 172.21.15.62:0/3597562435 shutdown_connections 2024-09-12T00:08:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.291+0000 7fe706266640 1 --2- 172.21.15.62:0/3597562435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7001082a0 0x7fe70010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.291+0000 7fe706266640 1 --2- 172.21.15.62:0/3597562435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7001036e0 0x7fe700107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.291+0000 7fe706266640 1 --2- 172.21.15.62:0/3597562435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe700102d30 0x7fe700103110 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.291+0000 7fe706266640 1 -- 172.21.15.62:0/3597562435 >> 172.21.15.62:0/3597562435 conn(0x7fe7000fc7d0 msgr2=0x7fe7000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.291+0000 7fe706266640 1 -- 172.21.15.62:0/3597562435 shutdown_connections 2024-09-12T00:08:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.291+0000 7fe706266640 1 -- 172.21.15.62:0/3597562435 wait complete. 2024-09-12T00:08:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.291+0000 7fe706266640 1 Processor -- start 2024-09-12T00:08:27.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.291+0000 7fe706266640 1 -- start start 2024-09-12T00:08:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe706266640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe700102d30 0x7fe700198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe706266640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7001036e0 0x7fe700198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe706266640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7001082a0 0x7fe70019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe706266640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe700112100 con 0x7fe700102d30 2024-09-12T00:08:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe706266640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe700111f80 con 0x7fe7001036e0 2024-09-12T00:08:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe706266640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe700112280 con 0x7fe7001082a0 2024-09-12T00:08:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe6fffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe700102d30 0x7fe700198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe6fffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe700102d30 0x7fe700198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59972/0 (socket says 172.21.15.62:59972) 2024-09-12T00:08:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe6fffff640 1 -- 172.21.15.62:0/1080752803 learned_addr learned my addr 172.21.15.62:0/1080752803 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe704803640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7001082a0 0x7fe70019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe6ff7fe640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7001036e0 0x7fe700198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe6fffff640 1 -- 172.21.15.62:0/1080752803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7001082a0 msgr2=0x7fe70019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe6fffff640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7001082a0 0x7fe70019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe6fffff640 1 -- 172.21.15.62:0/1080752803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7001036e0 msgr2=0x7fe700198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe6fffff640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7001036e0 0x7fe700198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe6fffff640 1 -- 172.21.15.62:0/1080752803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70019d540 con 0x7fe700102d30 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.292+0000 7fe6fffff640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe700102d30 0x7fe700198590 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe6f002a940 tx=0x7fe6f0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.293+0000 7fe6fd7fa640 1 -- 172.21.15.62:0/1080752803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f003f070 con 0x7fe700102d30 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.293+0000 7fe706266640 1 -- 172.21.15.62:0/1080752803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70019d7d0 con 0x7fe700102d30 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.293+0000 7fe6fd7fa640 1 -- 172.21.15.62:0/1080752803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6f0031c90 con 0x7fe700102d30 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.293+0000 7fe6fd7fa640 1 -- 172.21.15.62:0/1080752803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0035030 con 0x7fe700102d30 2024-09-12T00:08:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.293+0000 7fe706266640 1 -- 172.21.15.62:0/1080752803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7001a5110 con 0x7fe700102d30 2024-09-12T00:08:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.294+0000 7fe706266640 1 -- 172.21.15.62:0/1080752803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe700103110 con 0x7fe700102d30 2024-09-12T00:08:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.296+0000 7fe6fd7fa640 1 -- 172.21.15.62:0/1080752803 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe6f0042070 con 0x7fe700102d30 2024-09-12T00:08:27.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.297+0000 7fe6fd7fa640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe6e80a2cf0 0x7fe6e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:27.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.297+0000 7fe6fd7fa640 1 -- 172.21.15.62:0/1080752803 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8513+0+0 (secure 0 0 0) 0x7fe6f00efb70 con 0x7fe700102d30 2024-09-12T00:08:27.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.297+0000 7fe6ff7fe640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe6e80a2cf0 0x7fe6e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.297+0000 7fe6ff7fe640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe6e80a2cf0 0x7fe6e80a51b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe7000fed40 tx=0x7fe6f400b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:27.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.300+0000 7fe6fd7fa640 1 -- 172.21.15.62:0/1080752803 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6f00bc250 con 0x7fe700102d30 2024-09-12T00:08:27.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.488+0000 7fe706266640 1 -- 172.21.15.62:0/1080752803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe70010a2b0 con 0x7fe700102d30 2024-09-12T00:08:27.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.489+0000 7fe6fd7fa640 1 -- 172.21.15.62:0/1080752803 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v102) ==== 81+0+942 (secure 0 0 0) 0x7fe6f00c0550 con 0x7fe700102d30 2024-09-12T00:08:27.488 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/518295197 2024-09-12T23:52:05.426127+0000 2024-09-12T00:08:27.488 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/3106394779 2024-09-12T23:52:05.426127+0000 2024-09-12T00:08:27.488 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/352172300 2024-09-12T23:42:48.657359+0000 2024-09-12T00:08:27.489 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/1330539138 2024-09-12T23:42:48.657359+0000 2024-09-12T00:08:27.489 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/1330539138 2024-09-12T23:42:48.657359+0000 2024-09-12T00:08:27.489 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2045175448 2024-09-12T23:43:12.720281+0000 2024-09-12T00:08:27.489 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2225508066 2024-09-12T23:42:48.657359+0000 2024-09-12T00:08:27.489 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1351339337 2024-09-12T23:43:12.720281+0000 2024-09-12T00:08:27.490 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/3180930925 2024-09-12T23:43:12.720281+0000 2024-09-12T00:08:27.490 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/908998551 2024-09-12T23:52:05.426127+0000 2024-09-12T00:08:27.490 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/133064408 2024-09-12T23:52:05.426127+0000 2024-09-12T00:08:27.490 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/4057479111 2024-09-12T23:42:48.657359+0000 2024-09-12T00:08:27.490 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/3180930925 2024-09-12T23:43:12.720281+0000 2024-09-12T00:08:27.490 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/3106394779 2024-09-12T23:52:05.426127+0000 2024-09-12T00:08:27.490 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3170436522 2024-09-12T23:43:12.720281+0000 2024-09-12T00:08:27.490 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2894445674 2024-09-12T23:52:05.426127+0000 2024-09-12T00:08:27.491 INFO:teuthology.orchestra.run.smithi062.stderr:listed 16 entries 2024-09-12T00:08:27.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 -- 172.21.15.62:0/1080752803 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe6e80a2cf0 msgr2=0x7fe6e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:27.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe6e80a2cf0 0x7fe6e80a51b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe7000fed40 tx=0x7fe6f400b390 comp rx=0 tx=0).stop 2024-09-12T00:08:27.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 -- 172.21.15.62:0/1080752803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe700102d30 msgr2=0x7fe700198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:27.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe700102d30 0x7fe700198590 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe6f002a940 tx=0x7fe6f0003b90 comp rx=0 tx=0).stop 2024-09-12T00:08:27.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 -- 172.21.15.62:0/1080752803 shutdown_connections 2024-09-12T00:08:27.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7fe6e80a2cf0 0x7fe6e80a51b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:27.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7001082a0 0x7fe70019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:27.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7001036e0 0x7fe700198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:27.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 --2- 172.21.15.62:0/1080752803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe700102d30 0x7fe700198590 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:27.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 -- 172.21.15.62:0/1080752803 >> 172.21.15.62:0/1080752803 conn(0x7fe7000fc7d0 msgr2=0x7fe700106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:27.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 -- 172.21.15.62:0/1080752803 shutdown_connections 2024-09-12T00:08:27.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:27.491+0000 7fe706266640 1 -- 172.21.15.62:0/1080752803 wait complete. 2024-09-12T00:08:28.120 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:27 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1080752803' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T00:08:28.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:27 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1080752803' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T00:08:28.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:27 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1080752803' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T00:08:28.279 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T00:08:28.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:08:28.279 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /tmp/cephfs 2024-09-12T00:08:28.461 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-09-12T00:08:28.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:08:28.461 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-12T00:08:28.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:08:28.529 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-09-12T00:08:28.584 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-12T00:08:28.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:08:28.585 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-12T00:08:28.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:08:28.660 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-09-12T00:08:28.768 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T00:08:28.779 INFO:tasks.cephadm:Teardown begin 2024-09-12T00:08:28.779 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T00:08:28.779 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:08:29.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:28 smithi083 ceph-mon[34805]: pgmap v512: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2024-09-12T00:08:29.198 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:28 smithi062 ceph-mon[30958]: pgmap v512: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2024-09-12T00:08:29.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:28 smithi120 ceph-mon[35182]: pgmap v512: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2024-09-12T00:08:29.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.707+0000 7f60c93b2640 1 -- 172.21.15.62:0/3050785911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4101de0 msgr2=0x7f60c41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:29.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.707+0000 7f60c93b2640 1 --2- 172.21.15.62:0/3050785911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4101de0 0x7f60c41021c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009900 tx=0x7f60b402a410 comp rx=0 tx=0).stop 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.707+0000 7f60c93b2640 1 -- 172.21.15.62:0/3050785911 shutdown_connections 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.707+0000 7f60c93b2640 1 --2- 172.21.15.62:0/3050785911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c4106f10 0x7f60c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.707+0000 7f60c93b2640 1 --2- 172.21.15.62:0/3050785911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4102790 0x7f60c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.707+0000 7f60c93b2640 1 --2- 172.21.15.62:0/3050785911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4101de0 0x7f60c41021c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.707+0000 7f60c93b2640 1 -- 172.21.15.62:0/3050785911 >> 172.21.15.62:0/3050785911 conn(0x7f60c40fd8c0 msgr2=0x7f60c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.708+0000 7f60c93b2640 1 -- 172.21.15.62:0/3050785911 shutdown_connections 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.708+0000 7f60c93b2640 1 -- 172.21.15.62:0/3050785911 wait complete. 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.708+0000 7f60c93b2640 1 Processor -- start 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.708+0000 7f60c93b2640 1 -- start start 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c93b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c4101de0 0x7f60c4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c93b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4102790 0x7f60c41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c93b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4106f10 0x7f60c419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:29.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c93b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f60c4111f30 con 0x7f60c4102790 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c93b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60c4111db0 con 0x7f60c4101de0 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c93b2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60c41120b0 con 0x7f60c4106f10 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c27fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4102790 0x7f60c41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c27fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4102790 0x7f60c41988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59990/0 (socket says 172.21.15.62:59990) 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c27fc640 1 -- 172.21.15.62:0/1922305242 learned_addr learned my addr 172.21.15.62:0/1922305242 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c2ffd640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c4101de0 0x7f60c4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c37fe640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4106f10 0x7f60c419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c27fc640 1 -- 172.21.15.62:0/1922305242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4106f10 msgr2=0x7f60c419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c27fc640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4106f10 0x7f60c419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c27fc640 1 -- 172.21.15.62:0/1922305242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c4101de0 msgr2=0x7f60c4198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c27fc640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c4101de0 0x7f60c4198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c27fc640 1 -- 172.21.15.62:0/1922305242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c419d340 con 0x7f60c4102790 2024-09-12T00:08:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60c27fc640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4102790 0x7f60c41988d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f60b800c920 tx=0x7f60b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.709+0000 7f60abfff640 1 -- 172.21.15.62:0/1922305242 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8007c70 con 0x7f60c4102790 2024-09-12T00:08:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.710+0000 7f60abfff640 1 -- 172.21.15.62:0/1922305242 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60b8007e10 con 0x7f60c4102790 2024-09-12T00:08:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.710+0000 7f60c93b2640 1 -- 172.21.15.62:0/1922305242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c419d630 con 0x7f60c4102790 2024-09-12T00:08:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.710+0000 7f60abfff640 1 -- 172.21.15.62:0/1922305242 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8018890 con 0x7f60c4102790 2024-09-12T00:08:29.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.710+0000 7f60c93b2640 1 -- 172.21.15.62:0/1922305242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c41a4f10 con 0x7f60c4102790 2024-09-12T00:08:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.711+0000 7f60c93b2640 1 -- 172.21.15.62:0/1922305242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c41021c0 con 0x7f60c4102790 2024-09-12T00:08:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.712+0000 7f60abfff640 1 -- 172.21.15.62:0/1922305242 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f60b8018ad0 con 0x7f60c4102790 2024-09-12T00:08:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.713+0000 7f60abfff640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f60ac0a2cf0 0x7f60ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.713+0000 7f60c2ffd640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f60ac0a2cf0 0x7f60ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.713+0000 7f60abfff640 1 -- 172.21.15.62:0/1922305242 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8513+0+0 (secure 0 0 0) 0x7f60b80cf860 con 0x7f60c4102790 2024-09-12T00:08:29.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.713+0000 7f60c2ffd640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f60ac0a2cf0 0x7f60ac0a51b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009230 tx=0x7f60b40058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:29.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.716+0000 7f60abfff640 1 -- 172.21.15.62:0/1922305242 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60b8014030 con 0x7f60c4102790 2024-09-12T00:08:29.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.929+0000 7f60c93b2640 1 -- 172.21.15.62:0/1922305242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60c41996f0 con 0x7f60c4102790 2024-09-12T00:08:29.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.930+0000 7f60abfff640 1 -- 172.21.15.62:0/1922305242 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11914 (secure 0 0 0) 0x7f60c41996f0 con 0x7f60c4102790 2024-09-12T00:08:29.929 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:08:29.929 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":42,"btime":"2024-09-12T00:08:15:840883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:08:15.713627+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:08:29.929 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 42 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 -- 172.21.15.62:0/1922305242 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f60ac0a2cf0 msgr2=0x7f60ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f60ac0a2cf0 0x7f60ac0a51b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009230 tx=0x7f60b40058d0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 -- 172.21.15.62:0/1922305242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4102790 msgr2=0x7f60c41988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4102790 0x7f60c41988d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f60b800c920 tx=0x7f60b800cde0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 -- 172.21.15.62:0/1922305242 shutdown_connections 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f60ac0a2cf0 0x7f60ac0a51b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60c4106f10 0x7f60c419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4102790 0x7f60c41988d0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 --2- 172.21.15.62:0/1922305242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c4101de0 0x7f60c4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 -- 172.21.15.62:0/1922305242 >> 172.21.15.62:0/1922305242 conn(0x7f60c40fd8c0 msgr2=0x7f60c40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 -- 172.21.15.62:0/1922305242 shutdown_connections 2024-09-12T00:08:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:29.933+0000 7f60c93b2640 1 -- 172.21.15.62:0/1922305242 wait complete. 2024-09-12T00:08:30.724 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:08:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:30 smithi062 ceph-mon[30958]: pgmap v513: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 6 op/s 2024-09-12T00:08:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:30 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1922305242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:08:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:30 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:08:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:30 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:08:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:30 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:08:30.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:30 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:08:30.981 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:31.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:30 smithi083 ceph-mon[34805]: pgmap v513: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 6 op/s 2024-09-12T00:08:31.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:30 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1922305242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:08:31.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:30 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:08:31.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:30 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:08:31.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:30 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:08:31.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:30 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:08:31.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:30 smithi120 ceph-mon[35182]: pgmap v513: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 6 op/s 2024-09-12T00:08:31.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:30 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1922305242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:08:31.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:30 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:08:31.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:30 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:08:31.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:30 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:08:31.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:30 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' 2024-09-12T00:08:31.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.484+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3585997907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54102570 msgr2=0x7f4a541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.484+0000 7f4a5c13d640 1 --2- 172.21.15.62:0/3585997907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54102570 0x7f4a541029b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009900 tx=0x7f4a4802a480 comp rx=0 tx=0).stop 2024-09-12T00:08:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.484+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3585997907 shutdown_connections 2024-09-12T00:08:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.484+0000 7f4a5c13d640 1 --2- 172.21.15.62:0/3585997907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a54102ef0 0x7f4a5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.484+0000 7f4a5c13d640 1 --2- 172.21.15.62:0/3585997907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54102570 0x7f4a541029b0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.484+0000 7f4a5c13d640 1 --2- 172.21.15.62:0/3585997907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a54100e20 0x7f4a54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.485+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3585997907 >> 172.21.15.62:0/3585997907 conn(0x7f4a540fc7d0 msgr2=0x7f4a540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.485+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3585997907 shutdown_connections 2024-09-12T00:08:31.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.485+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3585997907 wait complete. 2024-09-12T00:08:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.485+0000 7f4a5c13d640 1 Processor -- start 2024-09-12T00:08:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.485+0000 7f4a5c13d640 1 -- start start 2024-09-12T00:08:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.485+0000 7f4a5c13d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a54100e20 0x7f4a541984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.485+0000 7f4a5c13d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54102570 0x7f4a541989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.485+0000 7f4a5c13d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a54102ef0 0x7f4a5419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a5c13d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54112060 con 0x7f4a54102570 2024-09-12T00:08:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a5c13d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54111ee0 con 0x7f4a54100e20 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a5c13d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541121e0 con 0x7f4a54102ef0 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a596d8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54102570 0x7f4a541989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a596d8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54102570 0x7f4a541989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58632/0 (socket says 172.21.15.62:58632) 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a596d8640 1 -- 172.21.15.62:0/3383081249 learned_addr learned my addr 172.21.15.62:0/3383081249 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a5a6da640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a54102ef0 0x7f4a5419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a59ed9640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a54100e20 0x7f4a541984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a596d8640 1 -- 172.21.15.62:0/3383081249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a54102ef0 msgr2=0x7f4a5419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a596d8640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a54102ef0 0x7f4a5419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a596d8640 1 -- 172.21.15.62:0/3383081249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a54100e20 msgr2=0x7f4a541984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a596d8640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a54100e20 0x7f4a541984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a596d8640 1 -- 172.21.15.62:0/3383081249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5419d450 con 0x7f4a54102570 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a596d8640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54102570 0x7f4a541989e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006010 tx=0x7f4a48003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:31.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a52ffd640 1 -- 172.21.15.62:0/3383081249 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4803f070 con 0x7f4a54102570 2024-09-12T00:08:31.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3383081249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5419d6e0 con 0x7f4a54102570 2024-09-12T00:08:31.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a52ffd640 1 -- 172.21.15.62:0/3383081249 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a48031c30 con 0x7f4a54102570 2024-09-12T00:08:31.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.486+0000 7f4a52ffd640 1 -- 172.21.15.62:0/3383081249 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48035030 con 0x7f4a54102570 2024-09-12T00:08:31.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.487+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3383081249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a541a5020 con 0x7f4a54102570 2024-09-12T00:08:31.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.488+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3383081249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a54101200 con 0x7f4a54102570 2024-09-12T00:08:31.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.489+0000 7f4a52ffd640 1 -- 172.21.15.62:0/3383081249 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4a48042070 con 0x7f4a54102570 2024-09-12T00:08:31.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.489+0000 7f4a52ffd640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a3c0a2cf0 0x7f4a3c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:31.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.489+0000 7f4a59ed9640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a3c0a2cf0 0x7f4a3c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:31.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.489+0000 7f4a52ffd640 1 -- 172.21.15.62:0/3383081249 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8513+0+0 (secure 0 0 0) 0x7f4a480ef820 con 0x7f4a54102570 2024-09-12T00:08:31.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.490+0000 7f4a59ed9640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a3c0a2cf0 0x7f4a3c0a51b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4a44005fd0 tx=0x7f4a44009870 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.494+0000 7f4a52ffd640 1 -- 172.21.15.62:0/3383081249 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a480bc440 con 0x7f4a54102570 2024-09-12T00:08:31.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.705+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3383081249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a54109090 con 0x7f4a54102570 2024-09-12T00:08:31.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.706+0000 7f4a52ffd640 1 -- 172.21.15.62:0/3383081249 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11914 (secure 0 0 0) 0x7f4a480bbd40 con 0x7f4a54102570 2024-09-12T00:08:31.705 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:08:31.705 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":42,"btime":"2024-09-12T00:08:15:840883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1692157360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1692157360},{"type":"v1","addr":"172.21.15.62:6835","nonce":1692157360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1495913627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1495913627},{"type":"v1","addr":"172.21.15.62:6837","nonce":1495913627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/392450380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":392450380},{"type":"v1","addr":"172.21.15.62:6839","nonce":392450380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/4034635892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":4034635892},{"type":"v1","addr":"172.21.15.62:6841","nonce":4034635892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3300661503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3300661503},{"type":"v1","addr":"172.21.15.83:6833","nonce":3300661503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1221552790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1221552790},{"type":"v1","addr":"172.21.15.120:6833","nonce":1221552790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:50:11.406773+0000","modified":"2024-09-12T00:08:15.713627+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24484,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3040146550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3040146550},{"type":"v1","addr":"172.21.15.83:6835","nonce":3040146550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/337493680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":337493680},{"type":"v1","addr":"172.21.15.83:6837","nonce":337493680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/2559115046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2559115046},{"type":"v1","addr":"172.21.15.120:6835","nonce":2559115046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.120:6837/1796168792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1796168792},{"type":"v1","addr":"172.21.15.120:6837","nonce":1796168792}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.120:6839/2283411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":2283411896},{"type":"v1","addr":"172.21.15.120:6839","nonce":2283411896}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6839/2530997346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2530997346},{"type":"v1","addr":"172.21.15.83:6839","nonce":2530997346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24484,24488]},"id":1}]} 2024-09-12T00:08:31.705 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 42 2024-09-12T00:08:31.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3383081249 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a3c0a2cf0 msgr2=0x7f4a3c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:31.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a3c0a2cf0 0x7f4a3c0a51b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4a44005fd0 tx=0x7f4a44009870 comp rx=0 tx=0).stop 2024-09-12T00:08:31.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3383081249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54102570 msgr2=0x7f4a541989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:31.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54102570 0x7f4a541989e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006010 tx=0x7f4a48003c10 comp rx=0 tx=0).stop 2024-09-12T00:08:31.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3383081249 shutdown_connections 2024-09-12T00:08:31.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f4a3c0a2cf0 0x7f4a3c0a51b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:31.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a54102ef0 0x7f4a5419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:31.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54102570 0x7f4a541989e0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:31.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 --2- 172.21.15.62:0/3383081249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a54100e20 0x7f4a541984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:31.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3383081249 >> 172.21.15.62:0/3383081249 conn(0x7f4a540fc7d0 msgr2=0x7f4a540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:31.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3383081249 shutdown_connections 2024-09-12T00:08:31.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:31.708+0000 7f4a5c13d640 1 -- 172.21.15.62:0/3383081249 wait complete. 2024-09-12T00:08:31.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:31 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3383081249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:08:32.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:31 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3383081249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:08:32.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:31 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3383081249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:08:32.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:08:32.705 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:33.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:32 smithi062 ceph-mon[30958]: pgmap v514: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 6 op/s 2024-09-12T00:08:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.153+0000 7f66e5b55640 1 -- 172.21.15.62:0/15340076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0102570 msgr2=0x7f66e01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.153+0000 7f66e5b55640 1 --2- 172.21.15.62:0/15340076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0102570 0x7f66e01029b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009900 tx=0x7f66d402a480 comp rx=0 tx=0).stop 2024-09-12T00:08:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.153+0000 7f66e5b55640 1 -- 172.21.15.62:0/15340076 shutdown_connections 2024-09-12T00:08:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.153+0000 7f66e5b55640 1 --2- 172.21.15.62:0/15340076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e0102ef0 0x7f66e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.153+0000 7f66e5b55640 1 --2- 172.21.15.62:0/15340076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0102570 0x7f66e01029b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.153+0000 7f66e5b55640 1 --2- 172.21.15.62:0/15340076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66e0100e20 0x7f66e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.153+0000 7f66e5b55640 1 -- 172.21.15.62:0/15340076 >> 172.21.15.62:0/15340076 conn(0x7f66e00fc7d0 msgr2=0x7f66e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.153+0000 7f66e5b55640 1 -- 172.21.15.62:0/15340076 shutdown_connections 2024-09-12T00:08:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.153+0000 7f66e5b55640 1 -- 172.21.15.62:0/15340076 wait complete. 2024-09-12T00:08:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66e5b55640 1 Processor -- start 2024-09-12T00:08:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66e5b55640 1 -- start start 2024-09-12T00:08:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66e5b55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e0100e20 0x7f66e0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66e5b55640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0102570 0x7f66e0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66e5b55640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66e0102ef0 0x7f66e019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66e5b55640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0112130 con 0x7f66e0102570 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66e5b55640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0111fb0 con 0x7f66e0100e20 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66e5b55640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01122b0 con 0x7f66e0102ef0 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66deffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0102570 0x7f66e0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66deffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0102570 0x7f66e0198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58650/0 (socket says 172.21.15.62:58650) 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66deffd640 1 -- 172.21.15.62:0/2674920906 learned_addr learned my addr 172.21.15.62:0/2674920906 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66df7fe640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e0100e20 0x7f66e0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66dffff640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66e0102ef0 0x7f66e019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66deffd640 1 -- 172.21.15.62:0/2674920906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66e0102ef0 msgr2=0x7f66e019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.154+0000 7f66deffd640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66e0102ef0 0x7f66e019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66deffd640 1 -- 172.21.15.62:0/2674920906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e0100e20 msgr2=0x7f66e0198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66deffd640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e0100e20 0x7f66e0198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66deffd640 1 -- 172.21.15.62:0/2674920906 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e019d600 con 0x7f66e0102570 2024-09-12T00:08:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66dffff640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66e0102ef0 0x7f66e019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:08:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66df7fe640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e0100e20 0x7f66e0198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:08:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66deffd640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0102570 0x7f66e0198b90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f66d4006010 tx=0x7f66d4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66dcff9640 1 -- 172.21.15.62:0/2674920906 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d403f070 con 0x7f66e0102570 2024-09-12T00:08:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66dcff9640 1 -- 172.21.15.62:0/2674920906 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66d4031e30 con 0x7f66e0102570 2024-09-12T00:08:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66dcff9640 1 -- 172.21.15.62:0/2674920906 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4035030 con 0x7f66e0102570 2024-09-12T00:08:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66e5b55640 1 -- 172.21.15.62:0/2674920906 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e019d890 con 0x7f66e0102570 2024-09-12T00:08:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.155+0000 7f66e5b55640 1 -- 172.21.15.62:0/2674920906 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e01a51d0 con 0x7f66e0102570 2024-09-12T00:08:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.156+0000 7f66e5b55640 1 -- 172.21.15.62:0/2674920906 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e0101280 con 0x7f66e0102570 2024-09-12T00:08:33.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.157+0000 7f66dcff9640 1 -- 172.21.15.62:0/2674920906 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f66d4042070 con 0x7f66e0102570 2024-09-12T00:08:33.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.158+0000 7f66dcff9640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f66c80a2cf0 0x7f66c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:33.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.158+0000 7f66df7fe640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f66c80a2cf0 0x7f66c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:33.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.158+0000 7f66dcff9640 1 -- 172.21.15.62:0/2674920906 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8513+0+0 (secure 0 0 0) 0x7f66d40f0440 con 0x7f66e0102570 2024-09-12T00:08:33.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.159+0000 7f66df7fe640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f66c80a2cf0 0x7f66c80a51b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009930 tx=0x7f66d0008040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:33.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.162+0000 7f66dcff9640 1 -- 172.21.15.62:0/2674920906 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66d40bd0f0 con 0x7f66e0102570 2024-09-12T00:08:33.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:32 smithi083 ceph-mon[34805]: pgmap v514: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 6 op/s 2024-09-12T00:08:33.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.338+0000 7f66e5b55640 1 -- 172.21.15.62:0/2674920906 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66e0109090 con 0x7f66e0102570 2024-09-12T00:08:33.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.340+0000 7f66dcff9640 1 -- 172.21.15.62:0/2674920906 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+23089 (secure 0 0 0) 0x7f66d40bc9f0 con 0x7f66e0102570 2024-09-12T00:08:33.339 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-12T00:08:33.339 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":102,"fsid":"56675698-7097-11ef-bcea-c7b262605968","created":"2024-09-11T23:42:20.574451+0000","modified":"2024-09-12T00:08:11.832699+0000","last_up_change":"2024-09-11T23:49:19.643515+0000","last_in_change":"2024-09-11T23:48:55.648490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:45:29.220992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:50:07.698272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:50:09.813785+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ed3595-d3b7-44d6-8aa3-1896f7602607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6803","nonce":3944758612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6805","nonce":3944758612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6809","nonce":3944758612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3944758612},{"type":"v1","addr":"172.21.15.62:6807","nonce":3944758612}]},"public_addr":"172.21.15.62:6803/3944758612","cluster_addr":"172.21.15.62:6805/3944758612","heartbeat_back_addr":"172.21.15.62:6809/3944758612","heartbeat_front_addr":"172.21.15.62:6807/3944758612","state":["exists","up"]},{"osd":1,"uuid":"f352db6b-9b13-4d9b-9a26-7d6f16e9dd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6801","nonce":2108490342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6803","nonce":2108490342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6807","nonce":2108490342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2108490342},{"type":"v1","addr":"172.21.15.83:6805","nonce":2108490342}]},"public_addr":"172.21.15.83:6801/2108490342","cluster_addr":"172.21.15.83:6803/2108490342","heartbeat_back_addr":"172.21.15.83:6807/2108490342","heartbeat_front_addr":"172.21.15.83:6805/2108490342","state":["exists","up"]},{"osd":2,"uuid":"5f110d93-58ae-4b7b-9d7f-93a25f5bbc6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6801","nonce":3994166921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6803","nonce":3994166921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6807","nonce":3994166921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3994166921},{"type":"v1","addr":"172.21.15.120:6805","nonce":3994166921}]},"public_addr":"172.21.15.120:6801/3994166921","cluster_addr":"172.21.15.120:6803/3994166921","heartbeat_back_addr":"172.21.15.120:6807/3994166921","heartbeat_front_addr":"172.21.15.120:6805/3994166921","state":["exists","up"]},{"osd":3,"uuid":"37bbe64a-1332-4aee-83f4-67fb8f0092b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6811","nonce":1879962337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6813","nonce":1879962337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6817","nonce":1879962337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1879962337},{"type":"v1","addr":"172.21.15.62:6815","nonce":1879962337}]},"public_addr":"172.21.15.62:6811/1879962337","cluster_addr":"172.21.15.62:6813/1879962337","heartbeat_back_addr":"172.21.15.62:6817/1879962337","heartbeat_front_addr":"172.21.15.62:6815/1879962337","state":["exists","up"]},{"osd":4,"uuid":"8cb698b7-3471-4503-8039-6519c4d2943e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6809","nonce":3966827611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6811","nonce":3966827611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6815","nonce":3966827611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3966827611},{"type":"v1","addr":"172.21.15.83:6813","nonce":3966827611}]},"public_addr":"172.21.15.83:6809/3966827611","cluster_addr":"172.21.15.83:6811/3966827611","heartbeat_back_addr":"172.21.15.83:6815/3966827611","heartbeat_front_addr":"172.21.15.83:6813/3966827611","state":["exists","up"]},{"osd":5,"uuid":"ce904ade-27b5-4171-af4c-7c4894f40c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6809","nonce":186624354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6811","nonce":186624354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6815","nonce":186624354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":186624354},{"type":"v1","addr":"172.21.15.120:6813","nonce":186624354}]},"public_addr":"172.21.15.120:6809/186624354","cluster_addr":"172.21.15.120:6811/186624354","heartbeat_back_addr":"172.21.15.120:6815/186624354","heartbeat_front_addr":"172.21.15.120:6813/186624354","state":["exists","up"]},{"osd":6,"uuid":"7ea78a9f-4447-413d-a698-12bf6a2f1398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6819","nonce":461634356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6821","nonce":461634356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6825","nonce":461634356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":461634356},{"type":"v1","addr":"172.21.15.62:6823","nonce":461634356}]},"public_addr":"172.21.15.62:6819/461634356","cluster_addr":"172.21.15.62:6821/461634356","heartbeat_back_addr":"172.21.15.62:6825/461634356","heartbeat_front_addr":"172.21.15.62:6823/461634356","state":["exists","up"]},{"osd":7,"uuid":"9eb34260-3535-4e62-8aa5-276282a679cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138900766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6819","nonce":3138900766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6823","nonce":3138900766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3138900766},{"type":"v1","addr":"172.21.15.83:6821","nonce":3138900766}]},"public_addr":"172.21.15.83:6817/3138900766","cluster_addr":"172.21.15.83:6819/3138900766","heartbeat_back_addr":"172.21.15.83:6823/3138900766","heartbeat_front_addr":"172.21.15.83:6821/3138900766","state":["exists","up"]},{"osd":8,"uuid":"f2244a62-94a3-4ebf-9e65-aadfc025c3c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6817","nonce":1046771944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6819","nonce":1046771944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6823","nonce":1046771944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1046771944},{"type":"v1","addr":"172.21.15.120:6821","nonce":1046771944}]},"public_addr":"172.21.15.120:6817/1046771944","cluster_addr":"172.21.15.120:6819/1046771944","heartbeat_back_addr":"172.21.15.120:6823/1046771944","heartbeat_front_addr":"172.21.15.120:6821/1046771944","state":["exists","up"]},{"osd":9,"uuid":"ce0d8be5-9f8a-449b-a52a-e6ccda54e2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6827","nonce":343213579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6829","nonce":343213579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6833","nonce":343213579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":343213579},{"type":"v1","addr":"172.21.15.62:6831","nonce":343213579}]},"public_addr":"172.21.15.62:6827/343213579","cluster_addr":"172.21.15.62:6829/343213579","heartbeat_back_addr":"172.21.15.62:6833/343213579","heartbeat_front_addr":"172.21.15.62:6831/343213579","state":["exists","up"]},{"osd":10,"uuid":"70f5b28b-2b5f-4c39-b8b7-fdda706fa3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6825","nonce":4134153557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6827","nonce":4134153557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6831","nonce":4134153557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":4134153557},{"type":"v1","addr":"172.21.15.83:6829","nonce":4134153557}]},"public_addr":"172.21.15.83:6825/4134153557","cluster_addr":"172.21.15.83:6827/4134153557","heartbeat_back_addr":"172.21.15.83:6831/4134153557","heartbeat_front_addr":"172.21.15.83:6829/4134153557","state":["exists","up"]},{"osd":11,"uuid":"37bcf111-5dde-49c5-8f40-0a1a5644a081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6825","nonce":4266702819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6827","nonce":4266702819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6831","nonce":4266702819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":4266702819},{"type":"v1","addr":"172.21.15.120:6829","nonce":4266702819}]},"public_addr":"172.21.15.120:6825/4266702819","cluster_addr":"172.21.15.120:6827/4266702819","heartbeat_back_addr":"172.21.15.120:6831/4266702819","heartbeat_front_addr":"172.21.15.120:6829/4266702819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:44:35.871616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:00.822676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:26.911985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:45:50.894511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:17.081771+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:46:42.298934+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:08.836275+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:47:34.045565+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:00.242230+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:26.024545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:48:51.500787+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:49:17.830500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/518295197":"2024-09-12T23:52:05.426127+0000","172.21.15.62:6801/3106394779":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/352172300":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6801/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6800/1330539138":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/2045175448":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2225508066":"2024-09-12T23:42:48.657359+0000","172.21.15.62:0/1351339337":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6800/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/908998551":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/133064408":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/4057479111":"2024-09-12T23:42:48.657359+0000","172.21.15.62:6801/3180930925":"2024-09-12T23:43:12.720281+0000","172.21.15.62:6800/3106394779":"2024-09-12T23:52:05.426127+0000","172.21.15.62:0/3170436522":"2024-09-12T23:43:12.720281+0000","172.21.15.62:0/2894445674":"2024-09-12T23:52:05.426127+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":26,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:08:33.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 -- 172.21.15.62:0/2674920906 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f66c80a2cf0 msgr2=0x7f66c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:33.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f66c80a2cf0 0x7f66c80a51b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009930 tx=0x7f66d0008040 comp rx=0 tx=0).stop 2024-09-12T00:08:33.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 -- 172.21.15.62:0/2674920906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0102570 msgr2=0x7f66e0198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:33.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0102570 0x7f66e0198b90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f66d4006010 tx=0x7f66d4003c80 comp rx=0 tx=0).stop 2024-09-12T00:08:33.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 -- 172.21.15.62:0/2674920906 shutdown_connections 2024-09-12T00:08:33.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f66c80a2cf0 0x7f66c80a51b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:33.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66e0102ef0 0x7f66e019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:33.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0102570 0x7f66e0198b90 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:33.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 --2- 172.21.15.62:0/2674920906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e0100e20 0x7f66e0198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:33.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 -- 172.21.15.62:0/2674920906 >> 172.21.15.62:0/2674920906 conn(0x7f66e00fc7d0 msgr2=0x7f66e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:33.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 -- 172.21.15.62:0/2674920906 shutdown_connections 2024-09-12T00:08:33.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:33.342+0000 7f66e5b55640 1 -- 172.21.15.62:0/2674920906 wait complete. 2024-09-12T00:08:33.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:32 smithi120 ceph-mon[35182]: pgmap v514: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 6 op/s 2024-09-12T00:08:34.081 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T00:08:34.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:33 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2674920906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:08:34.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:33 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2674920906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:08:34.339 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:34.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:33 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2674920906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:08:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.805+0000 7f6221c35640 1 -- 172.21.15.62:0/765986746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f621c103660 msgr2=0x7f621c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:34.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.805+0000 7f6221c35640 1 --2- 172.21.15.62:0/765986746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f621c103660 0x7f621c10bb50 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6210009900 tx=0x7f621002a480 comp rx=0 tx=0).stop 2024-09-12T00:08:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.805+0000 7f6221c35640 1 -- 172.21.15.62:0/765986746 shutdown_connections 2024-09-12T00:08:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.805+0000 7f6221c35640 1 --2- 172.21.15.62:0/765986746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f621c10c170 0x7f621c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.805+0000 7f6221c35640 1 --2- 172.21.15.62:0/765986746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f621c103660 0x7f621c10bb50 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.805+0000 7f6221c35640 1 --2- 172.21.15.62:0/765986746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f621c102d40 0x7f621c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.805+0000 7f6221c35640 1 -- 172.21.15.62:0/765986746 >> 172.21.15.62:0/765986746 conn(0x7f621c0fc7a0 msgr2=0x7f621c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.805+0000 7f6221c35640 1 -- 172.21.15.62:0/765986746 shutdown_connections 2024-09-12T00:08:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.805+0000 7f6221c35640 1 -- 172.21.15.62:0/765986746 wait complete. 2024-09-12T00:08:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.806+0000 7f6221c35640 1 Processor -- start 2024-09-12T00:08:34.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.806+0000 7f6221c35640 1 -- start start 2024-09-12T00:08:34.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.806+0000 7f6221c35640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f621c102d40 0x7f621c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:34.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.806+0000 7f6221c35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f621c103660 0x7f621c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:34.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.806+0000 7f6221c35640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f621c10c170 0x7f621c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:34.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.806+0000 7f6221c35640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f621c111fb0 con 0x7f621c10c170 2024-09-12T00:08:34.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.806+0000 7f6221c35640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f621c111e30 con 0x7f621c103660 2024-09-12T00:08:34.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.806+0000 7f6221c35640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f621c112130 con 0x7f621c102d40 2024-09-12T00:08:34.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.806+0000 7f621bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f621c10c170 0x7f621c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:34.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f621c10c170 0x7f621c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58674/0 (socket says 172.21.15.62:58674) 2024-09-12T00:08:34.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621bfff640 1 -- 172.21.15.62:0/1019674951 learned_addr learned my addr 172.21.15.62:0/1019674951 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621affd640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f621c103660 0x7f621c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621b7fe640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f621c102d40 0x7f621c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621bfff640 1 -- 172.21.15.62:0/1019674951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f621c102d40 msgr2=0x7f621c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621bfff640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f621c102d40 0x7f621c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621bfff640 1 -- 172.21.15.62:0/1019674951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f621c103660 msgr2=0x7f621c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621bfff640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f621c103660 0x7f621c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621bfff640 1 -- 172.21.15.62:0/1019674951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c19d3e0 con 0x7f621c10c170 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621affd640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f621c103660 0x7f621c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f621bfff640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f621c10c170 0x7f621c19cd00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f621400cce0 tx=0x7f621400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f6218ff9640 1 -- 172.21.15.62:0/1019674951 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621400f450 con 0x7f621c10c170 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f6221c35640 1 -- 172.21.15.62:0/1019674951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621c19d6d0 con 0x7f621c10c170 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f6218ff9640 1 -- 172.21.15.62:0/1019674951 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621400fad0 con 0x7f621c10c170 2024-09-12T00:08:34.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f6218ff9640 1 -- 172.21.15.62:0/1019674951 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214018c10 con 0x7f621c10c170 2024-09-12T00:08:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.807+0000 7f6221c35640 1 -- 172.21.15.62:0/1019674951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621c1a4fb0 con 0x7f621c10c170 2024-09-12T00:08:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.808+0000 7f6221c35640 1 -- 172.21.15.62:0/1019674951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621c103120 con 0x7f621c10c170 2024-09-12T00:08:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.810+0000 7f6218ff9640 1 -- 172.21.15.62:0/1019674951 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f621400f5f0 con 0x7f621c10c170 2024-09-12T00:08:34.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.810+0000 7f6218ff9640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f62040a2cf0 0x7f62040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:34.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.810+0000 7f621b7fe640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f62040a2cf0 0x7f62040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:34.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.810+0000 7f6218ff9640 1 -- 172.21.15.62:0/1019674951 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8513+0+0 (secure 0 0 0) 0x7f62140cfcb0 con 0x7f621c10c170 2024-09-12T00:08:34.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.811+0000 7f621b7fe640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f62040a2cf0 0x7f62040a51b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f620c009890 tx=0x7f620c008040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:34.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:34.813+0000 7f6218ff9640 1 -- 172.21.15.62:0/1019674951 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6214014030 con 0x7f621c10c170 2024-09-12T00:08:35.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.026+0000 7f6221c35640 1 -- 172.21.15.62:0/1019674951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f621c1a5290 con 0x7f621c10c170 2024-09-12T00:08:35.025 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:34 smithi062 ceph-mon[30958]: pgmap v515: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2024-09-12T00:08:35.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.028+0000 7f6218ff9640 1 -- 172.21.15.62:0/1019674951 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7f621409c900 con 0x7f621c10c170 2024-09-12T00:08:35.027 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T00:08:35.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 -- 172.21.15.62:0/1019674951 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f62040a2cf0 msgr2=0x7f62040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:35.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f62040a2cf0 0x7f62040a51b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f620c009890 tx=0x7f620c008040 comp rx=0 tx=0).stop 2024-09-12T00:08:35.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 -- 172.21.15.62:0/1019674951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f621c10c170 msgr2=0x7f621c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:35.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f621c10c170 0x7f621c19cd00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f621400cce0 tx=0x7f621400a620 comp rx=0 tx=0).stop 2024-09-12T00:08:35.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 -- 172.21.15.62:0/1019674951 shutdown_connections 2024-09-12T00:08:35.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f62040a2cf0 0x7f62040a51b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:35.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f621c10c170 0x7f621c19cd00 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:35.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f621c103660 0x7f621c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:35.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 --2- 172.21.15.62:0/1019674951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f621c102d40 0x7f621c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:35.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 -- 172.21.15.62:0/1019674951 >> 172.21.15.62:0/1019674951 conn(0x7f621c0fc7a0 msgr2=0x7f621c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:35.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 -- 172.21.15.62:0/1019674951 shutdown_connections 2024-09-12T00:08:35.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:35.032+0000 7f6221c35640 1 -- 172.21.15.62:0/1019674951 wait complete. 2024-09-12T00:08:35.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:34 smithi083 ceph-mon[34805]: pgmap v515: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2024-09-12T00:08:35.167 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b[64402]: ignoring --setuser ceph since I am not root 2024-09-12T00:08:35.167 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b[64402]: ignoring --setgroup ceph since I am not root 2024-09-12T00:08:35.167 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-mds[64406]: -- 172.21.15.83:0/3341437332 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56330aacbc20 con 0x56330aaa8800 2024-09-12T00:08:35.167 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b[64402]: starting mds.b at 2024-09-12T00:08:35.167 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e[63786]: ignoring --setuser ceph since I am not root 2024-09-12T00:08:35.168 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e[63786]: ignoring --setgroup ceph since I am not root 2024-09-12T00:08:35.168 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-mds[63790]: -- 172.21.15.83:0/3101850723 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646ff529860 con 0x5646ff507400 2024-09-12T00:08:35.168 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e[63786]: starting mds.e at 2024-09-12T00:08:35.168 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h[64086]: ignoring --setuser ceph since I am not root 2024-09-12T00:08:35.168 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h[64086]: ignoring --setgroup ceph since I am not root 2024-09-12T00:08:35.168 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-mds[64090]: -- 172.21.15.83:0/4291530238 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559069fd860 con 0x5559069db400 2024-09-12T00:08:35.168 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h[64086]: starting mds.h at 2024-09-12T00:08:35.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:34 smithi120 ceph-mon[35182]: pgmap v515: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2024-09-12T00:08:35.370 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c[65607]: ignoring --setuser ceph since I am not root 2024-09-12T00:08:35.371 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c[65607]: ignoring --setgroup ceph since I am not root 2024-09-12T00:08:35.371 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-mds[65611]: -- 172.21.15.120:0/3756257504 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b1c0697860 con 0x55b1c0675000 2024-09-12T00:08:35.371 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c[65607]: starting mds.c at 2024-09-12T00:08:35.371 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i[65905]: ignoring --setuser ceph since I am not root 2024-09-12T00:08:35.371 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i[65905]: ignoring --setgroup ceph since I am not root 2024-09-12T00:08:35.371 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-mds[65909]: -- 172.21.15.120:0/326493063 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56022dc69860 con 0x56022dc47400 2024-09-12T00:08:35.371 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i[65905]: starting mds.i at 2024-09-12T00:08:35.372 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-f[65307]: ignoring --setuser ceph since I am not root 2024-09-12T00:08:35.372 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-f[65307]: ignoring --setgroup ceph since I am not root 2024-09-12T00:08:35.372 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-mds[65311]: -- 172.21.15.120:0/584123104 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d52609f860 con 0x55d52607c800 2024-09-12T00:08:35.372 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-f[65307]: starting mds.f at 2024-09-12T00:08:35.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:35 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a[30954]: 2024-09-12T00:08:35.027+0000 7f190b1a8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T00:08:35.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T00:08:36.114 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:36.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1019674951' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T00:08:36.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-mon[34805]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T00:08:36.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-mon[34805]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T00:08:36.167 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/1019674951' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T00:08:36.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-mon[34805]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T00:08:36.168 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:35 smithi083 ceph-mon[34805]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T00:08:36.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:35 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1019674951' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T00:08:36.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:35 smithi062 ceph-mon[30958]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T00:08:36.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:35 smithi062 ceph-mon[30958]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T00:08:36.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:35 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/1019674951' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T00:08:36.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:35 smithi062 ceph-mon[30958]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T00:08:36.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:35 smithi062 ceph-mon[30958]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T00:08:36.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1019674951' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T00:08:36.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-mon[35182]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T00:08:36.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-mon[35182]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T00:08:36.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/1019674951' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T00:08:36.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-mon[35182]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T00:08:36.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:35 smithi120 ceph-mon[35182]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T00:08:36.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.592+0000 7f35aff3a640 1 -- 172.21.15.62:0/1094897280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8106a80 msgr2=0x7f35a8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:36.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.592+0000 7f35aff3a640 1 --2- 172.21.15.62:0/1094897280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8106a80 0x7f35a8106e60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009930 tx=0x7f35a402a420 comp rx=0 tx=0).stop 2024-09-12T00:08:36.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.592+0000 7f35aff3a640 1 -- 172.21.15.62:0/1094897280 shutdown_connections 2024-09-12T00:08:36.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.592+0000 7f35aff3a640 1 --2- 172.21.15.62:0/1094897280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a8101420 0x7f35a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:36.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.592+0000 7f35aff3a640 1 --2- 172.21.15.62:0/1094897280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a8100a80 0x7f35a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:36.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.592+0000 7f35aff3a640 1 --2- 172.21.15.62:0/1094897280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8106a80 0x7f35a8106e60 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.592+0000 7f35aff3a640 1 -- 172.21.15.62:0/1094897280 >> 172.21.15.62:0/1094897280 conn(0x7f35a80fc7a0 msgr2=0x7f35a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.592+0000 7f35aff3a640 1 -- 172.21.15.62:0/1094897280 shutdown_connections 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.592+0000 7f35aff3a640 1 -- 172.21.15.62:0/1094897280 wait complete. 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35aff3a640 1 Processor -- start 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35aff3a640 1 -- start start 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35aff3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a8100a80 0x7f35a8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35aff3a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8101420 0x7f35a8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35aff3a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a8106a80 0x7f35a819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35aff3a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8110e40 con 0x7f35a8101420 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35aff3a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8110cc0 con 0x7f35a8100a80 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35aff3a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8110fc0 con 0x7f35a8106a80 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35ad4d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8101420 0x7f35a8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35ad4d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8101420 0x7f35a8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58694/0 (socket says 172.21.15.62:58694) 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35ad4d5640 1 -- 172.21.15.62:0/3106101029 learned_addr learned my addr 172.21.15.62:0/3106101029 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35adcd6640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a8100a80 0x7f35a8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35ad4d5640 1 -- 172.21.15.62:0/3106101029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a8106a80 msgr2=0x7f35a819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35ad4d5640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a8106a80 0x7f35a819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35ad4d5640 1 -- 172.21.15.62:0/3106101029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a8100a80 msgr2=0x7f35a8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35ad4d5640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a8100a80 0x7f35a8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.593+0000 7f35ad4d5640 1 -- 172.21.15.62:0/3106101029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a819d3f0 con 0x7f35a8101420 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.594+0000 7f35ad4d5640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8101420 0x7f35a8198980 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f359800c920 tx=0x7f359800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.594+0000 7f359effd640 1 -- 172.21.15.62:0/3106101029 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598007c70 con 0x7f35a8101420 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.594+0000 7f359effd640 1 -- 172.21.15.62:0/3106101029 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3598007e10 con 0x7f35a8101420 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.594+0000 7f35aff3a640 1 -- 172.21.15.62:0/3106101029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a819d6e0 con 0x7f35a8101420 2024-09-12T00:08:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.594+0000 7f359effd640 1 -- 172.21.15.62:0/3106101029 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598018890 con 0x7f35a8101420 2024-09-12T00:08:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.594+0000 7f35aff3a640 1 -- 172.21.15.62:0/3106101029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81a4fc0 con 0x7f35a8101420 2024-09-12T00:08:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.595+0000 7f35aff3a640 1 -- 172.21.15.62:0/3106101029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a8100ee0 con 0x7f35a8101420 2024-09-12T00:08:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.597+0000 7f359effd640 1 -- 172.21.15.62:0/3106101029 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3598018a50 con 0x7f35a8101420 2024-09-12T00:08:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.597+0000 7f359effd640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f35940a2cf0 0x7f35940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.597+0000 7f359effd640 1 -- 172.21.15.62:0/3106101029 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8727+0+0 (secure 0 0 0) 0x7f35980cf9c0 con 0x7f35a8101420 2024-09-12T00:08:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.597+0000 7f35adcd6640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f35940a2cf0 0x7f35940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.598+0000 7f35adcd6640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f35940a2cf0 0x7f35940a51b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f35a4000c00 tx=0x7f35a4005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.601+0000 7f359effd640 1 -- 172.21.15.62:0/3106101029 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3598014030 con 0x7f35a8101420 2024-09-12T00:08:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:36.813+0000 7f35aff3a640 1 -- 172.21.15.62:0/3106101029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f35a802d7c0 con 0x7f35a8101420 2024-09-12T00:08:37.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.037+0000 7f359effd640 1 -- 172.21.15.62:0/3106101029 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7f359809c5a0 con 0x7f35a8101420 2024-09-12T00:08:37.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 -- 172.21.15.62:0/3106101029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f35940a2cf0 msgr2=0x7f35940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:37.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f35940a2cf0 0x7f35940a51b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f35a4000c00 tx=0x7f35a4005a20 comp rx=0 tx=0).stop 2024-09-12T00:08:37.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 -- 172.21.15.62:0/3106101029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8101420 msgr2=0x7f35a8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:37.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8101420 0x7f35a8198980 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f359800c920 tx=0x7f359800cde0 comp rx=0 tx=0).stop 2024-09-12T00:08:37.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 -- 172.21.15.62:0/3106101029 shutdown_connections 2024-09-12T00:08:37.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f35940a2cf0 0x7f35940a51b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:37.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a8106a80 0x7f35a819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:37.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8101420 0x7f35a8198980 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:37.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 --2- 172.21.15.62:0/3106101029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a8100a80 0x7f35a8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:37.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 -- 172.21.15.62:0/3106101029 >> 172.21.15.62:0/3106101029 conn(0x7f35a80fc7a0 msgr2=0x7f35a806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:37.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.039+0000 7f35aff3a640 1 -- 172.21.15.62:0/3106101029 shutdown_connections 2024-09-12T00:08:37.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:37.040+0000 7f35aff3a640 1 -- 172.21.15.62:0/3106101029 wait complete. 2024-09-12T00:08:37.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: pgmap v517: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 99 KiB/s wr, 9 op/s 2024-09-12T00:08:37.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.83:6834/4089840472,v1:172.21.15.83:6835/4089840472] up:boot 2024-09-12T00:08:37.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.120:6834/3529238375,v1:172.21.15.120:6836/3529238375] up:boot 2024-09-12T00:08:37.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.120:6835/611027111,v1:172.21.15.120:6838/611027111] up:boot 2024-09-12T00:08:37.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.120:6837/2627127308,v1:172.21.15.120:6839/2627127308] up:boot 2024-09-12T00:08:37.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.83:6837/3573572803,v1:172.21.15.83:6839/3573572803] up:boot 2024-09-12T00:08:37.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: mds.? [v2:172.21.15.83:6836/658719730,v1:172.21.15.83:6838/658719730] up:boot 2024-09-12T00:08:37.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T00:08:37.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T00:08:37.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T00:08:37.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T00:08:37.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T00:08:37.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T00:08:37.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T00:08:37.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:37 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3106101029' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T00:08:37.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: pgmap v517: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 99 KiB/s wr, 9 op/s 2024-09-12T00:08:37.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.83:6834/4089840472,v1:172.21.15.83:6835/4089840472] up:boot 2024-09-12T00:08:37.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.120:6834/3529238375,v1:172.21.15.120:6836/3529238375] up:boot 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.120:6835/611027111,v1:172.21.15.120:6838/611027111] up:boot 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.120:6837/2627127308,v1:172.21.15.120:6839/2627127308] up:boot 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.83:6837/3573572803,v1:172.21.15.83:6839/3573572803] up:boot 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: mds.? [v2:172.21.15.83:6836/658719730,v1:172.21.15.83:6838/658719730] up:boot 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:37 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3106101029' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T00:08:37.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: pgmap v517: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 99 KiB/s wr, 9 op/s 2024-09-12T00:08:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.83:6834/4089840472,v1:172.21.15.83:6835/4089840472] up:boot 2024-09-12T00:08:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.120:6834/3529238375,v1:172.21.15.120:6836/3529238375] up:boot 2024-09-12T00:08:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.120:6835/611027111,v1:172.21.15.120:6838/611027111] up:boot 2024-09-12T00:08:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.120:6837/2627127308,v1:172.21.15.120:6839/2627127308] up:boot 2024-09-12T00:08:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.83:6837/3573572803,v1:172.21.15.83:6839/3573572803] up:boot 2024-09-12T00:08:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: mds.? [v2:172.21.15.83:6836/658719730,v1:172.21.15.83:6838/658719730] up:boot 2024-09-12T00:08:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T00:08:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T00:08:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T00:08:37.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T00:08:37.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T00:08:37.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T00:08:37.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: from='mgr.15141 172.21.15.62:0/3987477396' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T00:08:37.418 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:37 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3106101029' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T00:08:37.777 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T00:08:38.035 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:38.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:38 smithi062 ceph-mon[30958]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T00:08:38.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:38 smithi062 ceph-mon[30958]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T00:08:38.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:38 smithi062 ceph-mon[30958]: Cluster is now healthy 2024-09-12T00:08:38.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:38 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3106101029' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T00:08:38.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:38 smithi062 ceph-mon[30958]: fsmap 12 up:standby 2024-09-12T00:08:38.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:38 smithi120 ceph-mon[35182]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T00:08:38.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:38 smithi120 ceph-mon[35182]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T00:08:38.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:38 smithi120 ceph-mon[35182]: Cluster is now healthy 2024-09-12T00:08:38.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:38 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3106101029' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T00:08:38.371 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:38 smithi120 ceph-mon[35182]: fsmap 12 up:standby 2024-09-12T00:08:38.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:38 smithi083 ceph-mon[34805]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T00:08:38.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:38 smithi083 ceph-mon[34805]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T00:08:38.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:38 smithi083 ceph-mon[34805]: Cluster is now healthy 2024-09-12T00:08:38.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:38 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3106101029' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T00:08:38.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:38 smithi083 ceph-mon[34805]: fsmap 12 up:standby 2024-09-12T00:08:38.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.512+0000 7f3a03922640 1 -- 172.21.15.62:0/2947720102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39fc10c170 msgr2=0x7f39fc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.512+0000 7f3a03922640 1 --2- 172.21.15.62:0/2947720102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39fc10c170 0x7f39fc10e610 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009ca0 tx=0x7f39f402a950 comp rx=0 tx=0).stop 2024-09-12T00:08:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.512+0000 7f3a03922640 1 -- 172.21.15.62:0/2947720102 shutdown_connections 2024-09-12T00:08:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.512+0000 7f3a03922640 1 --2- 172.21.15.62:0/2947720102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39fc10c170 0x7f39fc10e610 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.512+0000 7f3a03922640 1 --2- 172.21.15.62:0/2947720102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39fc103660 0x7f39fc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.512+0000 7f3a03922640 1 --2- 172.21.15.62:0/2947720102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc102d40 0x7f39fc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.512+0000 7f3a03922640 1 -- 172.21.15.62:0/2947720102 >> 172.21.15.62:0/2947720102 conn(0x7f39fc0fc7a0 msgr2=0x7f39fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.512+0000 7f3a03922640 1 -- 172.21.15.62:0/2947720102 shutdown_connections 2024-09-12T00:08:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.512+0000 7f3a03922640 1 -- 172.21.15.62:0/2947720102 wait complete. 2024-09-12T00:08:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.513+0000 7f3a03922640 1 Processor -- start 2024-09-12T00:08:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.513+0000 7f3a03922640 1 -- start start 2024-09-12T00:08:38.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.513+0000 7f3a03922640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39fc102d40 0x7f39fc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:38.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.513+0000 7f3a03922640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39fc103660 0x7f39fc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:38.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.513+0000 7f3a03922640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc10c170 0x7f39fc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:38.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.513+0000 7f3a03922640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc111fb0 con 0x7f39fc102d40 2024-09-12T00:08:38.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.513+0000 7f3a03922640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc111e30 con 0x7f39fc103660 2024-09-12T00:08:38.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.513+0000 7f3a03922640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc112130 con 0x7f39fc10c170 2024-09-12T00:08:38.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a016be640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39fc102d40 0x7f39fc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:38.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a016be640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39fc102d40 0x7f39fc1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58710/0 (socket says 172.21.15.62:58710) 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a016be640 1 -- 172.21.15.62:0/2315860059 learned_addr learned my addr 172.21.15.62:0/2315860059 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a00ebd640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39fc103660 0x7f39fc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a01ebf640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc10c170 0x7f39fc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a016be640 1 -- 172.21.15.62:0/2315860059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc10c170 msgr2=0x7f39fc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a016be640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc10c170 0x7f39fc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a016be640 1 -- 172.21.15.62:0/2315860059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39fc103660 msgr2=0x7f39fc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a016be640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39fc103660 0x7f39fc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a016be640 1 -- 172.21.15.62:0/2315860059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc19d380 con 0x7f39fc102d40 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a016be640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39fc102d40 0x7f39fc1983d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b4a0 tx=0x7f39f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f39f27fc640 1 -- 172.21.15.62:0/2315860059 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8014070 con 0x7f39fc102d40 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a03922640 1 -- 172.21.15.62:0/2315860059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc19d670 con 0x7f39fc102d40 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f39f27fc640 1 -- 172.21.15.62:0/2315860059 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f8010af0 con 0x7f39fc102d40 2024-09-12T00:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f39f27fc640 1 -- 172.21.15.62:0/2315860059 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8017d10 con 0x7f39fc102d40 2024-09-12T00:08:38.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.514+0000 7f3a03922640 1 -- 172.21.15.62:0/2315860059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc1a4f50 con 0x7f39fc102d40 2024-09-12T00:08:38.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.516+0000 7f3a03922640 1 -- 172.21.15.62:0/2315860059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc103120 con 0x7f39fc102d40 2024-09-12T00:08:38.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.517+0000 7f39f27fc640 1 -- 172.21.15.62:0/2315860059 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f39f8017070 con 0x7f39fc102d40 2024-09-12T00:08:38.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.517+0000 7f39f27fc640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f39e80a2cf0 0x7f39e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:38.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.518+0000 7f3a00ebd640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f39e80a2cf0 0x7f39e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:38.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.518+0000 7f39f27fc640 1 -- 172.21.15.62:0/2315860059 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8727+0+0 (secure 0 0 0) 0x7f39f80cec20 con 0x7f39fc102d40 2024-09-12T00:08:38.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.518+0000 7f3a00ebd640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f39e80a2cf0 0x7f39e80a51b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39ec005fd0 tx=0x7f39ec005f60 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:38.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.521+0000 7f39f27fc640 1 -- 172.21.15.62:0/2315860059 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39f809b880 con 0x7f39fc102d40 2024-09-12T00:08:38.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:38.696+0000 7f3a03922640 1 -- 172.21.15.62:0/2315860059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f39fc1a5230 con 0x7f39fc102d40 2024-09-12T00:08:39.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.048+0000 7f39f27fc640 1 -- 172.21.15.62:0/2315860059 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7f39f809b180 con 0x7f39fc102d40 2024-09-12T00:08:39.048 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-09-12T00:08:39.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.052+0000 7f3a03922640 1 -- 172.21.15.62:0/2315860059 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f39e80a2cf0 msgr2=0x7f39e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:39.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.052+0000 7f3a03922640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f39e80a2cf0 0x7f39e80a51b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39ec005fd0 tx=0x7f39ec005f60 comp rx=0 tx=0).stop 2024-09-12T00:08:39.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.052+0000 7f3a03922640 1 -- 172.21.15.62:0/2315860059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39fc102d40 msgr2=0x7f39fc1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:39.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.052+0000 7f3a03922640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39fc102d40 0x7f39fc1983d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b4a0 tx=0x7f39f800b960 comp rx=0 tx=0).stop 2024-09-12T00:08:39.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.052+0000 7f3a03922640 1 -- 172.21.15.62:0/2315860059 shutdown_connections 2024-09-12T00:08:39.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.052+0000 7f3a03922640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f39e80a2cf0 0x7f39e80a51b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:39.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.052+0000 7f3a03922640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39fc10c170 0x7f39fc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:39.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.052+0000 7f3a03922640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39fc103660 0x7f39fc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:39.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.052+0000 7f3a03922640 1 --2- 172.21.15.62:0/2315860059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39fc102d40 0x7f39fc1983d0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:39.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.052+0000 7f3a03922640 1 -- 172.21.15.62:0/2315860059 >> 172.21.15.62:0/2315860059 conn(0x7f39fc0fc7a0 msgr2=0x7f39fc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:39.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.053+0000 7f3a03922640 1 -- 172.21.15.62:0/2315860059 shutdown_connections 2024-09-12T00:08:39.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:39.053+0000 7f3a03922640 1 -- 172.21.15.62:0/2315860059 wait complete. 2024-09-12T00:08:39.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:39 smithi062 ceph-mon[30958]: pgmap v518: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 7 op/s 2024-09-12T00:08:39.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:39 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2315860059' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T00:08:39.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:39 smithi120 ceph-mon[35182]: pgmap v518: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 7 op/s 2024-09-12T00:08:39.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:39 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2315860059' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T00:08:39.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:39 smithi083 ceph-mon[34805]: pgmap v518: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 7 op/s 2024-09-12T00:08:39.417 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:39 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2315860059' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T00:08:39.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 56675698-7097-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T00:08:39.972 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:08:40.318 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:40 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/2315860059' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T00:08:40.318 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:40 smithi083 ceph-mon[34805]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T00:08:40.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:40 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/2315860059' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T00:08:40.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:40 smithi062 ceph-mon[30958]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T00:08:40.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:40 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/2315860059' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T00:08:40.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:40 smithi120 ceph-mon[35182]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T00:08:40.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 -- 172.21.15.62:0/874996575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8540102790 msgr2=0x7f85401067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:40.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 --2- 172.21.15.62:0/874996575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8540102790 0x7f85401067e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8534009900 tx=0x7f853402a410 comp rx=0 tx=0).stop 2024-09-12T00:08:40.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 -- 172.21.15.62:0/874996575 shutdown_connections 2024-09-12T00:08:40.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 --2- 172.21.15.62:0/874996575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8540106f10 0x7f854010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:40.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 --2- 172.21.15.62:0/874996575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8540102790 0x7f85401067e0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:40.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 --2- 172.21.15.62:0/874996575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8540101de0 0x7f85401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 -- 172.21.15.62:0/874996575 >> 172.21.15.62:0/874996575 conn(0x7f85400fd8c0 msgr2=0x7f85400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 -- 172.21.15.62:0/874996575 shutdown_connections 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 -- 172.21.15.62:0/874996575 wait complete. 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 Processor -- start 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.415+0000 7f854687b640 1 -- start start 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f854687b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8540101de0 0x7f8540198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f854687b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8540102790 0x7f8540198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f854687b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8540106f10 0x7f854019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f854687b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8540111f30 con 0x7f8540102790 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f854687b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8540111db0 con 0x7f8540106f10 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f854687b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85401120b0 con 0x7f8540101de0 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8540102790 0x7f8540198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:40.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8540102790 0x7f8540198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58732/0 (socket says 172.21.15.62:58732) 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f853f7fe640 1 -- 172.21.15.62:0/3137935131 learned_addr learned my addr 172.21.15.62:0/3137935131 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f8544e18640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8540106f10 0x7f854019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f853ffff640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8540101de0 0x7f8540198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f853f7fe640 1 -- 172.21.15.62:0/3137935131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8540101de0 msgr2=0x7f8540198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f853f7fe640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8540101de0 0x7f8540198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f853f7fe640 1 -- 172.21.15.62:0/3137935131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8540106f10 msgr2=0x7f854019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f853f7fe640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8540106f10 0x7f854019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.416+0000 7f853f7fe640 1 -- 172.21.15.62:0/3137935131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854019d3e0 con 0x7f8540102790 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.417+0000 7f853f7fe640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8540102790 0x7f8540198970 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8534006010 tx=0x7f85340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.417+0000 7f853d7fa640 1 -- 172.21.15.62:0/3137935131 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853403f030 con 0x7f8540102790 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.417+0000 7f853d7fa640 1 -- 172.21.15.62:0/3137935131 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8534003ce0 con 0x7f8540102790 2024-09-12T00:08:40.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.417+0000 7f854687b640 1 -- 172.21.15.62:0/3137935131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854019d670 con 0x7f8540102790 2024-09-12T00:08:40.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.417+0000 7f853d7fa640 1 -- 172.21.15.62:0/3137935131 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534035070 con 0x7f8540102790 2024-09-12T00:08:40.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.417+0000 7f854687b640 1 -- 172.21.15.62:0/3137935131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8540074420 con 0x7f8540102790 2024-09-12T00:08:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.418+0000 7f854687b640 1 -- 172.21.15.62:0/3137935131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85401032c0 con 0x7f8540102790 2024-09-12T00:08:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.419+0000 7f853d7fa640 1 -- 172.21.15.62:0/3137935131 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8534042030 con 0x7f8540102790 2024-09-12T00:08:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.420+0000 7f853d7fa640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f85280a2cf0 0x7f85280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:08:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.420+0000 7f853ffff640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f85280a2cf0 0x7f85280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:08:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.420+0000 7f853d7fa640 1 -- 172.21.15.62:0/3137935131 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8173+0+0 (secure 0 0 0) 0x7f85340ef8b0 con 0x7f8540102790 2024-09-12T00:08:40.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.420+0000 7f853ffff640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f85280a2cf0 0x7f85280a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f853000bbe0 tx=0x7f853000b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:08:40.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.424+0000 7f853d7fa640 1 -- 172.21.15.62:0/3137935131 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85340bc0f0 con 0x7f8540102790 2024-09-12T00:08:40.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:40.600+0000 7f854687b640 1 -- 172.21.15.62:0/3137935131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f854010a0f0 con 0x7f8540102790 2024-09-12T00:08:41.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.061+0000 7f853d7fa640 1 -- 172.21.15.62:0/3137935131 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7f85340c03f0 con 0x7f8540102790 2024-09-12T00:08:41.061 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-09-12T00:08:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.065+0000 7f854687b640 1 -- 172.21.15.62:0/3137935131 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f85280a2cf0 msgr2=0x7f85280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.065+0000 7f854687b640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f85280a2cf0 0x7f85280a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f853000bbe0 tx=0x7f853000b390 comp rx=0 tx=0).stop 2024-09-12T00:08:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.065+0000 7f854687b640 1 -- 172.21.15.62:0/3137935131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8540102790 msgr2=0x7f8540198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:08:41.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.065+0000 7f854687b640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8540102790 0x7f8540198970 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8534006010 tx=0x7f85340042f0 comp rx=0 tx=0).stop 2024-09-12T00:08:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.067+0000 7f854687b640 1 -- 172.21.15.62:0/3137935131 shutdown_connections 2024-09-12T00:08:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.067+0000 7f854687b640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.62:6800/3798569441,v1:172.21.15.62:6801/3798569441] conn(0x7f85280a2cf0 0x7f85280a51b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.067+0000 7f854687b640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8540106f10 0x7f854019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.067+0000 7f854687b640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8540102790 0x7f8540198970 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.067+0000 7f854687b640 1 --2- 172.21.15.62:0/3137935131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8540101de0 0x7f8540198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:08:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.067+0000 7f854687b640 1 -- 172.21.15.62:0/3137935131 >> 172.21.15.62:0/3137935131 conn(0x7f85400fd8c0 msgr2=0x7f85400fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:08:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.069+0000 7f854687b640 1 -- 172.21.15.62:0/3137935131 shutdown_connections 2024-09-12T00:08:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-12T00:08:41.069+0000 7f854687b640 1 -- 172.21.15.62:0/3137935131 wait complete. 2024-09-12T00:08:41.332 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:41 smithi083 ceph-mon[34805]: pgmap v520: 65 pgs: 65 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:08:41.332 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:41 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3137935131' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T00:08:41.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:41 smithi120 ceph-mon[35182]: pgmap v520: 65 pgs: 65 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:08:41.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:41 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3137935131' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T00:08:41.417 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:41 smithi062 ceph-mon[30958]: pgmap v520: 65 pgs: 65 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:08:41.418 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:41 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3137935131' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T00:08:41.750 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T00:08:41.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T00:08:41.852 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T00:08:41.897 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T00:08:41.898 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T00:08:41.936 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T00:08:41.968 DEBUG:teuthology.orchestra.run.smithi120:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T00:08:42.005 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T00:08:42.006 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T00:08:42.006 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mon.a 2024-09-12T00:08:42.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:42 smithi120 ceph-mon[35182]: from='client.? 172.21.15.62:0/3137935131' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T00:08:42.370 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:42 smithi120 ceph-mon[35182]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T00:08:42.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:42 smithi083 ceph-mon[34805]: from='client.? 172.21.15.62:0/3137935131' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T00:08:42.416 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:42 smithi083 ceph-mon[34805]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T00:08:42.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:42 smithi062 systemd[1]: Stopping Ceph mon.a for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:08:42.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:42 smithi062 ceph-mon[30958]: from='client.? 172.21.15.62:0/3137935131' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T00:08:42.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:42 smithi062 ceph-mon[30958]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T00:08:43.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:42 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a[30954]: 2024-09-12T00:08:42.730+0000 7f19109b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:08:43.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:42 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a[30954]: 2024-09-12T00:08:42.730+0000 7f19109b3640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T00:08:43.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:42 smithi062 podman[119052]: 2024-09-12 00:08:42.925403542 +0000 UTC m=+0.540780080 container died 577ea3d597ecdd3a9728a41ca756b69a136118806b5ce9b0e8a9d027840ef3e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T00:08:43.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:43 smithi062 podman[119052]: 2024-09-12 00:08:43.315764813 +0000 UTC m=+0.931141348 container cleanup 577ea3d597ecdd3a9728a41ca756b69a136118806b5ce9b0e8a9d027840ef3e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T00:08:43.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:43 smithi062 bash[119052]: ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a 2024-09-12T00:08:43.775 INFO:journalctl@ceph.mon.a.smithi062.stdout:Sep 12 00:08:43 smithi062 podman[119062]: 2024-09-12 00:08:43.626391046 +0000 UTC m=+0.698068975 container remove 577ea3d597ecdd3a9728a41ca756b69a136118806b5ce9b0e8a9d027840ef3e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T00:08:43.797 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T00:08:43.848 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:08:43.849 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T00:08:43.849 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T00:08:43.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mon.b 2024-09-12T00:08:44.166 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:43 smithi083 systemd[1]: Stopping Ceph mon.b for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:08:44.526 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:44 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mon-b[34801]: 2024-09-12T00:08:44.167+0000 7fc0bcf01640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:08:44.526 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:44 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mon-b[34801]: 2024-09-12T00:08:44.167+0000 7fc0bcf01640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T00:08:44.526 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:44 smithi083 podman[70912]: 2024-09-12 00:08:44.198972653 +0000 UTC m=+0.159915475 container died 123c1129f1957ab1d9ef94d9d8711ad0e6d4e6607ee44f48f674c80e1f4afce2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T00:08:44.852 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:44 smithi083 podman[70912]: 2024-09-12 00:08:44.635335391 +0000 UTC m=+0.596278211 container cleanup 123c1129f1957ab1d9ef94d9d8711ad0e6d4e6607ee44f48f674c80e1f4afce2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T00:08:44.852 INFO:journalctl@ceph.mon.b.smithi083.stdout:Sep 12 00:08:44 smithi083 bash[70912]: ceph-56675698-7097-11ef-bcea-c7b262605968-mon-b 2024-09-12T00:08:45.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T00:08:45.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:08:45.110 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T00:08:45.110 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T00:08:45.110 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mon.c 2024-09-12T00:08:45.525 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:45 smithi120 systemd[1]: Stopping Ceph mon.c for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:08:45.843 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:45 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mon-c[35178]: 2024-09-12T00:08:45.525+0000 7f32b0ade640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:08:45.843 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:45 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mon-c[35178]: 2024-09-12T00:08:45.525+0000 7f32b0ade640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T00:08:45.843 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:45 smithi120 podman[72498]: 2024-09-12 00:08:45.557379569 +0000 UTC m=+0.254749051 container died 1e4cf4400adba3d5de5f33dfe8fa511e3cf88938a7a44eda55d354c67c60a3c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T00:08:46.518 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:46 smithi120 podman[72498]: 2024-09-12 00:08:46.251694305 +0000 UTC m=+0.949063787 container cleanup 1e4cf4400adba3d5de5f33dfe8fa511e3cf88938a7a44eda55d354c67c60a3c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T00:08:46.518 INFO:journalctl@ceph.mon.c.smithi120.stdout:Sep 12 00:08:46 smithi120 bash[72498]: ceph-56675698-7097-11ef-bcea-c7b262605968-mon-c 2024-09-12T00:08:46.685 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T00:08:46.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:08:46.758 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T00:08:46.758 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T00:08:46.758 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.x 2024-09-12T00:08:47.087 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 12 00:08:46 smithi062 systemd[1]: Stopping Ceph mgr.x for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:08:47.405 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 12 00:08:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-12T00:08:47.087+0000 7f7811a08640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:08:47.405 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 12 00:08:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x[31184]: 2024-09-12T00:08:47.087+0000 7f7811a08640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-12T00:08:47.405 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 12 00:08:47 smithi062 podman[119167]: 2024-09-12 00:08:47.119328166 +0000 UTC m=+0.158316673 container died bff0f724599e2f53ce30551a7483a7e67958d4e590fb8ff3cae309fb50e7407e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:08:48.715 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 12 00:08:48 smithi062 podman[119167]: 2024-09-12 00:08:48.312960704 +0000 UTC m=+1.351949203 container cleanup bff0f724599e2f53ce30551a7483a7e67958d4e590fb8ff3cae309fb50e7407e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:08:48.715 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Sep 12 00:08:48 smithi062 bash[119167]: ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-x 2024-09-12T00:08:49.648 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T00:08:49.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:08:49.685 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T00:08:49.686 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T00:08:49.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.y 2024-09-12T00:08:50.166 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 12 00:08:49 smithi083 systemd[1]: Stopping Ceph mgr.y for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:08:50.907 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 12 00:08:50 smithi083 podman[71035]: 2024-09-12 00:08:50.631226896 +0000 UTC m=+0.753087535 container died 15708569c0cbf53b0a1382cc9710b509fa12f4eb9edf9944ab608e607d67c266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T00:08:51.166 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 12 00:08:51 smithi083 podman[71035]: 2024-09-12 00:08:51.024830726 +0000 UTC m=+1.146691354 container cleanup 15708569c0cbf53b0a1382cc9710b509fa12f4eb9edf9944ab608e607d67c266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:08:51.167 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 12 00:08:51 smithi083 bash[71035]: ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-y 2024-09-12T00:08:51.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T00:08:51.493 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:08:51.494 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T00:08:51.494 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T00:08:51.494 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.z 2024-09-12T00:08:51.870 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 12 00:08:51 smithi120 systemd[1]: Stopping Ceph mgr.z for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:08:53.181 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 12 00:08:52 smithi120 podman[72623]: 2024-09-12 00:08:52.880312736 +0000 UTC m=+1.191558356 container died 251ba42188807c0c33843fca6c71ce553c0fff016a06225e9674cbed9f4e7ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T00:08:54.040 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 12 00:08:53 smithi120 podman[72623]: 2024-09-12 00:08:53.832574892 +0000 UTC m=+2.143820511 container cleanup 251ba42188807c0c33843fca6c71ce553c0fff016a06225e9674cbed9f4e7ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T00:08:54.041 INFO:journalctl@ceph.mgr.z.smithi120.stdout:Sep 12 00:08:53 smithi120 bash[72623]: ceph-56675698-7097-11ef-bcea-c7b262605968-mgr-z 2024-09-12T00:08:54.265 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T00:08:54.299 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:08:54.299 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T00:08:54.299 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T00:08:54.299 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.0 2024-09-12T00:08:54.619 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:08:54 smithi062 systemd[1]: Stopping Ceph osd.0 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:08:54.965 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:08:54 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0[44685]: 2024-09-12T00:08:54.619+0000 7f949509e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:08:54.965 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:08:54 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0[44685]: 2024-09-12T00:08:54.619+0000 7f949509e640 -1 osd.0 105 *** Got signal Terminated *** 2024-09-12T00:08:54.965 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:08:54 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0[44685]: 2024-09-12T00:08:54.619+0000 7f949509e640 -1 osd.0 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:09:00.637 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:00 smithi062 podman[119289]: 2024-09-12 00:09:00.32221395 +0000 UTC m=+5.820216353 container stop 5637f0f671bfe57505097a8ecde89b63627cb48e81f041f393c9186116f6b35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T00:09:00.637 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:00 smithi062 podman[119289]: 2024-09-12 00:09:00.347936159 +0000 UTC m=+5.845938558 container died 5637f0f671bfe57505097a8ecde89b63627cb48e81f041f393c9186116f6b35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T00:09:00.946 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:00 smithi062 podman[119289]: 2024-09-12 00:09:00.738010329 +0000 UTC m=+6.236012726 container cleanup 5637f0f671bfe57505097a8ecde89b63627cb48e81f041f393c9186116f6b35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T00:09:00.947 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:00 smithi062 bash[119289]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0 2024-09-12T00:09:01.215 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:00 smithi062 podman[119306]: 2024-09-12 00:09:00.948288419 +0000 UTC m=+0.619609088 container remove 5637f0f671bfe57505097a8ecde89b63627cb48e81f041f393c9186116f6b35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T00:09:01.692 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:01 smithi062 podman[119392]: 2024-09-12 00:09:01.260980139 +0000 UTC m=+0.016356850 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:09:01.693 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:01 smithi062 podman[119392]: 2024-09-12 00:09:01.37438369 +0000 UTC m=+0.129760435 container create c44c46f5434881632489458ab9235cebb9549355c372bd998559019d11d4b240 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:09:01.693 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:01 smithi062 podman[119392]: 2024-09-12 00:09:01.602103292 +0000 UTC m=+0.357479991 container init c44c46f5434881632489458ab9235cebb9549355c372bd998559019d11d4b240 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T00:09:01.693 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:01 smithi062 podman[119392]: 2024-09-12 00:09:01.605727533 +0000 UTC m=+0.361104233 container start c44c46f5434881632489458ab9235cebb9549355c372bd998559019d11d4b240 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T00:09:01.693 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:01 smithi062 podman[119392]: 2024-09-12 00:09:01.693757077 +0000 UTC m=+0.449133788 container attach c44c46f5434881632489458ab9235cebb9549355c372bd998559019d11d4b240 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T00:09:01.965 INFO:journalctl@ceph.osd.0.smithi062.stdout:Sep 12 00:09:01 smithi062 podman[119392]: 2024-09-12 00:09:01.802801266 +0000 UTC m=+0.558177965 container died c44c46f5434881632489458ab9235cebb9549355c372bd998559019d11d4b240 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T00:09:02.487 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T00:09:02.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:09:02.544 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T00:09:02.544 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T00:09:02.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.1 2024-09-12T00:09:02.845 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:02 smithi083 systemd[1]: Stopping Ceph osd.1 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:09:03.166 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:02 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1[39266]: 2024-09-12T00:09:02.845+0000 7f7b112c4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:09:03.166 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:02 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1[39266]: 2024-09-12T00:09:02.845+0000 7f7b112c4640 -1 osd.1 105 *** Got signal Terminated *** 2024-09-12T00:09:03.167 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:02 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1[39266]: 2024-09-12T00:09:02.845+0000 7f7b112c4640 -1 osd.1 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:09:08.738 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:08 smithi083 podman[71161]: 2024-09-12 00:09:08.446373257 +0000 UTC m=+5.696426556 container died b0c24284392007b8a4ff0f3fc83a8abb73f8f85c85590188e47ef48ad5ded448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T00:09:09.105 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:08 smithi083 podman[71161]: 2024-09-12 00:09:08.821529717 +0000 UTC m=+6.071583015 container cleanup b0c24284392007b8a4ff0f3fc83a8abb73f8f85c85590188e47ef48ad5ded448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T00:09:09.105 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:08 smithi083 bash[71161]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1 2024-09-12T00:09:09.416 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:09 smithi083 podman[71176]: 2024-09-12 00:09:09.106478103 +0000 UTC m=+0.654321710 container remove b0c24284392007b8a4ff0f3fc83a8abb73f8f85c85590188e47ef48ad5ded448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T00:09:09.692 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:09 smithi083 podman[71266]: 2024-09-12 00:09:09.333150823 +0000 UTC m=+0.015864009 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:09:09.692 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:09 smithi083 podman[71266]: 2024-09-12 00:09:09.432342383 +0000 UTC m=+0.115055568 container create 8430978e216666d6ff7b1959e46dfe9a1d36030518e5592d5255384dfc5353ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T00:09:09.692 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:09 smithi083 podman[71266]: 2024-09-12 00:09:09.693671492 +0000 UTC m=+0.376384677 container init 8430978e216666d6ff7b1959e46dfe9a1d36030518e5592d5255384dfc5353ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:09:10.167 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:09 smithi083 podman[71266]: 2024-09-12 00:09:09.697327229 +0000 UTC m=+0.380040415 container start 8430978e216666d6ff7b1959e46dfe9a1d36030518e5592d5255384dfc5353ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T00:09:10.167 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:09 smithi083 podman[71266]: 2024-09-12 00:09:09.794002707 +0000 UTC m=+0.476715910 container attach 8430978e216666d6ff7b1959e46dfe9a1d36030518e5592d5255384dfc5353ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:09:10.167 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 12 00:09:09 smithi083 podman[71266]: 2024-09-12 00:09:09.89169024 +0000 UTC m=+0.574403427 container died 8430978e216666d6ff7b1959e46dfe9a1d36030518e5592d5255384dfc5353ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T00:09:10.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T00:09:10.575 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:09:10.576 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T00:09:10.576 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T00:09:10.576 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.2 2024-09-12T00:09:10.870 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:10 smithi120 systemd[1]: Stopping Ceph osd.2 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:09:11.370 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2[40143]: 2024-09-12T00:09:10.902+0000 7f13f5fac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:09:11.371 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2[40143]: 2024-09-12T00:09:10.902+0000 7f13f5fac640 -1 osd.2 105 *** Got signal Terminated *** 2024-09-12T00:09:11.371 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2[40143]: 2024-09-12T00:09:10.902+0000 7f13f5fac640 -1 osd.2 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:09:16.870 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:16 smithi120 podman[72749]: 2024-09-12 00:09:16.578478734 +0000 UTC m=+5.805212566 container died 0acfb330dc23c8a5d0424b488edf1ede81ecb0d12deb2dc5b7cb7df94662778b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T00:09:17.243 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:16 smithi120 podman[72749]: 2024-09-12 00:09:16.9857036 +0000 UTC m=+6.212437434 container cleanup 0acfb330dc23c8a5d0424b488edf1ede81ecb0d12deb2dc5b7cb7df94662778b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:09:17.244 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:16 smithi120 bash[72749]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2 2024-09-12T00:09:17.586 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:17 smithi120 podman[72778]: 2024-09-12 00:09:17.24550813 +0000 UTC m=+0.665036509 container remove 0acfb330dc23c8a5d0424b488edf1ede81ecb0d12deb2dc5b7cb7df94662778b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T00:09:17.870 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:17 smithi120 podman[72865]: 2024-09-12 00:09:17.491014023 +0000 UTC m=+0.015881734 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:09:17.870 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:17 smithi120 podman[72865]: 2024-09-12 00:09:17.613026439 +0000 UTC m=+0.137894183 container create f154b73fb8bda82ee38488bd6642d84ce690d23de966109c093e0f60be7c2d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T00:09:18.128 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:17 smithi120 podman[72865]: 2024-09-12 00:09:17.900558774 +0000 UTC m=+0.425426475 container init f154b73fb8bda82ee38488bd6642d84ce690d23de966109c093e0f60be7c2d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T00:09:18.128 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:17 smithi120 podman[72865]: 2024-09-12 00:09:17.903891648 +0000 UTC m=+0.428759346 container start f154b73fb8bda82ee38488bd6642d84ce690d23de966109c093e0f60be7c2d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T00:09:18.128 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:18 smithi120 podman[72865]: 2024-09-12 00:09:18.007346549 +0000 UTC m=+0.532214257 container attach f154b73fb8bda82ee38488bd6642d84ce690d23de966109c093e0f60be7c2d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T00:09:18.128 INFO:journalctl@ceph.osd.2.smithi120.stdout:Sep 12 00:09:18 smithi120 podman[72865]: 2024-09-12 00:09:18.098796901 +0000 UTC m=+0.623664609 container died f154b73fb8bda82ee38488bd6642d84ce690d23de966109c093e0f60be7c2d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T00:09:18.771 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T00:09:18.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:09:18.811 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T00:09:18.811 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T00:09:18.811 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.3 2024-09-12T00:09:19.143 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:18 smithi062 systemd[1]: Stopping Ceph osd.3 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:09:19.465 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:19 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3[51397]: 2024-09-12T00:09:19.143+0000 7efd1cf69640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:09:19.465 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:19 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3[51397]: 2024-09-12T00:09:19.143+0000 7efd1cf69640 -1 osd.3 105 *** Got signal Terminated *** 2024-09-12T00:09:19.465 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:19 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3[51397]: 2024-09-12T00:09:19.143+0000 7efd1cf69640 -1 osd.3 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:09:21.215 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:20 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3[51397]: 2024-09-12T00:09:20.781+0000 7efd1957f640 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:58.732479+0000 front 2024-09-12T00:08:58.732523+0000 (oldest deadline 2024-09-12T00:09:20.432651+0000) 2024-09-12T00:09:21.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:21 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:21.402+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:22.215 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:21 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3[51397]: 2024-09-12T00:09:21.788+0000 7efd1957f640 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:58.732479+0000 front 2024-09-12T00:08:58.732523+0000 (oldest deadline 2024-09-12T00:09:20.432651+0000) 2024-09-12T00:09:22.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:21 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:21.929+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:22.775 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:22 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:22.490+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:22.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:22 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:22.415+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:23.203 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:22 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3[51397]: 2024-09-12T00:09:22.776+0000 7efd1957f640 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:58.732479+0000 front 2024-09-12T00:08:58.732523+0000 (oldest deadline 2024-09-12T00:09:20.432651+0000) 2024-09-12T00:09:23.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:22 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:22.928+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:23.465 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:23 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:23.446+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:23.465 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:23 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:23.204+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:23.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:23.426+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:24.166 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:23 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:23.841+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:24.226 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:23 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3[51397]: 2024-09-12T00:09:23.727+0000 7efd1957f640 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:58.732479+0000 front 2024-09-12T00:08:58.732523+0000 (oldest deadline 2024-09-12T00:09:20.432651+0000) 2024-09-12T00:09:24.373 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:23.886+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:24.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:24 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:24.398+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:24.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:24 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:24.223+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:24.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:24 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:24.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:25.051 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:24 smithi062 podman[119509]: 2024-09-12 00:09:24.793923864 +0000 UTC m=+5.776014295 container died 103726feb9d631bb17fabb35623a0f5832de3fae7181e001983fdac7e370d495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-12T00:09:25.166 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:24 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:24.876+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:25.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:24 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:24.888+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:25.392 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:25 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:25.271+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:25.392 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:25 smithi062 podman[119509]: 2024-09-12 00:09:25.260325764 +0000 UTC m=+6.242416192 container cleanup 103726feb9d631bb17fabb35623a0f5832de3fae7181e001983fdac7e370d495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T00:09:25.392 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:25 smithi062 bash[119509]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3 2024-09-12T00:09:25.715 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:25 smithi062 podman[119526]: 2024-09-12 00:09:25.529136636 +0000 UTC m=+0.739097766 container remove 103726feb9d631bb17fabb35623a0f5832de3fae7181e001983fdac7e370d495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T00:09:25.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:25 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:25.393+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:25.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:25 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:25.464+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:26.181 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:25 smithi062 podman[119609]: 2024-09-12 00:09:25.786503744 +0000 UTC m=+0.016355977 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:09:26.181 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:25 smithi062 podman[119609]: 2024-09-12 00:09:25.921953877 +0000 UTC m=+0.151806098 container create 8e917b43faec0a8328f9e5268a2bd21a83bff85df75ad993e7df2db7f1dbb72b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T00:09:26.181 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:26 smithi062 podman[119609]: 2024-09-12 00:09:26.182910164 +0000 UTC m=+0.412762385 container init 8e917b43faec0a8328f9e5268a2bd21a83bff85df75ad993e7df2db7f1dbb72b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T00:09:26.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:25 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:25.902+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:26.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:25 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:25.918+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:26.465 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:26 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:26.407+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:26.466 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:26 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:26.268+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:26.466 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:26 smithi062 podman[119609]: 2024-09-12 00:09:26.186632495 +0000 UTC m=+0.416484717 container start 8e917b43faec0a8328f9e5268a2bd21a83bff85df75ad993e7df2db7f1dbb72b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-12T00:09:26.466 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:26 smithi062 podman[119609]: 2024-09-12 00:09:26.266353936 +0000 UTC m=+0.496206169 container attach 8e917b43faec0a8328f9e5268a2bd21a83bff85df75ad993e7df2db7f1dbb72b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:09:26.467 INFO:journalctl@ceph.osd.3.smithi062.stdout:Sep 12 00:09:26 smithi062 podman[119609]: 2024-09-12 00:09:26.381926639 +0000 UTC m=+0.611778870 container died 8e917b43faec0a8328f9e5268a2bd21a83bff85df75ad993e7df2db7f1dbb72b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-09-12T00:09:26.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:26 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:26.451+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:26.999 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T00:09:27.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:09:27.043 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T00:09:27.043 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T00:09:27.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.4 2024-09-12T00:09:27.367 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:27 smithi083 systemd[1]: Stopping Ceph osd.4 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:09:27.367 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:26 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:26.964+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:27.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:26 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:26.873+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:27.666 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:27 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-12T00:09:27.367+0000 7f4a1a80a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:09:27.666 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:27 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-12T00:09:27.367+0000 7f4a1a80a640 -1 osd.4 105 *** Got signal Terminated *** 2024-09-12T00:09:27.667 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:27 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-12T00:09:27.367+0000 7f4a1a80a640 -1 osd.4 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:09:27.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:27 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:27.363+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:27.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:27 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:27.276+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:27.845 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:27.472+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:28.120 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:27.846+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:28.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:27 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:27.922+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:28.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:28 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:28.403+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:28.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:28 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:28.259+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:28.849 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:28 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:28.507+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:29.120 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:28 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:28.849+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:29.229 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:28 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:28.940+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:29.229 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:28 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:28.940+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:29.668 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:29 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:29.229+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:29.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:29 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:29.414+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:29.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:29 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:29.298+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:29.853 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:29 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:29.486+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:29.853 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:29 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:29.490+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:30.120 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:29 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:29.853+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:30.263 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:29 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:29.986+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:30.263 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:29 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:29.986+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:30.666 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:30 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:30.263+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:30.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:30 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:30.433+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:30.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:30 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:30.295+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:30.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:30 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:30.479+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:30.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:30 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:30.489+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:31.254 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:30 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:30.991+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:31.254 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:30 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:30.991+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:31.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:30 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:30.899+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:31.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:30 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:30.899+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.798320+0000 front 2024-09-12T00:09:06.798341+0000 (oldest deadline 2024-09-12T00:09:30.298363+0000) 2024-09-12T00:09:31.666 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:31 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:31.254+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:31.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:31 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:31.420+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:31.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:31 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:31.420+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:31.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:31 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:31.250+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:31.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:31 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:31.250+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:31.863 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:31 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:31.485+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:31.864 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:31 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:31.507+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:32.120 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:31 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:31.864+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:32.120 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:31 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:31.864+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.798320+0000 front 2024-09-12T00:09:06.798341+0000 (oldest deadline 2024-09-12T00:09:30.298363+0000) 2024-09-12T00:09:32.416 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:32 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4[45711]: 2024-09-12T00:09:32.082+0000 7f4a12e18640 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.130092+0000 front 2024-09-12T00:09:06.130062+0000 (oldest deadline 2024-09-12T00:09:31.430180+0000) 2024-09-12T00:09:32.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:32 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:32.219+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:32.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:31 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:31.993+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:32.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:31 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:31.993+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:32.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:32 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:32.432+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:32.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:32 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:32.432+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:32.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:32 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:32.286+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:32.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:32 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:32.286+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:32.858 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:32 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:32.445+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:32.858 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:32 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:32.472+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:32.858 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:32 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:32.472+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:33.120 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:32 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:32.858+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:33.120 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:32 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:32.858+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.798320+0000 front 2024-09-12T00:09:06.798341+0000 (oldest deadline 2024-09-12T00:09:30.298363+0000) 2024-09-12T00:09:33.167 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:32 smithi083 podman[71382]: 2024-09-12 00:09:32.893539461 +0000 UTC m=+5.650501481 container died b0dc95dde0281978c24b0794b4d4379b315fb52d71d70b6f4bce7cb5f04bfc81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T00:09:33.167 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:32 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:32.998+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:33.167 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:32 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:32.998+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:33.559 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:33 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:33.185+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:33.560 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:33 smithi083 podman[71382]: 2024-09-12 00:09:33.293025086 +0000 UTC m=+6.049987105 container cleanup b0dc95dde0281978c24b0794b4d4379b315fb52d71d70b6f4bce7cb5f04bfc81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T00:09:33.560 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:33 smithi083 bash[71382]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4 2024-09-12T00:09:33.616 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:33 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:33.403+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:33.616 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:33 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:33.403+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:33.617 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:33 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:33.331+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:33.617 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:33 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:33.331+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:33.827 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:33 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:33.448+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:33.827 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:33 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:33.485+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:33.827 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:33 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:33.485+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:33.893 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:33 smithi083 podman[71397]: 2024-09-12 00:09:33.56143933 +0000 UTC m=+0.667177644 container remove b0dc95dde0281978c24b0794b4d4379b315fb52d71d70b6f4bce7cb5f04bfc81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T00:09:34.120 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:33 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:33.828+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:34.120 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:33 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:33.828+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.798320+0000 front 2024-09-12T00:09:06.798341+0000 (oldest deadline 2024-09-12T00:09:30.298363+0000) 2024-09-12T00:09:34.147 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:33 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:33.980+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:34.147 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:33 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:33.980+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:34.147 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:33 smithi083 podman[71487]: 2024-09-12 00:09:33.895511827 +0000 UTC m=+0.109573514 container create 82f485a7626c90018edd4a90364ffd9b963ca2de7486acf4dde8d8a6ed65956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T00:09:34.147 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:33 smithi083 podman[71487]: 2024-09-12 00:09:33.801639994 +0000 UTC m=+0.015701632 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:09:34.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:34 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:34.203+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:34.417 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:34 smithi083 podman[71487]: 2024-09-12 00:09:34.14871091 +0000 UTC m=+0.362772557 container init 82f485a7626c90018edd4a90364ffd9b963ca2de7486acf4dde8d8a6ed65956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T00:09:34.417 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:34 smithi083 podman[71487]: 2024-09-12 00:09:34.152291306 +0000 UTC m=+0.366352948 container start 82f485a7626c90018edd4a90364ffd9b963ca2de7486acf4dde8d8a6ed65956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T00:09:34.417 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:34 smithi083 podman[71487]: 2024-09-12 00:09:34.273454944 +0000 UTC m=+0.487516588 container attach 82f485a7626c90018edd4a90364ffd9b963ca2de7486acf4dde8d8a6ed65956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T00:09:34.417 INFO:journalctl@ceph.osd.4.smithi083.stdout:Sep 12 00:09:34 smithi083 podman[71487]: 2024-09-12 00:09:34.345520137 +0000 UTC m=+0.559581787 container died 82f485a7626c90018edd4a90364ffd9b963ca2de7486acf4dde8d8a6ed65956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T00:09:34.632 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:34 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:34.379+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:34.633 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:34 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:34.379+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:34.633 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:34 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:34.305+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:34.633 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:34 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:34.305+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:34.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:34 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:34.408+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:34.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:34 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:34.517+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:34.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:34 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:34.517+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:34.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T00:09:35.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:09:35.025 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T00:09:35.025 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T00:09:35.025 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.5 2024-09-12T00:09:35.247 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:34 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:34.950+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:35.247 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:34 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:34.950+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:35.355 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:34 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:34.877+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:35.356 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:34 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:34.877+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.798320+0000 front 2024-09-12T00:09:06.798341+0000 (oldest deadline 2024-09-12T00:09:30.298363+0000) 2024-09-12T00:09:35.356 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:35 smithi120 systemd[1]: Stopping Ceph osd.5 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:09:35.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:35.429+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:35.621 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:35.356+0000 7f8636dea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:09:35.621 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:35.356+0000 7f8636dea640 -1 osd.5 105 *** Got signal Terminated *** 2024-09-12T00:09:35.621 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:35.356+0000 7f8636dea640 -1 osd.5 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:09:35.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:35.566+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:35.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:35.566+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:35.666 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:35.248+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:35.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:35 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:35.402+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:35.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:35 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:35.402+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:35.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:35 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:35.313+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:35.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:35 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:35.313+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:36.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:35.908+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:36.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:35 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:35.908+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.798320+0000 front 2024-09-12T00:09:06.798341+0000 (oldest deadline 2024-09-12T00:09:30.298363+0000) 2024-09-12T00:09:36.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:36 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:36.207+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:36.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:35.972+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:36.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:35 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:35.972+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:36.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:36 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:36.401+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:36.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:36 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:36.401+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:36.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:36 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:36.337+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:36.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:36 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:36.337+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:36.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:36 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:36.447+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:36.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:36 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:36.520+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:36.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:36 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:36.520+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:37.246 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:36 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:36.962+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:37.247 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:36 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:36.962+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:37.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:36 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:36.940+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:37.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:36 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:36.940+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.798320+0000 front 2024-09-12T00:09:06.798341+0000 (oldest deadline 2024-09-12T00:09:30.298363+0000) 2024-09-12T00:09:37.666 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:37 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:37.247+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:37.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:37.359+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:37.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:37.359+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:37.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:37.299+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:37.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:37 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:37.299+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:37.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:37 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:37.465+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:37.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:37 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:37.503+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:37.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:37 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:37.503+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:38.217 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:37 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:37.949+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:38.218 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:37 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:37.949+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:38.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:37 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:37.979+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:38.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:37 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:37.979+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.798320+0000 front 2024-09-12T00:09:06.798341+0000 (oldest deadline 2024-09-12T00:09:30.298363+0000) 2024-09-12T00:09:38.669 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:38 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:38.218+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:38.669 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:38 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:38.218+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:38.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:38 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:38.328+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:38.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:38 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:38.328+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:38.716 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:38 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:38.328+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:10.604598+0000 front 2024-09-12T00:09:10.604508+0000 (oldest deadline 2024-09-12T00:09:37.605046+0000) 2024-09-12T00:09:38.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:38 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:38.340+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:38.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:38 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:38.340+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:38.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:38 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:38.340+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:38.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:38 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:38.448+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:38.873 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:38 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:38.485+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:38.873 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:38 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:38.485+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:39.208 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:38 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:38.941+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:39.208 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:38 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:38.941+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:39.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:38 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:38.980+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:39.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:38 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:38.980+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.798320+0000 front 2024-09-12T00:09:06.798341+0000 (oldest deadline 2024-09-12T00:09:30.298363+0000) 2024-09-12T00:09:39.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:38 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:38.980+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:13.799369+0000 front 2024-09-12T00:09:13.799405+0000 (oldest deadline 2024-09-12T00:09:38.499536+0000) 2024-09-12T00:09:39.666 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:39 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:39.209+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:39.666 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:39 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:39.209+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:39.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:39 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:39.287+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:39.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:39 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:39.287+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:39.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:39 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:39.287+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:10.604598+0000 front 2024-09-12T00:09:10.604508+0000 (oldest deadline 2024-09-12T00:09:37.605046+0000) 2024-09-12T00:09:39.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:39 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:39.387+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:39.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:39 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:39.387+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:39.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:39 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:39.387+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:39.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:39 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:39.419+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:39.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:39 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:39.525+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:39.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:39 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:39.525+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:40.166 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:39 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:39.897+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:40.167 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:39 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:39.897+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:40.167 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:39 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:39.897+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:40.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:39 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:39.997+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.996444+0000 front 2024-09-12T00:08:56.996254+0000 (oldest deadline 2024-09-12T00:09:21.096324+0000) 2024-09-12T00:09:40.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:39 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:39.997+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.798320+0000 front 2024-09-12T00:09:06.798341+0000 (oldest deadline 2024-09-12T00:09:30.298363+0000) 2024-09-12T00:09:40.370 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:39 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5[46863]: 2024-09-12T00:09:39.997+0000 7f862f3f8640 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:13.799369+0000 front 2024-09-12T00:09:13.799405+0000 (oldest deadline 2024-09-12T00:09:38.499536+0000) 2024-09-12T00:09:40.433 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:40 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:40.192+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:40.433 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:40 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:40.192+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:40.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:40 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:40.263+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:40.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:40 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:40.263+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:40.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:40 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:40.263+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:10.604598+0000 front 2024-09-12T00:09:10.604508+0000 (oldest deadline 2024-09-12T00:09:37.605046+0000) 2024-09-12T00:09:40.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:40 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:40.362+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:40.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:40 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:40.362+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:40.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:40 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:40.362+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:40.859 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:40 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:40.458+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:40.860 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:40 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:40.514+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:40.860 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:40 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:40.514+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:41.121 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:40 smithi120 podman[72967]: 2024-09-12 00:09:40.861623507 +0000 UTC m=+5.636694191 container died 2b358fcec06d5654bb62cdbc022cc1235a1d8afc709bf3e45810197c0377f57b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T00:09:41.381 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:41 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:41.176+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:41.381 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:41 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:41.176+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:41.381 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:40 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:40.931+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:41.382 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:40 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:40.931+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:41.382 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:40 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:40.931+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:41.383 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:41 smithi120 podman[72967]: 2024-09-12 00:09:41.264663369 +0000 UTC m=+6.039734049 container cleanup 2b358fcec06d5654bb62cdbc022cc1235a1d8afc709bf3e45810197c0377f57b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T00:09:41.383 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:41 smithi120 bash[72967]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5 2024-09-12T00:09:41.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:41 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:41.312+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:41.719 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:41 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:41.312+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:41.719 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:41 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:41.312+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:10.604598+0000 front 2024-09-12T00:09:10.604508+0000 (oldest deadline 2024-09-12T00:09:37.605046+0000) 2024-09-12T00:09:41.719 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:41 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:41.353+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:41.719 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:41 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:41.353+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:41.719 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:41 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:41.353+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:41.875 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:41 smithi120 podman[72985]: 2024-09-12 00:09:41.516105412 +0000 UTC m=+0.655912341 container remove 2b358fcec06d5654bb62cdbc022cc1235a1d8afc709bf3e45810197c0377f57b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T00:09:41.876 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:41 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:41.432+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:41.876 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:41 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:41.432+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:41.876 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:41 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:41.541+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:41.876 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:41 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:41.541+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:42.185 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:41 smithi120 podman[73083]: 2024-09-12 00:09:41.816011424 +0000 UTC m=+0.015973934 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:09:42.185 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:41 smithi120 podman[73083]: 2024-09-12 00:09:41.942194669 +0000 UTC m=+0.142157176 container create a7afeb0ae14c1710944c9cada09305c12d9ce483a85b5f1a34e38c4898c62000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T00:09:42.185 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:42 smithi120 podman[73083]: 2024-09-12 00:09:42.18635731 +0000 UTC m=+0.386319820 container init a7afeb0ae14c1710944c9cada09305c12d9ce483a85b5f1a34e38c4898c62000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T00:09:42.220 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:41 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:41.951+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:42.220 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:41 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:41.951+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:42.220 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:41 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:41.951+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:42.562 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:42 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:42.417+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:42.562 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:42 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:42.417+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:42.562 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:42 smithi120 podman[73083]: 2024-09-12 00:09:42.19001594 +0000 UTC m=+0.389978460 container start a7afeb0ae14c1710944c9cada09305c12d9ce483a85b5f1a34e38c4898c62000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T00:09:42.562 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:42 smithi120 podman[73083]: 2024-09-12 00:09:42.269699649 +0000 UTC m=+0.469662164 container attach a7afeb0ae14c1710944c9cada09305c12d9ce483a85b5f1a34e38c4898c62000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T00:09:42.562 INFO:journalctl@ceph.osd.5.smithi120.stdout:Sep 12 00:09:42 smithi120 podman[73083]: 2024-09-12 00:09:42.380361387 +0000 UTC m=+0.580323904 container died a7afeb0ae14c1710944c9cada09305c12d9ce483a85b5f1a34e38c4898c62000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T00:09:42.666 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:42 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:42.221+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:42.667 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:42 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:42.221+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:42.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:42 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:42.359+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:42.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:42 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:42.359+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:42.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:42 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:42.359+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:10.604598+0000 front 2024-09-12T00:09:10.604508+0000 (oldest deadline 2024-09-12T00:09:37.605046+0000) 2024-09-12T00:09:42.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:42 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:42.315+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:42.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:42 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:42.315+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:42.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:42 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:42.315+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:42.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:42 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:42.562+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:42.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:42 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:42.562+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:42.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:42 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:42.562+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:43.025 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T00:09:43.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:09:43.066 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T00:09:43.066 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T00:09:43.066 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.6 2024-09-12T00:09:43.209 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:42 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:42.950+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:43.210 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:42 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:42.950+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:43.210 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:42 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:42.950+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:43.381 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:43 smithi062 systemd[1]: Stopping Ceph osd.6 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:09:43.385 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:43.309+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:43.386 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:43.309+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:43.386 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:43.309+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:43.666 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:43 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:43.210+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:43.667 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:43 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:43.210+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:43.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:43.381+0000 7ff32b5c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:09:43.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:43.381+0000 7ff32b5c8640 -1 osd.6 105 *** Got signal Terminated *** 2024-09-12T00:09:43.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:43.381+0000 7ff32b5c8640 -1 osd.6 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:09:43.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:43.384+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:43.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:43.384+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:43.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:43.384+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:10.604598+0000 front 2024-09-12T00:09:10.604508+0000 (oldest deadline 2024-09-12T00:09:37.605046+0000) 2024-09-12T00:09:43.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:43 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:43.467+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:43.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:43 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:43.467+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:43.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:43 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:43.533+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:43.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:43 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:43.533+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:43.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:43 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:43.533+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:44.418 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:44 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:44.161+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:44.418 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:44 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:44.161+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:44.418 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:43 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:43.967+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:44.418 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:43 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:43.967+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:44.418 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:43 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:43.967+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:44.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:44.410+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:44.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:44.410+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:44.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:44.410+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:10.604598+0000 front 2024-09-12T00:09:10.604508+0000 (oldest deadline 2024-09-12T00:09:37.605046+0000) 2024-09-12T00:09:44.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:44.275+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:44.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:44.275+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:44.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:44 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:44.275+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:44.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:44 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:44.439+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:44.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:44 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:44.439+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:44.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:44 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:44.502+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:44.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:44 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:44.502+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:44.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:44 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:44.502+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:45.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:45 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:45.147+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:45.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:45 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:45.147+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:45.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:45 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:45.017+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:45.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:45 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:45.017+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:45.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:45 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:45.017+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:45.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:45 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:45.367+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:45.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:45 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:45.367+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:45.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:45 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:45.367+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:10.604598+0000 front 2024-09-12T00:09:10.604508+0000 (oldest deadline 2024-09-12T00:09:37.605046+0000) 2024-09-12T00:09:45.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:45 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:45.320+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:45.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:45 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:45.320+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:45.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:45 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:45.320+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:45.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:45 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:45.477+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:45.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:45 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:45.477+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:45.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:45 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:45.548+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:45.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:45 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:45.548+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:45.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:45 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:45.548+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:46.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:46 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:46.152+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:46.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:46 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:46.152+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:46.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:46 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:46.053+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:46.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:46 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:46.053+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:46.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:46 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:46.053+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:46.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:46 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:46.366+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:46.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:46 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:46.366+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:46.715 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:46 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:46.366+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:10.604598+0000 front 2024-09-12T00:09:10.604508+0000 (oldest deadline 2024-09-12T00:09:37.605046+0000) 2024-09-12T00:09:46.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:46 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:46.352+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:46.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:46 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:46.352+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:46.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:46 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:46.352+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:46.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:46 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:46.492+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:46.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:46 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:46.492+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:46.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:46 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:46.580+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:46.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:46 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:46.580+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:46.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:46 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:46.580+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:47.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:47 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:47.138+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:47.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:47 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:47.138+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:47.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:47 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:47.025+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:47.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:47 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:47.025+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:47.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:47 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:47.025+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:47.719 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:47.390+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:47.719 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:47.390+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:47.719 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:47.390+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:47.719 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:47.415+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.502745+0000 front 2024-09-12T00:08:56.502714+0000 (oldest deadline 2024-09-12T00:09:21.802854+0000) 2024-09-12T00:09:47.720 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:47.415+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:06.504043+0000 front 2024-09-12T00:09:06.503920+0000 (oldest deadline 2024-09-12T00:09:30.603905+0000) 2024-09-12T00:09:47.720 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:47 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6[58329]: 2024-09-12T00:09:47.415+0000 7ff323bd6640 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:10.604598+0000 front 2024-09-12T00:09:10.604508+0000 (oldest deadline 2024-09-12T00:09:37.605046+0000) 2024-09-12T00:09:47.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:47 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:47.479+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:47.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:47 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:47.479+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:47.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:47 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:47.479+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:47.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:47 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:47.574+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:47.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:47 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:47.574+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:47.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:47 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:47.574+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:48.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:48 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:48.163+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:48.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:48 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:48.163+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:48.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:48 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:48.063+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:48.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:48 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:48.063+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:48.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:48 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:48.063+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:48.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:48 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:48.368+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:48.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:48 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:48.368+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:48.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:48 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:48.368+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:48.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:48 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:48.368+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:48.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:48 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:48.495+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:48.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:48 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:48.495+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:48.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:48 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:48.495+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:48.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:48 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:48.528+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:48.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:48 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:48.528+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:48.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:48 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:48.528+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:49.157 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:48 smithi062 podman[119722]: 2024-09-12 00:09:48.857285987 +0000 UTC m=+5.588272488 container died 4424946309ad21c7cee727a2e5d9c6a21fa5deef2591189c07d5cd1e9c4f8433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T00:09:49.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:49 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:49.179+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:49.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:49 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:49.179+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:49.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:49 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:49.040+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:49.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:49 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:49.040+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:49.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:49 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:49.040+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:49.465 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:49 smithi062 podman[119722]: 2024-09-12 00:09:49.349480799 +0000 UTC m=+6.080467296 container cleanup 4424946309ad21c7cee727a2e5d9c6a21fa5deef2591189c07d5cd1e9c4f8433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T00:09:49.465 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:49 smithi062 bash[119722]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6 2024-09-12T00:09:49.465 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:49 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:49.337+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:49.465 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:49 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:49.337+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:49.465 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:49 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:49.337+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:49.465 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:49 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:49.337+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:49.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:49 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:49.494+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:49.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:49 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:49.494+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:49.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:49 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:49.494+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:49.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:49 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:49.484+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:49.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:49 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:49.484+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:49.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:49 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:49.484+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:49.934 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:49 smithi062 podman[119739]: 2024-09-12 00:09:49.601355328 +0000 UTC m=+0.739028169 container remove 4424946309ad21c7cee727a2e5d9c6a21fa5deef2591189c07d5cd1e9c4f8433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T00:09:50.211 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:49 smithi062 podman[119826]: 2024-09-12 00:09:49.935726098 +0000 UTC m=+0.104255516 container create 3a17ceb8a8b1df2b8cf87d1e9e4deac1656ac225c7e640cb9cc43c447dbb50c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T00:09:50.211 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:49 smithi062 podman[119826]: 2024-09-12 00:09:49.847855648 +0000 UTC m=+0.016385010 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:09:50.211 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:50 smithi062 podman[119826]: 2024-09-12 00:09:50.121310287 +0000 UTC m=+0.289839647 container init 3a17ceb8a8b1df2b8cf87d1e9e4deac1656ac225c7e640cb9cc43c447dbb50c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T00:09:50.212 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:50 smithi062 podman[119826]: 2024-09-12 00:09:50.124771774 +0000 UTC m=+0.293301136 container start 3a17ceb8a8b1df2b8cf87d1e9e4deac1656ac225c7e640cb9cc43c447dbb50c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T00:09:50.212 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:50 smithi062 podman[119826]: 2024-09-12 00:09:50.213064341 +0000 UTC m=+0.381593716 container attach 3a17ceb8a8b1df2b8cf87d1e9e4deac1656ac225c7e640cb9cc43c447dbb50c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T00:09:50.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:50 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:50.145+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:50.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:50 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:50.145+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:50.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:50 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:50.145+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:23.493866+0000 front 2024-09-12T00:09:23.493739+0000 (oldest deadline 2024-09-12T00:09:49.393787+0000) 2024-09-12T00:09:50.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:50 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:50.087+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:50.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:50 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:50.087+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:50.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:50 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:50.087+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:50.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:50 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:50.087+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:09:50.465 INFO:journalctl@ceph.osd.6.smithi062.stdout:Sep 12 00:09:50 smithi062 podman[119826]: 2024-09-12 00:09:50.32635715 +0000 UTC m=+0.494886516 container died 3a17ceb8a8b1df2b8cf87d1e9e4deac1656ac225c7e640cb9cc43c447dbb50c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T00:09:50.465 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:50 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:50.351+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:50.466 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:50 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:50.351+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:50.466 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:50 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:50.351+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:50.466 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:50 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:50.351+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:50.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:50 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:50.469+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:50.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:50 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:50.469+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:50.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:50 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:50.469+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:50.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:50 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:50.492+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:50.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:50 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:50.492+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:50.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:50 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:50.492+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:50.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:50 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:50.492+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:09:51.013 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T00:09:51.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:09:51.056 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T00:09:51.056 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T00:09:51.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.7 2024-09-12T00:09:51.166 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:51 smithi083 systemd[1]: Stopping Ceph osd.7 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:09:51.167 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:51 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:51.098+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:51.167 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:51 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:51.098+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:51.167 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:51 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:51.098+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:51.167 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:51 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:51.098+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:09:51.442 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:51 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:51.167+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:51.442 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:51 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:51.167+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:51.442 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:51 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:51.167+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:23.493866+0000 front 2024-09-12T00:09:23.493739+0000 (oldest deadline 2024-09-12T00:09:49.393787+0000) 2024-09-12T00:09:51.443 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:51 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:51.355+0000 7f4218e5d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:09:51.443 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:51 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:51.355+0000 7f4218e5d640 -1 osd.7 105 *** Got signal Terminated *** 2024-09-12T00:09:51.443 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:51 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:51.355+0000 7f4218e5d640 -1 osd.7 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:09:51.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:51.342+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:51.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:51.342+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:51.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:51.342+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:51.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:51 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:51.342+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:51.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:51 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:51.493+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:51.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:51 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:51.493+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:51.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:51 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:51.493+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:51.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:51 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:51.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:51.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:51 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:51.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:51.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:51 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:51.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:51.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:51 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:51.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:09:52.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:52.214+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:52.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:52.214+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:52.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:52.214+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:23.493866+0000 front 2024-09-12T00:09:23.493739+0000 (oldest deadline 2024-09-12T00:09:49.393787+0000) 2024-09-12T00:09:52.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:52.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:52.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:52.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:52.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:52.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:52.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:52.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:09:52.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:52 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:52.333+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:52.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:52 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:52.333+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:52.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:52 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:52.333+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:52.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:52 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:52.333+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:52.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:52 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:52.531+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:52.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:52 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:52.531+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:52.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:52 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:52.531+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:52.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:52 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:52.432+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:52.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:52 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:52.432+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:52.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:52 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:52.432+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:52.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:52 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:52.432+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:09:53.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:53 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:53.182+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:53.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:53 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:53.182+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:53.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:53 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:53.182+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:23.493866+0000 front 2024-09-12T00:09:23.493739+0000 (oldest deadline 2024-09-12T00:09:49.393787+0000) 2024-09-12T00:09:53.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:53 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:53.084+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:53.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:53 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:53.084+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:53.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:53 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:53.084+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:53.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:53 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:53.084+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:09:53.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:53.339+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:53.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:53.339+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:53.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:53.339+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:53.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:53 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:53.339+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:53.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:53 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:53.544+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:53.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:53 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:53.544+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:53.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:53 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:53.544+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:53.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:53 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:53.413+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:53.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:53 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:53.413+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:53.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:53 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:53.413+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:53.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:53 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:53.413+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:09:53.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:53 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:53.413+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:09:54.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:54 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:54.098+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:54.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:54 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:54.098+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:54.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:54 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:54.098+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:54.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:54 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:54.098+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:09:54.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:54 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:54.157+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:54.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:54 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:54.157+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:54.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:54 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:54.157+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:23.493866+0000 front 2024-09-12T00:09:23.493739+0000 (oldest deadline 2024-09-12T00:09:49.393787+0000) 2024-09-12T00:09:54.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:54 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:54.157+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.294791+0000 front 2024-09-12T00:09:32.294712+0000 (oldest deadline 2024-09-12T00:09:53.994754+0000) 2024-09-12T00:09:54.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:54 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:54.350+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:54.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:54 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:54.350+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:54.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:54 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:54.350+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:54.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:54 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:54.350+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:54.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:54 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:54.350+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:09:54.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:54 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:54.576+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:54.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:54 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:54.576+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:54.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:54 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:54.576+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:54.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:54 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:54.383+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:54.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:54 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:54.383+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:54.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:54 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:54.383+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:54.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:54 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:54.383+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:09:54.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:54 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:54.383+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:09:55.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:55 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:55.129+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:55.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:55 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:55.129+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:55.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:55 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:55.129+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:23.493866+0000 front 2024-09-12T00:09:23.493739+0000 (oldest deadline 2024-09-12T00:09:49.393787+0000) 2024-09-12T00:09:55.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:55 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:55.129+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.294791+0000 front 2024-09-12T00:09:32.294712+0000 (oldest deadline 2024-09-12T00:09:53.994754+0000) 2024-09-12T00:09:55.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:55 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:55.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:55.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:55 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:55.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:55.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:55 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:55.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:55.418 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:55 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:55.063+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:09:55.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:55 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:55.528+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:55.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:55 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:55.528+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:55.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:55 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:55.528+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:55.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:55 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:55.528+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.718193+0000 front 2024-09-12T00:09:31.718078+0000 (oldest deadline 2024-09-12T00:09:54.618045+0000) 2024-09-12T00:09:55.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:55 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:55.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:55.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:55 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:55.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:55.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:55 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:55.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:55.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:55 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:55.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:09:55.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:55 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:55.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:09:55.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:55 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:55.386+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:55.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:55 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:55.386+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:55.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:55 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:55.386+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:55.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:55 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:55.386+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:55.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:55 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:55.386+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:09:56.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:56.106+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.189784+0000 front 2024-09-12T00:09:03.189909+0000 (oldest deadline 2024-09-12T00:09:28.490106+0000) 2024-09-12T00:09:56.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:56.106+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.891530+0000 front 2024-09-12T00:09:14.891713+0000 (oldest deadline 2024-09-12T00:09:37.791494+0000) 2024-09-12T00:09:56.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:56.106+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:23.493866+0000 front 2024-09-12T00:09:23.493739+0000 (oldest deadline 2024-09-12T00:09:49.393787+0000) 2024-09-12T00:09:56.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7[52216]: 2024-09-12T00:09:56.106+0000 7f4215473640 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.294791+0000 front 2024-09-12T00:09:32.294712+0000 (oldest deadline 2024-09-12T00:09:53.994754+0000) 2024-09-12T00:09:56.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:56.025+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:56.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:56.025+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:56.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:56.025+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:56.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:56.025+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:09:56.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:56.025+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:09:56.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:56 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:56.514+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:56.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:56 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:56.514+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:56.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:56 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:56.514+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:56.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:56 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:56.514+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.718193+0000 front 2024-09-12T00:09:31.718078+0000 (oldest deadline 2024-09-12T00:09:54.618045+0000) 2024-09-12T00:09:56.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:56 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:56.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:56.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:56 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:56.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:56.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:56 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:56.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:56.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:56 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:56.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:09:56.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:56 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:56.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:09:56.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:56 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:56.436+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:56.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:56 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:56.436+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:56.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:56 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:56.436+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:56.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:56 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:56.436+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:56.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:56 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:56.436+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:09:57.122 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:56 smithi083 podman[71602]: 2024-09-12 00:09:56.839139279 +0000 UTC m=+5.589566099 container died bee4081ba0bf9e8247f8e2101cffbe816af5df8cb98bd3ab91f3c9ae4636b73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T00:09:57.122 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:57 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:57.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:57.122 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:57 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:57.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:57.122 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:57 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:57.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:57.122 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:57 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:57.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:09:57.122 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:57 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:57.062+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:09:57.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:57 smithi083 podman[71602]: 2024-09-12 00:09:57.230383775 +0000 UTC m=+5.980810592 container cleanup bee4081ba0bf9e8247f8e2101cffbe816af5df8cb98bd3ab91f3c9ae4636b73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:09:57.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:57 smithi083 bash[71602]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7 2024-09-12T00:09:57.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:57 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:57.465+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:57.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:57 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:57.465+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:57.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:57 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:57.465+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:57.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:57 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:57.465+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:57.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:57 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:57.465+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:09:57.788 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:57 smithi083 podman[71618]: 2024-09-12 00:09:57.456688779 +0000 UTC m=+0.614954845 container remove bee4081ba0bf9e8247f8e2101cffbe816af5df8cb98bd3ab91f3c9ae4636b73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T00:09:57.870 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:57 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:57.506+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:57.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:57 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:57.506+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:57.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:57 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:57.506+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:57.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:57 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:57.506+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.718193+0000 front 2024-09-12T00:09:31.718078+0000 (oldest deadline 2024-09-12T00:09:54.618045+0000) 2024-09-12T00:09:57.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:57 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:57.380+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:57.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:57 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:57.380+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:57.872 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:57 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:57.380+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:57.872 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:57 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:57.380+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:09:57.872 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:57 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:57.380+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:09:58.083 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:57 smithi083 podman[71709]: 2024-09-12 00:09:57.692687847 +0000 UTC m=+0.015882683 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:09:58.083 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:57 smithi083 podman[71709]: 2024-09-12 00:09:57.832520685 +0000 UTC m=+0.155715515 container create 1c4fe87ee554a757432c7dcfb4512165dc9dc3f472f59d7d7ce17c650e3ec499 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T00:09:58.083 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:58 smithi083 podman[71709]: 2024-09-12 00:09:58.085098658 +0000 UTC m=+0.408293489 container init 1c4fe87ee554a757432c7dcfb4512165dc9dc3f472f59d7d7ce17c650e3ec499 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T00:09:58.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:58 smithi083 podman[71709]: 2024-09-12 00:09:58.088320751 +0000 UTC m=+0.411515588 container start 1c4fe87ee554a757432c7dcfb4512165dc9dc3f472f59d7d7ce17c650e3ec499 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:09:58.416 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:58 smithi083 podman[71709]: 2024-09-12 00:09:58.201700619 +0000 UTC m=+0.524895456 container attach 1c4fe87ee554a757432c7dcfb4512165dc9dc3f472f59d7d7ce17c650e3ec499 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T00:09:58.417 INFO:journalctl@ceph.osd.7.smithi083.stdout:Sep 12 00:09:58 smithi083 podman[71709]: 2024-09-12 00:09:58.271935209 +0000 UTC m=+0.595130058 container died 1c4fe87ee554a757432c7dcfb4512165dc9dc3f472f59d7d7ce17c650e3ec499 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-12T00:09:58.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:58 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:58.105+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:58.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:58 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:58.105+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:58.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:58 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:58.105+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:58.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:58 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:58.105+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:09:58.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:58 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:58.105+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:09:58.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:58 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:58.518+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:58.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:58 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:58.518+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:58.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:58 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:58.518+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:58.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:58 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:58.518+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.718193+0000 front 2024-09-12T00:09:31.718078+0000 (oldest deadline 2024-09-12T00:09:54.618045+0000) 2024-09-12T00:09:58.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:58 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:58.343+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:58.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:58 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:58.343+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:58.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:58 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:58.343+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:58.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:58 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:58.343+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:09:58.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:58 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:58.343+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:09:58.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:58 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:58.484+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:58.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:58 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:58.484+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:58.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:58 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:58.484+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:58.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:58 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:58.484+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:58.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:58 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:58.484+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:09:58.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T00:09:59.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:09:59.039 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T00:09:59.039 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T00:09:59.039 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.8 2024-09-12T00:09:59.367 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:59 smithi120 systemd[1]: Stopping Ceph osd.8 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:09:59.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:59 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:59.084+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:09:59.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:59 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:59.084+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:09:59.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:59 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:59.084+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:09:59.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:59 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:59.084+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:09:59.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:09:59 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:09:59.084+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:09:59.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:59.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:09:59.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:59.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:09:59.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:59.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:09:59.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:59.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:09:59.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:09:59.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:09:59.622 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:59.371+0000 7fd28b91c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:09:59.622 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:59.371+0000 7fd28b91c640 -1 osd.8 105 *** Got signal Terminated *** 2024-09-12T00:09:59.622 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:59.371+0000 7fd28b91c640 -1 osd.8 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:09:59.622 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:59.531+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:09:59.622 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:59.531+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:09:59.622 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:59.531+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:09:59.622 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:09:59 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:09:59.531+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.718193+0000 front 2024-09-12T00:09:31.718078+0000 (oldest deadline 2024-09-12T00:09:54.618045+0000) 2024-09-12T00:09:59.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:59 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:59.477+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:09:59.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:59 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:59.477+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:09:59.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:59 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:59.477+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:09:59.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:59 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:59.477+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:09:59.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:09:59 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:09:59.477+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:00.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:00.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:00.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:00.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:00.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:00.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:00.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:00.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:00.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:00.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:00.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:00.572+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:10:00.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:00.572+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:10:00.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:00.572+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:10:00.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:00.572+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.718193+0000 front 2024-09-12T00:09:31.718078+0000 (oldest deadline 2024-09-12T00:09:54.618045+0000) 2024-09-12T00:10:00.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:00.338+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:00.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:00.338+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:00.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:00.338+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:00.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:00.338+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:00.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:00 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:00.338+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:00.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:00.458+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:00.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:00.458+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:00.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:00.458+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:00.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:00.458+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:00.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:00 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:00.458+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:01.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:01 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:01.061+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:01.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:01 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:01.061+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:01.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:01 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:01.061+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:01.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:01 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:01.061+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:01.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:01 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:01.061+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:01.554 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:01.352+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:01.554 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:01.352+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:01.554 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:01.352+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:01.554 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:01.352+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:01.554 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:01.352+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:01.776 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:01 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:01.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:01.776 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:01 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:01.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:01.776 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:01 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:01.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:01.776 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:01 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:01.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:01.776 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:01 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:01.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:01.871 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:01.554+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:10:01.872 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:01.554+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:10:01.872 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:01.554+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:10:01.872 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:01 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:01.554+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.718193+0000 front 2024-09-12T00:09:31.718078+0000 (oldest deadline 2024-09-12T00:09:54.618045+0000) 2024-09-12T00:10:02.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:02 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:02.102+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:02.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:02 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:02.102+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:02.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:02 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:02.102+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:02.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:02 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:02.102+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:02.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:02 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:02.102+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:02.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:02.601+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:10:02.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:02.601+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:10:02.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:02.601+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:10:02.622 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:02.601+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.718193+0000 front 2024-09-12T00:09:31.718078+0000 (oldest deadline 2024-09-12T00:09:54.618045+0000) 2024-09-12T00:10:02.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:02.368+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:02.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:02.368+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:02.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:02.368+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:02.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:02.368+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:02.623 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:02 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:02.368+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:02.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:02 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:02.522+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:02.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:02 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:02.522+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:02.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:02 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:02.522+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:02.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:02 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:02.522+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:02.716 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:02 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:02.522+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:03.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:03.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:03.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:03.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:03.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:03.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:03.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:03.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:03.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:03.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:03.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:03 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:03.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:03.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:03.557+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:03.913203+0000 front 2024-09-12T00:09:03.913129+0000 (oldest deadline 2024-09-12T00:09:28.613056+0000) 2024-09-12T00:10:03.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:03.557+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:12.014592+0000 front 2024-09-12T00:09:12.014659+0000 (oldest deadline 2024-09-12T00:09:40.815185+0000) 2024-09-12T00:10:03.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:03.557+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:20.815857+0000 front 2024-09-12T00:09:20.815739+0000 (oldest deadline 2024-09-12T00:09:46.715623+0000) 2024-09-12T00:10:03.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:03.557+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.718193+0000 front 2024-09-12T00:09:31.718078+0000 (oldest deadline 2024-09-12T00:09:54.618045+0000) 2024-09-12T00:10:03.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8[53493]: 2024-09-12T00:10:03.557+0000 7fd287f32640 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:37.518773+0000 front 2024-09-12T00:09:37.518835+0000 (oldest deadline 2024-09-12T00:10:02.818926+0000) 2024-09-12T00:10:03.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:03.347+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:03.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:03.347+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:03.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:03.347+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:03.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:03.347+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:03.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:03 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:03.347+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:03.902 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:03.478+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:03.902 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:03.478+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:03.902 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:03.478+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:03.902 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:03.478+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:03.902 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:03 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:03.478+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:04.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:04 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:04.092+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:04.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:04 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:04.092+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:04.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:04 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:04.092+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:04.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:04 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:04.092+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:04.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:04 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:04.092+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:04.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:04 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:04.092+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:04.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:04 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:04.382+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:04.872 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:04 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:04.382+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:04.872 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:04 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:04.382+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:04.872 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:04 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:04.382+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:04.872 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:04 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:04.382+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:04.910 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:04.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:04.910 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:04.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:04.910 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:04.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:04.910 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:04.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:04.910 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:04.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:04.910 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:04 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:04.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:38.685367+0000 front 2024-09-12T00:09:38.685276+0000 (oldest deadline 2024-09-12T00:10:03.984846+0000) 2024-09-12T00:10:05.328 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:05 smithi120 podman[73188]: 2024-09-12 00:10:05.020681178 +0000 UTC m=+5.781336001 container died dbc5bf1a6a5cb67b687e80eebb8ce1e2febc7cf21efac7825c629edddd163d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:10:05.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:05 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:05.079+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:05.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:05 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:05.079+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:05.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:05 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:05.079+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:05.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:05 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:05.079+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:05.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:05 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:05.079+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:05.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:05 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:05.079+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:05.620 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:05 smithi120 podman[73188]: 2024-09-12 00:10:05.429018209 +0000 UTC m=+6.189673031 container cleanup dbc5bf1a6a5cb67b687e80eebb8ce1e2febc7cf21efac7825c629edddd163d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T00:10:05.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:05 smithi120 bash[73188]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8 2024-09-12T00:10:05.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:05 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:05.333+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:05.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:05 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:05.333+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:05.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:05 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:05.333+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:05.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:05 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:05.333+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:05.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:05 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:05.333+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:05.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:05 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:05.333+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:05.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:05.558+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:05.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:05.558+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:05.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:05.558+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:05.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:05.558+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:05.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:05.558+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:05.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:05 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:05.558+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:38.685367+0000 front 2024-09-12T00:09:38.685276+0000 (oldest deadline 2024-09-12T00:10:03.984846+0000) 2024-09-12T00:10:06.037 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:05 smithi120 podman[73208]: 2024-09-12 00:10:05.680898664 +0000 UTC m=+0.659113590 container remove dbc5bf1a6a5cb67b687e80eebb8ce1e2febc7cf21efac7825c629edddd163d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T00:10:06.307 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:06 smithi120 podman[73297]: 2024-09-12 00:10:05.941490941 +0000 UTC m=+0.016046339 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:10:06.308 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:06 smithi120 podman[73297]: 2024-09-12 00:10:06.048395295 +0000 UTC m=+0.122950690 container create 6511cd885929459b5468632504c24bca26af5b59fa092e64d6b497bd1543863d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T00:10:06.308 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:06 smithi120 podman[73297]: 2024-09-12 00:10:06.309402293 +0000 UTC m=+0.383957688 container init 6511cd885929459b5468632504c24bca26af5b59fa092e64d6b497bd1543863d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T00:10:06.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:06 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:06.113+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:06.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:06 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:06.113+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:06.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:06 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:06.113+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:06.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:06 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:06.113+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:06.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:06 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:06.113+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:06.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:06 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:06.113+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:06.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:06 smithi120 podman[73297]: 2024-09-12 00:10:06.312891981 +0000 UTC m=+0.387447376 container start 6511cd885929459b5468632504c24bca26af5b59fa092e64d6b497bd1543863d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T00:10:06.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:06 smithi120 podman[73297]: 2024-09-12 00:10:06.426300637 +0000 UTC m=+0.500856044 container attach 6511cd885929459b5468632504c24bca26af5b59fa092e64d6b497bd1543863d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:10:06.621 INFO:journalctl@ceph.osd.8.smithi120.stdout:Sep 12 00:10:06 smithi120 podman[73297]: 2024-09-12 00:10:06.499460006 +0000 UTC m=+0.574015402 container died 6511cd885929459b5468632504c24bca26af5b59fa092e64d6b497bd1543863d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T00:10:06.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:06 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:06.354+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:06.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:06 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:06.354+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:06.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:06 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:06.354+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:06.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:06 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:06.354+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:06.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:06 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:06.354+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:06.622 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:06 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:06.354+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:06.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:06 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:06.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:06.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:06 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:06.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:06.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:06 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:06.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:06.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:06 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:06.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:06.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:06 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:06.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:06.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:06 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:06.521+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:38.685367+0000 front 2024-09-12T00:09:38.685276+0000 (oldest deadline 2024-09-12T00:10:03.984846+0000) 2024-09-12T00:10:07.190 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T00:10:07.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:07.232 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T00:10:07.233 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T00:10:07.233 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.9 2024-09-12T00:10:07.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:07 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:07.157+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:07.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:07 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:07.157+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:07.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:07 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:07.157+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:07.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:07 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:07.157+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:07.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:07 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:07.157+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:07.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:07 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:07.157+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:07.440 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:07 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:07.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:07.440 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:07 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:07.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:07.441 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:07 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:07.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:07.441 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:07 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:07.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:07.441 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:07 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:07.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:07.441 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:07 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:07.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:07.541 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:07 smithi062 systemd[1]: Stopping Ceph osd.9 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:10:07.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:07.541+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:07.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:07.541+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:07.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:07.541+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:07.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:07.541+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:07.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:07.541+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:07.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:07.541+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:38.685367+0000 front 2024-09-12T00:09:38.685276+0000 (oldest deadline 2024-09-12T00:10:03.984846+0000) 2024-09-12T00:10:07.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:07.561+0000 7ff8d80a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:10:07.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:07.561+0000 7ff8d80a4640 -1 osd.9 105 *** Got signal Terminated *** 2024-09-12T00:10:07.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:07 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:07.561+0000 7ff8d80a4640 -1 osd.9 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:10:08.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:08 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:08.136+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:08.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:08 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:08.136+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:08.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:08 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:08.136+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:08.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:08 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:08.136+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:08.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:08 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:08.136+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:08.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:08 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:08.136+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:08.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:08 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:08.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:08.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:08 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:08.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:08.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:08 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:08.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:08.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:08 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:08.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:08.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:08 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:08.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:08.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:08 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:08.365+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:08.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:08.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:08.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:08.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:08.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:08.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:08.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:08.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:08.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:08.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:08.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:08 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:08.508+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:38.685367+0000 front 2024-09-12T00:09:38.685276+0000 (oldest deadline 2024-09-12T00:10:03.984846+0000) 2024-09-12T00:10:09.666 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:09 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:09.171+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:09.666 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:09 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:09.171+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:09.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:09 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:09.171+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:09.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:09 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:09.171+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:09.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:09 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:09.171+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:09.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:09 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:09.171+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:09.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:09.388+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:09.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:09.388+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:09.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:09.388+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:09.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:09.388+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:09.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:09.388+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:09.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:09 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:09.388+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:09.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:09 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:09.505+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:09.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:09 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:09.505+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:09.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:09 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:09.505+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:09.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:09 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:09.505+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:09.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:09 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:09.505+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:09.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:09 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:09.505+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:38.685367+0000 front 2024-09-12T00:09:38.685276+0000 (oldest deadline 2024-09-12T00:10:03.984846+0000) 2024-09-12T00:10:10.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:10 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:10.131+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:10.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:10 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:10.131+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:10.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:10 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:10.131+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:10.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:10 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:10.131+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:10.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:10 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:10.131+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:10.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:10 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:10.131+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:10.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:10 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:10.131+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:10.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:10.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:10.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:10.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:10.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:10.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:10.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:10.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:10.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:10.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:10.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:10 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:10.357+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:10.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:10 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:10.553+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:10.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:10 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:10.553+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:10.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:10 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:10.553+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:10.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:10 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:10.553+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:10.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:10 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:10.553+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:10.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:10 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:10.553+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:38.685367+0000 front 2024-09-12T00:09:38.685276+0000 (oldest deadline 2024-09-12T00:10:03.984846+0000) 2024-09-12T00:10:10.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:10 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:10.553+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:46.186503+0000 front 2024-09-12T00:09:46.186528+0000 (oldest deadline 2024-09-12T00:10:09.686517+0000) 2024-09-12T00:10:11.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:11 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:11.126+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:11.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:11 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:11.126+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:11.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:11 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:11.126+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:11.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:11 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:11.126+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:11.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:11 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:11.126+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:11.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:11 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:11.126+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:11.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:11 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:11.126+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:11.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:11 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:11.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:11.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:11 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:11.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:11.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:11 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:11.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:11.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:11 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:11.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:11.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:11 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:11.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:11.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:11 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:11.369+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:11.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:11.583+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:11.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:11.583+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:11.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:11.583+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:11.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:11.583+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:11.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:11.583+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:11.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:11.583+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:38.685367+0000 front 2024-09-12T00:09:38.685276+0000 (oldest deadline 2024-09-12T00:10:03.984846+0000) 2024-09-12T00:10:11.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:11 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:11.583+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:46.186503+0000 front 2024-09-12T00:09:46.186528+0000 (oldest deadline 2024-09-12T00:10:09.686517+0000) 2024-09-12T00:10:12.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:12.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:12.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:12.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:12.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:12.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:12.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:12.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:12.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:12.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:12.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:12.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:12.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:12 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:12.122+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:12.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:12.417+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:12.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:12.417+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:12.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:12.417+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:12.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:12.417+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:12.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:12.417+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:12.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:12.417+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:12.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:12 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:12.561+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.579017+0000 front 2024-09-12T00:08:59.579151+0000 (oldest deadline 2024-09-12T00:09:23.079184+0000) 2024-09-12T00:10:12.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:12 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:12.561+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.780281+0000 front 2024-09-12T00:09:04.780455+0000 (oldest deadline 2024-09-12T00:09:30.680199+0000) 2024-09-12T00:10:12.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:12 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:12.561+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.181431+0000 front 2024-09-12T00:09:14.181315+0000 (oldest deadline 2024-09-12T00:09:37.681354+0000) 2024-09-12T00:10:12.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:12 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:12.561+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:21.782270+0000 front 2024-09-12T00:09:21.782416+0000 (oldest deadline 2024-09-12T00:09:48.182856+0000) 2024-09-12T00:10:12.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:12 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:12.561+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.684050+0000 front 2024-09-12T00:09:31.684137+0000 (oldest deadline 2024-09-12T00:09:53.383884+0000) 2024-09-12T00:10:12.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:12 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:12.561+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:38.685367+0000 front 2024-09-12T00:09:38.685276+0000 (oldest deadline 2024-09-12T00:10:03.984846+0000) 2024-09-12T00:10:12.966 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:12 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9[65287]: 2024-09-12T00:10:12.561+0000 7ff8d46ba640 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:46.186503+0000 front 2024-09-12T00:09:46.186528+0000 (oldest deadline 2024-09-12T00:10:09.686517+0000) 2024-09-12T00:10:13.390 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:13 smithi062 podman[119952]: 2024-09-12 00:10:13.104288845 +0000 UTC m=+5.665259263 container died 5c7c6b23b5ab31cf2979107b3b33cee7f4eb5a84a139d494b9af96eb9e6364be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T00:10:13.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:13 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:13.112+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:13.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:13 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:13.112+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:13.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:13 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:13.112+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:13.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:13 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:13.112+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:13.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:13 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:13.112+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:13.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:13 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:13.112+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:13.418 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:13 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:13.112+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:13.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:13.370+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:13.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:13.370+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:13.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:13.370+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:13.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:13.370+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:13.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:13.370+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:13.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:13 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:13.370+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:13.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:13 smithi062 podman[119952]: 2024-09-12 00:10:13.570701589 +0000 UTC m=+6.131672006 container cleanup 5c7c6b23b5ab31cf2979107b3b33cee7f4eb5a84a139d494b9af96eb9e6364be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T00:10:13.715 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:13 smithi062 bash[119952]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9 2024-09-12T00:10:14.215 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:13 smithi062 podman[119967]: 2024-09-12 00:10:13.881138273 +0000 UTC m=+0.767066554 container remove 5c7c6b23b5ab31cf2979107b3b33cee7f4eb5a84a139d494b9af96eb9e6364be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T00:10:14.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:14.075+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:14.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:14.075+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:14.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:14.075+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:14.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:14.075+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:14.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:14.075+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:14.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:14.075+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:14.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:14 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:14.075+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:14.508 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:14 smithi062 podman[120054]: 2024-09-12 00:10:14.151860397 +0000 UTC m=+0.016482725 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:10:14.508 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:14 smithi062 podman[120054]: 2024-09-12 00:10:14.256951566 +0000 UTC m=+0.121573926 container create 95ac3e050b5313dfe6c61cc0347ab4d1904defa92d14daa7a976dfd34559a087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T00:10:14.508 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:14 smithi062 podman[120054]: 2024-09-12 00:10:14.5094071 +0000 UTC m=+0.374029427 container init 95ac3e050b5313dfe6c61cc0347ab4d1904defa92d14daa7a976dfd34559a087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T00:10:14.873 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:14 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:14.411+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:14.874 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:14 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:14.411+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:14.874 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:14 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:14.411+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:14.874 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:14 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:14.411+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:14.874 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:14 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:14.411+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:14.874 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:14 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:14.411+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:14.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:14 smithi062 podman[120054]: 2024-09-12 00:10:14.513275338 +0000 UTC m=+0.377897656 container start 95ac3e050b5313dfe6c61cc0347ab4d1904defa92d14daa7a976dfd34559a087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T00:10:14.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:14 smithi062 podman[120054]: 2024-09-12 00:10:14.584354125 +0000 UTC m=+0.448976454 container attach 95ac3e050b5313dfe6c61cc0347ab4d1904defa92d14daa7a976dfd34559a087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-12T00:10:14.965 INFO:journalctl@ceph.osd.9.smithi062.stdout:Sep 12 00:10:14 smithi062 podman[120054]: 2024-09-12 00:10:14.710292178 +0000 UTC m=+0.574914496 container died 95ac3e050b5313dfe6c61cc0347ab4d1904defa92d14daa7a976dfd34559a087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T00:10:15.308 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T00:10:15.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:15.349 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T00:10:15.349 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T00:10:15.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.10 2024-09-12T00:10:15.378 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:15.072+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:15.378 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:15.072+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:15.378 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:15.072+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:15.378 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:15.072+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:15.378 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:15.072+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:15.378 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:15.072+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:15.378 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:15.072+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:15.666 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 systemd[1]: Stopping Ceph osd.10 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:10:15.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:15 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:15.424+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:15.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:15 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:15.424+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:15.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:15 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:15.424+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:15.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:15 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:15.424+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:15.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:15 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:15.424+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:15.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:15 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:15.424+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:16.050 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:15.683+0000 7f761ae16640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:10:16.050 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:15.683+0000 7f761ae16640 -1 osd.10 105 *** Got signal Terminated *** 2024-09-12T00:10:16.050 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:15 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:15.683+0000 7f761ae16640 -1 osd.10 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:10:16.050 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:16 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:16.050+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:16.050 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:16 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:16.050+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:16.050 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:16 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:16.050+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:16.051 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:16 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:16.050+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:16.051 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:16 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:16.050+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:16.051 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:16 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:16.050+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:16.051 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:16 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:16.050+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:16.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:16 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:16.461+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:16.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:16 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:16.461+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:16.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:16 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:16.461+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:16.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:16 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:16.461+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:16.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:16 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:16.461+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:16.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:16 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:16.461+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:17.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:17 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:17.094+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:17.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:17 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:17.095+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:17.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:17 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:17.095+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:17.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:17 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:17.095+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:17.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:17 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:17.095+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:17.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:17 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:17.095+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:17.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:17 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:17.095+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:17.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:17 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:17.492+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:17.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:17 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:17.492+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:17.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:17 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:17.492+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:17.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:17 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:17.492+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:17.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:17 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:17.492+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:17.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:17 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:17.492+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:18.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:18 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:18.064+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:18.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:18 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:18.064+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:18.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:18 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:18.064+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:18.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:18 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:18.064+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:18.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:18 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:18.064+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:18.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:18 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:18.064+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:18.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:18 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:18.064+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:18.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:18 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:18.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:18.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:18 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:18.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:18.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:18 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:18.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:18.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:18 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:18.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:18.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:18 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:18.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:18.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:18 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:18.459+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:19.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:19 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:19.114+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:19.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:19 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:19.115+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:19.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:19 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:19.115+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:19.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:19 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:19.115+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:19.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:19 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:19.115+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:19.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:19 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:19.115+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:19.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:19 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:19.115+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:19.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:19 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:19.477+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:19.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:19 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:19.477+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:19.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:19 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:19.477+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:19.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:19 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:19.477+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:19.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:19 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:19.477+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:19.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:19 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:19.477+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:20.416 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:20 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:20.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:59.609478+0000 front 2024-09-12T00:08:59.609321+0000 (oldest deadline 2024-09-12T00:09:23.709494+0000) 2024-09-12T00:10:20.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:20 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:20.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:04.810477+0000 front 2024-09-12T00:09:04.810412+0000 (oldest deadline 2024-09-12T00:09:28.910592+0000) 2024-09-12T00:10:20.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:20 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:20.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:14.112352+0000 front 2024-09-12T00:09:14.112339+0000 (oldest deadline 2024-09-12T00:09:39.412116+0000) 2024-09-12T00:10:20.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:20 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:20.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:24.013546+0000 front 2024-09-12T00:09:24.013426+0000 (oldest deadline 2024-09-12T00:09:49.913436+0000) 2024-09-12T00:10:20.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:20 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:20.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:31.014287+0000 front 2024-09-12T00:09:31.014330+0000 (oldest deadline 2024-09-12T00:09:55.114424+0000) 2024-09-12T00:10:20.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:20 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:20.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.116674+0000 front 2024-09-12T00:09:40.116529+0000 (oldest deadline 2024-09-12T00:10:02.416317+0000) 2024-09-12T00:10:20.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:20 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10[59201]: 2024-09-12T00:10:20.101+0000 7f761742c640 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-09-12T00:09:45.917610+0000 front 2024-09-12T00:09:45.917526+0000 (oldest deadline 2024-09-12T00:10:10.017287+0000) 2024-09-12T00:10:20.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:20 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:20.490+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:20.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:20 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:20.490+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:20.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:20 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:20.490+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:20.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:20 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:20.490+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:20.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:20 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:20.490+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:20.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:20 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:20.490+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:21.417 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:21 smithi083 podman[71826]: 2024-09-12 00:10:21.17518118 +0000 UTC m=+5.631371204 container died 8e8d2d85217bba56aa95028ec3549b2d7f556b8d2954ce129c530d5c0d323199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:10:21.729 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:21 smithi083 podman[71826]: 2024-09-12 00:10:21.541111585 +0000 UTC m=+5.997301608 container cleanup 8e8d2d85217bba56aa95028ec3549b2d7f556b8d2954ce129c530d5c0d323199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T00:10:21.729 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:21 smithi083 bash[71826]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10 2024-09-12T00:10:21.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:21 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:21.518+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:21.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:21 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:21.518+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:21.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:21 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:21.518+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:21.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:21 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:21.518+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:21.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:21 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:21.518+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:21.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:21 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:21.518+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:21.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:21 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:21.518+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-09-12T00:09:55.458996+0000 front 2024-09-12T00:09:55.458922+0000 (oldest deadline 2024-09-12T00:10:20.758893+0000) 2024-09-12T00:10:22.137 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:21 smithi083 podman[71841]: 2024-09-12 00:10:21.817691574 +0000 UTC m=+0.640869872 container remove 8e8d2d85217bba56aa95028ec3549b2d7f556b8d2954ce129c530d5c0d323199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T00:10:22.390 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:22 smithi083 podman[71928]: 2024-09-12 00:10:22.042011399 +0000 UTC m=+0.015990640 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:10:22.390 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:22 smithi083 podman[71928]: 2024-09-12 00:10:22.143479424 +0000 UTC m=+0.117458717 container create 26e6683f1513fdadd806891b9a613dbf88854a6975a7e909b6f074ad691353ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:10:22.390 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:22 smithi083 podman[71928]: 2024-09-12 00:10:22.387930309 +0000 UTC m=+0.361909556 container init 26e6683f1513fdadd806891b9a613dbf88854a6975a7e909b6f074ad691353ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T00:10:22.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:22 smithi083 podman[71928]: 2024-09-12 00:10:22.391792737 +0000 UTC m=+0.365771984 container start 26e6683f1513fdadd806891b9a613dbf88854a6975a7e909b6f074ad691353ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T00:10:22.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:22 smithi083 podman[71928]: 2024-09-12 00:10:22.496026466 +0000 UTC m=+0.470005707 container attach 26e6683f1513fdadd806891b9a613dbf88854a6975a7e909b6f074ad691353ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T00:10:22.667 INFO:journalctl@ceph.osd.10.smithi083.stdout:Sep 12 00:10:22 smithi083 podman[71928]: 2024-09-12 00:10:22.583209434 +0000 UTC m=+0.557188675 container died 26e6683f1513fdadd806891b9a613dbf88854a6975a7e909b6f074ad691353ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:10:22.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:22 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:22.502+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:22.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:22 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:22.502+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:22.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:22 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:22.502+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:22.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:22 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:22.502+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:22.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:22 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:22.502+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:22.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:22 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:22.502+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:22.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:22 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:22.502+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-09-12T00:09:55.458996+0000 front 2024-09-12T00:09:55.458922+0000 (oldest deadline 2024-09-12T00:10:20.758893+0000) 2024-09-12T00:10:23.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T00:10:23.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:23.306 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T00:10:23.306 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T00:10:23.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@osd.11 2024-09-12T00:10:23.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 systemd[1]: Stopping Ceph osd.11 for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:10:23.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:23.505+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:23.620 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:23.506+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:23.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:23.506+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:23.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:23.506+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:23.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:23.506+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:23.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:23.506+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:23.621 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:23.506+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-09-12T00:09:55.458996+0000 front 2024-09-12T00:09:55.458922+0000 (oldest deadline 2024-09-12T00:10:20.758893+0000) 2024-09-12T00:10:24.120 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:23.652+0000 7f03daf80640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:10:24.120 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:23.652+0000 7f03daf80640 -1 osd.11 105 *** Got signal Terminated *** 2024-09-12T00:10:24.120 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:23 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:23.652+0000 7f03daf80640 -1 osd.11 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T00:10:24.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:24 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:24.457+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:24.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:24 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:24.457+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:24.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:24 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:24.457+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:24.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:24 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:24.457+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:24.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:24 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:24.457+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:24.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:24 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:24.457+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:24.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:24 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:24.457+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-09-12T00:09:55.458996+0000 front 2024-09-12T00:09:55.458922+0000 (oldest deadline 2024-09-12T00:10:20.758893+0000) 2024-09-12T00:10:25.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:25 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:25.489+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:25.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:25 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:25.489+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:25.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:25 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:25.489+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:25.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:25 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:25.489+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:25.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:25 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:25.489+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:25.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:25 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:25.489+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:25.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:25 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:25.489+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-09-12T00:09:55.458996+0000 front 2024-09-12T00:09:55.458922+0000 (oldest deadline 2024-09-12T00:10:20.758893+0000) 2024-09-12T00:10:26.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:26 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:26.530+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:26.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:26 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:26.530+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:26.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:26 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:26.530+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:26.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:26 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:26.530+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:26.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:26 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:26.530+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:26.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:26 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:26.530+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:26.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:26 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:26.530+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-09-12T00:09:55.458996+0000 front 2024-09-12T00:09:55.458922+0000 (oldest deadline 2024-09-12T00:10:20.758893+0000) 2024-09-12T00:10:27.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:27.494+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:27.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:27.494+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:27.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:27.494+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:27.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:27.494+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:27.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:27.494+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:27.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:27.494+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:27.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:27.494+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-09-12T00:09:55.458996+0000 front 2024-09-12T00:09:55.458922+0000 (oldest deadline 2024-09-12T00:10:20.758893+0000) 2024-09-12T00:10:27.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:27 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:27.494+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6820 osd.8 since back 2024-09-12T00:10:02.960315+0000 front 2024-09-12T00:10:02.960366+0000 (oldest deadline 2024-09-12T00:10:27.060418+0000) 2024-09-12T00:10:28.870 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:28 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:28.448+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-09-12T00:08:56.050029+0000 front 2024-09-12T00:08:56.049999+0000 (oldest deadline 2024-09-12T00:09:20.650672+0000) 2024-09-12T00:10:28.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:28 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:28.448+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-09-12T00:09:05.951698+0000 front 2024-09-12T00:09:05.951752+0000 (oldest deadline 2024-09-12T00:09:31.851853+0000) 2024-09-12T00:10:28.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:28 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:28.448+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6804 osd.2 since back 2024-09-12T00:09:11.852304+0000 front 2024-09-12T00:09:11.852231+0000 (oldest deadline 2024-09-12T00:09:42.452741+0000) 2024-09-12T00:10:28.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:28 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:28.448+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-09-12T00:09:22.453315+0000 front 2024-09-12T00:09:22.453420+0000 (oldest deadline 2024-09-12T00:09:50.053787+0000) 2024-09-12T00:10:28.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:28 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:28.448+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-09-12T00:09:32.354784+0000 front 2024-09-12T00:09:32.354821+0000 (oldest deadline 2024-09-12T00:09:53.354866+0000) 2024-09-12T00:10:28.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:28 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:28.448+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6812 osd.5 since back 2024-09-12T00:09:40.356106+0000 front 2024-09-12T00:09:40.356153+0000 (oldest deadline 2024-09-12T00:10:04.456215+0000) 2024-09-12T00:10:28.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:28 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:28.448+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-09-12T00:09:55.458996+0000 front 2024-09-12T00:09:55.458922+0000 (oldest deadline 2024-09-12T00:10:20.758893+0000) 2024-09-12T00:10:28.871 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:28 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11[60480]: 2024-09-12T00:10:28.448+0000 7f03d7596640 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.120:6820 osd.8 since back 2024-09-12T00:10:02.960315+0000 front 2024-09-12T00:10:02.960366+0000 (oldest deadline 2024-09-12T00:10:27.060418+0000) 2024-09-12T00:10:29.424 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:29 smithi120 podman[73414]: 2024-09-12 00:10:29.165989477 +0000 UTC m=+5.658582906 container died b699d2e4f0f2eff1d6ab3003aa9dcadf8fea3359543b167f48fa35f68ec7a59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T00:10:29.799 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:29 smithi120 podman[73414]: 2024-09-12 00:10:29.51632667 +0000 UTC m=+6.008920099 container cleanup b699d2e4f0f2eff1d6ab3003aa9dcadf8fea3359543b167f48fa35f68ec7a59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T00:10:29.799 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:29 smithi120 bash[73414]: ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11 2024-09-12T00:10:30.120 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:29 smithi120 podman[73434]: 2024-09-12 00:10:29.80103178 +0000 UTC m=+0.630534469 container remove b699d2e4f0f2eff1d6ab3003aa9dcadf8fea3359543b167f48fa35f68ec7a59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T00:10:30.444 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:30 smithi120 podman[73521]: 2024-09-12 00:10:30.030145703 +0000 UTC m=+0.015729659 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:10:30.444 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:30 smithi120 podman[73521]: 2024-09-12 00:10:30.135170171 +0000 UTC m=+0.120754118 container create 2cc3cef398f637f9743b8d9be87da779bb4c3d5f645e00bd2cd1d75d30e6aab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T00:10:30.445 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:30 smithi120 podman[73521]: 2024-09-12 00:10:30.36285366 +0000 UTC m=+0.348437607 container init 2cc3cef398f637f9743b8d9be87da779bb4c3d5f645e00bd2cd1d75d30e6aab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T00:10:30.445 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:30 smithi120 podman[73521]: 2024-09-12 00:10:30.366043848 +0000 UTC m=+0.351627795 container start 2cc3cef398f637f9743b8d9be87da779bb4c3d5f645e00bd2cd1d75d30e6aab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T00:10:30.445 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:30 smithi120 podman[73521]: 2024-09-12 00:10:30.445968159 +0000 UTC m=+0.431552120 container attach 2cc3cef398f637f9743b8d9be87da779bb4c3d5f645e00bd2cd1d75d30e6aab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:10:30.855 INFO:journalctl@ceph.osd.11.smithi120.stdout:Sep 12 00:10:30 smithi120 podman[73521]: 2024-09-12 00:10:30.555833988 +0000 UTC m=+0.541417936 container died 2cc3cef398f637f9743b8d9be87da779bb4c3d5f645e00bd2cd1d75d30e6aab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T00:10:31.162 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T00:10:31.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:31.201 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T00:10:31.201 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T00:10:31.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.a 2024-09-12T00:10:31.506 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 12 00:10:31 smithi062 systemd[1]: Stopping Ceph mds.a for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:10:31.965 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 12 00:10:31 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-a[72463]: 2024-09-12T00:10:31.506+0000 7f50f7216640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:10:31.965 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 12 00:10:31 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-a[72463]: 2024-09-12T00:10:31.506+0000 7f50f7216640 -1 mds.a *** got signal Terminated *** 2024-09-12T00:10:34.465 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 12 00:10:34 smithi062 podman[120156]: 2024-09-12 00:10:34.153432155 +0000 UTC m=+2.751911453 container died 0ee0b356bad1fc23048d5968cffe6d72610bb3287acbb12b3cc3593b3890df84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T00:10:34.748 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 12 00:10:34 smithi062 podman[120156]: 2024-09-12 00:10:34.548234531 +0000 UTC m=+3.146713830 container cleanup 0ee0b356bad1fc23048d5968cffe6d72610bb3287acbb12b3cc3593b3890df84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T00:10:34.748 INFO:journalctl@ceph.mds.a.smithi062.stdout:Sep 12 00:10:34 smithi062 bash[120156]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-a 2024-09-12T00:10:34.882 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T00:10:34.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:34.918 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T00:10:34.918 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T00:10:34.918 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.d 2024-09-12T00:10:35.215 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 12 00:10:34 smithi062 systemd[1]: Stopping Ceph mds.d for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:10:35.715 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 12 00:10:35 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-d[71285]: 2024-09-12T00:10:35.253+0000 7f1f2821e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:10:35.715 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 12 00:10:35 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-d[71285]: 2024-09-12T00:10:35.253+0000 7f1f2821e640 -1 mds.d *** got signal Terminated *** 2024-09-12T00:10:39.340 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 12 00:10:39 smithi062 podman[120278]: 2024-09-12 00:10:39.040922513 +0000 UTC m=+3.900609853 container died 2d408df0c1e5dddfa72418fa5cabc67dc871a8f8bd1c22b35fdd24f6b87fa773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:10:39.715 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 12 00:10:39 smithi062 podman[120278]: 2024-09-12 00:10:39.502996416 +0000 UTC m=+4.362683788 container cleanup 2d408df0c1e5dddfa72418fa5cabc67dc871a8f8bd1c22b35fdd24f6b87fa773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T00:10:39.715 INFO:journalctl@ceph.mds.d.smithi062.stdout:Sep 12 00:10:39 smithi062 bash[120278]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-d 2024-09-12T00:10:39.889 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T00:10:39.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:39.925 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T00:10:39.926 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T00:10:39.926 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.g 2024-09-12T00:10:40.465 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 12 00:10:40 smithi062 systemd[1]: Stopping Ceph mds.g for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:10:40.465 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 12 00:10:40 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-g[72080]: 2024-09-12T00:10:40.249+0000 7f2608c8f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:10:40.465 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 12 00:10:40 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-g[72080]: 2024-09-12T00:10:40.249+0000 7f2608c8f640 -1 mds.g *** got signal Terminated *** 2024-09-12T00:10:42.964 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 12 00:10:42 smithi062 podman[120412]: 2024-09-12 00:10:42.689579877 +0000 UTC m=+2.532520241 container died 4589a60950bf4029192d42e5a65e55d0077760eed2b0b20ba6095fde2d19f38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T00:10:43.215 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 12 00:10:43 smithi062 podman[120412]: 2024-09-12 00:10:43.132328703 +0000 UTC m=+2.975269063 container cleanup 4589a60950bf4029192d42e5a65e55d0077760eed2b0b20ba6095fde2d19f38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T00:10:43.215 INFO:journalctl@ceph.mds.g.smithi062.stdout:Sep 12 00:10:43 smithi062 bash[120412]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-g 2024-09-12T00:10:43.562 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T00:10:43.597 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:43.597 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T00:10:43.597 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T00:10:43.597 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.j 2024-09-12T00:10:43.919 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 12 00:10:43 smithi062 systemd[1]: Stopping Ceph mds.j for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:10:44.215 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 12 00:10:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-j[70875]: 2024-09-12T00:10:43.919+0000 7f3ce9cec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:10:44.215 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 12 00:10:43 smithi062 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-j[70875]: 2024-09-12T00:10:43.919+0000 7f3ce9cec640 -1 mds.j *** got signal Terminated *** 2024-09-12T00:10:47.652 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 12 00:10:47 smithi062 podman[120535]: 2024-09-12 00:10:47.397783108 +0000 UTC m=+3.579799075 container died 200292cae1191b3f6574068ce6bb59cffbed917465a2e609ac979a8710441cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T00:10:48.293 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 12 00:10:47 smithi062 podman[120535]: 2024-09-12 00:10:47.98737262 +0000 UTC m=+4.169388627 container cleanup 200292cae1191b3f6574068ce6bb59cffbed917465a2e609ac979a8710441cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T00:10:48.405 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 12 00:10:47 smithi062 bash[120535]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-j 2024-09-12T00:10:48.405 INFO:journalctl@ceph.mds.j.smithi062.stdout:Sep 12 00:10:48 smithi062 podman[120548]: 2024-09-12 00:10:48.188617106 +0000 UTC m=+0.786915145 container remove 200292cae1191b3f6574068ce6bb59cffbed917465a2e609ac979a8710441cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T00:10:48.407 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T00:10:48.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:48.441 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T00:10:48.441 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T00:10:48.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.b 2024-09-12T00:10:48.740 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 12 00:10:48 smithi083 systemd[1]: Stopping Ceph mds.b for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:10:48.997 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 12 00:10:48 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b[64402]: 2024-09-12T00:10:48.741+0000 7fd9f6cf2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:10:48.998 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 12 00:10:48 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b[64402]: 2024-09-12T00:10:48.741+0000 7fd9f6cf2640 -1 mds.b *** got signal Terminated *** 2024-09-12T00:10:51.412 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 12 00:10:51 smithi083 podman[72029]: 2024-09-12 00:10:51.108733791 +0000 UTC m=+2.472946323 container died 0c5fa4907a3f4af0398f02771cd99af69238a94e0c05ba523743e93646ae0c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:10:51.673 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 12 00:10:51 smithi083 podman[72029]: 2024-09-12 00:10:51.499984149 +0000 UTC m=+2.864196680 container cleanup 0c5fa4907a3f4af0398f02771cd99af69238a94e0c05ba523743e93646ae0c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T00:10:51.673 INFO:journalctl@ceph.mds.b.smithi083.stdout:Sep 12 00:10:51 smithi083 bash[72029]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-b 2024-09-12T00:10:51.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T00:10:51.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:51.897 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T00:10:51.898 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T00:10:51.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.e 2024-09-12T00:10:52.416 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 12 00:10:51 smithi083 systemd[1]: Stopping Ceph mds.e for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:10:52.416 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 12 00:10:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e[63786]: 2024-09-12T00:10:52.236+0000 7f471c7ac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:10:52.417 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 12 00:10:52 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e[63786]: 2024-09-12T00:10:52.236+0000 7f471c7ac640 -1 mds.e *** got signal Terminated *** 2024-09-12T00:10:55.416 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 12 00:10:55 smithi083 podman[72162]: 2024-09-12 00:10:55.104116128 +0000 UTC m=+2.976256521 container died f8cbd6eead053188c46dd19c86db5afb333af9037862f5c3771087962062957d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T00:10:55.752 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 12 00:10:55 smithi083 podman[72162]: 2024-09-12 00:10:55.520224593 +0000 UTC m=+3.392364940 container cleanup f8cbd6eead053188c46dd19c86db5afb333af9037862f5c3771087962062957d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T00:10:55.753 INFO:journalctl@ceph.mds.e.smithi083.stdout:Sep 12 00:10:55 smithi083 bash[72162]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-e 2024-09-12T00:10:55.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T00:10:55.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:55.939 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T00:10:55.939 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T00:10:55.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.h 2024-09-12T00:10:56.275 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 12 00:10:56 smithi083 systemd[1]: Stopping Ceph mds.h for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:10:56.666 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 12 00:10:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h[64086]: 2024-09-12T00:10:56.274+0000 7f5778eac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:10:56.666 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 12 00:10:56 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h[64086]: 2024-09-12T00:10:56.274+0000 7f5778eac640 -1 mds.h *** got signal Terminated *** 2024-09-12T00:10:59.367 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 12 00:10:59 smithi083 podman[72280]: 2024-09-12 00:10:59.091953381 +0000 UTC m=+2.936128675 container died e72378e7512a938e859583a07f8a1b2075e58f6e4d5ec63af432681d3a918112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T00:10:59.666 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 12 00:10:59 smithi083 podman[72280]: 2024-09-12 00:10:59.476237577 +0000 UTC m=+3.320412877 container cleanup e72378e7512a938e859583a07f8a1b2075e58f6e4d5ec63af432681d3a918112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T00:10:59.666 INFO:journalctl@ceph.mds.h.smithi083.stdout:Sep 12 00:10:59 smithi083 bash[72280]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-h 2024-09-12T00:10:59.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T00:10:59.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:10:59.935 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T00:10:59.935 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T00:10:59.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.k 2024-09-12T00:11:00.416 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 12 00:11:00 smithi083 systemd[1]: Stopping Ceph mds.k for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:11:00.417 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 12 00:11:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-k[63473]: 2024-09-12T00:11:00.254+0000 7ff4bacf2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:11:00.417 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 12 00:11:00 smithi083 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-k[63473]: 2024-09-12T00:11:00.254+0000 7ff4bacf2640 -1 mds.k *** got signal Terminated *** 2024-09-12T00:11:04.561 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 12 00:11:04 smithi083 podman[72401]: 2024-09-12 00:11:04.26659903 +0000 UTC m=+4.114471671 container died b49664dd6ce399d04c2d05d250624485ae599aba8e0544fdb9fb12475195a232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:11:04.870 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 12 00:11:04 smithi083 podman[72401]: 2024-09-12 00:11:04.646214814 +0000 UTC m=+4.494087444 container cleanup b49664dd6ce399d04c2d05d250624485ae599aba8e0544fdb9fb12475195a232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T00:11:04.870 INFO:journalctl@ceph.mds.k.smithi083.stdout:Sep 12 00:11:04 smithi083 bash[72401]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-k 2024-09-12T00:11:04.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T00:11:05.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:11:05.026 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T00:11:05.027 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T00:11:05.027 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.c 2024-09-12T00:11:05.323 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 12 00:11:05 smithi120 systemd[1]: Stopping Ceph mds.c for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:11:05.590 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 12 00:11:05 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c[65607]: 2024-09-12T00:11:05.323+0000 7f5a64d21640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:11:05.591 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 12 00:11:05 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c[65607]: 2024-09-12T00:11:05.323+0000 7f5a64d21640 -1 mds.c *** got signal Terminated *** 2024-09-12T00:11:07.370 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 12 00:11:07 smithi120 podman[73625]: 2024-09-12 00:11:07.09445909 +0000 UTC m=+1.879078781 container died 518513c8e43a83968dcf6de9ad4501530bdc978fd60a242b739e9de0013fd9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T00:11:07.848 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 12 00:11:07 smithi120 podman[73625]: 2024-09-12 00:11:07.548808666 +0000 UTC m=+2.333428356 container cleanup 518513c8e43a83968dcf6de9ad4501530bdc978fd60a242b739e9de0013fd9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T00:11:07.848 INFO:journalctl@ceph.mds.c.smithi120.stdout:Sep 12 00:11:07 smithi120 bash[73625]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-c 2024-09-12T00:11:07.982 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T00:11:08.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:11:08.018 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T00:11:08.018 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T00:11:08.019 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.f 2024-09-12T00:11:08.370 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 12 00:11:08 smithi120 systemd[1]: Stopping Ceph mds.f for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:11:08.870 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 12 00:11:08 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-f[65307]: 2024-09-12T00:11:08.485+0000 7fc639f58640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:11:08.870 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 12 00:11:08 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-f[65307]: 2024-09-12T00:11:08.485+0000 7fc639f58640 -1 mds.f *** got signal Terminated *** 2024-09-12T00:11:11.368 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 12 00:11:11 smithi120 podman[73758]: 2024-09-12 00:11:11.097870756 +0000 UTC m=+2.850755864 container died 33a9ed429ceea66cdbdc090aedb31cb63027112df98af3d0c94c8051672473b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T00:11:11.620 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 12 00:11:11 smithi120 podman[73758]: 2024-09-12 00:11:11.478148927 +0000 UTC m=+3.231034037 container cleanup 33a9ed429ceea66cdbdc090aedb31cb63027112df98af3d0c94c8051672473b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T00:11:11.621 INFO:journalctl@ceph.mds.f.smithi120.stdout:Sep 12 00:11:11 smithi120 bash[73758]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-f 2024-09-12T00:11:11.960 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T00:11:11.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:11:11.994 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T00:11:11.994 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T00:11:11.994 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.i 2024-09-12T00:11:12.314 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:11:12 smithi120 systemd[1]: Stopping Ceph mds.i for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:11:12.620 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:11:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i[65905]: 2024-09-12T00:11:12.315+0000 7ff00f913640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:11:12.620 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:11:12 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i[65905]: 2024-09-12T00:11:12.315+0000 7ff00f913640 -1 mds.i *** got signal Terminated *** 2024-09-12T00:11:15.372 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:11:15 smithi120 podman[73879]: 2024-09-12 00:11:15.098360255 +0000 UTC m=+2.887879991 container died ab1501235c6d42b04a6e03e1c1df3c4916ab4a5e454001fb5d86bbdf008357f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T00:11:15.649 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:11:15 smithi120 podman[73879]: 2024-09-12 00:11:15.482272128 +0000 UTC m=+3.271791855 container cleanup ab1501235c6d42b04a6e03e1c1df3c4916ab4a5e454001fb5d86bbdf008357f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T00:11:15.649 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:11:15 smithi120 bash[73879]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i 2024-09-12T00:11:15.899 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T00:11:15.920 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:11:15 smithi120 podman[73891]: 2024-09-12 00:11:15.650326863 +0000 UTC m=+0.552524332 container remove ab1501235c6d42b04a6e03e1c1df3c4916ab4a5e454001fb5d86bbdf008357f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T00:11:15.920 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:11:15 smithi120 systemd[1]: ceph-56675698-7097-11ef-bcea-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-12T00:11:15.920 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:11:15 smithi120 systemd[1]: Stopped Ceph mds.i for 56675698-7097-11ef-bcea-c7b262605968. 2024-09-12T00:11:15.920 INFO:journalctl@ceph.mds.i.smithi120.stdout:Sep 12 00:11:15 smithi120 systemd[1]: ceph-56675698-7097-11ef-bcea-c7b262605968@mds.i.service: Consumed 10.052s CPU time. 2024-09-12T00:11:15.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:11:15.931 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T00:11:15.931 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T00:11:15.931 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-56675698-7097-11ef-bcea-c7b262605968@mds.l 2024-09-12T00:11:16.260 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 12 00:11:16 smithi120 systemd[1]: Stopping Ceph mds.l for 56675698-7097-11ef-bcea-c7b262605968... 2024-09-12T00:11:16.610 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 12 00:11:16 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-l[65006]: 2024-09-12T00:11:16.260+0000 7f3865627640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:11:16.610 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 12 00:11:16 smithi120 ceph-56675698-7097-11ef-bcea-c7b262605968-mds-l[65006]: 2024-09-12T00:11:16.260+0000 7f3865627640 -1 mds.l *** got signal Terminated *** 2024-09-12T00:11:17.868 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 12 00:11:17 smithi120 podman[73997]: 2024-09-12 00:11:17.549763207 +0000 UTC m=+1.389657550 container died 86a7bf9d1c1250931d7a1514c875921f31a94915ee9020d23f0180cff375b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T00:11:17.869 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 12 00:11:17 smithi120 podman[73997]: 2024-09-12 00:11:17.870388463 +0000 UTC m=+1.710282806 container cleanup 86a7bf9d1c1250931d7a1514c875921f31a94915ee9020d23f0180cff375b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T00:11:18.120 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 12 00:11:17 smithi120 bash[73997]: ceph-56675698-7097-11ef-bcea-c7b262605968-mds-l 2024-09-12T00:11:18.120 INFO:journalctl@ceph.mds.l.smithi120.stdout:Sep 12 00:11:18 smithi120 podman[74009]: 2024-09-12 00:11:18.079758695 +0000 UTC m=+0.521006891 container remove 86a7bf9d1c1250931d7a1514c875921f31a94915ee9020d23f0180cff375b001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-56675698-7097-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T00:11:18.207 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-56675698-7097-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T00:11:18.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T00:11:18.240 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T00:11:18.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 56675698-7097-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T00:11:18.453 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 56675698-7097-11ef-bcea-c7b262605968 2024-09-12T00:11:21.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 56675698-7097-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T00:11:21.254 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 56675698-7097-11ef-bcea-c7b262605968 2024-09-12T00:11:23.771 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 56675698-7097-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T00:11:23.972 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 56675698-7097-11ef-bcea-c7b262605968 2024-09-12T00:11:26.358 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T00:11:26.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T00:11:26.427 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T00:11:26.455 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T00:11:26.457 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489/remote/smithi062/crash 2024-09-12T00:11:26.457 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/crash -- . 2024-09-12T00:11:26.500 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T00:11:26.500 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2024-09-12T00:11:26.502 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489/remote/smithi083/crash 2024-09-12T00:11:26.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/crash -- . 2024-09-12T00:11:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T00:11:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-09-12T00:11:26.553 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489/remote/smithi120/crash 2024-09-12T00:11:26.554 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/crash -- . 2024-09-12T00:11:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:tar: /var/lib/ceph/56675698-7097-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T00:11:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:tar: Error is not recoverable: exiting now 2024-09-12T00:11:26.592 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T00:11:26.592 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T00:11:26.635 INFO:tasks.cephadm:Compressing logs... 2024-09-12T00:11:26.636 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T00:11:26.678 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T00:11:26.681 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T00:11:26.704 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T00:11:26.704 INFO:teuthology.orchestra.run.smithi062.stderr:: No such file or directory 2024-09-12T00:11:26.704 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T00:11:26.706 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T00:11:26.706 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T00:11:26.706 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T00:11:26.706 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T00:11:26.706 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T00:11:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T00:11:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mon.a.log: 84.4% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T00:11:26.712 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T00:11:26.713 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.log 2024-09-12T00:11:26.713 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T00:11:26.713 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T00:11:26.713 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.log: 94.6% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T00:11:26.716 INFO:teuthology.orchestra.run.smithi062.stderr: 86.9% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T00:11:26.717 INFO:teuthology.orchestra.run.smithi062.stderr: 94.6% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T00:11:26.717 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T00:11:26.717 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T00:11:26.718 INFO:teuthology.orchestra.run.smithi120.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T00:11:26.718 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T00:11:26.718 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T00:11:26.718 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T00:11:26.718 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T00:11:26.718 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T00:11:26.718 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T00:11:26.718 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T00:11:26.718 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip 81.8% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T00:11:26.719 INFO:teuthology.orchestra.run.smithi120.stderr: -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T00:11:26.719 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.log 2024-09-12T00:11:26.719 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T00:11:26.719 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T00:11:26.719 INFO:teuthology.orchestra.run.smithi120.stderr: 86.9% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T00:11:26.719 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T00:11:26.719 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T00:11:26.727 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mgr.x.log: 91.1% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T00:11:26.727 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T00:11:26.727 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T00:11:26.727 INFO:teuthology.orchestra.run.smithi062.stderr: 94.6% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T00:11:26.727 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.3.log: 94.6% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T00:11:26.728 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.8.log: 91.1% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T00:11:26.728 INFO:teuthology.orchestra.run.smithi120.stderr: 94.6% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T00:11:26.728 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T00:11:26.728 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T00:11:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.10.log: No such file or directory 2024-09-12T00:11:26.730 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-12T00:11:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T00:11:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T00:11:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.cephadm.log: 81.8% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr: 94.5% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.log 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mgr.y.log: /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.log: 86.9% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T00:11:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T00:11:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T00:11:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T00:11:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T00:11:26.732 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.c.log: 92.3% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T00:11:26.733 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T00:11:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T00:11:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T00:11:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T00:11:26.936 INFO:teuthology.orchestra.run.smithi083.stderr: 93.5% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T00:11:26.989 INFO:teuthology.orchestra.run.smithi083.stderr: 94.0% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T00:11:27.048 INFO:teuthology.orchestra.run.smithi062.stderr: 90.3% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T00:11:27.269 INFO:teuthology.orchestra.run.smithi120.stderr: 94.1% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T00:11:27.289 INFO:teuthology.orchestra.run.smithi120.stderr: 91.7% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T00:11:27.314 INFO:teuthology.orchestra.run.smithi083.stderr: 91.7% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T00:11:27.767 INFO:teuthology.orchestra.run.smithi062.stderr: 90.2% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T00:11:28.297 INFO:teuthology.orchestra.run.smithi120.stderr: 93.8% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T00:11:29.746 INFO:teuthology.orchestra.run.smithi120.stderr: 94.6% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T00:11:38.024 INFO:teuthology.orchestra.run.smithi083.stderr: 95.1% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T00:11:55.269 INFO:teuthology.orchestra.run.smithi062.stderr: 91.7% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T00:12:01.378 INFO:teuthology.orchestra.run.smithi120.stderr: 91.5% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T00:12:06.703 INFO:teuthology.orchestra.run.smithi062.stderr: 91.3% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T00:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr: 91.6% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T00:12:09.454 INFO:teuthology.orchestra.run.smithi083.stderr: 91.3% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T00:12:10.494 INFO:teuthology.orchestra.run.smithi120.stderr: 91.2% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T00:12:12.016 INFO:teuthology.orchestra.run.smithi120.stderr: 91.4% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T00:12:14.451 INFO:teuthology.orchestra.run.smithi120.stderr: 91.3% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T00:12:14.453 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-12T00:12:14.454 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m47.758s 2024-09-12T00:12:14.454 INFO:teuthology.orchestra.run.smithi120.stderr:user 2m51.461s 2024-09-12T00:12:14.454 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m5.300s 2024-09-12T00:12:15.845 INFO:teuthology.orchestra.run.smithi083.stderr: 91.4% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T00:12:21.570 INFO:teuthology.orchestra.run.smithi083.stderr: 91.3% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T00:12:21.574 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-12T00:12:21.575 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m54.876s 2024-09-12T00:12:21.575 INFO:teuthology.orchestra.run.smithi083.stderr:user 3m14.945s 2024-09-12T00:12:21.575 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m6.064s 2024-09-12T00:12:22.170 INFO:teuthology.orchestra.run.smithi062.stderr: 91.2% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T00:12:26.410 INFO:teuthology.orchestra.run.smithi062.stderr: 91.1% -- replaced with /var/log/ceph/56675698-7097-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T00:12:26.412 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-09-12T00:12:26.412 INFO:teuthology.orchestra.run.smithi062.stderr:real 0m59.719s 2024-09-12T00:12:26.412 INFO:teuthology.orchestra.run.smithi062.stderr:user 2m59.512s 2024-09-12T00:12:26.412 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m5.248s 2024-09-12T00:12:26.413 INFO:tasks.cephadm:Archiving logs... 2024-09-12T00:12:26.413 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489/remote/smithi062/log 2024-09-12T00:12:26.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T00:12:57.654 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489/remote/smithi083/log 2024-09-12T00:12:57.655 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T00:13:30.681 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489/remote/smithi120/log 2024-09-12T00:13:30.682 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T00:14:02.366 INFO:tasks.cephadm:Removing cluster... 2024-09-12T00:14:02.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 56675698-7097-11ef-bcea-c7b262605968 --force 2024-09-12T00:14:02.582 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 56675698-7097-11ef-bcea-c7b262605968 2024-09-12T00:14:03.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 56675698-7097-11ef-bcea-c7b262605968 --force 2024-09-12T00:14:03.322 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 56675698-7097-11ef-bcea-c7b262605968 2024-09-12T00:14:03.846 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 56675698-7097-11ef-bcea-c7b262605968 --force 2024-09-12T00:14:04.049 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 56675698-7097-11ef-bcea-c7b262605968 2024-09-12T00:14:04.581 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T00:14:04.581 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T00:14:04.602 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T00:14:04.620 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T00:14:04.636 INFO:tasks.cephadm:Teardown complete 2024-09-12T00:14:04.637 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T00:14:04.648 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T00:14:04.649 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T00:14:04.652 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T00:14:04.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T00:14:04.804 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T00:14:04.815 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T00:14:04.816 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T00:14:04.818 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T00:14:04.821 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T00:14:04.834 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-09-12T00:14:04.834 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-09-12T00:14:04.835 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-12T00:14:04.839 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T00:14:04.839 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-09-12T00:14:04.839 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 +1612us[+1600us] +/- 49ms 2024-09-12T00:14:04.839 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -1316us[-1328us] +/- 64ms 2024-09-12T00:14:04.839 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 -1171us[-1183us] +/- 99ms 2024-09-12T00:14:04.839 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:14:04.841 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T00:14:04.841 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-12T00:14:04.842 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv01.front.sepia.ceph.com 3 6 377 58 +1558us[+1575us] +/- 49ms 2024-09-12T00:14:04.842 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -1340us[-1323us] +/- 64ms 2024-09-12T00:14:04.842 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 -1211us[-1193us] +/- 99ms 2024-09-12T00:14:04.842 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:14:04.842 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T00:14:04.842 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-09-12T00:14:04.842 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 +1612us[+1627us] +/- 49ms 2024-09-12T00:14:04.842 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -1301us[-1301us] +/- 64ms 2024-09-12T00:14:04.842 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 -1170us[-1170us] +/- 99ms 2024-09-12T00:14:04.842 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:14:04.843 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T00:14:04.853 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T00:14:04.854 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T00:14:04.877 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T00:14:04.977 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T00:14:05.050 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T00:14:05.118 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T00:14:05.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:14:05.150 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T00:14:05.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:14:05.181 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T00:14:05.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:14:05.209 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T00:14:05.221 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T00:14:05.261 INFO:teuthology.task.internal:Duration was 2316.982382 seconds 2024-09-12T00:14:05.262 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T00:14:05.294 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T00:14:05.295 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T00:14:05.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T00:14:05.300 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T00:14:05.337 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T00:14:05.338 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T00:14:05.338 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T00:14:05.757 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T00:14:05.758 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-09-12T00:14:05.758 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T00:14:05.821 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-09-12T00:14:05.821 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T00:14:05.845 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-09-12T00:14:05.845 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T00:14:05.866 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T00:14:05.866 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T00:14:05.869 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T00:14:05.888 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T00:14:05.932 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T00:14:05.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T00:14:05.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T00:14:05.961 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T00:14:07.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T00:14:07.417 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T00:14:07.417 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T00:14:07.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T00:14:07.474 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T00:14:07.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T00:14:07.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:14:07.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:14:07.523 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:14:07.542 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-09-12T00:14:07.547 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-09-12T00:14:07.572 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-09-12T00:14:07.801 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:14:07.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:14:07.818 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:14:07.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:14:07.860 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T00:14:07.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:14:07.877 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T00:14:07.888 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T00:14:07.890 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489/remote/smithi062 2024-09-12T00:14:07.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T00:14:07.964 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489/remote/smithi083 2024-09-12T00:14:07.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T00:14:08.006 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901489/remote/smithi120 2024-09-12T00:14:08.006 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T00:14:08.049 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T00:14:08.049 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T00:14:08.052 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T00:14:08.055 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T00:14:08.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T00:14:08.113 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T00:14:08.113 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T00:14:08.137 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T00:14:08.138 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T00:14:08.140 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T00:14:08.143 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T00:14:08.153 INFO:teuthology.orchestra.run.smithi062.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 00:14 /home/ubuntu/cephtest 2024-09-12T00:14:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 00:14 /home/ubuntu/cephtest 2024-09-12T00:14:08.159 INFO:teuthology.orchestra.run.smithi120.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 00:14 /home/ubuntu/cephtest 2024-09-12T00:14:08.160 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T00:14:08.172 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T00:14:08.361 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 2316.9823820590973 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T00:14:08.362 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T00:14:08.450 INFO:teuthology.run:pass